Remix.run Logo
▲ jinseokim 3 hours ago

I just feel a bit sad and impotent that a number of security researchers outside the circle cannot join their "yo we found more crazy bugs" party because simply frontier labs only give access to those models with "Red Team" availability to the inner circle members/companies.

Now their "trusted cyber access" program is moat rather than safeguard.

▲ipsum2 3 hours ago | parent [-]

You can do a lot with the Chinese open source models.

▲wslh a minute ago | parent | next [-]

Not in my experience (yes, sample ~= 1), where they are subpar in terms of cybersecurity (e.g. source code review). This is one of my reports using GPT 5.6 (High), GLM 5.3 High, Fable 5.1 [1]:

> On the full benchmark of 31 audited revisions, GPT-5.6 Sol High reached a weighted recall of 38.27% and GLM-5.3 High reached 30.86%. On a common subset of 12 projects, Claude Fable 5.1 High reached 65.70%, against 42.98% for GPT and 35.54% for GLM.

[1] https://github.com/CoinFabrik/soroban-audit-harness/blob/mai...

▲_rs 3 hours ago | parent | prev [-]

That assumes you have the hardware capable of running inference on them at a reasonable speed

▲noosphr 3 hours ago | parent | next [-]

Or $10 on open router.

▲ipsum2 3 hours ago | parent | prev [-]

Or it assumes you have $5 to spend on the tokens for inference. Not everyone has the money.

▲r_lee 2 hours ago | parent [-]

it takes a lot more than that....

▲bhelkey 2 hours ago | parent [-]

On OpenRouter, GLM 5.3 Flash costs $0.0285 for 1M input tokens and $0.0923 for 1M output tokens. [1]

[1] https://openrouter.ai/z-ai/glm-5.3-flash

▲r_lee an hour ago | parent [-]

if you wanna have the same kind of capability as these researchers where you have tight loops calling cli stuff you won't get by with $5 at all, especially for major bugs

if you have a way to turn $5 easily into exploitable vulns, let me know cause that'd be a first for me