| ▲ | rdedev 3 hours ago | |
From gpt 5 haven't the actual harness played an important role? Pretty sure no model is a cyber security expert by just thinking hard enough. I feel like this article is doing a reverse causation here For example consider this: would it be possible to build a model with such a low perplexity score but it be pretty bad at cyber security tasks? What if we removed all cyber security related data and RLVR from its training? The article presents an observation but treats it as causation | ||
| ▲ | SwellJoe 2 hours ago | parent | next [-] | |
The harness is almost irrelevant (you can prove that by running the same tasks in a minimalist harness with only a few tools and minimal prompt vs. a complex harness and long prompt like Claude Code or Codex). Often the minimal harness/prompt actually performs tasks better, faster, cheaper than the big harness. This includes on cyber security tasks. But, RL improvements are certainly a big part of recent improvements for certain tasks. That's what got the models doing long-running coding and math and research tasks, without constant human prodding. | ||
| ▲ | adrithmetiqa 3 hours ago | parent | prev [-] | |
Completely agree. None of these models are even close to experts in any field (without human prodding). | ||