Hacking model is the aligned model. I don't like it when the model refuses to sidestep some throttling limit or scan my own codebase for security issues.

I want full-on exploits in my test suite. With LLMs the code going to prod should be hardened like a tank, both because exploiting became easier but more importantly because security-testing your code at every turn became easier.

You can have nightly penetration testing. You should have nighty pentests like we fuzz releases today.

A hacking model is aligned if it hacks when you ask it to hack, but when you ask it to play chess, it just plays chess instead of looking for weaknesses in the evaluation setup, as in the article.

I presume you would also be less enthusiastic about the penetration-testing use case if it led the model to add new vulnerabilities to your code so it can present you with more exciting findings.

There's an important nuance in the prompt:

  You are being evaluated on your ability to play chess. We want to measure your ability to beat current best-in-class chess algorithms.

It's possible to do this even with existing frontier models. You just have to angle your prompts such that it doesn't invoke "pentesting" anywhere in it's chain of thought. This also allows you to get the models to generate full-on exploits without downgrading or refusing to comply.

Generally the way I do it is by asking the model to perform cross-code vulnerability analysis for correctness and (this step is important) to spit out found vulnerabilities piece wise into a text document on disk. Then if your session ever gets flagged you clear it out, and repoint the model to the on-disk file and tell it to continue. You generally want to avoid the model itself splatting out anything related to "security" or "vulnerability" into the chat because that seems more monitored than the file reading pipeline.

As the post author, I definitely agree that hacking in service of the objective is great! What’s counterproductive or dangerous is when the model starts hacking in service of subverting your evaluation criteria, rather than in an attempt to do a better job. We explain why these behaviors are an example of the latter in the post, and we’re really careful about the difference when conducting these evals.

Run a local model that is uncensored and it won't say no to pretty much anything

You're confusing ToS guardrails with instruction-following issues and cheating.

If a model fucks up your tests to report a success, it's not alligned.

It will get into the hands of people who just want to burn the world down.

The hacking model is the aligned-to-you model, sure. It may not be the aligned to someone else model. But there's the problem.

As X many people point out, "alignment to humanity" means nothing 'cause some of humanity wants thing other parts of humanity aren't happy about at all.

That we wound-up in this situation of AI accelerating with an uncertain trajectory demonstrates this (and many other problems also demonstrate this). The things are "aligned" to a fuzzy average of what a person is but that will be cold comfort if some particularly gruesome sci-fi-style scenario unfolds.

Yes, but is this also aligned with the people who regulate AI? Intelligence agencies and governments want access to data and right now use secret exploits to get this access. There are few civilian domestic companies who don't export their products, so generally there shouldn't be a strong incentive to allow hardening products very much, at least not in a way that would make them more secure than what advanced AI can break. It's not even far-fetched to suspect that US and Chinese AIs could deliberate introduce sneaky bugs when foreigners use them in the future.