This assumes those who'd pay $500k don't have the skill to use GTP5.6 for the same purpose themselves?

Then why didn't we see the same writeup earlier? If you look through the writeup, you still need the skills to go through what the LLM gives you and actually create a valid proof of concept.

I've been using LLMs to find security vulnerabilities and there is no way I can just submit what I found and call it a day (many try).

You read articles regularly about how X authority is provided cloud LLM history and uses it as evidence to prosecute a defendant.

Yet you think professional criminals are too stupid to launder their activities through an unscrupulous yet legal intermediary?

People who make the money are not necessarily the people who can write the best code.

Elon Musk didn't write code for a rocket, he hired people who could.