Disclaimer: I am not an LLM maximalist, I like to use LLMs, but in a highly controlled way.

One exciting development that LLMs have recently started is automated vulnerability discovery. For instance, Chrome solved more bugs in June than over the past two years:

https://news.ycombinator.com/item?id=49120097

Similarly, the latest Apple security update and the June Android Security Bulletin fixes an insane number of vulnerabilities.

Of course, many of these vulnerabilities are caused by using unsafe languages (C/C++). But LLMs also help porting such code to Rust or other safe languages, because it is the type of transformation that LLMs are pretty good at (clearly defined problem, not many opportunities to go wild).

Yeah I can see this as being an area that will benefit from LLMs. Though, taking it one further, we don’t yet have evidence (so far as I know) that this actually produces more secure software.

It’s not that I’m an LLM hater at all, I use them, I feel like I’m being super productive so I’m not doubting people’s accounts. But if I honestly look back over the past 12 months and ask “have I shipped more value that I would have without LLMs” the answer is probably no.

I think it has helped me learn areas quicker, though for complex mathsy explanations I still ask for it to find me human written posts as Claude’s explanations can be super weird, using convoluted analogies etc..