> I am not talking about literally bruteforcing passwords (although LLMs are being used for that, too)

They are? Seems like a much worse way to brute force that a tight loop written in a compiled language.

PassGPT: Password Modeling and (Guided) Generation with Large Language Models

https://huggingface.co/papers/2306.01545

Although most activity is likely hidden (blackhat or state)