Can someone define “improvement” in this context? This concept feels like a buzz word otherwise.

Improvement means being able to do more complicated things more reliably. Relatedly, it means being able to learn to do new things with fewer and fewer examples. We are running out of easily verifiable or simulation-friendly or data-rich domains for LLMs to conquer. (Note that I didn't say "simple" or "easy" domains.)

Models that are strong enough to improve themselves without a human (I.e. ai researcher) in the loop