> as progress depends more on massive training runs, it becomes capital-intensive, less reproducible and more secretive; so you get a compute divide and less publication.

In the area of AI, secrecy and inability to reproduce/verify can become a huge systemic and social problem, the possible damage is literally unbounded.

That's why I like open source AI, including training data and process, it solves the above problem as well as the problem of duplication of effort which leads to a huge waste of resources, waste that is economically significant on national and global scales.