It's easy to switch the LLM API, but in practice this requires having a strong eval suite so that the expected behavior of whatever is built on top changes within acceptable limits. It's really the implications of the LLM switch that matter.

I think the most is small but it’s not gone yet: in these discussions I almost never see the people reporting productivity wins say they’re running local models, and there’s a standard response to problems that you should switch to a different vendor, which suggests that there are enough differences in the training and tooling to make switching non-trivial. This will especially be true for all of the non-tech specialist companies adopting things – if you bought an AI tool to optimize your logistics and the vendor pulls a Broadcom on your contract renewal, they’re probably going to do so only after you’ve designed your business around their architecture and price it below where it’d make sense to hire a team and spend years building something in house. Having laid off a lot of in-house knowledge directly helps them, too.

I know this is just a casual comment, but this is a genuine concern I have every day. However, I've been working for 10 years now and working in music/video streaming has been the most "societal value" I've had thus far.

I've worked at Apple, in finance, in consumer goods.. everywhere is just terrible. Music/Video streaming has been the closest thing I could find to actually being valuable, or at least not making the world worse.

I'd love to work at an NGO or something, but I'm honestly not that eager to lose 70% of my salary to do so. And I can't work in pure research because I don't have a PhD.

What industry do you work in, if you don't mind me asking?

I work in R&D, and although I haven't signed an NDA, I think it's best if I don't elaborate too much. But basically we have a large dataset of shows and movies for which we already know how well they did with specific audiences, but we didn't know why exactly. So we use LLMs to reverse-engineer a large amount of metadata about these shows, and then use traditional ML to train a model that learns which feature appeal to which audiences.

Most stuff is obvious: nobody needs to tell you what segment of society is drawn to soap operas or action movies, for example. But there's plenty of room for nuance in some areas.

This doesn't guarantee that it actually becomes a succesful movie or show, though. That's a different project and frankly, a lot harder. Things like which actors, which writers, which directors, which studio are involved, and how much budget the show has.. it feels more like Moneyball but with more intangible variables.