I'm wondering if the ROI will be worth it anytime soon for anything other than coding, and anything that can be publically scraped of the internet. Or more to the point things that are at a enterprise level and require paid staff to train the model for a particular domain at an expert level of quality - and the ROI of such a task to be positive.

The thing is none of this is really happening under typical economic assumptions like ROI, rate of return, net PV, etc.

You see - on a pure ROI basis none of this should of existed. Even for coding I think - a lot of this is fuelled by investor money and even if developers took up the tooling I'm not sure it would pay off the capital investment. DeepMind wouldn't of been funded by Google, transformers would of never been invented if it was just based on expected ROI, etc. Most companies can't afford engineers/AI researchers on the side "just in case" it pays off especially back then when AI was a pie in the sky kind of thing. The only reason why any of this works is because Big Tech has more money than they can invest, and the US system punishes dividends meaning companies can justify "expected bad" investments as long as they can be dressed up and some pay off. They almost operate like internal VC's/funds because they had the money to do so.

This allows "arms race" and "loss leading" dynamics to take hold and be funded - which isn't about economics as much anymore. Most other industries/domains don't have the war chest or investors with very very deep pockets to make that a reality.

Sadly I think we as SWE's think it will also be other professions; what if instead we just disrupted our own profession and a few other smaller targets?