Stuff it's not directly trained on is going to be flaky and sucky. It was like that with programming at first too and it still is sometimes. It's hard to imagine this won't improve with better more focused training. They focus on improving "CRUD" for obvious reasons. The specialization era hasn't begun yet.

Your domain, while I'm sure it is very interesting and complex, if it proves economically interesting will be cracked as well.

▲

jofer 6 hours ago | parent [-]

Just for some context, the domain we're talking about is oil and gas and mineral exploration. E.g. At my previous job, I used to personally manage a >$400 million per year budget and that wasn't even considered significant. We had multiple >$10 billion per year projects ongoing. That was 10 years ago. The amounts are larger now.

The issue isn't a lack of economic interest.

It might be a lack of training data in addition to inherent complexity, but it's certainly not a lack of economic interest.

▲

llmssuck 5 hours ago | parent | next [-]

I have no idea how and why GenAI would be useful in your profession. I'm sure a lot of money is moved there (not sure about the profits though), but it's not clear to me how software itself is budging that needle. I suppose better algorithms and better understanding of geology will do it, but software itself seems just subservient to that goal.

I guess what I'm saying is that "domain knowledge" is taking software development for a ride here. The software is just the vehicle, the science is the engine here and I can see why companies like OpenAI start going for the low-hanging fruits first instead.

Your specific company might be profitable, but does automating "mineral exploration" give you leverage over quite literally all other domains? My guess is not. For "CRUD" it is a resounding yes, it provides gigantic leverage. Once you automate basic software development you enter a new world. 10 billion, 10 trillion, all bets are off. You automate the creation of the next iteration of automation and on we go. Let's hope it takes a while for this take off. I can't see ourselves being ready for it.

My guess is it'll take a decade or so for real AI science to start taking off though - if that soon - so you're probably fine for now.

	▲	jofer 5 hours ago \| parent [-]
		Yes. My point was that LLMs aren't currently good for everything. The original commenter literally said they were good at everything and I offered a counterpoint of something they're not good at: Most science. (And yes, a lot of science is software. Analysis is software.)

▲

woeirua 4 hours ago | parent | prev [-]

Skill issue. I've seen LLMs used in this domain to get mindblowing results. You won't see it published anywhere though.... =).

▲

calf 3 hours ago | parent [-]

Disagree, someone like the other commenter who points out LLMs don't even understand the domain concepts correctly versus someone who uses it anyways for corporate proprietary results have very different standards for what is acceptable. If you wrangle an LLM with harnesses and clever prompts you could use it to get some amazing results but that has more to do with trial and error and creativity, not some kind of fundamental skill of using LLMs.

	▲	woeirua 14 minutes ago \| parent [-]
		It definitely understands the concepts well enough if you give it the right context. I'm not the only one saying this either. Like I said, it's a skill issue.