The one thing that poisons the entire online discourse regarding LLMs to me is just simply not knowing who you're dealing with and what their problems are. This, like all things, is complex and nuanced.

For all my efforts to get LLMs to work for me, they've failed. I figure it is because the tech space I work in (Industrial Automation) simply doesn't have the training data that someone building webapps does. The few times I've put LLM to the test its failed horribly, and since my job has me existing in safety critical scenarios I don't humor what AI gives me at all.

So when I bitch about LLMs and their useless nature, its because of how I've experienced them in my craft. They get 95% of the information right, but that 5% it gets wrong can have very serious repercussions.

But when I'm talking with individuals online who see them as revolutionary? I don't know if their job is editing templates or formatting data or any other host of problems that AI can be revolutionary for. I don't know if they're full of themselves, having drank the AI koolaid. I don't know if they're just really deep into the LLM world and figured out things I have yet to learn (or am unwilling to pay for). But from a HN or Reddit discussion, they're all one in the same and without trying to figure out who they are, their opinions all carry the same weight.

▲

phil21 20 hours ago | parent [-]

> For all my efforts to get LLMs to work for me, they've failed. I figure it is because the tech space I work in (Industrial Automation) simply doesn't have the training data that someone building webapps does.

Putting safety critical aside (given that is a giant aside) for the moment, I'm a little surprised at this.

One of the most useful things I've found for AI Agent stuff is having an agent live on my home network and finally get the hundred or so random sensors and other IoT devices finally configured and talking together.

It's definitely not perfect though, and not anything I would trust my life to. This year however, I was able to set up enough sensors in my garden to have by far the largest hot pepper harvest I've ever had by min/maxing watering and fertilizing via way too many sensors and data fusing. This was mostly describing the sensors involved, the outcome and dashboarding desired, and pointing it to the various vendor API docs.

I could have done all that stuff manually, but never would have gotten around to it since it was just a tedious boring task. But with AI help I now can tell you precisely the soil moisture level, EC reading, ambient temp, etc. for each of my planters. And with some combining of my rain gauge data and sunlight sensing, tie that into automated irrigation runs. Then the bot just pings me when it thinks the next fertilizing cycle is due.

I don't know if that's revolutionary, but it took a project that would take me weeks to accomplish myself due to around a hundred sensors to configure/setup/etc. to a couple nights of messing around learning what all this talk was over these AI agent things.

I think there are plenty of use-cases where if AI gets things wrong 5% of the time it really doesn't matter a whole lot. I have plenty of these areas at my job.

	▲	TheCapn 7 hours ago \| parent [-]
		The biggest part of that is how there's a lot more community involvement for IoT and things like Home Assistant than there are for the proprietary PLC systems. In the PLC/SCADA space a lot of the development and resources are closed source. There's been a big push in the last few years for Open standards but that's slow to adopt and I'm sure in time we'll see the Controls space catch up (it is notoriously dated compared to the rest of the software world). But in the meantime its like the perfect storm of terrible-ness for AI. The vast majority of my SCADA work is in AVEVA products, mainly Wonderware and Edge. Wonderware's scripting language is something called QuickScript.NET which is like 80% of VB.NET. The issue with AI is that being a statistical engine it very quickly starts on the right path, then veers into what is very obvious VB.NET training data and starts doing things that are impossible in Wonderware. Edge uses VBScript, but a limited subset of it. So querying AI for help mostly just spits out useless code in the same way; confidently declaring what it gave is possible when it only results in runtime exceptions. In the PLC space you can get AI to do pretty mundane things with ease. Writing ladder logic or structured text is pretty okay, but like most things that was never the bottleneck in project development: you're not saving a lot of time for the cost of an AI subscription license. As mentioned, Open standards are starting to be adopted pretty quickly and a lot of PLC systems allow import/export of the proprietary formats into XML now which is going to be beneficial in the long run, but we're not quite there yet. The problem is always that time it messes up, and it can be very subtle. Off by one errors which would be perfect fine in one environment can be catastrophic totally depending on how your I/O Comms were configured. I was trying to interface with some hardware that I asked AI to see if I could solve the pain points of remote enable/disable via network comms. It very eagerly told me I could use OPC-UA Methods to achieve what I'm after, but I was working a platform that didn't support Methods. It even fabricated a path to fake call a method which obviously did not work when applied to a real world scenario. So I asked it how to use Modbus which requires use of FC16 to act on the hardware. Again AI very eager to tell me how to do that, but hallucinated what could be done. Lastly I went for the TCP route which again had AI giving incorrect information - it was far closer this time, having instructed me to send a command in a format which doesn't work out of box, but could be manually configured in the hardware to work. Fortunately I was testing to see if it could save me time, not trying to see if it could do something I was unable to do, but I try to imagine what would happen if I allowed it to do that type of work on a VFD, or PID loop. That tiny little bit of misunderstanding by the agent could be catastrophic. Each time I share this type of experience (and I have others) with the AI community I'm told I'm doing it wrong. That if I was using better models or different configurations that wouldn't be an issue. Telling me that I need to pay for models - as though I want to fork over cash when their demos are wildly underwhelming. It gives me a lot of overall distrust in the AI community because of this. It feels a lot like interacting with Linux fanatics who are unable to accept that it has faults that aren't tolerable by the layperson. That these people cannot see the faults because it works great in their narrow needs. But at the same time, using AI to give little tips. How can I speed up this SQL Stored Proc? Why is my DataBinding failing to update? What is causing this OS upgrade to fail? Its done pretty good work. I see it as a tool that works along with me and an IDE, not as a human replacement like so many feel it is ready to be.