Learn to work on interesting problems? If the problem you are working on is novel and hard, the AI will stumble.

Generalizing your experience to everyone else's betrays a lack of imagination.

> Generalizing your experience to everyone else's betrays a lack of imagination.

One guy is generalizing from "they don't work for me" to "they don't work for anyone."

The other one is saying "they do work for me, therefore they do work for some people."

Note that the second of these is a logically valid generalization. Note also that it agrees with folks such as Tim Gowers, who work on novel and hard problems.

This is my experience. For rote generation, it's great, saves me from typing out the same boilerplate unit test bootstrap, or refactoring something that exists, etc.

Any time I try to get a novel insight, it flails wildly, and nothing of value comes out. And yes, I am prompting incrementally and building up slowly.

Even people who do actual hard work need a lot of ordinary scaffolding done for them.

A secretary who works for an inventor is still thinking.

Research mathematicians have been finding the tools useful [1][2]. I think those problems are interesting, novel, and hard. The AI might stumble sometimes, but it also produces meaningful, quality results sometimes. For experts working on interesting problems, that is enough to be useful.

[1] https://mathstodon.xyz/@tao/115420236285085121 [2] https://xcancel.com/wtgowers/status/1984340182351634571

Even most humans will stumble on hard problems, that is the reason they are hard in the first place

I'm genuinely curious what you work on that is so "novel" that an LLM doesn't work well on?

I feel like so little is TRUELY novel. Almost everything is built on older concepts and to some degree expertise can be applied or repurposed.

Dude. We don't all work for NASA. Most day to day problems aren't novel. Most jobs aren't novel. Most jobs can't keep a variety of sometimes useful experts on hand. I do my job and I go home and do my hobbies. Anything I can use at work to keep friction down and productivity up is extremely valuable.

Example prompt (paraphrasing and dumbed down, but not a ton): Some users across the country can't get to some fileshares. I know networking, but I'm not on the networking team so I don't have full access to switch, router, and firewall logs/configurations. It looks kind of random, but there must be a root cause, let's find it.

I can't use Python(security team says so) and I don't have access to a Linux box that's joined to the domain and has access the shares.

We are on a Windows domain controller. Write me a PowerShell 5.1 compatible script to be run remotely on devices. Use AD Sites and Services to find groups of random workstations and users at each office and tries to connect to all shares at each other site. Show me progress in the terminal and output an Excel file and Dot file that clearly illustrates successful and failed connections.

---

And it works. Ok, I can see the issue is from certain sites that use x AND y VPN ipsec tunnels to get to particular cloud resources. I give this info to networking and they fix it right away. Problem resolved in less than an hour.

First of all, a couple years ago, I wouldn't have been able to justify writing something like this while an outage is occuring. Could I do it myself? Sure, but I'm going to have to look up the specifics of syntax and certain commands and modules. I don't write PowerShell for a living or fun, but I do need to use it. I am familiar and know how to write it. But I sure as fuck couldn't sit down and spend an hour or two screwing around working on building a goddamn Dot file generator. Yes, years ago I had a whole pile of little utility modules I could use. But that's a far cry from what I can do now to fit the exact situation < 15 minutes while I do other things like pick up the phone, message coworkers, etc.

Secondly, rather than building little custom tools to hook together as I need, I can just ask for the whole thing. I don't need to save any of that stuff anymore and re-figure out what CheckADFSConns(v2).PS1 that I wrote 8 months ago does and how to use it. "Oh, that's not the one, what the did I name that? Where did I put it?"

I work in an environment that is decades old, the company is over 100 years old, I didn't build any of it myself, is not a tech company, and has tons of tech debt and weird shit. AI is insanely useful. For any given problem, there are dozens of different rabbit holes I could go down because of decades of complete system overhaul changes. Today, I can toss a variety of logs at AI and if nothing else, get a sense of direction of why a handful of PCs are rejecting some web certificates. (Combination of a new security policy and their times mismatching the domain controller, because it was new, and NTP wasn't configured properly. I wasn't even looking for timestamps, but it noticed event offsets and pointed it out).

I feel like this community isn't very familiar with what that's like. We aren't all working on self driving cars or whatever seems hard at a brand new company with new everything and no budget. Some of us need to keep the systems running that help people to make actual things. These environments are far from pristine and are held together by underpaid and underappreciated normies through sheer willpower.

Is this kind of work breaking technical frontiers? No. But it's complicated, difficult, and unpredictable. Is it novel? The problems are, sometimes.

Generalizing your experience to everyone else's betrays your lack of self-awareness, sir.