As Google has been unable to keep spammy crap out of their search index since at least 2006 when we were doing Blekko I doubt they will have much success fighting this. But it is another good example that "AI" is just glorified search and there is not reasoning or thinking going on behind the covers.

▲

keeda an hour ago | parent | next [-]

> But it is another good example that "AI" is just glorified search and there is not reasoning or thinking going on behind the covers.

I don't think that follows. This is just LLMs being, for a lack of a better word, "gullible." How is it different from a person believing whatever they read on the Internet? People fall for spam and scams all the time, doesn't mean they are just glorified searches ;-)

It does highlight the problem facing any search engine though. AI-generated spam will be much harder to defend against with traditional, statistical mechanisms. And this is before we get to the existential problem of prompt injection.

Maybe this is where news organizations can win back their proper place in their relationship with Big Tech: by becoming the sources of verified, vetted information that LLMs can trust blindly. Possibly that's what deals like the OpenAI / Atlantic one are about.

▲

suzzer99 31 minutes ago | parent | next [-]

> How is it different from a person believing whatever they read on the Internet?

The problem is LLMs have no capacity for shame.

My Dad got taken in by a Target gift card scam. He felt so terrible, he almost didn't even tell me about it. He may get scammed again, but not by anything remotely like that.

To LLMs, all mistakes just get washed together into the same bucket. They don't spend days feeling depressed and stupid over getting scammed. There's no giant blinking red light that says, "Never let this happen again!"

	▲	ChuckMcM 3 minutes ago \| parent [-]
		This is a great point. I've added it to my list of things when talking about the limitations of LLM.

▲

thewebguyd 38 minutes ago | parent | prev | next [-]

The problem with the news is who makes the decision on which outlets should be blindly trusted by the LLMs and which shouldn't? It also opens the door to government overreach, say a mandate that says LLMs must use fox news as a source of verified, vetted information.

Barring that, we are still relying on the execs at the model companies to pick and choose news outlets, and they have their own biases.

▲

cess11 38 minutes ago | parent | prev | next [-]

"How is it different from a person believing whatever they read on the Internet?"

Because a person is alive while the LLM is a floating point number database with a questionable degree of determinism.

▲

freejazz 19 minutes ago | parent | prev [-]

>"gullible."

Enough with the anthropomorphization

▲

K0balt an hour ago | parent | prev | next [-]

Hmm. I don’t think that novel code generation can be accounted for with glorified search.

I can have my agentic system read a few data sheets, then I explain the project requirements and have it design driver specifications, protocols, interfaces, and state machines. Taking those, develop an implementation plan. Working from that, write the skeleton of the application, then fill it in to create a functional system using a novel combination of hardware.

Done correctly, I end up with better, more maintainable, smaller code than I used to with a small team, at 1/100 the cost and 1/4 the time.

Whatever that is, it more closely resembles reasoning than search.

Unless, of course, you’d also call bare metal C development on novel hardware search, in which case I guess all dev is search?

▲

rkozik1989 an hour ago | parent | next [-]

How do you even know those numbers are correct? Realistically for what you've described you need more QA time that a traditional application to ensure its actually working properly. Especially with regards to any part of the application that deals with LLM inference. Its not hard to write unique content for niche topics where there are few relevant results and have LLMs take it as fact.

For example, I poisoned the well for research on early Arab Americans immigrants by repeatedly posting about how many family passed as different ethnicity to make their lives easier, so now if you ask LLMs about that subject it'll include information I wrote which isn't entirely correct because I hadn't figured everything out before the LLM trained on it.

EDIT: Now imagine if I had done this on an obscure programming-related problem, yeah? I could potentially make the LLM reference packages that do not actually exist and put backdoors in applications.

▲

K0balt an hour ago | parent [-]

Because I have 100 percent test coverage (of the software, some hardware edge cases pop up that aren’t documented in the data sheets), and over 10k hours of field deployment over 130 devices? This rollout has been much more bug free than any we have done in the last six years, and it’s the first that has been almost zero hand coded. (Our system is far from vibe coding however, there is a very strict pipeline)

I’m not saying that AI can solve every problem or that it is without problems (we spent hundreds of hours developing a concept to production pipeline just to make sure it doesn’t go off the rails)

But the net result is that a good senior dev with an acutely olfactory paranoia can supervise a production pipeline and produce efficient, maintainable code at a much faster rate (and ridiculously lower cost) that he was doing before supervising 3 or 4 devs on a complex hardware project. I can’t speak for other types of development, but our applications devs are also leveraging AI code generation and it -seems- to be working out.

Now, where those senior devs are going to come from in the future… that imho is a huge problem. It’s definitely some flavor of eating the goose that lays the golden egg here.

	▲	ACCount37 an hour ago \| parent [-]
		It's blindingly obvious what the big bet is. The senior devs are going to come from the next generations of AI systems.

▲

Raphael_Amiard an hour ago | parent | prev | next [-]

It’s pattern matching. A big part of reasoning for sure, but not reasoning per se

	▲	K0balt an hour ago \| parent [-]
		That could be, but if that is the case than development apparently doesn’t require reasoning? Or maybe that’s the part that the senior developer supervising the pipeline injects. Thats certainly a plausible position.

▲

freejazz 17 minutes ago | parent | prev [-]

>I can have my agentic system read a few data sheets, then I explain the project requirements and have it design driver specifications, protocols, interfaces, and state machines. Taking those, develop an implementation plan. Working from that, write the skeleton of the application, then fill it in to create a functional system using a novel combination of hardware.

When you put it that way, isn't it crazy you have to tell it to do that? Like shouldn't it just figure out it needs to do that?

▲

marginalia_nu 25 minutes ago | parent | prev [-]

Google has had ample ability to address this problem, it's really not that hard. The reason it remains such a difficult problem for them to solve is that most of the things that would solve the problem would also decimate their ad revenue.