It’s no longer true that AI tools primarily get knowledge from their pre-training input data. That gives them a baseline, but nowadays AI chatbots and coding agents routinely assume they need to get up-to-date information in other ways, via web searches and other tool calling.

So I don’t see accuracy declining at least for programming.

▲

gortok an hour ago | parent [-]

> nowadays AI chatbots and coding agents routinely assume they need to get up-to-date information in other ways, via web searches and other tool calling. So I don’t see accuracy declining at least for programming.

How do those chat bots discern that the ‘web searches’ they’re using are returning human generated information only that’s been vetted instead of LLM output?

▲

ako 33 minutes ago | parent | next [-]

They don't need to, they can use tools to validate their assumptions.

▲

taurath 24 minutes ago | parent [-]

If someone has a vibe coded website with bad statistics but shows up on a web search, what tools will it use to check those statistics? How will it know what data it needs to validate? What tools will it use?

	▲	foldr 8 minutes ago \| parent [-]
		Humans face the same problem. So this at least shouldn't make AI perform worse relative to humans, even if AI slop degrades the performance of both over time.

▲

skybrian an hour ago | parent | prev | next [-]

It’s true that they are only as good as their input data, but the same is true if you do your own web searches.

▲

taurath 22 minutes ago | parent [-]

I know within a second whether the search result I am looking at is obvious AI slop (search for almost any health condition, recipe, esoteric questions etc almost always have slop at the top of the result), but LLMs regularly source those sites as the basis for conclusions.

▲

kristiandupont 17 minutes ago | parent [-]

>I know within a second

I am sorry to be pedantic, but the correct phrasing would be "I think I know within a second", which seems like a pretty important distinction.

	▲	taurath 14 minutes ago \| parent [-]
		Sure and thats fair, there's probably many things that are good enough that I miss, but I'm talking about the most obvious possible websites being used regularly as sources in web search. Identifying poor sourcing or misinformation on the internet and what is a credible source has been a lifelong skill I've had to build from the early days of the web, and in the AI boom its only gotten more necessary to be able to not get taken by hucksters with billion dollar budgets.

▲

WarmWash an hour ago | parent | prev | next [-]

Every lab is already training on synthetic data and has been for years now.

▲

vitally3643 44 minutes ago | parent | prev [-]

The same way you and I do: vibes

Welcome to the postmodern internet. It's vibes all the way down.

	▲	gortok 40 minutes ago \| parent [-]
		Upvoted you, of course; but it’s worse than that. It’s vibes being marketed as correctness. To the lay person (and unfortunately, to more than a few folks who should know better), computers don’t “make up” information. Maybe some good (in some weird way) that comes from all of this is that we stop using LLMs for recitation of facts.