The problem we're seeing across many professions is AI output is not getting vetted by knowledgeable people, whether it's an experienced analyst, senior engineer, expert attorney, or the resident physician. At best they skim, at worst they don't even see it at all before it's published, pushed to production, distributed to clients, or submitted to the court.

In many cases the skills are available in house to do the necessary vetting, but these people are already overwhelmed with their existing day to day.

Anyone remember that item a few months back about Amazon now having senior engineers vet generative AI output (https://news.ycombinator.com/item?id=47323017)? I had to LOL when I read that. These folks are already slammed. And the idea that Amazon would allow human bottlenecks to multiply across projects and underlying infrastructure development is ridiculous.

Part of the problem: you get given a complete document to review after it's been fully baked.

I'm pushing the need for basic engineering principles across whole organisations.

You wouldn't give an engineer 1000 lines of code to review without the original spec of what you're trying to achieve for context (at a minimum, ideally the reviewer was in the room when the work was introduced, and has full context).

So, these docs, they're given as an all or nothing.

Do you push back on the 39th metric that is defined to the utmost detail? Or just resign yourself to the fact that it is what it is?

A one (6 is the goto if we're talking Amazon?!) pager.. "this is what I am proposing" at least gives the skeleton of the idea to push back at the general shape of the idea, refine it, before all the emotional investment of your precious report being complete.

Y'know.. the traditional product running through the spec in a SCRUM* environment.. the engineers doing proper code reviews..

* Yes SCRUM is dead, but that's another thing.

> AI output is not getting vetted by knowledgeable people

You mean the people they fired and demoralized?

One of the things that "great [wo]men" like about "vibe-coding" (and that includes blindly producing non-code product), is that they, and they alone can now do what used to require the painful process of "passing it to context experts."

Now, the LLM is a "built-in context expert," and they don't need to vet the output anymore.

As an attorney, I feel like vetting AI output takes longer than just doing it from scratch, let alone versus just using a traditional form.

With AI, I have to read through everything, often explain why it's wrong, and then rewrite everything anyways. I mean, I get way more billables, but I think it's symptomatic of how AI loses its advantage of being quick and accessible to those who don't understand the subject matter.

>>The problem we're seeing across many professions is AI output is not getting vetted by knowledgeable people

I am particularly interested in Education and Human Knowledge Management. I have seen the rate of IT training going to zero. Think about specialized training, where if you make a mistake, the consequence of your errors, are talked about on the tv news of the evening.

The whole idea everybody is just planning to save their butt, using these strings coming out of these numeric matrices, while suspending judgement, just shudders me in horror. A bit like those South Asia Airline companies, that were forbidding their pilots from landing airplanes with manual piloting, leading to an increase loss of skills causing some well known disasters...

If well paid consultants cant even bother to check their links...

> In many cases the skills are available in house to do the necessary vetting, but these people are already overwhelmed with their existing day to day.

This is an interesting topic. We treat vetting output the same as doing the work ourselves, but that is not the case.

Doing the work is not the same as reviewing work done by others.

I have heard reports of software engineering companies that have gone full agentic. Their seniors only review stuff written by LLMs and it burns them out, because they have to switch context constantly.

I find this interesting because part of being a senior developer is that you are experienced enough that you won‘t make grave mistakes anymore. This is the case in many professions: you are relied upon to not make grave mistakes.

But those same people are now swamped with stuff that they are not able to review, so they will let a grave mistake slip through at some point.

So they really can‘t trust themselves anymore?

> The problem we're seeing across many professions is AI output is not getting vetted by knowledgeable people

The problem is that output sometimes take longer to verify than to create in the first place.

That turns AI into a deeply negative ROI system for many applications.

Also wondering on this whole review process with someone who wrote it with AI. Even if you comment and noted all issues. Do they have skills or willingness to correctly correct it all? And how many times would you need to keep the loop going for error free outcome? Is there even enough calendar time for that?

But wait, if knowledgeable people have to vet the output, the process will not be 10X faster and you will not be able to fire the knowledgeable people. Therefore, your objection makes no sense. QED.

> the idea that Amazon would allow human bottlenecks to appear across projects and underlying infrastructure is ridiculous.

Why?

> The problem we're seeing across many professions is AI output is not getting vetted by knowledgeable people, whether it's an experienced analyst, senior engineer, expert attorney, or the resident physician.

Yeah probably not for the same reason I left VFX rather than have a lifetime of completely disregarding my own generative creativity and cleaning up LLM-generated bullshit. Fuck that. Double-fuck creating ‘content’ to train the models.

In code, LLMs automate away a lot of the drudgery. I wasn’t sad to avoid spending a couple hours looking up the usage patterns and idioms for some ported library, or do some rote task that didn’t make the project significantly better. In most other jobs, they automate away the only fun part and leave humans with all of the drudgery.

The tech industry has always been arrogant to some extent, but assuming the world of talented professional knowledge workers and creatives would be content to professionally proofread, apply lipstick to pigs, and polish turds is a whole new level of out-of-touch. I’d rather live out of my car and dig through the garbage for bottles with deposits.

> In many cases the skills are available in house to do the necessary vetting, but these people are already overwhelmed with their existing day to day.

I think a lot of the time it's just pure laziness. AI gives people a magical "do all the work for me" button and it can bring out the worst in them.

If the main job is putting out a report, starting with AI is wrong in any case. What's the value of an AI-generated report, even if experts fix the biggest issues with it? Maybe this kind of report didn't have all that much value before, I don't know. But starting with AI just makes sure it's generic drivel.

It is harder to check everything then to create a thing without lying in the first place.