I think what they're saying is that you need the summaries to do these things

It's easier and simpler to use an LLM service than to maintain those ad hoc models. Many replaced their old NLP pipelines with LLMs.

Sentiment analysis was not solved and companies were paying analyst firms shit tons of money to do that for them manually.

So, I wouldn't be surprised if someone in charge of a QA/ops department chose LLMs over similarly effective existing ML models in part because the AI hype is hitting so hard right now.

Two things _would_ surprise me, though:

- That they'd integrate it into any meaningful process without having done actual analysis of the LLM based perf vs their existing tech

- That they'd integrate the LLM into a core process their department is judged on knowing it was substantially worse when they could find a less impactful place to sneak it in

I'm not saying those are impossible realities. I've certainly known call center senior management to make more hairbrained decisions than that, but barring more insight I personally default to assuming OP isn't among the hairbrained.

Yeah, I was a QA data analyst supporting three multi-thousand agent call-centers for an F500 in 2012 and we were using phoneme matching for transcript categorization. It was definitely good enough for pretty nuanced analysis.

I'm not saying any given department should, by some objective measure, switch to LLMs and I actually default to a certain level of skepticism whenever my department talks about applications.

I'm just saying I can imagine plausible realities where an intelligent and competent person would choose to switch toward using LLMs in a call center context.

There are also a ton of plausible realities where someone is just riding the hype train gunning for the next promotion.

I think it's useful to talk about alternate strategies and how they might compare, but I'm personally just defaulting to assuming the OP made a reasonable decision and didn't want to write a novel to justify it (a trait I don't suffer from, apparently), vs assuming they just have no idea what they're doing.

Everyone is free to decide which assumed reality they want to respond to. I just have a different default.

Also waiting music being interrupted every minute to tell:

1) my call is very important to them (it's not)

2) listen carefully because options changed (when? 5 years ago?)

3) they have a website where I can do things (you can't, otherwise why would I call?)

4) please stay at the end of call to give them feedback (sure, I will waste more of my time)

Perhaps doing this suggested auto-summarizing would be what finally solves that problem?

I'm in love with email-based support as I am on both sides of the chain. When I raise a problem, the engineers on the other side can work at their pace, escalate when needed, and I almost always get a reasonably good reply. I can dig deeper if I wish, and I'm pretty sure the guys on the other end are doing their best.

It works the same way when I'm helping someone else: most reasonable people don't expect that if they make an audio call I will magically solve their problem faster. Maybe it will be slower and they will get a lower-quality ad-hoc solution.

> But in fact, customer call centers tend not to be able to even know that you called in yesterday, three days ago and last week.

Nor what you told the person you talked to three minutes earlier, during the same call, before they transferred you to someone else. Because their performance is measured on how quickly they can get rid of you.

I've always guessed that they are able to tell when you called/what you called about, but they simply don't give that level of information to their frontline folks.

I am sorry about your bad experience. Maybe the ones you called did not have AI transcribed summaries and were not managed by GP?

Product teams analyse call summaries at scale to guide the roadmap to reduce future calls. It’s not just about case management.

I can assure you that people care very much about searching and mining calls, especially for compliance and QA reasons.

I would imagine OP is probably mining service call summaries to find common service issues, or at least that's what I would do.

Advanced organizations (think not startups, but companies that have had years of decades of profit in the public market) might have solved all the low-hanging fruit problems and have staff doing things like automated quality audits (search summaries for swearing, abusive language, etc).

we do

Thanks for the detailed domain-specific explanation, if we assume that some whale clients of the company will end up in the call center is it not more probable that more competent human agents will be responsible for the call, whereas it's pretty much the same AI agent adressing the whale client as the regular customers in the alternative scenario?

I genuinely don't think that the GP is actually making someone actually listen to the transcription and summary and check if the summary is wrong.

I almost have this gut feeling that its the case (I may be wrong though)

Like imagine this, if the agent could just spend 3 minutes writing a summary, why would you use AI to create a summary and then have some other person listen to the whole audio recording and check if the summary is right

like it would take an agent 3 minutes out of lets say a 1 hour long conversation / (call?)

on the other hand you have someone listen to 1 hour whole recording and then check the summary? that's now 1 hour compared to 3 minutes Nah, I don't think so.

Even if we assume that multiple agents are contacted in the same call, they can all simply write the summary of what they did and to whom they redirected and just follow that line of summaries.

And after this, I think that your summary of seeing that they are really screwing away is accurately true.

Kinda funny how the gp comment was the first thing that I saw in this post and how even I was kinda convinced that they are one of the more smarter ones integrating AI but your comment made me come to realization of them actually just screwing themselves.

Imagine the irony, that a post about how AI companies are screwing themselves by burning a lot of money and then the people using them don't get any value out of it.

And then the one on Hn that sounded like it finally made sense for them is also not making sense... and they are screwing over themselves.

The irony is just ridiculous. So funny it made me giggle

>in the context of call centers in particular I actually can believe that a moderately inaccurate AI model could be better on average than harried humans

You're free to believe that of course, but you're assuming the point that has to be proven. Not all fuck ups are equal. Missing information is one thing, but writing literally opposite of what is said is way higher on the fuck up list. A human agent would be achieving an impressive level of incompetence if they kept on repeating such a mistake, and would definately have been jettisoned from the task after at most three strikes (assuming someone notices). But firing a specific AI agent that repeats such mistakes is out of the question for some reason.

Feel free to expand on why no amount of mistakes in AI summaries will outweigh the benefits in call centers.

Especially humans whose jobs are performance-graded on how quickly they can start talking to the next customer.

A company that could save millions by not having staff write up their own call notes almost surely is already doing that.

This is the bread and butter of call centers and the companies that use them. The transcripts and summaries are used from everything from product improvement to agent assessment. This data is used continuously. Its not like they use this transcript for the one rare time someone sues because they claim an agent lied. That rarely happens.

The "last second" is right after the call.

For example, if 60% of your calls this month mention assembly issues with the product, that information will help you improve it.

This is practical, not theoretical.