If I'm getting citations, and the distillation has strong alignment with citations, then it's fine. In other words: the if the summary is a highlight and not a rewrite then I'm happy. I've studied this extensively and have even created metrics around this problem: https://maxirwin.com/articles/llm-rag/

▲

irishcoffee 4 hours ago | parent [-]

So, if you review the entire dataset? At that point just write the summary yourself.

I use "AI" a lot, it is a fantastic tool. We need to stop pretending it is some kind of panacea. It's a tool.

	▲	Forgeties79 3 hours ago \| parent [-]
		> I use "AI" a lot, it is a fantastic tool. We need to stop pretending it is some kind of panacea. It's a tool. I have been repeating some variation of this for months. If people would stop acting like it’s THE tech solution to ALL things ALL the time I bet a lot of critics would quiet down. The overhyping has become exhausting. It’s been going on for years. GPT messed up the math for me the other day when I was simply adding 10 durations for a TRT. Couple of HH:MM:SS inputs, annoying to add up and I had it open. It got it wrong, I told it it was wrong, it got it wrong again. Then out of curiosity I provided it the answer, asked for it to confirm it against the original numbers, and it went “you’re absolute right, it’s [original/wrong answer from earlier].” This stuff happens probably 10-15% of the time for me regardless of the model. Not just math, just super simple crap. It’s wild to see at this point after 3-4 solid years of “hyperscaling” and overhyping. And it’s the kind of thing that keeps people like me from buying in beyond the foot or two we’ve stuck in the water.