It's not that direct a counterexample. We have no idea what underlying data from the Fallout show they gave to the model to summarize. Surely it wasn't the scripts of the episodes. The nature of the error makes me think it might have been given stills of the show to analyze visually. In this case we know it is the text of the book.

▲

ceejayoz 3 days ago | parent [-]

> It's not that direct a counterexample.

Amazon made a video with AI summarizing their own show, and got it broadly wrong. Why would we expect their book analysis to be dramatically better - especially as far fewer human eyes are presumably on the summaries of some random book that sold 500 copies than official marketing pushes for the Fallout show.

▲

dcre 3 days ago | parent | next [-]

For the reason I gave in my answer: it would be answering based on the text of the book. I don't expect it to be particularly great regardless because these features always use cheap models.

▲

ceejayoz 3 days ago | parent [-]

> For the reason I gave in my answer: it would be answering based on the text of the book.

Why would that not also be true for the Fallout season one recap video?

▲

dcre 3 days ago | parent [-]

Did you read a word past the part of my answer you quoted?

	▲	ceejayoz 2 days ago \| parent [-]
		Of course. I’m just not sure how “the Kindle feature’s cheap models are gonna be even worse at the task” helps your point.

▲

catgary 3 days ago | parent | prev [-]

Because text analysis is substantially easier than video analysis?

	▲	ceejayoz 3 days ago \| parent [-]
		Amazon has the Fallout scripts, subtitles, internal show bibles, etc. all available to them.