Remix.run Logo
kace91 8 hours ago

The guide is generally very well thought, but I see an issue in this part:

It sets the rule that things must be actually read when there’s a social expectation (code interviews for example) but otherwise… remarks that use of LLMs to assist comprehension has little downside.

I find two problems with this:

- there is incoherence there. If LLMs are flawless in reading and summarization, there is no difference with reading the original. And if they aren’t flawless, then that flaw also extends to non social stuff.

- in practice, I haven’t found LLMs so good as reading assistants. I’ve send them to check a linked doc and they’ve just read the index and inferred the context, for example. Just yesterday I asked for a comparison of three technical books on a similar topic, and it wrongly guessed the third one rather than follow the three links.

There is a significant risk in placing a translation layer between content and reader.

fastball 3 hours ago | parent | next [-]

> It sets the rule that things must be actually read when there’s a social expectation (code interviews for example) but otherwise… remarks that use of LLMs to assist comprehension has little downside.

I think you got this backwards, because I don't think the RFD said that at all. The point was about a social expectation for writing, not for reading.

kace91 an hour ago | parent [-]

This is what I’m referencing:

>using LLMs to assist comprehension should not substitute for actually reading a document where such reading is socially expected.

gpm 6 hours ago | parent | prev [-]

> Just yesterday I asked for a comparison of three technical books on a similar topic, and it wrongly guessed the third one rather than follow the three links.

I would consider this a failure in their tool use capabilities, not their reading ones.

To use them to read things (without relying on their much less reliable tool use) take the thing and put it in the context window yourself.

They still aren't perfect of course, but they are reasonably good.

Three whole books likely exceeds their context window size of course, I'd take this as a sign that they aren't up to a task of that magnitude yet.

kace91 an hour ago | parent [-]

>Three whole books likely exceeds their context window size of course

This was not “read all three books”, this was “check these three links with the (known) book synopsis/reviews there” and it made up the third one.

>I would consider this a failure in their tool use capabilities, not their reading ones.

Id give it to you if I got an error message, but the text being enhanced with wrong-but-plausible data is clearly a failure of reliability.