Remix.run Logo
derefr 4 hours ago

I haven't stress-tested it, but I would imagine it approaches complex problems the same way a human with a phone in their pocket would — that is, by having a degree of awareness of the confidence it has in its own knowledge in some areas; where, when it "realizes that it doesn't know", it blocks the conversation with statements like "I don't know, let me check."

I say this because this is already how ChatGPT works internally when using its "auto" mode; the version of the "fast" model used in the "auto" mode does the same "notice your ignorance and bring in the heavy model" thing, just silently, rather than mentioning that it's doing it.

(If someone has actually run the experiment, please chime in!)