| ▲ | dbbk 4 hours ago |
| When they say "can't hallucinate" they mean they produce a confidence value for every result, so you could see for example it has 0.1 confidence, and you can disregard the result - that'd be different from hallucinating where it believes it's correct |
|
| ▲ | 8note 16 minutes ago | parent | next [-] |
| if it puts a high confidence value on a wrong answer, thats still hallucinating, no? llm hallucinations are high probability tokens that are incorrect vs the real world |
|
| ▲ | janalsncm 3 hours ago | parent | prev | next [-] |
| Technically speaking when you send the prefix “The capital of France is “ into an LLM it will also produce probabilities across its whole vocabulary. |
| |
| ▲ | sothatsit an hour ago | parent | next [-] | | The probability values don’t really represent confidence in modern LLMs though, especially after RLHF and RLVR. System One says they use RLCD, Reinforcement Learning for Calibrated Decisions, which presumably has accurate probabilities as an explicit optimisation goal. | |
| ▲ | jiggawatts 2 hours ago | parent | prev [-] | | … which they could provide in their APIs but are vehemently opposed to because it makes distillation much easier, and faster. |
|
|
| ▲ | CompleteSkeptic 3 hours ago | parent | prev | next [-] |
| that's right, but because these models are probabilistic, it's also possible to be confidently wrong (and all future models will be smarter still and still have that possibility) |
|
| ▲ | orbital-decay 2 hours ago | parent | prev [-] |
| Yeah but what stops it from producing confidently incorrect outputs... |
| |
| ▲ | zenlikethat an hour ago | parent [-] | | Nothing, but imagine using LLMs for a classification task People out there are so resigned to the models being unreliable that they are really doing things like hallucinating deliberately, and then matching the hallucinations to embeddings - https://softwaredoug.com/blog/2026/08/10/hypothetical-classi... You could do that or you could just... use a model that will never produce unreliable outputs in the first place. | | |
| ▲ | threecheese an hour ago | parent [-] | | But we're going from "Apple" to "Apple: 99% - trust me". It could still be an image of an orange :) | | |
| ▲ | zenlikethat an hour ago | parent [-] | | It's pretty darn smart. If you did want to hack on it in earnest and find out for yourself, send me an email - nathan@typesafe.ai |
|
|
|
