|
| ▲ | brailsafe 4 days ago | parent | next [-] |
| > It is search if you ask it to produce a list of links. Not in the example I mentioned. It can imagine the links, and the content of the links, and be very confident about it. It literally invented an obituary that didn't exist, gave me a link to a funeral home that 404'd, came up with "in-memoriam" references from regional newsletters that never contained her name. It's actually really scary how specifically fake it was. I asked it to produce verbatim references from any sources and the links to them, and none of the text it produced could be searched with quotes on any search engine. |
| |
| ▲ | xupybd 4 days ago | parent [-] | | Yeah it does punt sometimes but often it works well. Especially on more well written about topics. | | |
| ▲ | brailsafe 4 days ago | parent [-] | | I think that's the tricky thing. I'm not saying it's not useful when it is, but you really do need a keen and skeptical eye to be able to know. The problem kind of reminds of bloom filters, such that they're useful for situations when you want to know something might exist or definitely does not exist in a set. Exact truth has some level of permissible error rate, as it does in any situation, but definitely wrong is pretty important to know about. |
|
|
|
| ▲ | duskdozer 5 days ago | parent | prev | next [-] |
| The issue as I see it is just straight copy/pasting its output. You want to use it as a search tool to give you pointers on things to look up and links to read? Great. Then use that as a basis to read the sources and write your own response. If you aren't familiar enough with the subject area to do that, then you also shouldn't be pasting LLM output on it. |
| |
| ▲ | brailsafe 4 days ago | parent [-] | | It's not even copy/pasting in some cases. In my example, it confidentially produced "verbatim" references that don't exist anywhere, to specific pages that never mentioned this person's name or contained any of the text. Sometimes completely different people, 404 pages, huge waste of time | | |
| ▲ | duskdozer 3 days ago | parent [-] | | Yeah I agree, I've seen the hallucinated references also. Sometimes used by people in internet arguments to make their bullshit seem more legitimate. What I meant though by copy/pasting is people getting the LLM output and then just directly feeding that to the people they're conversing with, instead of looking into what it's saying or really even engaging with it in any way. |
|
|
|
| ▲ | anjel 5 days ago | parent | prev | next [-] |
| Ask it to solve a tough Euler Math puzzle with the search button on and it just copies the answer from the web. Turn search off and it actually computes the answer.
Funny how the search button is taken away though. |
|
| ▲ | roarcher 5 days ago | parent | prev [-] |
| > It is search if you ask it to produce a list of links. Unfortunately it can hallucinate those too. I've had ChatGPT cite countless nonexistent academic papers, complete with links that go nowhere. |
| |
| ▲ | tharkun__ 5 days ago | parent [-] | | Which is "fine" so to speak. We do this with using AIs for coding all the time, don't we? As in, we ask it to do things or tell us things about our code base (which we might be new to as well) but essentially use it as a "search engine+" so to speak. Hopefully it's faster and can provide some sort of understanding faster than we could with searching ourselves and building a mental model while doing it. But we still need to ask it for and then follow file and line number references (aka "links") and verify it's true and it got the references right and build enough of a mental model ourselves. With code (at least for our code base) it usually does get that right (the references) and I can verify. I might be biased because I both know our code base very well already (but not everything in detail) and I'm a very suspicious person, questioning everything. With humans it sometimes "drives them crazy" but the LLM doesn't mind when I call its BS over and over. I'm always "right" :P The problem is when you just trust anything it says. I think we need to treat it like a super junior that's trained to very convincingly BS you if it's out of its depth. But it's still great to have said junior do your bidding while you do other things and faster than an actual junior and this junior is available 24/7 (barring any outages ;)). |
|
