I asked: "How many resistors are used in fuzzhugger phantom octave guitar pedal?". It replied 29 resistors and provided a long list. Answer is 2 resistors: https://tagboardeffects.blogspot.com/2013/04/fuzzhugger-phan...

▲

iFire 3 days ago | parent | next [-]

> How many resistors are used in fuzzhugger phantom octave guitar pedal?

Weird, as someone not having a database of the web, I wouldn't be able to calculate either result.

▲

dvh 3 days ago | parent | next [-]

"I don't know" would be perfectly reasonable answer

▲

MaxikCZ 3 days ago | parent [-]

I feel like theres a time in near future where LLMs will be too cautious to answer any questions they arent sure about, and most of the human effort will go into pleading the LLM to at least try to give an answer, which will almost always be correct anyways.

	▲	plufz 3 days ago \| parent \| next [-]
		That would be a great if you could have a setting like temperature 0.0-1.0 (Only answer if you are 100% to guess as much as you like).
	▲	littlestymaar 3 days ago \| parent \| prev [-]
		It's not going to happen as the user would just leave the platform. It would be better for most API usage though, as for business doing just a fraction of the job with 100% accuracy is often much preferable than claiming to do 100% but 20% is garbage.

▲

kaoD 3 days ago | parent | prev | next [-]

> as someone not having a database of the web, I wouldn't be able to calculate either result

And that's how I know you're not an LLM!

▲

iFire 3 days ago | parent | prev [-]

I tend to pick things where I think the answer is in the introduction material like exams that test what was taught.

▲

esafak 3 days ago | parent | prev | next [-]

This is just trivia. I would not use it to test computers -- or humans.

▲

littlestymaar 3 days ago | parent | next [-]

It's good way to assess the model with respect to hallucinations though.

I don't think a model should know the answer, but it must be able to know that it doesn't know if you want to use it reliably.

	▲	esafak 3 days ago \| parent [-]
		No model is good at this yet. I'd expect the flagships to solve the first.

▲

parineum 3 days ago | parent | prev [-]

Everything is just trivia until you have a use for the answer.

OP provided a we link with the answer, aren't these models supposed to be trained on all of that data?

▲

esafak 3 days ago | parent | next [-]

There is nothing useful you can do with this information. You might as well memorize the phone book.

The model has a certain capacity -- quite limited in this case -- so there is an opportunity cost in learning one thing over another. That's why it is important to train on quality data; things you can build on top of.

▲

parineum 3 days ago | parent [-]

What if you are trying to fix one of these things and needed a list of replacement parts?

	▲	esafak 3 days ago \| parent [-]
		Not the right problem for this model. Any RAG-backed SLM would do; the important part is being backed by a search engine, like https://google.com/ai

▲

DennisP 3 days ago | parent | prev [-]

Just because it's in the training data doesn't mean the model can remember it. The parameters total 60 gigabytes, there's only so much trivia that can fit in there so it has to do lossy compression.

▲

brookst 3 days ago | parent | prev | next [-]

Where did you try it? I don’t see this model listed in the linked Qwen chat.

▲

bongodongobob 3 days ago | parent | prev | next [-]

Lol I asked it how many rooms I have in my house and it got that wrong. Llms are useless amirite

	▲	cindyllm 3 days ago \| parent [-]
		[dead]

▲

strangattractor 3 days ago | parent | prev [-]

Maybe it thinks some of those 29 are in series:)