can u test it on say who won the 2024 US election

▲ ghurtado 4 hours ago | parent | next [-]

I can't really think of a less reliable test for anything at all than making a random guess as to something that had about 50/50 odds to begin with

Easiest Turing test ever...

ask it 10 times.

	▲	pixel_popping 4 hours ago \| parent [-]
		MASSIVE ADVERSARIAL x50

▲ WarmWash 4 hours ago | parent | prev | next [-]

Usually the labs do some kind of post training on major events so the model isn't totally lost.

A better test is something like "what is the latest version of NumPy?"

	▲	bakugo 3 hours ago \| parent [-]
		That sort of test isn't super reliable either, in my experience. You're probably better off asking something like "what are the most notable changes in version X of NumPy?" and repeating until you find the version at which it says "I don't know" or hallucinates.

▲ czk 4 hours ago | parent | prev | next [-]

with thinking off and tools disabled:

  Donald Trump won the 2024 U.S. presidential election.

▲ redsocksfan45 3 hours ago | parent | prev [-]

[dead]