Isn't Google Maps the main culprit here if it didn't update the info? And more generally, also the castle management for not updating Google Maps to reflect its closed?

Also this is the Flash-Lite model, which they specifically don't want you to pay for.

▲

password54321 an hour ago | parent | next [-]

It failed to make a connection between "now" and the current time, nothing to do with Google Maps.

▲

stephbook 2 hours ago | parent | prev [-]

You're right, this is Lite. I change my charge to "this is the AI you're going to get on the free tier."

I mean, sure you can blame Google Maps, but it wasn't my choice to consult Google Maps.

I think noticeable here is how hard it is to steer this trash model even when fed already correct information.

I asked "why closed?" and it corrected me to "it's open" with wrong information. Then it got the current time wrong.

▲

altmanaltman an hour ago | parent [-]

> I mean, sure you can blame Google Maps, but it wasn't my choice to consult Google Maps.

Okay but isn't that obvious that's what would happen? Like even if you ask a human, that's where they will consult and if it shows its open, it doesn't really have any incentive to question the validity of the data.

But more so, it is on the management of the place as well that they have a Google My Business Page but do not update it. If they say its open, why would Google Maps mark it as closed? If it's marked as open on Google Maps, why would Gemini Flash Lite refute that?

Blaming the LLM for this makes little sense imo. Sure it's a dumb model but in this case, it was set up to lose.

▲

nickspacek an hour ago | parent [-]

The point is that it got to the right answer eventually, which conceivably it could have done at the first step:

1. Open on Google Maps?

2. Validate no extra notices posted on their public website

Even the fact that there is are special notices sometimes posted on their public website is something a model could have been trained to understand, same as a human being.

	▲	simianwords an hour ago \| parent [-]
		Yeah but are you willing to pay additional cost for marginal improvement in accuracy that only pays off sometimes? If so, take the judgement and use a better model.