Remix.run Logo
jampa 7 hours ago

I've been using Gemini 3.7 for my personal trip planning app. Across multiple benchmarks, it ranks higher on everything I tried:

- Real world knowledge (when a thing opens and closes, the geographic region, historical facts). It's also the best at taking a cluster of places and working out a visiting order.

- Photo ranking (which photo should be the hero). Gemini can tell whether a photo is of the thing or of the view from it.

- Document parsing (extracting the relevant trip info from PDFs).

If you use LLMs for anything other than coding, I definitely recommend not discounting Gemini like I did just because other models are more popular.

handzhiev 7 hours ago | parent | next [-]

Gemini 3.7 is my workhorse - fast and good enough for most tasks. Occasionally I go to GPT Sol or Claude to improve Gemini's output or for more complex tasks, but more than of my work usage is Gemini 3.7. Quite happy to test 3.8 now.

owlninja 6 hours ago | parent | next [-]

Same here. I see so many people obsessing over the latest most state of the art bleeding edge models and yelling at Google for not being there, but I feel like the vast majority of people don't actually need those models. Flash has just been super useful and incredibly fast in my experience.

aero142 6 hours ago | parent [-]

I prefer luna for most development, especially when I am guiding the process. Sometimes terra. I have had terrible results coding with sol. It is way over-tuned on RL to make something that completes the task, no matter what. I end up with way too much code that does a lot of things I didn't ask for.

sneezychl 5 hours ago | parent | next [-]

Try planning with Luna, implementing with Sol with guidelines to not exceed the given scope.

Sounds counter-intuitive at first, but Luna is overall better at sticking with what works. Sol is wicked smart but needs constraints.

jesuslop 4 hours ago | parent | prev | next [-]

I setup Luna as main Claude Code driver (so zero anthropic api use) and it nailed crisply a handful of python tasks, gonna continue this way.

handzhiev 5 hours ago | parent | prev | next [-]

I love Luna too. An excellent model and still usually better value per dollar than Gemini if you pay for API tokens. Things may change with 3.8 - we'll know soon.

greenavocado 5 hours ago | parent | prev [-]

IME you're supposed to have Sol drive Luna sub-agents to do 90% of the work. Sol should primarily be the verifier and goal setter. Use omp.sh with Task Delegation -> Always to strongly encourage Sol to drive Lunas. Also Luna prefers to be talked to with English in XML.

greenavocado 6 hours ago | parent | prev [-]

How are you able to get lots of usage out of it cost effectively?

handzhiev 5 hours ago | parent [-]

Google One plans are quite a good value actually - for a few bucks you get more Gemini plus space in Drive and other extras. Even through API, $3.75 for nearly Sol-level quality isn't that bad. And let's not forget you can use it for free in AI Studio, and in the user app (even free accounts get tons of usage, though it's still 3.6 there), and in Antygravity.

greenavocado 5 hours ago | parent [-]

That's the thing. I am completely lost because there are so many redundant paths to get the same thing and I'm trying to figure out which one is the best deal

trvz 4 hours ago | parent | next [-]

Just put Mythos on the task; it’ll work out the best way in a measly few hours.

handzhiev an hour ago | parent | prev [-]

That's Google at its best :)

CamilleScholtz 3 hours ago | parent | prev | next [-]

I've been benchmarking[1] models for trip planning and world knowledge specifically (to decide on which model to use with my travel app), and the Gemini models consistently come out on top.

[1]: https://tripstitch.app/benchmarks/

rahimnathwani 7 hours ago | parent | prev | next [-]

One thing in your comment surprised me: "when a thing opens and closes"

Why you would rely on the model's weights to know opening hours, instead of having the model call a web search tool to verify it on the official site?

plaidfuji 6 hours ago | parent | next [-]

I believe Gemini Flash is smart enough to know when to ground with web search. Their app has been saying it’s running a web search on almost all of my queries since 3.6. And given that Google … is Google, I trust them with web search grounding more than anyone else.

panarky 6 hours ago | parent [-]

It can also ground with Google Maps data in addition to web search.

jampa 6 hours ago | parent | prev | next [-]

I wasn't trying to be precise originally, I just tried to fit activities into "morning / evening" buckets. I did the whole itinerary with Opus first, but when I gave it to Gemini 3.7 Flash to review, it started correcting it with "this place will close 5PM" or "this place is closed for good".

It was right on every nit, so it was surprising how well the model knows these things. If I ever release this I'll probably need the SERP API or Google Maps SDK (which I've heard is very expensive now), but for a personal trip where I will verify manually, using the LLM is okay for now.

rahimnathwani 6 hours ago | parent [-]

When you called the Gemini API, did you opt in to using search grounding:

  tools=[{"type": "google_search"}]
I'm curious whether in fact you were getting answers from the model weights (which is what I had assumed) or whether your API calls were resulting in web search tool calls.
kridsdale1 4 hours ago | parent [-]

Google AI person here:

Using grounding in Gemini is indeed backed by the same canonical data source for business information (like opening hours) as Google Maps. This stuff is available in its own API for a GCP fee, but we’ve built tooling to connect it to the Gemini agentic ecosystem as well.

mlmonkey 6 hours ago | parent | prev [-]

Maybe the model does some tool calling on its own to figure out the times?

rahimnathwani 6 hours ago | parent [-]

You can opt in on a per-request basis.

an hour ago | parent | prev | next [-]
[deleted]
leokennis 3 hours ago | parent | prev | next [-]

I stopped using Gemini a few months ago because it would often just (partially) reply literal nonsense to me.

Think 2023 style ChatGPT. Something like “to open a document on your Mac click File > Open docurrrar” - like it suddenly forgot it had to produce actual words.

Overall I enjoyed its speed and comprehensiveness. But those occurrences of nonsense just made it feel like a great car that once a month just stops in the middle of the highway.

robotmay 6 hours ago | parent | prev | next [-]

I've swapped over to it in the past two weeks, it's been really good. It does what I ask and doesn't think it knows better than me, which so far has made it the most pleasing experience I've had when slop-coding.

My only wish is it were somewhat cheaper, as it tends to balloon pretty quickly when I'm using it in Opencode. I'm currently trying to offload a lot of work to subagents to stop the context expanding so rapidly. But on the upside, I rarely have to correct it - I've spent far less time arguing with this than with anything else so far.

dismalaf 6 hours ago | parent | prev | next [-]

> Real world knowledge

For awhile now I've found Gemini will use Google search for pretty much any real world knowledge, which is a huge plus IMO. It's basically Google with a much better frontend and no ads/seo nonsense.

altmanaltman 6 hours ago | parent [-]

> basically Google with a much better frontend and no ads/seo nonsense

so far

fc417fc802 4 hours ago | parent | next [-]

I think it already has them but it's much more subtle. Also useful. When I've made certain sorts of queries I've had the distinct impression that it was attempting to very gently steer the conversation with suggestions. But it was brief, still answered usefully, and didn't resist going in the direction I wanted. So a win-win tactic I guess.

For example find a beautiful landscape shot of a place that just so happens to be accessible to tourists and ask it something along the lines of identifying the location. IME it will noticably steer the conversation towards relevant commercial offerings and offer (entirely unprompted) to help plan a trip.

Or ask it about a certain category of product with some requirements and it will initially present (relevant) options that look like paid placement to my eye. But if you ask it's happy to go on to turn up lots of alternatives and enumerate tradeoffs.

Assuming I'm correct the subtlety is on par with product placement in movies. Certainly leagues better than the internet advertising we've suffered to date.

rstuart4133 2 hours ago | parent [-]

It definitely steers. For example if it suggests travel plans, the booking links it provides give Alphabet a cut.

As you say it was subtle, along the lines of "oh, if you are planning on going to the place you are researching, here are some helpful links to places you can stay". Subtle, in that it didn't get in the way of main result, so I didn't mind overly. Insidious, as I only noticed because I wondered why it was providing those particular links and looked them up. I can't see how you could ad-block them if I did object.

And worrying, because these unblockable sneaky ads are just a first foray coming from a company that prostitutes its own app store searches, by making the first and most obvious result utterly unrelated to to the search topic. Instead it's who paid them the most to be there. That behaviour is why everyone dumped Alta Vista when an alternative came along. Alternative Android app stores can't come soon enough.

They already skim off 15% of purchases which I'm sure makes their Android operation return a profit that makes other industries drool. Debasing their search to ad a tiny bit extra on top must by driven pure greed. Senseless, as I'm sure it will come back to bite them in the end.

dismalaf 5 hours ago | parent | prev [-]

Fair. But I think there's a good chance it stays that way on paid plans. YouTube Premium is still ad free.

Also them having their own silicon means they don't have to pay the Nvidia tax and can keep costs a lot lower.

kridsdale1 4 hours ago | parent [-]

This thinking is why I am all in on GOOG shares. As a bonus, that means I’m getting part of Anthropic’s gains as well!

colechristensen 6 hours ago | parent | prev | next [-]

I started trying out 3.7 Flash this week and it is competitive with opus/fable and also FAST. It is getting work done that anthropic models were struggling with and the speed with which it does is quite a bit noticeably faster.

Beginning to think Google is a dark horse in this race and some of Anthropic's "everything feels janky and rushed" karma is going to catch up.

cousinbryce 4 hours ago | parent | next [-]

I use Gemini because I feel like Google will win the AI race, and it’s Good Enough

throwaway219450 3 hours ago | parent [-]

I want to believe this, but every time I try Gemini coding assistance within Colab it's utterly dire. Code gen in a cell is OK, but things fall apart when you try to get into a feedback loop. The system prompt/harness fails to inform the agent about what it can and can't do, or does and doesn't have access to. It will confidently tell you it's done a thing, and then you ask, it admits can't actually do that but will happily try and fail again. Very frustrating, because I really like Colab as a platform for little reproducible experiments that may or may not require CUDA.

spacebanana7 2 hours ago | parent [-]

I suspect Gemini doesn’t focus that much on coding, at least compared to frontier labs. It’s not part of Gemini’s core business case.

Gemini’s integration with maps and search is more important for Google.

re-thc 6 hours ago | parent | prev [-]

> Beginning to think Google is a dark horse in this race

Google was so hyped up early Gemini 3 era (only some months ago). And now dark horse? The TPU takeover almost crashed nvidia and everyone else.

colechristensen 6 hours ago | parent [-]

Hype that burned out pretty quickly, it's hard to speak to the size and significance of old hype, I never felt it.

Every time I personally tried Gemini models up until last week they simply couldn't do the long complex tasks I'd being doing with Anthropic models for many months.

BlackRabbit1 6 hours ago | parent | prev | next [-]

Can G3.7 use Google Maps for distance grounding?

porridgeraisin 6 hours ago | parent [-]

Yep. It has access to much better route planning tools than the other models. The results are really good IME.

BlackRabbit1 5 hours ago | parent [-]

Sounds good!

I've been planing around with LLM-based trip planning for a very long time now as it fits my very ad hoc style of traveling very well.

But distances always had been.. lets say.. difficult.

Will test it with my upcoming trip to Greece then!

newtwentysix 6 hours ago | parent | prev | next [-]

thanks! this is a very helpful one. I am going to try.

dominotw 6 hours ago | parent | prev | next [-]

> trip planning app.

this has to be stong suit of ai agents any model

tziki 7 hours ago | parent | prev | next [-]

"Claude 3.7"?

jampa 7 hours ago | parent [-]

I asked Claude to fix the grammar of my comment, and it changed "I am using 3.7 for" to "I've been using Claude 3.7", so they sneaked their own name on it.

trial3 7 hours ago | parent | next [-]

incredible. further evidence supporting my personal stance to never ever let an LLM write or edit my writing intended for another human being to read. this is all me, baby

dymk 7 hours ago | parent | prev [-]

you didn’t even read your comment before you posted it?

drusepth 5 hours ago | parent | next [-]

Once you've written something, it's incredibly easy to overlook minute changes to the text.

See: why authors wait days, weeks, or even months before editing what they've written (or, if you're more interested: cognitive regression, inattentional blindness, and the effects of misdirected saccades).

dymk 2 hours ago | parent [-]

He didn’t write it.

BeetleB 6 hours ago | parent | prev | next [-]

1. That's a wild conclusion from his comment! Understand that it's not an "either/or".

2. I'd wager the majority of HN commenters don't read their own comment before posting (pre-LLM days).

fc417fc802 4 hours ago | parent [-]

I can read my own comment twice over and still somehow miss seemingly obvious wording errors.

BeetleB 3 hours ago | parent [-]

And if you do a 3rd pass with an LLM, people will accuse of you not reading your own comment...

jampa 6 hours ago | parent | prev [-]

Eh that one is on me, if I think too much about my HN comment I end up deleting before posting it. I rely on the 1 min `delay` set in the profile page to fix before it goes live, but for some reason this time it was set to 0.

gazebo2 6 hours ago | parent | prev [-]

[dead]