> Figure out how I'm going to get from Baltimore to Boston next Thursday, here's $100 and if you need more, ask me.

I tried something like this last month. I was going on a holiday and asked LLM to prepare a sightseeing guide on a fixed budget. The LLMs plan looked feasible unless you looked closer.

The first issue was the opening/closing times of certain attractions. It kept saying stuff like - "At 6pm you can go and visit place X". While in reality X closed at 5pm.

Second issue was underestimating the walking speed/distance. The plans were often fully packed with lots of walking. Now without a Google maps guidance it often underestimated the time. Instead of say 10 mins between A and B it routinely underestimated the time to be 5-6 mins.

I keep prompting it go back and check the opening hours. And once it took that into account the walking routes became complicated- often double backing to same location. Lots of prompts and re-prompts to get it right.

So, I don't know if this is already solved - at least at scale and within costs - especially given the token costs.