I assume you're aware but there are some open source assistants that can get you this (at least to some extent).

E.g. Codename Goose https://block.github.io/goose/docs/quickstart/

Which I think supports Gemini among with all the other major AI providers, plus MCP. I have heard anecdotally that it doesn't work as well as Claude Code, so maybe there are additional smarts required to make a top notch agent.

I've also heard that Claude is just the best LLM at tool use at the moment, so YMMV with other models.