I'm very pleased with the performance of the largest gemma4 model (which I tested through ollama). My singular data point on whether an LLM remembers things well is whether it can translate toki pona to (and from) English. I find it easy to evaluate because I know the language. This local LLM marks the first version that 1) doesn't hallucinate words - at least, for the largest model - and 2) uses common word-phrases that other toki pona speakers use, and most importantly 3) can actually run on my laptop.

We're doing multilingual testing and I can confirm what you've observed: Gemma 4 is surprisingly good at multilingual tasks, especially given its size. This is mostly true for the dense 31B model.