Are you using Google Gemini directly? I've found the Vertex API seems to be significantly less strict.