I think you can. RC toys do that all the time. Maybe the measurement is less precise, but you usually don't need super precise voltage, especially if you know your capacitance and measure current.

Current doesn't tell you if the battery is leaking charge, or otherwise failing to store it though. Vastly increased losses could be a sign of failure, for example. In any case it's important to be sure.