We had these in the 80s:

https://en.wikipedia.org/wiki/Casio_Film_Card_calculators

I don't think it's too wild to expect Linux to run in this form factor, 40 years later.

Especially if you’re not looking for a gui. Just one lean SOC, stuff for power, and 80x100 character display.