As someone who saw this and was interested but also skeptical of this being low effort are there other open projects for running small models locally on android / iOS?

I've found https://github.com/alichherawalla/off-grid-mobile-ai but haven't tried anything in this space yet.

Google's AI Edge Gallery worked really well for me. It's still in early access.