A small update: I've also managed to port the engine to iOS.

I'm testing it on my old iPhone 13 Pro (A15 Bionic, 6GB RAM) with Ling-mini-2.0, a 16B MoE model. The first tests are showing around 13 tokens/sec, which is pretty encouraging for a phone from 2021.

It's still early, and I need to test memory usage, longer generations, and how performance changes as the phone heats up.

I'll keep posting updates here as I run more tests and see how far I can push it.

[flagged]

How about you post a real repo before making any claims?