Yeah, this is good to point out. The primary constraint I was working around was "this needs to run at a smooth 60FPS on mobile devices" which limits the type and amount of work one can do on each frame.

I'd probably arrive at a very different solution if coming at this from a "you've got infinite compute resources, maximize quality" angle.