> inference is CPU bound

This was a typo. It should have been “inference is memory bandwidth bound”.