If you've trained your own models you would be aware of quantization aware training.