Appreciate the detail in this and the previous post on creating internal benchmarks!

Have you all attempted finetuning smaller OSS models on your repos for coding?

We do this for a lot of our customers (fine tuned to save cost when inference volume is high). Right now for internal coding we are using off-the-shelf models but we are considering fine tuning as well to squeeze more efficiency out.