It's at https://github.com/peterc/pure_jpeg .. and a lot of the recent speedups actually came from contributions by Ufuk/paracycle who, I'm guessing from your bio, you possibly work with? :-)

But yeah, I agree with your point about native extensions. Ruby has gotten so much faster in every form in the past couple of years that I think we could bring a lot more "in house". I think there have been some efforts with this regarding Psych in core as well?