Codecs not harder than compilers? Sounds like an unsubstantiated claim!

Modern video codecs are harder than compilers. You have to have good ASIC development expertise to do them right, for example, which you don't need for compilers. It's totally feasible for a single company to develop a leading edge compiler whereas you don't see that in video codecs, historically they've been collaborations.

(I've worked on both codecs and compilers. You may be underestimating the difficulty of implementing sound optimizers).

Hardware vendors don't benefit from the patent pools. They usually get nothing from them, and are burdened by having to pass per-unit licensing costs on to their customers.

It's true that designing an ASIC-friendly codec needs special considerations, and benefits from close collaboration with hardware vendors, but it's not magic. The general constraints are well-known to codec designers (in open-source too). The commercial incentives for collaboration are already there — HW vendors will profit from selling the chipsets or licensing the HW design.

The patent situation is completely broken. The commercial codecs "invent" coding features of dubious utility, mostly unnecessary tweaks on old stuff, because everyone wants to have their patent in the pool. It ends up being a political game, because the engineering goal is to make the simplest most effective codec, but the financial incentive is to approve everyone's patented add-ons regardless of whether they're worth the complexity or not.

Meanwhile everything that isn't explicitly covered by a patent needs to be proven to be 20 years old, and this limits MPEG too. Otherwise nobody can prove that there won't be any submarine patent that could be used to set up a competing patent pool and extort MPEG's customers.

So our latest-and-greatest codecs are built on 20-year-old ideas, with or without some bells and whistles added. The ASICs often don't use the bells and whistles anyway, because the extra coding features may not even be suitable for ASICs, and usually have diminishing returns (like 3x slower encode for 1% better quality/filesize ratio).

With all due respect, to say that codecs are more difficult to get right than optimizing compilers is absurd.

The only reason I can think of why you would say this is that nowadays we have good compiler infrastructure that works with many hardware architectures and it has become easy to create or modify compilers. But that's only due to the fact that it was so insanely complicated that it had to be redone from scratch to become generalizible, which led to LLVM and the subsequent direct and indirect benefits everywhere. That's the work of thousands of the smartest people over 30 years.

There is no way that a single company could develop a state of the art compiler without using an existing one. Intel had a good independent compiler and gave up because open source had become superior.

For what it's worth, look at the state of FPGA compilers. They are so difficult that every single one of them that exists is utter shit. I wish it were different.