▲ | derf_ 4 days ago | |
> I think HN has a lot of "if I can't see how it can be done then it can't be done" people. No, HN has, "This has been thought of a thousand times before and it's not actually that good of an idea," people. The motion search in a video encoder is highly optimized. Take your side-scroller as an example. If several of your neighboring blocks have the same MV, that is the first candidate your search is going to check, and if the match is good, you will not check any others. The check itself has specialized CPU instructions to accelerate it. If the bulk of the screen really has the same motion, the entire search will take a tiny fraction of the encoding time, even in a low-latency, real-time scenario. Even if you reduce that to zero, you will barely notice. On the other end of the spectrum, consider a modern 3D engine. There will be many things not describable by block-based motion of the underlying geometry: shadows, occlusions, reflections, clouds or transparency, shader effects, water or atmospheric effects, etc. Even if you could track the "real" motion through all of that, the best MV to use for compression does not need to match the real motion (which might be very expensive to code, while something "close enough" could be much cheaper, as just one possible reason), it might come from any number of frames (not necessarily the most recent), etc., so you still need to do a search, and it's not obvious the real motion is much better as a starting point than the heuristics an encoder already uses, where they even differ. All of that said, some encoder APIs do allow providing motion hints [0], you will find research papers and theses on the topic, and of course, patents. That the technique is not more widespread is not because no one ever tried to make it work. [0] https://docs.nvidia.com/video-technologies/video-codec-sdk/1... as the first random result of a simple search. | ||
▲ | WantonQuantum 4 days ago | parent [-] | |
> If several of your neighboring blocks have the same MV I think we’re mostly agreeing here. Finding the MVs in any block takes time. Time that can be saved by hints about the direction of motion. Sure, once some motion vectors are found then other blocks benefit by initially assuming similar vectors. To speed things up why not give the hints right away if they’re known a priori? |