I'm pretty sure there is some compression pipeline that gives you a mask for every frame.
Also
https://www.google.com/search?q=DIS+Optical+Flow