Maybe you need a minimal downscale version as well, I heard is very common technique in the video editing world.

Based on my experience, sampling rate can be tricky if what you are looking for lasted less than interval period.

Proxies. You transcode proxies from the original media, edit off those, then you use OM for the final render. NLE’s usually let you flip between them.