You'll need to settle on a decoder. I personally just use my video player for this, mpc-hc.

In mpc-hc, you can framestep using CTRL+LeftArrow (steps a frame backward) or CTRL+RightArrow (steps a frame forward). This lets you select the frame you want to capture. You do not need to be on a keyframe. These keybinds are configurable and may be different on the latest version.

Then in the File menu, there's an export image option. It directly exports the frame you're currently on, to disk. Make sure to use a lossless format for comparisons (e.g. PNG).

I'm aware this can be done in other players - like mpv - as well, although there I believe no keybinds are set up for this by default, and the default export format is JPEG.