Doesn't that happen only after UEFI starts the boot process, and only if Secure Boot is enabled?

I don't understand what "UEFI starts the boot process" means? The firmware is what initialises the hardware. If the code needed to initialise your GPU doesn't have a trusted signature then it won't be executed, and you won't have any working graphics, so you won't have a UI to let you disable secure boot. If secure boot isn't enabled in the first place then yes this isn't a problem.