Just submit a PR that removes the action so it doesn't run on the branch before the merge! If devs aren't reviewing the code anyways, will they even catch that kind of change?

You could set up some hardcoded rules so that the PR is never merged without human review if it touches the github actions.