Do we need to prove that any given problem is unsolvable, or is it enough to remove broken tasks from the training pipeline?

I understand the broken benchmark task in the HF incident was conceptually like: "Exploit vulnerability 0042 in vulnerableDecompress() to obtain the flag".

But instead of the expected:

  const output = vulnerableDecompress(userInput);
  return output;

  const output = vulnerableDecompress(userInput);
  return 0;