Sure, but the point is that code written against such a compiler is not C and is not portable. It is written in a dialect of C, and that comes with drawbacks.

Writing C (or any language) means adhering to the standard, because that's the definition of the language.