Because one property doesn't guarantee the other. A modular system may imply that it can be extended. An extensible system is not necessarily modular.

Wayland, the protocol, may be extensible, but the implementations of it are monolithic. E.g. I can't use the xdg-shell implementation from KWin on Mutter, and so on. I'm stuck with whatever my compositor and applications support. This is the opposite of modularity.

So all this protocol extensibility creates in practice is fragmentation. When a compositor proposes a new protocol, it's only implemented by itself. Implementations by other compositors can take years, and implementations by client applications decades. This is why it's taken 18 years to get close to anything we can refer to as "stable".