To me that raises the opposite question, why are there so few fields? (Compared to what I'd imagine, infinite)

[Edit: I suppose I'm imagining waves or frequencies of waves, rather than fields, hence why in my imagination there would be an infinite variety]

Not all fields interact with all other fields. You can think of them as a loosely coupled graph…

There might be any number of graph components with no connectivity to our fields at all, and we’d never know. Assuming, of course, that we’re including gravity in this logic.

There’s also might be any number of arbitrarily complex components which are only connected through gravity. That’s a decent candidate for what the dark sector actually is.