It was worse than you remember. You could have 640x256 in monochrome, or 320x256 with 4 colours, or 160x256 with 16 colours (which IIRC was actually 8 distinct colours plus 8 flashing versions of them).

The game Elite did something extremely evil and clever: it was actually able to switch between modes partway through each frame, so that it could display higher-resolution wireframe graphics in the upper part of the screen and lower-resolution more-colourful stuff for the radar/status display further down.

AlexandertheOk's documentary on Elite and the BBC Micro: https://www.youtube.com/watch?v=lC4YLMLar5I