You can charge for the source code too, just not separately.

But you can definitely have an opensource project that is not available free of charge from any official source. Your clients can redistribute for free or not, but you are not required to just let the whole world have your code directly from you.