Is banking not deemed a right in Germany ? Aren't there "banks of last resort" ? Or does that right somehow not extend to non-EU refugees ?

Yes, sort of. You can always force it, but it takes time, resources and knowledge that recent immigrants lack. Usually they get funneled towards more immigrant-friendly banks.