Yes, although you probably want to calibrate your model if you want the probabilities to actually be meaningful.