Les Mis is a great treatment of exactly this, even if fictional. It takes more than justice to reform the soul. It takes making room by society to forgive the repentant. We call this mercy, and it is the higher ideal.

If it's too much for society to forgive someone who has done their time, the very least society could do is to stop actively fighting their rehabilitation.

Whenever a read a story about someone who's been to prison and then ends up a solid, productive member of society, I can't help but think: "This person must have extraordinary grit and determination!" Because when a criminal gets out of prison, the entire system and the entire society is set up to try to oppose his rehabilitation and get him back into prison. Overcoming this active hostility must take a remarkable person.