It's not just adjustment, there's a decent amount of evidence it's just bad for your health in general, apart from the issues when coming back to earth. Stuff like your circulation also gets messed up.