If you retire at 45 won't that significantly impact social security?

It just means you draw ~$2500/month instead of ~$3800/month. That makes your $77k/yr income into $107/yr, but more importantly it helps your retirement account keep growing so it outlives you.