I remember early cell phones (not smart phones, mind you) having weeks of standby time, or something like 20 hours of talk time. These had replaceable batteries. I don't recall people carrying spare batteries being a thing..?

Some of my early phones had spare batteries. They most certainly did NOT have weeks of standby time or 20 hours of talk time. We are talking late 90's.

Later, as phones and batteries got better, the spare batteries became unnecessary. They still degraded fast enough that there was a market for replacement batteries and they could indeed easily be replaced. We are talking things like the Nokia 3310.

Even later, the need for user replaceable batteries pretty much disappeared.

These days, it is entirely gone.

Standby times were indeed great in those days because those phones weren’t doing very much when they were idle. (Weeks may be an exaggeration, though!)

You might also be misremembering talk times, unless you had a phone with an exceptionally large battery.

A typical device like the Nokia 3210 had 3-4 hours talk time, which is far less than modern smartphones.

You can in fact still buy those kinds of phones and they still have removable batteries.