How hard would it be to fix this? Could we theoretically add or subtract enough material or make whole thing slightly more dense or less dense to compensate?

We should just redefine it to make the speed of light a nice even 300.000 km/s - we are so close already!

Don't forget packs of cigarettes as a more convenient unit for measuring volumes significantly smaller than Olympic-sized pools.

There is, of course, no more need to standardize on a specific brand or style of cigarette than on a specific depth of Olympic-sized pool.

You jest, but times around the Earth is the actual origin of the Meter. Kinda.

The history is quite interesting and well worth checking out.

I can't recommend a book on the subject, but I do heartily recommend "Longitude", which is about the challenges of inventing the first maritime chronometers for the purpose of accurately measuring longitude.

For smaller lengths and radiation bananas are also acceptable.

Don’t forget cheetahs for velocity and elephants for weight.

For the floor area or length/width it is, but if you want the height then that's in Empire State Buildings.