I was thinking about this recently, the way to do is to define a radius, and then imagine rolling a circle of that radius around the outside of the coastline (or around the inside! Define that as well) and then take the length of the equivalent track that never leaves contact with the circle.

So you get a different length depending on the radius you choose, but at least you get an answer.

You could define the radius in a scale-invariant way (proportional to the perimeter of the convex hull of the land mass for example) so that scaling the land mass up/down would also scale our declared coastline length proportionally.

It's in no way a meaningful solution. If you're settling for a resolution, you don't need a ball-rolling analogy. We already know the length of a given coastline at given resolutions (ignoring the constant changing of the coastline itself). What's practically not feasible is getting every country on earth to agree on the right resolutions. And that's for good reasons, because the desired accuracy depends on many factors, some situational and harder to quantify than just size of the enclosed land mass.

Not a bad idea - one issue would be when the circle approaches a 'narrow' section that widens out again. If too big to fit into the gap, the circle method would simply not count any of this as land. I think it would be unreliable compared to moving along the coastline in fixed increments (IE one-mile increments or one-foot increments, depending on your goal)

https://en.wikipedia.org/wiki/Roulette_(curve)

A radius of plank length is the only true answer