I am also in Sweden, and learned recently that a large part of my property seems to actually belong to the neighbour according to the online map! But there is a page in the relevant authority's website which clarifies that the online map can be 10s of meters off (in Swedish): https://www.lantmateriet.se/sv/kartor/vara-karttjanster/Visa...

There, it even explains some history and methodology for defining the borders. Mostly, they are defined by physical markers that hopefully the original surveryors left on the ground. I found a couple around my property (which is on hills so it's likely difficult to mark properly on a map from above) and it seems the borders are actually almost correct. As my fences have been up for over 20 years in the same location, I believe they also count now as de-facto borders now!

The official map of your property will not be exactly the same as the one on Lantmäteriet.se.

In more densely populated areas, there will be a local coordinate system, where each property is defined in terms of the neighbouring ones. This also applies to newly formed properties in old areas.

The property borders on digital maps are machine approximations of the mapping from the local coordinate system onto an absolute global coordinate system. This mapping can never be perfect, and it is often much less perfect than it could have been.

When the physical markers are missing or suspected of having moved from their original location (happens all the time for all sorts of reasons), Lantmäteriet will review the original documents of your and any number of neighbouring properties and deduce where the markers ought to be.

Regarding your fence, 20 years is very far from enough to establish "urminnes hävd". I suggest you wait another 100 years before you start assuming that they could act as facts on the ground in a property disputes! :-) And even then I wouldn't bet on it, unless the national archives are all destroyed...