Usually you do "pointer-fixup" where you convert them to relative-offsets on write and then back to absolute-offsets on read.