Then you need to re-pad everything (clean looking git diff be damned). It's just the reality of dealing with bounding boxes. Maybe we don't notice it in HTML and such since the browser redraws them for us for free.

A reasonable format would not insist you lay out tables visually any more than it would insist you center your headers if you'd like them horizontally centered when rendered. For instance, Asciidoctor has syntax for table cells that requires no whitespace and lets you put any content at all in a cell.