Architectural restoration often solves this by using an inoffense, but still visibly detectable, "new material color". Some British castles have been rebuilt this way.