You'd still need one real measurement at least: this might get proportions right if background can be clearly separated, but the absolute size of an object can be worlds apart.

That's true. And there's lens correction and all that, but it would be nice to accelerate the CAD modeling.