I've tested different ways to use agents to model stuff from images with accuracy. Maybe this is something that can help or give you ideas: https://janne-alatalo.fi/en/articles/silhouette-modeling/

If you have some physical reference, for example a coffee table, you can create an accurate 3D model of that quickly. The agent creates model that can be converted to STEP format and opened with any CAD software.

This might help you? You get a good model that you can start editing towards your own design.

Here is agent skill that works with OpenAI models and Codex https://github.com/janne-alatalo/image-to-cad-skill

Basically you take few pictures. Three is usually enough of furnitures. The agent uses the pictures to find the exact camera poses and creates a 3D model that minimizes the difference between the object's silhouette in images and in the 3D-model when projected with the virtual cameras to the 2D planes. The agents create the model with python scripts. First using numpy for faster optimization and finally CadQuery for easy exports to CAD formats.