AI coding agents are notoriously bad at anything that involves spatial awareness.

Over the weekend I took pictures of the four walls of my office and asked Claude Desktop to examine them and give me a plan for tackling it. It absolutely “understood” my room, identifying the different (messy) workspaces and various piles of stuff on the ground. It generated a checklist with targeted advice and said that I should be motivated to clean up because the “welcome back daddy” sign up on the wall indicates that my kids love me and want a nice space to share with me.

I vibe-code TUI and GUI by making statements like “make the panel on the right side two pixels thinner”.

Related to this thread, I explored agentic looping for 3d models (with a swift library, could be done with this Rust one by following the workflow: https://github.com/ConAcademy/WeaselToonCadova

I've found they are actually quite good at semantic geometry even if they struggle with visual or pixel-based reasoning. Since this is parametric the agent just needs to understand the API and constraints rather than visualize the final output. It seems like a code-first interface is exactly what you want for this.