Similar experience building a product solo with AI. The spec-first workflow you describe is very real. I converged on something similar after getting burned way too many times :(

One thing I'd add: even with good specs, the agent still cuts corners in ways that are hard to catch. It'll implement a feature but quietly add a fallback that returns mock data when the real path fails. Your app looks like it works. It doesn't. You find out in production.

Or it'll say "done" and what it did was add a placeholder component with a TODO. So now I have trust issues and I review everything, which kind of defeats the "walk away from the computer" part.

The "just one more prompt" loop is so true lol.