| The biggest thing AI enables is cheap code. That means you could choose to try three (or more) genuine implementations and explore their tradeoffs, instead of making three proposals in a document with one recommended (and the other two usually only provided for contrast). I do think the design is important to keep around - in particular, the constraints, the communication points, schema, tacit things that might not be clear in code. I am not certain that the design should precede the implementation for features below a certain size though. Larger efforts need milestones and collaboration and will have multiple people doing implementation, so there's more need to agree schemas, APIs etc up front there. |
| |
| ▲ | mtlynch an hour ago | parent [-] | | > The biggest thing AI enables is cheap code. Agree, but in my experience that doesn't change much about the design doc. I think it's helpful to the author to be able to say to an AI agent, "Hey, put together this quick prototype," and that informs the design doc. But if the goal is to review the design decisions with the team, I don't see how you get around the design doc. I don't want a teammate to send me 10 KLOC of AI-generated code and ask me to review the design. Even if you told AI to try 10 different ideas and pick the best, I don't trust AI to make the same decisions as my human teammates. | | |
| ▲ | barrkel an hour ago | parent | next [-] | | I'm not suggesting using AI generated code as a proposed design. I would try and get an understanding of design space by giving a good agent a high level goal and seeing what it does, then getting a summary of the approach. When you do this several times, especially if you give it a steer on some non-functional requirement, you can compare and contrast different approaches. The idea isn't to prototype so much as to gather information by doing. Prototype, to my mind, suggests other things; shortcuts, stubs, incompleteness. I would actually ask agents to do the whole thing, and find out the full scope. It can be particularly useful revealing side effects. Pair it with code auditors wearing different hats, of course. | |
| ▲ | luc_ an hour ago | parent | prev [-] | | Why do you need to review design decisions with a team anymore? I get the impression that Fable, when well directed, is better than maybe 80% of SWEs. Maybe more. [edit: Yes, I'm maybe baiting other users, but I want to know your honest opinions on this.] | | |
| ▲ | mtlynch 32 minutes ago | parent | next [-] | | I find that LLMs are still worse than humans at limiting complexity, which is one of the most important outcomes of a design review. If I tell a senior SWE that I'm creating a Discourse-like discussion forum, and I want users to have three options for selecting an avatar: (1) import from Gravatar, (2) upload a JPG or SVG or PNG or GIF, or (3) let the user draw their avatar on a canvas, the LLM will happily go and design that and write a 5 KLOC implementation, whereas a good SWE would push back and say, "That's like 10x the complexity of just allowing JPGs. How about we simplify it to say that in v1, the only option is to upload a JPG." I've tried working with Fable/Sol and saying, "Look for features that we can simplify to reduce complexity," and they don't understand. They'll guess at features we can cut entirely, but they fail to see how to capture the essence of the feature without the complexity. I've noticed this a lot with Fable recently. Like I'll say, "Show an error message in the web UI if X fails," and Fable comes back with this like 800 LOC error message generator that has switch-cases and combines inputs from three different sources when all I wanted was something like, "Update failed: database is locked." | |
| ▲ | lirolero 17 minutes ago | parent | prev [-] | | [dead] |
|
|
|
| I think we definitely need to have alignment, and documentation to support it. I think this can be at the PRD level, mostly. For many systems, I'd argue technical documentation to understand how the internals are working can simply be handed off to the robots. Or generated on the fly. And if a requirement on the product level is not met, that can be changed under the hood. As the other commenter said, "code is cheap" now. |
