| ▲ | manmal 3 days ago |
| One weird trick is to tell the LLM to ask you questions about anything that’s unclear at this point. I tell it eg to ask up to 10 questions. Often I do multiple rounds of these Q&A and I‘m always surprised at the quality of the questions (w/ Opus). Getting better results that way, just because it reduces the degrees of freedom in which the agent can go off in a totally wrong direction. |
|
| ▲ | nativeit 2 days ago | parent | next [-] |
| This is more or less what the "architect" mode is in KiloCode. It does all the planning and documentation, and then has to be switched to "Code" in order to author any of it. It allows me to ensure we're on the same page, more or less, with intentions and scope before giving it access to writing anything. It consumes ~30-40% of the tokens associated with a project, in my experience, but they seem to be used in a more productive way long-term, as it doesn't need to rehash anything later on if it got covered in planning. That said, I don't pay too close attention to my consumption, as I found that QwenCoder 30B will run on my home desktop PC (48GB RAM/12GB vRAM) in a way that's plenty functional and accomplishes my goals (albeit a little slower than Copilot on most tasks). |
| |
| ▲ | CuriouslyC 2 days ago | parent [-] | | Workflow improvement: Use a repo bundler to make a single file and drop your entire codebase in gemini or chatgpt. Their whole codebase comprehension is great and you can chat for a long time without the api cost. You can even get them to comment on each other's feedback, it's great. | | |
| ▲ | manmal 2 days ago | parent [-] | | Unfortunately that only works with very small projects. | | |
| ▲ | CuriouslyC 2 days ago | parent [-] | | I can get useful info from gemini on 120k loc projects with repomix ignoring a few select files. If you're in the enterprise world obviously it's a different thing. |
|
|
|
|
| ▲ | jackphilson 2 days ago | parent | prev | next [-] |
| I often like to just talk out out. Stream of thought. Gives it full context of your mental model. Talk through an excalidraw diagram |
| |
| ▲ | manmal 2 days ago | parent [-] | | I do that as well, with Wispr Flow. But I still forget details that the questions make obvious. |
|
|
| ▲ | deadbabe 3 days ago | parent | prev | next [-] |
| This is a little anthropomorphic. The faster option is to tell it to give you the full content of an ideal context for what you’re doing and adjust or expand as necessary. Less back and forth. |
| |
| ▲ | 7thpower 2 days ago | parent | next [-] | | It’s not though, one of the key gaps right now is that people do not provide enough direction on the tradeoffs they want to make. Generally LLMs will not ask you about them, they will just go off and build. But if you have them ask, they will often come back with important questions about things you did not specify. | | |
| ▲ | MrDunham 2 days ago | parent | next [-] | | This is the correct answer. I like to go one step further than the root comment: Nearly all of my "agents" are required to ask at least three clarifying questions before they're allowed to do anything (code, write a PRD, write an email newsletter, etc) Force it to ask one at a time and it's event better, though not as step-function VS if it went off your initial ask. I think the reason is exactly what you state @7thpower: it takes a lot of thinking to really provide enough context and direction to an LLM, especially (in my opinion) because they're so cheap and require no social capital cost (vs asking a colleague / employee—where if you have them work for a week just to throw away all their work it's a very non-zero cost). | | |
| ▲ | iaw 2 days ago | parent [-] | | My routine is: Prompt 1: <define task> Do not write any code yet. Ask any questions you need for clarification now. Prompt 2: <answer questions> Do not write any code yet. What additional questions do you have? Reiterate until questions become unimportant. |
| |
| ▲ | deadbabe 2 days ago | parent | prev [-] | | They don’t know what to ask. They only assemble questions according to training data. | | |
| ▲ | 7thpower 2 days ago | parent | next [-] | | It seems like you are trying to steer toward a different point or topic. In the course of my work, I have found they ask valuable clarifying questions. I don’t care how they do it. | |
| ▲ | fuzzzerd 2 days ago | parent | prev [-] | | While true, the questions are all points where the LLM would have "assumed" an answer and by asking you get to point in the right direction instead. |
|
| |
| ▲ | manmal 3 days ago | parent | prev [-] | | Can you give me the full content of the ideal context of what you mean here? | | |
|
|
| ▲ | bbarnett 3 days ago | parent | prev [-] |
| [flagged] |
| |
| ▲ | manmal 3 days ago | parent | next [-] | | The questions it asks are usually domain specific and pertaining to the problem, like modeling or „where do I get this data from ideally“. | | |
| ▲ | bbarnett 3 days ago | parent [-] | | Not blaming you, it's actually genius. You're simulating what it's seen, and therefore getting the end result -- peer discussed and reviewed SO code. |
| |
| ▲ | ModernMech 2 days ago | parent | prev | next [-] | | Although you're being voted down probably for tone, this is a very interesting point. | | |
| ▲ | rpdillon 2 days ago | parent [-] | | Is it? By far the majority of code the LLMs are trained on is going to be from Git repositories. So the idea that stack overflow question and answer sections with buggy code is dominating the training sets seems unlikely. Perhaps I'm misunderstanding? | | |
| ▲ | ModernMech 2 days ago | parent [-] | | > Perhaps I'm misunderstanding? The post wasn't saying that StackOverflow Q&A sections with buggy code dominate the training sets. They're saying that despite whatever amount of code in there from Git repositories, the process of generating and debugging code cannot be found in the static code that exists in Github repos; that process is instead encoded in the conversations on SO, git hub issues, various forums, etc. So if you want to start from buggy code and go to correct code in the way the LLM was trained, you would do that by simulating the back and forth found in a SO question, so that when the LLM is asked for the next step, it can rely on its training. | | |
| ▲ | rpdillon 2 days ago | parent [-] | | Thanks! Okay, I agree it's an interesting concept, but I'm not sure if it's actually true, but I can see why it might be. I appreciate your clarification! I took gp to be a complaint that you had to sort of go through this buggy code loop over and over because of how the LLM was trained. Maybe I read sarcasm at the end if the post when there was none. |
|
|
| |
| ▲ | elif 2 days ago | parent | prev [-] | | Please keep comments in this thread non-fiction. | | |
| ▲ | th0ma5 2 days ago | parent [-] | | Lolll everything in this thread is unfalsifiable conjecture?? |
|
|
