| ▲ | hopa 3 hours ago |
| I feel like AI is amazing for read only “search” tasks (query this, look for bugs, find vulnerabilities) but it’s not quite as good for “write” tasks like “add this feature”. Search tasks are easily verified, code changes are much harder to verify |
|
| ▲ | bheztio 3 hours ago | parent | next [-] |
| "Add this feature" is often too vague, unless it's yet another button to a list of buttons. If you want your feature in a particular way, you need to specify it. If you're not even sure what you want, it's granted you won't be satisfied by the result, so you need to spend time figuring that out first. In other words, they're definitely great for write tasks too, but it does require more effort from the operator to formulate the problem. |
| |
| ▲ | codebje 2 hours ago | parent | next [-] | | That problem is as old as commercial programming; it doesn't matter that now your actors are prompt engineer and LLM, the same story can be told with manager and programmer. | |
| ▲ | Forgeties79 3 hours ago | parent | prev [-] | | This feels like just another variation of “it’s not the tool it’s the user,” which is too frequently leveraged during any discussion about AI. It makes it impossible to honestly assess the quality of the tool. You can always just say “people are doing it wrong.” | | |
| ▲ | aleksiy123 2 hours ago | parent | next [-] | | Idk, I think it pretty clear to me that “Find the issues in this code” vs “write this feature” are different problems. The first one is easier because the feedback loop is llm to code/test and trivially scalable. The second one is llm to human which is going to be slower because it can’t read your mind. The tighter and more deterministic the loop the more llms excel. | |
| ▲ | 8note 2 hours ago | parent | prev | next [-] | | its certainly benchmarkable. make a bunch of "build the feature" vs "heres a detailed spec" prompts, and see which one does better. if you take out the AI boogieman, and instead go with "next token prediction" then "people are doing it wrong" is much more reasonable | |
| ▲ | int3 2 hours ago | parent | prev [-] | | I mean that's what you would expect from a new and rapidly-evolving tool. Being able to identify the kinds of work where people typically use it sub-optimally is a useful thing though! |
|
|
|
| ▲ | simonw 3 hours ago | parent | prev | next [-] |
| I'm not sure about that. The most common error for a search task is omission, and that's really hard to spot - how do you verify that some answer that should be there wasn't included? For things like "add this feature" you can verify by trying out the feature and seeing if it works. Of course, that won't tell you if there are edge-case bugs or security holes or other design flaws, but you can get to a positive confirmation of the happy case pretty quickly. |
| |
|
| ▲ | ipdashc 3 hours ago | parent | prev | next [-] |
| I've slowly been coming around to this viewpoint as well. It's interesting that there's so much focus on using AI to write code (which, don't get me wrong, does work well but makes you lose track of what you're building so quickly) and seemingly less on analyzing code, finding bugs, etc |
| |
| ▲ | mihaelm 3 hours ago | parent | next [-] | | The planning, which can also be done in concert with AI, before getting the AI to write the code helps with not losing track of what you're building. In my current workflow, I never tell AI to just code X, I always approach it in such a way to leave space for research and breaking it up into multiple units of work (tickets) if the scope of the feature or idea calls for it. | |
| ▲ | kibwen 2 hours ago | parent | prev [-] | | I wouldn't say it's "interesting", rather that it's a natural result of incentive structures. If the AI labs try to sell their product as being good at finding bugs, then their expected valuation is the entire market cap of the existing static analysis industry, which is not very high. If they instead sell their product as being good at writing software, then their expected valuation is the "market cap" of the code-writing industry, a.k.a. the total spent on software development labor, which is very high. It doesn't matter what's true, only what promises to make them the most money. |
|
|
| ▲ | hoppp 2 hours ago | parent | prev | next [-] |
| If the entire project is developed with AI it can add features effectively but for large old codebases it becomes an issue |
|
| ▲ | MBCook 3 hours ago | parent | prev | next [-] |
| But it DIDNT “write that feature”. They made it extremely clear on the delay announcement the code was written without AI. AI’s only part of this was helping review things and finding bugs. More bugs, bugs that would have been found later, whatever. Postgres did not have a slop issue and it’s wrong for people to insinuate it. |
| |
|
| ▲ | r_lee 3 hours ago | parent | prev | next [-] |
| I mean the transformer model came out of Google, and it's literally a query thing so yeah. any kind of pattern matching / transformation is going to be an ideal task for LLMs |
| |
| ▲ | jeremyjh 2 hours ago | parent [-] | | It is not in any way a “query thing” in the same manner as PageRank, it was based on prior work in deep learning and it is irrelevant that Deepmind was purchased by the same company that was behind Google Search. |
|
|
| ▲ | vatsachak 3 hours ago | parent | prev | next [-] |
| What? AI can't easily implement features. Just write a spec, ask if to expand it and then ask it to implement it |
|
| ▲ | DIE_DIE_DIE 3 hours ago | parent | prev [-] |
| [dead] |
