I think the main cause of different views is just the fact that people have very different products and codebases they're working on. Therefore, the requirements and applicability of AI varies a lot, case-by-case. It would be useful if people always explained their exact use cases when making claims one way or another. I understand it's not feasible for many use cases to reveal exact details, but at least some transparency would be great.

I've found AI to be immensely useful in gamedev, mainly UX development, research, and curation. In those areas I let it go quite wild, with mechanical and visual tests for UX, and summaries for research and curation. I've also made skills, custom subagents, hooks, and scripts (all made with AI assistance) to streamline development.

Gamedev is a good example of where AI shines, since the risks of fucking things up are not so big. If I worked on, let's say banking, I'd be much more careful. On my day job, as a data scientist (moving towards product owner) in a large industrial company, I'm much more careful. UX can run on vibes, but anything that touches data processing or business logic needs to be looked at by a human, row by row. Depending on use cases, it may be fine to look at only the tests. The main problem is that there's so much stuff to review, and maybe 98% of it is fine. It's then very boring to try to spot the 2%. I don't have good solutions for that yet, and idk if anyone does.

I've found that these models are extremely good at reasoning since late last year, but they're basically 200 IQ Amelia Bedelias. So my working model earlier this year was to give it a brief instruction and watch how it failed, which was generally a lack of context that it couldn't have possibly known. Then I'd add that context to the team AI docs repo, try again from scratch, and iterate until it did what I wanted without me having to give a bunch of detail in the prompt. Over the course of 2-3 months, I needed to give new context and steering less and less.

I also encouraged my team to paste any inquiries or alerts we got in our slack channel verbatim into codex and follow that procedure until it could figure it out as a one-shot.

I still treat it as more of a design and coding partner day-to-day, so I'm typically not trying to one-shot, but it can e.g. be triggered by a pagerduty alert, reference our code, check grafana panels, query application state across different servers/clusters, and come up with bug fix PRs all autonomously, and then post its analysis, mitigation suggestions, and PR link to slack for the on-caller to review. This was all just some docs, scripts, and a little listener service to trigger a one-shot model prompt that we built.

As a design partner, it's made it easier to add more telemetry or create rapid prototypes to check my assumptions. I've encouraged my team to regularly ponder on what facts they wish they knew, no matter how difficult it would be to find out. Pretend you could just ask God anything you want to know. Then ask the AI to go add whatever's needed to find out. We should all move toward a perfect intuition for what our system is doing because all questions can now be easily answered if we can only think to ask them.

It’s a very soft skill that changes as the capabilities of AI change.

But one thing that doesn’t change is the need to specify an end goal correctly and precisely. I think the emphasis of knowledge/information-processing work is going to increasingly be placed on verification mechanisms. This is practically equivalent to precisely defining an end goal.

Spend time deeply thinking about what it means for a solution to be correct. What properties will a correct solution have? What of those properties are testable? Write those things down and tell the agent.

As models get better, agents will be able to target more and more difficult end goals. The strategy just becomes more useful. (It’s useful for people as well.)

For example, if you want an agent to write a photo editor, think about what end properties the editor should have. There are reference images for color space and rendering transformations. That’s a good start.

Sometimes the goal will be fuzzier. “I want a feature like CaptureOne where I provide a reference image and it makes my image look like that.” Well, time to think really hard about what that means. Iterate with AI on how to test for that precisely. Come up with some good metrics/heuristics. Maybe it means local contrast should match. Maybe the overall distribution of colors. Maybe something more complicated.

Then you have a target and you can let an implementation agent work against that. If it fails, it’s either because the agent is bad or your target was incorrect or incomplete. As models get better, the limiting factor becomes your ability to correctly define a problem.

The reality of the current situation is that we have dozens of families of different model; each of which is differentiated further into various revision numbers and power levels, and depending on which one you use and what the nature of the tasks is, you very much need a different usage style that can only be discerned after multiple (Sometimes very many) sessions of using that individual model for that category of task. It is not one size fits all, and trying to pretend that it is will lead to failure.

Latest models like Opus 5.0 work well in both scenarios. You can just give it a goal and let it make a plan, review the plan and then let it go ham.

Or you can also use it piece-meal and have it do parts of work that you want. It works in both ways and works really well.

I don't think there is any model right now that you can use with 0 oversight, but you can do pretty complex stuff without writing single line of code at this point.

I don't generally use AI to write code I feel I can write efficiently. In these cases, I'll just ask AI for a review. But when I need to write code in areas I'm less confident about, I'll let AI take the reins and the do a thorough review myself. These instances are not too common, though.

I also ask direct questions to AI for brainstorming and advanced programming language usage. In these cases, AI isn't touching the code.

What can be concluded is that some get immense value out of Ai and some are incredibly critical of that claim.

I do not know what the secret sauce is.

But I have a responsibility to the team I lead. I need to protect our velocity. Which means that my hiring practices now include specific checks that people are able to work with ai.

In the end, it might be a step thing. Some people are just inherently incapable of working with this technology...

The truth is that using AI well is a skill much like anything else. There's no one single way to use it much like there's no one single way to write a program that serves hackernews.

You can today develop a mobile app with 20 screens without ever reviewing any of the code generated.

If you test each feature and have it iterate on your feedback, you can build a decent product in this way.

Does it write too much code? Perhaps. Could I bring those 100k lines through code review in a team where some members nitpick? No.

That does not change the fact that it works, and that you do not need to read the code even today.

Unless progress slows down dramatically and soon, it seems rather likely that most of us will not be reading code in the near future. Their marketing is not wrong.

I find plenty of nuanced takes on HN. Honestly I think it's easier here than elsewhere; I just have to filter down to them. On other platforms, polarized takes are incentivized by the struggle for "engagement". (Youtubers often have fairly nuanced takes, but are incentivized to clickbait by misrepresenting themselves as believing something more extreme. Then they're incentivized to make a longer video, wasting your time and distributing the interesting bits across the middle.)

I think LinkedIn belongs in a natural category with Reddit and Twitter far more than HN does.

This right here is the truth. Too often people talk in abstract ways, not disclosing what they're actually working on. And then they tell their experiences as universal facts.

It's true that metacommenting about leaving a discussion in order to leave a supercilious last word is a flamebait trope. But please don't respond by breaking the site guidelines yourself. That only makes things worse.

https://news.ycombinator.com/newsguidelines.html

[flagged]