I've recently been using AI a lot for performance optimisation during a particularly busy period at work. I would say it was almost completely useless at the high-level direction - it would point out suspicious parts of SQL queries for example but on back to back testing these almost never resulted in any performance change.

In fact, if it wasn't for the fact that it made making the actual changes I identified much easier (move these joins into a CTE etc) it would have been a detriment. Not only did I get sidetracked by a bunch of useless suggestions but I also had to put up with others dumping their raw AI output at me as if it was somehow a meaningful contribution.

The useless suggestions are exhausting. One of my co-workers uses Claude for all asynchronous communication, including Slack messages, Jira comments, code reviews, and emails.

Every single message from him, literally every time he communicates, it's a massive wall of text, overflowing with scope-creep suggestions like nothing I've ever seen before. It's impossible to ask him a simple question and get a simple answer.

And this seems to be a broader trend with AI in general. It's getting more and more difficult to retain some semblance of brevity with AI tools. I'm constantly asking them to "be brief," and "only answer the question I asked," and "don't provide more information than I requested," but they keep churning out more than I want in their responses.

It looks like a subtle attempt to use up more tokens.

The thing that makes it work really well is to make sure it has all the tooling to verify its hypotheses. If you allow it to run the full lifecycle in loops you will be surprised how well it works.

The mistake there is to point it at code to figure out performance optimizations.

The place to find them would be performance profiles, query plans, telemetry. The guidance for perf still applies, measure before and after change.

The issue is that the code often does not contain the information to do a perf optimization. Eg. you can't tell your cache size, the volumes of data in your DB or the latency of your network through just the text. Should you provide this context, you can get better results.

I've consistently found that I see performance issues that the AI misses. It often says "that's not going to be what improves performance, it's noise" and then I get it to do it and it's like a global 30% throughput win lol. I think a lot of performance guidance it'll be trained on is shit - I see devs consistently misunderstand performance too and downplay the impact of anything other than "IO".

Yup, I do this regularly now. Whenever I think that something takes longer than it should I just tell it to set up a testing harness to measure each part of the process.

Once I have the baseline I'll just set a goal to improve performance by 10x. Works better than it has any right to. Have to be careful to exclude changes that massively increase complexity for tiny gains after its done, but the big improvements are often sensible choices that I'd also make as an engineer (i.e. more efficient data representation, caching and memoization where it matters, parallel processing, etc.)

Which model are you using? There is monumental difference between models. Even between "frontier models". When people tell these stories, it would be great to also add which model you were using.

As an example Opus 5.0 is in completely different class compared to Cursor Grok 4.5 even if the benchmarks don't show such massive difference. Not even talking about regular stuff like Sonnet or Composer or stuff like that.

You're using it wrong. No really you are.

Give Fabe 5 access to a test database with some data, or even restricted access to your live DB and tell it to optimize then.

I've had stunning success optimizing for performance this way. In a single day, I made the core part of our app 2-3x faster.

Which model/agent/harness tool did you use? I've found what you describe was my exact experience some ~6-8 months ago, but since about a month or so the game has completely changed. Using 5.6 Sol with highest reasoning setting in Codex or Fable in Code, the models come up with a list of possible improvements from static analysis (ranked by complexity/benefit), write and run their own custom profilers and deliver significant performance improvements with barely any input needed from my side. So this is no longer a model issue, it's a user toolchain issue.

I've had a completely different experience. I usually know the problematic part and then ask the AI to optimize that exact part with benchmarks, telling it to try various different approaches, and it very often gains massive performance increases on various tasks, with me of course steering it and giving it pointers every now and then.

I was able to optimize a physical water simulation that was almost unrunnable on browsers to a buttery smooth 60fps version.

A self validating loop goes a long way with getting AI to optimize stuff. We had some builds that used to take a long time (15-20~ minutes), that i just asked Claude to reduce the build time. It brought it down to 2 minutes and under 10 seconds for incremental builds.

Turns out folks had been copy pasting some setup code which took 2 minutes to run, and only needed to be run once per environment. Claude refactored the code to get it to run once, and since we had tests which validated the build artifacts, it was able to roll back if its change had broke the build.

You have to learn to use your tools, not try whatever intuitively made sense to you at first (expecting the tool to do all the work) and then whining on Hacker News when it doesn’t work out for you.

My favorite thing lately is that it's very good at resolving dependency security vulnerabilities. GitHub already does a good job at detecting and notifying which packages are vulnerable and at what versions they're fixed, and without AI, it's just a super tedious process to figure out what things all depend on that package, find a version that'll work for all of them (potentially including upgrading the things that depend on it), wire it up, test things, and make any minor changes necessary in our code.

It's not "hard" or novel--it's just tedious grunt work. I'm happy to have AI solving that one.

I did a lot of SQL optimisation with claude code over the last year, what made it work for me was making sure I verify everything with explain analyze. The whole loop was not fully “agentic” because of this but it was still faster and better than when done entirely manually. Claude also suggested plenty of small optimisations I would otherwise not do.

One of my tasks is to take various AI models the data science team has produced/fine-tuned and make them runnable with optimizations on GPUs (TensorRT, vLLM, Triton Inference Server, etc.), involving conversion, deploy, smoke-test, packaging, documentation, all within a uniform framework. The documentation includes model-provenance, simple deployment instructions (usually a Docker run), simple deployment test. The DevOps team takes this and tailors it to Kubernetes / helm-charts or whatever the target environment needs.

I had worked on some earlier deployment environments for a few models where we focused on one version of Triton and one runtime technology. Even upgrading to a newer Triton version was brittle, involving a lot of command-line changes at various phases. This was written mostly pre-Claude-getting-real-good. I decided that probably Claude had matured and was way better at understanding the particulars of AI-model-GPU-deployment-and-technologies. I worked with Claude to make the framework a much more lightweight wrapper, ignorant for the most part of a lot of the deployment internals.

After this refactoring and doing the first couple of models, it's quite amazing at how well Claude can figure things out. For any new model we now set up its "specialization" directory and its documentation subdirectory, point Claude at the proper AI-model files, point Claude to the sample non-optimized inference code and test data, and point Claude to a similar conversion we've done in the past. There a multi-layer class hierarchy dealing with various tiers. I ask Claude to explore the existing conversion/packaging, the model, any documentation that comes with it (a lot of times there are unexpected twists), the sample code, and the desired multiple use cases the model is meant to address, and the test data. Claude has been trained, I'm sure, on a lot of AI model conversion, so it's able to synthesize the full multi-stage conversion/deployment pipeline, come up with appropriate test cases for all the use cases involved. There are usually between 3 and 10 refinements after initial synthesis, fixing outright errors, refinemnts that the data-science team requests after playing with test deployments, etc. The options and pitfalls are vast, and without Claude each preparation likely would 10x or more longer. I just put most details in Claude's hands, and make sure the general framework is good enough to provide external uniformity. When all is working, it takes a couple of hours to make sure the documentation is good.

All this to say that at least for this domain, Claude / AI has been a game-changer and has sped up the process amazingly.

Echoing the other folks, I have a different experience.

I profile sql performance and LLMs find more opportunities than I could. All it takes is real data, a sql repl and an agent. Just ask the agent to use the repl to EXPLAIN and profile the sql. It works amazingly most of the time.

It is very cheap to:

1. Ask Claude Code or coding agent to research the internet, documentation, and Github for examples and learning working with similar problems.

2. Make the ten best examples of solving the problem based on the research.

3. Run each against the database enough times (100, 1000, or 10,000,000 depending) to empirically prove which is optimized. You can set other criteria based on your knowledge.

4. ???

5. Profit

The point is, it is cheap to research combing through 1,000s of examples and learnings and test the best and most relevant.

Ask it to create a small sample to reproduce the issue in isolation. If it can't reproduce the performance changes then it drops it.

That might work better. I haven't tried it with SQL but it worked with a few complex UI issues I had. It identified the actual issue after a few false starts.

For sql, i found that it can recommend indexes that i didnt have in place. So for that part of it, its great. For optimizing code, it does an ok job, i guess.

I'm surprised to hear this. I've found AI incredibly useful for optimizing queries. Give it the EXPLAIN ANALYZE output of your query and it will have no problem optimizing it

>using AI

What model? There is no "AI", there are only specific models.

Is this with the good Anthropic models? With extended thinking? And a way for it to validate the changes? My experience is the complete opposite.

This is the research taste part - often they can be good but human experts are really good at this (also you know your codebase, without the prompt these models have no other background).

Then being able to suggest several things to try and have them go off and build, measure and tweak is hugely useful in my experience.

Also things like making custom visualisations for comparing changes.

I am not denying what you are saying but I am interested in what models you were using for these tasks?

AI can uncover certain issues and resolve certain problems but it also has its blind spots.

I've been using Claude to create a software architecture diagram. It came up with a lot of useful functionality that we had neglected to show but upon further examination a lot of steps didn't make any sense. It added a box to do validation on some data then it put another box downstream to do validation again on the exact same piece of data (the second box had a different name but upon questioning, Claude conceded that it was the exact same validation step). When doing inference on user input, it put a box to extract specific insights from the prompt and then it put another box to do inference again on the inferred data to categorize it... And this would add latency and unnecessarily strip out useful context when doing the second inference... It really only needed a single inference step...

I re-uploaded the diagram to Claude and confronted it and it conceded to all of my points and it even noticed a theme between them and suggested that the diagram exhibited "a pattern of redundant steps." But it was incapable of suggesting viable solutions besides name changes. All of its solutions made the design more complicated. It was incapable of simplifying the design. I've noticed this with AI code; it can make things more complicated, add more features but struggles to simplify things.

Also, AI is horrible at rating things... It's just too superficial. For example, if you write a huge amount of intentionally over-engineered, tightly coupled code with poor separation of concerns but you do proper linting, define all the types, add a lot of comments, it will give it a higher score than a shorter, more readable and maintainable snippet which exhibits loose coupling and high cohesion because it lacks the superficial aspects.

It's horrible because now business people who understand nothing about coding may ask AI about code quality and it will consistently rank low quality over-engineered projects that are full of bugs, unmaintable and less secure, with a higher score.

Frustratingly, if you point out problems in its judgement, it will concede to your points without reservation, even building on top of your argument... but it will never actually tell you this stuff up-front, unprompted if you don't already know it! It never seems to reveal new insights, at best it can only expand on existing insight which you've already had.

It tells you what you want to know but not what you need to know.

People in the comments sound like they're literally asking claude: "hey claude, can you find areas for performance improvements." Nah, it doesn't work like that. You have to be more like: "can you profile [...] and identify hot spots." then looks at results that are suspicious. if you don't ground it to the real world reality of your project then don't be surprised when it makes up its own.

that's the first mistake i made as a new claude user. i only knew what prompts people were sharing on forums. but since they had been written by vibe coders it was all complete non-sense. also think about this: software engineering is filled with highly specialised tooling that can easily improve software quality. the thing is, most developers dont know how to use them, and if they do -- using them is so slow that its almost not worth it.

any tool you can name claude can use. obscure academic proof verifiers, memory corruption scanners, debuggers, profilers, coverage tools, backdoor supply chain pattern scanners... it can run these and have results in seconds. really is like living in the future.

Then you're not grounding it to reality properly. LLMs get bad pretty quickly when you just tell them to pull the answer out of thin air. Pin them to reality with actual performance tests to run to test theories and the results will be much better.

Performance optimization involves simulating real world loads and measurable results. Give them the opportunity to actually have a closed loop if you want more than trivial improvements.

What model is “AI”?

CTEs are often slower than using temp tables with indexes.

Were you using Fable 5?

Did it ever tell you not to optimize what you should redesign?

Did you give it access to a planner/analyzer?

I've noticed the same thing.

The trend I've noticed is that AI struggles to think outside the box when making optimisations, which exactly what's needed when you've made all of the practical DB and logic optimisations to the existing code.

Often you need to take a step back and question how the system is working and if there would be better ways to design it so the bottlenecks you're hitting wouldn't exist in the first place. Caching things, adding indexes, tweaking logic – these can help, but you'll quickly hit diminishing returns once you've done all of the obvious stuff.

I've seen people here say how AI is great at optimising code though, but I'm not sure if that's because they're giving it optimisation problems with a lot of low hanging fruit or if they're successfully getting AI to rework their systems to remove bottlenecks. This one area I find AI to still be particularly bad at.

Which model/mode?

As others have said, these were uninformed guesses which LLMs seem still to be willing to hand out deliberately. If you provide more information (your schemas, the distribution of values in your database tables, the EXPLAIN ANALYZE outputs, DB configuration etc.) it will do a much better job.

I had good results when I started AI with building the testing for the actual problem, then it can just run itself over and over without wasting much time on verifying its output.

And at worst you still saved a bunch of time on writing the tooling to test the issue

What's the complain? That AI can't do your job yet?

Yep, as a performance specialist I've had the same outcomes - it'll find "smoking guns" galore which will sound plausible and be completely wrong.

But that's ok - I still get a lot of value of the tool elsewhere (e.g. writing data analysis scripts, exploring code, breaking down bug artifacts...), but it doesn't stop the tiring BS from other people going "why not just AI it?" (especially from managers).

I usually let them try to "just AI it" and most of them learn quickly that it's not that simple. (The others stay deranged and are making my job miserable.)

> I've recently been using AI

That's uninteresting as long as you don't specify the model you used. For example, Mythos was far better at finding security bugs than previous models.

Why are you sidetracked instead of piping that to another AI?

I would never undertake a big performance optimization without first measuring that area's total impact. The LLM would then be great to work out the refactoring and you can give it your bench suite (which it probably wrote) to work against.

I guess just another area where the LLM is useful only as long as you remain in charge using your own programming experience as a guide.

[dead]

[dead]

[flagged]