The mistake there is to point it at code to figure out performance optimizations.

The place to find them would be performance profiles, query plans, telemetry. The guidance for perf still applies, measure before and after change.

The issue is that the code often does not contain the information to do a perf optimization. Eg. you can't tell your cache size, the volumes of data in your DB or the latency of your network through just the text. Should you provide this context, you can get better results.

▲

flohofwoe 11 hours ago | parent | next [-]

> The place to find them would be performance profiles, query plans, telemetry.

Tbh, once this information is available (which is the tricky part), in 99% of cases there's really no AI needed to analyze the data, since the 'low hanging fruits' will usually stand out anyway. And once you get into the area where optimization hotspots are no longer obvious, you're already deep in the diminishing returns area and optimizations for one use case or hardware configuration may degrade performance on others. That's my experience anyway.

▲

dewey 11 hours ago | parent | next [-]

> there's really no AI needed once this information is available

Most people are not arguing that problems are too tricky for a human to solve once presented with it, but AI can look at 1000 things at the same time across a whole code base and then just come back with the results a human can review.

	▲	flohofwoe 10 hours ago \| parent [-]
		True, but at least IME, once you get beyond fixing the 'obvious bugs' (how many there are depends on the initial quality of the code base - but I agree that AI is really useful to find those), you'll get into a murky grey area of "maybe false positives" which require a lot of time to validate. As a result you sink a lot of time creating and tweaking code-base-specific rules to try filtering out false positives, and IMHO this is exactly the tipping point where the whole thing becomes pointless because it becomes a bureaucratic monster. E.g. a good code analysis tool needs to work predictably at button press on any code base. Still better than nothing of course, e.g. I actually think bug scanning / code analsysis is indeed the one area where LLMs are actually useful, but it suffers from the same 'diminishing returns' problem as traditional approaches, if not worse (e.g. still no silver bullet, but a mostly useful additional tool in the toolbox).

▲

duskdozer 5 hours ago | parent | prev [-]

>in 99% of cases there's really no AI needed to analyze the data

As I've seen it, there's a large chunk of those getting the most out of AI doing so because they weren't aware of or couldn't be bothered by already existing, more reliable alternatives.

▲

Orphis 9 hours ago | parent | prev | next [-]

It really depends on the context. On a pure computational code, it can work really well.

I recently optimized some code asking Claude to "make it faster" 2 different programs. For each, it wrote a benchmark, gathered initial data. Emitted some hypothesis and measured data around them with profilers, then did some changes (behind feature flags), checked the output was identical in either side and through profiling that the right code path was taken, and then benchmarked both.

And it did that for multiple successive changes in both codebases. Some changes were purely algorithmic (better complexity), some were related to tradeoffs between memory and computation (caching intermediate results better), some were about creating less intermediate objects to relieve the memory pressure, some where about a better memory layout to improve data locality.

The annoying part is that some of the code that was optimized was generated by Claude in the first place, so it's a bit frustrating that it didn't do those in the first place. But I guess, my iterative approach to making those tools didn't work on big enough data sets at first where it would have been an issue.

▲

insanitybit 7 hours ago | parent | prev [-]

You can very easily spot performance issues through code. Allocations are often visible, slower hash maps are often visible, loops are visible, etc.

▲

inigyou 5 hours ago | parent [-]

No you can't.

▲

insanitybit 3 hours ago | parent | next [-]

Are you serious? Of course you can. Some hash algorithms are slower than others, you can read that and know that. Allocating in a hot loop is probably slower than allocating outside of it. Duh? I can't believe this even has to be justified, of course you can look at code and get an idea of its performance properties lol

Obviously you want empirical evidence to justify changes, but like... duh, you can read code and understand how it executes.

	▲	inigyou 11 minutes ago \| parent [-]
		Sometimes when you look at code you know is slow, you can immediately see why it's slow. Obviously you didn't see it when you wrote it, which disproves your hypothesis.

▲

jackling 3 hours ago | parent | prev [-]

Not really a rebuttal. There's a lot of low hanging fruit in code bases were you can read code and see what's making things slow. Like the parent comment stated, you can visually tell if there's a lot of allocations in a certain part, and easily see if those allocs are present in a loop and often reason simply how many times that loop can be called.

I can see in the code when data layouts aren't optimal, and fix that.

There's a lot of optimizations that need more of a deep dive, but you can get a lot of gains by just reading and reasoning about your code/data.

EDIT: To add, there are cases were you specifically can't read code and understand performance issue, but you should first ask, is that because you just don't understand the APIs/Libs/tools you're using, or is it fundamentally difficult. For example, often at work I see people complain about their torch code being slow and needing to bust out a profiler, but often those people just don't understand how tensor operations work internally so of course they can't reason about the code and see the way they're using the lib is suboptimal.