LLMs are also good for writing quick experiments and benchmarks to satisfy someone's curiosity. For example, once I was wondering, how much time does it take to migrate a cache line between cores when several processes access the same variable - and after I wrote a detailed benchmark algorithm, LLM generated the code instantly. Note that I described the algorithm completely and what it did is just translated it into the code. Obviously I could write the code myself, but I might need to lookup a function (how does one measure elapsed time?), I might make mistakes in C, etc. Another time a made a benchmark to compare linear vs tree search for finding a value in a small array.

It's very useful when you get the answer in several minutes rather than half a hour.

▲ codedokode 5 days ago | parent [-]

Also I wanted to add that LLMs (at least free ones) are pretty dumb sometimes and do not notice obvious thing. For example, when writing tests they generate lot of duplicated code and do not move it into a helper function, or do not combine tests using parametrization. I have to do it manually every time.

Maybe it is because they generate the code in one pass and cannot return back and fix the issues. LLM makers, you should allow LLMs to review and edit the generated code.

▲ kelnos 5 days ago | parent | next [-]

I see that often enough too, but if I then ask it to review what it's done and look for opportunities to factor out duplicated code, it does a decent job.

▲ 4 days ago | parent | prev | next [-]

[deleted]

▲ nikki93 5 days ago | parent | prev | next [-]

https://github.com/terryyin/lizard has been useful to track when functions get too convoluted or long, or when there's too much duplication -- in code generated by agents. Still have to see how well it works long term but it's caught things here and there, I have it in the build steps in my scripts so the agent sees its output.

▲ jlei523 5 days ago | parent | prev | next [-]

  Also I wanted to add that LLMs (at least free ones) are pretty dumb sometimes and do not notice obvious thing. For example, when writing tests they generate lot of duplicated code and do not move it into a helper function, or do not combine tests using parametrization. I have to do it manually every time.

Do you prompt it to reduce duplicated code?

▲

codedokode 5 days ago | parent [-]

I can prompt anything but I would prefer it not to make obvious mistakes from the start.

	▲	jlei523 5 days ago \| parent [-]
		"Use DRY coding". 3 words can solve this problem. Maybe put it in the parent prompt.

▲ scotty79 5 days ago | parent | prev [-]

> I have to do it manually every time.

You can tell it to move it and they'll move it and use this shared code from now on.

▲ codedokode 5 days ago | parent [-]

Sometimes it seems like explaining what I want could take more time than actually editing the code.

For example, imagine if you test a vector-like collection. In every test case dumb LLM creates vector manually and makes inserts/deletes. It could be replaced by adding a helper function that accepts a sequence of operations and returns the processed vector. Furthermore, once you have that function, you can merge multiple tests with parametrization, by having a test function accept a sequence of operation and expected result:

    parametrize('actions, result', (
        # Test that remove removes items from vector
        ([Ins(1, 2, 3, 4), Remove(4)], [1, 2, 3]),
        ...
    )

But it takes time to write this explanation, and dumb LLM might not merge all tests from the first time.

	▲	scotty79 5 days ago \| parent [-]
		Try something like: "Don't create vector manually inline in every test case, make a helper function for that." and see what agent does. It might do something smart. It might do something a bit dumb but by understanding why exactly it's dumb, you can communicate what correction is needed pretty smoothly.