Yes, that's a good point: you really need to work on the prompts, even if the harness is the same. Ambiguity is a real problem as is what we call 'the negative space', what isn't in the prompt but what is implied by the prompt to be undesirable. You need to spell that out in more detail it is not enough just to say 'do x' but you also need to make sure to say 'and not y or z', and knowing what y or z are requires a bit more insight into what hobby horses are encoded in a model.

To give one example: I'm doing a lot of stuff that involves hardware. When settings things up for fixing a bug I have to make sure to include that the hardware has been tested and works well. If I don't do that the model will almost certainly spend a ridiculous amount of time speculating about all the different ways in which the hardware could be broken (including theorizing about chip mask errors and all kinds of other obscure and rarely seen in practice defects). It's going to end up telling itself that for sure the 500 lines it just produced are absolutely perfect and the device that is on the i2c bus is broken in a way that it just really can't progress with solving the issue because there is nothing wrong with the code.

These kinds of blind-spots are funny when they happen once in a month but when they happen regularly it begins to be irritating. Adding a line or two to the prompt will take care of the issue. Frontier models tend to be better at this sort of thing.

That is very interesting. So far I am extremely careful to avoid putting negative statements into my instructions, to avoid poisoning the AI's thinking with exactly the concepts I do not want it to have. This might be from my use of much older models where if you say "do not X", then "X" gets put into the context and the "do not" kind of gets lost. It was especially visible with stable diffusion where you would ask for a picture without a feature and get exactly that feature. Maybe current models are better at that.

Us too! Hardware is very interesting - I find in fact that 'smart' (aka expensive) models can overthink complex hardware state machines and end up doing atrocious code because it will try to handle every possible edge case (unbidden, mind you). What has worked very well is harvesting and aggregating lessons and feeding this automatically into our pi sessions, this allows the models which listen to instructions to avoid a lot of time burning tokens on internal dialogue to ascertain the nature of reality.. but the expensive ones: a) they dont listen b) they ironically know too much!