Hey, thank you so much for jumping in and for your elaboration.

> SOC and piece of test equipment might have it's own control language

OK, so apart from code generation the LLMs are providing language integration and data integration.

But how would you control quality, or make sure the LLms does not create some unwordly control code? I am trying to understand if they are using type checking or any other methodology that keeps the LLM generated code in check.

I am not from the hardware field, but I do understand that we are trying to build deterministic mashines. It is quite mindboggling that we are using statistical models for building the test suite, instead of, say, permutating through all critical input states and testing the output parameters for it.

I am asking because this isnt certainly the only mode of operation for LLMs in testing and quality control, and I would love to get a better understanding how to build such a system.

AI's can write deterministic code and have been doing so for a while. Humans do this as well. We write deterministic code with a goal, and generally it works out okay.

If you say that you can't trust AI because LLM's are statistical models, that's a true statement. Though it also turns out that we don't trust humans to write code correct either, which is why we have code reviews.