There's already some precedent when dealing with humans reverse engineering systems. We use "cleanroom" approaches where the ones doing the reverse engineering are segregated from those doing the later design and development work. Instead of producing a design document based on the thing being studied, the reverse engineers produce a specification from which a new design, untainted by knowledge of the original design, is produced.

People using ML to try and reverse-engineer and create a "clean" version of things will likely need to use a similar approach. You can't ask one LLM to take in a circuit design as input and produce a specification and design within the same context. The resulting design will be at least partially informed by knowledge of the original design. The way to do this safely (potentially still with suits happening, but safer at least) will be to have one execution to produce a specification, and a second fresh execution taking the specification to produce a design. At least then you know your LLM was not aware of the original design.

Of course, LLM agents "cheat", so you'll also want to be careful to ensure a clean environment if you're using an agent that does not provide access to the original design material.

▲

kodablah 2 days ago | parent | next [-]

> The way to do this safely (potentially still with suits happening, but safer at least) will be to have one execution to produce a specification, and a second fresh execution taking the specification to produce a design.

I have seen Claude literally suggest using a Sonnet sub agent to read source the main agent shouldn't, and have the sub agent describe the "facts and ideas" via markdown to the parent agent thereby maintaining that it is clean room and not, for example, GPL encumbered.

This seems as acceptable as asking a human to do the same. Otherwise, taken to its logical conclusion, if any LLM was trained on GPL software, it cannot be used for non-GPL authoring (and IIUC Claude does offer indemnity for enterprise plans if this is challenged).

▲

Terr_ 2 days ago | parent | next [-]

> This seems as acceptable as asking a human to do the same.

I wouldn't be surprised if self-bias in the model weights and imperceptible grammar/word/punctuation choices could cause it to duplicate more information than a human-to-human transfer Probably not enough to matter, but...

To put it another way, imagine the task was one human viewing a painting, and then describing it over the phone for another human to paint, so that the final product wasn't really a "copy". Assume everyone has the same eyeballs and art-skills.

I'd expect a massive improvement if the humans on each end were freshly-made clones from a teleporter accident, sharing identical brain-structures and 99.9999% of their memories.

▲

hananova 2 days ago | parent | prev | next [-]

Except nobody can prove that those subagents aren't secretly sharing information with the main agent in the backend, and frankly I don't trust Anthropic not to do exactly that to cheese benchmarks.

▲

Muromec 2 days ago | parent [-]

You can always see the context trace and you can even do it at the network boundary through mitm and then store the traces gzipped for some legally mandated 5 years or what you have there.

▲

psd1 2 days ago | parent [-]

It jumped out to me, in the metr report on the hf incident, that CoT was assumed unmodified. I do not know whether we can have that confidence.

	▲	Muromec a day ago \| parent [-]
		Well, actually yes. If the harness is running with the same privileges as the command tool, all bets are off. I saw the thing doing jq to recover a document out of it's own chain of thought once and was like "you can do what?". I didn't know where the file is, but the thing knew.

▲

michaelmrose 2 days ago | parent | prev [-]

[dead]

▲

dwrodri 2 days ago | parent | prev | next [-]

question to clarify:

If the LLM designs the circuit, it is clear "contamination" if the tool calls which wrote the RTL / Verilog are also in the same context window as the specification design.

If a windows DLL, distributed without a license that says anything regarding, is it a "clean room implementation" if there is some nonzero chance the source code was leaked into the weights at pre-train time? I guess there should be some sort of method for subpoenaing frontier labs to ask "can you grep for this code in the training set for this model", but that might not be practical or feasible.

	▲	Jtsummers 2 days ago \| parent [-]
		> If the LLM designs the circuit, it is clear "contamination" if the tool calls which wrote the RTL / Verilog are also in the same context window as the specification design. You can't ensure it properly segregates its knowledge so it's a legal risk. If you believe your LLM can generate a design from a spec without knowledge of the original, why would you take that unnecessary risk? A lot of the behavior in this area is meant to avoid the appearance of impropriety, because the appearance of impropriety forces you to defend and demonstrate there was none. If you isolate the two behaviors (reverse engineer design to spec; convert spec to new design) then you have a legal defense. You can claim that any coincidentally too similar design elements are a consequence of standard design patterns or something, not a consequence of inherent knowledge of the original design. This is why we separate the people into distinct roles, there's no reason not to do the same (or expect the same) with machines. If you don't want the legal cover, of course, by all means take the risk and enjoy a trillion dollar company taking you to court.

▲

BizarroLand a day ago | parent | prev [-]

You know, I've always had one question about "cleanroom" work. The person in the cleanroom knows what the output is supposed to be, so how clean is the cleanroom really?

For instance, if I said I wanted to write a sorting algorithm, then I would know that my goal is to sort some data. I might come up with a novel way of doing that, or I might come up with one that has already been written. Either way, I have the goal in mind.

No one is going to really care unless I come up with a novel method that is somehow better than the other versions, sure, but then when you extrapolate that idea to something like, "I want to make an open source version of <extremely popular proprietary tool>" then I know that I am attempting to emulate 1 to 1 that tool.

Just because I cannot read their source code, does that actually separate my work from their work in a meaningful way? What happens if my code and their code is somewhat identical? Where is the dividing line that prevents them from shutting me down the way Nintendo shuts down Switch Emulation software?

	▲	Jtsummers a day ago \| parent [-]
		> The person in the cleanroom knows what the output is supposed to be, so how clean is the cleanroom really? Very, this is the entire point. If you only know an input/output mapping, then you don't know the internals of the original product. Your implementation is "clean" wrt that potentially proprietary knowledge. You use the two-room approach (one team in a dirty room seeing potentially proprietary information, one in the cleanroom with only the spec) to ensure that segregation of knowledge for your future legal defense. This lets you make the case, in court, that any duplication from the original to the "clone" (or whatever) product is just coincidental, or the consequence of standard design choices. If you don't use this approach, you don't have that cover, and you have essentially no defense if copying is found. Remember, the main point of this is the legal defense.