Remix.run Logo
NitpickLawyer 4 hours ago

> Using ciphertext for inference would mean it's not a very secure ciphertext.

Inference is done in plain text. It's just that some parts of the response can be encrypted. While I haven't looked into this specific implementation, here's a short "how I'd do it" if I wanted to implement this:

Before:

[] - encrypted {} - plain text

1. user -> please do this -> server

2. user <- a) [thinking1] encrypted; b) {answer1} plain text <- server

3. user -> please do this -> [thinking1] (sent encrypted as received) -> {answer1} -> good but do this instead -> server

4. user <- [...] <- [thinking2] ; {answer2}

(here the server decrypts the thinking parts, adds them to the conversation, does the inference, and sends back the new thinking trace (encrypted as well) and the new answer

After:

1. user -> please do this long task -> server

2. user <- [thinking1] ; {tool_agent_spawn([params1])} ; {answer1} (e.g. would you like me to explore or do a quick hack?) <- server

3. user -> please do this long task -> (decides if explore or message) spawn([params1]) / message -> server

3. a) if no explore -> send message as usual 3. b) if explore execute spawn that in turns begins 2 channels

4. user <- [channel_1_thinking] ; {channel_1_answer} ; [channel_2_thinking] ; {channel_2_answer} ... <- server

So the server always does inference on plain text. But it sends the "important" bits encrypted, and you only send those back if you as the user want to (or need to, or choose to, etc). The idea is that the client still gets to decide on "local" things, but the server keeps the important bits from reaching the client. In this particular case, the [params] are encrypted bits that can include prompts, etc.