Using ciphertext for inference would mean it's not a very secure ciphertext.

These two ideas don't compute for me.

Same thing with homomorphic encryption. I don't get it. If you can gain any knowledge from a ciphertext, you just found a way to exploit the ciphertext to me.

▲

NitpickLawyer 4 hours ago | parent | next [-]

> Using ciphertext for inference would mean it's not a very secure ciphertext.

Inference is done in plain text. It's just that some parts of the response can be encrypted. While I haven't looked into this specific implementation, here's a short "how I'd do it" if I wanted to implement this:

Before:

[] - encrypted {} - plain text

1. user -> please do this -> server

2. user <- a) [thinking1] encrypted; b) {answer1} plain text <- server

3. user -> please do this -> [thinking1] (sent encrypted as received) -> {answer1} -> good but do this instead -> server

4. user <- [...] <- [thinking2] ; {answer2}

(here the server decrypts the thinking parts, adds them to the conversation, does the inference, and sends back the new thinking trace (encrypted as well) and the new answer

After:

1. user -> please do this long task -> server

2. user <- [thinking1] ; {tool_agent_spawn([params1])} ; {answer1} (e.g. would you like me to explore or do a quick hack?) <- server

3. user -> please do this long task -> (decides if explore or message) spawn([params1]) / message -> server

3. a) if no explore -> send message as usual 3. b) if explore execute spawn that in turns begins 2 channels

4. user <- [channel_1_thinking] ; {channel_1_answer} ; [channel_2_thinking] ; {channel_2_answer} ... <- server

So the server always does inference on plain text. But it sends the "important" bits encrypted, and you only send those back if you as the user want to (or need to, or choose to, etc). The idea is that the client still gets to decide on "local" things, but the server keeps the important bits from reaching the client. In this particular case, the [params] are encrypted bits that can include prompts, etc.

▲

3form 5 hours ago | parent | prev | next [-]

The idea of homomorphic encryption is to do things without the knowledge, and not gaining the knowledge. If ciphertext contains a number, and you don't need to know what number it does to always be able to multiply it by 2, you succeeded - as a simple example.

▲

smalltorch 5 hours ago | parent [-]

It still just sounds like fancy obfuscation to me. I've read alot of examples trying to understand but I can't get past that being able to run processes on ciphertext in a way you can learn something doesn't make sense without me changing my definition of what I think encryption means.

▲

_flux 4 hours ago | parent [-]

I thought though the idea is that you cannot learn anything from the ciphertext, how it is processed, or what the final result is?

Unless you are a participant of the computation and you have the key, that is.

▲

smalltorch 3 hours ago | parent [-]

To actually make it work, you would need to preformat your data very specifically, and the data you want to allow to be processed would need a subkey to unlock the parts you want processed.

I don't see a way to make it a open standard. The processing steps would need to be part of the key.

Anyway If someone figured it out I would be very interested to be sure they weren't just trying to slap a it's encrypted to meet some standards required. And..also, what amount of data processing does Alice need to do that required outsourcing to Bobs machines. Data processing and anaysis is cheap.

▲

3form 2 hours ago | parent [-]

The point is to not have subkeys or any other way to access any part of the plaintex - as the goal is generally to offset the computation and the costs of it to another party. And yes, most likely it will require a very particular format of the data, and in all likelihood will allow only for a limited set of operations.

▲

smalltorch 23 minutes ago | parent [-]

I wish I understood the actual data Alice needed processed to wrap my head around the problem space. Reading examples of what it could be used for leave me more confused.

Without knowing the actual pointed problem space the generalities of the schema just sounds like a snake game to me.

I can't get past where the veil is dropped for bob to take some ciphertext and operate on it in any meaningful way for him to create some machine that simultaneously can process encrypted data while being completely blinded to what the data is or the insights learned. Any form of ability to process a datum while simultaneously being blinded to it is logically incompatible to me.

Now, could we obfuscated it and lower the chances of an attacker being able to do anything useful with it or knowing what is being processed? Yes...but why muddy the definition of encryption in order to do this?

	▲	3form 3 minutes ago \| parent [-]
		One example I can think of: privacy preserving ML models. Company wants to keep their model safe and also perhaps not to force you to run some classification on your phone for battery savings. You want your inference features to be private, because they e.g. contain your location data. You send them encrypted, you get encrypted results back and decrypt them. It could have been obfuscated, but assuming we have HmE, obfuscation is more tricky to be done right. Do we change the definition of encryption meaningfully in this process, though? If so, I don't see how really. It's just that out of set of potential encryption algorithms, for this purpose we would pick ones that are fit for it. It wouldn't be AES I guess... But it would still be encryption. Maybe a weaker one, but that weakness could be coincidental. I'm not familiar with any theorems in that space.

▲

tybit 4 hours ago | parent | prev [-]

They’re not using ciphertext in inference. They are encrypting agent responses on their servers if it’s going to a subagent on the client. The subagent will send it back to their servers for inference. Only their servers have the keys, so they can decrypt when running inference.