Nvidia GPUs let you load encrypted weights onto them and to decrypt on chip so Google would need to have some sort of Nvidia management firmware attack to read from HBM while the GPU is live

▲

jayd16 5 hours ago | parent | next [-]

Isn't distillation more of a black box type affair?

▲

zeroxfe 5 hours ago | parent | prev | next [-]

Distillation does not require access to weights.

▲

binlog 6 hours ago | parent | prev | next [-]

Doesn’t really matter what happens at the GPU level when Google can see all API traffic.

▲

worldsavior 6 hours ago | parent | prev [-]

Except google cloud doesn't use Nvidia GPUs but TPUs.

▲

Mond_ 5 hours ago | parent [-]

Google has both. Afaik we don't know whether OpenAI models will run on TPUs or GPUs here.

	▲	tgma 4 hours ago \| parent [-]
		I bet it would be the NVIDIA, at least initially, not for the encryption capability per se, but for simple fact that adapting to TPU at high performance is non-trivial.