| ▲ | npn 2 hours ago | |
It is technically impossible, considering they need to serve them live and the token generation speed is a hard limitation. Just think about it, since us models hide their CoT, there will be dozens seconds delay before the first output token retuning to the user. And to predict the fake CoT correctly you need to at least read enough output tokens to make it believable. Then you have to hide their own model CoT to generate said synthetic CoT, which mean further delays. Adding up it will be at least 30s or 1m before you first seeing the thinking process. I don't think nobody would notice such delays, because Chinese models do not need to hide the CoT like US models so usually they start emit text pretty soon | ||
| ▲ | staticman2 2 hours ago | parent [-] | |
Wouldn't a delay due to synthetic COT generation look pretty much the same as a delay due to insufficient compute for all customers? | ||