Remix.run Logo
SyneRyder 5 hours ago

Most of that is paywalled, but this one paragraph in the Bloomberg article suggests it might be more to do with investors leaking information:

"The suspension stemmed in part from Liang’s frustration over online reports about his comments to investors during his first financing deal"

The part of the transcript I'd seen floating around online was this part from around 1 hour 26 min:

"With the largest models available today, we simply cannot afford to train them. Even if we spent all five hundred billion yuan, we still wouldn't be able to do so. Even if we could accumulate the resources, we wouldn't have the means to utilize them. The current largest model requires approximately 800 billion activations; domestically, we are still at a scale of several dozen billion activations, and even the largest domestic model may only require several dozen billion activations—a difference of an order of magnitude. To train a model of the same size as an AI system, we would need around 50,000 GB300 GPUs or Huawei 950 GPUs, totaling two hundred thousand cards. This is merely training; research has not yet been considered. Therefore, the biggest gap between us and the United States lies in resources."

carterschonwald 4 hours ago | parent | next [-]

amusingly ive been working on ultra sparse llm inference/ training/ model design because nature loaths a dense graph/matrix and cause i think it shoukd be possible. i actually stood up a 20-25 percent faster than sota causal fast attention kernel yesterday, will be standing up cuda/metal/armv8 kernels too and thats gonna be fun.

i genuinely think these models should be like 0.1 percent sparse for same capabilities we associate with them today, but theres no sane way to do that with extent tools. i built the right core tech for that in 2014 when there wasnt a market, but now there is and the experimentation velocity is wild.

amusingly llms really have a hard time using my simple apis because its not in distribution array programs. but i literally stood up cpu custom memory format and micro kernel for dense causal attention in less than 24-36 hours and outperforms the equivalent fused ggml/llama cpp fast oath by like 20-25 percent

jacktang 3 hours ago | parent [-]

Look forwarding your future releases

carterschonwald 2 hours ago | parent [-]

i definitely will be doing some drop of some faster attention kernels in the next few weeks.

like i can do all sorts of memory layout of tensors/matrices etc tricks that if you dont have the abstractions for it would just never happen. so i can optimize the kernel flops

3 hours ago | parent | prev [-]
[deleted]