| ▲ | torben-friis 3 hours ago | ||||||||||||||||||||||||||||||||||||||||
Can you explain how this removes their moat? Genuine question, I'm not sure I understand enough about LLMs to get how an open source inference engine undermines them. | |||||||||||||||||||||||||||||||||||||||||
| ▲ | mbesto a minute ago | parent | next [-] | ||||||||||||||||||||||||||||||||||||||||
When the model is fungible and the switching cost is near zero (business logic lies in the harness), then there really isn't a moat for models. | |||||||||||||||||||||||||||||||||||||||||
| ▲ | frays 3 hours ago | parent | prev | next [-] | ||||||||||||||||||||||||||||||||||||||||
Today the closed-source giants maintain a substantial cost-per-token advantage due to proprietary inference tricks like speculative decoding kernels, ultra-optimized FlashAttention derivatives, multi-GPU pipeline orchestration and dynamic prefix-caching algorithms. I'm not sure whether the DeepSeek's open-sourced inference engine does all of these, but it appears that plenty of inference tricks have just been revealed for the open source community to learn from and start improving upon. | |||||||||||||||||||||||||||||||||||||||||
| |||||||||||||||||||||||||||||||||||||||||
| ▲ | trentor 3 hours ago | parent | prev | next [-] | ||||||||||||||||||||||||||||||||||||||||
What moat? | |||||||||||||||||||||||||||||||||||||||||
| |||||||||||||||||||||||||||||||||||||||||
| ▲ | throwa356262 3 hours ago | parent | prev [-] | ||||||||||||||||||||||||||||||||||||||||
In theory, this can allow a small US lab to compete with O/A/G and take marketshare from them with small open models deployed centrally or (and this is think is the real moat) on-prem for enterprises. | |||||||||||||||||||||||||||||||||||||||||