| ▲ | NiloCK 5 hours ago | |
Data has historically been a moat, but I think now more than ever it's a moat of bounded size / utility. The biggest data hoarders now compress their data into oracles whose job is to say whatever to whoever - leaking an ever-improving approximation of the data back out. DeepSeek was a big early example of adversarial distillation, but it seems inevitable to me that frontier models can and will always be siphoned off in order to produce reasonably strong fast-follow grey market competition. | ||