| ▲ | natrys 3 hours ago |
| Some official benchmark numbers posted in Chinese social media (I am sure they will publish an English blogpost later too): https://mp.weixin.qq.com/s/V4xhEIy8xDXSMDPrPkmUAQ Generally looks like a Sol/Fable tier model, better across the board than Opus 4.8. (Edit) English blogpost is up now: https://www.kimi.com/blog/kimi-k3 |
|
| ▲ | GodelNumbering 3 hours ago | parent | next [-] |
| The link has 6 well-known benchmarks where this beats Fable (out of 14 I counted). If the numbers hold up scrutiny, this is scary good. Forget about their pricing but the companies that do have means to host such models fully on-prem are also the same companies that are paying tens of millions of $ in inference cost every month, and are by extension the biggest customers of OAI and Anthropic |
| |
| ▲ | InsideOutSanta an hour ago | parent | next [-] | | > If the numbers hold up scrutiny, this is scary good. After using it for a few hours, I believe these benchmarks. | |
| ▲ | echelon 2 hours ago | parent | prev [-] | | Open Source >>> Closed Source [1] I don't want to cheer against my country, but we've given up on open source. The way Anthropic and OpenAI treat their customers as adversaries is embarrassing. I will cheer for China, for Kimi, and for z.ai until we have something in the same category. [1] I'd even be fine with open weights, fair source, or anything that let us have direct access to the weights. Even if that came with stipulations. Don't hide the weights from us. | | |
| ▲ | GodelNumbering 2 hours ago | parent [-] | | I am with you in the spirit of openweights but I am trying to hard-avoid bringing countries into this. The narrative of US vs China only benefits those who want regulatory capture in the US since attacking China is politically much easier than attacking open-weights, so certain groups like to repeatedly call them 'Chinese models'. | | |
| ▲ | echelon 8 minutes ago | parent [-] | | It's much more a rallying cry for open weights funding than it is for regulatory capture. The argument on our side wins - if America or the West don't do open source, China will. And that means -- with certainty -- that China wins the market. Every politician and VC should hear that loud and clear. |
|
|
|
|
| ▲ | tgtweak 3 hours ago | parent | prev | next [-] |
| I think given how much benchmaxxing we're seeing - the anecdotal evidence of how competent this model is (and efficient) will depend on user's actual real-world use cases. Given the pricing, it suggests that this model is much more efficient/competent than previous-gen OS/distilled models. |
|
| ▲ | zarzavat 3 hours ago | parent | prev | next [-] |
| It's like reading Anthropic's obituary. |
| |
| ▲ | austinthetaco 3 hours ago | parent | next [-] | | This is weird and reactionary. Lots of organizations are continuing to refuse to use chinese models due to security and IP concerns. Anthropic/american models aren't going anywhere anytime soon. | | |
| ▲ | cromka 2 hours ago | parent | next [-] | | > Lots of organizations are continuing to refuse to use chinese models due to security and IP concerns This is such a common omission: the Chinese models are open, you can host them yourself on your premises. So privacy and independence. | | |
| ▲ | baq a few seconds ago | parent | next [-] | | Good luck hosting 2.8T params yourself. A box capable of this is at least $100k. | |
| ▲ | spongebobstoes 2 hours ago | parent | prev [-] | | it's well documented that models can be adversarially trained with essentially backdoors in response to special inputs while I am skeptical that this is happening atm, there are probably many industries where the risk does not seem worthwhile | | |
| ▲ | anon373839 an hour ago | parent | next [-] | | I suppose this is like when Anthropic was using “prompt modification, steering vectors, or parameter-efficient fine-tuning” to poison the work of people working in the LLM field, including academic researchers. | |
| ▲ | Giefo6ah an hour ago | parent | prev | next [-] | | When the model is open weights you can even pass every token (including the chain of thought) though a fourth-party lightweight model like gpt-oss-safeguard to check that it has not become adversarial. | |
| ▲ | selectodude 2 hours ago | parent | prev | next [-] | | I feel like that's a threat that isn't super difficult to block. Unplug it from the internet, require it to go through an API intermediary to access web pages. Maybe I just don't have any imagination. | | | |
| ▲ | 38 minutes ago | parent | prev [-] | | [deleted] |
|
| |
| ▲ | oceanplexian 2 hours ago | parent | prev | next [-] | | > Lots of organizations are continuing to refuse to use chinese models Correction: Lots of organizations are refusing to use Anthropic Fable because they have forced opt-in data collection as part of their privacy policy, even for Enterprise. | | |
| ▲ | ben_w 2 hours ago | parent [-] | | Both things, and both reasons, can be true at the same time. Not everyone's going to care about Anthropic requiring data collection (a similar debate plays out with regards to "pay or consent" on website tracking), just as not everyone cares about China with regards to security/IP issues (if they did, a lot more would be banned besides occasionally-Huawei). |
| |
| ▲ | sscaryterry 3 hours ago | parent | prev | next [-] | | Nope, but I think this is maybe the critical mass needed to finally crash the AI hype/datacenter cost problem everyones is talking about. With Oracle being junk before this, more will follow. | | |
| ▲ | reissbaker 2 hours ago | parent | next [-] | | I would assume the opposite is true — with an open-weight Fable-class model, doesn't demand for GPUs go up? Plenty of companies can now look at what Anthropic is offering — high per token costs for a very intelligent model — and do the math, and at some point it makes sense to just rent the GPU yourself and run Kimi on it if you get similar intelligence without paying Anthropic's margins (albeit with high upfront capital cost). This would drive down Anthropic's margins, but drive up demand for datacenter and GPU capacity. It's not that people would be using fewer GPUs, they'd just shift demand from high priced token vendors to direct GPU rental, which benefits datacenter companies while hurting Anthropic. | | | |
| ▲ | stevefan1999 2 hours ago | parent | prev | next [-] | | Oracle is fine, it's just that they can't really expect political decisions that hindered it to accquire TikTok which will be slated to be the biggest customer if the deal went through. Now they are betting with Project Stargate but it also seems to be crumbling down. But don't forget that they literally hold the biggest databases, both in commercial and open source, that is, Oracle Database and MySQL. Plus Oracle Java they literally controls at least 30% of the internet's software infrastructure. And also with a good team of attorneies enforcing the licenses, they can squeeze so much money at the cost of morality. Also recently they downgraded the always free OCI ARM instance from 4C24G to 2C12G without telling anyone. | | |
| ▲ | calgoo 13 minutes ago | parent | next [-] | | New enterprise java licenses are going to milk enterprise just like broadcom is doing. New license deals makes you pay for employee total number (including contractors) instead of for users of oracle java. | |
| ▲ | re-thc 2 hours ago | parent | prev [-] | | > Oracle is fine They're drowning in debt and risk is increasing. If these US models don't keep holding up their valuation will tank further and some will recall the loans or ask for different terms. |
| |
| ▲ | ai-x 2 hours ago | parent | prev [-] | | Models need datacenters to run. It also need other services to do anything useful | | |
| ▲ | sscaryterry 2 hours ago | parent [-] | | The point: Fable isn't worth what Anthropic says it is, so Anthropic isn't as valuable as they make themselves out to be. The DeepSeek incident has already shown it, this is a reminder. |
|
| |
| ▲ | jml78 3 hours ago | parent | prev | next [-] | | If it ends up being open weights, companies will use it running in US data centers. | |
| ▲ | adastra22 3 hours ago | parent | prev | next [-] | | You can run open weight models anywhere. | |
| ▲ | woadwarrior01 2 hours ago | parent | prev [-] | | Cursor will rebrand it as Composer 3.0 to assuage any such concerns, as they did with the previous Kimi models. | | |
| ▲ | re-thc an hour ago | parent [-] | | More likely for them to use Kimi 2.7 since Grok is now the flagship product. |
|
| |
| ▲ | scrollop an hour ago | parent | prev | next [-] | | Nah: https://www.youtube.com/watch?v=LSlV206xPqM These real world examples show it's one tier away. | | |
| ▲ | VulgarExigency 39 minutes ago | parent | next [-] | | These "real world" examples are nothing like the way I use LLMs from within a harness. GPT 5.6 Sol and Fable are clearly more impressive, but how does this translate to interactive agent use, or use under an agent orchestration framework? | | |
| ▲ | pimeys 7 minutes ago | parent [-] | | This is a question I am going to get an answer tomorrow with evals. Extremely interesting... |
| |
| ▲ | an hour ago | parent | prev [-] | | [deleted] |
| |
| ▲ | refulgentis 3 hours ago | parent | prev [-] | | Fable is by Anthropic, and this is too expensive, GLM 5.2 is roughly the same quality at a much cheaper price. (I mantain a client with llama.cpp and 101 models across 14 companies by http) | | |
| ▲ | pimeys 5 minutes ago | parent | next [-] | | GLM has issues with tool calls and nested JSON and it wastes tokens pretty often. I see it being a bit above half the price of Opus in a bit more complex eval tasks. With some RL you could probably get the tool calls sorted and the price down. | |
| ▲ | LaurensBER 3 hours ago | parent | prev [-] | | As much as I like GLM 5.2 it's clearly a step below Opus (or even Fable) for more complicated tasks. I would place it at Opus 4.6/4.7 level. Having said that, the safety system on Fable makes it an extremely unattractive model. It feels that half of the time you're paying double for Opus level performance. | | |
| ▲ | jml78 3 hours ago | parent [-] | | Fable won’t even generate a jwt to test endpoints because it is security related. It is crazy capable but useless for real work | | |
| ▲ | weego an hour ago | parent [-] | | Unless your real work is outside the scope of one tiny niche of work. |
|
|
|
|
|
| ▲ | scrollop an hour ago | parent | prev | next [-] |
| Meh, not fable/sol tier: https://www.youtube.com/watch?v=LSlV206xPqM |
| |
| ▲ | natrys an hour ago | parent [-] | | If anecdote is data, then here's another point: https://nitter.net/synthwavedd/status/2077537805715005724#m (As an aside, I don't know how it was professional of Arena to unmask an unreleased cloaked model on their platform. Also practically, upstream could have been A/B testing multiple variants under same endpoint, casting validity of such pre-announcement tests into question) |
|
|
| ▲ | vonneumannstan 2 hours ago | parent | prev [-] |
| Crazy how their models always come out after the US labs and just lag the performance of top models. Almost like they are performing distillation attacks... how strange. |
| |
| ▲ | twobitshifter 42 minutes ago | parent | next [-] | | distillation attack? why the violent word choice? When OpenAI crawled Github was that an attack? | |
| ▲ | MaxPock 2 hours ago | parent | prev | next [-] | | Do you have moat if your advanced model can be distilled in a month or two ? | |
| ▲ | rowanG077 34 minutes ago | parent | prev | next [-] | | Distillation is not an attack. It simply a way to train a model. Not doing it when you are behind is akin to snatching defeat from the jaws of victory. | |
| ▲ | PepegaRoach a minute ago | parent | prev [-] | | [dead] |
|