| ▲ | DCKing 4 hours ago |
| This model seems to be pretty benchmaxed. In actual use, it doesn't survive the comparison to models in the benchmark. This model was released as a "stealth model" in OpenCode Go for a while as "Space Bunny" (they mention this in the blog). So people have been using without knowing what the model was exactly, but it failed to impress anyone as far as I can tell. This is despite the model being served for free (with some asterisks). GLM 5.3 Flash and DeepSeek V4.1 Flash, even before last week's update, seemed to be much preferred. Stealth releases happen often with OpenCode subscriptions, and GLM 5.3 Flash (stealth name "Ox Alpha") impressed a lot for example. The perverse incentives around benchmarks are now so bad that benchmark scores are effectively meaningless. Models can score highly and still be crap to use. It doesn't matter whether it's Minimax or Google, or anyone really. |
|
| ▲ | dom96 8 minutes ago | parent | next [-] |
| For what it's worth, it does perform on par with DeepSeek V4 Pro on my adversarial programming language benchmark[1]. I wouldn't think they would benchmax on that. 1 - https://bench.killswitch-lang.org/ |
|
| ▲ | Rebelgecko 4 hours ago | parent | prev | next [-] |
| I used Space Bunny from some providers that offered it for free and it was a very much a "you get what you pay for" experience. If the release is comparable, I'll be sticking with Deepseek v4 for sure. |
| |
| ▲ | adinb 3 hours ago | parent [-] | | I’m one of the unimpressed. I tested SpaceBunny intensely on a research project for a week. The model wasn’t horrible at coding, it just made a ton of initial mistakes—but then would catch most of them later on. It can get work done if coaxed, but its peers are just cheaper and/or easier to work with. | | |
| ▲ | CapsAdmin 6 minutes ago | parent | next [-] | | What kind of mistakes? I've used local models extensively (mostly qwen 3.8 27b) and now Claude sonnet/opus 5.5 since launch. Sometimes I really wonder if not being able to see what's going on in thinking is a large contributor to thinking the models are stronger. That assumes you bother skimming through thinking traces and the output in-between. In my experience both qwen and claude fumbles and misunderstand, but will in a lot of cases self correct. Qwen models can feel very indecisive but somehow makes it in the end. If the harness was able to hide that and only highlight the end results, I'd probably think qwen models are smarter. Reading the tool calls Claude does while thinking shows a lot of trial and error about even simple things like editing code and getting the whitespace wrong. But it happens so fast compared to qwen that it's sort of acceptable. With qwen, it can feel frustrating because it spends so much time on it. This is all assuming that's what you meant by mistakes, but maybe you meant something different. | |
| ▲ | taytus 2 hours ago | parent | prev [-] | | Same experience. For me was "OK", not "Now we are cooking!" |
|
|
|
| ▲ | jstummbillig 2 hours ago | parent | prev | next [-] |
| > The perverse incentives around benchmarks I am not sure. The people who actually care enough to look at benchmarks will run evals, and find out quickly. If you post silly benchmark scores too often, you will simply be ignored. You are paying with your reputation. |
| |
| ▲ | eli 2 hours ago | parent | next [-] | | All the benchmarks are silly but people still want a quick way to put a new release in context | |
| ▲ | Forgeties79 an hour ago | parent | prev [-] | | Those consequences never stopped blog spam previously. Won’t stop it now lol |
|
|
| ▲ | hgoel 28 minutes ago | parent | prev | next [-] |
| Oh yeah, Space Bunny was awful, massive step back from even Qwen3.8-Flash-Next |
| |
| ▲ | bitexploder 8 minutes ago | parent [-] | | I know I mention it way too often right now, but I am just so impressed with Qwen Flash Next right now. I have been using it locally and it's just such a great little model to work with. It makes mistakes and has its quirks, but in general, it just gets things done and its default manner is just right. It is not overly proactive or making messes for me, even compared to DeepSeek Flash 4.1. I really like DS Flash too, it can just be /really/ annoying in some roles and to chat with some of the time. I can run Qwen Flash Next on hopes, dreams, DDR4, and ancient data center GPUs at 200 t/s :) |
|
|
| ▲ | toddmorey 2 hours ago | parent | prev | next [-] |
| I don’t get the endgame. If your model underperforms in real life usage, the backlash comes quickly. I guess I’m asking what exactly the perverse incentive is? A day of press at most? |
| |
| ▲ | phainopepla2 37 minutes ago | parent [-] | | A day of press is probably better than no press at all, if you're the CEO of a publicly traded company |
|
|
| ▲ | RussianCow 2 hours ago | parent | prev | next [-] |
| This is my experience with all MiniMax models: They're good on paper but fail to live up to expectations in practice. IMO, at no point has any MiniMax model been the superior choice over others, all things considered. |
| |
| ▲ | arcanemachiner an hour ago | parent [-] | | Yeah, I think of MiniMax as a good video generation model provider. Their languages models, by comparison, are not as impressive. |
|
|
| ▲ | em3rgent0rdr 3 hours ago | parent | prev [-] |
| "When a measure becomes a target, it ceases to be a good measure." https://en.wikipedia.org/wiki/Goodhart%27s_law |
