Remix.run Logo
scotttrinh 3 hours ago

I like to compare models with a similar score on cost per task and output tokens per task since those measure two things I'm interested in: cost efficiency and token efficiency. Here's how GLM-5.3 compares to other models in a similar score and against GLM-5.2 to save a few clicks for others who care about these metrics:

    Model                        Score    Cost / Task    Output Tokens / Task
    -------------------------------------------------------------------------
    GLM-5.3 (max)                 59.5          $0.68                  41,107
    GLM-5.2 (max)                 53.0          $0.56                  32,200
    Claude Opus 5 (high)          61.5          $1.52                  21,353
    GPT-5.6 Sol (max)             60.9          $1.23                  16,879
    Grok 4.6 (high)               60.9          $0.84                  21,735
    Kimi K3 (max)                 59.7          $0.84                  25,474
    GPT-5.6 Sol (xhigh)           59.0          $0.87                  11,098
    Claude Opus 5 (medium)        58.6          $0.98                  12,459
    Qwen3.8 Max                   58.1          $1.13                  38,287
    Qwen3.8 2.4T A95B             57.7          $0.95                  32,472
    Claude Opus 4.8 (max)         57.3          $1.65                  33,557
    GPT-5.6 Sol (high)            57.3          $0.52                   7,545
    Muse Spark 1.2 (xhigh)        56.8          $0.40                  30,430
    GPT-5.6 Terra (max)           56.6          $0.51                  20,838
    GPT-5.5 (xhigh)               56.3          $0.69                  16,893
    Gemini 3.7 Flash (high)       56.0          $0.40                  36,847
Edited for accuracy and more models.
sourcecodeplz 3 hours ago | parent | next [-]

Muse Spark has a nice balance. not to mentions the Contribs version is old deepseek flash prices.

glub 2 hours ago | parent | next [-]

Tested muse spark 1.2 because it was rated so high on design arena, and I've missed a model that can do nice UI in the hands of an operator with no UI skills.

It produced worse UI mockups than GPT and GPT models are already the bottom of the barrel here. The only model that performed well was Kimi K3 - insanely good, but expensive.

It's hard to trust benchmarks these days.

sscaryterry 3 hours ago | parent | prev [-]

I found the sweetspot here: GPT-5.6 Sol (high) 57.3 $0.52 7,545

(Edit: TLDR; It gets on with it, makes the same mistakes you would, without overthinking and overengineering, most of the time)

dudeinhawaii 2 hours ago | parent | prev | next [-]

It would make reading and comparing a bit easier if the data was sorted by a dimension.

ac29 an hour ago | parent [-]

Cost per task:

  Model                        Score    Cost / Task    Output Tokens / Task
  -------------------------------------------------------------------------
  Muse Spark 1.2 (xhigh)        56.8          $0.40                  30,430
  Gemini 3.7 Flash (high)       56.0          $0.40                  36,847
  GPT-5.6 Terra (max)           56.6          $0.51                  20,838
  GPT-5.6 Sol (high)            57.3          $0.52                   7,545
  GLM-5.2 (max)                 53.0          $0.56                  32,200
  GLM-5.3 (max)                 59.5          $0.68                  41,107
  GPT-5.5 (xhigh)               56.3          $0.69                  16,893
  Grok 4.6 (high)               60.9          $0.84                  21,735
  Kimi K3 (max)                 59.7          $0.84                  25,474
  GPT-5.6 Sol (xhigh)           59.0          $0.87                  11,098
  Qwen3.8 2.4T A95B             57.7          $0.95                  32,472
  Claude Opus 5 (medium)        58.6          $0.98                  12,459
  Qwen3.8 Max                   58.1          $1.13                  38,287
  GPT-5.6 Sol (max)             60.9          $1.23                  16,879
  Claude Opus 5 (high)          61.5          $1.52                  21,353
  Claude Opus 4.8 (max)         57.3          $1.65                  33,557
Benchmark score:

  Model                        Score    Cost / Task    Output Tokens / Task
  -------------------------------------------------------------------------
  Claude Opus 5 (high)          61.5          $1.52                  21,353
  GPT-5.6 Sol (max)             60.9          $1.23                  16,879
  Grok 4.6 (high)               60.9          $0.84                  21,735
  Kimi K3 (max)                 59.7          $0.84                  25,474
  GLM-5.3 (max)                 59.5          $0.68                  41,107
  GPT-5.6 Sol (xhigh)           59.0          $0.87                  11,098
  Claude Opus 5 (medium)        58.6          $0.98                  12,459
  Qwen3.8 Max                   58.1          $1.13                  38,287
  Qwen3.8 2.4T A95B             57.7          $0.95                  32,472
  Claude Opus 4.8 (max)         57.3          $1.65                  33,557
  GPT-5.6 Sol (high)            57.3          $0.52                   7,545
  Muse Spark 1.2 (xhigh)        56.8          $0.40                  30,430
  GPT-5.6 Terra (max)           56.6          $0.51                  20,838
  GPT-5.5 (xhigh)               56.3          $0.69                  16,893
  Gemini 3.7 Flash (high)       56.0          $0.40                  36,847
  GLM-5.2 (max)                 53.0          $0.56                  32,200
tw1984 2 hours ago | parent | prev | next [-]

this is not very useful.

for over 1 billion real world users living in China, they don't have the option of paying $1.52 per task to use Opus 5, they are banned doing that due to US politics.

Barbing an hour ago | parent [-]

What’s the reseller situation?

teravor 3 hours ago | parent | prev [-]

these $/task figures aren't very useful in my experience. it doesn't tell you how well it did the task.

generally I choose models by their intelligence and then personal preference from direct experience.

kolinko 3 hours ago | parent [-]

they make a ton of sense for tasks that are achievable with no prob by all models - e.g. writing scripts that do a specific thing etc.

teravor 2 hours ago | parent [-]

    > writing scripts
you can use a Gemini model completely for free doing that. no agent. aistudio.google.com