Remix.run Logo
▲ simonw an hour ago

Qwen4-Max pelicans at four reasoning efforts (none, low, high, xhigh): https://tools.simonwillison.net/markdown-svg-renderer?url=ht...

For comparison, here's one I got from Qwen3.8 Max (at default reasoning effort): https://simonwillison.net/2026/Jul/20/afraid-of-chinese-mode...

This inspired me to re-run pelicans through 3.8 Max - but OpenRouter has qwen3.8-max-prime and qwen3.8-max-0902 - prime is twice the price and faster - so I went with the 0902 one.

Turns out Qwen4-Max has a none setting, but 3.8 Max didn't allow disabling reasoning - you had to pick from these five levels instead: minimal, low, medium, high, xhigh.

Here are those 3.8 Max 0902 pelicans: https://tools.simonwillison.net/markdown-svg-renderer?url=ht... - they're mostly not as good as 4, but I prefer the 3.8 xhigh one.

▲Anon1096 an hour ago | parent | next [-]

I feel like the xhigh result is one of the first times I've seen a model go for a non-red colored bicycle (teal). Still haven't seen the leftward facing bike though.

▲inshard an hour ago | parent [-]

Astra and the OpenAI models do this color as well

▲gpt5 an hour ago | parent [-]

Evidence from where they distilling from /s

▲Starlevel004 an hour ago | parent | prev | next [-]

Feels like Pelicanbench is getting saturated now. Maybe time to switch to a different animal and vehicle combo?

▲slashtrends an hour ago | parent | next [-]

The 3d one still produces interesting variations - https://www.reddit.com/r/ClaudeCode/comments/1xs4ll7/fable_5...

▲sorrywhat 31 minutes ago | parent | prev [-]

First thing I do when a new model drop is scroll down to find the pelican comment, and then the "the pelican benchmark is no longer useful" debate under it.

▲lcnPylGDnU4H9OF an hour ago | parent | prev [-]

I kinda like the no-effort result more than low, but high and xhigh return to what one would expect for reasoning->quality.