Remix.run Logo
ray__ 14 hours ago

Any previewers have hot takes? I've really preferred gpt-5.5 over Opus 4.8 for data analysis and scientific software work. It seems much more reliable. Fable is unusable for the type of work that I do (due to guardrails). Really looking forward to trying these new OpenAI models out.

CjHuber 14 hours ago | parent | next [-]

Interesting to hear people like gpt-5.5. For me it feels smart only at one shot prompts, but if you try to build up session context before doing something it feels magnitudes inferior to Claude. I'm almost sure its because the thinking of previous turns is stripped with the responses API, so if I tell it to analyse something deeply, what remains of the understanding in future turns is only the short response text of that analysis

porker 14 hours ago | parent | next [-]

For gpt-5.5 I build up that session context into a markdown file, and then I start a new session and give it the markdown file with the instructions for what to do.

I'm guessing this works better because it can always go back and re-analyze the saved context.

prodigycorp 13 hours ago | parent | prev | next [-]

Do you have a source for this? I'm pretty sure responses api is only there to obfuscate reasoning, but that they're still keeping reasoning traces in the backend.

tedsanders 12 hours ago | parent | next [-]

I work at OpenAI and can confirm that's correct: reasoning tokens are discarded after each new user turn (though not after each message or tool call).

Our docs show a diagram here:

https://developers.openai.com/api/docs/guides/reasoning

> Input and output tokens from each step are carried over, while reasoning tokens are discarded.

Keeping reasoning tokens around is better for caching and for remembering past insights, so you might reasonably wonder why we designed it this way. The main benefit of dropping reasoning tokens is that you can fit a lot more work inside the model's context window before you're forced into a slow and lossy compaction step. This was a larger consideration with our earlier reasoning models that had shorter context windows (~200k), longer thinking times (up to ~100k per message), and poor compaction. However, now that we've shipped longer context windows, we've trained our models think much more efficiently, and we've made compaction way better than it used to be, the balance of factors is changing. Tune in Thursday!

CjHuber 11 hours ago | parent | next [-]

> fit a lot more work inside the model's context window before you're forced into a slow and lossy compaction step

This is something I never understood. Why the reasoning is not included until the context is full, then the reasoning stripped optionally to allow the conversation to continue. and only then when its truly full offer a compaction. Was it to optimize caching? Well I guess it doesn't matter now that you hinted that this choice was made because of prior limitations and may change very soon

embedding-shape 9 hours ago | parent [-]

> Why the reasoning is not included until the context is full, then the reasoning stripped optionally to allow the conversation to continue. and only then when its truly full offer a compaction.

Models are typically trained (at longer conversations/more turns) either with or without the reasoning still in the conversation. If you train a model with those, then using it without them, the model will perform a lot worse, same vice-versa if you train without but then end up using the model with them.

That's why you'll see some models have it and others don't, and trying to use them another way, will make them worse, they weren't trained like that.

So why aren't the models trained with both? I'm guessing that sort of permutation in the training would lead to double the amount of training time being needed, as you know effectively will have two variants of every session you train on, with and without the reasoning.

throwuxiytayq 12 hours ago | parent | prev [-]

dude what the hell

wahnfrieden 13 hours ago | parent | prev [-]

[flagged]

CjHuber 11 hours ago | parent | next [-]

Well, obviously its true as you can see in the silbing comment. But still interesting how experiences differ. I wonder did you ever use codex in more demanding workflows, because that at least something like that is going on is obvious if you use it for a few hours

satvikpendem 11 hours ago | parent | prev | next [-]

Looks like you're wrong, if you look at the sibling

simiones 11 hours ago | parent | prev [-]

You answered at more or less the same time as an OpenAI engineer confirming this is the case, explaining why they did it, and pointing to their docs that also very explicitly explain this "made-up nonsense". You should really consider why you felt so certain about something you were plain wrong about.

orliesaurus 11 hours ago | parent | next [-]

> You should really consider why you felt so certain about something you were plain wrong about.

I applaud this reply!

satvikpendem 10 hours ago | parent | prev [-]

Exactly, it's ironic that they thought they were right while talking about not having sources for another comment.

throwuxiytayq 13 hours ago | parent | prev [-]

> thinking of previous turns is stripped with the responses API

Why do they store an encrypted reasoning payload in the session file and pass it to the API? Just a ruse? Reasoning isn’t even that many tokens, you think they’d degrade their model quality like that?

Reasoning messages would be lost immediately after a single tool call, unless you mean they sometimes go back and strip the reasoning channel retroactively, but that would increase costs via cache invalidation. I just don’t see any way it would make sense for them to do.

And wouldn’t this be noticeable by reasoning tokens not being accounted for in the context window usage?

pavpanchekha 14 hours ago | parent | prev | next [-]

For compiler work I found that Sol is noticably better than 5.5 (and I generally use OAI models because I like the Codex app), but Fable was still obviously better.

cbg0 10 hours ago | parent [-]

Better in what way? Does it follow the goals better, does the code produce have higher quality in a testable/maintainable sense or is it just closer to how you would usually program something?

ottoboney 14 hours ago | parent | prev | next [-]

It seems comparable to Fable to me in my uses.

scottmf 14 hours ago | parent | next [-]

What types of use cases?

wahnfrieden 14 hours ago | parent | prev [-]

That's great to hear - and for the same price as 5.5, and reportedly with much lower token use per task.

Were you able to try Sol Ultra?

ottoboney 14 hours ago | parent [-]

No, my organization limits access to xhigh.

11 hours ago | parent | prev | next [-]
[deleted]
hodgehog11 9 hours ago | parent | prev | next [-]

I'm sorry to hear you are unable to use Fable; my partner is in the same boat and it frustrates her immensely to see what I've been able to do with it. As someone who is working with developing new linear algebra routines, Fable is so far ahead of GPT-5.5 and Opus that it's obscene. Massively better insights and far better at handling delicate corner cases without needing to mention them. I would be stunned if GPT-5.6 is at that level, but one can hope.

petesergeant 13 hours ago | parent | prev [-]

Interesting, data analysis work is the only thing I’ll use Gemini for