Remix.run Logo
vorticalbox 8 hours ago

This is a problem I find with opus is will spend so long thinking then going “but wait what if”

To point where I stop it and simple tell it to “start writing code you can work it out as you go along”

Seems writers block also effects LLM

robertkarl 4 hours ago | parent | next [-]

https://arxiv.org/abs/2606.00206

In this paper they nerf an LLMs ability to emit waffling thinking tokens like "wait", "but", "alternatively", and the models (they're old, small models in the paper) terminate reasoning faster and perform better. I bet Anthropic is tuning this on their backend.

meatmanek an hour ago | parent [-]

This is super cool. Do you know if any of the inference backends (llama.cpp, vllm, etc) support this technique?

giancarlostoro 6 hours ago | parent | prev | next [-]

I usually have Claude build a plan first, then I put it into an XML file it updates with phases, usually we talk about some of those tasks, and then once its good and I like it, I have Claude implement the plan.

Another thing I tell Claude to do is to not guess, but look at documentation, it messes up a lot less, might use some tokens reading docs, but at least it has a higher success rate code wise.

xstas1 6 hours ago | parent [-]

XML??

giancarlostoro 5 hours ago | parent | next [-]

Apparently because of how Claude is trained, even the system level prompts go through as XML, it works better with XML "prompting" so I figured I could have it write plans in XML. I need to update my ticketing tool to output XML maybe by default.

https://www.reddit.com/r/ClaudeAI/comments/1psxuv7/anthropic...

saltsucker 5 hours ago | parent [-]

Comments later in thread say markdown works just as fine and that it’s more important to organize your plan into sections.

Also just think about it, why would a model trained on the world’s corpus of text (that isnt formatted in xml) perform better with XML? It would be a better study if that post tested markdown, org, xml, json, etc. 10 times to see if their is a difference

swingboy 3 hours ago | parent | next [-]

Anthropic’s best practices still include the use of XML: https://platform.claude.com/docs/en/build-with-claude/prompt...

adastra22 3 hours ago | parent | prev [-]

A year or so ago XML worked more reliably for long-lived prompt instructions. Now it is cargo culting.

root-parent 5 hours ago | parent | prev [-]

XML stands for Xtra ML....

noworriesnate 3 hours ago | parent [-]

I'd like to switch to a sales career--can you give me any pointers?

mikeocool 7 hours ago | parent | prev | next [-]

Seriously. Whenever I read the thinking output I get mad and turn down effort to medium or low.

Just output the code and we’ll work through it!

I feel similarly about having codex review claude’s plans. I don’t think I’ve ever seen it catch a major issue. It just points out things that would have inevitably been addressed during implementation anyway.

SubiculumCode 2 hours ago | parent [-]

A lot of times this is how humans work. Just start 'putting words on paper', 'think by doing', etc. sometimes it's more efficient to see why something won't work after writing a bit of it, and sometimes you get lucky and it works right off the bat

drob518 2 hours ago | parent | prev | next [-]

Qwen is notorious for this, too. It’ll sometimes spin in a long loop of “But wait…” paragraphs.

epolanski 7 hours ago | parent | prev | next [-]

Fable was 20 times worse on that.

It's clear it was the vibe coding model, as like no other model before, fully turned you into his assistant instead of the other way around.

RyanHamilton 7 hours ago | parent | next [-]

Could it be possible, these firms are optimizing for two things: a) Better performance. b) Gathering data from you to further improve performance later. I've also found the huge amount of planning rather than iteration frustrating. I've felt like I'm teaching a junior!

epolanski 6 hours ago | parent | next [-]

I think they simply optimize around E2E benchmarks, none of those benchmarks is designed as multi turn assistance to the user, but going from a prompt straight to the final solution.

celrod 21 minutes ago | parent [-]

Exactly. How can "we" develop and encourage benchmarks for multi-turn user assistance? That is what I want. I feel like the models and harnesses push much too hard against this workflow -- that they push you towards letting go and vibe coding, with only your discipline (and desire for a quality and maintainable product) holding it back.

happyPersonR 5 hours ago | parent | prev [-]

more thinking == more tokens === more money LOLL

overfeed 3 hours ago | parent | next [-]

Os there a cost benchmark out there? I wonder how frontier models are doing over time for cost per problem solved.

drob518 2 hours ago | parent | prev [-]

I think they are optimizing for one-shot performance because that will drive usage. They can’t afford to look bad in the benchmarks. And if that means consuming an order of magnitude more tokens, well, that’s good for business, too.

7 hours ago | parent | prev [-]
[deleted]
thinkingtoilet 7 hours ago | parent | prev [-]

I've been having success with Opus but you REALLY have to tame it. Long prompts that list what files to look at, relationships between entities, etc... I went from regularly hitting my daily limit to almost never hitting it. Oh, and also I was being lazy with small changes and stopping that helped a lot too. As you said, it gets in these loops where it's just churning and if you don't stop it it can go on for way too long.