Remix.run Logo
dudisubekti 5 days ago

But harness is relatively easy to code yourself?

They're just system prompt composer, with some tool functions that the LLM can invoke. I've vibe coded my own in just one day.

SOLAR_FIELDS 5 days ago | parent | next [-]

I don't understand why this is being presented as an either/or thing.

The moat is actually the harness AND the model, and one of the reasons that Claude works so well is because the model is actually trained with its usage in that specific harness in mind, and the harness is designed to deal with Claude model's idiosyncracies. Easy to validate, just run Claude through some other harness and compare, then just run some other model through Claude's harness and compare

Paradigma11 5 days ago | parent | prev [-]

But is there anything preventing them from putting their own proprietary wolfram alpha/prolog/super duper expert system in there?

dudisubekti 5 days ago | parent | next [-]

I guess... but I think, at its core, a good coding harness usually includes:

- well-crafted system prompt that follows best practices

- good contextual reminder prompts (when an llm got stuck in an infinite loop and times out, forgets how to use tools, or needs recurring best practice reminders, etc)

- well-written ergonomic tools the llm can use (read/write files, read diffs, browse the internet, etc)

I dont think these are anything special. The deepest moat I can think of is, proprietary models can be specifically trained to use their proprietary harnesses, so they are more token-efficient and make less tool call and file editing mistakes.

However in my experience, I'm as comfortable working with my own homemade harness as with Claude Code, so I don't think it's a deep moat...

SwellJoe 5 days ago | parent | prev [-]

Only that it would just slow down the model and make it dumber.

You can't tool and harness a weak model into strength and you probably don't improve top models with boondoggles.