Terra has not been able to do any of the technical tasks I've asked of it correctly. I'm surprised others get use out of it. Anything below Sol high tends to give me mostly unreliable results. I'm using codex as my main harness but maybe it performs better with a different one.

▲

mixedCase 3 hours ago | parent | next [-]

With Pi as a harness I've been using OpenAI models as a worker with an Opus 5 (in Claude Code) planner. I've only had a few issues with Terra High/Medium and absolutely none with Sol Medium+ on a fairly complex Rust project that targets Linux, Mac, Windows and Web, with plenty of nasty FFI, VMs, remotely debugging systems, among some other things within a monorepo.

I think the key is to give them a nice assortment of self-verification tools, an AGENTS.md or reference document that they're encouraged to routinely check, and asking the planner to be thorough with the ACs but give the model some space.

The planner routinely finds issues with the worker's output, but that's what it is for.

▲

ericfr11 31 minutes ago | parent | next [-]

Harness is the key. I built my own to "talk" our institutional knowledge and it's working great

▲

sejje an hour ago | parent | prev [-]

Plan with sol-med, implement with luna-high. Rarely a problem.

	▲	ericfr11 33 minutes ago \| parent [-]
		Same for me, with Claude Opus/Sonnet. All the models are almost equivalent if well steered

▲

Art9681 an hour ago | parent | prev | next [-]

Terra is great. It's wild how different our experiences are.

Install the Superpowers plugin.

Behold.

▲

freakynit 4 hours ago | parent | prev | next [-]

Depends on project complexity. For one of my more complex projects, I exclusively use sol-high ... nothing below that works correctly.

For this however, a comparatively much simpler task, tarra-high works fine.

▲

Foobar8568 4 hours ago | parent [-]

Right now, sol-xhigh is my favorite model. I feel that Opus 5 is dumber than 4.8. Fable is too expensive to do anything (limit of $50, started a prompt at $25, ended up at $75, is bullshit, but at least it's "free credits").

DeepSeek is okay for random API-based stuff, as it's cheap.

Local open models running on a 5090 are hit or miss. I feel that most GGUFs/quants are awful...

▲

ericfr11 35 minutes ago | parent | next [-]

I am still on Opus 4.8, with a custom built harness and it works very well even on multi-repos, across stack, deep changes. I also have a very solid test suite which is helping the coding agent a lot

▲

miohtama 2 hours ago | parent | prev [-]

Opus 5 degrades to word salad.

I wonder if it is because of watermarking.

	▲	SwellJoe 17 minutes ago \| parent [-]
		Opus 5 doesn't really even speak coherent English. I'm not sure what's going on, but it can't explain anything. It still does an excellent job with code and writing tests and code review and creating and completing a plan, and it seems to be able to understand English instructions, but it sure as hell can't explain what it did or how to use the code it wrote. That was true before they announced the watermarking, I'd already started to back off of using Opus as much because I like to understand what the model is doing and have it write documentation I can use to reproduce its results, but maybe watermarking was already in there unannounced.

▲

zeven7 3 hours ago | parent | prev | next [-]

I bounce between Sol high/medium and Luna max. I don't know why you'd use anything between Luna max and Sol medium. Luna is so extremely cheap and cranked up to max it does anything I'd want Terra to do for a fraction of the cost. What is Terra for?

	▲	Juvination 2 hours ago \| parent [-]
		One thing I've really noticed with Luna Max is its speed. I've got a review script setup on a custom Pi extension. Luna finds some issues/some false positives, while Sol finds issues but disregards false positives. The biggest thing is Sol finishes in about half the time.

▲

bob1029 3 hours ago | parent | prev | next [-]

I am seeing essentially deterministic results with Terra running a custom browser automation agent across >100 interaction events.

The harness is everything. If I just threw something like Codex at this and said "good luck" I wouldn't make it beyond 5-10 interactions. I tried that already. Carefully designing the views and tools over the environment is where you can go from 50% to 99.9999%.

	▲	tanishqkanc 3 hours ago \| parent [-]
		Curious about what you found. I agree harness for browser automation is vital - I work on https://libretto.sh

▲

cyanydeez 4 hours ago | parent | prev [-]

the breadth and width of the universe of oneshot challenges are all arbitrary. It's unsurprising different workflows oneshot better than others.

All the more reason to favor local models under your control, as once you find that sweet spot model, no one can change it, upgrade it, align it, take it down or otherwise harm the time investment you made it making it your own.

I can't really believe no one understands, after decades, how valueable a rock solid development environment is.

	▲	Phemist 3 hours ago \| parent [-]
		Exactly! I am not opposed to cloud-based models, but I do only stick to open-weight models because I know I can move my whole stack to local (given enough hardware) and continue development without any of the LLM interaction contracts being broken. I would like to see some development where proof of authenticity certs are generated alongside the actual output of the model. Prove to me (or at least claim to me liable to breach of contract) that this output was generated by FP8 DeepSeek V4 Pro 0813. Not some cheaper quantization of the model.