> Do you then recommend Sol for technical documentation?

I don't know, to be honest. Technical documentation is such a multidimensional skill, and I also don't know your budget. I'd imagine Terra might have the most readable prose, but Astra might be more likely to catch bugs or mistakes to highlight. If the documentation is about a more esoteric language or situation, Astra might have a greater edge than if it's a common language or situation. If you have a long list of documentation guidelines, that might also give more of an edge to Astra. I wouldn't want to make a recommendation without doing a lot of testing, and I haven't done a lot of testing.

▲

flockonus a day ago | parent [-]

Some food for thought, if someone from oAI can't make a direct recommendation for something as direct as "Technical documentation is such a multidimensional skill" - how could we know assess what model to use?

▲

tedsanders a day ago | parent | next [-]

To elaborate, I can ask each model to write a doc about something I know well and tell you which output I like better.

But to assess whether my preference on my problem is going to generalize across other people, other programming languages, other documentation types/formats, etc., and then analyze statistical significance is a thornier question that I don't have the data to answer. I'd rather make no recommendation than a weakly informed recommendation.

▲

mrandish 9 hours ago | parent [-]

> I can ask each model to write a doc about something I know well and tell you which output I like better.

That's what users currently end up having to do on their own. Or they just give up in confusion, which presents in your site analytics as either bounce, buy later or buy less.

Why not have a model comparison site? Visitors choose from a dozen-ish broad categories of work (eg technical documentation, market communications, etc), then pick from a dozen sample prompts across typical tasks in that category, and see the model responses with avg time and approximate cost. Especially helpful would be having a toggle overlay with annotation highlighting relevant differences, then if they have (or create) an account, they can edit the annotations and add their own with a sharing link.

If my job was to actually help my co-workers and friends make better informed choices they feel more confident in, I'd be doing something like that. The extra credit version would help users narrow down to two models and then let them convert to a 'Test Drive' trial - same workflow but from their own prompts, side-by-side A/B response comparisons with annotation and sharing link.

As you said, the answer to "Which model is best for me?" is actually complicated, conditional and inevitably a bit confusing. When that's the reality, the right pattern isn't 'Communicate to Convert', it's 'Build credibility and confidence with maximum clarity and genuine helpfulness'.

▲

tedsanders 9 hours ago | parent [-]

Sounds like arena.ai: https://arena.ai/leaderboard

People vote on pairs of model responses and they aggregate everyone's votes into a leaderboard.

One thing to be aware of that the distribution of prompts on arena.ai can be pretty different than real-world usage. The people who spend time on that website casting votes tend to be more interested in entertainment than professional productivity. And turn-based voting can punish useful behavior like asking clarifying questions instead of guessing at assumptions.

	▲	mrandish 9 hours ago \| parent [-]
		Yes, I've seen Arena.ai and the reasons you cited are why I suggested something better that's more private, personalized and shareable internally without being skewed and gamed.

▲

scarab92 a day ago | parent | prev [-]

I use Astra 6 Extra High for everything.

It’s overkill most of the time, but frankly all these models are “cheap”, and I’d rather waste compute than brain cycles trying to discern which dials are best for each individual as-hoc query.

▲

junnan 18 hours ago | parent | next [-]

But Astra is simply bad at writing. I have to use Argon to rewrite everything Astra wrote in an academic setting. Astra always tries to cover every small detail and engages in defensive writing that is unreadable.

▲

selestify a day ago | parent | prev | next [-]

How are they cheap? Does your organization offer unlimited spend, or does your subscription not run into the same limits as mine?

	▲	hsuduebc2 a day ago \| parent [-]
		Maybe if you have subsidized 200/500 dollars subscription it may look that way.

▲

thirtygeo 20 hours ago | parent | prev | next [-]

Totally agree. I'm working a lot faster when being just about everything I want on the first hop, without model switching and goofing around to find the sweet spot. When I need a janky answer I run a model locally with llama.cpp

▲

noahmichael89 a day ago | parent | prev [-]

[flagged]