Good ideas are usually still good across time and tools

Indeed. What a deep cut

Mixture of Models, perhaps? tbf to OP, the setup they're proposing has also been recently evangelized by other "ai gateway" products (like OpenRouter, JusCode, Fireworks etc), so there's likely something useful here.

timecube?

The evaluator is public here: https://echo.tracerml.ai/eval/

It currently exposes 907 stored rows across seven benchmark families, with prompts, outputs, grades, and cost records. More benchmarks are coming soon.

Echo does not disclose its per-request routing decision because that policy is the product. We can, however, publish some of the eligible open-weight model pool, version dates, aggregate allocation mix, and evaluation settings without exposing the request-level recipe.

New video is also being made.

> No benchmarks, no info on which models are used, […]

The benchmarks are here → https://echo.tracerml.ai/eval/

They are not good benchmarks but at least they exist.

It's basically trying to replicate OpenRouter, which works pretty well and has a lot of nice features to abstract away any single provider, such as failover, metering, autoswitching, etc. It's actually a really smart infrastructure abstraction.

I just wish this were solving an actual problem rather than being a fairly transparent attempt to say something approximating, "Hey VCs, OpenRouter just became a unicorn but I can basically vibe code it"

Calling it "Fable-level" feels intellectually lazy / dishonest, but then again, what do you expect when there's so much money on the table.

Unrelated - but reminds me of my favorite quote by tenderlove:

"microservices turn function calls into distributed computing problems"

It's easy enough to copy and paste in a prompt, no?

Eval tests while giving general indicators might not be similar for each use case.

Fusion is a totally different approach, though similar in the sense that it leverages different models.

Fusion generates many replies then synthesizes. This adds a ton of latency and cost, so it's going to be better only for cases where you're willing to wait a lot and pay a lot more.

Routers (like this project) are a different thing, they can theoretically improve performance and cost at the same time without increasing latency much. I'm a bit skeptical though, since knowing which LLM is going to be better on a cost adjusted basis is hard (see https://artificialanalysis.ai/models/capabilities/coding?cos..., where the cost per task vs. performance is not what you expect, for example comparing Qwen 3.7 Max to GPT Sol.

A project I'm working on is aimed at improving performance without added latency but from a different angle. Instead of waiting for all replies for synthesis (like OpenRouter Fusion), it streams the "best" reply immediately (using a router to pick the best model) then synthesizes with emoji reactions and optional replies from the background models. It's free to use here with no login: http://pellmell.ai

I can get it to write win32 unsafe rust code.

I can't get it to review win32 unsafe rust code.

Make it make sense.

You are right that the privacy wording was too broad. We are fixing it now so it states explicitly that Echo does not use customer prompts, files, chats, or outputs to train or fine-tune models. We are updating the matching Terms language at the same time.

Echo also starts users with free credit and does not require a credit card to try it. The current signup flow did not make that clear enough, so we are fixing that presentation too.

Thanks for calling both out.

If you want to try a similar idea that you can chat with immediately (free tier uses open weight models), I'm working on this: http://pellmell.ai

I assume trying without a credit card will bankrupt him immediately.

Thanks for checking the individual rows. HumanEval+ is one small code slice, not the whole basis for the launch claim. The public evaluator currently contains 907 rows across seven benchmark families, and matched SWE-bench Verified and BigCodeBench runs are the next code evidence being added.

You also found a real UI bug: the inspector should show both stored answers and currently does not in some rows. We are fixing that.

On the row you reran: the page records a frozen matched run. It does not claim that Fable is incapable of solving that prompt on another run. We are adding repeated matched trials and making run count and variance explicit. Your rerun is exactly the kind of external check the row-level page is intended to make possible.

The broader result remains: Echo is competitive with Fable across the evaluated task mix at materially lower measured inference cost. We are filling in the harder agentic-code evidence now rather than asking anyone to infer it from HumanEval+.

You might be interested in a project that I'm working on, which is kind of like OpenRouter Fusion, but instead of waiting for all models to synthesize, we stream the best model immediately and background the rest. The background models then reconcile with an emoji reaction and optional reply. It gets similar results to Fusion and is a lot faster! There's a free version that leverages open weight models here: http://pellmell.ai.

In MoE systems the routing decision is made per-token, not per prompt or task. It’s one of ML’s many confusing naming conventions.

Even more confusingly, there are older pre-LLM MoE systems which ensemble and pool the predictions from multiple sub-components. For example in a random forest you could take the majority vote of the decision trees or the average of their numerical predictions.

After that, we developed neural net architectures for predicting a single thing like whether the user will click on your ad. An MMoE is in the same family.

And so now we are at massive MoE networks for LLMs which have similarities with MMoE in that the “decision” is about the very next token to predict.

Enterprise plans pay API costs, they're only subsidizing individual accounts and that's only because they have to compete with open-source models.

When they are successful at making those illegal/inaccessible, both Anthropic and ChatGPT are going to rip the band-aid off and inference will only be sold to those that can afford it.

The good ole American way.

Because that is a short term solution, it won’t be offered forever. Large organisations have to purchase credits at $/tokens. Eventually everyone else will too.

I think most people assume the subsidized plans will go away or get more limited eventually. They are basically a loss leader and a marketing cost that is very flexible and easy to change w/o directly impacting their primary customers.

You may want to consider the incomes of developers outside the US, students, unemployed. $200/month is a lot to a lot of people.

doesnt exist for enterprise plans

The entirety of "agents" and tool calls is a process of combining LLMs to get better results. Is it the same LLM in many cases? Yes. But it doesn't have to be.

It's the natural move that happened after people realized you couldn't throw away half a century of AI research.

Most of these focus on costs. But it is simply the case that the one-shot output did not scale for harder problems on workflows.

how does this differs from Sakana Fugu?

Looks like it but with open-weights only.