Using metaphors is dangerous, but I would dare to say that big tech AI is like cement suppliers. It's too low level of a service. In civil engineering you have the option to contract value added suppliers that will give you prefabricated pieces in concrete or steel you could be using to build your construction.

I'm seeing a lot of AI firms building value added services on top of big tech "foundational" AI offerings. Value addition can start very early at a clear plans/billing structure, going through rate limiting, documentation and extra features that will bring stability or consistency to our AI enhanced products.

Going the other way around (I tried) and building things on top of big tech AI is challenging starting at the fundamentals as the OP described well.

▲

wongarsu 2 days ago | parent [-]

Openrouter is roughly at that level of value-add. With plenty of competition now, since being able to charge 5% on your AI spend just for having sane billing, spending controls (actually enforced per-api-token budget limits!) and easy sign-up is an insanely profitable business proposition

On the other hand I think it's fair to criticize the model hosts for not offering the same

	▲	numlocked 2 days ago \| parent [-]
		(I work at OpenRouter) Certainly for individual developers / hobby projects that's the primary value prop; super easy access to all of the models. But there's a lot more functionality that becomes relevant when building in production. We do automatic fallbacks, route between providers based on data policies, syndicate your data to agent observability tools / your logging platform of choice, user-level and api-key-level budget management and model allow/block lists, programmatic API key management, etc, etc. More good stuff shipping all the time!