This year, a lot of big companies are building the agentic layer.

This has to be build and takes time.

But this is the major thing were ai will be introduced into every single layer of software we have.

The HR System which might have needed x people? The agent has solved 90% of tickets already before they hit a human. Agents will do the repetitive but not that easy to automate tasks.

Building the middle system that can take any model as a back end, based on cost/performance/client choices/etc, and API style present the same interface and functionality to end users is huge right now.

Local LLM usage is bouncing across major runtimes, custom runtimes and forks plus custom quants and variations of open models. llama-swap and some other tools integrate those and provide metrics but as the field expands can't keep up even with AI support.

I need to look at tailscales dashboard/etc again. I started on that when it launched but couldn't get my runtimes to go through it. I think the easiest tie in for metrics/etc is grafana + Prometheus but you need that stood up, and then a custom parsing job for each runtime based on metrics it reports and processing that data into useful dash.

I would love to see more standards here like OpenAI endpoints with metrics and information to link into. ./metrics is great but not standardized.

This is the point. hmontazeri is imagining where AI still has left to grow as a consumer product, but AI is currently more of a business product. Even without a new consumer use case, it's easy to see demand for tokens keep going up.

> But this is the major thing were ai will be introduced into every single layer of software we have.

How is that even possible? I thought they already put AI chats into everything. Maybe AI chat inside an AI chat.