| ▲ | Control LLM Spend and Access with any-LLM-gateway(blog.mozilla.ai) | ||||||||||||||||
| 43 points by aittalam 7 days ago | 11 comments | |||||||||||||||||
| ▲ | verdverm 3 hours ago | parent | next [-] | ||||||||||||||||
This service (llm proxy to all providers) are a dime-a-dozen This one has very little on monitoring and no reference to OTEL in the docs | |||||||||||||||||
| |||||||||||||||||
| ▲ | bitpush 3 hours ago | parent | prev | next [-] | ||||||||||||||||
I'm conflicted on what Mozilla is doing here. On the one hand, it is nice that they are getting involved but com'on, dont you all have Firefox to work on? This is a classic case of an over enthusiastic engineer who says yes / raises hand to everything, but doesnt do any one thing properly. At some point, you have to sit down and tell them to focus on one thing and do it properly. | |||||||||||||||||
| |||||||||||||||||
| ▲ | ouk 3 hours ago | parent | prev | next [-] | ||||||||||||||||
There is also PydanticAI Gateway (https://ai.pydantic.dev/gateway/). I use it with the PydanticAI framework and it's quite nice. | |||||||||||||||||
| ▲ | smcleod 3 hours ago | parent | prev | next [-] | ||||||||||||||||
Interested to see how this stacks up against Bifrost (fast but many features paywalled) and LiteLLM Proxy (featureful but garbage code quality). Especially if it gets a web admin / reporting frontend and high availability. | |||||||||||||||||
| |||||||||||||||||
| ▲ | bravura 3 hours ago | parent | prev [-] | ||||||||||||||||
Thoughts on any-llm-gateway versus litellm-proxy? litellm is a great library, but one team using litellm-proxy reported having many issues with it to me. I haven't tried it yet. | |||||||||||||||||
| |||||||||||||||||