For a new project, I’ve been exploring options to develop a backend to query multiple large language models and just came across this great solution.
It’s an open source project called LiteLLM and it provides a unified interface to call 100+ LLMs using the same input and output format, including OpenAI, Anthropic, models on Hugging Face, Azure etc.
There is cost tracking and rate limits. To make things easier, there is even a user interface. What I found most useful is the ease of comparison and benchmarking between LLMs. Kudos to the developer team.
I can see so many business use cases for integrations like this: rapid prototyping and experimentation, performance benchmarking and optimization, cost control…