Keep AI on your own servers
What is it?
Paddler is an open-source LLM load balancer and serving platform that lets teams run inference and deploy LLMs on their own infrastructure.
Who made this?
This is an open-source project originally started by Intentee.
Why?
- Cost control
- Data privacy
- Reliability
- Extensive model support (uses llama.cpp for inference)