Introduction
What is Paddler? Why Paddler? Before you begin Installation Changelog
Starting out
Set up a basic LLM cluster Using the web admin panel Generating tokens and embeddings Using function calling Multi-agent fleet Going beyond a single device Running the balancer with state database Managing Paddler through API Reporting metrics Model swapping
Paddler in your project
For ops teams For product teams
Best practices
How to control response quality How to handle the load
Migrating to Paddler
OpenAI compatibility
Internals
How Paddler works Buffered requests How to enable embeddings CORS configuration Chat templates

What is Paddler?

Paddler allows you to deploy and scale open source LLMs (Large Language Models) in your own infrastructure.

The primary goal for Paddler is to provide great developer experience, guide you through the best practices, and make sure that the models you deploy work in the optimal way.

Why Paddler?