Introduction
What is Paddler? Why Paddler? Before you begin Installation Changelog
Starting out
Set up a basic LLM cluster Using the web admin panel Generating tokens and embeddings Inference parameters Using function calling Using grammars Using multimodal models Multi-agent fleet Going beyond a single device Running the balancer with state database Managing Paddler through API Reporting metrics Model swapping
Paddler in your project
For ops teams For product teams
Best practices
How to control response quality How to handle the load
Migrating to Paddler
OpenAI compatibility
Desktop app (beta feature)
Introduction Installation Using the desktop app
Internals
How Paddler works Buffered requests How to enable embeddings CORS configuration Chat templates

How to enable embeddings

You can enable embeddings either in the web admin panel or through API.

Enabling embeddings in the web admin panel

In the "Model" section, check the enable_embeddings checkbox at the bottom of the inference parameters list. You can also select the pooling type from the dropdown. Next, apply your changes.

Testing the request to generate tokens
Testing the request to generate tokens

Enabling embeddings through API

enable_embeddings should be set to true in the PUT request to change the balancer's desired state.

Pooling type can also be specified in that request.

Mixing embedding and generation requests

An agent processes either token generation requests or embedding requests at a time, but not both. If an embedding request reaches an agent currently generating tokens, it is rejected.

In a multi-agent fleet, the balancer routes embedding requests to an idle agent. With a single agent serving mixed (tokens and embeddings) traffic, you may expect occasional rejections when an embedding request overlaps with active generation; adding more agents resolves this.

Buffered requests CORS configuration