How to Manage LLM Models

An LLM configuration is a named connection to one model at one vendor. Agents, dictation and image generation refer to it by name, and rules grant it by name (model:<name>). Add one per model you want to offer.

Add a model

1. Open Tenants → <tenant> and click the pencil on the LLM Configurations card.

2. Click Add LLM Configuration and fill in:

   Field	What to enter
   Name	The name everything else uses, for example gpt-5.2. Choose it once: agents and policies break when it changes.
   Vendor	OpenAI, Azure OpenAI, Anthropic (also for AWS Bedrock), Azure Anthropic (Azure AI Foundry), or Ollama
   Vendor credentials	API key, endpoint, API version, or for Azure the Auth Type (API Key, Client Secret or Workload Identity). Pick a stored secret or enter a new value, see Storing secrets. Ollama needs none.
   Model Name	The vendor's model identifier. On Azure it is the deployment name.
   Model Type	Chat for most agents; Responses for OpenAI reasoning models; Realtime Transcription for dictation; Image for image generation

3. Click Save Changes.

Temperature, max tokens, reasoning effort and the other model settings are under Model Configuration and Advanced Model Settings. What each field does is in the Agent Configuration Reference.

Use the model

To use it for	Do this
An agent's answers	Pick it as the agent's LLM Configuration, see Creating and editing agents
A model users can switch to in the chat	Add it to the agent's Allowed extra models
Dictation	Features card → Dictation → Transcription Model (a Realtime Transcription configuration)
Image generation	Agent editor → Capabilities → Policies → Image model, with the agent's Image Generation Tools on

Decide which users get which models

Give the agent every model anyone should use, then narrow per group with rules on model:<name>. Who may use it on the model's card edits those rules in place. A user sees only the allowed models in the model picker, and an agent with none of its models allowed is hidden from them.

Trace model calls

Under the same editor, Tracing Configurations → Add Tracing Configuration connects a Langfuse project (host, public key, secret key, sample rate). Pick it per agent as Tracing Configuration.

Pitfalls

• The last LLM configuration cannot be removed; a tenant always keeps one.
• Removing or renaming a configuration does not update the agents and policies that name it. Change those first.
• A model type must match its use: a Chat model cannot transcribe, and an Image model cannot answer chats.
• Tool search methods lists the methods a model supports (tool_search_methods); server and client are offered only for OpenAI responses and Anthropic chat models. For OpenAI models prefer client: OpenAI's server search is slower. See model_configuration. Unset, a model supports only the emulated method. The agent page warns when none of an agent's methods fits a model, see Tool search.