Skip to main content

How to Deploy Self-Hosted Agents

In this guide, we will explore how to deploy agents on your own infrastructure using the Agents SDK.

Prerequisites​

Before you begin, make sure you have completed the Getting Started with the Agents SDK tutorial and you are familiar with How to Configure Agents.

Serving the REST API​

The Agents SDK serves a REST API that allows you to interact with the agents. There API contains streaming and non-streaming endpoints for sending a conversation and receiving a response from the agent. To serve the REST API, you need to run the following command:

rag_agents serve "<agents project directory>"

This will start the API server on 127.0.0.1 at port 8000 by default. To override the default host and port, you can use the --host and --port options. You can access the API documentation at http://<host>:<port>/docs.

Deploying the Server​

To deploy the server on your own infrastructure, you can use any cloud provider or hosting service that supports running your own applications.

🚧 Under Construction 🚧

Using your Deployed Agents in the Zeta Alpha Platform​

Once you have deployed your agents, you can use them in the Zeta Alpha Platform by updating your tenant configuration. For example you can add the following to your chat_bot_setups configuration, one entry for each agent:

{
  "bot_identifier": "<agent identifier>",
  "agent_name": "<agent name>",
  "llm_configuration_name": "",
  "serving_url": "https://<host>:<port>"
}

Replace <agent identifier>, <agent name>, <host>, and <port> with the appropriate values for your deployed agents. Note that for self-hosted agents, the llm_client_configuration will be taken from the agent_setups.json file, therefore we leave llm_configuration_name empty.

Using your Deployed Agents outside the Zeta Alpha Platform​

The REST API provided by the Agents SDK allows you to interact with the agents from any application or service. You can send a conversation to the agent and receive a response by making HTTP requests to the API endpoints.

warning

When running in production, it is recommended to secure the API endpoints with authentication and authorization mechanisms to protect your agents and data. You can achieve this by using API gateways, firewalls, and other security measures that your hosting service provides.

Alternatively, you can also extend the API server to add custom endpoints and security features.