> ## Documentation Index
> Fetch the complete documentation index at: https://langchain-5e9cc07a-preview-patchr-1773857969-df0cef9.mintlify.site/llms.txt
> Use this file to discover all available pages before exploring further.

# Agent Server

LangSmith Deployment's **Agent Server** offers an API for creating and managing agent-based applications. It is built on the concept of [assistants](/langsmith/assistants), which are agents configured for specific tasks, and includes built-in [persistence](/oss/python/langgraph/persistence#memory-store) and a **task queue**. This versatile API supports a wide range of agentic application use cases, from background processing to real-time interactions.

Use Agent Server to create and manage [assistants](/langsmith/assistants), [threads](/oss/python/langgraph/persistence#threads), [runs](/langsmith/assistants#execution), [cron jobs](/langsmith/cron-jobs), [webhooks](/langsmith/use-webhooks), and more.

<Tip>
  **API reference**<br />
  For detailed information on the API endpoints and data models, refer to the [Agent Server API reference](/langsmith/server-api-ref).
</Tip>

## Application structure

To deploy an Agent Server application, you need to specify the graph(s) you want to deploy, as well as any relevant configuration settings, such as dependencies and environment variables.

Read the [application structure](/langsmith/application-structure) guide to learn how to structure your LangGraph application for deployment.

## Parts of a deployment

When you deploy Agent Server, you are deploying one or more [graphs](#graphs), a database for [persistence](/oss/python/langgraph/persistence), and a task queue.

### Graphs

When you deploy a graph with Agent Server, you are deploying a "blueprint" for an [Assistant](/langsmith/assistants).

An [Assistant](/langsmith/assistants) is a graph paired with specific configuration settings. You can create multiple assistants per graph, each with unique settings to accommodate different use cases
that can be served by the same graph.

Upon deployment, Agent Server will automatically create a default assistant for each graph using the graph's default configuration settings.

<Note>
  We often think of a graph as implementing an [agent](/oss/python/langgraph/workflows-agents), but a graph does not necessarily need to implement an agent. For example, a graph could implement a simple
  chatbot that only supports back-and-forth conversation, without the ability to influence any application control flow. In reality, as applications get more complex, a graph will often implement a more complex flow that may use [multiple agents](/oss/python/langchain/multi-agent) working in tandem.
</Note>

### Persistence and task queue

Agent Server leverages a database for [persistence](/oss/python/langgraph/persistence) and a task queue.

[PostgreSQL](https://www.postgresql.org/) is supported as a database for Agent Server and [Redis](https://redis.io/) as the task queue. Checkpoints can optionally be stored using custom adapters, or in MongoDB using our native implementation. See [Configure checkpointer backend](/langsmith/configure-checkpointer) for details.

If you're deploying using [LangSmith cloud](/langsmith/cloud), these components are managed for you. If you're deploying Agent Server on your [own infrastructure](/langsmith/self-hosted), you'll need to set up and manage these components yourself.

For more information on how these components are set up and managed, review the [hosting options](/langsmith/platform-setup) guide.

### How to deploy

Agent Server can be deployed using different methods depending on your infrastructure:

* [Cloud](/langsmith/deploy-to-cloud): Deploy from GitHub repositories with fully managed infrastructure.
* [Hybrid or self-hosted with control plane](/langsmith/deploy-with-control-plane): Build Docker images and deploy via the UI.
* [Standalone servers](/langsmith/deploy-standalone-server): Deploy Agent Servers directly without the control plane.

<Note>
  Cloud deployments are available on all LangSmith plans. Hybrid and self-hosted options require an Enterprise plan and license key. To acquire a license key, [contact our sales team](https://www.langchain.com/contact-sales).
</Note>

## Runtime architecture

The following description applies to the non-distributed runtime variant of LangSmith Deployment.

### Container architecture

A typical deployment consists of two kinds of long-running containers, both built from the same Docker image (a base image with your project code installed on top):

* **API servers** handle client requests (creating runs, reading thread state, streaming results) but do not execute agent code themselves.
* **Queue workers** are the execution engine. They listen to the durable task queue, execute your graph code, and write checkpoints.

Containers are **stateless** but persistent. At least 1 queue worker must listen to the task queue at any time to ensure no runs are orphaned. The containers can serve many runs over their lifetime.

API servers and queue workers are separate container pools and [scale independently](/langsmith/data-plane#autoscaling).

```mermaid theme={null}
flowchart TB
    User["User"]

    API["API Servers"]

    subgraph WorkerContainer["Worker Containers"]
        QueueLoop["Queue Loop"]
        W1["Worker"]
        W2["Worker"]
        Wn["..."]
        QueueLoop -->|dispatch| W1
        QueueLoop -->|dispatch| W2
    end

    DB[(Postgres)]
    Redis[(Redis)]

    User -->|request| API
    API -->|create run| DB
    API -->|notify| Redis

    Redis -->|wake| QueueLoop
    QueueLoop -->|claim next run| DB

    WorkerContainer -->|save checkpoints / update status| DB
    WorkerContainer -->|publish events| Redis

    Redis -->|stream events| API
    API -->|SSE response| User

    style User fill:#F3F4F6,stroke:#9CA3AF,stroke-width:2px,color:#374151
    style API fill:#F3E8FF,stroke:#9333EA,stroke-width:2px,color:#581C87
    style DB fill:#DBEAFE,stroke:#2563EB,stroke-width:2px,color:#1E3A8A
    style Redis fill:#FEE2E2,stroke:#DC2626,stroke-width:2px,color:#7F1D1D
    style WorkerContainer fill:#DCFCE7,stroke:#16A34A,stroke-width:2px,color:#14532D
    style QueueLoop fill:#FEF3C7,stroke:#F59E0B,stroke-width:2px,color:#78350F
    style W1 fill:#F3F4F6,stroke:#9CA3AF,stroke-width:2px,color:#374151
    style W2 fill:#F3F4F6,stroke:#9CA3AF,stroke-width:2px,color:#374151
    style Wn fill:#F3F4F6,stroke:#9CA3AF,stroke-width:2px,color:#374151
```

### Run execution lifecycle

When you invoke a run, the request flows through several components:

1. A client sends a request to an API server, which creates a pending run in the durable task queue.
2. A queue worker picks up the run, acquires a lease on it, loads the appropriate graph, and begins execution. The queue enforces that at most 1 run can be executed for a given thread at one time.
3. As the graph executes, the worker writes checkpoints to the persistence layer (the frequency depends on the [durability mode](/oss/python/langgraph/durable-execution#durability-modes)) and broadcasts streaming events over the configured pubsub provider.
4. If the client opened a `/stream` connection, the API server subscribes to the pubsub channel and forwards events to the client via server-sent events in real time.
5. When execution completes, the worker updates the run status and releases its slot for the next run.

Each worker handles up to [`N_JOBS_PER_WORKER`](/langsmith/env-var#n_jobs_per_worker) runs concurrently (default: 10), so a single worker container serves many runs in parallel. See [Configure Agent Server for scale](/langsmith/agent-server-scale) for tuning guidance.

### Graph loading and compilation

How and when your graph is compiled depends on how you register it in your [application structure](/langsmith/application-structure):

1. **Compiled graph**: If you export an already-compiled graph (a `CompiledGraph` instance), it is loaded once at container startup and reused for every run. This is the most efficient path.
2. **Factory function**: If you export an agent factory function, the server invokes it each time it needs the graph. Factories receive the run's configuration, enabling per-run graph customization (for example, choosing different models or tools based on the assistant config). Keep factory functions lightweight for best performance.

In both cases, the server automatically injects the checkpointer and memory store configured for that deployment at runtime. **You should not configure these in your graph code**, since the server needs to manage these for other operations.

## Learn more

* [Application Structure](/langsmith/application-structure) guide explains how to structure your application for deployment.
* The [API Reference](https://docs.langchain.com/langsmith/server-api-ref) provides detailed information on the API endpoints and data models.

***

<div className="source-links">
  <Callout icon="edit">
    [Edit this page on GitHub](https://github.com/langchain-ai/docs/edit/main/src/langsmith/agent-server.mdx) or [file an issue](https://github.com/langchain-ai/docs/issues/new/choose).
  </Callout>

  <Callout icon="terminal-2">
    [Connect these docs](/use-these-docs) to Claude, VSCode, and more via MCP for real-time answers.
  </Callout>
</div>