# Conduit

> Secure, self-hosted observability for AI agents: traces, logs and metrics over mTLS, stored in ClickHouse and VictoriaMetrics, with a server-rendered UI. Go services, Go and Python SDKs.

Conduit collects traces from AI agents. Each trace is a run broken into spans for planning, tool calls and LLM calls, with status, errors, token counts and cost. Conduit stores the traces and shows them in a fast, server-rendered UI. It's designed for teams that want agent telemetry inside their own infrastructure, with a small and auditable attack surface.

## Highlights

- **Identity you can trust.** Agents authenticate with mTLS client certificates, and the certificate's CommonName is the agent's identity. An agent can't spoof it with an attribute.
- **Hardened ingestion.** Every span is validated before it reaches storage. Rate limits apply per agent and per telemetry item, so batching can't get around them.
- **LLM-aware traces.** Model, tokens, cost and errors are recorded per span and rolled up per trace. Prompts and completions are stored separately from span metadata.
- **A UI that's quick to read.** One row per trace, filtering by time, agent and errors, stable paging, and a span tree with a waterfall timeline.
- **Small and auditable.** Each service is a static Go binary in a `FROM scratch` image. The UI uses no JavaScript framework, has a strict Content-Security-Policy, and embeds its own assets.
- **Production basics included.** UI sign-in with a login form or Google Cloud IAP, health and readiness endpoints, Kubernetes manifests, and a CI pipeline that runs the whole stack end to end.

## How it works

```mermaid
flowchart LR
    A["Agent<br/>Go or Python SDK"] -- "gRPC + mTLS :4317" --> G["Gateway<br/>auth · validation · rate limits"]
    G -- "JetStream (work queue)" --> N[("NATS")]
    N --> C["Collector"]
    C --> CH[("ClickHouse<br/>spans, LLM data")]
    C -. "best effort" .-> VM[("VictoriaMetrics<br/>span durations")]
    C -. "OTLP/HTTP, best effort" .-> T[("Tempo<br/>traces")]
    CH --> U["UI<br/>login or GCP IAP"]
```

The gateway accepts spans only from agents holding a certificate signed by your CA. The collector writes each span to ClickHouse, the system of record, and redelivers it if that write fails. It also copies spans to VictoriaMetrics and Tempo on a best-effort basis. The UI reads from ClickHouse. See [Architecture](architecture.md) for details.

## Quick start

You need Docker (Compose v2, 2 GiB of memory). One command runs the whole
platform from the signed release images:

```bash
git clone https://github.com/tjscanlan/conduit && cd conduit
make quickstart          # or: ./deployments/quickstart/quickstart.sh
```

It generates a dev CA, agent certificates, and a UI login, waits until
everything is healthy, and prints your login. Open <http://localhost:8080>:
a simulated `demo-agent` is already sending traces, and a Get started page
walks you through sending your own agent's first trace. The banner also shows how
to send a trace from your own code as `my-agent`; CI checks that a cold start
reaches the first trace in under five minutes.

Releases also ship [`conduitctl`](cli.md), the same
quickstart as a single binary with no checkout needed: `conduitctl
quickstart`, plus `agent-cert` and `doctor`.

`quickstart.sh status` shows the login again, `down` stops the stack (keeping
data), and `reset` deletes everything. Next, run one of the
[examples](examples.md) (hello-trace, tool calling, asyncio, Claude,
OpenAI) or instrument your own agent with the [Go SDK](sdk-go.md) or
[Python SDK](sdk-python.md). To work
on Conduit itself (building images from source), see
[Deployment](deployment.md#local-docker-compose-from-source).

## Documentation

**Read the docs at <https://tjscanlan.github.io/conduit/>**, including the [visual roadmap](https://tjscanlan.github.io/conduit/roadmap.html). The same pages are in [`docs/`]().

**For AI agents:**
- [`llms.txt`](https://tjscanlan.github.io/conduit/llms.txt) indexes the docs and links each page's raw Markdown.
- [`llms-full.txt`](https://tjscanlan.github.io/conduit/llms-full.txt) has all the docs in one file.
- [`AGENTS.md`](AGENTS.md) tells coding agents how to build, test and change this repo.

| Guide | What's in it |
|-------|--------------|
| [Architecture](architecture.md) | Components, data flow, delivery guarantees and storage schema |
| [Security](security.md) | The threat model, mTLS identity, rate limiting, UI authentication and browser hardening |
| [Deployment](deployment.md) | Docker Compose, Kubernetes, health checks, upgrades and configuration reference |
| [Go SDK](sdk-go.md) · [Python SDK](sdk-python.md) | Instrumenting agents |
| [Roadmap](roadmap.md) | What's shipped, what's next and the open decisions |
| [Releasing](releasing.md) | Versioning policy, verifying signed images, upgrading and cutting a release |
| [Changelog](CHANGELOG.md) | What changed in each release |
| [Contributing](contributing.md) | Development setup, tests, code generation and pull requests |
| [Brand](brand.md) | Logo, palette and accessibility rules for the UI |

## Project status

Conduit is **pre-1.0**. The core pipeline is tested end to end in CI on every push: SDK → gateway → NATS → collector → ClickHouse, VictoriaMetrics, Tempo → UI. APIs and the storage schema may still change between releases. See the [roadmap](roadmap.md).

## License

[Apache License 2.0](LICENSE)