hermes-router

Why hermes-router?

Free and evaluation tiers are rate-limited. hermes-router sits between your app and a pool of providers (Gemini, OpenRouter, Groq, and more). When one hits its limit, it tries the next configured candidate. A request can still fail when all candidates are exhausted or unavailable.

Reduce interruptions

Key rotation and provider failover keep trying while another configured candidate is available.

Common API compatibility

Supports OpenAI chat, embeddings, and models plus Anthropic Messages, including streaming and tool translation.

Smart, cheap routing

Each request goes to the cheapest model that can handle it; unhealthy providers are skipped.

Run many routers

The dashboard can monitor existing routers or launch Docker-backed instances for agents and teams.

Yours to run

A single self-hosted Python file. Your keys live in your own auth.json — nothing hidden.

Start here

New here? The docs are built to read in order — each page ends with a Next link, so you can follow the path straight through from zero to a running, monitored agent.

1. Getting Started Zero-experience intro and your first message.

2. Core Concepts Plain-language glossary — LLM, token, tool, agent…

3. Routing Features How the router picks a provider for chat, tools, vision, and embeddings.

4. Deployment Run it on Windows, macOS, Linux, Docker, or Hugging Face Spaces.

5. Providers Add provider API keys — access notes, Codex, and local models.

6. Usage Call it from the OpenAI or Anthropic SDK; tools, streaming.

7. Build an Agent Chatbot → memory → tools, copy-paste.

Already comfortable? Jump to How it works for the internals or the Monitoring page to run multiple router instances from the dashboard.

Quick start

curl -fsSL https://raw.githubusercontent.com/Shaf2665/Hermes-router/main/get.sh | bash
hr setup        # add a key, start the router

from openai import OpenAI

client = OpenAI(base_url="http://localhost:8319/v1", api_key="YOUR_ROUTER_KEY")
resp = client.chat.completions.create(
    model="hermes-router",
    messages=[{"role": "user", "content": "Hello!"}],
)
print(resp.choices[0].message.content)

On Windows or hosting in the cloud? See Deployment.