Monitoring

Web dashboard

The router serves a built-in browser dashboard — no install, no extra service. It ships inside router.py, so it’s available the moment the router is running. Just open the router in a browser:

http://localhost:8319/          # redirects to the dashboard
http://localhost:8319/dashboard

On first load it asks for your proxy API key (one of PROXY_API_KEYS — see below if you don’t have one yet) and remembers it in the browser’s local storage. It’s a full control panel, not just a read-only view — a left sidebar splits it into pages, refreshed every 5 seconds:

• Overview — a plain-language status card, the endpoint/model to point your app at, a setup checklist, and summary stats (requests, tokens, spend, cache hit-rate, error rate)
• Providers — health cards (worst-first) plus a detailed table (rating, latency, keys, breaker state, cost)
• Instances — track other Hermes Router base URLs, or create Docker-managed routers on new host ports for agents, tests, or isolated workloads
• Provider Keys — add a key for any provider, set the key-rotation mode, and see live per-key request counts and daily budget usage
• Access Keys — mint new PROXY_API_KEYS for teammates/other apps, with optional rate/ budget limits, and revoke them — see below
• Models — override a provider’s model(s), and a capability table (rating, tool support, reasoning) for every configured model
• Add-ons — toggle optional features on/off, plus live cache stats
• Request Log — the last requests (endpoint, provider, model, latency, complexity score, cascade count, tokens, status), filterable by status and endpoint

Every write (add a key, change a model, toggle an add-on, mint/revoke an access key) shows a “Restart Required” banner — click it to restart the router in place; the page reconnects automatically once it’s back.

It’s pure HTML/JS (no framework, no external CDN) and reads/writes only the router’s own /v1/* endpoints — so it adds essentially no memory or CPU to the router itself.

Accessing it remotely. By default the router binds to 0.0.0.0 (all interfaces). If you set HOST=127.0.0.1 (localhost-only, recommended on a shared/VPS host), reach the dashboard over an SSH tunnel: ssh -L 8319:127.0.0.1:8319 user@server, then open http://localhost:8319/ locally. With Docker the mapped port (-p 8319:8319) exposes it to your host automatically. The raw API endpoints stay key-protected either way.

From VS Code, the extension’s dashboard panel has a ⬈ Web dashboard button (and a globe icon in the panel header) that opens this page in your browser.

Instances: run and monitor more than one router

The dashboard’s Instances page turns one Hermes Router into a small control plane for other Hermes Router processes. This is useful when you want separate routers for agents, teams, experiments, or isolated provider-key pools. Each instance has its own base URL and proxy key, so an agent only needs the usual OpenAI-compatible settings:

client = OpenAI(base_url="http://localhost:8320/v1", api_key="sk-router-agent-a")

There are two modes:

• Connect existing — register a router that is already running somewhere. Enter a name, its OpenAI base URL (for example http://localhost:8320/v1), and optionally its proxy key. Hermes checks /health, and when a key is provided it also verifies authenticated /v1/models.
• Launch Docker — define a managed Docker container. Pick a host port, such as 8320, and the dashboard fills http://localhost:8320/v1 automatically. Hermes creates a container from HERMES_INSTANCE_IMAGE (default hermes-router:latest), maps the host port to the container port, generates a proxy key when you leave one blank, and then lets you start, stop, restart, or delete the container from the Instances table.

The Use existing router keys picker appears under Docker settings. It shows provider names and counts only, never raw key values. When you select a provider such as gemini, the manager copies the current router’s existing Gemini provider keys into the new container as GEMINI_API_KEYS. This lets a new instance come up with the same provider pool without exposing secrets in the browser or API responses.

Manual provider env vars are still available in the same advanced Docker settings box. Use them when an instance needs a different provider pool than the manager router.

Instance definitions are stored in instances.json, next to .env and auth.json by default. It can contain generated proxy keys and copied provider keys, so it is git-ignored and written with owner-only permissions (0600) where the OS allows that. See Configuration → Instance manager settings for the file path and Docker defaults.

The same feature is available over HTTP for scripts or orchestrators:

# Set this to one of your PROXY_API_KEYS values.
export HERMES_ROUTER_KEY='replace-with-your-router-key'

# list instances (secrets are masked)
curl -H "Authorization: Bearer $HERMES_ROUTER_KEY" http://localhost:8319/v1/instances

# register an already-running router
curl -X POST http://localhost:8319/v1/instances \
  -H "Authorization: Bearer $HERMES_ROUTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name":"agent-a","mode":"external","base_url":"http://localhost:8320/v1","api_key":"sk-router-agent-a"}'

# define and start a Docker instance, copying the manager's Gemini/OpenAI provider keys into it
curl -X POST http://localhost:8319/v1/instances \
  -H "Authorization: Bearer $HERMES_ROUTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name":"agent-b","mode":"docker","host_port":8321,"copy_provider_keys":["gemini","openai"],"start":true}'

Proxy API keys & the dashboard key

PROXY_API_KEYS is the credential your app uses to call the router and the key that unlocks the web dashboard — there’s only one tier, no separate “admin” vs. “chat” key. If you never set one, the router generates a real random key on first boot and saves it to .env, logging it once so you can copy it (this also replaces the placeholder value .env.example ships with, so copying that file verbatim doesn’t leave every install on the same public default). Once you have one real key, it’s left alone — nothing regenerates it out from under you.

Adding more keys — for a teammate, a CI pipeline, or another app — is done from the dashboard’s Access Keys page: give it an optional name and optional limits (requests/min, requests/day, tokens/day, cost/day — blank means unlimited), and it generates a new key. The full key is shown exactly once, in a copy box — after that, only its last 6 characters are ever displayed again, matching every other key in this project. A key needs a restart (the dashboard prompts for one) before it can actually authenticate.

Existing access keys can have their name/limits edited in place, or be revoked — revoking removes it from PROXY_API_KEYS so it can no longer authenticate. You can’t revoke the last remaining key; that would lock out the dashboard itself.

Terminal dashboard (hr status)

hr status prints a live, per-provider dashboard — rating, health (circuit-breaker state), key pool, latency, and cache stats — without needing curl or an API key:

hr status
hr status --json   # raw JSON for scripts

Prometheus metrics (/metrics)

A Prometheus-compatible endpoint is exposed at /metrics. It contains operational metrics, provider names, and the last six characters of proxy keys, but no request content or full keys. It is unauthenticated by default, like /health; set METRICS_REQUIRE_AUTH=1 when those identifiers should not be public.

curl http://localhost:8319/metrics

Point Prometheus/Grafana at it to track per-provider traffic and the cache over time.

Exposed metrics

Metric	Type	Labels	Meaning
hermes_router_uptime_seconds	gauge	—	Seconds since the router started
hermes_router_providers	gauge	—	Number of configured providers
hermes_router_requests_total	counter	provider	Total requests routed per provider
hermes_router_errors_total	counter	provider	Total errored requests per provider
hermes_router_avg_latency_ms	gauge	provider	Mean successful-request latency (ms)
hermes_router_circuit_breaker_open	gauge	provider	1 if the breaker is open, else 0
hermes_router_cache_hits_total	counter	—	Response-cache hits
hermes_router_cache_misses_total	counter	—	Response-cache misses
hermes_router_cache_size	gauge	—	Entries currently in the response cache
hermes_router_semantic_cache_hits_total	counter	—	Semantic-cache hits
hermes_router_tokens_total	counter	provider	Tokens served per provider (non-streaming)
hermes_router_cost_usd_total	counter	provider	Estimated USD cost served per provider
hermes_router_key_requests_total	counter	key	Requests per proxy key (key tail)

Alerting (Prometheus + Alertmanager)

/metrics is already native Prometheus format, so you can wire up real alerts — “notify me when spend spikes” or “tell me when a provider goes down” — with just config, no extra gateway or proxy in front of the router.

Don’t stack another LLM gateway in front of Hermes just for this. If you’re running something like LiteLLM between your app and Hermes purely to get usage/cost visibility, and Hermes only sees one key pointed at that gateway, you’re losing Hermes’s whole reason to exist: rotating across many real provider keys with per-provider rating and failover. Give Hermes your real provider keys directly (hr auth add <provider>) and point Prometheus at Hermes’s own /metrics — no middleman needed.

1. Scrape it — add Hermes as a target in prometheus.yml:

scrape_configs:
  - job_name: hermes-router
    scrape_interval: 30s
    static_configs:
      - targets: ["localhost:8319"]
    metrics_path: /metrics

If you’ve set METRICS_REQUIRE_AUTH=1, add your proxy key as a bearer token:

    authorization:
      credentials: YOUR_ROUTER_KEY

2. Alert on it — an example rules file covering the failure modes that actually matter (spend, dead providers, error spikes, latency), using the metrics from the table above:

groups:
  - name: hermes-router
    rules:
      - alert: HermesRouterProviderDown
        expr: hermes_router_circuit_breaker_open == 1
        for: 5m
        labels: { severity: warning }
        annotations:
          summary: "{{ $labels.provider }}'s circuit breaker has been open for 5+ minutes"

      - alert: HermesRouterHighErrorRate
        expr: |
          rate(hermes_router_errors_total[5m])
            / clamp_min(rate(hermes_router_requests_total[5m]), 1e-9) > 0.5
        for: 5m
        labels: { severity: warning }
        annotations:
          summary: "{{ $labels.provider }} error rate above 50% over 5m"

      - alert: HermesRouterDailySpendHigh
        # cost_usd_total is cumulative since the last restart, not a daily counter —
        # increase() over 24h approximates "spend today" (skews only right after a restart).
        expr: increase(hermes_router_cost_usd_total[24h]) > 5
        labels: { severity: warning }
        annotations:
          summary: "{{ $labels.provider }} has cost an estimated ${{ $value }} in the last 24h"

      - alert: HermesRouterSlow
        expr: hermes_router_avg_latency_ms > 5000
        for: 10m
        labels: { severity: info }
        annotations:
          summary: "{{ $labels.provider }} average latency above 5s for 10m"

      - alert: HermesRouterUnreachable
        expr: up{job="hermes-router"} == 0
        for: 2m
        labels: { severity: critical }
        annotations:
          summary: "Prometheus can't scrape hermes-router — it may be down"

Adjust the thresholds (> 5 for daily spend, 0.5 for error rate, etc.) to your own budget and tolerance — these are starting points, not fixed rules. Wire the resulting alerts into whatever Alertmanager already sends to (Slack, PagerDuty, email, …); nothing else on the Hermes side needs to change.

For per-key budget enforcement (as opposed to alerting after the fact), see Configuration → Per-key budgets & rate limits — hr limit set <key> --cost-day 5 rejects a caller’s requests with 429 before they’re ever sent to a provider, once its daily spend crosses the limit.

Usage analytics (/v1/usage)

GET /v1/usage (proxy key required) returns a JSON summary for dashboards and billing:

• per provider — requests, errors, tokens served, and estimated cost ({"usd": …}, plus a converted currency when COST_FX_RATE is set)
• per key — request, token, and cost totals (lifetime + today, plus the live RPM window); keys are shown by their last 6 chars only, never in full
• cache — hits, misses, hit-rate, semantic hits
• totals — total tokens, total estimated cost, and uptime

Cost is estimated from a manually maintained price table; entries marked free/subscription are $0, and unknown or changed upstream pricing may not be reflected. See Configuration → Cost awareness.

curl -H "Authorization: Bearer $HERMES_ROUTER_KEY" http://localhost:8319/v1/usage

JSON status (/v1/status)

GET /v1/status (proxy key required) returns the full picture as JSON: per-provider key cooldown state, rating, model, latency, supports_tools, reasoning, tokens served, circuit-breaker status, plus cache (incl. semantic + a persistent flag), routing, and per-key limit/usage config. This is what hr status renders.

Each entry in a provider’s keys array also reports requests — how many times that specific provider key (last 6 chars only) has been handed out since the router started. This is the direct evidence that round-robin is actually spreading load evenly: add more keys to a provider and each one’s requests count should climb roughly in step with the others. The built-in web dashboard (/dashboard) shows this as a tooltip on each key’s status dot.

The rotation block reports the active key-rotation mode ({"rotation": {"mode": "round-robin"}}); the limits block reports per-key budgets and live usage; hr status shows both in the footer. See configuration.md for details.

Request log (/v1/logs)

GET /v1/logs (proxy key required) returns the most recent requests from an in-memory ring buffer — the data source behind the web dashboard’s live log. The log never writes to disk: the last REQUEST_LOG_SIZE entries (default 500) are kept in RAM and the oldest fall off as new ones arrive (~250 KB at the default size). Set REQUEST_LOG_SIZE=0 to disable it entirely.

Each entry records: timestamp, endpoint (chat/messages/embeddings), caller (key tail), streaming flag, complexity score (1–5), estimated tokens, chosen provider + model, latency, cascade count, status (success/error/cache_hit), and prompt/completion token counts. The request log stores metadata only. The separate response cache does retain request/response data in memory and, with CACHE_PERSIST=1, in SQLite; see Configuration.

Query parameters (all optional): limit (default 100), provider, status (success/error/cache_hit), and endpoint (chat/messages/embeddings).

curl -H "Authorization: Bearer $HERMES_ROUTER_KEY" \
  "http://localhost:8319/v1/logs?limit=20&status=error"

---

Next: How it works — the full request pipeline and every moving part, under the hood.