Monitoring

Monitoring

Web dashboard

The router serves a built-in browser dashboard — no install, no extra service. It ships as dashboard.html, served by router.py, so it’s available the moment the router is running. Just open the router in a browser:

http://localhost:8319/          # redirects to the dashboard
http://localhost:8319/dashboard

Sign in using ADMIN_API_KEY from the router’s private environment file. The browser uses an expiring HttpOnly session; it does not retain the administrator key in local storage. Home, Providers, Connect, Activity and Settings organize the existing controls. Polling is limited to visible data and pauses when the tab is hidden; failed panels retain their own error state. The following capabilities remain available within those pages:

• Home — a plain-language status card, the endpoint/model to point your app at, a setup checklist, and summary stats (requests, tokens, spend, cache hit-rate, error rate)
• Providers — health cards (worst-first) plus a detailed table (rating, latency, keys, breaker state, cost)
• Instances — track other Hermes Router base URLs, or create Docker-managed routers on new host ports for agents, tests, or isolated workloads
• Provider Keys — add a key for any provider, set the key-rotation mode, and see live per-key request counts and daily budget usage
• Access Keys — mint new PROXY_API_KEYS for teammates/other apps, with optional rate/ budget limits, and revoke them — see below
• Models — override a provider’s model(s), and a capability table (rating, tool support, reasoning) for every configured model
• Add-ons — toggle optional features on/off, plus live cache stats
• Context length — choose a minimum context and confirm actual model capacities; extension 0.9.10 refreshes its limits automatically. See Context synchronization.
• Request Log — the last requests (endpoint, provider, model, latency, complexity score, cascade count, tokens, status), filterable by status and endpoint

Context settings and application-key changes apply to new requests immediately. For changes that show a Restart Required banner, use the supported restart action or restart the hosting process as described in Setup. Restarting signs browser sessions out and may interrupt in-flight requests.

It’s pure HTML/JS (no framework, no external CDN) and reads/writes only the router’s own /v1/* endpoints — so it adds essentially no memory or CPU to the router itself.

Accessing it remotely. By default the router binds to 0.0.0.0 (all interfaces). If you set HOST=127.0.0.1 (localhost-only, recommended on a shared/VPS host), reach the dashboard over an SSH tunnel: ssh -L 8319:127.0.0.1:8319 user@server, then open http://localhost:8319/ locally. With Docker the mapped port (-p 8319:8319) exposes it to your host automatically. The raw API endpoints stay key-protected either way.

From VS Code, the extension’s dashboard panel has a ⬈ Web dashboard button (and a globe icon in the panel header) that opens this page in your browser.

Instances: run and monitor more than one router

The dashboard’s Settings → Instances section turns one Hermes Router into a small control plane for other Hermes Router processes. This is useful when you want separate routers for agents, teams, experiments, or isolated provider-key pools. Each instance has its own base URL and proxy key, so an agent only needs the usual OpenAI-compatible settings:

client = OpenAI(base_url="http://localhost:8320/v1", api_key="sk-router-agent-a")

There are two modes:

• Connect existing — register a router that is already running somewhere. Enter a name, its OpenAI base URL (for example http://localhost:8320/v1), and optionally its proxy key. Hermes checks /health, and when a key is provided it also verifies authenticated /v1/models.
• Launch Docker — define a managed Docker container. Pick a host port, such as 8320, and the dashboard fills http://localhost:8320/v1 automatically. Hermes creates a container from HERMES_INSTANCE_IMAGE (default hermes-router:latest), maps the host port to the container port, generates a proxy key when you leave one blank, and then lets you start, stop, restart, or delete the container from the Instances table.

The Use existing router keys picker appears under Docker settings. It shows provider names and counts only, never raw key values. When you select a provider such as gemini, the manager copies the current router’s existing Gemini provider keys into the new container as GEMINI_API_KEYS. This lets a new instance come up with the same provider pool without exposing secrets in the browser or API responses.

Manual provider env vars are still available in the same advanced Docker settings box. Use them when an instance needs a different provider pool than the manager router.

Instance definitions are stored in instances.json, next to .env and auth.json by default. It can contain generated proxy keys and copied provider keys, so it is git-ignored and written with owner-only permissions (0600) where the OS allows that. See Configuration → Instance manager settings for the file path and Docker defaults.

The same feature is available over HTTP for scripts or orchestrators:

# Set this to your ADMIN_API_KEY for global monitoring.
export HERMES_ROUTER_KEY='replace-with-your-router-key'

# list instances (secrets are masked)
curl -H "Authorization: Bearer $HERMES_ROUTER_KEY" http://localhost:8319/v1/instances

# register an already-running router
curl -X POST http://localhost:8319/v1/instances \
  -H "Authorization: Bearer $HERMES_ROUTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name":"agent-a","mode":"external","base_url":"http://localhost:8320/v1","api_key":"sk-router-agent-a"}'

# define and start a Docker instance, copying the manager's Gemini/OpenAI provider keys into it
curl -X POST http://localhost:8319/v1/instances \
  -H "Authorization: Bearer $HERMES_ROUTER_KEY" \
  -H "Content-Type: application/json" \
  -d '{"name":"agent-b","mode":"docker","host_port":8321,"copy_provider_keys":["gemini","openai"],"start":true}'

Application keys and administrator access

PROXY_API_KEYS authenticates inference. ADMIN_API_KEY unlocks the dashboard and configuration/global monitoring APIs. Both are generated if absent and saved to the private environment file without printing full credentials in logs. See setup.md for retrieval and migration. An old application key no longer grants administration privileges.

Create named application keys from Connect or Settings. The full key is shown once. Creation, revocation and policy updates take effect immediately for new requests; an admitted stream may finish. Access metadata is persisted in auth.json. Administrator access remains separate even if there are no application keys. Limits and reservations survive restart in usage.db; provider-reported totals and conservative estimates are distinguishable.

Terminal dashboard (hr status)

hr status prints a live, per-provider dashboard — rating, health (circuit-breaker state), key pool, latency, and cache stats — without needing curl or an API key:

hr status
hr status --json   # raw JSON for scripts

Prometheus metrics (/metrics)

A Prometheus-compatible endpoint is exposed at /metrics. It contains operational metrics, provider names, and the last six characters of proxy keys, but no request content or full keys. It requires the administrator key. The old METRICS_REQUIRE_AUTH toggle is no longer used: global telemetry cannot be made public through that flag.

curl -H "Authorization: Bearer $ADMIN_API_KEY" http://localhost:8319/metrics

Point Prometheus/Grafana at it to track per-provider traffic and the cache over time.

Exposed metrics

Metric	Type	Labels	Meaning
hermes_router_uptime_seconds	gauge	—	Seconds since the router started
hermes_router_providers	gauge	—	Number of configured providers
hermes_router_requests_total	counter	provider	Total requests routed per provider
hermes_router_errors_total	counter	provider	Total errored requests per provider
hermes_router_avg_latency_ms	gauge	provider	Mean successful-request latency (ms)
hermes_router_circuit_breaker_open	gauge	provider	1 if the breaker is open, else 0
hermes_router_cache_hits_total	counter	—	Response-cache hits
hermes_router_cache_misses_total	counter	—	Response-cache misses
hermes_router_cache_size	gauge	—	Entries currently in the response cache
hermes_router_semantic_cache_hits_total	counter	—	Semantic-cache hits
hermes_router_tokens_total	counter	provider	Tokens served per provider (non-streaming)
hermes_router_cost_usd_total	counter	provider	Estimated USD cost served per provider
hermes_router_key_requests_total	counter	key	Requests per proxy key (key tail)

Alerting (Prometheus + Alertmanager)

/metrics is already native Prometheus format, so you can wire up real alerts — “notify me when spend spikes” or “tell me when a provider goes down” — with just config, no extra gateway or proxy in front of the router.

Don’t stack another LLM gateway in front of Hermes just for this. If you’re running something like LiteLLM between your app and Hermes purely to get usage/cost visibility, and Hermes only sees one key pointed at that gateway, you’re losing Hermes’s whole reason to exist: rotating across many real provider keys with per-provider rating and failover. Give Hermes your real provider keys directly (hr auth add <provider>) and point Prometheus at Hermes’s own /metrics — no middleman needed.

1. Scrape it — add Hermes as a target in prometheus.yml:

scrape_configs:
  - job_name: hermes-router
    scrape_interval: 30s
    static_configs:
      - targets: ["localhost:8319"]
    metrics_path: /metrics

Add administrator credentials to the scrape job. Prefer a private file readable by Prometheus:

    authorization:
      credentials_file: /etc/prometheus/hermes-admin-key

2. Alert on it — an example rules file covering the failure modes that actually matter (spend, dead providers, error spikes, latency), using the metrics from the table above:

groups:
  - name: hermes-router
    rules:
      - alert: HermesRouterProviderDown
        expr: hermes_router_circuit_breaker_open == 1
        for: 5m
        labels: { severity: warning }
        annotations:
          summary: "{{ $labels.provider }}'s circuit breaker has been open for 5+ minutes"

      - alert: HermesRouterHighErrorRate
        expr: |
          rate(hermes_router_errors_total[5m])
            / clamp_min(rate(hermes_router_requests_total[5m]), 1e-9) > 0.5
        for: 5m
        labels: { severity: warning }
        annotations:
          summary: "{{ $labels.provider }} error rate above 50% over 5m"

      - alert: HermesRouterDailySpendHigh
        # cost_usd_total is cumulative since the last restart, not a daily counter —
        # increase() over 24h approximates "spend today" (skews only right after a restart).
        expr: increase(hermes_router_cost_usd_total[24h]) > 5
        labels: { severity: warning }
        annotations:
          summary: "{{ $labels.provider }} has cost an estimated ${{ $value }} in the last 24h"

      - alert: HermesRouterSlow
        expr: hermes_router_avg_latency_ms > 5000
        for: 10m
        labels: { severity: info }
        annotations:
          summary: "{{ $labels.provider }} average latency above 5s for 10m"

      - alert: HermesRouterUnreachable
        expr: up{job="hermes-router"} == 0
        for: 2m
        labels: { severity: critical }
        annotations:
          summary: "Prometheus can't scrape hermes-router — it may be down"

Adjust the thresholds (> 5 for daily spend, 0.5 for error rate, etc.) to your own budget and tolerance — these are starting points, not fixed rules. Wire the resulting alerts into whatever Alertmanager already sends to (Slack, PagerDuty, email, …); nothing else on the Hermes side needs to change.

For per-key budget enforcement (as opposed to alerting after the fact), see Configuration → Per-key budgets & rate limits — hr limit set <key> --cost-day 5 rejects a caller’s requests with 429 before they’re ever sent to a provider, once its daily spend crosses the limit.

Usage analytics (/v1/usage)

GET /v1/usage returns only the caller’s usage for an application key, or the global summary for administrator authentication:

• per provider — requests, errors, tokens served, and estimated cost ({"usd": …}, plus a converted currency when COST_FX_RATE is set)
• per key — request, token, and cost totals (lifetime + today, plus the live RPM window); keys are shown by their last 6 chars only, never in full
• cache — hits, misses, hit-rate, semantic hits
• totals — total tokens, total estimated cost, and uptime

Cost is estimated from a manually maintained price table; entries marked free/subscription are $0, and unknown or changed upstream pricing may not be reflected. See Configuration → Cost awareness.

curl -H "Authorization: Bearer $HERMES_ROUTER_KEY" http://localhost:8319/v1/usage

JSON status (/v1/status)

GET /v1/status (administrator authentication required) returns the full picture as JSON: per-provider key cooldown state, rating, model, latency, supports_tools, reasoning, tokens served, circuit-breaker status, plus cache (incl. semantic + a persistent flag), routing, and per-key limit/usage config. This is what hr status renders.

Each entry in a provider’s keys array also reports requests — how many times that specific provider key (last 6 chars only) has been handed out since the router started. This is the direct evidence that round-robin is actually spreading load evenly: add more keys to a provider and each one’s requests count should climb roughly in step with the others. The built-in web dashboard (/dashboard) shows this as a tooltip on each key’s status dot.

The rotation block reports the active key-rotation mode ({"rotation": {"mode": "round-robin"}}); the limits block reports per-key budgets and live usage; hr status shows both in the footer. See configuration.md for details.

Request log (/v1/logs)

GET /v1/logs (administrator authentication required) returns the most recent requests from an in-memory ring buffer — the data source behind the web dashboard’s live log. The log never writes to disk: the last REQUEST_LOG_SIZE entries (default 500) are kept in RAM and the oldest fall off as new ones arrive (~250 KB at the default size). Set REQUEST_LOG_SIZE=0 to disable it entirely.

Each entry records: timestamp, endpoint (chat/messages/embeddings), caller (key tail), streaming flag, complexity score (1–5), estimated tokens, chosen provider + model, latency, cascade count, status (success/error/cache_hit), and prompt/completion token counts. The request log stores metadata only. The separate response cache does retain request/response data in memory and, with CACHE_PERSIST=1, in SQLite; see Configuration.

Query parameters (all optional): limit (default 100), provider, status (success/error/cache_hit), and endpoint (chat/messages/embeddings).

curl -H "Authorization: Bearer $HERMES_ROUTER_KEY" \
  "http://localhost:8319/v1/logs?limit=20&status=error"