Retrieved article excerpt
Open article · Retrieved 2026-09-17T18:22:35.581217+00:00
# Coding Agent Observability
[agento11y capturing a coding agent session](https://github.com/grafana/agento11y/blob/main/.github/img/agento11y.gif)
Monitor the coding agents you already use — Cursor, Claude Code, Codex, Copilot CLI, OpenCode, Pi, Vibe, and others. Observe usage, cost, tokens, and tools across all of them in one place. Keep sessions on your machine with the local Agent Observability app, or send them to [Grafana Agent Observability](https://grafana.com/docs/grafana-cloud/machine-learning/agent-observability/).
## Quick start
1. [Install](https://github.com/grafana/agento11y/tree/main/plugins/agento11y#install) `agento11y`.
2. [Configure](https://github.com/grafana/agento11y/tree/main/plugins/agento11y#configure) with `agento11y login` (use local Agent Observability app or Grafana Cloud Agent Observability).
3. [Launch a coding agent](https://github.com/grafana/agento11y/tree/main/plugins/agento11y#launch-a-coding-agent) with `agento11y <agent>` (for example `agento11y claude`).
4. If something looks wrong, run [`agento11y doctor`](https://github.com/grafana/agento11y/tree/main/plugins/agento11y#troubleshooting).
Or hand setup to a coding agent already in your terminal — see [Skills](https://github.com/grafana/agento11y/tree/main/plugins/agento11y#skills).
## Install
**Quick install (Linux/macOS):**
```
curl -fsSL https://raw.githubusercontent.com/grafana/agento11y/main/plugins/agento11y/scripts/install.sh | sh
```
Installs to `~/.local/bin`. Put that directory on your `PATH` if it is not already.
**Homebrew (macOS):**
```
brew install grafana/grafana/agento11y
```
**mise:**
```
mise use -g aqua:grafana/agento11y@latest
```
Make sure [mise is activated in your shell](https://mise.jdx.dev/cli/activate.html) so `agento11y` is on your `PATH`.
**Go install (Windows, or any platform with Go 1.25+):**
```
go install github.com/grafana/agento11y/plugins/agento11y/cmd/agento11y@latest
```
Installs to `$(go env GOPATH)/bin` (or `GOBIN`). Put that directory on your `PATH`.
**Windows (prebuilt binary):** download the `windows_amd64` or `windows_arm64` zip from the [releases page](https://github.com/grafana/agento11y/releases), extract `agento11y.exe`, and put it on your `PATH`.
Verify with `agento11y --version`.
> **Note:** The command was renamed from `sigil`; the old name still works but will be removed.
## Configure
Run `agento11y login` to configure capture. On first run it asks where sessions go: **Local only**, or **Grafana Cloud**. **Local only** needs no Cloud credentials and does not forward sessions. **Grafana Cloud** prints your stack's coding-agent setup page and asks you to paste the connection block from that page.
Interactive Cloud setup on macOS and Linux also asks for **Local web UI** in the preferences. The question starts on **Yes** by default and remembers an earlier answer. **Yes** keeps a full local copy and forwards a copy to Grafana Cloud. **No** sends directly to Grafana Cloud without a local copy. Windows has no local receiver, so it does not show either local choice. Complete credential flags also skip the preferences and keep direct Cloud behavior.
Run `agento11y login` again to change the Cloud connection, local web UI, content capture, tags, or guard settings. With a complete saved Cloud connection, a rerun shows its Grafana stack first. **Keep this connection** jumps to preferences without changing the connection. **Change connection** opens the stack and credential questions. A rerun asks where sessions go only when neither a destination nor credentials are saved.
For scripts and unattended rollout (register an agent without prompts), see [Noninteractive agent setup](https://github.com/grafana/agento11y/tree/main/plugins/agento11y#noninteractive-agent-setup) and [Fleet reconciliation](https://github.com/grafana/agento11y/tree/main/plugins/agento11y#fleet-reconciliation).
## Launch a coding agent
Run `agento11y <agent>` with your coding agent's command name:
```
agento11y claude
```
| Agent | How to run |
| --- | --- |
| [Claude Code](https://docs.anthropic.com/en/docs/claude-code) | `agento11y claude` |
| [Codex](https://developers.openai.com/codex) | `agento11y codex` |
| [Copilot CLI](https://docs.github.com/en/copilot/github-copilot-in-the-cli/using-github-copilot-in-the-cli) | `agento11y copilot` |
| [Cursor](https://cursor.com) | `agento11y cursor install`, then start Cursor |
| [OpenCode](https://opencode.ai) | `agento11y opencode` |
| [Pi](https://github.com/earendil-works/pi) | `agento11y pi` |
| [Vibe](https://github.com/mistralai/vibe) | `agento11y vibe` |
Cursor has no launcher. Run `agento11y cursor install` once, then start Cursor normally. Remove its hooks with `agento11y cursor uninstall`. See also [`cursor/README.md`](https://github.com/grafana/agento11y/blob/main/plugins/cursor/README.md). Per-agent notes and glue live under [`plugins/`](https://github.com/grafana/agento11y/blob/main/plugins).
## Claude plugin evals
Export `claude plugin eval` results, conversations, traces, and usage without
another runner or judge. Run Claude with `--keep-temp`, then:
```
agento11y claude eval import results.json --trace-root /tmp --include-content
```
Use `--dry-run` to preview and `--expect-tenant <stack-id>` to guard the destination.
`--trace-root` authorizes retained trace files under that directory; omit it for
results-only import. Content requires `--include-content`; without it, trajectory
capture preserves identities, model/tool spans and usage but not message bodies.
The command imports both baseline arms and reuses the saved Cloud/OTLP connection.
See the [example and CI workflow](https://github.com/grafana/agento11y/blob/main/examples/experiments/claude-plugin-evals)
for build instructions, scoring details, and verification with `gcx`. This is a
new command in this checkout, not a promise about older installed releases.
## Skills
The binary carries agent skills: markdown workflows a coding agent reads and follows. They ship inside the binary, so there is nothing to fetch and no second CLI to install. Upgrading `agento11y` upgrades them.
```
agento11y skills list # name and one-line description
agento11y skills show setup-coding-agent # the raw SKILL.md on stdout
```
`get` is accepted as an alias for `show`, matching `gcx agent skills get`.
`setup-coding-agent` walks a coding agent through the whole setup: reading `agento11y doctor --json`, installing the binary, saving credentials, wiring the host agent, verifying one session, and diagnosing a broken pipeline. To hand setup to the agent already open in your terminal, paste this:
```
Run `agento11y skills show setup-coding-agent` and follow it to set up Grafana Agent observability for my coding agent.
```
`agento11y doctor` and `agento11y login` both name that command when they finish.
For help choosing and testing local guard packs, paste this into your coding agent:
```
Run `agento11y skills show setup-local-guards` and follow it to help me choose, configure, and test local guard packs.
```
The skills for instrumenting your own application code are separate and ship with [`gcx`](https://github.com/grafana/gcx) instead: `gcx agent skills install agento11y-instrument`.
## Local mode
`agento11y <agent> --local` records the session to a JSONL store and starts the local Agent Observability app. The command prints the app URL (tries `http://127.0.0.1:8765`, then a higher port if needed).
`AGENTO11Y_LOCAL=true` in the shell or `config.env` enables local mode for every launch and installed hook. **Local only** writes `AGENTO11Y_LOCAL` and `SIGIL_LOCAL` as true, and writes `AGENTO11Y_LOCAL_FORWARD` and `SIGIL_LOCAL_FORWARD` as false. Sessions then stay on the machine. **Local web UI = Yes** writes all four keys as true, so the daemon keeps a full local copy and also forwards to Grafana Cloud. **Local web UI = No** writes all four keys as false and sends directly to Cloud. Use `--no-local` to override a saved local destination for one launcher session without changing those keys.
The daemon always stores full session content locally. It forwards full generation content only when `AGENTO11Y_CONTENT_CAPTURE_MODE=full`. Every other selected capture mode is reduced to `metadata_only` for the forwarded copy. Local mode is available on macOS and Linux only; Windows has no local receiver.
Manage the app with `agento11y local start|open|status|stop|restart`. `agento11y local open` starts the receiver if needed, prints its address, and tries to open the app.
### Local guards
For the six bundled packs, setup steps, blocked and allowed examples, and enforcement limits, refer to [Local guard packs](https://github.com/grafana/agento11y/blob/main/docs/local-guards.md).
You can select packs in **Settings** > **Local** > **Guards**, or ask your coding agent to follow `agento11y skills show setup-local-guards`.
With `AGENTO11Y_GUARDS_ENABLED=true`, each host POSTs preflight and tool-call checks to the daemon. Put `guards.toml` next to `config.env` (`~/.config/agento11y/guards.toml`). A local deny always denies. `AGENTO11Y_GUARDS_FAIL_OPEN` only applies to Cloud relay failures. The daemon skips rules that cannot compile and still evaluates valid rules. An unknown `action_on_fail` is reported and treated as `deny`. An unreadable, unparsable, or empty file allows every call locally.
`reject = true` blocks when the pattern matches. `config.target = "shell_command"` evaluates the decoded command line of a shell tool instead of the JSON-escaped tool-call text. Cloud-only evaluator kinds (`llm_judge`, `heuristic`, `prompt_guard`, `json_schema`) load and never fire locally.
```
[[rules]]
rule_id = "block.reset"
phase = "postflight"
action_on_fail = "deny"
[[rules.evaluators]]
kind = "regex"
config.target = "shell_command"
config.reject = true
config.patterns = ['(?i)\bgit\s+reset\s+--hard\b']
```
`agento11y doctor` reports the file path, compile errors, and how many rules can enforce locally.
#### Test local guards offline
`agento11y guards test` evaluates saved local policy even when host guard requests are disabled. It does not execute the submitted command, contact endpoints, start a daemon, or write files. A local dry run does not predict host enforcement or Cloud decisions.
```
agento11y guards test 'rm -rf ~/.ssh'
agento11y guards test --json --rules ./guards.toml 'git reset --hard'
printf '%s\n' 'echo first' 'echo second' | agento11y guards test --stdin --tool shell --agent pi
```
Flags must precede one quoted command argument. Use `--` before a command starting with a dash. Alternatively, `--stdin` reads one command through EOF, preserving newlines. Blank commands and empty tool names are errors.
Without `--rules`, the command reads `guards.toml` beside the resolved `config.env`, including the legacy `sigil` directory fallback. An explicit path bypasses configuration discovery. A missing default file or a valid file with no enforceable rules returns allow with a notice. A missing explicit file is an error. Disabled rules stay disabled; absent packs stay absent. Unchanged legacy packs receive the daemon's in-memory upgrade without changing the file.
The synthetic request is postflight, with one assistant tool call, ID `guards-test`. `--tool` defaults to `Bash`; another name still receives `{"command": ...}` arguments. Conditional rules see only `--agent`, which defaults to empty. There is no model, tags, agent version, preflight history, or system prompt. This command does not replay arbitrary host payloads.
Plain output starts with `allow (local dry run)`, `deny (local dry run)`, or `error (local dry run)`. It reports the rules path, enforceable count, rule-level results, diagnostics, and any transformed input. Failed evaluations can be warnings or dropped redactions, not just denials. A response rule ID does not identify every transform that ran.
`--json` writes one deterministic document to s