Skip to main content

ag-skill observe

observe is a Typer sub-app with four sub-commands for recording and displaying local telemetry after a skill or agent run. All data is stored in .ag-skill/telemetry/runs.jsonl inside the repository — no external service required.

:::info What observe does NOT do ag-skill observe only reads and displays previously recorded entries. It does not execute agents. Run your agent first, then record afterwards. :::


ag-skill observe​

Display the most recent telemetry records as a Rich table.

ag-skill observe

Shows the last 20 records by default.

Options​

OptionDefaultDescription
--tail N / -n N20Number of records to show. Use 0 to show all.

Examples​

ag-skill observe
ag-skill observe --tail 50
ag-skill observe -n 0 # show everything

Expected output​

ag-skill Telemetry (last 3 of 3 records)
┏━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━━┳━━━━━━━━┳ ...
┃ Timestamp ┃ Skill ┃ Agent ┃ Model ┃ ...
┡━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━━╇━━━━━━━━╇ ...
│ 2026-07-29T11:30:39Z │ ddd-aggregate-design │ ag-coder │ gpt-5 │ ...
└──────────────────────┴───────────────────────┴──────────┴────────┴ ...

When no data exists yet:

No telemetry data found. Run ag-skill observe record to add entries.

ag-skill observe record​

Append one telemetry record to .ag-skill/telemetry/runs.jsonl. The timestamp is always auto-generated as UTC ISO 8601.

You do not need to count tokens by hand. Supply the prompt/response text and ag-skill estimates tokens automatically (1 token ≈ 4 characters). Text can be captured without copy-paste via:

  1. Piped stdin (--response-stdin) — pipe the agent's output directly.
  2. A file (--prompt-file / --response-file) — point at output already on disk.
  3. Inline text (--prompt-text / --response-text) — for short strings.

Resolution order per side is stdin → file → inline text. An explicit --prompt-tokens / --completion-tokens always overrides any estimate. --response-stdin and --prompt-stdin are mutually exclusive (stdin can only be read once).

ag-skill observe record --skill <name> [OPTIONS]

Options​

OptionRequiredDefaultDescription
--skill TEXTYes—Skill name that was executed
--agent TEXTNo""Agent name (e.g. ag-architect)
--model TEXTNo""Model used (e.g. gpt-4o)
--response-stdinNooffRead the response from piped stdin — tokens estimated
--prompt-stdinNooffRead the prompt from piped stdin
--prompt-file PATHNo""Read prompt text from a file
--response-file PATHNo""Read response text from a file
--prompt-text TEXTNo""Inline prompt text
--response-text TEXTNo""Inline response text
--prompt-tokens INTNo0Exact prompt token count (overrides estimate)
--completion-tokens INTNo0Exact completion token count (overrides estimate)
--duration-ms INTNo0Execution duration in milliseconds
--status TEXTNosuccessOutcome — success, failure, error, or custom

Examples​

Simplest — just record that the skill completed successfully (tokens default to 0):

ag-skill observe record --skill ddd-aggregate-design

Recommended — pipe the agent's output directly in (no pasting, tokens auto-estimated):

my-agent "design DDD aggregates for the order service" |
ag-skill observe record --skill ddd-aggregate-design --agent ag-architect --model gpt-5 --response-stdin --duration-ms 8200

Read the response from a file the tool wrote:

ag-skill observe record `
--skill ddd-aggregate-design `
--agent ag-architect `
--model gpt-5 `
--response-file .\out\answer.md `
--duration-ms 8200

Inline text (short prompts):

ag-skill observe record `
--skill ddd-aggregate-design `
--prompt-text "Design DDD aggregates for the order service..." `
--response-text "Here are the aggregates: Order, OrderItem, Customer..." `
--duration-ms 8200

Exact counts when your platform's API reports usage:

ag-skill observe record `
--skill ddd-aggregate-design `
--agent ag-architect `
--model gpt-4o `
--prompt-tokens 3200 `
--completion-tokens 1800 `
--duration-ms 12400

Record a failure:

ag-skill observe record --skill dotnet-clean-architecture --status failure --duration-ms 4200

Expected output (with auto-estimation)​

✓ Recorded ddd-aggregate-design / ag-architect (gpt-5)
24 tokens 8,200 ms [success]
(completion tokens auto-estimated)

ag-skill observe estimate-tokens​

Check the token estimate for any text string before recording a run. Useful to sanity-check lengths.

ag-skill observe estimate-tokens "<text>"

Example​

ag-skill observe estimate-tokens "Design DDD aggregates for the order service..."
# → 11 estimated tokens (47 characters)

ag-skill observe from-copilot​

Record a telemetry entry directly from your latest VS Code Copilot Chat session — no pasting or piping required.

ag-skill locates the most-recently modified transcript file under:

%APPDATA%\Code\User\workspaceStorage\*\GitHub.copilot-chat\transcripts\*.jsonl (Windows)
~/Library/Application Support/Code/User/workspaceStorage/... (macOS)
~/.config/Code/User/workspaceStorage/... (Linux)

It extracts the last user message and last non-empty assistant response, then estimates tokens. Copilot Chat transcripts do not include provider token counts, so tokens are estimated at 1 token ≈ 4 characters (~10% accuracy).

Omit --skill to preview what would be recorded without writing anything.

ag-skill observe from-copilot [OPTIONS]

Options​

OptionRequiredDefaultDescription
--skill TEXTNo""Skill name — required to record; omit for dry run
--agent TEXTNo""Agent name (e.g. ag-architect)
--model TEXTNo""Model used (e.g. gpt-4o)
--duration-ms INTNo0Execution duration in milliseconds
--status TEXTNosuccessOutcome — success, failure, error, or custom
--file / -f PATHNoautoPath to a specific transcript .jsonl (overrides auto-detection)
--dry-runNooffPreview without writing

Examples​

# Preview (no --skill)
ag-skill observe from-copilot

# Record after finishing a Copilot Chat session
ag-skill observe from-copilot --skill ddd-aggregate-design --agent ag-architect --model gpt-4o --duration-ms 12400

# Point at a specific transcript file
ag-skill observe from-copilot --skill react-frontend --file "C:\Users\you\AppData\Roaming\Code\User\workspaceStorage\abc123\GitHub.copilot-chat\transcripts\session.jsonl"

Expected output​

Source: C:\Users\...\transcripts\4bb78274-...jsonl
Turns: 10
Prompt: 'yes build per tool and update the documentation '
Response: 'All four commands show up. Let me do a quick smoke test…'
Tokens: 12 prompt + 22 completion = 34 (estimated — Copilot Chat has no usage data)

✓ Recorded ddd-aggregate-design / ag-architect (gpt-4o) 34 tokens 12,400 ms [success]

ag-skill observe from-claude​

Record a telemetry entry from your latest Claude Code session — with exact token counts including cache hits.

ag-skill looks for the session under:

~\.claude\projects\<encoded-cwd>\*.jsonl

The project folder name is derived from your current working directory by replacing : and \ with - (e.g. d:\Projects\ag-skill → d--Projects-ag-skill). If no matching folder exists, ag-skill falls back to the most recently modified session across all Claude Code projects.

Claude Code sessions include a usage block on every assistant message with exact token counts — including cache_creation_input_tokens and cache_read_input_tokens. ag-skill sums all of these across the session, so reported counts reflect what you actually paid for.

Omit --skill to preview without recording.

ag-skill observe from-claude [OPTIONS]

Options​

OptionRequiredDefaultDescription
--skill TEXTNo""Skill name — required to record; omit for dry run
--agent TEXTNo""Agent name (e.g. ag-coder)
--model TEXTNo""Model used (overrides value detected from session)
--duration-ms INTNo0Execution duration in milliseconds
--status TEXTNosuccessOutcome — success, failure, error, or custom
--file / -f PATHNoautoPath to a specific session .jsonl (overrides auto-detection)
--dry-runNooffPreview without writing

Examples​

# Preview
ag-skill observe from-claude

# Record after a Claude Code session
ag-skill observe from-claude --skill ddd-aggregate-design --duration-ms 18500

# Point at a specific session file
ag-skill observe from-claude --skill ddd-aggregate-design --file ~/.claude/projects/d--Projects-myapp/abc123.jsonl

Expected output​

Source: C:\Users\...\.claude\projects\d--Projects-ag-skill\45103e57-...jsonl
Turns: 3
Prompt: 'from this file d:\Projects\...\Program.cs create the Agent b…'
Response: 'Let me first read the file and explore the codebase structure…'
Tokens: 127,004 prompt + 1,113 completion = 128,117 (exact — from Claude usage data)

✓ Recorded ddd-aggregate-design 128,117 tokens 18,500 ms [success]
tip

Token counts from from-claude are session totals — they sum all turns including cache operations. This is the most accurate telemetry ag-skill can provide and requires no manual effort.


Help​

ag-skill observe --help
ag-skill observe record --help
ag-skill observe estimate-tokens --help
ag-skill observe from-copilot --help
ag-skill observe from-claude --help