Skip to content

Commit ed999ec

Browse files
committed
docs: add metrics documentation and new CLI commands
- Add --status and --metrics-reset to CLI options table - Add comprehensive Metrics section documenting: - Viewing metrics with --status command - Resetting metrics with --metrics-reset command - All tracked metrics fields and descriptions - Cost estimation for different AI model providers - Tips for using metrics effectively - Update Directory Structure to include metrics.json file - Add example output showing metrics display format - Add confirmation prompt example for safe reset Signed-off-by: leocavalcante <leo@cavalcante.dev>
1 parent 951124f commit ed999ec

File tree

1 file changed

+117
-0
lines changed

1 file changed

+117
-0
lines changed

README.md

Lines changed: 117 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -129,6 +129,11 @@ opencoder -P anthropic/claude-opus-4 -B anthropic/claude-sonnet-4
129129
| `-B, --build-model MODEL` | Model for build phase |
130130
| `-p, --project DIR` | Project directory (default: current directory) |
131131
| `-v, --verbose` | Enable verbose logging |
132+
| `--status` | Display metrics summary and exit (no model required) |
133+
| `--metrics-reset` | Reset all metrics to default values (requires confirmation) |
134+
| `--no-auto-commit` | Disable automatic commits after tasks |
135+
| `--no-auto-push` | Disable automatic push after cycles |
136+
| `-s, --signoff` | Add Signed-off-by line to commits (DCO) |
132137
| `-h, --help` | Show help message |
133138
| `-V, --version` | Show version |
134139

@@ -210,6 +215,7 @@ OpenCoder creates a `.opencode/opencoder/` directory in your project:
210215
└── opencoder/
211216
├── config.json # Configuration file
212217
├── state.json # Current state (JSON)
218+
├── metrics.json # Performance metrics and statistics
213219
├── current_plan.md # Active task plan
214220
├── ideas/ # Drop .md files here to queue tasks
215221
│ ├── feature-x.md
@@ -302,6 +308,117 @@ The AI prioritizes based on:
302308
- **No naming convention** - Any `.md` filename works
303309
- **Auto-cleanup** - Empty/invalid ideas are automatically deleted
304310

311+
## Metrics
312+
313+
OpenCoder automatically tracks detailed metrics about your development sessions, including cycles, tasks, token usage, and estimated costs. Monitor your progress and efficiency with built-in metrics commands.
314+
315+
### Viewing Metrics
316+
317+
Display a summary of all tracked metrics without starting the loop:
318+
319+
```bash
320+
opencoder --status
321+
322+
# With a specific project
323+
opencoder --status -p ./myproject
324+
```
325+
326+
This displays:
327+
- **Cycles**: Total completed and timed out
328+
- **Tasks**: Completed, failed, and skipped
329+
- **Retries**: Total retry attempts across all operations
330+
- **Ideas**: Number of queued ideas processed
331+
- **Token Usage**: Input and output tokens consumed
332+
- **Estimated Cost**: Approximate API costs incurred
333+
- **Duration**: Average cycle duration
334+
- **Activity**: Timestamp of last activity
335+
336+
Example output:
337+
```
338+
OpenCoder Metrics
339+
=================
340+
341+
Cycles: 5 completed, 0 timed out
342+
Tasks: 42 completed, 2 failed, 0 skipped
343+
Success rate: 95.5%
344+
Retries: 3 total
345+
Ideas: 1 processed
346+
Avg cycle duration: 1m 0s
347+
Tokens: 50.0K in, 25.0K out
348+
Estimated cost: $1.50
349+
350+
Last activity: 2026-01-18T17:30:00Z
351+
```
352+
353+
### Resetting Metrics
354+
355+
Reset all metrics to default values with confirmation:
356+
357+
```bash
358+
opencoder --metrics-reset
359+
360+
# With a specific project
361+
opencoder --metrics-reset -p ./myproject
362+
```
363+
364+
This command:
365+
1. Displays your current metrics
366+
2. Asks for confirmation before proceeding
367+
3. Resets all counters to zero (preserves timestamps)
368+
4. Confirms successful reset
369+
370+
The confirmation prompt helps prevent accidental data loss:
371+
```
372+
Current Metrics:
373+
================
374+
375+
Cycles: 5 completed, 0 timed out
376+
Tasks: 42 completed, 2 failed, 0 skipped
377+
...
378+
379+
Are you sure you want to reset all metrics to default values? (yes/no): no
380+
Metrics reset cancelled.
381+
```
382+
383+
### Tracked Metrics
384+
385+
OpenCoder tracks the following metrics:
386+
387+
| Metric | Description |
388+
|--------|-------------|
389+
| **cyclesCompleted** | Number of complete plan-build-eval cycles |
390+
| **cyclesTimedOut** | Number of cycles that exceeded the timeout limit |
391+
| **tasksCompleted** | Number of tasks successfully completed |
392+
| **tasksFailed** | Number of tasks that failed after max retries |
393+
| **tasksSkipped** | Number of tasks skipped due to timeout |
394+
| **totalRetries** | Total retry attempts across all operations |
395+
| **ideasProcessed** | Number of ideas from the queue that were processed |
396+
| **totalInputTokens** | Cumulative input tokens consumed from API |
397+
| **totalOutputTokens** | Cumulative output tokens generated by API |
398+
| **totalCostUsd** | Estimated total cost of all API calls in USD |
399+
| **totalCycleDurationMs** | Total time spent in completed cycles (milliseconds) |
400+
| **firstRunTime** | ISO timestamp of first OpenCoder run with this project |
401+
| **lastActivityTime** | ISO timestamp of most recent activity |
402+
403+
### Cost Estimation
404+
405+
OpenCoder estimates costs based on actual token usage and current model pricing for major providers:
406+
407+
- **Claude models** (Opus, Sonnet, Haiku) - Anthropic pricing
408+
- **GPT models** (GPT-4o, GPT-4 Turbo, GPT-3.5) - OpenAI pricing
409+
- **Gemini models** - Google pricing
410+
- **Default pricing** - Used for unknown models
411+
412+
Costs are estimated during operation and recorded in metrics. Use the `--status` flag to see total estimated costs.
413+
414+
### Tips for Metrics
415+
416+
- **Monitor regularly** - Check metrics with `--status` between sessions to track progress
417+
- **Reset between projects** - Use `--metrics-reset` when starting a new project
418+
- **Track efficiency** - Compare success rates and token usage across different models
419+
- **Cost awareness** - Monitor token consumption and estimated costs to stay within budget
420+
- **Metrics file location** - Metrics are stored in `.opencode/opencoder/metrics.json`
421+
305422
## Tips
306423

307424
- **Start with a clear hint** - The more specific your instruction, the better the initial plan

0 commit comments

Comments
 (0)