diff --git a/docs/user-guide.md b/docs/user-guide.md index 7b5fa9a..4388d3d 100644 --- a/docs/user-guide.md +++ b/docs/user-guide.md @@ -3,22 +3,18 @@ layout: page title: User Guide --- -Complete guide to installing, configuring, and using AgentReady to assess your repositories. +Quick reference for installing and using AgentReady to assess and bootstrap your repositories. ## Table of Contents - [Installation](#installation) - [Quick Start](#quick-start) -- [Bootstrap Your Repository](#bootstrap-your-repository) ⭐ **NEW** - - [What is Bootstrap?](#what-is-bootstrap) - - [When to Use Bootstrap vs Assess](#when-to-use-bootstrap-vs-assess) - - [Step-by-Step Tutorial](#step-by-step-tutorial) - - [Generated Files Explained](#generated-files-explained) - - [Post-Bootstrap Checklist](#post-bootstrap-checklist) +- [Bootstrap Your Repository](#bootstrap-your-repository) - [Running Assessments](#running-assessments) +- [Batch Assessment](#batch-assessment) - [Understanding Reports](#understanding-reports) - [Configuration](#configuration) -- [CLI Reference](#cli-reference) + - ⚙️ [Custom Configuration](#custom-configuration) - [Troubleshooting](#troubleshooting) --- @@ -27,1713 +23,324 @@ Complete guide to installing, configuring, and using AgentReady to assess your r ### Prerequisites -- **Python 3.12 or 3.13** (AgentReady supports versions N and N-1) -- **Git** (for repository analysis) -- **pip** or **uv** (Python package manager) +- **Python 3.12 or 3.13** +- **Git** +- **pip** or **uv** ### Install from PyPI ```bash -# Using pip pip install agentready +# Or: uv pip install agentready (recommended) -# Using uv (recommended) -uv pip install agentready - -# Verify installation +# Verify agentready --version ``` -### Install from Source - -```bash -# Clone the repository -git clone https://github.com/ambient-code/agentready.git -cd agentready - -# Create virtual environment -python3 -m venv .venv -source .venv/bin/activate # On Windows: .venv\Scripts\activate - -# Install in development mode -pip install -e . - -# Or using uv -uv pip install -e . -``` - --- ## Quick Start ### Bootstrap-First Approach (Recommended) -Transform your repository with one command: +Transform your repository with complete CI/CD infrastructure: ```bash -# Navigate to your repository cd /path/to/your/repo - -# Bootstrap infrastructure agentready bootstrap . - -# Review generated files -git status - -# Commit and push +git status # Review generated files git add . git commit -m "build: Bootstrap agent-ready infrastructure" git push ``` -Bootstrap generates complete CI/CD infrastructure: GitHub Actions workflows (tests, security, assessment), pre-commit hooks, issue/PR templates, and Dependabot configuration. Assessment runs automatically on your next PR. **Duration**: <60 seconds. - -[See detailed Bootstrap tutorial →](#bootstrap-your-repository) +Generates GitHub Actions workflows (tests, security, assessment), pre-commit hooks, issue/PR templates, and Dependabot configuration. **Duration**: <60 seconds. -### Batch Assessment Approach +### Assess-Only Approach -Assess multiple repositories at once for organizational insights: +For one-time analysis without infrastructure: ```bash -# Navigate to directory containing multiple repos -cd /path/to/repos - -# Run batch assessment -agentready batch repo1/ repo2/ repo3/ --output-dir ./batch-reports - -# View comparison report -open batch-reports/comparison-summary.html +cd /path/to/your/repo +agentready assess . +open .agentready/report-latest.html ``` -Batch assessment generates individual reports for each repository plus a comparison table, aggregate statistics, and trend analysis for multi-repo projects. **Duration**: ~5 seconds per repository. - -[See detailed batch assessment guide →](#batch-assessment) +**Duration**: <5 seconds for most repositories. -### Manual Assessment Approach +### Batch Assessment -For one-time analysis without infrastructure changes: +Assess multiple repositories for organizational insights: ```bash -# Navigate to your repository -cd /path/to/your/repo - -# Run assessment -agentready assess . - -# View the HTML report -open .agentready/report-latest.html # macOS -xdg-open .agentready/report-latest.html # Linux -start .agentready/report-latest.html # Windows +cd /path/to/repos +agentready batch repo1/ repo2/ repo3/ --output-dir ./batch-reports +open batch-reports/comparison-summary.html ``` - -**Output location**: `.agentready/` directory in your repository root. - -**Duration**: Most assessments complete in under 5 seconds. - --- ## Bootstrap Your Repository ### What is Bootstrap? -**Bootstrap is AgentReady's automated infrastructure generator.** Instead of manually implementing recommendations from assessment reports, Bootstrap creates complete GitHub setup in one command: +Bootstrap is AgentReady's automated infrastructure generator. One command creates: -**Generated Infrastructure:** - -- **GitHub Actions workflows** — Tests, security scanning, AgentReady assessment +- **GitHub Actions workflows** — Tests, security scanning, assessment - **Pre-commit hooks** — Language-specific formatters and linters -- **Issue/PR templates** — Structured bug reports, feature requests, PR checklist +- **Issue/PR templates** — Structured bug reports, feature requests - **CODEOWNERS** — Automated review assignments - **Dependabot** — Weekly dependency updates - **Contributing guidelines** — If not present - **Code of Conduct** — Red Hat standard (if not present) -**Language Detection:** -Bootstrap automatically detects your primary language (Python, JavaScript, Go) via `git ls-files` and generates appropriate configurations. +**Language Detection**: Automatically detects your primary language (Python, JavaScript, Go) via `git ls-files`. -**Safe to Use:** +**Safe by Design**: -- Use `--dry-run` to preview changes without creating files -- All files are created, never overwritten +- Use `--dry-run` to preview changes +- Never overwrites existing files - Review with `git status` before committing ---- - ### When to Use Bootstrap vs Assess - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
ScenarioUse BootstrapUse Assess
New project✅ Start with best practicesLater, to track progress
Missing GitHub Actions✅ Generate workflows instantlyShows it's missing
No pre-commit hooks✅ Configure automaticallyShows it's missing
Understanding current stateUse after bootstrapping✅ Detailed analysis
Existing infrastructureSafe (won't overwrite)✅ Validate setup
Tracking improvementsOne-time setup✅ Run repeatedly
CI/CD integrationGenerates the workflows✅ Runs in CI (via Bootstrap)
- -**Recommended workflow:** - -1. **Bootstrap first** — Generate infrastructure -2. **Review and commit** — Inspect generated files -3. **Assess automatically** — Every PR via GitHub Actions -4. **Manual assess** — When validating improvements - ---- - -### Step-by-Step Tutorial +| Scenario | Use Bootstrap | Use Assess | +|----------|---------------|------------| +| **New project** | ✅ Start with best practices | Later, to track progress | +| **Missing GitHub Actions** | ✅ Generate workflows instantly | Shows it's missing | +| **Understanding current state** | Use after bootstrapping | ✅ Detailed analysis | +| **Tracking improvements** | One-time setup | ✅ Run repeatedly | +| **CI/CD integration** | Generates the workflows | ✅ Runs in CI | -#### Step 1: Preview Changes (Dry Run) +**Recommended workflow**: Bootstrap first → Review and commit → Assess automatically on PRs → Manual assess when validating improvements -Always start with `--dry-run` to see what will be created: +### Basic Usage ```bash -cd /path/to/your/repo +# Preview changes (recommended first step) agentready bootstrap . --dry-run -``` - -**Example output:** - -``` -Detecting primary language... -✓ Detected: Python (42 files) - -Files that will be created: - .github/workflows/agentready-assessment.yml - .github/workflows/tests.yml - .github/workflows/security.yml - .github/ISSUE_TEMPLATE/bug_report.md - .github/ISSUE_TEMPLATE/feature_request.md - .github/PULL_REQUEST_TEMPLATE.md - .github/CODEOWNERS - .github/dependabot.yml - .pre-commit-config.yaml - CONTRIBUTING.md (not present, will create) - CODE_OF_CONDUCT.md (not present, will create) - -Run without --dry-run to create these files. -``` - -**Review the list carefully:** - -- Files marked "(not present, will create)" are new -- Existing files are never overwritten -- Check for conflicts with existing workflows - ---- - -#### Step 2: Run Bootstrap -If dry-run output looks good, run without flag: - -```bash +# Generate infrastructure agentready bootstrap . -``` - -**Example output:** - -``` -Detecting primary language... -✓ Detected: Python (42 files) - -Creating infrastructure... - ✓ .github/workflows/agentready-assessment.yml - ✓ .github/workflows/tests.yml - ✓ .github/workflows/security.yml - ✓ .github/ISSUE_TEMPLATE/bug_report.md - ✓ .github/ISSUE_TEMPLATE/feature_request.md - ✓ .github/PULL_REQUEST_TEMPLATE.md - ✓ .github/CODEOWNERS - ✓ .github/dependabot.yml - ✓ .pre-commit-config.yaml - ✓ CONTRIBUTING.md - ✓ CODE_OF_CONDUCT.md - -Bootstrap complete! 11 files created. - -Next steps: -1. Review generated files: git status -2. Customize as needed (CODEOWNERS, workflow triggers, etc.) -3. Commit: git add . && git commit -m "build: Bootstrap infrastructure" -4. Enable GitHub Actions in repository settings -5. Push and create PR to see assessment in action! -``` - ---- - -#### Step 3: Review Generated Files - -Inspect what was created: - -```bash -# View all new files -git status - -# Inspect key files -cat .github/workflows/agentready-assessment.yml -cat .pre-commit-config.yaml -cat .github/CODEOWNERS -``` - -**What to check:** -- **CODEOWNERS** — Add actual team member GitHub usernames -- **Workflows** — Adjust triggers (e.g., only main branch, specific paths) -- **Pre-commit hooks** — Add/remove tools based on your stack -- **Issue templates** — Customize labels and assignees - ---- - -#### Step 4: Install Pre-commit Hooks (Local) - -Bootstrap creates `.pre-commit-config.yaml`, but you must install locally: - -```bash -# Install pre-commit (if not already) -pip install pre-commit - -# Install git hooks -pre-commit install - -# Test hooks on all files -pre-commit run --all-files -``` - -**Expected output:** +# Force specific language +agentready bootstrap . --language python +# Bootstrap different directory +agentready bootstrap /path/to/repo ``` -black....................................................................Passed -isort....................................................................Passed -ruff.....................................................................Passed -``` - -**If failures occur:** -- Review suggested fixes -- Run formatters: `black .` and `isort .` -- Fix linting errors: `ruff check . --fix` -- Re-run: `pre-commit run --all-files` +### Post-Bootstrap Steps ---- +1. **Review generated files**: -#### Step 5: Commit and Push + ```bash + git status + cat .github/workflows/agentready-assessment.yml + cat .pre-commit-config.yaml + ``` -```bash -# Stage all generated files -git add . +2. **Customize CODEOWNERS**: Replace placeholder usernames with actual team members +3. **Install pre-commit hooks locally**: -# Commit with conventional commit message -git commit -m "build: Bootstrap agent-ready infrastructure + ```bash + pip install pre-commit + pre-commit install + pre-commit run --all-files + ``` -- Add GitHub Actions workflows (tests, security, assessment) -- Configure pre-commit hooks (black, isort, ruff) -- Add issue and PR templates -- Enable Dependabot for weekly updates -- Add CONTRIBUTING.md and CODE_OF_CONDUCT.md" +4. **Enable GitHub Actions**: Settings → Actions → General → Allow all actions +5. **Commit and push**: -# Push to repository -git push origin main -``` + ```bash + git add . + git commit -m "build: Bootstrap agent-ready infrastructure" + git push + ``` ---- +6. **Test with PR**: -#### Step 6: Enable GitHub Actions + ```bash + git checkout -b test-bootstrap + echo "# Test" >> README.md + git add README.md + git commit -m "test: Verify AgentReady workflow" + git push origin test-bootstrap + gh pr create --title "Test: AgentReady Bootstrap" --body "Testing assessment" + ``` -If this is the first time using Actions: +### Generated Files -1. **Navigate to repository on GitHub** -2. **Go to Settings → Actions → General** -3. **Enable Actions** (select "Allow all actions") -4. **Set workflow permissions** to "Read and write permissions" -5. **Save** +- **Workflows**: Assessment, tests, security (CodeQL) +- **Pre-commit hooks**: black/isort/ruff (Python), prettier/eslint (JS), gofmt/golint (Go) +- **Templates**: Bug reports, feature requests, PR template, CODEOWNERS, Dependabot --- -#### Step 7: Test with a PR +## Running Assessments -Create a test PR to see Bootstrap in action: +### Basic Usage ```bash -# Create feature branch -git checkout -b test-agentready-bootstrap - -# Make trivial change -echo "# Test" >> README.md +# Assess current directory +agentready assess . -# Commit and push -git add README.md -git commit -m "test: Verify AgentReady assessment workflow" -git push origin test-agentready-bootstrap +# Assess specific repository +agentready assess /path/to/repo -# Create PR on GitHub -gh pr create --title "Test: AgentReady Bootstrap" --body "Testing automated assessment" +# Custom output directory +agentready assess . --output-dir ./custom-reports ``` -**What happens automatically:** - -1. **Tests workflow** runs pytest (Python) or appropriate tests -2. **Security workflow** runs CodeQL analysis -3. **AgentReady assessment workflow** runs assessment and posts results as PR comment +### Assessment Output -**PR comment example:** +Reports are saved in `.agentready/` directory: ``` -## AgentReady Assessment - -**Score:** 75.4/100 (🥇 Gold) - -**Tier Breakdown:** -- Tier 1 (Essential): 80/100 -- Tier 2 (Critical): 70/100 -- Tier 3 (Important): 65/100 -- Tier 4 (Advanced): 50/100 - -**Passing:** 15/25 | **Failing:** 8/25 | **Skipped:** 2/25 - -[View full HTML report](link-to-artifact) +.agentready/ +├── assessment-YYYYMMDD-HHMMSS.json # Machine-readable data +├── report-YYYYMMDD-HHMMSS.html # Interactive web report +├── report-YYYYMMDD-HHMMSS.md # Markdown report +├── assessment-latest.json # Symlink to latest +├── report-latest.html # Symlink to latest +└── report-latest.md # Symlink to latest ``` --- -### Generated Files Explained +## Batch Assessment -#### GitHub Actions Workflows +Assess multiple repositories for organizational insights: -**`.github/workflows/agentready-assessment.yml`** +```bash +# Assess all repos in a directory +agentready batch /path/to/repos --output-dir ./reports -```yaml -# Runs on every PR and push to main -# Posts assessment results as PR comment -# Fails if score drops below configured threshold (default: 60) +# Assess specific repos +agentready batch /path/repo1 /path/repo2 /path/repo3 -Triggers: pull_request, push (main branch) -Duration: ~30 seconds -Artifacts: HTML report, JSON data +# Generate comparison report +agentready batch . --compare ``` -**`.github/workflows/tests.yml`** - -```yaml -# Language-specific test workflow - -Python: - - Runs pytest with coverage - - Coverage report posted as PR comment - - Requires test/ directory - -JavaScript: - - Runs jest with coverage - - Generates lcov report +### Batch Output -Go: - - Runs go test with race detection - - Coverage profiling enabled ``` - -**`.github/workflows/security.yml`** - -```yaml -# Comprehensive security scanning - -CodeQL: - - Analyzes code for vulnerabilities - - Runs on push to main and PR - - Supports 10+ languages - -Dependency Scanning: - - GitHub Advisory Database - - Fails on high/critical vulnerabilities +reports/ +├── comparison-summary.html # Interactive comparison table +├── comparison-summary.md # Markdown summary +├── aggregate-stats.json # Machine-readable statistics +├── repo1/ +│ ├── assessment-latest.json +│ └── report-latest.html +└── repo2/ + └── ... ``` ---- - -#### Pre-commit Configuration - -**`.pre-commit-config.yaml`** - -Language-specific hooks configured: - -**Python:** - -- `black` — Code formatter (88 char line length) -- `isort` — Import sorter -- `ruff` — Fast linter -- `trailing-whitespace` — Remove trailing spaces -- `end-of-file-fixer` — Ensure newline at EOF - -**JavaScript/TypeScript:** - -- `prettier` — Code formatter -- `eslint` — Linter -- `trailing-whitespace` -- `end-of-file-fixer` - -**Go:** - -- `gofmt` — Code formatter -- `golint` — Linter -- `go-vet` — Static analysis - -**To customize:** -Edit `.pre-commit-config.yaml` and adjust hook versions or add new repos. - ---- - -#### GitHub Templates - -**`.github/ISSUE_TEMPLATE/bug_report.md`** - -- Structured bug report with reproduction steps -- Environment details (OS, version) -- Expected vs actual behavior -- Auto-labels as `bug` - -**`.github/ISSUE_TEMPLATE/feature_request.md`** - -- Structured feature proposal -- Use case and motivation -- Proposed solution -- Auto-labels as `enhancement` - -**`.github/PULL_REQUEST_TEMPLATE.md`** - -- Checklist for PR authors: - - [ ] Tests added/updated - - [ ] Documentation updated - - [ ] Passes all checks - - [ ] Breaking changes documented -- Links to related issues -- Change description - -**`.github/CODEOWNERS`** - -``` -# Auto-assign reviewers based on file paths -# CUSTOMIZE: Replace with actual GitHub usernames +### Interactive Heatmap Visualization -* @yourteam/maintainers -/docs/ @yourteam/docs -/.github/ @yourteam/devops -``` +Generate interactive Plotly heatmap showing attribute scores across repositories: -**`.github/dependabot.yml`** +```bash +# Generate heatmap with batch assessment +agentready assess-batch --repos /path/repo1 --repos /path/repo2 --generate-heatmap -```yaml -# Weekly dependency update checks -# Creates PRs for outdated dependencies -# Supports Python, npm, Go modules - -Updates: - - package-ecosystem: pip (or npm, gomod) - schedule: weekly - labels: [dependencies] +# Custom heatmap output +agentready assess-batch --repos-file repos.txt --generate-heatmap --heatmap-output ./heatmap.html ``` --- -#### Development Guidelines - -**`CONTRIBUTING.md`** (created if missing) - -- Setup instructions -- Development workflow -- Code style guidelines -- PR process -- Testing requirements - -**`CODE_OF_CONDUCT.md`** (created if missing) - -- Red Hat standard Code of Conduct -- Community guidelines -- Reporting process -- Enforcement policy - ---- +## Understanding Reports -### Post-Bootstrap Checklist +### HTML Report (Interactive) -After running `agentready bootstrap`, complete these steps: +**File**: `report-YYYYMMDD-HHMMSS.html` -#### 1. Customize CODEOWNERS +Interactive web report with score card, tier breakdown, sortable attribute table, and expandable findings. Self-contained (no CDN), safe to share via email or wikis. Open in browser to explore ✅/❌/⊘ findings, filter by status, and copy remediation commands. -```bash -# Edit .github/CODEOWNERS -vim .github/CODEOWNERS +### Markdown Report (Version Control) -# Replace placeholder usernames with actual team members -# * @yourteam/maintainers → * @alice @bob -# /docs/ @yourteam/docs → /docs/ @carol -``` +**File**: `report-YYYYMMDD-HHMMSS.md` -#### 2. Review Workflow Triggers +GitHub-Flavored Markdown for tracking progress over time. Commit after each assessment to see improvements in git diffs. -```bash -# Check if workflow triggers match your branching strategy -cat .github/workflows/*.yml | grep "on:" +### JSON Report (Machine-Readable) -# Common adjustments: -# - Change 'main' to 'master' or 'develop' -# - Add path filters (e.g., only run tests when src/ changes) -# - Adjust schedule (e.g., nightly instead of push) -``` +**File**: `assessment-YYYYMMDD-HHMMSS.json` -#### 3. Install Pre-commit Hooks +Complete data for CI/CD integration. Example: ```bash -pip install pre-commit -pre-commit install -pre-commit run --all-files # Test on existing code +# Fail build if score < 70 +score=$(jq '.overall_score' .agentready/assessment-latest.json) +(( $(echo "$score < 70" | bc -l) )) && exit 1 ``` -#### 4. Enable GitHub Actions - -- Repository Settings → Actions → General -- Enable "Allow all actions" -- Set "Read and write permissions" for workflows - -#### 5. Configure Branch Protection (Recommended) - -- Settings → Branches → Add rule for `main` -- Require status checks: `tests`, `security`, `agentready-assessment` -- Require PR reviews (at least 1 approval) -- Require branches to be up to date - -#### 6. Test the Workflows +--- -Create a test PR to verify: +## Configuration -```bash -git checkout -b test-workflows -echo "# Test" >> README.md -git add README.md -git commit -m "test: Verify automated workflows" -git push origin test-workflows -gh pr create --title "Test: Verify workflows" --body "Testing Bootstrap" -``` +### Default Behavior -**Verify:** +AgentReady works out-of-the-box with sensible defaults. No configuration required for basic usage. -- ✅ All workflows run successfully -- ✅ AgentReady posts PR comment with assessment results -- ✅ Test coverage report appears -- ✅ Security scan completes without errors +### Custom Configuration -#### 7. Update Documentation +Create `.agentready-config.yaml` to customize: -Add Badge to README.md: +```yaml +# Custom attribute weights (must sum to 1.0) +weights: + claude_md_file: 0.15 # Increase from default 0.10 + readme_structure: 0.12 + type_annotations: 0.08 -```markdown -# MyProject +# Exclude specific attributes +excluded_attributes: + - performance_benchmarks + - container_setup -![AgentReady](https://img.shields.io/badge/AgentReady-Bootstrap-blue) -![Tests](https://github.com/yourusername/repo/workflows/tests/badge.svg) -![Security](https://github.com/yourusername/repo/workflows/security/badge.svg) +# Custom output directory +output_dir: ./reports ``` -Mention Bootstrap in README: +### Generate/Validate Configuration -```markdown -## Development Setup - -This repository uses AgentReady Bootstrap for automated quality assurance. +```bash +# Generate example configuration +agentready --generate-config > .agentready-config.yaml -All PRs are automatically assessed for agent-readiness. See the PR comment -for detailed findings and remediation guidance. +# Validate configuration +agentready --validate-config .agentready-config.yaml ``` --- -### Language-Specific Notes - -#### Python Projects - -Bootstrap generates: - -- `pytest` workflow with coverage (`pytest-cov`) -- Pre-commit hooks: `black`, `isort`, `ruff`, `mypy` -- Dependabot for pip dependencies - -**Customizations:** - -- Adjust `pytest` command in `tests.yml` if using different test directory -- Add `mypy` configuration in `pyproject.toml` if type checking required -- Modify `black` line length in `.pre-commit-config.yaml` if needed - -#### JavaScript/TypeScript Projects - -Bootstrap generates: - -- `jest` or `npm test` workflow -- Pre-commit hooks: `prettier`, `eslint` -- Dependabot for npm dependencies - -**Customizations:** - -- Update test command in `tests.yml` to match `package.json` scripts -- Adjust `prettier` config (`.prettierrc`) if different style -- Add TypeScript type checking (`tsc --noEmit`) to workflow - -#### Go Projects - -Bootstrap generates: - -- `go test` workflow with race detection -- Pre-commit hooks: `gofmt`, `golint`, `go-vet` -- Dependabot for Go modules - -**Customizations:** - -- Add build step to workflow if needed (`go build ./...`) -- Configure `golangci-lint` for advanced linting -- Add coverage reporting (`go test -coverprofile=coverage.out`) - ---- - -### Bootstrap Command Reference - -```bash -agentready bootstrap [REPOSITORY] [OPTIONS] -``` - -**Arguments:** - -- `REPOSITORY` — Path to repository (default: current directory) - -**Options:** - -- `--dry-run` — Preview files without creating -- `--language TEXT` — Override auto-detection: `python|javascript|go|auto` (default: auto) - -**Examples:** +## Troubleshooting -```bash -# Bootstrap current directory (auto-detect language) -agentready bootstrap . +### Common Issues -# Preview without creating files -agentready bootstrap . --dry-run +**"No module named 'agentready'"** — `pip install agentready` -# Force Python configuration -agentready bootstrap . --language python +**"Permission denied"** — `agentready assess . --output-dir ~/reports` -# Bootstrap different directory -agentready bootstrap /path/to/repo +**"Repository not found"** — `git init` to initialize repository -# Combine dry-run and language override -agentready bootstrap /path/to/repo --dry-run --language go -``` +**"Assessment taking too long"** — AgentReady warns before scanning >10,000 files. Check: `agentready assess . --verbose` -**Exit codes:** +**"File already exists" (Bootstrap)** — Bootstrap never overwrites files by design. Remove existing files first if regenerating. -- `0` — Success -- `1` — Error (not a git repository, permission denied, etc.) +**"Language detection failed" (Bootstrap)** — `agentready bootstrap . --language python` to force language ---- +**"GitHub Actions not running"** — Enable in Settings → Actions → General. Set "Read and write permissions" in Workflow permissions. -## Running Assessments +**"Pre-commit hooks not running"** — `pip install pre-commit && pre-commit install` -### Basic Usage - -```bash -# Assess current directory -agentready assess . - -# Assess specific repository -agentready assess /path/to/repo - -# Assess with verbose output -agentready assess . --verbose - -# Custom output directory -agentready assess . --output-dir ./custom-reports -``` - -### Assessment Output - -AgentReady creates a `.agentready/` directory containing: - -``` -.agentready/ -├── assessment-YYYYMMDD-HHMMSS.json # Machine-readable data -├── report-YYYYMMDD-HHMMSS.html # Interactive web report -├── report-YYYYMMDD-HHMMSS.md # Markdown report -├── assessment-latest.json # Symlink to latest -├── report-latest.html # Symlink to latest -└── report-latest.md # Symlink to latest -``` - -**Timestamps**: All files are timestamped for historical tracking. - -**Latest links**: Symlinks always point to the most recent assessment. - -### Verbose Mode - -Get detailed progress information during assessment: - -```bash -agentready assess . --verbose -``` - -**Output includes**: - -- Repository path and detected languages -- Each assessor's execution status -- Finding summaries (pass/fail/skip) -- Final score calculation breakdown -- Report generation progress - ---- - -## Batch Assessment - -Assess multiple repositories in one command to gain organizational insights and identify patterns across projects. - -### Basic Usage - -```bash -# Assess all repos in a directory -agentready batch /path/to/repos --output-dir ./reports - -# Assess specific repos -agentready batch /path/repo1 /path/repo2 /path/repo3 - -# Generate comparison report -agentready batch . --compare -``` - -### Batch Output - -AgentReady batch assessment creates: - -``` -reports/ -├── comparison-summary.html # Interactive comparison table -├── comparison-summary.md # Markdown summary -├── aggregate-stats.json # Machine-readable statistics -├── repo1/ -│ ├── assessment-latest.json -│ ├── report-latest.html -│ └── report-latest.md -├── repo2/ -│ └── ... -└── repo3/ - └── ... -``` - -### Comparison Report Features - -**comparison-summary.html** includes: - -- Side-by-side score comparison table -- Certification level distribution (Platinum/Gold/Silver/Bronze) -- Average scores by tier -- Outlier detection (repos significantly above/below average) -- Sortable columns (by score, name, certification) -- Filterable view (show only failing repos) - -**Example comparison table:** - -| Repository | Overall Score | Cert Level | Tier 1 | Tier 2 | Tier 3 | Tier 4 | -|------------|---------------|------------|--------|--------|--------|--------| -| agentready | 80.0/100 | Gold | 90.0 | 75.0 | 70.0 | 60.0 | -| project-a | 75.2/100 | Gold | 85.0 | 70.0 | 65.0 | 55.0 | -| project-b | 62.5/100 | Silver | 70.0 | 60.0 | 55.0 | 45.0 | - -### Aggregate Statistics - -**aggregate-stats.json** provides: - -```json -{ - "total_repositories": 3, - "average_score": 72.6, - "median_score": 75.2, - "certification_distribution": { - "Platinum": 0, - "Gold": 2, - "Silver": 1, - "Bronze": 0, - "Needs Improvement": 0 - }, - "tier_averages": { - "tier_1": 81.7, - "tier_2": 68.3, - "tier_3": 63.3, - "tier_4": 53.3 - }, - "common_failures": [ - {"attribute": "pre_commit_hooks", "failure_rate": 0.67}, - {"attribute": "lock_files", "failure_rate": 0.33} - ] -} -``` - -### Interactive Heatmap Visualization - -Generate an interactive Plotly heatmap showing attribute scores across all repositories: - -```bash -# Generate heatmap with batch assessment -agentready assess-batch --repos /path/repo1 --repos /path/repo2 --generate-heatmap - -# Custom heatmap output path -agentready assess-batch --repos-file repos.txt --generate-heatmap --heatmap-output ./heatmap.html -``` - -The heatmap visualization includes color-coded scores for instant visual identification of strong/weak attributes, cross-repo comparison to see patterns, interactive exploration with hover details and zoom, and export capability as a self-contained HTML file for sharing with teams. Use heatmaps to identify organization-wide patterns, spot outliers, track improvements over time, and guide training efforts on commonly failing attributes. - -### Use Cases - -**Organization-wide assessment**: - -```bash -# Clone all org repos, then batch assess -gh repo list myorg --limit 100 --json name --jq '.[].name' | \ - xargs -I {} gh repo clone myorg/{} - -agentready batch repos/* --output-dir ./org-assessment -``` - -**Multi-repo project**: - -```bash -# Assess all microservices -agentready batch services/* --compare -``` - -**Trend tracking**: - -```bash -# Monthly assessment -agentready batch repos/* --output-dir ./assessments/2025-11 -``` - ---- - -## Report Validation & Migration - -AgentReady v1.27.2 includes schema versioning for backwards compatibility and evolution. - -### Validate Reports - -Verify assessment reports conform to their schema version: - -```bash -# Strict validation (default) -agentready validate-report .agentready/assessment-latest.json - -# Lenient validation (allow extra fields) -agentready validate-report --no-strict .agentready/assessment-latest.json -``` - -**Output examples:** - -**Valid report:** - -``` -✅ Report is valid! -Schema version: 1.0.0 -Repository: agentready -Overall score: 80.0/100 -``` - -**Invalid report:** - -``` -❌ Validation failed! 3 errors found: - - Missing required field: 'schema_version' - - Invalid type for 'overall_score': expected number, got string - - Extra field not allowed in strict mode: 'custom_field' -``` - -### Migrate Reports - -Convert reports between schema versions: - -```bash -# Migrate to specific version -agentready migrate-report old-report.json --to 2.0.0 - -# Custom output path -agentready migrate-report old.json --to 2.0.0 --output new.json - -# Explicit source version (auto-detected by default) -agentready migrate-report old.json --from 1.0.0 --to 2.0.0 -``` - -**Migration output:** - -``` -🔄 Migrating report... -Source version: 1.0.0 -Target version: 2.0.0 - -✅ Migration successful! -Migrated report saved to: assessment-20251123-migrated.json -``` - -### Schema Compatibility - -**Current schema version**: 1.0.0 - -**Supported versions**: - -- 1.0.0 (current) - -**Future versions** will maintain backwards compatibility: - -- Read old versions via migration -- Write new versions with latest schema -- Migration paths provided for all versions - -[Learn more about schema versioning →](schema-versioning.html) - ---- - -## Understanding Reports - -AgentReady generates three complementary report formats. - -### HTML Report (Interactive) - -**File**: `report-YYYYMMDD-HHMMSS.html` - -The HTML report provides an interactive, visual interface: - -#### Features - -- **Overall Score Card**: Certification level, score, and visual gauge -- **Tier Summary**: Breakdown by attribute tier (Essential/Critical/Important/Advanced) -- **Attribute Table**: Sortable, filterable list of all attributes -- **Detailed Findings**: Expandable sections for each attribute -- **Search**: Find specific attributes by name or ID -- **Filters**: Show only passed, failed, or skipped attributes -- **Copy Buttons**: One-click code example copying -- **Offline**: No CDN dependencies, works anywhere - -#### How to Use - -1. **Open in browser**: Double-click the HTML file -2. **Review overall score**: Check certification level and tier breakdown -3. **Explore findings**: - - Green ✅ = Passed - - Red ❌ = Failed (needs remediation) - - Gray ⊘ = Skipped (not applicable or not yet implemented) -4. **Click to expand**: View detailed evidence and remediation steps -5. **Filter results**: Focus on specific attribute statuses -6. **Copy remediation commands**: Use one-click copy for code examples - -#### Security - -HTML reports include Content Security Policy (CSP) headers for defense-in-depth: - -- Prevents unauthorized script execution -- Mitigates XSS attack vectors -- Safe to share and view in any browser - -The CSP policy allows only inline styles and scripts needed for interactivity. - -#### Sharing - -The HTML report is self-contained and can be: - -- Emailed to stakeholders -- Uploaded to internal wikis -- Viewed on any device with a browser -- Archived for compliance/audit purposes - -### Markdown Report (Version Control Friendly) - -**File**: `report-YYYYMMDD-HHMMSS.md` - -The Markdown report is optimized for git tracking: - -#### Features - -- **GitHub-Flavored Markdown**: Renders beautifully on GitHub -- **Git-Diffable**: Track score improvements over time -- **ASCII Tables**: Attribute summaries without HTML -- **Emoji Indicators**: ✅❌⊘ for visual status -- **Certification Ladder**: Visual progress chart -- **Prioritized Next Steps**: Highest-impact improvements first - -#### How to Use - -1. **Commit to repository**: - - ```bash - git add .agentready/report-latest.md - git commit -m "docs: Add AgentReady assessment report" - ``` - -2. **Track progress**: - - ```bash - # Run new assessment - agentready assess . - - # Compare to previous - git diff .agentready/report-latest.md - ``` - -3. **Review on GitHub**: Push and view formatted Markdown - -4. **Share in PRs**: Reference in pull request descriptions - -#### Recommended Workflow - -```bash -# Initial baseline -agentready assess . -git add .agentready/report-latest.md -git commit -m "docs: AgentReady baseline (Score: 65.2)" - -# Make improvements -# ... implement recommendations ... - -# Re-assess -agentready assess . -git add .agentready/report-latest.md -git commit -m "docs: AgentReady improvements (Score: 72.8, +7.6)" -``` - -### JSON Report (Machine-Readable) - -**File**: `assessment-YYYYMMDD-HHMMSS.json` - -The JSON report contains complete assessment data: - -#### Structure - -```json -{ - "metadata": { - "timestamp": "2025-11-21T10:30:00Z", - "repository_path": "/path/to/repo", - "agentready_version": "1.0.0", - "duration_seconds": 2.35 - }, - "repository": { - "path": "/path/to/repo", - "name": "myproject", - "languages": {"Python": 42, "JavaScript": 18} - }, - "overall_score": 75.4, - "certification_level": "Gold", - "tier_scores": { - "tier_1": 85.0, - "tier_2": 70.0, - "tier_3": 65.0, - "tier_4": 50.0 - }, - "findings": [ - { - "attribute_id": "claude_md_file", - "attribute_name": "CLAUDE.md File", - "tier": 1, - "weight": 0.10, - "status": "pass", - "score": 100, - "evidence": "Found CLAUDE.md at repository root", - "remediation": null - } - ] -} -``` - -#### Use Cases - -**CI/CD Integration**: - -```bash -# Fail build if score < 70 -score=$(jq '.overall_score' .agentready/assessment-latest.json) -if (( $(echo "$score < 70" | bc -l) )); then - echo "AgentReady score too low: $score" - exit 1 -fi -``` - -**Trend Analysis**: - -```python -import json -import glob - -# Load all historical assessments -assessments = [] -for file in sorted(glob.glob('.agentready/assessment-*.json')): - with open(file) as f: - assessments.append(json.load(f)) - -# Track score over time -for a in assessments: - print(f"{a['metadata']['timestamp']}: {a['overall_score']}") -``` - -**Custom Reporting**: - -```python -import json - -with open('.agentready/assessment-latest.json') as f: - assessment = json.load(f) - -# Extract failed attributes -failed = [ - f for f in assessment['findings'] - if f['status'] == 'fail' -] - -# Create custom report -for finding in failed: - print(f"❌ {finding['attribute_name']}") - print(f" {finding['evidence']}") - print() -``` - ---- - -## Configuration - -### Default Behavior - -AgentReady works out-of-the-box with sensible defaults. No configuration required for basic usage. - -### Custom Configuration File - -Create `.agentready-config.yaml` to customize: - -```yaml -# Custom attribute weights (must sum to 1.0) -weights: - claude_md_file: 0.15 # Increase from default 0.10 - readme_structure: 0.12 # Increase from default 0.10 - type_annotations: 0.08 # Decrease from default 0.10 - # ... other 22 attributes - -# Exclude specific attributes -excluded_attributes: - - performance_benchmarks # Skip this assessment - - container_setup # Not applicable to our project - -# Custom output directory -output_dir: ./reports - -# Verbosity (true/false) -verbose: false -``` - -### Weight Customization Rules - -1. **Must sum to 1.0**: Total weight across all attributes (excluding excluded ones) -2. **Minimum weight**: 0.01 (1%) -3. **Maximum weight**: 0.20 (20%) -4. **Automatic rebalancing**: Excluded attributes' weights redistribute proportionally - -### Example: Security-Focused Configuration - -```yaml -# Emphasize security attributes -weights: - dependency_security: 0.15 # Default: 0.05 - secrets_management: 0.12 # Default: 0.05 - security_scanning: 0.10 # Default: 0.03 - # Other weights adjusted to sum to 1.0 - -excluded_attributes: - - performance_benchmarks -``` - -### Example: Documentation-Focused Configuration - -```yaml -# Emphasize documentation quality -weights: - claude_md_file: 0.20 # Default: 0.10 - readme_structure: 0.15 # Default: 0.10 - inline_documentation: 0.12 # Default: 0.08 - api_documentation: 0.10 # Default: 0.05 - # Other weights adjusted to sum to 1.0 -``` - -### Validate Configuration - -```bash -# Validate configuration file -agentready --validate-config .agentready-config.yaml - -# Generate example configuration -agentready --generate-config > .agentready-config.yaml -``` - ---- - -## CLI Reference - -### Main Commands - -#### `agentready assess PATH` - -Assess a repository at the specified path. - -**Arguments**: - -- `PATH` — Repository path to assess (required) - -**Options**: - -- `--verbose, -v` — Show detailed progress information -- `--config FILE, -c FILE` — Use custom configuration file -- `--output-dir DIR, -o DIR` — Custom report output directory - -**Examples**: - -```bash -agentready assess . -agentready assess /path/to/repo -agentready assess . --verbose -agentready assess . --config custom.yaml -agentready assess . --output-dir ./reports -``` - -### Configuration Commands - -#### `agentready --generate-config` - -Generate example configuration file. - -**Output**: Prints YAML configuration to stdout. - -**Example**: - -```bash -agentready --generate-config > .agentready-config.yaml -``` - -#### `agentready --validate-config FILE` - -Validate configuration file syntax and weights. - -**Example**: - -```bash -agentready --validate-config .agentready-config.yaml -``` - -### Research Commands - -#### `agentready --research-version` - -Show bundled research document version. - -**Example**: - -```bash -agentready --research-version -# Output: Research version: 1.0.0 (2025-11-20) -``` - -### Utility Commands - -#### `agentready --version` - -Show AgentReady version. - -#### `agentready --help` - -Show help message with all commands. - ---- - -## Troubleshooting - -### Common Issues - -#### "No module named 'agentready'" - -**Cause**: AgentReady not installed or wrong Python environment. - -**Solution**: - -```bash -# Verify Python version -python --version # Should be 3.11 or 3.12 - -# Check installation -pip list | grep agentready - -# Reinstall if missing -pip install agentready -``` - -#### "Permission denied: .agentready/" - -**Cause**: No write permissions in repository directory. - -**Solution**: - -```bash -# Use custom output directory -agentready assess . --output-dir ~/agentready-reports - -# Or fix permissions -chmod u+w . -``` - -#### "Repository not found" - -**Cause**: Path does not point to a git repository. - -**Solution**: - -```bash -# Verify git repository -git status - -# If not a git repo, initialize one -git init -``` - -#### "Assessment taking too long" - -**Cause**: Large repository with many files. - -**Solution**: -AgentReady should complete in <10 seconds for most repositories. If it hangs: - -1. **Check verbose output**: - - ```bash - agentready assess . --verbose - ``` - -2. **Verify git performance**: - - ```bash - time git ls-files - ``` - -3. **Report issue** with repository size and language breakdown. - -**Note**: AgentReady will now warn you before scanning repositories with more than 10,000 files: - -``` -⚠️ Warning: Large repository detected (12,543 files). -Assessment may take several minutes. Continue? [y/N]: -``` - -#### "Warning: Scanning sensitive directory" - -**Cause**: Attempting to scan system directories like `/etc`, `/sys`, `/proc`, `/.ssh`, or `/var`. - -**Solution**: -AgentReady includes safety checks to prevent accidental scanning of sensitive system directories: - -``` -⚠️ Warning: Scanning sensitive directory /etc. Continue? [y/N]: -``` - -**Best practices**: - -- Only scan your own project repositories -- Never scan system directories or sensitive configuration folders -- If you need to assess a project in `/var/www`, copy it to a user directory first -- Use `--output-dir` to avoid writing reports to sensitive locations - -#### "Invalid configuration file" - -**Cause**: Malformed YAML or incorrect weight values. - -**Solution**: - -```bash -# Validate configuration -agentready --validate-config .agentready-config.yaml - -# Check YAML syntax -python -c "import yaml; yaml.safe_load(open('.agentready-config.yaml'))" - -# Regenerate from template -agentready --generate-config > .agentready-config.yaml -``` - ---- - -### Bootstrap-Specific Issues - -#### "File already exists" error - -**Cause**: Bootstrap refuses to overwrite existing files. - -**Solution**: -Bootstrap is safe by design—it never overwrites existing files. This is expected behavior: - -```bash -# Review what files already exist -ls -la .github/workflows/ -ls -la .pre-commit-config.yaml - -# If you want to regenerate, manually remove first -rm .github/workflows/agentready-assessment.yml -agentready bootstrap . - -# Or keep existing and only add missing files -agentready bootstrap . # Safely skips existing -``` - ---- - -#### "Language detection failed" - -**Cause**: No recognizable language files in repository. - -**Solution**: - -```bash -# Check what files git tracks -git ls-files - -# If empty, add some files first -git add *.py # or *.js, *.go - -# Force specific language -agentready bootstrap . --language python - -# Or if mixed language project -agentready bootstrap . --language auto # Uses majority language -``` - ---- - -#### "GitHub Actions not running" - -**Cause**: Actions not enabled or insufficient permissions. - -**Solution**: - -1. **Enable Actions**: - - Repository Settings → Actions → General - - Select "Allow all actions" - - Save - -2. **Check workflow permissions**: - - Settings → Actions → General → Workflow permissions - - Select "Read and write permissions" - - Save - -3. **Verify workflow files**: - - ```bash - # Check files were created - ls -la .github/workflows/ - - # Validate YAML syntax - cat .github/workflows/agentready-assessment.yml - ``` - -4. **Trigger manually**: - - Actions tab → Select workflow → "Run workflow" - ---- - -#### "Pre-commit hooks not running" - -**Cause**: Hooks not installed locally. - -**Solution**: - -```bash -# Install pre-commit framework -pip install pre-commit - -# Install git hooks -pre-commit install - -# Verify installation -ls -la .git/hooks/ -# Should see pre-commit file - -# Test hooks -pre-commit run --all-files -``` - -**If hooks fail:** - -```bash -# Update hook versions -pre-commit autoupdate - -# Clear cache -pre-commit clean - -# Reinstall -pre-commit uninstall -pre-commit install -``` - ---- - -#### "Dependabot PRs not appearing" - -**Cause**: Dependabot not enabled for repository or incorrect config. - -**Solution**: - -1. **Check Dependabot is enabled**: - - Repository Settings → Security & analysis - - Enable "Dependabot alerts" and "Dependabot security updates" - -2. **Verify config**: - - ```bash - cat .github/dependabot.yml - - # Should have correct package-ecosystem: - # - pip (for Python) - # - npm (for JavaScript) - # - gomod (for Go) - ``` - -3. **Check for existing dependency issues**: - - Security tab → Dependabot - - View pending updates - -4. **Manual trigger**: - - Wait up to 1 week for first scheduled run - - Or manually trigger via GitHub API - ---- - -#### "CODEOWNERS not assigning reviewers" - -**Cause**: Invalid usernames or team names in CODEOWNERS. - -**Solution**: - -```bash -# Edit CODEOWNERS -vim .github/CODEOWNERS - -# Use valid GitHub usernames (check they exist) -* @alice @bob - -# Or use teams (requires org ownership) -* @myorg/team-name - -# Verify syntax -# Each line: -*.py @python-experts -/docs/ @documentation-team -``` - -**Common mistakes:** - -- Using email instead of GitHub username -- Typo in username -- Team name without org prefix (@myorg/team) -- Missing @ symbol - ---- - -#### "Assessment workflow failing" - -**Cause**: Various potential issues with workflow execution. - -**Solution**: - -1. **Check workflow logs**: - - Actions tab → Select failed run → View logs - -2. **Common failures**: - - **Python not found:** - - ```yaml - # In .github/workflows/agentready-assessment.yml - # Ensure correct Python version - - uses: actions/setup-python@v4 - with: - python-version: '3.11' # Or '3.12' - ``` - - **AgentReady not installing:** - - ```yaml - # Check pip install step - - run: pip install agentready - - # Or use specific version - - run: pip install agentready==1.1.0 - ``` - - **Permission denied:** - - ```yaml - # Add permissions to workflow - permissions: - contents: read - pull-requests: write # For PR comments - ``` - -3. **Test locally**: - - ```bash - # Run same commands as workflow - pip install agentready - agentready assess . - ``` - ---- +**"CODEOWNERS not assigning reviewers"** — Edit `.github/CODEOWNERS` with valid usernames (`* @alice @bob`) ### Report Issues -If you encounter issues not covered here: - -1. **Check GitHub Issues**: [github.com/ambient-code/agentready/issues](https://github.com/ambient-code/agentready/issues) -2. **Search Discussions**: Someone may have encountered similar problems -3. **Create New Issue**: Use the bug report template with: - - AgentReady version (`agentready --version`) - - Python version (`python --version`) - - Operating system - - Complete error message - - Steps to reproduce +[GitHub Issues](https://github.com/ambient-code/agentready/issues) — Include AgentReady/Python version, OS, error message, and steps to reproduce. ---