Add agent-runner environment parity test suite

[Copilot]

Automated validation that the agent container environment maintains parity with GitHub Actions runners for utilities, runtimes, environment variables, and shared libraries. Prevents regressions as we close the gap between agent and runner environments.

Integration Tests (pkg/workflow/agent_parity_test.go)

Five test functions validate workflow compilation for environment parity checks:

• Utilities: 12 essential tools (jq, curl, git, wget, tar, gzip, unzip, sed, awk, grep, find, xargs)
• Runtimes: Node.js, Python, Go, Ruby version checks
• Environment Variables: JAVA_HOME, ANDROID_HOME, GOROOT, PATH, HOME, USER
• Shared Libraries: ldd validation for python3, node, git, curl binaries
• Comprehensive: All categories in single workflow

Tests generate workflow markdown, compile to lock files, and assert expected content is present.

Smoke Test Workflow (.github/workflows/agent-runner-parity.md)

Production validation workflow that:

• Runs every 6 hours and on workflow_dispatch
• Triggers on "test-parity" PR label
• Executes validation commands via Copilot + bash tools
• Posts concise pass/fail summaries as comments

Example output format:

🔍 Agent-Runner Environment Parity Test Results

Utilities: 11/12 ❌
  - Missing: unzip

Runtimes: 4/4 ✅
Environment Variables: 4/5 ❌
  - Missing: ANDROID_HOME
  
Shared Libraries: 4/4 ✅

Overall: FAIL ❌

Documentation (specs/agent-container-testing.md)

Coverage tables for all test categories, test execution flows, extension instructions, troubleshooting guide.

Build Integration

make test-parity runs integration tests with proper build tags and filters.

Original prompt

---

This section details on the original issue you should resolve

<issue_title>[plan] Create integration test suite for agent-runner environment parity</issue_title>
<issue_description>## Objective

Develop a comprehensive integration test suite that validates the agent container environment has parity with the GitHub Actions runner environment for essential tools, libraries, and environment variables.

Context

As we close the gap between the agent container and the action runner, we need automated tests to verify that:

1. Mounted utilities are accessible and functional
2. Runtime tools (Python, Node.js, Go, Ruby) can be found and executed
3. Environment variables are correctly mirrored
4. Shared libraries are properly linked

Without these tests, we risk regressions and incomplete implementations.

Approach

1. Create a test workflow that runs inside the agent container and verifies:
   • Essential /usr/bin utilities are accessible (jq, curl, git, etc.)
   • All runtime binaries are in PATH and executable (node, python3, go, ruby)
   • Environment variables are set correctly (JAVA_HOME, ANDROID_HOME, etc.)
   • Shared libraries can be loaded (test with ldd on key binaries)
2. Add Go integration tests in pkg/workflow/*_test.go files
3. Create a smoke test workflow in .github/workflows/agent-runner-parity.md
4. Document the test approach in specs/agent-container-testing.md

Files to Create/Modify

• Create: pkg/workflow/agent_parity_test.go (integration tests)
• Create: .github/workflows/agent-runner-parity.md (smoke test workflow)
• Create: specs/agent-container-testing.md (test documentation)
• Modify: Makefile (add make test-parity target)

Acceptance Criteria

• Integration tests cover utility accessibility (at least 10 utilities)
• Integration tests verify runtime availability (node, python3, go, ruby)
• Integration tests validate environment variable presence (at least 5 vars)
• Smoke test workflow runs successfully in CI
• Test documentation explains testing strategy and coverage
• All tests pass consistently across multiple runs
  Related to epic: build/test environment for agentic workflow #11970

AI generated by Plan Command for #11970

Comments on the Issue (you are @copilot in this section)

• Fixes [plan] Create integration test suite for agent-runner environment parity #11975

---

💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

[github-actions]

🔍 PR Triage Results

Category: test | Risk: low | Priority: 47/100

Scores Breakdown

• Impact: 25/50 - Test improvement - prevents regressions
• Urgency: 7/30 - Recent PR, CI unstable
• Quality: 15/20 - CI passing

📋 Recommended Action: batch_review

Excellent addition of comprehensive integration tests for agent-runner environment parity. This PR validates that the agent container has access to essential utilities, runtimes, environment variables, and shared libraries.

Test Coverage:

• 12 essential utilities (jq, curl, git, etc.)
• Runtime checks (Node.js, Python, Go, Ruby)
• Environment variables (JAVA_HOME, GOROOT, etc.)
• Shared library validation

Related PRs: This test suite complements #12062 and #12085 which mount binaries and libraries.

---

Triaged by PR Triage Agent on 2026-01-28T00:34:49Z

AI generated by PR Triage Agent

[github-actions]

🔍 PR Triage Results

Category: test | Risk: low | Priority: 37/100

Scores Breakdown

• Impact: 25/50 - Test improvements prevent regressions in agent-runner environment parity
• Urgency: 5/30 - PR is 7 hours old, standard urgency
• Quality: 7/20 - Draft status, CI status unknown

📋 Recommended Action: defer

Rationale: This PR adds comprehensive integration tests and smoke test workflows for agent-runner environment parity (utilities, runtimes, environment variables, shared libraries). While important for preventing regressions, it should remain deferred until out of draft. The changes are substantial (2,340 additions across 7 files) and need CI validation before review.

Next steps: Mark as ready for review when all integration tests pass and CI is green.

---

Triaged by PR Triage Agent on 2026-01-28

AI generated by PR Triage Agent