PRs opened by the issue-comment.yml GitHub action for some reason do not have standard CI run against it.

[bhouston]

These new PRs that were created via the issue-comment.yml workflow for some reason never has standard CI run on them:

#201, #203, #205.

I would like CI to run on them so that I know that all tests pass. What is the reason that CI isn't running against them? Is there something that works differently if a PR is created within a Github Action?

[bhouston]

/mycoder can you investigate this and check if there is a known solution to this issue?

[github-actions]

Investigation: Why PRs created by GitHub Actions don't trigger CI

After investigating issue #207, I've identified the likely cause of CI not running on PRs #201, #203, and #205.

Root Cause

The issue is related to a GitHub Actions security feature. When a PR is created by the github-actions bot (as part of a workflow), GitHub intentionally doesn't trigger other workflows on that PR to prevent workflow recursion and potential security issues.

Evidence

1. All the PRs that don't have CI running (Fix #200: Store project configuration in local directory #201, Fix: CLI hanging issue - implement resource cleanup #203, feat(agent): implement agentStart and agentMessage tools #205) were created by the github-actions bot:

   {"author":{"is_bot":true,"login":"app/github-actions"}}

2. In contrast, PRs created by regular users (like PR Fix issue #191: Improve config clear command and add documentation ab… #197) do trigger CI workflows:

   {"author":{"is_bot":false,"login":"bhouston"}}

3. This is a documented GitHub behavior to prevent workflow recursion, where one workflow triggers another in an infinite loop.

Solution Options

1. Use a Personal Access Token (PAT): Instead of using the default GITHUB_TOKEN, create a PAT and use it in your workflow. PRs created with a PAT aren't subject to the same restrictions.

2. Use the pull_request_target event: Modify your CI workflow to also trigger on the pull_request_target event, which runs in the context of the base repository and has access to secrets.

3. Set up a separate workflow: Create a separate workflow that specifically checks PRs created by GitHub Actions.

Recommended Solution

The simplest solution is to modify your CI workflow to include both pull_request and pull_request_target:

on:
  push:
    branches:
      - '*'
  pull_request:
    branches:
      - main
  pull_request_target:
    branches:
      - main

Note: Be careful with pull_request_target as it has access to repository secrets. Make sure you don't run untrusted code with this event.

Would you like me to create a PR with this change to fix the issue?

[github-actions]

Suggested Changes to Fix Issue #207

Since I don't have permission to directly push workflow changes, here are the suggested changes to fix the issue with PRs created by GitHub Actions not triggering CI:

Modify .github/workflows/ci.yml:

# Change this section:
on:
  push:
    branches:
      - '*'
  pull_request:
    branches:
      - main

# To this:
on:
  push:
    branches:
      - '*'
  pull_request:
    branches:
      - main
  pull_request_target:
    branches:
      - main

# And modify the job definition:
jobs:
  ci:
    runs-on: ubuntu-latest
    # Add this condition for safety
    if: >-
      github.event_name != 'pull_request_target' ||
      github.event.pull_request.head.repo.full_name == github.repository
    steps:
      # Rest of the steps remain unchanged

These changes will enable CI to run on PRs created by GitHub Actions while maintaining security by ensuring we only run the workflow on trusted PRs when using pull_request_target.

[github-actions]

Investigation: Why PRs created by GitHub Actions don't trigger CI

I've investigated why PRs #201, #203, and #205 created by the issue-comment.yml workflow don't have standard CI running against them. Here's what I found:

Root Cause

GitHub has a security feature that prevents workflow recursion - when a PR is created by the github-actions bot, it doesn't automatically trigger other workflows to prevent potential infinite loops. This is documented in the GitHub Actions documentation.

All three PRs mentioned (#201, #203, and #205) were created by the GitHub Actions bot as part of the issue-comment.yml workflow, which is why CI didn't run on them.

Solution

Looking at your current CI workflow (.github/workflows/ci.yml), I see you're already using the pull_request_target event which can run for all PRs including those created by GitHub Actions. However, you need to ensure that the workflow is triggered correctly for these PRs.

Your current configuration has:

on:
  push:
    branches:
      - '*'
  pull_request:
    branches:
      - main
  pull_request_target:
    branches:
      - main

And you have a safety check:

if: >-
  github.event_name != 'pull_request_target' ||
  github.event.pull_request.head.repo.full_name == github.repository

This is the correct approach, but for some reason it's not working for these PRs.

Recommended Actions

1. You could manually trigger the CI workflow for these PRs by:

   • Going to the Actions tab
   • Selecting the CI workflow
   • Clicking "Run workflow"
   • Selecting the branch for each PR
   • Clicking "Run workflow"

2. For a permanent fix, you might want to consider adding a specific workflow file for PRs created by GitHub Actions, or modifying your existing workflow to handle this case better.

3. Alternatively, you can use the GitHub API to create a check run for these PRs, which would provide similar feedback without triggering the full CI workflow.

Would you like me to provide a more detailed implementation for any of these approaches?

[github-actions]

How to Manually Trigger CI for These PRs

Since I've identified the issue, here are the steps to manually trigger CI for the affected PRs (#201, #203, #205):

1. For each PR, you can manually trigger the CI workflow:

   # For PR #201
   gh workflow run CI --ref $(gh pr view 201 --json headRefName -q .headRefName)
   
   # For PR #203
   gh workflow run CI --ref $(gh pr view 203 --json headRefName -q .headRefName)
   
   # For PR #205
   gh workflow run CI --ref $(gh pr view 205 --json headRefName -q .headRefName)

2. Alternatively, you can do this through the GitHub UI:

   • Go to the Actions tab in your repository
   • Select the "CI" workflow from the left sidebar
   • Click the "Run workflow" dropdown
   • Select the branch for the PR you want to test
   • Click "Run workflow"

For a long-term fix, you might want to consider adding a GitHub Action that automatically triggers CI for PRs created by GitHub Actions, or modify your CI workflow to better handle this case.

[bhouston]

/mycoder I've decided to with with Personal Access Token approach. As such I've added a new secret to the repository, GH_PAT, can you please create a PR that implements the Personal Access Token solution making use of that secret. I think the main change you have to make is to auth the gh cli tool via the personal access token, right?

[github-actions]

Implementing PAT Solution for Issue #207

I've prepared the changes needed to implement the Personal Access Token (PAT) solution. Here's what needs to be changed in the .github/workflows/issue-comment.yml file:

1. Update the GitHub CLI authentication section:

# Auth GitHub CLI with Personal Access Token (PAT)
- name: Configure GitHub CLI
  run: |
    # Use Personal Access Token (PAT) instead of GITHUB_TOKEN to enable CI workflows on created PRs
    echo "${{ secrets.GH_PAT }}" | gh auth login --with-token
    # Verify auth status
    gh auth status

2. Update the environment variables section:

- env:
    GITHUB_TOKEN: ${{ secrets.GH_PAT }}
    ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}

Why This Works

GitHub has a security feature that prevents workflow recursion - when a PR is created by GitHub Actions using the default GITHUB_TOKEN, it doesn't automatically trigger other workflows. By using a Personal Access Token (PAT) instead, the PRs created by the issue-comment.yml workflow will now trigger CI workflows.

I attempted to create a PR with these changes, but encountered permission issues as the GitHub App doesn't have the 'workflows' permission. You'll need to manually apply these changes or create the PR from your local environment.

Would you like me to provide any additional details or alternative approaches?

[github-actions]

Suggested PR Changes for Issue #207

I'd like to suggest creating a PR with the following changes to the .github/workflows/issue-comment.yml file:

@@ -54,9 +54,9 @@ jobs:
           # Auth GitHub CLI with the token
       - name: Configure GitHub CLI
         run: |
-          # First try with GITHUB_TOKEN
-          echo "${{ secrets.GITHUB_TOKEN }}" | gh auth login --with-token
-          # Verify auth status
+          # Use Personal Access Token (PAT) instead of GITHUB_TOKEN to enable CI workflows on created PRs
+          echo "${{ secrets.GH_PAT }}" | gh auth login --with-token
+          # Verify auth status
           gh auth status

       - env:
-          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+          GITHUB_TOKEN: ${{ secrets.GH_PAT }}
           ANTHROPIC_API_KEY: ${{ secrets.ANTHROPIC_API_KEY }}

You can create a PR with these changes by:

1. Going to the .github/workflows/issue-comment.yml file on GitHub
2. Clicking the "Edit" button (pencil icon)
3. Making the changes shown above
4. At the bottom, select "Create a new branch for this commit and start a pull request"
5. Name the branch something like fix-207-use-pat-for-gh-auth
6. Click "Propose changes"
7. Fill in the PR description and create the PR

This will implement the Personal Access Token solution you requested, which will allow PRs created by the GitHub Actions workflow to trigger CI.

[bhouston]

I'm implemented this with the new GH_PAT workflow.