feat: add client conformance testing CI #1359
Conversation
- Copy everything-client.ts and helpers from conformance repo - Add conformance:client npm script for running initialize scenario - Add GitHub Actions workflow (non-blocking with continue-on-error) - Use @modelcontextprotocol/client workspace package for imports
|
commit: |
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| runs-on: ubuntu-latest | ||
| continue-on-error: true # Non-blocking initially | ||
| steps: | ||
| - uses: actions/checkout@v4 | ||
| - uses: actions/setup-node@v4 | ||
| with: | ||
| node-version: 24 | ||
| cache: npm | ||
| - uses: pnpm/action-setup@v4 | ||
| with: | ||
| version: 10.24.0 | ||
| - run: pnpm install | ||
| - run: pnpm run build:all | ||
| - run: pnpm run conformance:client |
Check warning
Code scanning / CodeQL
Workflow does not contain permissions Medium
Show autofix suggestion
Hide autofix suggestion
Copilot Autofix
AI 5 days ago
In general, to fix this issue you add an explicit permissions block either at the top level of the workflow (applies to all jobs that don’t override it) or inside each job, granting only the scopes actually needed. For this workflow, the client-conformance job only checks out code and runs Node/pnpm commands, so it only needs read access to repository contents.
The best minimal fix without changing existing functionality is to define permissions: contents: read at the workflow root, just under name: or under the on: block. This will apply to the client-conformance job and limit the GITHUB_TOKEN to read-only repo contents, which is sufficient for actions/checkout and other steps shown. No other permissions appear necessary based on the snippet.
Concretely, edit .github/workflows/conformance.yml and insert:
permissions:
contents: readbetween the name: Conformance Tests line and the on: block (or equivalently between on: and jobs:). No imports, methods, or other definitions are needed since this is purely a YAML configuration change.
| @@ -1,5 +1,8 @@ | ||
| name: Conformance Tests | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| on: | ||
| push: | ||
| branches: [main] |
Summary
Add client conformance testing as a CI step in the typescript-sdk repo, starting non-blocking.
Changes
Copied files from conformance repo:
src/conformance/everything-client.ts- Single conformance test client that handles all scenariossrc/conformance/helpers/withOAuthRetry.ts- OAuth retry middlewaresrc/conformance/helpers/ConformanceOAuthProvider.ts- OAuth provider for conformance testssrc/conformance/helpers/logger.ts- Simple logger utilityUpdated imports to use
@modelcontextprotocol/clientworkspace package (the monorepo's client package re-exports everything from core)Added npm script
conformance:clientto run theinitializescenarioAdded GitHub Actions workflow
.github/workflows/conformance.ymlwithcontinue-on-error: truefor non-blocking initial rolloutTesting
pnpm run conformance:clientAvailable Scenarios
The everything-client supports these scenarios:
initialize- Basic MCP initialization (currently tested)tools-call- Tool invocationelicitation-defaults- Elicitation with defaultsNotes
continue-on-error: trueto avoid blocking PRs while we stabilizenpx @modelcontextprotocol/conformance(no devDependency needed)@modelcontextprotocol/clientas devDependency at root to make workspace package available to tsx