Add a new prompt & tool for diagnosing FPs/FNs from query runs. by MichaelRFairhurst · Pull Request #70 · advanced-security/codeql-development-mcp-server

MichaelRFairhurst · 2026-02-22T23:39:18Z

Rather than relying on SARIF, give the MCP server a tool to allow agents to read the source code from the zip archive of a database.

Create a prompt that instructs the agent to try to find a prior runs, rerun if necessary, and then use this tool to find FPs/FNs.

Example results of usage:

False Positive Analysis on gRPC
Redis: 0 results — clean.

gRPC: 290 results — all false positives, one root cause.

Root cause: ABSL_DCHECK → assertion failure handler → demangle.cc
Every alert points to MapKeyComparator::operator() at wire_format.cc:1176:

// wire_format.cc ~1174
bool operator()(const MapKey& a, const MapKey& b) const {
    ABSL_DCHECK(a.type() == b.type());   // <-- flagged
        switch (a.type()) { ...compare... }
    }

The ABSL_DCHECK macro expands to code that, on assertion failure, invokes a diagnostic handler that calls into abseil's stack symbolizer, which calls Demangle() in demangle.cc. That function operates on a locally-allocated State* struct:

// demangle.cc ~319
static void InitState(State* state, ...) {
    state->mangled_begin = mangled;  // AliasVariableModification
    state->out = out;                // AliasVariableModification
    state->recursion_depth = 0;      // AliasVariableModification
    ...
}

This produces ~290 duplicate alerts for one call site, one per pointer-write in demangle.cc.

There are two compounding false positive mechanisms here:

Mechanism	Problem
`AliasVariableModification`	Any pointer/reference parameter write is treated as a `GlobalSideEffect`. Here `State*` is locally allocated — not a persistent global. The query can't distinguish pointer-to-local from pointer-to-global.
Transitive assertion paths	`ABSL_DCHECK` is only executed on crash/assertion paths in debug builds. The query treats all reachable call paths equally, with no distinction for assertion/diagnostic code.

Recommended fix: The most impactful single change would be to restrict to direct side effects only — replacing getAnExternalOrGlobalSideEffectInFunction(f) (which recurses through all callees) with only effects where effect.getEnclosingFunction() = f. This eliminates all transitive false positives at the cost of missing cases where a predicate calls a small helper that performs a side effect.

github-actions · 2026-02-22T23:39:30Z

Dependency Review

The following issues were found:

✅ 0 vulnerable package(s)
✅ 0 package(s) with incompatible licenses
✅ 0 package(s) with invalid SPDX license definitions
⚠️ 1 package(s) with unknown licenses.

See the Details below.

License Issues

server/package.json

Package	Version	License	Issue Type
adm-zip	^0.5.16	Null	Unknown License

OpenSSF Scorecard

Package

Version

Score

Details

npm/@types/adm-zip

0.5.7

🟢 6.4

Details

Check	Score	Reason
Packaging	⚠️ -1	packaging workflow not detected
CII-Best-Practices	⚠️ 0	no effort to earn an OpenSSF best practices badge detected
Code-Review	🟢 7	Found 21/29 approved changesets -- score normalized to 7
Maintained	🟢 10	30 commit(s) and 1 issue activity found in the last 90 days -- score normalized to 10
License	🟢 9	license file detected
Token-Permissions	⚠️ 0	detected GitHub workflow tokens with excessive permissions
Branch-Protection	⚠️ -1	internal error: error during branchesHandler.setup: internal error: some github tokens can't read classic branch protection rules: https://github.com/ossf/scorecard-action/blob/main/docs/authentication/fine-grained-auth-token.md
Security-Policy	🟢 10	security policy file detected
Dangerous-Workflow	🟢 10	no dangerous workflow patterns detected
Signed-Releases	⚠️ -1	no releases found
SAST	⚠️ 0	SAST tool is not run on all commits -- score normalized to 0
Binary-Artifacts	🟢 10	no binaries found in the repo
Pinned-Dependencies	🟢 8	dependency not pinned by hash detected -- score normalized to 8
Fuzzing	⚠️ 0	project is not fuzzed

npm/adm-zip

0.5.16

🟢 3.6

Details

Check	Score	Reason
Code-Review	🟢 3	Found 6/19 approved changesets -- score normalized to 3
Maintained	⚠️ 0	0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0
Dangerous-Workflow	🟢 10	no dangerous workflow patterns detected
Packaging	⚠️ -1	packaging workflow not detected
Binary-Artifacts	🟢 10	no binaries found in the repo
Token-Permissions	⚠️ 0	detected GitHub workflow tokens with excessive permissions
Pinned-Dependencies	⚠️ 0	dependency not pinned by hash detected -- score normalized to 0
CII-Best-Practices	⚠️ 0	no effort to earn an OpenSSF best practices badge detected
Security-Policy	⚠️ 0	security policy file not detected
License	🟢 10	license file detected
Fuzzing	⚠️ 0	project is not fuzzed
Signed-Releases	⚠️ -1	no releases found
Branch-Protection	⚠️ 0	branch protection not enabled on development/release branches
SAST	🟢 7	SAST tool detected but not run on all commits

npm/@types/adm-zip

^0.5.7

Unknown

npm/adm-zip

^0.5.16

Unknown

Scanned Files

package-lock.json
server/package.json

MichaelRFairhurst · 2026-02-22T23:40:08Z

Mostly written by a local agent, especially the zip decoding stuff.

Copilot

Pull request overview

This PR adds a new MCP tool and workflow prompt to help developers analyze false positives in their CodeQL query results by reading source code directly from database archives. The implementation enables agents to explore code at alert locations without requiring the original source tree, which is particularly valuable for diagnosing query precision issues.

Changes:

Added read_database_source tool to read source files from CodeQL database archives (src.zip or src/ directory)
Added run_query_and_summarize_false_positives prompt to guide agents through FP analysis workflows
Added comprehensive test coverage with 12 test cases covering both src.zip and src/ directory modes

Reviewed changes

Copilot reviewed 10 out of 11 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
server/src/tools/codeql/read-database-source.ts	New tool implementation with archive reading, path resolution, and line-range slicing capabilities
server/test/src/tools/codeql/read-database-source.test.ts	Comprehensive test suite covering listing, reading, URI resolution, line ranges, and error cases
server/src/tools/codeql/index.ts	Export for the new read_database_source tool
server/src/tools/codeql-tools.ts	Registration of the new tool in alphabetical order
server/src/prompts/run-query-and-summarize-false-positives.prompt.md	New prompt with YAML frontmatter, task instructions, and output format guidance
server/src/prompts/workflow-prompts.ts	Schema and registration for the new prompt
server/test/src/tools/codeql-tools.test.ts	Updated tool count from 34 to 35
server/test/src/prompts/workflow-prompts.test.ts	Updated prompt count from 10 to 11
server/package.json	Added adm-zip dependency and `@types/adm-zip` dev dependency
package-lock.json	Added dependency entries for adm-zip packages; contains engine version inconsistency
docs/ql-mcp/prompts.md	Updated prompt count and added new prompt to reference table

server/src/prompts/run-query-and-summarize-false-positives.prompt.md

docs/ql-mcp/prompts.md

server/src/prompts/run-query-and-summarize-false-positives.prompt.md

Copilot

Pull request overview

Copilot reviewed 10 out of 13 changed files in this pull request and generated 7 comments.

server/src/tools/codeql/read-database-source.ts

server/src/prompts/run-query-and-summarize-false-positives.prompt.md

server/src/prompts/workflow-prompts.ts

server/test/src/prompts/workflow-prompts.test.ts

server/src/tools/codeql/read-database-source.ts

Rather than relying on SARIF, give the MCP server a tool to allow agents to read the source code from the zip archive of a database. Create a prompt that instructs the agent to try to find a prior runs, rerun if necessary, and then use this tool to find FPs/FNs.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Nathan Randall <70299490+data-douser@users.noreply.github.com>

Adds client integration tests for the new 'read_database_source' MCP server tool.

Copilot

Pull request overview

Copilot reviewed 10 out of 13 changed files in this pull request and generated 5 comments.

server/test/src/tools/codeql/read-database-source.test.ts

server/src/prompts/run-query-and-summarize-false-positives.prompt.md

docs/ql-mcp/prompts.md

server/src/prompts/workflow-prompts.ts

Copilot

Pull request overview

Copilot reviewed 16 out of 20 changed files in this pull request and generated 6 comments.

server/src/prompts/run-query-and-summarize-false-positives.prompt.md

server/src/prompts/workflow-prompts.ts

server/src/tools/codeql/read-database-source.ts

server/test/src/tools/codeql/read-database-source.test.ts

…lse-positives

- Register run-query-and-summarize-false-positives prompt in PROMPT_TEMPLATES so it is embedded in the bundle instead of returning the fallback message - Replace backtick tool references with #tool_name form in the prompt template for consistent Copilot Chat rendering - Fix CRLF line splitting in applyLineRange (split on /\r?\n/) - Add default listing cap (1000 entries) to prevent oversized MCP responses from large src.zip archives - Enforce max uncompressed size (10 MB) before decompressing zip entries - Use dirname() instead of join(.., '..') in test helper for clarity

Copilot

Pull request overview

Copilot reviewed 17 out of 21 changed files in this pull request and generated 2 comments.

Copilot · 2026-02-23T12:09:23Z

server/src/tools/codeql/read-database-source.ts

+function* walkDirectory(dir: string, base: string = dir): Generator<string> {
+  for (const entry of readdirSync(dir)) {
+    const fullPath = join(dir, entry);
+    if (statSync(fullPath).isDirectory()) {
+      yield* walkDirectory(fullPath, base);
+    } else {
+      // Return path relative to base, normalised with forward slashes
+      yield fullPath.slice(base.length).replace(/\\/g, '/').replace(/^\//, '');
+    }
+  }
+}


The walkDirectory function does not explicitly check for or skip symbolic links when recursively traversing the src/ directory. This could potentially allow reading files outside the intended database source directory if symlinks are present. Consider adding a check to skip symlinks or resolve them safely within the database boundary.

For example, you could check lstatSync instead of statSync to detect symlinks before following them, or verify that resolved paths remain within the database directory.

Copilot · 2026-02-23T12:09:23Z

server/src/tools/codeql/read-database-source.ts

+  // 3. Suffix match — collect all candidates, prefer the longest (most specific)
+  const candidates: { entry: string; length: number }[] = [];
+  for (const entry of available) {
+    const entryNorm = entry.replace(/^\//, '');
+    if (entryNorm.endsWith(withoutLeading) || withoutLeading.endsWith(entryNorm)) {
+      candidates.push({ entry, length: entryNorm.length });
+    }
+  }


The suffix matching logic at line 143 uses a bidirectional match condition: entryNorm.endsWith(withoutLeading) || withoutLeading.endsWith(entryNorm). The second condition (withoutLeading.endsWith(entryNorm)) could match unintended entries when requesting a longer path that happens to end with a shorter entry name.

For example, if requesting foo/bar/File.java and the archive contains just File.java, this would match even though the requested path has additional directory components. This behavior seems intentional based on the comment about "handles absolute SARIF URIs", but the bidirectional match could lead to unexpected results in some cases.

Consider documenting this behavior more explicitly or restricting the second condition to only apply when it makes sense semantically.

Copilot AI review requested due to automatic review settings February 22, 2026 23:39

MichaelRFairhurst requested review from a team, data-douser and enyil as code owners February 22, 2026 23:39

Copilot started reviewing on behalf of MichaelRFairhurst February 22, 2026 23:39 View session

Copilot AI reviewed Feb 22, 2026

View reviewed changes

server/src/prompts/run-query-and-summarize-false-positives.prompt.md Outdated Show resolved Hide resolved

docs/ql-mcp/prompts.md Outdated Show resolved Hide resolved

server/src/prompts/run-query-and-summarize-false-positives.prompt.md Outdated Show resolved Hide resolved

Copilot AI review requested due to automatic review settings February 23, 2026 00:24

Copilot started reviewing on behalf of data-douser February 23, 2026 00:25 View session

Copilot AI reviewed Feb 23, 2026

View reviewed changes

MichaelRFairhurst and others added 4 commits February 23, 2026 04:05

Commit bundled changes

8e53674

Apply suggestions from code review

5003f24

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com> Signed-off-by: Nathan Randall <70299490+data-douser@users.noreply.github.com>

Fix tidy and sync package-lock.json

1b02f68

Copilot AI review requested due to automatic review settings February 23, 2026 11:07

data-douser force-pushed the michaelrfairhurst/run-query-and-summarize-false-positives branch from b85e017 to 1b02f68 Compare February 23, 2026 11:07

Copilot started reviewing on behalf of data-douser February 23, 2026 11:07 View session

Integration tests for read_database_source tool

21ab5d3

Adds client integration tests for the new 'read_database_source' MCP server tool.

Copilot AI reviewed Feb 23, 2026

View reviewed changes

Address PR review comments

10e85e4

Copilot AI review requested due to automatic review settings February 23, 2026 11:38

Copilot started reviewing on behalf of data-douser February 23, 2026 11:39 View session

Copilot AI reviewed Feb 23, 2026

View reviewed changes

data-douser approved these changes Feb 23, 2026

View reviewed changes

data-douser and others added 2 commits February 23, 2026 04:56

Merge branch 'main' into michaelrfairhurst/run-query-and-summarize-fa…

8e4cc04

…lse-positives

Copilot AI review requested due to automatic review settings February 23, 2026 12:05

Copilot started reviewing on behalf of data-douser February 23, 2026 12:05 View session

Copilot AI reviewed Feb 23, 2026

View reviewed changes

data-douser merged commit de790bb into main Feb 23, 2026
19 checks passed

data-douser deleted the michaelrfairhurst/run-query-and-summarize-false-positives branch February 23, 2026 12:24

Comments

Conversation

MichaelRFairhurst commented Feb 22, 2026

Uh oh!

github-actions bot commented Feb 22, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Dependency Review

License Issues

server/package.json

OpenSSF Scorecard

Scanned Files

Uh oh!

MichaelRFairhurst commented Feb 22, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Feb 23, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 23, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

github-actions bot commented Feb 22, 2026 •

edited

Loading