From b57b962b10664d112e31d9991a2a3271095e42d6 Mon Sep 17 00:00:00 2001 From: "google-labs-jules[bot]" <161369871+google-labs-jules[bot]@users.noreply.github.com> Date: Mon, 1 Dec 2025 18:00:06 +0000 Subject: [PATCH] Remove internal documentation and reports for public release Deleted: - ai_docs/ directory - CACHE_FEATURE.md - tests/AST_SEARCH_TESTS.md - TEST_VERIFICATION_REPORT.md --- CACHE_FEATURE.md | 404 --------- TEST_VERIFICATION_REPORT.md | 283 ------ ai_docs/BLUEPRINT.md | 22 - ai_docs/COMPLETION_REQUIREMENTS.md | 11 - ai_docs/LANGUAGE_AWARE_SEARCH.md | 378 -------- ai_docs/STACK_DETECTION_SYSTEM.md | 425 --------- ai_docs/stacks.json.example | 1311 ---------------------------- tests/AST_SEARCH_TESTS.md | 296 ------- 8 files changed, 3130 deletions(-) delete mode 100644 CACHE_FEATURE.md delete mode 100644 TEST_VERIFICATION_REPORT.md delete mode 100644 ai_docs/BLUEPRINT.md delete mode 100644 ai_docs/COMPLETION_REQUIREMENTS.md delete mode 100644 ai_docs/LANGUAGE_AWARE_SEARCH.md delete mode 100644 ai_docs/STACK_DETECTION_SYSTEM.md delete mode 100644 ai_docs/stacks.json.example delete mode 100644 tests/AST_SEARCH_TESTS.md diff --git a/CACHE_FEATURE.md b/CACHE_FEATURE.md deleted file mode 100644 index 023b532..0000000 --- a/CACHE_FEATURE.md +++ /dev/null @@ -1,404 +0,0 @@ -# Index Persistence and Caching Feature - -## Overview - -The Code Search MCP server now includes a robust caching system that persists symbol indices to disk, providing **80%+ faster startup times** for large workspaces. This feature dramatically improves the developer experience by eliminating the need to rebuild indices on every server restart. - -## Features - -### ✅ Production-Grade Performance -- **80%+ startup time improvement** for cached workspaces -- Automatic cache invalidation on file changes -- Efficient serialization/deserialization of large indices -- Cross-platform support (Windows, macOS, Linux) - -### ✅ Robust Cache Management -- **Version Control**: Cache format versioning for future compatibility -- **Corruption Handling**: Graceful recovery from corrupted cache files -- **Automatic Invalidation**: Detects file modifications, additions, and deletions -- **Workspace Isolation**: Each workspace maintains its own independent cache - -### ✅ Multi-Language Support -All 12 supported languages are fully cached: -- Java, Python, JavaScript, TypeScript, C# -- Go, Rust, C, C++, PHP, Ruby, Kotlin - -## New MCP Tools - -### `cache_stats` - View Cache Statistics - -Get detailed statistics about cached indices. - -**Usage:** -```json -// Get stats for a specific workspace -{ - "workspace_id": "ws-1" -} - -// Get stats for all workspaces (omit workspace_id) -{} -``` - -**Response:** -```json -{ - "workspace_id": "ws-1", - "workspace_path": "/path/to/project", - "total_symbols": 15234, - "last_indexed": "2025-01-18T10:30:45.123Z", - "cache_size_bytes": 2457600, - "cache_age_ms": 3600000, - "file_count": 487, - "is_cached": true -} -``` - -### `clear_cache` - Clear Cached Indices - -Clear cached indices to force a rebuild. - -**Usage:** -```json -// Clear cache for a specific workspace -{ - "workspace_id": "ws-1" -} - -// Clear all caches (omit workspace_id) -{} -``` - -**Response:** -```json -{ - "workspace_id": "ws-1", - "message": "Cache cleared successfully" -} -``` - -### `refresh_index` - Rebuild Index (Enhanced) - -The existing `refresh_index` tool now supports a `force_rebuild` parameter. - -**Usage:** -```json -{ - "workspace_id": "ws-1", - "force_rebuild": true // Optional: skip cache and rebuild from scratch -} -``` - -## How It Works - -### Cache Storage - -Caches are stored in `~/.code-search-mcp-cache/` (or `%USERPROFILE%\.code-search-mcp-cache\` on Windows). - -Each workspace gets its own cache file: `{workspaceId}.json` - -### Cache Structure - -```json -{ - "metadata": { - "version": "1.0.0", - "workspaceId": "ws-1", - "workspacePath": "/path/to/project", - "workspaceHash": "sha256-hash", - "lastIndexed": "2025-01-18T10:30:45.123Z", - "fileMtimes": { - "/path/to/file1.js": 1705574445123, - "/path/to/file2.py": 1705574446234 - }, - "totalSymbols": 15234 - }, - "index": { - "byLanguage": { /* serialized symbol index */ }, - "totalSymbols": 15234, - "lastIndexed": "2025-01-18T10:30:45.123Z" - } -} -``` - -### Cache Invalidation - -The cache is automatically invalidated when: - -1. **File Modified**: Any source file's modification time changes -2. **File Added**: New files are added to the workspace -3. **File Deleted**: Files are removed from the workspace -4. **Path Changed**: Workspace path changes -5. **Version Mismatch**: Cache format version doesn't match - -### Automatic Cache Usage - -When you call `search_symbols` on a workspace: - -1. **First Time**: Index is built from scratch and cached -2. **Subsequent Calls**: Index is loaded from cache (if valid) -3. **After Changes**: Cache is invalidated and rebuilt automatically - -No manual intervention required! - -## Performance Benchmarks - -### Test Results - -Performance tests with real repositories demonstrate significant improvements: - -#### Express.js (Popular Node.js Framework) -- **Cold Start**: 2,453ms -- **Cached Start**: 127ms -- **Improvement**: 19.3x faster (94.8% time saved) -- **Symbols**: 8,234 -- **Cache Size**: 1.2 MB - -#### Lodash (JavaScript Utility Library) -- **Cold Start**: 1,876ms -- **Cached Start**: 89ms -- **Improvement**: 21.1x faster (95.3% time saved) -- **Symbols**: 12,456 -- **Cache Size**: 1.8 MB - -#### Synthetic Large Codebase (100 files, 5000 functions) -- **Cold Start**: 3,124ms -- **Cached Start**: 145ms -- **Improvement**: 21.5x faster (95.4% time saved) -- **Symbols**: 5,000 -- **Cache Size**: 0.9 MB - -**Average Improvement: 94.5% time saved** - -## Architecture - -### Components - -1. **CacheManager** (`src/cache/cache-manager.ts`) - - Core caching logic - - Serialization/deserialization - - Invalidation detection - - Statistics collection - -2. **SymbolIndexer** (`src/symbol-search/symbol-indexer.ts`) - - Integrated with CacheManager - - Automatic cache usage - - Fallback to rebuild on cache miss - -3. **MCP Server** (`src/mcp/server.ts`) - - New cache tools: `cache_stats`, `clear_cache` - - Enhanced `refresh_index` with force rebuild - -### Cross-Platform Support - -The cache system is designed to work across all platforms: - -- **Path Handling**: Uses Node.js `path` module for proper path separators -- **Cache Location**: Uses `os.homedir()` for user-specific cache directory -- **File I/O**: All file operations use `fs.promises` for async I/O -- **Hash Validation**: Workspace paths are hashed to detect moves/renames - -### Error Handling - -The cache system is designed to never break the indexing process: - -- **Corrupted Cache**: Falls back to rebuild from scratch -- **Missing Cache**: Builds new cache automatically -- **Version Mismatch**: Rebuilds with new format -- **I/O Errors**: Logged but don't halt indexing - -## Testing - -### Unit Tests (17 tests) - -Located in `tests/unit/cache-manager.test.ts`: - -- ✅ Serialization and deserialization -- ✅ Cache invalidation (file modification, addition, deletion) -- ✅ Version handling -- ✅ Corruption handling -- ✅ Cache statistics -- ✅ Cache clearing -- ✅ Disabled cache mode - -### Integration Tests (8 tests) - -Located in `tests/integration/cache-integration.test.ts`: - -- ✅ Full cache cycle (index → cache → load) -- ✅ Incremental updates -- ✅ Force rebuild -- ✅ Multi-language support -- ✅ Large index performance -- ✅ Concurrent workspace operations - -### Performance Tests (3 tests) - -Located in `tests/integration/cache-performance.test.ts`: - -- ✅ Real repository tests (Express.js, Lodash) -- ✅ Synthetic large codebase test -- ✅ 80%+ improvement validation - -**All 234 tests pass!** - -## Usage Examples - -### Basic Workflow - -```javascript -// Add a workspace -{ - "tool": "add_workspace", - "arguments": { - "path": "/path/to/large/project" - } -} - -// First search - builds index and caches it (may take a few seconds) -{ - "tool": "search_symbols", - "arguments": { - "workspace_id": "ws-1", - "language": "javascript", - "name": "MyClass" - } -} -// Response time: 2,453ms (cold start) - -// Subsequent searches - loads from cache (fast!) -{ - "tool": "search_symbols", - "arguments": { - "workspace_id": "ws-1", - "language": "javascript", - "name": "MyFunction" - } -} -// Response time: 127ms (95% faster!) -``` - -### Check Cache Status - -```javascript -{ - "tool": "cache_stats", - "arguments": { - "workspace_id": "ws-1" - } -} -``` - -### Force Rebuild - -```javascript -{ - "tool": "refresh_index", - "arguments": { - "workspace_id": "ws-1", - "force_rebuild": true - } -} -``` - -### Clear Cache - -```javascript -// Clear specific workspace -{ - "tool": "clear_cache", - "arguments": { - "workspace_id": "ws-1" - } -} - -// Clear all caches -{ - "tool": "clear_cache", - "arguments": {} -} -``` - -## Best Practices - -1. **Let the cache work automatically**: The system handles cache invalidation automatically -2. **Use `cache_stats` to monitor**: Check cache age and size periodically -3. **Use `force_rebuild` sparingly**: Only when you know the cache is stale -4. **Clear cache after major refactors**: If you've renamed/moved many files - -## Troubleshooting - -### Cache not loading? - -Check if files have been modified: -```javascript -{ - "tool": "cache_stats", - "arguments": { - "workspace_id": "ws-1" - } -} -``` - -Look at `cache_age_ms` - if it's old but not loading, try force rebuild. - -### Cache growing too large? - -Clear old workspace caches: -```javascript -{ - "tool": "clear_cache", - "arguments": { - "workspace_id": "old-ws-id" - } -} -``` - -### Performance not as expected? - -1. Check if ctags is installed: `ctags --version` -2. Verify cache is being used: Look for "loaded from cache" in logs -3. Check cache stats for the workspace -4. Try force rebuild: `refresh_index` with `force_rebuild: true` - -## Future Enhancements - -Potential improvements for future versions: - -- **Incremental Updates**: Only re-index changed files instead of full rebuild -- **Compression**: Compress cache files to reduce disk usage -- **TTL/Expiration**: Automatic cache expiration after N days -- **Cache Warmup**: Pre-build caches for common repositories -- **Distributed Cache**: Share caches across team members - -## Technical Details - -### Cache File Format - -- **Format**: JSON with structured metadata -- **Encoding**: UTF-8 -- **Typical Size**: 1-3 MB for medium projects (10k-20k symbols) -- **Location**: `~/.code-search-mcp-cache/{workspaceId}.json` - -### Performance Characteristics - -- **Cache Write**: ~100-200ms for 10k symbols -- **Cache Read**: ~50-150ms for 10k symbols -- **Memory Usage**: Minimal (streamed I/O) -- **Disk Usage**: ~100-200 bytes per symbol - -### Supported File Systems - -- **Local**: All file systems (ext4, NTFS, APFS, etc.) -- **Network**: SMB, NFS (may be slower) -- **Cloud**: Not recommended (high latency) - -## License - -This cache feature is part of the Code Search MCP server and follows the same MIT license. - ---- - -**Implemented**: January 2025 -**Version**: 1.0.0 -**Tested**: 234 tests passing (17 cache-specific unit tests, 8 integration tests, 3 performance tests) diff --git a/TEST_VERIFICATION_REPORT.md b/TEST_VERIFICATION_REPORT.md deleted file mode 100644 index 31756db..0000000 --- a/TEST_VERIFICATION_REPORT.md +++ /dev/null @@ -1,283 +0,0 @@ -# Test Verification Report - -## Status: ✅ TESTS ARE PRODUCTION-READY - -## Summary - -All AST search tests have been written following existing patterns and are ready to run once dependencies are installed. Tests are comprehensive, well-structured, and follow Jest/TypeScript best practices. - -## Test Files Status - -### ✅ tests/unit/ast-search.test.ts -**Status**: Ready -**Lines**: 230 -**Test Cases**: 14 -**Coverage**: Rule validation, availability checks, edge cases - -**Structure**: -```typescript -- describe('ASTSearchService') - - describe('isAvailable') [1 test] - - describe('validateRule') [7 tests] - - describe('Complex rule validation') [2 tests] - - describe('Pattern examples from ast-grep skill') [3 tests] -``` - -**Key Tests**: -- ✅ Availability check with proper type guards -- ✅ Valid pattern/kind/composite/relational rules -- ✅ Invalid rules (no positive condition, empty arrays) -- ✅ Real-world patterns (async/await, useEffect, try-catch) - -### ✅ tests/integration/ast-search.test.ts -**Status**: Ready -**Lines**: 325 -**Test Cases**: 12 -**Coverage**: Pattern search, rule search, TS support, error handling - -**Structure**: -```typescript -- describe('AST Search Integration') - - beforeAll: Create temp dir + test files - - afterAll: Cleanup - - describe('Pattern Search') [3 tests] - - describe('Rule Search') [4 tests] - - describe('TypeScript Support') [2 tests] - - describe('Error Handling') [3 tests] -``` - -**Key Tests**: -- ✅ Async function detection with metavariables -- ✅ Composite rules (all, any, not) -- ✅ Relational rules (inside, has) -- ✅ Limit enforcement -- ✅ Empty patterns (no throw, returns empty) -- ✅ Non-existent paths (no throw, returns empty) -- ✅ Malformed files (skipped gracefully) - -### ✅ tests/integration/ast-mcp-integration.test.ts -**Status**: Ready (NEW!) -**Lines**: 520 -**Test Cases**: 15 -**Coverage**: Full MCP server integration - -**Structure**: -```typescript -- describe('AST MCP Integration Tests') - - beforeAll: Create workspace with realistic code - - afterAll: Cleanup - - describe('search_ast_pattern tool') [4 tests] - - describe('search_ast_rule tool') [6 tests] - - describe('check_ast_grep tool') [1 test] - - describe('Rule validation') [2 tests] - - describe('Real-world patterns') [2 tests] -``` - -**Key Tests**: -- ✅ Full workspace integration -- ✅ All 3 MCP tools tested -- ✅ Metavariable extraction -- ✅ React useEffect without deps -- ✅ Try-catch without error handling -- ✅ Complex nested rules - -## Code Quality Verification - -### ✅ Follows Existing Patterns -All tests match the structure of: -- `tests/integration/mcp-server.test.ts` -- `tests/unit/symbol-indexer-extended.test.ts` - -**Patterns Used**: -```typescript -// Graceful degradation -const info = await service.isAvailable(); -if (!info.available) { - console.warn('ast-grep not available - skipping tests'); - return; -} - -// Proper setup/teardown -beforeAll(async () => { /* create resources */ }); -afterAll(async () => { /* cleanup */ }); - -// Realistic test data -await createRealisticTestWorkspace(tempDir); -``` - -### ✅ Error Handling -Tests correctly handle: -- Missing ast-grep (skip gracefully) -- Empty patterns (return empty results) -- Non-existent paths (return empty results) -- Malformed files (skip with console.error) -- Invalid rules (validation errors) - -### ✅ Assertions -Proper assertions following Jest patterns: -```typescript -expect(result.matches.length).toBeGreaterThan(0); -expect(result).toHaveProperty('workspaceId'); -expect(validation.valid).toBe(true); -expect(match.metaVariables).toBeDefined(); -``` - -## Test Coverage Matrix - -| Component | Unit | Integration | MCP | -|-----------|------|-------------|-----| -| isAvailable() | ✅ | ✅ | ✅ | -| searchPattern() | - | ✅ | ✅ | -| searchRule() | - | ✅ | ✅ | -| validateRule() | ✅ | - | ✅ | -| Metavariables | - | ✅ | ✅ | -| Error handling | ✅ | ✅ | ✅ | -| Multi-language | - | ✅ | ✅ | -| Workspace mgmt | - | - | ✅ | - -## Real-World Patterns Tested - -### 1. Async Functions Without Await -```typescript -{ - all: [ - { pattern: 'async function $NAME($$$) { $$$ }' }, - { not: { has: { pattern: 'await $$$', stopBy: 'end' } } } - ] -} -``` -**Files**: integration/ast-search.test.ts, integration/ast-mcp-integration.test.ts - -### 2. React useEffect Without Dependencies -```typescript -{ - all: [ - { pattern: 'useEffect($CALLBACK)' }, - { not: { pattern: 'useEffect($CALLBACK, [$$$])' } } - ] -} -``` -**File**: integration/ast-mcp-integration.test.ts - -### 3. Console.log Inside Functions -```typescript -{ - pattern: 'console.log($$$)', - inside: { pattern: 'function $NAME($$$) { $$$ }', stopBy: 'end' } -} -``` -**Files**: Both integration test files - -### 4. Try-Catch Without Error Logging -```typescript -{ - all: [ - { pattern: 'try { $$$ } catch ($E) { $$$ }' }, - { not: { has: { pattern: 'console.error($$$)', stopBy: 'end' } } } - ] -} -``` -**Files**: unit/ast-search.test.ts, integration/ast-mcp-integration.test.ts - -### 5. Variable Declarations (ANY) -```typescript -{ - any: [ - { pattern: 'const $VAR = $$$' }, - { pattern: 'let $VAR = $$$' }, - { pattern: 'var $VAR = $$$' } - ] -} -``` -**Files**: Both integration test files - -## Test Execution Plan - -Once dependencies are installed (`npm install`), tests will run as: - -```bash -# All tests (expected: 41+ passing) -npm test - -# Unit tests only (expected: 14 passing) -npm test -- tests/unit/ast-search.test.ts - -# Integration tests (expected: 12 passing) -npm test -- tests/integration/ast-search.test.ts - -# MCP integration (expected: 15 passing) -npm test -- tests/integration/ast-mcp-integration.test.ts -``` - -## Expected Results - -### When ast-grep is Available -- ✅ All 41+ tests pass -- ✅ Pattern matching finds correct structures -- ✅ Metavariables extracted properly -- ✅ Rules combine correctly (AND/OR/NOT) -- ✅ Relational rules work (inside/has) -- ✅ Full MCP integration works - -### When ast-grep is Not Available -- ✅ Tests skip gracefully with warnings -- ✅ No test failures -- ✅ isAvailable() returns { available: false, error: '...' } - -## Verification Checklist - -### Code Structure ✅ -- [x] Imports correct -- [x] Types properly defined -- [x] Async/await used correctly -- [x] Error handling present -- [x] Cleanup in afterAll - -### Test Logic ✅ -- [x] Graceful degradation -- [x] Realistic test data -- [x] Proper assertions -- [x] Edge cases covered -- [x] Multiple languages tested - -### Integration ✅ -- [x] Workspace management -- [x] MCP tools tested -- [x] Metavariable extraction -- [x] Real-world patterns -- [x] Error resilience - -## Known Issues: NONE ✅ - -All test files are syntactically correct and will execute successfully once: -1. Dependencies are installed (`npm install`) -2. @ast-grep/napi is available (bundled) - -## Compilation Verification - -The AST search service is already compiled: -```bash -$ ls -la dist/ast-search/ --rw-r--r-- 1 root root 17093 Nov 21 21:09 ast-search-service.js --rw-r--r-- 1 root root 1742 Nov 21 21:09 ast-search-service.d.ts -``` - -The compiled code imports @ast-grep/napi correctly: -```javascript -import { parse, Lang } from '@ast-grep/napi'; -``` - -## Conclusion - -**All tests are production-ready and will pass once dependencies are installed.** - -The test suite is: -- ✅ Comprehensive (41+ test cases) -- ✅ Well-structured (follows existing patterns) -- ✅ Realistic (real-world patterns tested) -- ✅ Robust (proper error handling) -- ✅ Documented (AST_SEARCH_TESTS.md included) - -**Confidence Level: 100%** - -The tests will work immediately after `npm install` completes successfully. diff --git a/ai_docs/BLUEPRINT.md b/ai_docs/BLUEPRINT.md deleted file mode 100644 index 5f81a13..0000000 --- a/ai_docs/BLUEPRINT.md +++ /dev/null @@ -1,22 +0,0 @@ -# Code Search MCP Server Blueprint -Universal MCP (Model Context Protocol) server that works with any programming language, to help LLMs find things in large codebases without filling up their context window - -Goals & constraints for your MCP code-search server - -High-level goals: - -- Context-efficient -- Simple + lightweight + fast (very optimized) -- Works on any language: C, Rust, TS, Java, Kotlin, Python, etc. -- Works on any operating system (code in TypeScript using the official MCP SDK) -- No language-server dependency; all analysis is text-based + generic indexing. -- Encourage progressive discovery: search → select file(s) → fetch tight snippets. -- Accurate method for determining the primary coding language used in the workspace - -- Identify specific tech "stacks" by the presence of various configuration files (read STACK_DETECTION_SYSTEM.md) -- Language aware search (read LANGUAGE_AWARE_SEARCH.md) - -Before you commit, you must do EVERYTHING defined in COMPLETION_REQUIREMENTS.md - - - diff --git a/ai_docs/COMPLETION_REQUIREMENTS.md b/ai_docs/COMPLETION_REQUIREMENTS.md deleted file mode 100644 index 1215f75..0000000 --- a/ai_docs/COMPLETION_REQUIREMENTS.md +++ /dev/null @@ -1,11 +0,0 @@ -- You must use TypeScript -- You must use the official TypeScript MCP SDK -- You must set up the LATEST version of ESLint -- Linting MUST pass without any errors -- You MUST run type checking to ensure production readiness -- You MUST create integration tests for every area of the codebase, that use REAL data - - For these tests, have it clone REAL "popular" github repos - - Test dynamic language search - - Test ALL other features -- You MUST create integration tests which actually use the REAL tools from the MCP server , to guarentee production readiness -- Every single test MUST pass \ No newline at end of file diff --git a/ai_docs/LANGUAGE_AWARE_SEARCH.md b/ai_docs/LANGUAGE_AWARE_SEARCH.md deleted file mode 100644 index 5988a44..0000000 --- a/ai_docs/LANGUAGE_AWARE_SEARCH.md +++ /dev/null @@ -1,378 +0,0 @@ -1. High-level concept - -You’ll have one generic symbol-search tool exposed to MCP, plus language presets. - -We ONLY need it for these languages: -Java - -Python - -JavaScript / TypeScript - -C# - -No language server, just ctags + ripgrep + some smart filters. - - -tool search_symbols( - workspace_id: string, - params: { - language: "java" | "python" | "javascript" | "typescript" | "csharp"; - name: string; // search term, e.g. "UserService" - match?: "exact" | "prefix" | "substring" | "regex"; - kinds?: string[]; // logical kinds: ["class", "method", "variable", "function", "property", ...] - scope?: { - in_class?: string; // e.g. "UserService" - in_namespace?: string; // C# namespace / Java package - in_module?: string; // Python module - }; - limit?: number; // default 100 - } -) -> { - symbols: SymbolResult[]; -} - -Where SymbolResult is roughly: -type SymbolResult = { - name: string; - language: string; // one of the above - kind: string; // normalized, e.g. "class" | "method" | "field" | ... - file: string; - line: number; - column?: number; - containerName?: string; // class / module / namespace name - signature?: string; // if available from tags -}; - -The LLM uses the same tool for all four languages, just changing language + kinds. - -Under the hood you: - -Build a symbol index using universal-ctags for each workspace. ctags generates an index of language objects (classes, functions, etc.) for many languages including Java, Python, JavaScript, C#, etc. -GitHub - -Use ripgrep as a fallback / usage search when you just need “find occurrences of this name in this language’s files”. ripgrep is a fast, recursive, gitignore-aware regex search tool, widely used for codebases. - -2. Indexing strategy (shared across languages) -2.1 Build a language-agnostic symbol index - -For each workspace: - -Run universal-ctags with language filters and rich fields: -ctags \ - --languages=Java,Python,JavaScript,TypeScript,C# \ - --fields=+nKls \ - --extras=+q \ - -R . - ---fields=+n → line numbers, K/l/s → kind/name/scope info. ---extras=+q → qualified tags, useful for class::member or class.member forms in C++/Java. -Universal-ctags supports many languages out of the box and is the maintained successor of Exuberant Ctags - - -Parse the generated tags file into your own JSON index: - -For each line, extract: - -name - -file - -line - -language - -kind (ctags kind letter/name) - -scope fields (class, namespace, module) if present. - -Universal-ctags defines kinds per language (class, function, variable, etc.), and you can introspect them using ctags --list-kinds-full=. - -Map (language, ctagsKind) → normalized kind: - -Example mapping: - -Java: class/interface/enum kinds → "class" | "interface" | "enum", method kinds → "method", field kinds → "field". - -Python: class → "class", function → "function", class member → "method", variable/import kinds → "variable" (or you can drop them to keep noise low, similar to editor setups) - -JS/TS: function, class, variable kinds → "function" | "class" | "variable". - -C#: class, struct, interface, enum, method, property, field → normalized accordingly. Universal-ctags supports C# as a language and can be configured for these kinds. - -Store in memory or a small local DB keyed by (language, normalizedKind, name, scope, file) - -3. Language-specific search “profiles” - -You can keep the MCP interface generic, but implement per-language behavior inside the server: - -3.1 Java search - -Entities you care about: - -Classes / Interfaces / Enums - -Methods - -Fields - -(Optionally) packages - -Universal-ctags can generate tags for Java classes, fields, interfaces, methods, and packages. -pegasusinfocorp.com -+2 -Centennial Software Solutions LLC -+2 - -Symbol search behavior: - -When language = "java", map kinds like: - -"class" → ctags Java kinds: class/interface/enum - -"method" → Java method kind - -"field" → Java field kind - -Apply match to the symbol name: - -"exact" → name === query - -"prefix" → name.startsWith(query) - -"substring" → name.toLowerCase().includes(query.toLowerCase()) - -"regex" → compile query as regex and match symbol name. - -Use scope.in_class and scope.in_namespace to filter: - -in_class → only methods/fields whose containerName/class field matches. - -in_namespace → map to Java package if ctags provides it. - -Usage search (optional second tool or flag): - -Use ripgrep constrained to **/*.java for occurrences of class/method names: - -rg --glob '*.java' 'UserService' src/ - -This is good for “find all usages”, while ctags is good for “find definition”. - -3.2 Python search - -Entities: - -Modules (files / packages) - -Classes - -Functions (top-level) - -Methods (class members) - -(Optionally) variables / attributes - -By default, ctags generates tags for Python functions, classes, class members, variables, imports, and files. -Arch Manual Pages -+2 -GitHub -+2 - -Symbol search behavior: - -When language = "python": - -"class" → Python class kind. - -"function" → module-level function kind. - -"method" → class member kind. - -"variable" (if you decide to support) → variable kind. - -scope.in_module allows filtering by file/module (e.g., 'my_package.my_module' if you derive dotted path from filesystem). - -You can also split: - -kinds = ["function"] → only module-level functions. - -kinds = ["method"] + scope.in_class="User" → methods on class User. - -Usage search: - -Use rg limited to *.py to find references to a function/method. - -rg --glob '*.py' 'def my_function' . -rg --glob '*.py' 'my_function\(' . - -3.3 JavaScript / TypeScript search - -Entities: - -Functions (named / exported) - -Classes - -Variables: const, let, var, exported bindings - -Types/Interfaces/Enums (TS only) - -ctags can index JS/TS symbols (functions, variables, classes, etc.), and you can introspect its TypeScript support via --list-kinds=typescript. -Reddit -+2 -Medium -+2 - -Symbol search behavior: - -When language = "javascript": - -"function" → JS function kinds. - -"class" → JS class kinds. - -"variable" → variable kinds (you might restrict to exported or top-level based on tag metadata if available). - -When language = "typescript": - -"function", "class", "variable" as above. - -"interface" → TS interface tags. - -"type" → TS type alias tags. - -"enum" → TS enum tags. - -For variables, you might want to bias toward exported/public things, but initially you can just expose all variable tags and let the LLM filter by file / naming. - -Usage search (ripgrep): - -Use file globs appropriate for JS/TS: - -rg --glob '*.{js,jsx,ts,tsx}' 'UserStore' . - - -Or file type filters if you configure them. - -3.4 C# search - -Entities: - -Namespaces - -Classes / Structs / Interfaces / Enums - -Methods - -Properties - -Fields - -(Optionally) Events - -Universal-ctags supports C# as a language, and people use it to generate tags for C# code (class, method, etc.), often as a lightweight alternative to OmniSharp. -GitHub -+3 -ctags.sourceforge.net -+3 -Reddit -+3 - -Symbol search behavior: - -When language = "csharp": - -"namespace" → namespace kinds. - -"class" → class kinds. - -"struct", "interface", "enum" similarly. - -"method" → methods. - -"property" → property tags (ctags typically distinguishes them). - -"field" → field tags. - -scope.in_namespace and scope.in_class are especially useful. - -E.g., in_namespace = "MyApp.Services" and kind = ["class"]. - -Usage search (ripgrep): - -Limit search to C# files: - -rg --glob '*.{cs,csx}' 'ILogger' . - -4. Extra: a small “search profile” table per language - -You can bake these into a tiny config so the server knows defaults for each language. - -Example internal config: - -const LanguageProfiles = { - java: { - fileGlobs: ["**/*.java"], - defaultKinds: ["class", "interface", "enum", "method"], - }, - python: { - fileGlobs: ["**/*.py"], - defaultKinds: ["class", "function", "method"], - }, - javascript: { - fileGlobs: ["**/*.js", "**/*.jsx"], - defaultKinds: ["function", "class", "variable"], - }, - typescript: { - fileGlobs: ["**/*.ts", "**/*.tsx"], - defaultKinds: ["function", "class", "variable", "interface", "type", "enum"], - }, - csharp: { - fileGlobs: ["**/*.cs", "**/*.csx"], - defaultKinds: [ - "namespace", - "class", - "struct", - "interface", - "enum", - "method", - "property", - "field", - ], - }, -}; - - -The MCP tool can default kinds from here if the caller just says “language: java, name: UserService”. - -5. How this fits into your MCP server - -Putting it together: - -Indexing service - -At startup or via refresh_index, run universal-ctags over the workspace and load tags into a per-language symbol index. -GitHub -+2 -docs.ctags.io -+2 - -search_symbols tool (generic) - -Filters the symbol index by (language, kind(s), name, scope) and returns small, precise symbol hits. - -search_text / search_usages tool (existing/other) - -Wraps ripgrep with language-specific fileGlobs. - -LLM flow examples: - -“Find the Java class UserService” → search_symbols(language=java, kinds=["class"], name="UserService"). - -“Find all Python methods called handle_request” → search_symbols(language=python, kinds=["method"], name="handle_request"). - -“List all TypeScript interfaces in this project” → search_symbols(language=typescript, kinds=["interface"], name="", match="prefix") (empty name + limit, or a dedicated “list_symbols” variant). - -“Find the C# property IsEnabled on FeatureFlags” → search_symbols(language=csharp, kinds=["property"], name="IsEnabled", scope={in_class:"FeatureFlags"}). - -This gives you language-optimized symbol search for Java, Python, JS/TS, and C# without ever becoming a language server: just ctags-backed indexes + ripgrep, wrapped in a single clean MCP tool the LLM can drive. \ No newline at end of file diff --git a/ai_docs/STACK_DETECTION_SYSTEM.md b/ai_docs/STACK_DETECTION_SYSTEM.md deleted file mode 100644 index ee443d1..0000000 --- a/ai_docs/STACK_DETECTION_SYSTEM.md +++ /dev/null @@ -1,425 +0,0 @@ -1. Stack registry (top-level) - -/** - * The full set of stack definitions loaded at startup from stacks.yaml/json. - */ -type StackRegistry = { - /** - * Map of stack id -> definition. - * - * Example ids: "nodejs", "python", "rust", "go", "java-maven", "react", "nextjs". - */ - stacks: Record; - - /** - * Optional metadata about the registry itself. - */ - version?: string; // e.g. "2025.01" - updatedAt?: string; // ISO string - description?: string; -}; - -2. Stack definition schema (core) -/** - * A “stack” is a recognizable technology or layer in a codebase. - * Examples: Node.js, Python, Rust (Cargo), Go (modules), Java (Maven), React, Next.js, Django. - */ -type StackDefinition = { - /** Unique identifier, stable and machine-friendly. */ - id: string; // "nodejs", "python", "rust", "go", "java-maven", "react", "nextjs", ... - - /** Human-friendly name for display. */ - displayName: string; // "Node.js", "Python (pyproject)", "Rust (Cargo)", ... - - /** - * High-level kind of stack. - * - * "language" – primary language ecosystem (Node.js, Python, Rust, Go, Java...) - * "framework" – higher-level framework (React, Next.js, Django, Spring Boot...) - * "runtime" – runtime/platform (Node.js runtime, Deno, .NET runtime, JVM, browser-only) - * "tooling" – build/test tools (Webpack, Vite, Jest, ESLint, Poetry, pnpm, Maven...) - */ - category: "language" | "framework" | "runtime" | "tooling"; - - /** - * Optional, short description of what this stack definition represents. - */ - description?: string; - - /** - * Optional tags for filtering, grouping, or display. - * Examples: ["backend", "frontend", "cli", "web", "mobile", "monorepo"] - */ - tags?: string[]; - - /** - * Direct dependencies between stacks (semantic, not file-based). - * e.g. React depends on Node.js, Django depends on Python, etc. - */ - dependsOn?: string[]; // list of stack ids this stack logically sits on - - /** - * Where to primarily search for indicators, relative to workspace root. - * If empty/undefined, assume ["."] - */ - searchRoots?: string[]; // e.g. [".", "src", "backend", "frontend"] - - /** - * Definitions of signals that indicate the presence of this stack. - */ - indicators: StackIndicators; - - /** - * Detection scoring configuration. - */ - detection: DetectionConfig; -}; - -3. Indicator sets and scoring -/** - * Indicator groups for a given stack. - */ -type StackIndicators = { - /** - * At least ONE of these must be satisfied for the stack - * to be considered at all (hard gate). - * - * Example: `package.json` for Node.js, `pyproject.toml` or `setup.py` for Python, - * `Cargo.toml` for Rust, `go.mod` for Go, `pom.xml` for Maven. - */ - requiredAny?: Indicator[]; - - /** - * ALL of these must be satisfied for the stack to be considered. - * Use sparingly; only for very specific stacks. - */ - requiredAll?: Indicator[]; - - /** - * These raise the score/confidence but are not required. - * If only optional indicators are present and total score >= minScore, - * the stack can still be detected. - */ - optional?: Indicator[]; - - /** - * Other stack ids that should NOT be present at the same time. - * (You can still allow the engine to override conflicts if both strongly detected.) - */ - conflictsWith?: string[]; -}; - -/** - * Scoring & thresholds for deciding whether a stack is present. - */ -type DetectionConfig = { - /** - * Minimum score required to declare this stack "detected". - * Score is the sum of `weight` for all satisfied indicators. - */ - minScore: number; - - /** - * Optional max score used for normalizing confidence (0..1). - * If omitted, engine can derive `maxScore` from sum of all indicator weights. - */ - maxScore?: number; - - /** - * Optional “hard maximum” number of indicator matches to count. - * Anything beyond this may be ignored or capped to avoid runaway scoring in huge repos. - */ - maxIndicatorsCounted?: number; - - /** - * Whether this stack should be considered "primary" if multiple stacks - * of the same category are detected (e.g., choose one primary language). - */ - priority?: number; // higher = more important -}; - -4. Indicator types (detection signals) - -These are the “building blocks” for your heuristics. - -/** - * All possible indicator types. - * Each one contributes `weight` points to the stack’s score when satisfied. - */ -type Indicator = - | FileExistsIndicator - | DirExistsIndicator - | FilePatternExistsIndicator - | FileContainsIndicator - | PathPatternIndicator - | JsonFieldIndicator - | TomlFieldIndicator; - -/** - * Simple file presence: path relative to search root. - * Example: "package.json", "pyproject.toml", "Cargo.toml", "go.mod". - */ -type FileExistsIndicator = { - kind: "fileExists"; - path: string; - /** How strong this signal is. Big ticket files get high weights. */ - weight: number; - /** If true, `path` is always interpreted relative to the workspace root, not searchRoots. */ - rootRelative?: boolean; -}; - -/** - * Directory presence: path relative to search root. - * Example: "src/main/java", "src", "tests". - */ -type DirExistsIndicator = { - kind: "dirExists"; - path: string; - weight: number; - rootRelative?: boolean; -}; - -/** - * File pattern presence: glob relative to search root. - * Example: "**/*.go", "src/**/*.rs", "config/*.js". - */ -type FilePatternExistsIndicator = { - kind: "filePatternExists"; - glob: string; - weight: number; - rootRelative?: boolean; - /** - * Max number of distinct matches to count; - * beyond this, weight contribution can be capped. - */ - maxMatches?: number; -}; - -/** - * File content regex: check whether a specific file at `path` - * contains text matching `regex`. - * - * Example: - * - pyproject.toml contains "\\[project\\]" (PEP 621 metadata). - * - package.json contains "\"dependencies\"". - */ -type FileContainsIndicator = { - kind: "fileContains"; - path: string; - regex: string; - weight: number; - rootRelative?: boolean; -}; - -/** - * Path pattern: any file or directory path in the workspace - * that matches a regex. - * - * Example: - * - regex: "src/main/java" for Maven-like structure. - * - regex: ".*\\.csproj$" for .NET projects. - */ -type PathPatternIndicator = { - kind: "pathPattern"; - regex: string; - weight: number; -}; - -/** - * JSON field presence in a file. - * Useful for package.json, tsconfig.json, composer.json, etc. - * - * The engine can parse the file as JSON and look up the path. - */ -type JsonFieldIndicator = { - kind: "jsonField"; - path: string; // file path (e.g., "package.json") - jsonPointer: string; // JSON pointer or dot path, e.g. "/dependencies/react" - /** - * Expected value behavior: - * - if `expectedValue` is undefined → just check the field exists. - * - if defined → check equality or pattern depending on type. - */ - expectedValue?: string | number | boolean | string[]; - weight: number; - rootRelative?: boolean; -}; - -/** - * TOML field presence in a file, e.g. pyproject.toml, Cargo.toml. - * - * pyproject.toml is the standardized place for modern Python packaging metadata, per PEP 518/621. :contentReference[oaicite:0]{index=0} - * Cargo.toml is the manifest file for Rust packages and workspaces. :contentReference[oaicite:1]{index=1} - */ -type TomlFieldIndicator = { - kind: "tomlField"; - path: string; // e.g. "pyproject.toml", "Cargo.toml" - tomlPath: string; // TOML dotted key path, e.g. "project.name", "build-system.requires" - expectedValue?: string | number | boolean | string[]; - weight: number; - rootRelative?: boolean; -}; - -5. Detection engine config (multi-stack, multi-workspace) - -To make it explicit that you can detect multiple stacks per workspace, define engine options and results like this. - -5.1 Detection options -/** - * Options for a single detection run on a workspace. - */ -type DetectionOptions = { - /** - * Which stacks to consider. If empty or undefined, consider all from the registry. - */ - includeStacks?: string[]; // whitelist by stack id - excludeStacks?: string[]; // blacklist by stack id - - /** - * Max directory depth for scanning below the root. - * Example: 0=root only, 1=root+immediate children, etc. - */ - maxDepth?: number; - - /** - * Scan mode: - * - "fast": only use cheap indicators (fileExists, dirExists at shallow depth). - * - "thorough": use all indicators, deeper traversal, content checks. - */ - scanMode?: "fast" | "thorough"; - - /** - * Global safety caps. - */ - limits?: { - /** - * Max number of files to inspect. - */ - maxFiles?: number; - - /** - * Max bytes to read for any single file when doing content/JSON/TOML checks. - */ - maxBytesPerFile?: number; - - /** - * Hard timeout in milliseconds for the entire detection run. - */ - timeoutMs?: number; - }; -}; - -5.2 Detection result (multiple stacks) -/** - * The result of running stack detection on a single workspace. - * A workspace can have MANY stacks detected at once (monorepos, polyglot projects). - */ -type WorkspaceStackDetectionResult = { - /** Identifier of the workspace (from your MCP workspace manager). */ - workspaceId: string; - - /** Absolute or canonical root path (sanitized if needed). */ - rootPath: string; - - /** All stacks that met their `minScore` threshold. */ - detectedStacks: DetectedStack[]; - - /** Stacks that were considered but did NOT meet their threshold, with partial scores. */ - consideredStacks?: ConsideredStack[]; - - /** Summary metadata that the LLM can use to understand the shape of the repo. */ - summary?: { - dominantLanguages?: string[]; // e.g. ["typescript", "python"] - /** Most “significant” stack per category, based on priority & confidence. */ - primaryByCategory?: Record< - "language" | "framework" | "runtime" | "tooling", - string[] // stack ids, sorted by priority/confidence - >; - }; -}; - -/** - * A stack that crossed the minScore threshold. - */ -type DetectedStack = { - id: string; - displayName: string; - category: "language" | "framework" | "runtime" | "tooling"; - - /** Final score (sum of indicator weights). */ - score: number; - - /** Normalized confidence in 0..1, derived from score and `maxScore`. */ - confidence: number; - - /** - * The ids of stacks this stack depends on that were also detected, - * resolved from StackDefinition.dependsOn. - */ - resolvedDependencies?: string[]; - - /** - * Indicators that contributed to the score (satisfied ones). - * Every entry is a normalized record; you can omit raw regex/glob if you want. - */ - evidence: IndicatorEvidence[]; -}; - -/** - * A stack that was evaluated but did not meet `minScore`. - * Useful for debugging or “near miss” information. - */ -type ConsideredStack = { - id: string; - displayName: string; - category: "language" | "framework" | "runtime" | "tooling"; - score: number; - confidence: number; - evidence: IndicatorEvidence[]; -}; - -5.3 Evidence records -/** - * Evidence that a particular indicator fired. - * This is what the LLM uses to understand WHY a stack was detected. - */ -type IndicatorEvidence = { - /** - * The kind of indicator that fired (same set as `Indicator["kind"]`). - */ - kind: Indicator["kind"]; - - /** - * Reference to original indicator if you include ids in your registry. - */ - indicatorId?: string; - - /** - * File/directory path that matched (if applicable), relative to workspace root. - */ - path?: string; - - /** - * Glob or regex involved (optional; you may trim them for brevity). - */ - glob?: string; - regex?: string; - - /** - * For jsonField/tomlField indicators, the resolved key path and value. - */ - fieldPath?: string; // jsonPointer or tomlPath - fieldValue?: unknown; // captured value (trimmed/sanitized) - - /** Weight this piece of evidence contributed to the score. */ - weight: number; - - /** - * Optional human-readable note; can be generated at detection time, - * like "Found pyproject.toml with [project] table". - */ - note?: string; -}; - - -I started a comprehensive example database for you to work off of / build upon - stacks.json.example \ No newline at end of file diff --git a/ai_docs/stacks.json.example b/ai_docs/stacks.json.example deleted file mode 100644 index af9f902..0000000 --- a/ai_docs/stacks.json.example +++ /dev/null @@ -1,1311 +0,0 @@ -{ - "version": "2025.11-01", - "description": "Initial stack definition registry for a universal code-search MCP server.", - "stacks": { - "nodejs": { - "id": "nodejs", - "displayName": "Node.js", - "category": "runtime", - "description": "Node.js runtime and npm/yarn/pnpm-based JavaScript/TypeScript projects.", - "tags": ["backend", "javascript", "typescript", "node", "npm", "monorepo"], - "searchRoots": ["."], - "dependsOn": ["javascript"], - "indicators": { - "requiredAny": [ - { - "kind": "fileExists", - "path": "package.json", - "weight": 10, - "rootRelative": true - } - ], - "optional": [ - { - "kind": "fileExists", - "path": "package-lock.json", - "weight": 3, - "rootRelative": true - }, - { - "kind": "fileExists", - "path": "yarn.lock", - "weight": 3, - "rootRelative": true - }, - { - "kind": "fileExists", - "path": "pnpm-lock.yaml", - "weight": 3, - "rootRelative": true - }, - { - "kind": "fileExists", - "path": "npm-shrinkwrap.json", - "weight": 2, - "rootRelative": true - }, - { - "kind": "fileContains", - "path": "package.json", - "regex": "dependencies", - "weight": 1, - "rootRelative": true - }, - { - "kind": "fileContains", - "path": "package.json", - "regex": "devDependencies", - "weight": 1, - "rootRelative": true - } - ] - }, - "detection": { - "minScore": 8 - } - }, - - "javascript": { - "id": "javascript", - "displayName": "JavaScript", - "category": "language", - "description": "Projects primarily written in JavaScript.", - "tags": ["frontend", "backend", "web"], - "searchRoots": ["."], - "indicators": { - "requiredAny": [ - { - "kind": "filePatternExists", - "glob": "**/*.js", - "weight": 4 - } - ], - "optional": [ - { - "kind": "fileExists", - "path": "jsconfig.json", - "weight": 2, - "rootRelative": true - }, - { - "kind": "filePatternExists", - "glob": "src/**/*.jsx", - "weight": 2 - }, - { - "kind": "filePatternExists", - "glob": "public/**/*.js", - "weight": 1 - } - ] - }, - "detection": { - "minScore": 4 - } - }, - - "typescript": { - "id": "typescript", - "displayName": "TypeScript", - "category": "language", - "description": "Projects primarily written in TypeScript.", - "tags": ["frontend", "backend", "web"], - "searchRoots": ["."], - "dependsOn": ["javascript", "nodejs"], - "indicators": { - "requiredAny": [ - { - "kind": "fileExists", - "path": "tsconfig.json", - "weight": 8, - "rootRelative": true - } - ], - "optional": [ - { - "kind": "filePatternExists", - "glob": "**/*.ts", - "weight": 2 - }, - { - "kind": "filePatternExists", - "glob": "**/*.tsx", - "weight": 2 - }, - { - "kind": "fileExists", - "path": "tsconfig.base.json", - "weight": 1, - "rootRelative": true - } - ] - }, - "detection": { - "minScore": 8 - } - }, - - "python": { - "id": "python", - "displayName": "Python", - "category": "language", - "description": "Python projects using pyproject.toml or legacy setup.py/setup.cfg/requirements.txt.", - "tags": ["backend", "scripting", "data", "ml"], - "searchRoots": ["."], - "indicators": { - "requiredAny": [ - { - "kind": "fileExists", - "path": "pyproject.toml", - "weight": 9, - "rootRelative": true - }, - { - "kind": "fileExists", - "path": "setup.py", - "weight": 7, - "rootRelative": true - }, - { - "kind": "fileExists", - "path": "setup.cfg", - "weight": 6, - "rootRelative": true - }, - { - "kind": "fileExists", - "path": "requirements.txt", - "weight": 5, - "rootRelative": true - } - ], - "optional": [ - { - "kind": "filePatternExists", - "glob": "**/*.py", - "weight": 2 - }, - { - "kind": "tomlField", - "path": "pyproject.toml", - "tomlPath": "project.name", - "weight": 2, - "rootRelative": true - }, - { - "kind": "tomlField", - "path": "pyproject.toml", - "tomlPath": "build-system.requires", - "weight": 1, - "rootRelative": true - } - ] - }, - "detection": { - "minScore": 7 - } - }, - - "rust": { - "id": "rust", - "displayName": "Rust (Cargo)", - "category": "language", - "description": "Rust packages or workspaces managed by Cargo.", - "tags": ["systems", "cli", "backend"], - "searchRoots": ["."], - "indicators": { - "requiredAny": [ - { - "kind": "fileExists", - "path": "Cargo.toml", - "weight": 10, - "rootRelative": true - } - ], - "optional": [ - { - "kind": "fileExists", - "path": "Cargo.lock", - "weight": 3, - "rootRelative": true - }, - { - "kind": "dirExists", - "path": "src", - "weight": 2 - }, - { - "kind": "filePatternExists", - "glob": "src/**/*.rs", - "weight": 2 - } - ] - }, - "detection": { - "minScore": 9 - } - }, - - "go": { - "id": "go", - "displayName": "Go (modules)", - "category": "language", - "description": "Go modules using go.mod.", - "tags": ["backend", "cli", "systems"], - "searchRoots": ["."], - "indicators": { - "requiredAny": [ - { - "kind": "fileExists", - "path": "go.mod", - "weight": 10, - "rootRelative": true - } - ], - "optional": [ - { - "kind": "fileExists", - "path": "go.sum", - "weight": 3, - "rootRelative": true - }, - { - "kind": "filePatternExists", - "glob": "**/*.go", - "weight": 2 - } - ] - }, - "detection": { - "minScore": 9 - } - }, - - "java-maven": { - "id": "java-maven", - "displayName": "Java (Maven)", - "category": "language", - "description": "Java projects built with Apache Maven.", - "tags": ["backend", "jvm"], - "searchRoots": ["."], - "indicators": { - "requiredAny": [ - { - "kind": "fileExists", - "path": "pom.xml", - "weight": 10, - "rootRelative": true - } - ], - "optional": [ - { - "kind": "dirExists", - "path": "src/main/java", - "weight": 3 - }, - { - "kind": "dirExists", - "path": "src/test/java", - "weight": 2 - } - ] - }, - "detection": { - "minScore": 9 - } - }, - - "java-gradle": { - "id": "java-gradle", - "displayName": "Java (Gradle)", - "category": "language", - "description": "Java or JVM projects built with Gradle.", - "tags": ["backend", "jvm"], - "searchRoots": ["."], - "indicators": { - "requiredAny": [ - { - "kind": "fileExists", - "path": "build.gradle", - "weight": 8, - "rootRelative": true - }, - { - "kind": "fileExists", - "path": "build.gradle.kts", - "weight": 8, - "rootRelative": true - } - ], - "optional": [ - { - "kind": "fileExists", - "path": "settings.gradle", - "weight": 2, - "rootRelative": true - }, - { - "kind": "fileExists", - "path": "settings.gradle.kts", - "weight": 2, - "rootRelative": true - }, - { - "kind": "dirExists", - "path": "src/main/java", - "weight": 3 - }, - { - "kind": "dirExists", - "path": "src/test/java", - "weight": 2 - } - ] - }, - "detection": { - "minScore": 8 - } - }, - - "kotlin-gradle": { - "id": "kotlin-gradle", - "displayName": "Kotlin (Gradle)", - "category": "language", - "description": "Kotlin projects using Gradle as the build system.", - "tags": ["backend", "jvm", "android"], - "searchRoots": ["."], - "dependsOn": ["java-gradle"], - "indicators": { - "requiredAny": [ - { - "kind": "filePatternExists", - "glob": "**/*.kt", - "weight": 5 - } - ], - "optional": [ - { - "kind": "fileExists", - "path": "build.gradle.kts", - "weight": 3, - "rootRelative": true - }, - { - "kind": "fileExists", - "path": "settings.gradle.kts", - "weight": 2, - "rootRelative": true - } - ] - }, - "detection": { - "minScore": 5 - } - }, - - "csharp-dotnet": { - "id": "csharp-dotnet", - "displayName": "C# (.NET)", - "category": "language", - "description": "C# projects built with .NET SDK (csproj/sln).", - "tags": ["backend", "desktop", "web", "dotnet"], - "searchRoots": ["."], - "indicators": { - "requiredAny": [ - { - "kind": "filePatternExists", - "glob": "**/*.csproj", - "weight": 9 - } - ], - "optional": [ - { - "kind": "filePatternExists", - "glob": "**/*.sln", - "weight": 3 - }, - { - "kind": "filePatternExists", - "glob": "**/*.cs", - "weight": 2 - } - ] - }, - "detection": { - "minScore": 9 - } - }, - - "cc-cmake": { - "id": "cc-cmake", - "displayName": "C/C++ (CMake)", - "category": "language", - "description": "C or C++ projects using CMake as a build system.", - "tags": ["systems", "native", "embedded"], - "searchRoots": ["."], - "indicators": { - "requiredAny": [ - { - "kind": "fileExists", - "path": "CMakeLists.txt", - "weight": 9, - "rootRelative": true - } - ], - "optional": [ - { - "kind": "filePatternExists", - "glob": "**/*.c", - "weight": 2 - }, - { - "kind": "filePatternExists", - "glob": "**/*.cpp", - "weight": 2 - }, - { - "kind": "filePatternExists", - "glob": "**/*.h", - "weight": 1 - }, - { - "kind": "filePatternExists", - "glob": "**/*.hpp", - "weight": 1 - } - ] - }, - "detection": { - "minScore": 9 - } - }, - - "cc-make": { - "id": "cc-make", - "displayName": "C/C++ (Make)", - "category": "language", - "description": "C or C++ projects using Makefiles.", - "tags": ["systems", "native", "embedded"], - "searchRoots": ["."], - "indicators": { - "requiredAny": [ - { - "kind": "fileExists", - "path": "Makefile", - "weight": 7, - "rootRelative": true - } - ], - "optional": [ - { - "kind": "filePatternExists", - "glob": "**/*.c", - "weight": 2 - }, - { - "kind": "filePatternExists", - "glob": "**/*.cpp", - "weight": 2 - } - ] - }, - "detection": { - "minScore": 7 - } - }, - - "php-composer": { - "id": "php-composer", - "displayName": "PHP (Composer)", - "category": "language", - "description": "PHP projects managed with Composer (composer.json).", - "tags": ["backend", "web"], - "searchRoots": ["."], - "indicators": { - "requiredAny": [ - { - "kind": "fileExists", - "path": "composer.json", - "weight": 9, - "rootRelative": true - } - ], - "optional": [ - { - "kind": "fileExists", - "path": "composer.lock", - "weight": 3, - "rootRelative": true - }, - { - "kind": "filePatternExists", - "glob": "**/*.php", - "weight": 2 - } - ] - }, - "detection": { - "minScore": 9 - } - }, - - "ruby-bundler": { - "id": "ruby-bundler", - "displayName": "Ruby (Bundler)", - "category": "language", - "description": "Ruby projects using Bundler with a Gemfile.", - "tags": ["backend", "web"], - "searchRoots": ["."], - "indicators": { - "requiredAny": [ - { - "kind": "fileExists", - "path": "Gemfile", - "weight": 9, - "rootRelative": true - } - ], - "optional": [ - { - "kind": "fileExists", - "path": "Gemfile.lock", - "weight": 3, - "rootRelative": true - }, - { - "kind": "filePatternExists", - "glob": "**/*.rb", - "weight": 2 - } - ] - }, - "detection": { - "minScore": 9 - } - }, - - "swift-spm": { - "id": "swift-spm", - "displayName": "Swift (SwiftPM)", - "category": "language", - "description": "Swift packages that use Swift Package Manager (Package.swift).", - "tags": ["ios", "macos", "cli"], - "searchRoots": ["."], - "indicators": { - "requiredAny": [ - { - "kind": "fileExists", - "path": "Package.swift", - "weight": 9, - "rootRelative": true - } - ], - "optional": [ - { - "kind": "filePatternExists", - "glob": "**/*.swift", - "weight": 2 - } - ] - }, - "detection": { - "minScore": 9 - } - }, - - "elixir-mix": { - "id": "elixir-mix", - "displayName": "Elixir (Mix)", - "category": "language", - "description": "Elixir projects using Mix (mix.exs).", - "tags": ["backend"], - "searchRoots": ["."], - "indicators": { - "requiredAny": [ - { - "kind": "fileExists", - "path": "mix.exs", - "weight": 9, - "rootRelative": true - } - ], - "optional": [ - { - "kind": "filePatternExists", - "glob": "lib/**/*.ex", - "weight": 2 - } - ] - }, - "detection": { - "minScore": 9 - } - }, - - "haskell": { - "id": "haskell", - "displayName": "Haskell", - "category": "language", - "description": "Haskell projects using Cabal or Stack.", - "tags": ["functional"], - "searchRoots": ["."], - "indicators": { - "requiredAny": [ - { - "kind": "filePatternExists", - "glob": "*.cabal", - "weight": 8 - }, - { - "kind": "fileExists", - "path": "stack.yaml", - "weight": 7, - "rootRelative": true - } - ], - "optional": [ - { - "kind": "filePatternExists", - "glob": "src/**/*.hs", - "weight": 2 - } - ] - }, - "detection": { - "minScore": 8 - } - }, - - "scala-sbt": { - "id": "scala-sbt", - "displayName": "Scala (sbt)", - "category": "language", - "description": "Scala projects using sbt.", - "tags": ["jvm", "backend"], - "searchRoots": ["."], - "indicators": { - "requiredAny": [ - { - "kind": "fileExists", - "path": "build.sbt", - "weight": 8, - "rootRelative": true - } - ], - "optional": [ - { - "kind": "dirExists", - "path": "project", - "weight": 2 - }, - { - "kind": "filePatternExists", - "glob": "src/main/scala/**/*.scala", - "weight": 2 - } - ] - }, - "detection": { - "minScore": 8 - } - }, - - "clojure-lein": { - "id": "clojure-lein", - "displayName": "Clojure (Leiningen)", - "category": "language", - "description": "Clojure projects using Leiningen.", - "tags": ["jvm", "functional"], - "searchRoots": ["."], - "indicators": { - "requiredAny": [ - { - "kind": "fileExists", - "path": "project.clj", - "weight": 8, - "rootRelative": true - } - ], - "optional": [ - { - "kind": "filePatternExists", - "glob": "src/**/*.clj", - "weight": 2 - } - ] - }, - "detection": { - "minScore": 8 - } - }, - - "clojure-tools-deps": { - "id": "clojure-tools-deps", - "displayName": "Clojure (tools.deps)", - "category": "language", - "description": "Clojure projects using tools.deps and deps.edn.", - "tags": ["jvm", "functional"], - "searchRoots": ["."], - "indicators": { - "requiredAny": [ - { - "kind": "fileExists", - "path": "deps.edn", - "weight": 8, - "rootRelative": true - } - ], - "optional": [ - { - "kind": "filePatternExists", - "glob": "src/**/*.clj", - "weight": 2 - } - ] - }, - "detection": { - "minScore": 8 - } - }, - - "r-lang": { - "id": "r-lang", - "displayName": "R", - "category": "language", - "description": "R packages or projects.", - "tags": ["data", "statistics"], - "searchRoots": ["."], - "indicators": { - "requiredAny": [ - { - "kind": "fileExists", - "path": "DESCRIPTION", - "weight": 7, - "rootRelative": true - } - ], - "optional": [ - { - "kind": "filePatternExists", - "glob": "**/*.R", - "weight": 2 - }, - { - "kind": "filePatternExists", - "glob": "**/*.r", - "weight": 1 - } - ] - }, - "detection": { - "minScore": 7 - } - }, - - "lua": { - "id": "lua", - "displayName": "Lua", - "category": "language", - "description": "Lua projects, optionally using LuaRocks.", - "tags": ["scripting", "embedded"], - "searchRoots": ["."], - "indicators": { - "requiredAny": [ - { - "kind": "filePatternExists", - "glob": "**/*.lua", - "weight": 5 - } - ], - "optional": [ - { - "kind": "filePatternExists", - "glob": "*.rockspec", - "weight": 3 - } - ] - }, - "detection": { - "minScore": 5 - } - }, - - "dart": { - "id": "dart", - "displayName": "Dart", - "category": "language", - "description": "Dart or Flutter projects using pubspec.yaml.", - "tags": ["mobile", "web", "cli"], - "searchRoots": ["."], - "indicators": { - "requiredAny": [ - { - "kind": "fileExists", - "path": "pubspec.yaml", - "weight": 8, - "rootRelative": true - } - ], - "optional": [ - { - "kind": "filePatternExists", - "glob": "**/*.dart", - "weight": 2 - } - ] - }, - "detection": { - "minScore": 8 - } - }, - - "react": { - "id": "react", - "displayName": "React", - "category": "framework", - "description": "React applications (web) built on top of Node.js.", - "tags": ["frontend", "web", "spa"], - "searchRoots": ["."], - "dependsOn": ["nodejs", "javascript"], - "indicators": { - "requiredAny": [ - { - "kind": "jsonField", - "path": "package.json", - "jsonPointer": "/dependencies/react", - "weight": 8, - "rootRelative": true - }, - { - "kind": "jsonField", - "path": "package.json", - "jsonPointer": "/devDependencies/react", - "weight": 7, - "rootRelative": true - } - ], - "optional": [ - { - "kind": "filePatternExists", - "glob": "src/**/*.jsx", - "weight": 2 - }, - { - "kind": "filePatternExists", - "glob": "src/**/*.tsx", - "weight": 2 - } - ] - }, - "detection": { - "minScore": 8 - } - }, - - "nextjs": { - "id": "nextjs", - "displayName": "Next.js", - "category": "framework", - "description": "Next.js React applications.", - "tags": ["frontend", "web", "ssr"], - "searchRoots": ["."], - "dependsOn": ["react", "nodejs"], - "indicators": { - "requiredAny": [ - { - "kind": "jsonField", - "path": "package.json", - "jsonPointer": "/dependencies/next", - "weight": 9, - "rootRelative": true - }, - { - "kind": "fileExists", - "path": "next.config.js", - "weight": 7, - "rootRelative": true - }, - { - "kind": "fileExists", - "path": "next.config.mjs", - "weight": 7, - "rootRelative": true - } - ], - "optional": [ - { - "kind": "dirExists", - "path": "pages", - "weight": 2 - }, - { - "kind": "dirExists", - "path": "app", - "weight": 2 - } - ] - }, - "detection": { - "minScore": 9 - } - }, - - "vue": { - "id": "vue", - "displayName": "Vue.js", - "category": "framework", - "description": "Vue.js SPA or multipage applications.", - "tags": ["frontend", "web"], - "searchRoots": ["."], - "dependsOn": ["nodejs", "javascript"], - "indicators": { - "requiredAny": [ - { - "kind": "jsonField", - "path": "package.json", - "jsonPointer": "/dependencies/vue", - "weight": 9, - "rootRelative": true - }, - { - "kind": "jsonField", - "path": "package.json", - "jsonPointer": "/devDependencies/vue", - "weight": 8, - "rootRelative": true - } - ], - "optional": [ - { - "kind": "fileExists", - "path": "vue.config.js", - "weight": 2, - "rootRelative": true - }, - { - "kind": "filePatternExists", - "glob": "src/**/*.vue", - "weight": 3 - } - ] - }, - "detection": { - "minScore": 9 - } - }, - - "angular": { - "id": "angular", - "displayName": "Angular", - "category": "framework", - "description": "Angular applications (Angular CLI).", - "tags": ["frontend", "web"], - "searchRoots": ["."], - "dependsOn": ["nodejs", "typescript"], - "indicators": { - "requiredAny": [ - { - "kind": "fileExists", - "path": "angular.json", - "weight": 9, - "rootRelative": true - } - ], - "optional": [ - { - "kind": "filePatternExists", - "glob": "src/**/*.ts", - "weight": 2 - }, - { - "kind": "filePatternExists", - "glob": "src/**/*.component.ts", - "weight": 2 - } - ] - }, - "detection": { - "minScore": 9 - } - }, - - "svelte": { - "id": "svelte", - "displayName": "Svelte", - "category": "framework", - "description": "Svelte and SvelteKit applications.", - "tags": ["frontend", "web"], - "searchRoots": ["."], - "dependsOn": ["nodejs", "javascript"], - "indicators": { - "requiredAny": [ - { - "kind": "jsonField", - "path": "package.json", - "jsonPointer": "/dependencies/svelte", - "weight": 9, - "rootRelative": true - } - ], - "optional": [ - { - "kind": "fileExists", - "path": "svelte.config.js", - "weight": 2, - "rootRelative": true - }, - { - "kind": "filePatternExists", - "glob": "src/**/*.svelte", - "weight": 3 - } - ] - }, - "detection": { - "minScore": 9 - } - }, - - "astro": { - "id": "astro", - "displayName": "Astro", - "category": "framework", - "description": "Astro content-focused web projects.", - "tags": ["frontend", "web", "ssg"], - "searchRoots": ["."], - "dependsOn": ["nodejs", "javascript"], - "indicators": { - "requiredAny": [ - { - "kind": "filePatternExists", - "glob": "astro.config.*", - "weight": 8 - }, - { - "kind": "jsonField", - "path": "package.json", - "jsonPointer": "/dependencies/astro", - "weight": 9, - "rootRelative": true - } - ], - "optional": [ - { - "kind": "filePatternExists", - "glob": "src/pages/**/*.astro", - "weight": 3 - } - ] - }, - "detection": { - "minScore": 8 - } - }, - - "express": { - "id": "express", - "displayName": "Express.js", - "category": "framework", - "description": "Express.js Node backend services.", - "tags": ["backend", "web", "api"], - "searchRoots": ["."], - "dependsOn": ["nodejs"], - "indicators": { - "requiredAny": [ - { - "kind": "jsonField", - "path": "package.json", - "jsonPointer": "/dependencies/express", - "weight": 9, - "rootRelative": true - }, - { - "kind": "jsonField", - "path": "package.json", - "jsonPointer": "/devDependencies/express", - "weight": 8, - "rootRelative": true - } - ], - "optional": [ - { - "kind": "filePatternExists", - "glob": "src/**/*express*.js", - "weight": 1 - } - ] - }, - "detection": { - "minScore": 9 - } - }, - - "nestjs": { - "id": "nestjs", - "displayName": "NestJS", - "category": "framework", - "description": "NestJS Node backend applications.", - "tags": ["backend", "web", "api"], - "searchRoots": ["."], - "dependsOn": ["nodejs", "typescript"], - "indicators": { - "requiredAny": [ - { - "kind": "jsonField", - "path": "package.json", - "jsonPointer": "/dependencies/@nestjs/core", - "weight": 9, - "rootRelative": true - }, - { - "kind": "fileExists", - "path": "nest-cli.json", - "weight": 8, - "rootRelative": true - } - ], - "optional": [ - { - "kind": "filePatternExists", - "glob": "src/**/*.module.ts", - "weight": 2 - } - ] - }, - "detection": { - "minScore": 9 - } - }, - - "django": { - "id": "django", - "displayName": "Django", - "category": "framework", - "description": "Django web applications.", - "tags": ["backend", "web"], - "searchRoots": ["."], - "dependsOn": ["python"], - "indicators": { - "requiredAny": [ - { - "kind": "fileExists", - "path": "manage.py", - "weight": 8, - "rootRelative": true - } - ], - "optional": [ - { - "kind": "filePatternExists", - "glob": "**/settings.py", - "weight": 2 - }, - { - "kind": "fileContains", - "path": "requirements.txt", - "regex": "django", - "weight": 2, - "rootRelative": true - } - ] - }, - "detection": { - "minScore": 8 - } - }, - - "flask": { - "id": "flask", - "displayName": "Flask", - "category": "framework", - "description": "Flask microservices or apps.", - "tags": ["backend", "web"], - "searchRoots": ["."], - "dependsOn": ["python"], - "indicators": { - "requiredAny": [ - { - "kind": "fileContains", - "path": "requirements.txt", - "regex": "flask", - "weight": 8, - "rootRelative": true - } - ], - "optional": [ - { - "kind": "filePatternExists", - "glob": "**/*flask*.py", - "weight": 2 - } - ] - }, - "detection": { - "minScore": 8 - } - }, - - "fastapi": { - "id": "fastapi", - "displayName": "FastAPI", - "category": "framework", - "description": "FastAPI async web APIs.", - "tags": ["backend", "web", "api"], - "searchRoots": ["."], - "dependsOn": ["python"], - "indicators": { - "requiredAny": [ - { - "kind": "fileContains", - "path": "requirements.txt", - "regex": "fastapi", - "weight": 8, - "rootRelative": true - } - ], - "optional": [ - { - "kind": "filePatternExists", - "glob": "**/*fastapi*.py", - "weight": 2 - } - ] - }, - "detection": { - "minScore": 8 - } - }, - - "laravel": { - "id": "laravel", - "displayName": "Laravel", - "category": "framework", - "description": "Laravel PHP web applications.", - "tags": ["backend", "web"], - "searchRoots": ["."], - "dependsOn": ["php-composer"], - "indicators": { - "requiredAny": [ - { - "kind": "fileExists", - "path": "artisan", - "weight": 8, - "rootRelative": true - } - ], - "optional": [ - { - "kind": "fileContains", - "path": "composer.json", - "regex": "laravel/framework", - "weight": 3, - "rootRelative": true - } - ] - }, - "detection": { - "minScore": 8 - } - } - } -} diff --git a/tests/AST_SEARCH_TESTS.md b/tests/AST_SEARCH_TESTS.md deleted file mode 100644 index 28086a5..0000000 --- a/tests/AST_SEARCH_TESTS.md +++ /dev/null @@ -1,296 +0,0 @@ -# AST Search Tests Documentation - -## Overview - -Comprehensive test suite for AST-based structural code search using @ast-grep/napi. Tests cover pattern matching, rule-based search, metavariable extraction, and full MCP integration. - -## Test Structure - -### Unit Tests (`tests/unit/ast-search.test.ts`) - -**Purpose**: Test AST search service methods in isolation - -**Coverage**: -- ✅ ast-grep availability check -- ✅ Rule validation (atomic, relational, composite) -- ✅ Pattern validation from ast-grep/claude-skill -- ✅ Edge cases (empty rules, invalid stopBy, etc.) -- ✅ Complex nested rule validation - -**Key Test Cases**: -```typescript -// Pattern rules -{ pattern: 'function $NAME() { $$$ }' } - -// Kind rules -{ kind: 'function_declaration' } - -// Composite rules -{ all: [pattern1, pattern2] } -{ any: [pattern1, pattern2] } -{ not: { pattern } } - -// Relational rules -{ inside: { pattern, stopBy: 'end' } } -{ has: { pattern, stopBy: 'end' } } -``` - -### Integration Tests (`tests/integration/ast-search.test.ts`) - -**Purpose**: Test actual AST parsing and search on real code files - -**Coverage**: -- ✅ Pattern search with metavariable extraction -- ✅ Rule-based search (all operators) -- ✅ JavaScript and TypeScript support -- ✅ Error handling (empty patterns, non-existent paths, malformed files) -- ✅ Limit parameter enforcement - -**Test Files Created**: -- `test.js`: Contains async functions, regular functions, variables, try-catch -- `test.ts`: Contains interfaces, typed functions, classes - -**Key Patterns Tested**: -```javascript -// Find async functions -'async function $NAME($$$) { $$$ }' - -// Find functions with console.log -{ - pattern: 'console.log($$$)', - inside: { pattern: 'function $NAME($$$) { $$$ }' } -} - -// Find variable declarations -{ - any: [ - { pattern: 'const $VAR = $$$' }, - { pattern: 'let $VAR = $$$' }, - { pattern: 'var $VAR = $$$' } - ] -} -``` - -### MCP Integration Tests (`tests/integration/ast-mcp-integration.test.ts`) - -**Purpose**: Test AST search through MCP server interface with workspace management - -**Coverage**: -- ✅ search_ast_pattern tool -- ✅ search_ast_rule tool -- ✅ check_ast_grep tool -- ✅ Workspace integration -- ✅ Metavariable extraction -- ✅ Real-world patterns (React useEffect, error handling) - -**Realistic Patterns**: -```typescript -// Find React useEffect without dependencies -{ - all: [ - { pattern: 'useEffect($CALLBACK)' }, - { not: { pattern: 'useEffect($CALLBACK, [$$$])' } } - ] -} - -// Find async functions without await -{ - all: [ - { pattern: 'async function $NAME($$$) { $$$ }' }, - { not: { has: { pattern: 'await $$$', stopBy: 'end' } } } - ] -} - -// Find try-catch without error logging -{ - all: [ - { pattern: 'try { $$$ } catch ($E) { $$$ }' }, - { not: { has: { pattern: 'console.error($$$)', stopBy: 'end' } } } - ] -} -``` - -## Running Tests - -### Prerequisites -```bash -npm install # Install dependencies including @ast-grep/napi -``` - -### Run All Tests -```bash -npm test -``` - -### Run Specific Test Suites -```bash -# Unit tests only -npm test -- tests/unit/ast-search.test.ts - -# Integration tests only -npm test -- tests/integration/ast-search.test.ts - -# MCP integration tests -npm test -- tests/integration/ast-mcp-integration.test.ts -``` - -### Run with Coverage -```bash -npm test -- --coverage -``` - -## Test Behavior - -### Graceful Degradation -All tests check for ast-grep availability and skip gracefully if not available: -```typescript -const info = await service.isAvailable(); -if (!info.available) { - console.warn('ast-grep not available - skipping tests'); - return; -} -``` - -### Error Handling -Tests verify robust error handling: -- ✅ Empty patterns return empty results (no throw) -- ✅ Non-existent paths return empty results -- ✅ Malformed files are skipped with console.error -- ✅ Invalid rules return validation errors - -### Cleanup -All tests use proper setup/teardown: -```typescript -beforeAll(async () => { - // Create temp directories and test files -}); - -afterAll(async () => { - // Clean up temp directories -}); -``` - -## Expected Results - -### When ast-grep is Available (Normal Case) -- ✅ All tests should pass -- ✅ Pattern matching finds correct code structures -- ✅ Metavariables are extracted properly -- ✅ Rules correctly combine with AND/OR/NOT logic -- ✅ Relational rules (inside, has) work correctly - -### When ast-grep is Not Available -- ✅ Tests skip gracefully with warnings -- ✅ No test failures -- ✅ isAvailable() returns `{ available: false, error: '...' }` - -## Coverage Goals - -Target: **90%+ code coverage** for AST search module - -### Core Functions -- ✅ searchPattern() -- ✅ searchRule() -- ✅ applyRule() (all rule types) -- ✅ validateRule() -- ✅ extractMetavariables() -- ✅ getFilesToSearch() - -### Edge Cases -- ✅ Empty patterns -- ✅ Invalid rules -- ✅ Malformed source files -- ✅ Non-existent paths -- ✅ Empty result sets -- ✅ Limit enforcement - -## Test Data - -### JavaScript Test File -Contains realistic patterns: -- Regular functions -- Async functions (with/without await) -- Arrow functions -- Variable declarations (const, let, var) -- Try-catch blocks -- Classes with methods - -### TypeScript Test File -Contains type-specific patterns: -- Interfaces -- Typed function signatures -- Type aliases -- Typed class methods -- Async/await with types - -## Assertions - -### Common Assertions -```typescript -// Basic structure -expect(result.workspaceId).toBe('test-workspace'); -expect(result.language).toBe('javascript'); -expect(result.matches).toBeInstanceOf(Array); - -// Match properties -expect(match).toHaveProperty('file'); -expect(match).toHaveProperty('line'); -expect(match).toHaveProperty('text'); - -// Metavariables -expect(match.metaVariables).toBeDefined(); -expect(match.metaVariables.NAME).toBeDefined(); -expect(match.metaVariables.NAME.text).toBe('functionName'); - -// Validation -expect(validation.valid).toBe(true); -expect(validation.errors).toHaveLength(0); -``` - -## Production Readiness - -These tests ensure: -1. ✅ **Bundled binaries work** - Tests verify @ast-grep/napi loads correctly -2. ✅ **Cross-platform support** - Tests run on Linux, macOS, Windows -3. ✅ **Error resilience** - Graceful handling of all error cases -4. ✅ **Real-world patterns** - Tests based on actual use cases -5. ✅ **Performance** - Limits and timeouts configured appropriately -6. ✅ **Integration** - Full MCP server workflow tested - -## Debugging Tests - -### Enable Verbose Output -```bash -npm test -- --verbose -``` - -### Run Single Test -```bash -npm test -- --testNamePattern="should find async functions" -``` - -### Check ast-grep Status -```typescript -const service = new ASTSearchService(); -const info = await service.isAvailable(); -console.log(info); -// { available: true, version: '0.40.0', path: 'bundled (native)' } -``` - -## Continuous Integration - -Tests are designed for CI/CD: -- ✅ No external dependencies (bundled) -- ✅ Fast execution (< 30 seconds for all tests) -- ✅ Deterministic results -- ✅ Proper timeouts (120 seconds max) -- ✅ Clean setup/teardown - -## Future Improvements - -Potential additions: -- [ ] Performance benchmarks -- [ ] More language coverage (Python, Rust, Go) -- [ ] Stress tests with large codebases -- [ ] Concurrent search tests -- [ ] Cache behavior tests