From b7444230fbb1990f430456288118d4f05494a8f6 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Wed, 21 Jan 2026 11:22:41 +0000 Subject: [PATCH 1/9] Initial plan From cb85b4239005fb61163a8a6c43fafbd3d4776968 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Wed, 21 Jan 2026 11:30:34 +0000 Subject: [PATCH 2/9] Add comprehensive migration strategy and architecture documentation Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com> --- FEATURE_MIGRATION_MATRIX.md | 266 ++++++++++++++ MIGRATION_STRATEGY.md | 550 ++++++++++++++++++++++++++++ PLUGIN_ARCHITECTURE.md | 692 ++++++++++++++++++++++++++++++++++++ 3 files changed, 1508 insertions(+) create mode 100644 FEATURE_MIGRATION_MATRIX.md create mode 100644 MIGRATION_STRATEGY.md create mode 100644 PLUGIN_ARCHITECTURE.md diff --git a/FEATURE_MIGRATION_MATRIX.md b/FEATURE_MIGRATION_MATRIX.md new file mode 100644 index 00000000..5a7de5bb --- /dev/null +++ b/FEATURE_MIGRATION_MATRIX.md @@ -0,0 +1,266 @@ +# Feature Migration Matrix: ObjectQL → @objectstack/runtime + +This matrix maps current ObjectQL features to their destinations in the @objectstack/runtime architecture. + +## Legend +- ✅ **Keep in @objectql** - Query-specific, stays in this repository as plugin +- ⬆️ **Move to @objectstack/runtime** - Core functionality, should be in base runtime +- 🔄 **Refactor as Plugin** - Convert to plugin architecture +- 🗑️ **Deprecate** - Remove or replace with runtime equivalent + +## Feature Comparison Matrix + +| Feature | Current Location | Decision | Target Location | Priority | Effort | Notes | +|---------|-----------------|----------|-----------------|----------|--------|-------| +| **Application Lifecycle** | +| App initialization | @objectql/core/app.ts | ⬆️ | @objectstack/runtime | P0 | High | Core runtime responsibility | +| App context management | @objectql/core/app.ts | ⬆️ | @objectstack/runtime | P0 | High | Unified context across all plugins | +| Plugin system | @objectql/core/app.ts | ⬆️ | @objectstack/runtime | P0 | High | Standard plugin architecture | +| Lifecycle hooks | @objectql/core/app.ts | ⬆️ | @objectstack/runtime | P0 | Medium | Standardize lifecycle events | +| **Metadata & Schema** | +| Metadata registry | @objectql/types | ⬆️ | @objectstack/runtime | P0 | High | Central schema registry | +| Object registration | @objectql/core/object.ts | ⬆️ | @objectstack/runtime | P0 | Medium | Runtime manages all objects | +| Field definitions | @objectql/types | ⬆️ | @objectstack/runtime | P0 | Medium | Standard field types | +| Relationship definitions | @objectql/types | ⬆️ | @objectstack/runtime | P0 | Medium | Standard relationship model | +| Index definitions | @objectql/types | ⬆️ | @objectstack/runtime | P1 | Low | Database-agnostic indexes | +| **Driver Management** | +| Driver interface | @objectstack/spec | ✅ | @objectstack/spec | P0 | Low | Already using spec | +| Driver registration | @objectql/core/app.ts | ⬆️ | @objectstack/runtime | P0 | Medium | Runtime orchestrates drivers | +| Driver lifecycle | @objectql/core/app.ts | ⬆️ | @objectstack/runtime | P0 | Medium | Connect/disconnect management | +| Multi-datasource support | @objectql/core/app.ts | ⬆️ | @objectstack/runtime | P0 | Medium | Runtime feature | +| **Query Functionality** | +| Query AST definition | @objectstack/spec | ✅ | @objectstack/spec | P0 | Low | Standard protocol | +| Query validation | @objectql/core/validator.ts | 🔄 | @objectql/query-validation | P0 | High | Query-specific validation plugin | +| Query optimization | Not implemented | 🔄 | @objectql/query-optimizer | P2 | High | New plugin capability | +| Query caching | Not implemented | 🔄 | @objectql/query-cache | P2 | Medium | New plugin capability | +| Query builder | @objectql/core/repository.ts | 🔄 | @objectql/query-builder | P1 | Medium | Fluent query construction | +| **Repository Pattern** | +| ObjectRepository class | @objectql/core/repository.ts | 🔄 | @objectql/advanced-repository | P0 | High | Enhanced repository as plugin | +| CRUD operations | @objectql/core/repository.ts | ✅ | @objectql/advanced-repository | P0 | Medium | Query-specific operations | +| Batch operations | @objectql/core/repository.ts | ✅ | @objectql/advanced-repository | P1 | Low | Advanced query feature | +| Transaction support | @objectql/core/repository.ts | ⬆️ | @objectstack/runtime | P0 | High | Runtime manages transactions | +| **Validation Engine** | +| Field validators | @objectql/core/validator.ts | 🔄 | @objectql/query-validation | P0 | Medium | Query data validation | +| Cross-field validation | @objectql/core/validator.ts | 🔄 | @objectql/query-validation | P1 | Medium | Query-level validation | +| Custom validators | @objectql/core/validator.ts | 🔄 | @objectql/query-validation | P1 | Low | Plugin extensibility | +| Validation messages | @objectql/core/validator.ts | 🔄 | @objectql/query-validation | P1 | Low | Error formatting | +| Validation rules YAML | @objectql/types | ⬆️ | @objectstack/runtime | P1 | Medium | Metadata-driven validation | +| **Hooks System** | +| Hook registration | @objectql/core/hook.ts | ⬆️ | @objectstack/runtime | P0 | Medium | Standard event system | +| Hook execution | @objectql/core/hook.ts | ⬆️ | @objectstack/runtime | P0 | Medium | Runtime event dispatcher | +| CRUD hooks | @objectql/core/hook.ts | ✅ | @objectql/advanced-repository | P0 | Low | Query lifecycle events | +| Validation hooks | @objectql/core/hook.ts | 🔄 | @objectql/query-validation | P1 | Low | Validation lifecycle | +| **Actions System** | +| Action registration | @objectql/core/action.ts | ⬆️ | @objectstack/runtime | P0 | Medium | Runtime command pattern | +| Action execution | @objectql/core/action.ts | ⬆️ | @objectstack/runtime | P0 | Medium | Runtime dispatcher | +| Custom actions | @objectql/core/action.ts | 🔄 | Plugin-based | P1 | Low | Plugin actions | +| **Formula Engine** | +| Formula parsing | @objectql/core/formula-engine.ts | 🔄 | @objectql/formula-engine | P1 | Medium | Specialized plugin | +| Expression evaluation | @objectql/core/formula-engine.ts | 🔄 | @objectql/formula-engine | P1 | Medium | Query expression support | +| Built-in functions | @objectql/core/formula-engine.ts | 🔄 | @objectql/formula-engine | P1 | Low | Standard function library | +| Custom functions | @objectql/core/formula-engine.ts | 🔄 | @objectql/formula-engine | P2 | Low | Plugin extensibility | +| **AI Integration** | +| AI agent interface | @objectql/core/ai-agent.ts | 🔄 | @objectql/ai-query-generator | P2 | Medium | AI query generation plugin | +| Natural language to query | @objectql/core/ai-agent.ts | 🔄 | @objectql/ai-query-generator | P2 | High | LLM integration | +| Query suggestions | Not implemented | 🔄 | @objectql/ai-query-generator | P3 | High | AI-powered suggestions | +| **Type System** | +| Core types | @objectql/types | ⬆️ | @objectstack/spec | P0 | High | Standard protocol types | +| Driver types | @objectql/types | ✅ | @objectstack/spec | P0 | Low | Already using spec | +| Query types | @objectql/types | ✅ | @objectstack/spec | P0 | Low | Standard QueryAST | +| Repository types | @objectql/types | ✅ | @objectql/types | P1 | Low | Query-specific types | +| **Drivers** | +| SQL Driver | @objectql/driver-sql | 🔄 | @objectql/driver-sql | P0 | Medium | Convert to plugin | +| MongoDB Driver | @objectql/driver-mongo | 🔄 | @objectql/driver-mongo | P0 | Medium | Convert to plugin | +| Memory Driver | @objectql/driver-memory | 🔄 | @objectql/driver-memory | P0 | Medium | Convert to plugin | +| LocalStorage Driver | @objectql/driver-localstorage | 🔄 | @objectql/driver-localstorage | P1 | Medium | Convert to plugin | +| FS Driver | @objectql/driver-fs | 🔄 | @objectql/driver-fs | P1 | Low | Convert to plugin | +| Excel Driver | @objectql/driver-excel | 🔄 | @objectql/driver-excel | P2 | Low | Convert to plugin | +| Redis Driver | @objectql/driver-redis | 🔄 | @objectql/driver-redis | P2 | Low | Convert to plugin | +| SDK Driver | @objectql/driver-sdk | 🔄 | @objectql/driver-sdk | P0 | Medium | Convert to plugin | +| **Platform Integration** | +| Node.js utilities | @objectql/platform-node | ⬆️ | @objectstack/platform-node | P0 | Medium | Platform-specific runtime | +| File system integration | @objectql/platform-node | ⬆️ | @objectstack/platform-node | P0 | Low | Runtime filesystem access | +| YAML loader | @objectql/platform-node | ⬆️ | @objectstack/platform-node | P0 | Low | Metadata loading | +| Plugin loader | @objectql/platform-node | ⬆️ | @objectstack/platform-node | P0 | Medium | Dynamic plugin loading | +| **Server & Runtime** | +| HTTP server | @objectql/server | 🔄 | @objectql/server | P1 | Medium | Query API server plugin | +| REST API | @objectql/server | 🔄 | @objectql/server | P1 | Low | REST query endpoints | +| GraphQL API | Not implemented | 🔄 | @objectql/graphql-server | P2 | High | GraphQL query plugin | +| Metadata API | @objectql/server | ⬆️ | @objectstack/runtime | P1 | Medium | Runtime metadata API | +| **Developer Tools** | +| CLI | @objectql/cli | 🔄 | @objectql/cli | P1 | High | Generate runtime-compatible code | +| Project scaffolding | @objectql/create | 🔄 | @objectql/create | P1 | Medium | Runtime-based templates | +| VSCode extension | @objectql/vscode-objectql | 🔄 | @objectql/vscode-objectql | P1 | Medium | Update for runtime | +| Dev server | @objectql/cli | 🔄 | @objectql/cli | P1 | Low | Runtime dev mode | +| **Security & Permissions** | +| RBAC system | Not implemented | ⬆️ | @objectstack/runtime | P0 | High | Core security feature | +| Permission checking | Not implemented | ⬆️ | @objectstack/runtime | P0 | High | Runtime authorization | +| Field-level security | Not implemented | ⬆️ | @objectstack/runtime | P1 | Medium | Metadata-driven security | +| Row-level security | Not implemented | 🔄 | @objectql/query-security | P1 | High | Query-level security plugin | +| **Performance** | +| Connection pooling | Driver-specific | ⬆️ | @objectstack/runtime | P0 | Medium | Runtime manages pools | +| Query caching | Not implemented | 🔄 | @objectql/query-cache | P2 | Medium | Query optimization plugin | +| Result caching | Not implemented | 🔄 | @objectql/query-cache | P2 | Low | Query optimization plugin | +| Lazy loading | Partial | 🔄 | @objectql/advanced-repository | P2 | Low | Repository feature | +| **Testing & Quality** | +| Test utilities | Various | ⬆️ | @objectstack/testing | P1 | Medium | Standard test helpers | +| Mock drivers | @objectql/driver-memory | ⬆️ | @objectstack/testing | P1 | Low | Testing infrastructure | +| Test fixtures | Various | ⬆️ | @objectstack/testing | P2 | Low | Reusable test data | + +## Summary Statistics + +### By Decision Type +- ⬆️ **Move to @objectstack/runtime**: 25 features (42%) +- 🔄 **Refactor as Plugin**: 28 features (47%) +- ✅ **Keep in @objectql**: 7 features (11%) +- 🗑️ **Deprecate**: 0 features (0%) + +### By Priority +- **P0 (Critical)**: 35 features (58%) +- **P1 (High)**: 18 features (30%) +- **P2 (Medium)**: 6 features (10%) +- **P3 (Low)**: 1 feature (2%) + +### By Effort +- **High**: 18 features (30%) +- **Medium**: 30 features (50%) +- **Low**: 12 features (20%) + +## Migration Waves + +### Wave 1: Foundation (Weeks 1-4) +**Dependencies:** None +**Features:** Core runtime integration, metadata registry, driver management, plugin system + +**Includes:** +- App initialization → @objectstack/runtime +- Context management → @objectstack/runtime +- Metadata registry → @objectstack/runtime +- Driver registration → @objectstack/runtime +- Plugin system → @objectstack/runtime + +### Wave 2: Drivers (Weeks 5-6) +**Dependencies:** Wave 1 +**Features:** Convert all drivers to plugin architecture + +**Includes:** +- SQL Driver → Plugin +- MongoDB Driver → Plugin +- Memory Driver → Plugin +- SDK Driver → Plugin +- All other drivers → Plugins + +### Wave 3: Core Plugins (Weeks 7-10) +**Dependencies:** Wave 1, Wave 2 +**Features:** Essential query functionality as plugins + +**Includes:** +- Query validation plugin +- Advanced repository plugin +- Query builder plugin +- Formula engine plugin + +### Wave 4: Enhanced Plugins (Weeks 11-14) +**Dependencies:** Wave 3 +**Features:** Advanced query capabilities + +**Includes:** +- Query optimizer plugin +- Query cache plugin +- AI query generator plugin +- Query security plugin + +### Wave 5: Tools & Integration (Weeks 15-18) +**Dependencies:** All previous waves +**Features:** Developer tools and integrations + +**Includes:** +- CLI updates +- VSCode extension updates +- Server updates +- Create tool updates + +### Wave 6: Polish & Release (Weeks 19-22) +**Dependencies:** All previous waves +**Features:** Testing, documentation, migration support + +**Includes:** +- Comprehensive testing +- Documentation updates +- Migration guides +- Example updates + +## Critical Path Analysis + +### Must-Have for MVP +1. Runtime kernel integration (Wave 1) +2. SQL/Memory driver plugins (Wave 2) +3. Query validation plugin (Wave 3) +4. Advanced repository plugin (Wave 3) +5. CLI updates (Wave 5) + +### Can Be Deferred +1. Query optimizer plugin +2. Query cache plugin +3. AI query generator plugin +4. GraphQL server plugin +5. Non-critical drivers (Excel, Redis) + +## Risk Assessment by Feature + +### High Risk Features +1. **App initialization** - Complex state management +2. **Transaction support** - Distributed transaction complexity +3. **Natural language to query** - AI integration challenges +4. **RBAC system** - Security critical +5. **Connection pooling** - Performance critical + +### Medium Risk Features +1. **Metadata registry migration** - Data migration needed +2. **Hook system refactor** - Event system compatibility +3. **Driver plugin conversion** - Multiple driver types +4. **Query validation** - Complex validation rules + +### Low Risk Features +1. **Type exports** - Already using @objectstack/spec +2. **YAML loader** - Well-defined functionality +3. **Custom validators** - Clear plugin API +4. **File system integration** - Platform-specific utilities + +## Backward Compatibility Strategy + +### Deprecated APIs (v4.0.0) +```typescript +// OLD: Direct ObjectQL instantiation +const app = new ObjectQL({ datasources: {...} }); + +// NEW: Runtime-based +const runtime = createRuntime({ + plugins: [sqlDriverPlugin({...})] +}); +``` + +### Compatibility Layer Duration +- **v4.0.x**: Full backward compatibility with warnings +- **v4.1.x**: Deprecated API warnings +- **v5.0.x**: Remove deprecated APIs + +### Migration Tools +1. Automated code migration script +2. Configuration converter +3. Plugin wrapper generator +4. Compatibility checker CLI + +## Next Steps + +1. **Review & Approve** this matrix with the team +2. **Prioritize** features based on business needs +3. **Design** plugin interfaces for Wave 1 +4. **Prototype** core runtime integration +5. **Plan** detailed sprints for each wave + +--- + +**Document Version:** 1.0.0 +**Last Updated:** 2026-01-21 +**Status:** Draft - Pending Review diff --git a/MIGRATION_STRATEGY.md b/MIGRATION_STRATEGY.md new file mode 100644 index 00000000..fea9aa62 --- /dev/null +++ b/MIGRATION_STRATEGY.md @@ -0,0 +1,550 @@ +# ObjectQL Migration to @objectstack/runtime Architecture + +## Executive Summary + +This document outlines the comprehensive strategy for migrating ObjectQL from a standalone framework to a **plugin-based architecture** built on top of @objectstack/runtime. The goal is to reposition this repository as a specialized **query extension plugin** for the ObjectStack ecosystem. + +## Vision & Objectives + +### New Repository Role +**From:** Full-featured, standalone ORM framework +**To:** Specialized query plugin repository for @objectstack/runtime + +### Core Principles +1. **Plugin-First Architecture**: All query-related functionality as composable plugins +2. **Runtime Delegation**: Leverage @objectstack/runtime for application lifecycle, metadata, and core operations +3. **Backward Compatibility**: Maintain API compatibility during transition period +4. **Minimal Core**: Keep only query-specific logic in this repository + +## Current State Analysis + +### Existing @objectstack Integration (v3.0.1) + +✅ **Already Integrated:** +- `@objectstack/spec@0.1.2` - Driver interface protocol +- `@objectstack/runtime@0.1.1` - Runtime kernel (type exports only) +- `@objectstack/objectql@0.1.1` - Core engine (type exports only) + +❌ **Not Yet Using:** +- @objectstack/runtime application lifecycle +- @objectstack/runtime plugin system +- @objectstack/runtime metadata registry +- @objectstack/runtime context management +- @objectstack/runtime validation engine + +### Current Architecture + +``` +@objectql/core (Current - Standalone) +├── Own metadata registry +├── Own driver management +├── Own hooks system +├── Own actions system +├── Own validation engine +├── Own formula engine +├── Own AI agent integration +└── Own repository pattern +``` + +### Target Architecture + +``` +@objectstack/runtime (Foundation) +├── Application lifecycle +├── Metadata registry +├── Plugin system +├── Context management +├── Core validation +└── Driver orchestration + +@objectql/* (Query Plugins) +├── @objectql/query-validation (plugin) +├── @objectql/query-optimizer (plugin) +├── @objectql/query-cache (plugin) +├── @objectql/formula-engine (plugin) +├── @objectql/ai-query-generator (plugin) +└── @objectql/advanced-repository (plugin) +``` + +## Migration Strategy + +### Phase-Based Approach + +#### Phase 1: Foundation Analysis (Week 1-2) +**Goal:** Deep understanding and planning + +**Tasks:** +1. Map all @objectql/core functionality to @objectstack/runtime equivalents +2. Identify functionality that belongs in @objectstack/runtime vs. plugins +3. Create detailed dependency graph +4. Design plugin interfaces and boundaries +5. Plan backward compatibility strategy + +**Deliverables:** +- Detailed feature comparison matrix +- Plugin architecture design document +- Migration decision tree +- Rollback plan + +#### Phase 2: Runtime Kernel Integration (Week 3-4) +**Goal:** Replace core engine with @objectstack/runtime + +**Tasks:** +1. Replace ObjectQL app class with @objectstack/runtime kernel +2. Migrate initialization flow to runtime bootstrap +3. Update metadata registry to use runtime's registry +4. Implement compatibility layer for existing APIs +5. Update tests to validate runtime integration + +**Breaking Changes:** +- ObjectQL constructor signature changes +- Initialization pattern changes +- Internal APIs deprecated + +**Compatibility Layer:** +```typescript +// Old API (deprecated but supported) +const app = new ObjectQL({ datasources: {...} }); +await app.init(); + +// New API (preferred) +import { createRuntime } from '@objectstack/runtime'; +import { queryPlugin } from '@objectql/query-plugin'; + +const runtime = createRuntime({ + plugins: [queryPlugin()] +}); +await runtime.initialize(); +``` + +#### Phase 3: Driver Plugin Migration (Week 5-6) +**Goal:** Drivers as @objectstack/runtime plugins + +**Tasks:** +1. Create driver plugin wrapper for @objectstack/runtime +2. Migrate each driver to plugin model +3. Update driver registration to runtime plugin system +4. Implement driver lifecycle hooks +5. Update driver documentation + +**Plugin Structure:** +```typescript +export function sqlDriverPlugin(config: SQLConfig): RuntimePlugin { + return { + name: '@objectql/driver-sql', + version: '4.0.0', + + async setup(runtime: Runtime) { + const driver = new SQLDriver(config); + runtime.registerDriver(driver); + }, + + async teardown(runtime: Runtime) { + await runtime.getDriver('sql').disconnect(); + } + }; +} +``` + +#### Phase 4: Query Functionality as Plugins (Week 7-10) +**Goal:** Extract all query-related logic into composable plugins + +**Plugins to Create:** + +1. **@objectql/query-validation** + - Field validation + - Cross-field validation + - Custom validators + - Error formatting + +2. **@objectql/query-optimizer** + - Query plan optimization + - Index hints + - Join optimization + - Projection pruning + +3. **@objectql/query-cache** + - Result caching + - Query normalization + - Cache invalidation + - TTL management + +4. **@objectql/formula-engine** + - Formula parsing + - Expression evaluation + - Built-in functions + - Custom function registration + +5. **@objectql/ai-query-generator** + - Natural language to query AST + - Query suggestion + - Semantic understanding + - Context-aware generation + +6. **@objectql/advanced-repository** + - Enhanced CRUD operations + - Batch operations + - Transaction management + - Event tracking + +**Plugin API Design:** +```typescript +export interface QueryPlugin extends RuntimePlugin { + // Query lifecycle hooks + beforeQuery?(ast: QueryAST): Promise; + afterQuery?(result: any): Promise; + + // Validation hooks + validateQuery?(ast: QueryAST): Promise; + + // Optimization hooks + optimizeQuery?(ast: QueryAST): Promise; +} +``` + +#### Phase 5: Hooks & Actions as Runtime Features (Week 11-12) +**Goal:** Align with @objectstack/runtime event system + +**Tasks:** +1. Map ObjectQL hooks to runtime events +2. Migrate action system to runtime commands +3. Update registration APIs +4. Implement backward compatibility +5. Update event documentation + +**Mapping:** +```typescript +// Old: ObjectQL hooks +app.on('beforeCreate', 'users', async (ctx) => {}); + +// New: Runtime events +runtime.on('entity.beforeCreate', 'users', async (ctx) => {}); + +// Old: ObjectQL actions +app.registerAction('users', 'approve', async (ctx) => {}); + +// New: Runtime commands +runtime.registerCommand('users.approve', async (ctx) => {}); +``` + +#### Phase 6: Repository Pattern Enhancement (Week 13-14) +**Goal:** Repository as thin wrapper over runtime context + +**Tasks:** +1. Refactor ObjectRepository to use runtime context +2. Delegate operations to runtime +3. Add plugin hooks to repository methods +4. Update query builder integration +5. Enhance with plugin capabilities + +**New Repository:** +```typescript +class RuntimeRepository { + constructor( + private runtime: Runtime, + private objectName: string, + private context: RuntimeContext + ) {} + + async find(query: QueryAST) { + // Plugin hooks: beforeQuery + const optimized = await this.runtime.executeHook( + 'query.beforeExecute', + query + ); + + // Execute via runtime + const result = await this.runtime.execute(optimized); + + // Plugin hooks: afterQuery + return await this.runtime.executeHook( + 'query.afterExecute', + result + ); + } +} +``` + +#### Phase 7: Tools & CLI Update (Week 15-16) +**Goal:** Tools that generate @objectstack/runtime compatible code + +**Tasks:** +1. Update CLI to generate runtime-based projects +2. Update create templates +3. Update VSCode extension schemas +4. Update dev server for runtime +5. Update code generation + +#### Phase 8: Documentation & Examples (Week 17-18) +**Goal:** Comprehensive documentation for new architecture + +**Documents to Create/Update:** +1. Migration Guide (detailed steps) +2. Plugin Development Guide +3. Architecture Overview +4. API Reference (new APIs) +5. Breaking Changes Guide +6. FAQ for migration + +**Examples to Update:** +- Hello World → Runtime-based +- Project Tracker → Plugin composition +- Enterprise ERP → Advanced plugins +- Browser Integration → Runtime in browser +- Custom Driver → Runtime driver plugin + +#### Phase 9: Testing & Validation (Week 19-20) +**Goal:** Ensure quality and compatibility + +**Testing Strategy:** +1. Unit tests for all plugins +2. Integration tests for runtime +3. Backward compatibility tests +4. Performance benchmarks +5. Security audit +6. Cross-environment testing + +**Quality Gates:** +- ✅ 90%+ test coverage +- ✅ All existing tests pass with compatibility layer +- ✅ No performance regression +- ✅ Zero high-severity security issues +- ✅ All examples functional + +#### Phase 10: Release & Support (Week 21-22) +**Goal:** Smooth transition for users + +**Release Plan:** +1. Alpha release (internal testing) +2. Beta release (early adopters) +3. RC release (community feedback) +4. GA release (production ready) + +**Version Strategy:** +- v3.x → v4.0.0 (major breaking change) +- v3.x maintenance for 6 months +- Deprecation warnings in v3.x +- Migration tools provided + +## Technical Decisions + +### 1. Backward Compatibility Strategy + +**Approach:** Dual API support during transition + +```typescript +// @objectql/core v4.0.0 +export class ObjectQL { + private runtime: Runtime; + + constructor(config: ObjectQLConfig | RuntimeConfig) { + if (isLegacyConfig(config)) { + console.warn('Legacy API detected. Please migrate to @objectstack/runtime'); + this.runtime = createRuntimeFromLegacy(config); + } else { + this.runtime = createRuntime(config); + } + } +} +``` + +**Deprecation Timeline:** +- v4.0.0: Legacy API supported with warnings +- v4.1.0: Legacy API deprecated (still functional) +- v5.0.0: Legacy API removed + +### 2. Plugin Loading Strategy + +**Options:** +1. **Static Import** - Compile-time plugin inclusion +2. **Dynamic Import** - Runtime plugin loading +3. **Hybrid** - Core plugins static, extensions dynamic + +**Decision:** Hybrid approach +- Core query plugins: Static (bundled) +- Extended plugins: Dynamic (on-demand) +- Driver plugins: Dynamic (environment-specific) + +### 3. Metadata Ownership + +**Decision:** Migrate to @objectstack/runtime metadata registry + +**Rationale:** +- Centralized schema management +- Cross-plugin metadata sharing +- Consistent validation +- Better tooling support + +**Migration:** +```typescript +// Old: Local registry +this.metadata = new MetadataRegistry(); + +// New: Runtime registry +const registry = runtime.getRegistry(); +registry.registerObject({...}); +``` + +### 4. Context Management + +**Decision:** Use @objectstack/runtime context + +**Benefits:** +- Unified context across plugins +- Better security (RBAC integrated) +- Consistent transaction management +- Audit trail built-in + +### 5. Query AST Standard + +**Decision:** Adopt @objectstack/spec QueryAST + +**Impact:** +- All drivers use same AST format +- Query builder generates standard AST +- Validation against standard schema +- Cross-driver query portability + +## Risk Assessment + +### High Risk +1. **Breaking Changes Impact** + - Mitigation: Comprehensive compatibility layer, clear migration guide + +2. **Performance Regression** + - Mitigation: Continuous benchmarking, optimization focus + +3. **Plugin System Complexity** + - Mitigation: Clear plugin API, extensive examples, helper utilities + +### Medium Risk +1. **Documentation Lag** + - Mitigation: Docs-first approach, community feedback + +2. **Third-party Integration Issues** + - Mitigation: Early beta release, partnership program + +### Low Risk +1. **Tool Compatibility** + - Mitigation: VSCode extension update included + +2. **Example Maintenance** + - Mitigation: Automated example testing + +## Success Metrics + +### Technical Metrics +- [ ] 100% driver compatibility with @objectstack/spec +- [ ] 90%+ test coverage maintained +- [ ] <10% performance overhead vs. v3.x +- [ ] Zero critical security vulnerabilities + +### User Metrics +- [ ] 80%+ user migration within 6 months +- [ ] <5% increase in issue reports post-release +- [ ] Positive community feedback on plugin system +- [ ] Active plugin ecosystem (5+ third-party plugins) + +### Code Quality Metrics +- [ ] All packages follow @objectstack conventions +- [ ] Reduced core package size by 60%+ +- [ ] Improved modularity score +- [ ] Decreased coupling between packages + +## Timeline Summary + +| Phase | Duration | Completion | +|-------|----------|------------| +| Analysis | 2 weeks | Week 2 | +| Runtime Integration | 2 weeks | Week 4 | +| Driver Migration | 2 weeks | Week 6 | +| Plugin Development | 4 weeks | Week 10 | +| Hooks/Actions | 2 weeks | Week 12 | +| Repository Update | 2 weeks | Week 14 | +| Tools Update | 2 weeks | Week 16 | +| Documentation | 2 weeks | Week 18 | +| Testing | 2 weeks | Week 20 | +| Release | 2 weeks | Week 22 | + +**Total Duration:** ~5-6 months + +## Next Steps + +### Immediate Actions (This Week) +1. ✅ Create migration strategy document (this document) +2. ⏭️ Set up @objectstack/runtime development environment +3. ⏭️ Create feature comparison matrix +4. ⏭️ Design plugin architecture diagram +5. ⏭️ Schedule team review meeting + +### Phase 1 Kickoff (Next Week) +1. Deep dive into @objectstack/runtime codebase +2. Map ObjectQL features to runtime equivalents +3. Prototype first plugin (query-validation) +4. Create migration decision tree +5. Draft backward compatibility design + +## Appendix + +### A. Package Dependency Changes + +**Before Migration:** +``` +@objectql/core +└── @objectql/types + +@objectql/driver-sql +├── @objectql/types +└── knex +``` + +**After Migration:** +``` +@objectql/query-validation (plugin) +├── @objectstack/runtime +└── @objectstack/spec + +@objectql/driver-sql (plugin) +├── @objectstack/runtime +├── @objectstack/spec +└── knex +``` + +### B. Configuration Changes + +**Before (v3.x):** +```typescript +const app = new ObjectQL({ + datasources: { + default: new SQLDriver({ connection: '...' }) + }, + plugins: [validationPlugin] +}); +``` + +**After (v4.x):** +```typescript +import { createRuntime } from '@objectstack/runtime'; +import { sqlDriverPlugin } from '@objectql/driver-sql'; +import { queryValidationPlugin } from '@objectql/query-validation'; + +const runtime = createRuntime({ + plugins: [ + sqlDriverPlugin({ connection: '...' }), + queryValidationPlugin() + ] +}); +``` + +### C. Reference Links + +- [@objectstack/runtime Documentation](https://github.com/objectstack-ai/runtime) +- [@objectstack/spec Specification](https://github.com/objectstack-ai/spec) +- [ObjectQL v3.x Documentation](./README.md) +- [Plugin Development Guide](./docs/guide/plugin-development.md) (To be created) + +--- + +**Document Version:** 1.0.0 +**Last Updated:** 2026-01-21 +**Status:** Draft - Awaiting Review diff --git a/PLUGIN_ARCHITECTURE.md b/PLUGIN_ARCHITECTURE.md new file mode 100644 index 00000000..4f06d484 --- /dev/null +++ b/PLUGIN_ARCHITECTURE.md @@ -0,0 +1,692 @@ +# ObjectQL Plugin Architecture Design + +## Overview + +This document defines the plugin architecture for ObjectQL 4.0, where the repository transitions from a standalone framework to a **collection of specialized query plugins** built on @objectstack/runtime. + +## Design Principles + +### 1. Plugin-First Philosophy +**Every feature is a plugin.** Even core query functionality is modular and composable. + +### 2. Clear Separation of Concerns +- **@objectstack/runtime**: Application lifecycle, metadata, security, transactions +- **@objectql plugins**: Query-specific functionality, optimizations, extensions + +### 3. Minimal Core, Maximal Flexibility +Keep the base runtime lightweight. Add functionality through plugins. + +### 4. Type-Safe Plugin API +All plugin interfaces are strictly typed with TypeScript. + +## Plugin Types + +### 1. Driver Plugins +**Purpose:** Data source adapters (SQL, MongoDB, Redis, etc.) + +**Interface:** +```typescript +import { RuntimePlugin } from '@objectstack/runtime'; +import { DriverInterface } from '@objectstack/spec'; + +export interface DriverPlugin extends RuntimePlugin { + driver: DriverInterface; +} + +export function createDriverPlugin( + driver: DriverInterface, + options?: DriverPluginOptions +): DriverPlugin { + return { + name: `@objectql/driver-${driver.name}`, + version: driver.version, + type: 'driver', + + driver, + + async setup(runtime: Runtime) { + await driver.connect(); + runtime.registerDriver(driver); + }, + + async teardown(runtime: Runtime) { + await driver.disconnect(); + runtime.unregisterDriver(driver.name); + } + }; +} +``` + +**Example Usage:** +```typescript +import { createRuntime } from '@objectstack/runtime'; +import { SQLDriver } from '@objectql/driver-sql'; +import { createDriverPlugin } from '@objectql/core'; + +const runtime = createRuntime({ + plugins: [ + createDriverPlugin(new SQLDriver({ + client: 'postgresql', + connection: 'postgresql://localhost/db' + })) + ] +}); +``` + +### 2. Query Processing Plugins +**Purpose:** Enhance query execution pipeline + +**Interface:** +```typescript +import { QueryAST } from '@objectstack/spec'; + +export interface QueryProcessorPlugin extends RuntimePlugin { + // Pre-processing hook + beforeQuery?( + ast: QueryAST, + context: RuntimeContext + ): Promise; + + // Post-processing hook + afterQuery?( + result: any, + context: RuntimeContext + ): Promise; + + // Validation hook + validateQuery?( + ast: QueryAST, + context: RuntimeContext + ): Promise; + + // Optimization hook + optimizeQuery?( + ast: QueryAST, + context: RuntimeContext + ): Promise; +} +``` + +**Example: Query Validation Plugin** +```typescript +export function queryValidationPlugin( + options?: ValidationOptions +): QueryProcessorPlugin { + return { + name: '@objectql/query-validation', + version: '4.0.0', + type: 'query-processor', + + async validateQuery(ast: QueryAST, context: RuntimeContext) { + const errors: ValidationError[] = []; + + // Validate object exists + if (!context.runtime.hasObject(ast.object)) { + errors.push({ + field: 'object', + message: `Object '${ast.object}' not found` + }); + } + + // Validate fields exist + const objectMeta = context.runtime.getObject(ast.object); + for (const field of ast.fields || []) { + if (!objectMeta.fields[field]) { + errors.push({ + field, + message: `Field '${field}' not found on object '${ast.object}'` + }); + } + } + + // Validate filter expressions + if (ast.filters) { + errors.push(...validateFilters(ast.filters, objectMeta)); + } + + return { + valid: errors.length === 0, + errors + }; + }, + + async beforeQuery(ast: QueryAST, context: RuntimeContext) { + const validation = await this.validateQuery(ast, context); + if (!validation.valid) { + throw new QueryValidationError(validation.errors); + } + return ast; + } + }; +} +``` + +### 3. Repository Extension Plugins +**Purpose:** Extend repository capabilities with advanced features + +**Interface:** +```typescript +export interface RepositoryPlugin extends RuntimePlugin { + // Extend repository with custom methods + extendRepository?( + repository: BaseRepository, + context: RuntimeContext + ): void; + + // CRUD lifecycle hooks + beforeCreate?(data: any, context: RuntimeContext): Promise; + afterCreate?(result: any, context: RuntimeContext): Promise; + beforeUpdate?(id: string, data: any, context: RuntimeContext): Promise; + afterUpdate?(result: any, context: RuntimeContext): Promise; + beforeDelete?(id: string, context: RuntimeContext): Promise; + afterDelete?(id: string, context: RuntimeContext): Promise; + beforeFind?(query: QueryAST, context: RuntimeContext): Promise; + afterFind?(results: any[], context: RuntimeContext): Promise; +} +``` + +**Example: Advanced Repository Plugin** +```typescript +export function advancedRepositoryPlugin(): RepositoryPlugin { + return { + name: '@objectql/advanced-repository', + version: '4.0.0', + type: 'repository', + + extendRepository(repo: BaseRepository, context: RuntimeContext) { + // Add batch operations + repo.createMany = async (records: any[]) => { + const results = []; + for (const record of records) { + results.push(await repo.create(record)); + } + return results; + }; + + // Add upsert operation + repo.upsert = async (id: string, data: any) => { + const exists = await repo.findOne({ filters: { _id: id } }); + return exists + ? repo.update(id, data) + : repo.create({ ...data, _id: id }); + }; + + // Add soft delete + repo.softDelete = async (id: string) => { + return repo.update(id, { + _deleted: true, + _deletedAt: new Date() + }); + }; + }, + + async beforeCreate(data: any, context: RuntimeContext) { + // Add audit fields + return { + ...data, + _createdBy: context.userId, + _createdAt: new Date() + }; + }, + + async beforeUpdate(id: string, data: any, context: RuntimeContext) { + // Add audit fields + return { + ...data, + _updatedBy: context.userId, + _updatedAt: new Date() + }; + } + }; +} +``` + +### 4. Feature Plugins +**Purpose:** Add specialized capabilities (formulas, AI, caching, etc.) + +**Interface:** +```typescript +export interface FeaturePlugin extends RuntimePlugin { + // Plugin-specific features + features: { + [key: string]: any; + }; + + // Register feature with runtime + registerFeature?( + runtime: Runtime, + context: RuntimeContext + ): Promise; +} +``` + +**Example: Formula Engine Plugin** +```typescript +export function formulaEnginePlugin( + options?: FormulaOptions +): FeaturePlugin { + const engine = new FormulaEngine(options); + + return { + name: '@objectql/formula-engine', + version: '4.0.0', + type: 'feature', + + features: { + evaluate: (expression: string, context: any) => { + return engine.evaluate(expression, context); + }, + + registerFunction: (name: string, fn: Function) => { + engine.registerFunction(name, fn); + } + }, + + async setup(runtime: Runtime) { + // Register built-in functions + engine.registerFunction('NOW', () => new Date()); + engine.registerFunction('SUM', (...args) => + args.reduce((a, b) => a + b, 0) + ); + + // Make engine available on runtime + runtime.registerFeature('formula', engine); + } + }; +} +``` + +**Example: AI Query Generator Plugin** +```typescript +export function aiQueryGeneratorPlugin( + options: AIOptions +): FeaturePlugin { + return { + name: '@objectql/ai-query-generator', + version: '4.0.0', + type: 'feature', + + features: { + generateQuery: async ( + naturalLanguage: string, + context: RuntimeContext + ): Promise => { + const schema = context.runtime.getSchema(); + const prompt = buildPrompt(naturalLanguage, schema); + + const response = await callLLM(prompt, options); + return parseQueryAST(response); + }, + + suggestQueries: async ( + partial: string, + context: RuntimeContext + ): Promise => { + // AI-powered query suggestions + return suggestionsFromContext(partial, context); + } + }, + + async setup(runtime: Runtime) { + runtime.registerFeature('ai', this.features); + } + }; +} +``` + +### 5. Query Optimization Plugins +**Purpose:** Improve query performance + +**Example: Query Cache Plugin** +```typescript +export function queryCachePlugin( + options?: CacheOptions +): QueryProcessorPlugin { + const cache = new QueryCache(options); + + return { + name: '@objectql/query-cache', + version: '4.0.0', + type: 'query-processor', + + async beforeQuery(ast: QueryAST, context: RuntimeContext) { + const cacheKey = generateCacheKey(ast, context); + const cached = await cache.get(cacheKey); + + if (cached) { + // Return cached result (skip query execution) + context.skipExecution = true; + context.cachedResult = cached; + } + + return ast; + }, + + async afterQuery(result: any, context: RuntimeContext) { + if (!context.skipExecution) { + const cacheKey = generateCacheKey(context.query, context); + await cache.set(cacheKey, result, options?.ttl); + } + return result; + } + }; +} +``` + +**Example: Query Optimizer Plugin** +```typescript +export function queryOptimizerPlugin(): QueryProcessorPlugin { + return { + name: '@objectql/query-optimizer', + version: '4.0.0', + type: 'query-processor', + + async optimizeQuery(ast: QueryAST, context: RuntimeContext) { + let optimized = { ...ast }; + + // Remove redundant fields + optimized = pruneFields(optimized); + + // Optimize filters + optimized = optimizeFilters(optimized); + + // Add index hints + optimized = addIndexHints(optimized, context); + + // Optimize joins + optimized = optimizeJoins(optimized); + + return optimized; + }, + + async beforeQuery(ast: QueryAST, context: RuntimeContext) { + return await this.optimizeQuery(ast, context); + } + }; +} +``` + +## Plugin Composition + +### Plugin Loading Order + +1. **Driver Plugins** - Initialize data sources +2. **Query Processor Plugins** - Set up query pipeline +3. **Repository Plugins** - Extend repository capabilities +4. **Feature Plugins** - Add specialized features + +### Plugin Dependencies + +```typescript +export interface PluginMetadata { + name: string; + version: string; + type: PluginType; + + // Plugin dependencies + dependencies?: string[]; + + // Optional dependencies + optionalDependencies?: string[]; + + // Conflicts with other plugins + conflicts?: string[]; +} +``` + +**Example:** +```typescript +export function advancedRepositoryPlugin(): RepositoryPlugin { + return { + name: '@objectql/advanced-repository', + version: '4.0.0', + type: 'repository', + + // Requires at least one driver + dependencies: ['@objectql/driver-*'], + + // Works better with validation + optionalDependencies: ['@objectql/query-validation'], + + // Conflicts with legacy repository + conflicts: ['@objectql/legacy-repository'] + }; +} +``` + +### Plugin Configuration + +```typescript +import { createRuntime } from '@objectstack/runtime'; +import { sqlDriverPlugin } from '@objectql/driver-sql'; +import { queryValidationPlugin } from '@objectql/query-validation'; +import { queryOptimizerPlugin } from '@objectql/query-optimizer'; +import { queryCachePlugin } from '@objectql/query-cache'; +import { advancedRepositoryPlugin } from '@objectql/advanced-repository'; +import { formulaEnginePlugin } from '@objectql/formula-engine'; + +const runtime = createRuntime({ + plugins: [ + // 1. Drivers (required) + sqlDriverPlugin({ + client: 'postgresql', + connection: process.env.DATABASE_URL + }), + + // 2. Query processors (optional but recommended) + queryValidationPlugin({ + strict: true, + allowUnknownFields: false + }), + queryOptimizerPlugin(), + queryCachePlugin({ + ttl: 60000, // 1 minute + maxSize: 1000 + }), + + // 3. Repository extensions + advancedRepositoryPlugin(), + + // 4. Features + formulaEnginePlugin({ + customFunctions: { + CUSTOM_FN: (arg) => arg.toUpperCase() + } + }) + ] +}); + +await runtime.initialize(); +``` + +## Plugin Development Guide + +### Creating a New Plugin + +#### Step 1: Define Plugin Interface +```typescript +// src/types.ts +import { RuntimePlugin } from '@objectstack/runtime'; + +export interface MyPluginOptions { + enabled?: boolean; + config?: any; +} + +export interface MyPlugin extends RuntimePlugin { + // Plugin-specific methods + myMethod(arg: any): Promise; +} +``` + +#### Step 2: Implement Plugin +```typescript +// src/plugin.ts +import { Runtime, RuntimeContext } from '@objectstack/runtime'; +import { MyPlugin, MyPluginOptions } from './types'; + +export function myPlugin(options?: MyPluginOptions): MyPlugin { + return { + name: '@objectql/my-plugin', + version: '1.0.0', + type: 'custom', + + async setup(runtime: Runtime) { + // Initialize plugin + console.log('MyPlugin initialized'); + }, + + async teardown(runtime: Runtime) { + // Cleanup + console.log('MyPlugin teardown'); + }, + + async myMethod(arg: any) { + // Plugin logic + return processArg(arg); + } + }; +} +``` + +#### Step 3: Add Tests +```typescript +// src/plugin.test.ts +import { createRuntime } from '@objectstack/runtime'; +import { myPlugin } from './plugin'; + +describe('MyPlugin', () => { + it('should initialize correctly', async () => { + const runtime = createRuntime({ + plugins: [myPlugin({ enabled: true })] + }); + + await runtime.initialize(); + expect(runtime.hasPlugin('@objectql/my-plugin')).toBe(true); + }); + + it('should execute myMethod', async () => { + const plugin = myPlugin(); + const result = await plugin.myMethod('test'); + expect(result).toBeDefined(); + }); +}); +``` + +#### Step 4: Document Plugin +```markdown +# @objectql/my-plugin + +My awesome plugin for ObjectQL. + +## Installation + +\`\`\`bash +npm install @objectql/my-plugin +\`\`\` + +## Usage + +\`\`\`typescript +import { createRuntime } from '@objectstack/runtime'; +import { myPlugin } from '@objectql/my-plugin'; + +const runtime = createRuntime({ + plugins: [ + myPlugin({ + enabled: true, + config: {} + }) + ] +}); +\`\`\` + +## API + +### Options + +- `enabled` - Enable/disable plugin +- `config` - Plugin configuration + +### Methods + +- `myMethod(arg)` - Does something cool +``` + +## Plugin Registry + +### Official Plugins + +| Plugin | Type | Status | Description | +|--------|------|--------|-------------| +| @objectql/driver-sql | Driver | ✅ Stable | SQL databases (PostgreSQL, MySQL, SQLite) | +| @objectql/driver-mongo | Driver | ✅ Stable | MongoDB database | +| @objectql/driver-memory | Driver | ✅ Stable | In-memory storage | +| @objectql/driver-sdk | Driver | ✅ Stable | Remote HTTP API | +| @objectql/query-validation | Processor | 🚧 Beta | Query validation | +| @objectql/query-optimizer | Processor | 🚧 Beta | Query optimization | +| @objectql/query-cache | Processor | 🚧 Beta | Query caching | +| @objectql/advanced-repository | Repository | 🚧 Beta | Enhanced repository | +| @objectql/formula-engine | Feature | 🚧 Beta | Formula evaluation | +| @objectql/ai-query-generator | Feature | 🔄 Alpha | AI query generation | + +### Community Plugins + +*(To be populated as community develops plugins)* + +## Best Practices + +### 1. Single Responsibility +Each plugin should do one thing well. + +### 2. Minimal Dependencies +Avoid heavy dependencies. Keep plugins lightweight. + +### 3. Type Safety +Always use TypeScript with strict mode. + +### 4. Error Handling +Handle errors gracefully and provide meaningful messages. + +```typescript +async beforeQuery(ast: QueryAST, context: RuntimeContext) { + try { + return await this.validateQuery(ast, context); + } catch (error) { + throw new PluginError( + `${this.name}: Validation failed`, + { cause: error } + ); + } +} +``` + +### 5. Performance +Minimize overhead in hot paths (query execution). + +### 6. Testing +Maintain >90% test coverage. + +### 7. Documentation +Provide clear examples and API documentation. + +## Plugin Versioning + +Follow Semantic Versioning (semver): +- **Major (4.0.0)**: Breaking changes +- **Minor (4.1.0)**: New features, backward compatible +- **Patch (4.0.1)**: Bug fixes + +## Next Steps + +1. Review plugin interfaces with team +2. Prototype first plugin (query-validation) +3. Establish plugin testing standards +4. Create plugin development template +5. Build plugin discovery mechanism + +--- + +**Document Version:** 1.0.0 +**Last Updated:** 2026-01-21 +**Status:** Draft - Pending Review From a5a21486647e96b1ae1e66899fa1b9e9e5ab5d8a Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Wed, 21 Jan 2026 11:34:23 +0000 Subject: [PATCH 3/9] Add package restructuring and implementation roadmap Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com> --- IMPLEMENTATION_ROADMAP.md | 668 ++++++++++++++++++++++++++++++++++++++ PACKAGE_RESTRUCTURING.md | 621 +++++++++++++++++++++++++++++++++++ 2 files changed, 1289 insertions(+) create mode 100644 IMPLEMENTATION_ROADMAP.md create mode 100644 PACKAGE_RESTRUCTURING.md diff --git a/IMPLEMENTATION_ROADMAP.md b/IMPLEMENTATION_ROADMAP.md new file mode 100644 index 00000000..5271e6a7 --- /dev/null +++ b/IMPLEMENTATION_ROADMAP.md @@ -0,0 +1,668 @@ +# Implementation Roadmap: ObjectQL v4.0 Migration + +## Executive Summary + +**Objective:** Transform ObjectQL from a standalone ORM framework into a specialized **query plugin ecosystem** built on @objectstack/runtime. + +**Timeline:** 22 weeks (5-6 months) +**Effort:** ~1800 engineering hours +**Team Size:** 2-3 engineers (full-time) + +**Key Results:** +- ✅ 60+ features migrated or refactored +- ✅ 8+ drivers converted to plugins +- ✅ 6+ new query plugins created +- ✅ 100% backward compatibility (v4.0.x) +- ✅ <10% performance overhead + +## Week-by-Week Breakdown + +### Week 1-2: Foundation & Planning ⚡ + +**Goals:** +- Deep dive into @objectstack/runtime +- Set up development environment +- Create migration automation tools + +**Deliverables:** +- [ ] Development environment configured +- [ ] Migration automation scripts +- [ ] Team training on @objectstack/runtime +- [ ] Detailed sprint plans for next 20 weeks + +**Tasks:** +1. Study @objectstack/runtime codebase (8 hours) +2. Set up monorepo for new package structure (4 hours) +3. Create package scaffolding tool (8 hours) +4. Create automated migration scripts (12 hours) +5. Document plugin development workflow (4 hours) +6. Team knowledge sharing session (4 hours) + +**Success Criteria:** +- ✅ Can create new plugin package in <5 minutes +- ✅ Automated type migration scripts working +- ✅ All team members trained on runtime + +--- + +### Week 3-4: Core Types Migration 📦 + +**Goals:** +- Migrate @objectql/types to v4.0 +- Remove duplicated types +- Establish type-safe plugin interfaces + +**Deliverables:** +- [ ] @objectql/types@4.0.0-alpha.1 +- [ ] Plugin type definitions +- [ ] Type migration guide + +**Tasks:** +1. Audit all types in @objectql/types (6 hours) +2. Remove types now in @objectstack/* (8 hours) +3. Create plugin interface types (10 hours) +4. Update type exports and re-exports (4 hours) +5. Write type migration guide (4 hours) +6. Update dependent packages (8 hours) + +**Code Changes:** +```typescript +// Before +import { Driver, ObjectConfig } from '@objectql/types'; + +// After +import { DriverInterface } from '@objectstack/spec'; +import { QueryPluginOptions } from '@objectql/types'; +``` + +**Success Criteria:** +- ✅ Zero duplicate types between @objectql and @objectstack +- ✅ All plugin interfaces defined +- ✅ Type tests passing + +--- + +### Week 5-6: Driver Plugin Migration 🔌 + +**Goals:** +- Convert all drivers to plugin architecture +- Implement driver plugin wrapper +- Update driver tests + +**Priority Drivers:** +1. **@objectql/driver-memory** (P0 - for testing) +2. **@objectql/driver-sql** (P0 - most popular) +3. **@objectql/driver-mongo** (P0 - popular) +4. **@objectql/driver-sdk** (P0 - remote API) + +**Deliverables:** +- [ ] Driver plugin template +- [ ] 4 critical drivers migrated +- [ ] Driver tests updated + +**Tasks per Driver (×4):** +1. Implement DriverInterface (4 hours) +2. Create plugin wrapper (2 hours) +3. Add setup/teardown hooks (2 hours) +4. Update tests (4 hours) +5. Update documentation (2 hours) + +**Total:** 56 hours for 4 drivers + +**Example Migration:** +```typescript +// packages/drivers/sql/src/index.ts + +// Export driver class +export class SQLDriver implements DriverInterface { + name = 'sql'; + version = '4.0.0'; + // ... implementation +} + +// Export plugin factory +export function sqlDriverPlugin(config: SQLConfig): RuntimePlugin { + return { + name: '@objectql/driver-sql', + version: '4.0.0', + type: 'driver', + + async setup(runtime: Runtime) { + const driver = new SQLDriver(config); + await driver.connect(); + runtime.registerDriver(driver); + } + }; +} +``` + +**Success Criteria:** +- ✅ All drivers implement DriverInterface +- ✅ All drivers exportPlugin factory +- ✅ All driver tests passing +- ✅ Can use drivers with runtime + +--- + +### Week 7-8: Query Validation Plugin 🛡️ + +**Goals:** +- Extract validation logic from core +- Create @objectql/query-validation plugin +- Implement comprehensive validation + +**Deliverables:** +- [ ] @objectql/query-validation@4.0.0-alpha.1 +- [ ] 30+ validation rules +- [ ] Custom validator API + +**Tasks:** +1. Design plugin architecture (6 hours) +2. Extract validation code from core (10 hours) +3. Implement QueryProcessorPlugin interface (8 hours) +4. Add field validation (6 hours) +5. Add cross-field validation (6 hours) +6. Add custom validators API (4 hours) +7. Write tests (12 hours) +8. Write documentation (4 hours) + +**Key Features:** +- Field existence validation +- Type validation +- Required field validation +- Cross-field validation +- Custom validator registration +- Detailed error messages + +**Usage Example:** +```typescript +import { queryValidationPlugin } from '@objectql/query-validation'; + +const runtime = createRuntime({ + plugins: [ + queryValidationPlugin({ + strict: true, + customValidators: { + email: (value) => /^[^\s@]+@[^\s@]+\.[^\s@]+$/.test(value) + } + }) + ] +}); +``` + +**Success Criteria:** +- ✅ All core validation rules implemented +- ✅ Custom validators working +- ✅ 90%+ test coverage +- ✅ Performance: <1ms validation overhead + +--- + +### Week 9-10: Advanced Repository Plugin 📚 + +**Goals:** +- Extract repository logic from core +- Create @objectql/advanced-repository plugin +- Add advanced CRUD features + +**Deliverables:** +- [ ] @objectql/advanced-repository@4.0.0-alpha.1 +- [ ] Batch operations +- [ ] Audit tracking +- [ ] Soft delete + +**Tasks:** +1. Design repository plugin architecture (6 hours) +2. Extract repository code from core (12 hours) +3. Implement RepositoryPlugin interface (8 hours) +4. Add batch operations (6 hours) +5. Add upsert functionality (4 hours) +6. Add soft delete (4 hours) +7. Add audit tracking (6 hours) +8. Write tests (12 hours) +9. Write documentation (4 hours) + +**Key Features:** +- Enhanced CRUD operations +- Batch create/update/delete +- Upsert (create or update) +- Soft delete with restore +- Automatic audit fields (_createdBy, _updatedBy, etc.) +- Transaction support +- Event hooks + +**Usage Example:** +```typescript +import { advancedRepositoryPlugin } from '@objectql/advanced-repository'; + +const runtime = createRuntime({ + plugins: [ + advancedRepositoryPlugin({ + auditFields: true, + softDelete: true + }) + ] +}); + +// Use enhanced repository +const repo = runtime.object('users'); +await repo.createMany([user1, user2, user3]); // Batch +await repo.upsert(id, data); // Create or update +await repo.softDelete(id); // Soft delete +``` + +**Success Criteria:** +- ✅ All repository features working +- ✅ Batch operations efficient +- ✅ 90%+ test coverage +- ✅ Backward compatible with v3.x API + +--- + +### Week 11-12: Formula Engine Plugin 🧮 + +**Goals:** +- Extract formula engine from core +- Create @objectql/formula-engine plugin +- Enhance with new functions + +**Deliverables:** +- [ ] @objectql/formula-engine@4.0.0-alpha.1 +- [ ] 50+ built-in functions +- [ ] Custom function API + +**Tasks:** +1. Extract formula engine code (8 hours) +2. Implement FeaturePlugin interface (6 hours) +3. Add built-in functions (12 hours) +4. Add custom function registration (4 hours) +5. Add expression parser (8 hours) +6. Write tests (12 hours) +7. Write documentation (6 hours) + +**Built-in Functions:** +- Math: SUM, AVG, MIN, MAX, ROUND, ABS, etc. +- String: CONCAT, UPPER, LOWER, TRIM, etc. +- Date: NOW, DATE, DATEADD, DATEDIFF, etc. +- Logical: IF, AND, OR, NOT, etc. +- Lookup: VLOOKUP, RELATED, etc. + +**Usage Example:** +```typescript +import { formulaEnginePlugin } from '@objectql/formula-engine'; + +const runtime = createRuntime({ + plugins: [ + formulaEnginePlugin({ + customFunctions: { + TAX: (amount) => amount * 0.2 + } + }) + ] +}); + +// Use in object definition +runtime.registerObject({ + name: 'orders', + fields: { + subtotal: { type: 'number' }, + tax: { + type: 'formula', + formula: 'TAX(subtotal)' + }, + total: { + type: 'formula', + formula: 'subtotal + tax' + } + } +}); +``` + +**Success Criteria:** +- ✅ 50+ functions implemented +- ✅ Custom functions working +- ✅ Expression parsing robust +- ✅ Performance: <5ms per evaluation + +--- + +### Week 13-14: Query Optimizer Plugin ⚡ + +**Goals:** +- Create query optimization plugin +- Implement optimization strategies +- Performance benchmarking + +**Deliverables:** +- [ ] @objectql/query-optimizer@4.0.0-alpha.1 +- [ ] 10+ optimization rules +- [ ] Performance improvements + +**Tasks:** +1. Design optimizer architecture (8 hours) +2. Implement QueryProcessorPlugin (6 hours) +3. Add projection pruning (6 hours) +4. Add filter optimization (8 hours) +5. Add join optimization (8 hours) +6. Add index hints (6 hours) +7. Write tests (10 hours) +8. Performance benchmarking (6 hours) +9. Write documentation (4 hours) + +**Optimization Rules:** +1. Remove unnecessary fields +2. Push filters down +3. Eliminate redundant conditions +4. Optimize boolean expressions +5. Index hint generation +6. Join order optimization +7. Subquery optimization +8. Limit/offset optimization + +**Success Criteria:** +- ✅ 20%+ query performance improvement +- ✅ Smart index usage +- ✅ Works with all drivers + +--- + +### Week 15-16: Query Cache Plugin 🗄️ + +**Goals:** +- Create query caching plugin +- Implement cache strategies +- Cache invalidation + +**Deliverables:** +- [ ] @objectql/query-cache@4.0.0-alpha.1 +- [ ] Multiple cache backends +- [ ] Smart invalidation + +**Tasks:** +1. Design cache architecture (6 hours) +2. Implement QueryProcessorPlugin (4 hours) +3. Add memory cache backend (6 hours) +4. Add Redis cache backend (8 hours) +5. Implement cache key generation (6 hours) +6. Implement TTL management (4 hours) +7. Implement invalidation strategies (8 hours) +8. Write tests (10 hours) +9. Write documentation (4 hours) + +**Cache Backends:** +- In-memory (LRU) +- Redis +- Custom (extensible) + +**Invalidation Strategies:** +- Time-based (TTL) +- Event-based (on write) +- Manual +- Tag-based + +**Success Criteria:** +- ✅ 50%+ cache hit rate +- ✅ Smart invalidation working +- ✅ Multiple backends supported + +--- + +### Week 17-18: AI Query Generator Plugin 🤖 + +**Goals:** +- Create AI query generation plugin +- Natural language to QueryAST +- Query suggestions + +**Deliverables:** +- [ ] @objectql/ai-query-generator@4.0.0-alpha.1 +- [ ] LLM integration +- [ ] Query suggestions + +**Tasks:** +1. Design AI plugin architecture (8 hours) +2. Implement FeaturePlugin interface (4 hours) +3. Create prompt engineering (12 hours) +4. Add LLM integration (OpenAI) (8 hours) +5. Add query suggestion (8 hours) +6. Add context-aware generation (10 hours) +7. Write tests (10 hours) +8. Write documentation (6 hours) + +**Features:** +- Natural language to QueryAST +- Query suggestions based on context +- Semantic understanding +- Schema-aware generation + +**Usage Example:** +```typescript +import { aiQueryGeneratorPlugin } from '@objectql/ai-query-generator'; + +const runtime = createRuntime({ + plugins: [ + aiQueryGeneratorPlugin({ + apiKey: process.env.OPENAI_API_KEY, + model: 'gpt-4' + }) + ] +}); + +// Generate query from natural language +const query = await runtime.generateQuery( + "Find all active users who signed up last month" +); +// Returns: QueryAST +``` + +**Success Criteria:** +- ✅ 80%+ query generation accuracy +- ✅ Context-aware suggestions +- ✅ Fast response (<2s) + +--- + +### Week 19-20: Tools & Integration Update 🛠️ + +**Goals:** +- Update CLI for v4.0 +- Update create templates +- Update VSCode extension +- Update server integration + +**Deliverables:** +- [ ] @objectql/cli@4.0.0-alpha.1 +- [ ] @objectql/create@4.0.0-alpha.1 +- [ ] @objectql/vscode-objectql@4.0.0 +- [ ] @objectql/server@4.0.0-alpha.1 + +**Tasks:** + +**CLI (20 hours):** +1. Update project templates (6 hours) +2. Add plugin management commands (6 hours) +3. Add migration command (4 hours) +4. Update documentation (4 hours) + +**Create (12 hours):** +1. Create runtime-based templates (6 hours) +2. Add plugin composition examples (4 hours) +3. Update documentation (2 hours) + +**VSCode (16 hours):** +1. Update schema validation (6 hours) +2. Add plugin IntelliSense (6 hours) +3. Update snippets (2 hours) +4. Update documentation (2 hours) + +**Server (12 hours):** +1. Integrate with runtime (6 hours) +2. Update HTTP API (4 hours) +3. Update documentation (2 hours) + +**Success Criteria:** +- ✅ Can scaffold v4.0 project in <2 minutes +- ✅ Plugin IntelliSense working +- ✅ Server works with runtime + +--- + +### Week 21-22: Testing, Documentation & Release 📋 + +**Goals:** +- Comprehensive testing +- Complete documentation +- Beta release + +**Deliverables:** +- [ ] 90%+ test coverage +- [ ] Complete documentation +- [ ] v4.0.0-beta.1 release +- [ ] Migration guide + +**Tasks:** + +**Testing (30 hours):** +1. Integration tests (12 hours) +2. Performance benchmarks (8 hours) +3. Cross-environment testing (6 hours) +4. Security audit (4 hours) + +**Documentation (30 hours):** +1. Update README.md (4 hours) +2. Update API docs (8 hours) +3. Create migration guide (8 hours) +4. Update all examples (6 hours) +5. Create video tutorials (4 hours) + +**Release Prep (12 hours):** +1. Version bumps (2 hours) +2. Changelog generation (3 hours) +3. Release notes (3 hours) +4. Community announcement (2 hours) +5. Blog post (2 hours) + +**Success Criteria:** +- ✅ All quality gates passed +- ✅ Documentation complete +- ✅ Beta release published +- ✅ Migration path clear + +--- + +## Resource Allocation + +### Engineering Hours by Phase + +| Phase | Hours | Weeks | +|-------|-------|-------| +| Foundation & Planning | 40 | 1-2 | +| Types Migration | 40 | 3-4 | +| Driver Migration | 56 | 5-6 | +| Query Validation | 56 | 7-8 | +| Advanced Repository | 62 | 9-10 | +| Formula Engine | 56 | 11-12 | +| Query Optimizer | 62 | 13-14 | +| Query Cache | 56 | 15-16 | +| AI Generator | 66 | 17-18 | +| Tools Update | 60 | 19-20 | +| Testing & Release | 72 | 21-22 | +| **Total** | **626** | **22** | + +### Team Structure + +**Recommended Team:** +- 1× Tech Lead (full-time) +- 2× Senior Engineers (full-time) + +**Alternative (smaller team):** +- 2× Senior Engineers (full-time, 22 weeks) + +**Alternative (faster):** +- 3× Senior Engineers (full-time, 14-16 weeks) + +## Risk Management + +### High-Risk Items + +1. **Runtime Integration Complexity** + - Mitigation: Prototype in weeks 1-2 + - Fallback: Extend timeline by 2 weeks + +2. **Performance Regression** + - Mitigation: Continuous benchmarking + - Fallback: Optimization sprint (week 23) + +3. **Breaking Changes** + - Mitigation: Comprehensive compatibility layer + - Fallback: Extend v3.x support to 12 months + +### Contingency Buffer + +- **Built-in buffer:** 10% (2.2 weeks) +- **Recommended total:** 24 weeks (includes buffer) + +## Quality Gates + +### Weekly Quality Checks + +Every week: +- ✅ All tests passing +- ✅ No TypeScript errors +- ✅ No ESLint errors +- ✅ Code reviewed +- ✅ Documentation updated + +### Phase-End Quality Gates + +After each phase: +- ✅ Integration tests passing +- ✅ Performance benchmarks met +- ✅ Security scan clean +- ✅ Documentation complete +- ✅ Changelog updated + +## Success Metrics (v4.0 GA) + +### Technical Metrics +- ✅ 100% drivers migrated to plugins +- ✅ 90%+ test coverage maintained +- ✅ <10% performance overhead +- ✅ Zero critical vulnerabilities +- ✅ All examples working + +### User Metrics +- ✅ Clear migration path documented +- ✅ 80%+ positive feedback from beta users +- ✅ <5% increase in GitHub issues +- ✅ Active community plugin development + +### Business Metrics +- ✅ On-time delivery (22 weeks) +- ✅ On-budget delivery +- ✅ Smooth v3.x→v4.x migration for existing users + +## Post-Release Plan + +### v4.0.x Maintenance (Months 6-12) +- Bug fixes +- Performance improvements +- Community support +- Plugin ecosystem growth + +### v4.1.0 (Month 9) +- Additional query plugins +- Enhanced AI capabilities +- Performance optimizations + +### v5.0.0 (Month 18) +- Remove deprecated APIs +- Additional runtime integrations +- Advanced features + +--- + +**Document Version:** 1.0.0 +**Last Updated:** 2026-01-21 +**Prepared By:** ObjectQL Migration Team +**Status:** Ready for Execution diff --git a/PACKAGE_RESTRUCTURING.md b/PACKAGE_RESTRUCTURING.md new file mode 100644 index 00000000..fb969465 --- /dev/null +++ b/PACKAGE_RESTRUCTURING.md @@ -0,0 +1,621 @@ +# Package Restructuring Plan for @objectstack/runtime Migration + +## Overview + +This document outlines the detailed package restructuring plan to transform ObjectQL from a standalone framework into a plugin-based architecture built on @objectstack/runtime. + +## Current Package Structure + +``` +packages/ +├── foundation/ +│ ├── types/ # TypeScript type definitions +│ ├── core/ # Core ObjectQL engine +│ └── platform-node/ # Node.js platform utilities +├── drivers/ +│ ├── sql/ # SQL driver (Knex) +│ ├── mongo/ # MongoDB driver +│ ├── memory/ # In-memory driver +│ ├── localstorage/ # Browser LocalStorage driver +│ ├── fs/ # File system driver +│ ├── excel/ # Excel file driver +│ ├── redis/ # Redis driver +│ └── sdk/ # HTTP Remote driver +├── runtime/ +│ └── server/ # HTTP server adapter +└── tools/ + ├── cli/ # Command-line interface + ├── create/ # Project scaffolding + └── vscode-objectql/ # VS Code extension +``` + +## Target Package Structure (v4.0.0) + +``` +packages/ +├── plugins/ +│ ├── query-validation/ # Query validation plugin +│ ├── query-optimizer/ # Query optimization plugin +│ ├── query-cache/ # Query caching plugin +│ ├── advanced-repository/ # Enhanced repository plugin +│ ├── formula-engine/ # Formula evaluation plugin +│ └── ai-query-generator/ # AI query generation plugin +├── drivers/ +│ ├── sql/ # SQL driver plugin +│ ├── mongo/ # MongoDB driver plugin +│ ├── memory/ # Memory driver plugin +│ ├── localstorage/ # LocalStorage driver plugin +│ ├── fs/ # File system driver plugin +│ ├── excel/ # Excel driver plugin +│ ├── redis/ # Redis driver plugin +│ └── sdk/ # SDK driver plugin +├── core/ +│ └── types/ # Query-specific types (minimal) +├── integrations/ +│ └── server/ # HTTP server plugin +└── tools/ + ├── cli/ # CLI (runtime-aware) + ├── create/ # Scaffolding (runtime-based) + └── vscode-objectql/ # VS Code extension +``` + +## Package-by-Package Migration Plan + +### 1. Foundation Layer → Core Types + +#### @objectql/types (v3.0.1 → v4.0.0) + +**Current Responsibilities:** +- All TypeScript interfaces +- Driver interface (being replaced by @objectstack/spec) +- Core types, query types, metadata types + +**Target Responsibilities:** +- Query-specific types only +- Plugin interfaces for ObjectQL plugins +- Thin type re-exports from @objectstack/spec + +**Changes:** +```diff +- Remove: Driver interface (use @objectstack/spec) +- Remove: MetadataRegistry types (use @objectstack/runtime) +- Remove: Context types (use @objectstack/runtime) +- Remove: Hook types (use @objectstack/runtime) +- Remove: Action types (use @objectstack/runtime) ++ Add: QueryPlugin interface ++ Add: RepositoryPlugin interface ++ Add: Plugin-specific types ++ Keep: Query-specific helper types +``` + +**Migration Steps:** +1. Create `/packages/core/types/` (new location) +2. Move query-specific types +3. Update imports to use @objectstack packages +4. Remove deprecated types +5. Update dependent packages + +**Package.json Changes:** +```json +{ + "name": "@objectql/types", + "version": "4.0.0", + "dependencies": { + "@objectstack/spec": "^0.2.0", + "@objectstack/runtime": "^0.2.0" + } +} +``` + +#### @objectql/core (v3.0.1 → DEPRECATED) + +**Current Responsibilities:** +- Full ObjectQL engine +- Metadata registry +- Driver management +- Hooks, actions, validation +- Repository pattern +- Formula engine +- AI agent integration + +**Target:** **DEPRECATED** - Split into multiple plugins + +**Migration Path:** +``` +@objectql/core (v3.x) +├── app.ts → @objectstack/runtime (lifecycle) +├── repository.ts → @objectql/advanced-repository (plugin) +├── validator.ts → @objectql/query-validation (plugin) +├── formula-engine.ts → @objectql/formula-engine (plugin) +├── ai-agent.ts → @objectql/ai-query-generator (plugin) +├── hook.ts → @objectstack/runtime (events) +├── action.ts → @objectstack/runtime (commands) +├── object.ts → @objectstack/runtime (metadata) +└── util.ts → @objectql/types (utilities) +``` + +**Package Status:** +- v4.0.0: Compatibility wrapper (re-exports from plugins) +- v4.1.0: Deprecation warnings +- v5.0.0: Removed (use plugins directly) + +#### @objectql/platform-node (v3.0.1 → MIGRATE) + +**Current Responsibilities:** +- Node.js file system utilities +- YAML loading +- Plugin loading +- Environment detection + +**Target:** Most functionality moves to @objectstack/platform-node + +**Remaining Responsibilities:** +- ObjectQL-specific Node.js utilities +- Query file loading helpers + +**Migration Steps:** +1. Identify ObjectQL-specific vs. general utilities +2. Contribute general utilities to @objectstack/platform-node +3. Keep only query-specific helpers +4. Update to use @objectstack/platform-node + +**Package.json Changes:** +```json +{ + "name": "@objectql/platform-node", + "version": "4.0.0", + "dependencies": { + "@objectstack/platform-node": "^0.2.0", + "@objectql/types": "workspace:*" + } +} +``` + +### 2. Drivers Layer → Driver Plugins + +**All drivers follow the same pattern:** + +#### Template: @objectql/driver-* (v3.0.1 → v4.0.0) + +**Changes:** +```diff +- Implement Driver interface from @objectql/types ++ Implement DriverInterface from @objectstack/spec ++ Export as RuntimePlugin ++ Add plugin setup/teardown +``` + +**Example: @objectql/driver-sql** + +**Before (v3.x):** +```typescript +import { Driver } from '@objectql/types'; + +export class SQLDriver implements Driver { + async connect() { } + async find(object, query) { } + // ... +} +``` + +**After (v4.x):** +```typescript +import { DriverInterface } from '@objectstack/spec'; +import { RuntimePlugin } from '@objectstack/runtime'; + +export class SQLDriver implements DriverInterface { + name = 'sql'; + version = '4.0.0'; + + async connect() { } + async disconnect() { } + async find(object, query, options) { } + async create(object, data, options) { } + async update(object, id, data, options) { } + async delete(object, id, options) { } +} + +export function sqlDriverPlugin(config): RuntimePlugin { + const driver = new SQLDriver(config); + return { + name: '@objectql/driver-sql', + version: '4.0.0', + type: 'driver', + async setup(runtime) { + await driver.connect(); + runtime.registerDriver(driver); + }, + async teardown(runtime) { + await driver.disconnect(); + } + }; +} +``` + +**Migration Checklist for Each Driver:** +- [ ] Update to DriverInterface from @objectstack/spec +- [ ] Add plugin wrapper function +- [ ] Add setup/teardown lifecycle +- [ ] Update tests to use Runtime +- [ ] Update documentation +- [ ] Update examples + +**Drivers to Migrate:** +1. @objectql/driver-sql (P0 - Critical) +2. @objectql/driver-memory (P0 - Critical for testing) +3. @objectql/driver-mongo (P0 - Popular) +4. @objectql/driver-sdk (P0 - Remote API) +5. @objectql/driver-localstorage (P1 - Browser) +6. @objectql/driver-fs (P1 - Development) +7. @objectql/driver-excel (P2 - Niche) +8. @objectql/driver-redis (P2 - Template/Example) + +### 3. Query Plugins (NEW) + +#### @objectql/query-validation (NEW) + +**Location:** `/packages/plugins/query-validation/` + +**Source:** +- Extracted from `@objectql/core/validator.ts` +- Extracted from `@objectql/core/object.ts` (field validation) + +**Responsibilities:** +- Query AST validation +- Field existence checking +- Type validation +- Cross-field validation +- Custom validators + +**Package.json:** +```json +{ + "name": "@objectql/query-validation", + "version": "4.0.0", + "main": "dist/index.js", + "types": "dist/index.d.ts", + "dependencies": { + "@objectstack/runtime": "^0.2.0", + "@objectstack/spec": "^0.2.0", + "@objectql/types": "workspace:*" + } +} +``` + +**Exports:** +```typescript +export { queryValidationPlugin } from './plugin'; +export type { ValidationOptions, ValidationError, ValidationResult } from './types'; +``` + +#### @objectql/advanced-repository (NEW) + +**Location:** `/packages/plugins/advanced-repository/` + +**Source:** +- Extracted from `@objectql/core/repository.ts` +- Enhanced with new capabilities + +**Responsibilities:** +- Enhanced CRUD operations +- Batch operations +- Upsert functionality +- Soft delete +- Audit tracking +- Event hooks + +**Package.json:** +```json +{ + "name": "@objectql/advanced-repository", + "version": "4.0.0", + "dependencies": { + "@objectstack/runtime": "^0.2.0", + "@objectstack/spec": "^0.2.0", + "@objectql/types": "workspace:*" + } +} +``` + +#### @objectql/query-optimizer (NEW) + +**Location:** `/packages/plugins/query-optimizer/` + +**Source:** New implementation + +**Responsibilities:** +- Query plan optimization +- Index hint generation +- Join optimization +- Projection pruning +- Filter optimization + +**Package.json:** +```json +{ + "name": "@objectql/query-optimizer", + "version": "4.0.0", + "dependencies": { + "@objectstack/runtime": "^0.2.0", + "@objectstack/spec": "^0.2.0", + "@objectql/types": "workspace:*" + } +} +``` + +#### @objectql/query-cache (NEW) + +**Location:** `/packages/plugins/query-cache/` + +**Source:** New implementation + +**Responsibilities:** +- Query result caching +- Cache key generation +- TTL management +- Cache invalidation +- Memory management + +**Package.json:** +```json +{ + "name": "@objectql/query-cache", + "version": "4.0.0", + "dependencies": { + "@objectstack/runtime": "^0.2.0", + "@objectstack/spec": "^0.2.0", + "@objectql/types": "workspace:*" + } +} +``` + +#### @objectql/formula-engine (NEW) + +**Location:** `/packages/plugins/formula-engine/` + +**Source:** +- Extracted from `@objectql/core/formula-engine.ts` + +**Responsibilities:** +- Formula parsing +- Expression evaluation +- Built-in functions +- Custom function registration + +**Package.json:** +```json +{ + "name": "@objectql/formula-engine", + "version": "4.0.0", + "dependencies": { + "@objectstack/runtime": "^0.2.0", + "@objectql/types": "workspace:*" + } +} +``` + +#### @objectql/ai-query-generator (NEW) + +**Location:** `/packages/plugins/ai-query-generator/` + +**Source:** +- Extracted from `@objectql/core/ai-agent.ts` +- Enhanced with new capabilities + +**Responsibilities:** +- Natural language to QueryAST +- Query suggestions +- Semantic understanding +- Context-aware generation + +**Package.json:** +```json +{ + "name": "@objectql/ai-query-generator", + "version": "4.0.0", + "dependencies": { + "@objectstack/runtime": "^0.2.0", + "@objectstack/spec": "^0.2.0", + "@objectql/types": "workspace:*", + "openai": "^4.28.0" + } +} +``` + +### 4. Server Integration + +#### @objectql/server (v3.0.1 → v4.0.0) + +**Location:** `/packages/integrations/server/` + +**Changes:** +```diff +- Own ObjectQL app initialization ++ Use @objectstack/runtime ++ Provide HTTP API plugin ++ Expose metadata endpoints via runtime +``` + +**Package.json:** +```json +{ + "name": "@objectql/server", + "version": "4.0.0", + "dependencies": { + "@objectstack/runtime": "^0.2.0", + "@objectstack/spec": "^0.2.0", + "@objectql/types": "workspace:*", + "express": "^4.18.0" + } +} +``` + +### 5. Tools + +#### @objectql/cli (v3.0.1 → v4.0.0) + +**Changes:** +```diff +- Generate v3.x ObjectQL projects ++ Generate v4.x runtime-based projects ++ New templates with plugin composition ++ Migration command for v3.x → v4.x +``` + +**New Commands:** +```bash +# Create new project with runtime +objectql create my-app --runtime + +# Migrate v3.x project to v4.x +objectql migrate --from 3.x + +# Plugin management +objectql plugin add @objectql/query-cache +objectql plugin list +``` + +**Package.json:** +```json +{ + "name": "@objectql/cli", + "version": "4.0.0", + "dependencies": { + "@objectstack/runtime": "^0.2.0", + "@objectql/types": "workspace:*" + } +} +``` + +#### @objectql/create (v3.0.1 → v4.0.0) + +**Changes:** +```diff +- v3.x templates ++ v4.x runtime-based templates ++ Plugin composition examples +``` + +**Package.json:** +```json +{ + "name": "@objectql/create", + "version": "4.0.0", + "dependencies": { + "@objectstack/runtime": "^0.2.0" + } +} +``` + +#### @objectql/vscode-objectql (v3.0.1 → v4.0.0) + +**Changes:** +```diff +- v3.x schema validation ++ v4.x runtime schema validation ++ Plugin configuration IntelliSense +``` + +## Migration Timeline + +### Week 1-2: Foundation +- [ ] Set up new package structure +- [ ] Migrate @objectql/types +- [ ] Create plugin base classes + +### Week 3-4: Core Plugins +- [ ] Create @objectql/query-validation +- [ ] Create @objectql/advanced-repository + +### Week 5-6: Drivers +- [ ] Migrate @objectql/driver-sql +- [ ] Migrate @objectql/driver-memory +- [ ] Migrate @objectql/driver-mongo +- [ ] Migrate @objectql/driver-sdk + +### Week 7-8: Feature Plugins +- [ ] Create @objectql/formula-engine +- [ ] Create @objectql/query-optimizer +- [ ] Create @objectql/query-cache + +### Week 9-10: Advanced Plugins +- [ ] Create @objectql/ai-query-generator +- [ ] Migrate remaining drivers + +### Week 11-12: Integration & Tools +- [ ] Migrate @objectql/server +- [ ] Update @objectql/cli +- [ ] Update @objectql/create +- [ ] Update @objectql/vscode-objectql + +### Week 13-14: Testing & Documentation +- [ ] Update all tests +- [ ] Update all documentation +- [ ] Create migration guides + +## Package Publishing Strategy + +### Phase 1: Alpha (Week 8) +Publish core plugins and drivers for internal testing: +- @objectql/types@4.0.0-alpha.1 +- @objectql/query-validation@4.0.0-alpha.1 +- @objectql/advanced-repository@4.0.0-alpha.1 +- @objectql/driver-sql@4.0.0-alpha.1 +- @objectql/driver-memory@4.0.0-alpha.1 + +### Phase 2: Beta (Week 12) +Publish all plugins for community testing: +- All packages@4.0.0-beta.1 + +### Phase 3: RC (Week 16) +Release candidate with docs and examples: +- All packages@4.0.0-rc.1 + +### Phase 4: GA (Week 20) +Production release: +- All packages@4.0.0 + +## Backward Compatibility + +### Compatibility Package: @objectql/legacy + +**Purpose:** Provide v3.x API compatibility + +```typescript +// @objectql/legacy +export class ObjectQL { + constructor(config: ObjectQLConfig) { + console.warn('Using legacy API. Please migrate to @objectstack/runtime'); + // Wrap runtime with v3.x API + } +} +``` + +**Usage:** +```typescript +// Old code continues to work +import { ObjectQL } from '@objectql/legacy'; + +const app = new ObjectQL({ + datasources: { default: sqlDriver } +}); +``` + +### Deprecation Timeline +- v4.0.0: @objectql/legacy available +- v4.5.0: @objectql/legacy deprecated +- v5.0.0: @objectql/legacy removed + +## Next Steps + +1. Review this restructuring plan +2. Create new package directories +3. Begin Phase 1 migration +4. Set up CI/CD for new packages +5. Create migration automation scripts + +--- + +**Document Version:** 1.0.0 +**Last Updated:** 2026-01-21 +**Status:** Draft - Pending Review From f004dd034ac31c7387195426ecb83771cc2a08df Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Wed, 21 Jan 2026 11:37:44 +0000 Subject: [PATCH 4/9] Add executive summary and migration index Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com> --- EXECUTIVE_SUMMARY.md | 440 +++++++++++++++++++++++++++++++++++++++++++ MIGRATION_INDEX.md | 239 +++++++++++++++++++++++ 2 files changed, 679 insertions(+) create mode 100644 EXECUTIVE_SUMMARY.md create mode 100644 MIGRATION_INDEX.md diff --git a/EXECUTIVE_SUMMARY.md b/EXECUTIVE_SUMMARY.md new file mode 100644 index 00000000..92e94bcb --- /dev/null +++ b/EXECUTIVE_SUMMARY.md @@ -0,0 +1,440 @@ +# ObjectQL v4.0 Migration to @objectstack/runtime - Executive Summary + +## Overview + +This document provides a comprehensive evaluation of the work required to migrate the ObjectQL repository from a standalone ORM framework to a **specialized query plugin ecosystem** built on top of @objectstack/runtime. + +## Problem Statement + +**Original Requirement (Chinese):** +> 评估现有代码全面迁移到@objectstack/runtime 新架构的工作清单，原则上这个仓库只是一个插件仓库，在objectstack的基础框架中开发插件扩展查询相关的功能 + +**Translation:** +> Evaluate the work checklist for fully migrating existing code to the @objectstack/runtime new architecture. In principle, this repository should be a plugin repository, developing plugin extensions for query-related functionality on the objectstack base framework. + +## Current State Assessment + +### Existing Integration (v3.0.1) + +ObjectQL already has partial integration with @objectstack packages: + +| Package | Version | Current Usage | +|---------|---------|---------------| +| `@objectstack/spec` | 0.1.2 | Type exports only (DriverInterface) | +| `@objectstack/runtime` | 0.1.1 | Type exports only (not actively used) | +| `@objectstack/objectql` | 0.1.1 | Type exports only (not actively used) | + +**Gap:** While ObjectQL imports types from @objectstack packages, it does NOT actually use the @objectstack/runtime kernel for application lifecycle, plugin management, or driver orchestration. + +### Current Architecture Issues + +1. **Monolithic Core**: @objectql/core contains everything (metadata, validation, hooks, actions, formulas, AI) +2. **Own Lifecycle**: ObjectQL manages its own application initialization instead of using runtime +3. **Duplicated Functionality**: Many features (metadata, hooks, actions) should be in @objectstack/runtime +4. **Not Plugin-Based**: Functionality is tightly coupled, not modular + +## Proposed Architecture + +### Vision: ObjectQL as Query Plugin Ecosystem + +``` +┌─────────────────────────────────────────────────────┐ +│ @objectstack/runtime (Foundation) │ +│ - Application Lifecycle │ +│ - Metadata Registry │ +│ - Plugin System │ +│ - Security & RBAC │ +│ - Transaction Management │ +└─────────────────────────────────────────────────────┘ + │ + ▼ +┌─────────────────────────────────────────────────────┐ +│ ObjectQL (Plugin Ecosystem) │ +│ │ +│ Query Plugins: │ +│ ├── @objectql/query-validation │ +│ ├── @objectql/query-optimizer │ +│ ├── @objectql/query-cache │ +│ ├── @objectql/advanced-repository │ +│ ├── @objectql/formula-engine │ +│ └── @objectql/ai-query-generator │ +│ │ +│ Driver Plugins: │ +│ ├── @objectql/driver-sql │ +│ ├── @objectql/driver-mongo │ +│ ├── @objectql/driver-memory │ +│ └── @objectql/driver-* (8+ total) │ +└─────────────────────────────────────────────────────┘ +``` + +## Comprehensive Documentation Created + +We have created 5 comprehensive planning documents totaling **75+ pages**: + +### 1. 📋 MIGRATION_STRATEGY.md (14.5 KB) +**Purpose:** High-level strategic plan + +**Contents:** +- 12 detailed migration phases +- Current vs. target architecture comparison +- Phase-by-phase approach (22 weeks total) +- Risk assessment and mitigation +- Success metrics and quality gates +- Backward compatibility strategy +- Deprecation timeline + +**Key Insight:** Migration will take ~5-6 months with proper planning and execution. + +### 2. 📊 FEATURE_MIGRATION_MATRIX.md (13.8 KB) +**Purpose:** Feature-level migration mapping + +**Contents:** +- 60+ features analyzed +- Each feature categorized: + - ⬆️ Move to @objectstack/runtime (42%) + - 🔄 Refactor as Plugin (47%) + - ✅ Keep in @objectql (11%) +- Priority levels (P0-P3) +- Effort estimates (High/Medium/Low) +- 6 migration waves with dependencies +- Critical path analysis + +**Key Insight:** Almost half of current ObjectQL code should move to @objectstack/runtime, the other half becomes plugins. + +### 3. 🔌 PLUGIN_ARCHITECTURE.md (17.1 KB) +**Purpose:** Plugin system design specification + +**Contents:** +- 5 plugin types defined: + 1. Driver Plugins (database adapters) + 2. Query Processor Plugins (validation, optimization) + 3. Repository Plugins (enhanced CRUD) + 4. Feature Plugins (formulas, AI) + 5. Optimization Plugins (caching, performance) +- Complete TypeScript interfaces +- Plugin composition examples +- Development guide +- Best practices + +**Key Insight:** Plugin-first architecture enables modularity and extensibility. + +### 4. 📦 PACKAGE_RESTRUCTURING.md (14.7 KB) +**Purpose:** Package-level reorganization plan + +**Contents:** +- Current vs. target package structure +- Package-by-package migration plan +- New packages to create (6+ plugins) +- Dependencies and version changes +- Publishing strategy (Alpha → Beta → RC → GA) +- @objectql/legacy compatibility package + +**Key Insight:** Major restructuring required, but backward compatibility maintained. + +### 5. 🗓️ IMPLEMENTATION_ROADMAP.md (15.7 KB) +**Purpose:** Week-by-week execution plan + +**Contents:** +- 22 weeks broken down in detail +- Specific tasks for each week +- Hour estimates (626 total engineering hours) +- Team structure recommendations (2-3 engineers) +- Quality gates per week +- Resource allocation +- Contingency planning + +**Key Insight:** Clear, actionable roadmap with realistic time estimates. + +## Migration Summary + +### By the Numbers + +| Metric | Value | +|--------|-------| +| **Total Features** | 60+ | +| **Features → Runtime** | 25 (42%) | +| **Features → Plugins** | 28 (47%) | +| **Features → Keep** | 7 (11%) | +| **Packages to Migrate** | 15+ | +| **New Plugins to Create** | 6+ | +| **Drivers to Convert** | 8 | +| **Timeline** | 22 weeks | +| **Engineering Hours** | 626+ | +| **Recommended Team** | 2-3 engineers | +| **Documentation Pages** | 75+ | + +### Migration Waves + +**Wave 1: Foundation (Weeks 1-4)** +- Core types migration +- Plugin system setup +- Development tooling + +**Wave 2: Drivers (Weeks 5-6)** +- Convert all 8 drivers to plugins +- Driver plugin template + +**Wave 3: Core Plugins (Weeks 7-10)** +- Query validation plugin +- Advanced repository plugin + +**Wave 4: Enhanced Plugins (Weeks 11-18)** +- Formula engine plugin +- Query optimizer plugin +- Query cache plugin +- AI query generator plugin + +**Wave 5: Tools (Weeks 19-20)** +- CLI updates +- VSCode extension +- Server integration + +**Wave 6: Release (Weeks 21-22)** +- Testing & validation +- Documentation +- Beta release + +## Breaking Changes & Compatibility + +### API Changes + +**Before (v3.x):** +```typescript +import { ObjectQL } from '@objectql/core'; +import { SQLDriver } from '@objectql/driver-sql'; + +const app = new ObjectQL({ + datasources: { + default: new SQLDriver({ connection: 'postgresql://...' }) + } +}); + +await app.init(); +``` + +**After (v4.x):** +```typescript +import { createRuntime } from '@objectstack/runtime'; +import { sqlDriverPlugin } from '@objectql/driver-sql'; +import { queryValidationPlugin } from '@objectql/query-validation'; +import { advancedRepositoryPlugin } from '@objectql/advanced-repository'; + +const runtime = createRuntime({ + plugins: [ + sqlDriverPlugin({ connection: 'postgresql://...' }), + queryValidationPlugin({ strict: true }), + advancedRepositoryPlugin() + ] +}); + +await runtime.initialize(); +``` + +### Compatibility Strategy + +**v4.0.x:** Full backward compatibility via @objectql/legacy package +- Old API works with deprecation warnings +- Wrapper around new plugin system +- 6 months support + +**v4.1.x-v4.9.x:** Deprecated API still functional +- Encourage migration to new API +- Migration tools provided + +**v5.0.x:** Remove deprecated APIs +- Clean plugin-only architecture +- No legacy support + +## Resource Requirements + +### Team Structure (Recommended) + +**Option A: 3-Engineer Team (Faster)** +- 1× Tech Lead +- 2× Senior Engineers +- Duration: 14-16 weeks +- Total: ~450 engineering hours + +**Option B: 2-Engineer Team (Standard)** +- 1× Tech Lead +- 1× Senior Engineer +- Duration: 22 weeks +- Total: ~626 engineering hours + +**Option C: 2-Engineer Team (Part-time)** +- 2× Senior Engineers (60% allocation) +- Duration: 30-32 weeks +- Total: ~626 engineering hours + +### Budget Estimate + +Assuming average senior engineer cost of $100/hour: +- **Minimum:** $45,000 (Option A) +- **Standard:** $62,600 (Option B) +- **Extended:** $62,600 (Option C, longer timeline) + +Plus: +- Infrastructure costs (minimal) +- Testing/QA resources +- Documentation/technical writing +- Community support during migration + +## Risks & Mitigation + +### High Risks + +1. **Runtime Integration Complexity** + - **Risk:** @objectstack/runtime may not support all ObjectQL features + - **Mitigation:** Prototype in weeks 1-2, collaborate with @objectstack team + - **Fallback:** Extend timeline by 2-4 weeks + +2. **Performance Regression** + - **Risk:** Plugin overhead may impact performance + - **Mitigation:** Continuous benchmarking, optimization focus + - **Target:** <10% overhead acceptable + +3. **Breaking Changes Impact** + - **Risk:** Users may struggle with migration + - **Mitigation:** Comprehensive compatibility layer, migration tools + - **Support:** 6-month dual-version support + +### Medium Risks + +4. **Documentation Lag** + - **Risk:** Docs fall behind implementation + - **Mitigation:** Docs-first approach, weekly updates + +5. **Community Adoption** + - **Risk:** Slow migration to v4.x + - **Mitigation:** Early beta program, migration incentives + +## Success Criteria + +### Technical Success + +- ✅ 100% drivers converted to plugins +- ✅ 90%+ test coverage maintained +- ✅ <10% performance overhead vs v3.x +- ✅ Zero critical security vulnerabilities +- ✅ All examples functional + +### User Success + +- ✅ Clear migration documentation +- ✅ 80%+ user migration within 6 months +- ✅ <5% increase in GitHub issues +- ✅ Positive community feedback + +### Business Success + +- ✅ On-time delivery (22 weeks) +- ✅ On-budget delivery +- ✅ Active plugin ecosystem (5+ community plugins) + +## Recommendations + +### Immediate Actions (Week 1) + +1. **Stakeholder Review** + - Review all 5 planning documents + - Align on vision and timeline + - Approve budget and resources + +2. **Team Assembly** + - Hire/allocate 2-3 engineers + - Assign tech lead + - Set up project tracking + +3. **Environment Setup** + - Set up development environment + - Access to @objectstack packages + - CI/CD pipeline preparation + +4. **Kickoff Meeting** + - Present migration strategy + - Q&A with engineering team + - Begin Week 1 tasks + +### Go/No-Go Decision Factors + +**GO if:** +- ✅ @objectstack/runtime is stable and production-ready +- ✅ Engineering resources available (2-3 engineers for 22 weeks) +- ✅ Budget approved (~$60k-$80k) +- ✅ Community notified and supportive +- ✅ Leadership aligned on plugin-first vision + +**NO-GO if:** +- ❌ @objectstack/runtime not ready +- ❌ Insufficient engineering resources +- ❌ High community resistance +- ❌ Competing priorities + +## Conclusion + +The migration to @objectstack/runtime is **feasible and well-planned**. We have: + +1. ✅ **Clear Vision:** ObjectQL as specialized query plugin ecosystem +2. ✅ **Detailed Plan:** 75+ pages of comprehensive documentation +3. ✅ **Realistic Timeline:** 22 weeks with buffer +4. ✅ **Resource Estimates:** 626 engineering hours, 2-3 engineers +5. ✅ **Risk Mitigation:** Strategies for all identified risks +6. ✅ **Compatibility:** Backward compatibility maintained +7. ✅ **Quality Assurance:** Clear success criteria and gates + +**Recommendation:** **PROCEED** with migration following the documented roadmap. + +The transition will: +- ✅ Position ObjectQL as a specialized plugin ecosystem +- ✅ Reduce code duplication with @objectstack/runtime +- ✅ Improve modularity and extensibility +- ✅ Enable community plugin development +- ✅ Align with ObjectStack ecosystem vision + +## Next Steps + +1. **Review & Approve** (Week 0) + - Stakeholder review of all documents + - Budget and resource approval + - Timeline confirmation + +2. **Team Setup** (Week 0-1) + - Hire/allocate engineers + - Environment setup + - Knowledge transfer + +3. **Begin Execution** (Week 1) + - Start Phase 1: Foundation & Planning + - Follow IMPLEMENTATION_ROADMAP.md + - Weekly progress reviews + +4. **Beta Release** (Week 22) + - v4.0.0-beta.1 published + - Community feedback collection + - Preparation for GA + +5. **GA Release** (Week 26-28) + - v4.0.0 production release + - Migration support for v3.x users + - Plugin ecosystem launch + +--- + +## Document Index + +All planning documents created: + +1. **MIGRATION_STRATEGY.md** - High-level strategic plan +2. **FEATURE_MIGRATION_MATRIX.md** - Feature-by-feature mapping +3. **PLUGIN_ARCHITECTURE.md** - Plugin system design +4. **PACKAGE_RESTRUCTURING.md** - Package reorganization +5. **IMPLEMENTATION_ROADMAP.md** - Week-by-week execution plan +6. **EXECUTIVE_SUMMARY.md** - This document (overview) + +--- + +**Document Version:** 1.0.0 +**Date:** 2026-01-21 +**Status:** Complete - Ready for Review +**Prepared By:** ObjectQL Migration Planning Team +**Approved By:** _Pending Stakeholder Review_ diff --git a/MIGRATION_INDEX.md b/MIGRATION_INDEX.md new file mode 100644 index 00000000..fbbc0910 --- /dev/null +++ b/MIGRATION_INDEX.md @@ -0,0 +1,239 @@ +# Migration Documentation Index + +This directory contains comprehensive planning documentation for migrating ObjectQL from a standalone framework to a plugin-based architecture built on @objectstack/runtime. + +## Quick Navigation + +### 🎯 Start Here + +**[EXECUTIVE_SUMMARY.md](./EXECUTIVE_SUMMARY.md)** - **READ THIS FIRST** +- High-level overview of the entire migration +- Problem statement and current state +- Summary of all documents +- Go/No-Go decision factors +- Recommendations and next steps + +### 📚 Detailed Planning Documents + +#### 1. Strategic Planning + +**[MIGRATION_STRATEGY.md](./MIGRATION_STRATEGY.md)** (14.5 KB) +- **What:** High-level strategic approach +- **Who:** Decision-makers, tech leads +- **When:** Phase-by-phase breakdown (12 phases, 22 weeks) +- **Why:** Architecture rationale and vision +- **Key Sections:** + - Current state vs. target architecture + - Phase-based approach + - Technical decisions (5 major decisions) + - Risk assessment + - Success metrics + +#### 2. Feature Analysis + +**[FEATURE_MIGRATION_MATRIX.md](./FEATURE_MIGRATION_MATRIX.md)** (13.8 KB) +- **What:** Feature-by-feature migration mapping +- **Who:** Engineers, project managers +- **When:** 6 migration waves +- **Why:** Detailed breakdown of what goes where +- **Key Sections:** + - 60+ features analyzed + - Priority and effort estimates + - Migration waves with dependencies + - Critical path analysis + - Backward compatibility strategy + +#### 3. Technical Architecture + +**[PLUGIN_ARCHITECTURE.md](./PLUGIN_ARCHITECTURE.md)** (17.1 KB) +- **What:** Plugin system design specification +- **Who:** Plugin developers, architects +- **When:** Reference throughout development +- **Why:** Standards for plugin development +- **Key Sections:** + - 5 plugin types with interfaces + - Plugin composition patterns + - Development guide + - Best practices + - Official plugin registry + +#### 4. Package Organization + +**[PACKAGE_RESTRUCTURING.md](./PACKAGE_RESTRUCTURING.md)** (14.7 KB) +- **What:** Package-level reorganization plan +- **Who:** Build engineers, release managers +- **When:** Package migration timeline (14 weeks) +- **Why:** How code is organized and published +- **Key Sections:** + - Current vs. target package structure + - Package-by-package migration + - Publishing strategy + - Backward compatibility package + +#### 5. Execution Plan + +**[IMPLEMENTATION_ROADMAP.md](./IMPLEMENTATION_ROADMAP.md)** (15.7 KB) +- **What:** Week-by-week implementation guide +- **Who:** Engineering team, project managers +- **When:** Detailed weekly breakdown (22 weeks) +- **Why:** Actionable tasks and time estimates +- **Key Sections:** + - Week-by-week tasks + - Hour estimates (626 total) + - Resource allocation + - Quality gates + - Team structure recommendations + +## How to Use This Documentation + +### For Decision-Makers + +1. **Start:** [EXECUTIVE_SUMMARY.md](./EXECUTIVE_SUMMARY.md) +2. **Review:** [MIGRATION_STRATEGY.md](./MIGRATION_STRATEGY.md) (Sections: Vision, Timeline, Risks) +3. **Decide:** Go/No-Go based on resources and timeline +4. **Approve:** Budget (~$60k-$80k) and team allocation (2-3 engineers) + +### For Technical Leads + +1. **Start:** [EXECUTIVE_SUMMARY.md](./EXECUTIVE_SUMMARY.md) +2. **Deep Dive:** [MIGRATION_STRATEGY.md](./MIGRATION_STRATEGY.md) (All sections) +3. **Architecture:** [PLUGIN_ARCHITECTURE.md](./PLUGIN_ARCHITECTURE.md) +4. **Planning:** [FEATURE_MIGRATION_MATRIX.md](./FEATURE_MIGRATION_MATRIX.md) +5. **Execution:** [IMPLEMENTATION_ROADMAP.md](./IMPLEMENTATION_ROADMAP.md) + +### For Engineers + +1. **Start:** [EXECUTIVE_SUMMARY.md](./EXECUTIVE_SUMMARY.md) +2. **Architecture:** [PLUGIN_ARCHITECTURE.md](./PLUGIN_ARCHITECTURE.md) (Plugin types and examples) +3. **Tasks:** [IMPLEMENTATION_ROADMAP.md](./IMPLEMENTATION_ROADMAP.md) (Your assigned week) +4. **Reference:** [FEATURE_MIGRATION_MATRIX.md](./FEATURE_MIGRATION_MATRIX.md) (Feature details) +5. **Packages:** [PACKAGE_RESTRUCTURING.md](./PACKAGE_RESTRUCTURING.md) (Package structure) + +### For Project Managers + +1. **Start:** [EXECUTIVE_SUMMARY.md](./EXECUTIVE_SUMMARY.md) +2. **Timeline:** [IMPLEMENTATION_ROADMAP.md](./IMPLEMENTATION_ROADMAP.md) (Week-by-week) +3. **Resources:** [MIGRATION_STRATEGY.md](./MIGRATION_STRATEGY.md) (Team structure) +4. **Risks:** [MIGRATION_STRATEGY.md](./MIGRATION_STRATEGY.md) (Risk assessment) +5. **Tracking:** [FEATURE_MIGRATION_MATRIX.md](./FEATURE_MIGRATION_MATRIX.md) (Progress tracking) + +## Document Statistics + +| Document | Size | Pages | Purpose | +|----------|------|-------|---------| +| EXECUTIVE_SUMMARY.md | 13.4 KB | ~8 | Overview & recommendations | +| MIGRATION_STRATEGY.md | 14.5 KB | ~25 | Strategic approach | +| FEATURE_MIGRATION_MATRIX.md | 13.8 KB | ~20 | Feature mapping | +| PLUGIN_ARCHITECTURE.md | 17.1 KB | ~30 | Technical design | +| PACKAGE_RESTRUCTURING.md | 14.7 KB | ~22 | Package organization | +| IMPLEMENTATION_ROADMAP.md | 15.7 KB | ~28 | Weekly execution | +| **Total** | **89.2 KB** | **~133** | Complete planning | + +## Key Metrics Summary + +### Migration Scope +- **Features to Migrate:** 60+ +- **New Plugins to Create:** 6+ +- **Drivers to Convert:** 8 +- **Packages to Restructure:** 15+ + +### Timeline & Resources +- **Total Duration:** 22 weeks (5-6 months) +- **Engineering Hours:** 626+ hours +- **Team Size:** 2-3 engineers +- **Budget Estimate:** $60,000 - $80,000 + +### Quality Targets +- **Test Coverage:** 90%+ +- **Performance Overhead:** <10% +- **Security Issues:** Zero critical +- **User Migration:** 80%+ within 6 months + +## Migration Phases Quick Reference + +| Phase | Weeks | Focus | +|-------|-------|-------| +| 1. Foundation | 1-4 | Types, dev environment, automation | +| 2. Drivers | 5-6 | Convert 8 drivers to plugins | +| 3. Core Plugins | 7-10 | Validation, repository | +| 4. Features | 11-18 | Formulas, optimizer, cache, AI | +| 5. Tools | 19-20 | CLI, VSCode, server | +| 6. Release | 21-22 | Testing, docs, beta | + +## Migration Waves Quick Reference + +| Wave | Dependencies | Features | +|------|--------------|----------| +| 1 | None | Runtime integration, types, plugins | +| 2 | Wave 1 | All drivers as plugins | +| 3 | Waves 1-2 | Query validation, repository | +| 4 | Waves 1-3 | Optimizer, cache, AI | +| 5 | All previous | Tools, integrations | +| 6 | All previous | Testing, documentation | + +## Document Relationships + +``` +EXECUTIVE_SUMMARY + │ + ├── MIGRATION_STRATEGY ──┐ + │ └── Phases 1-12 │ + │ │ + ├── FEATURE_MIGRATION ───┤ + │ └── 60+ features │ + │ ├── IMPLEMENTATION_ROADMAP + ├── PLUGIN_ARCHITECTURE ─┤ └── Week 1-22 + │ └── 5 plugin types │ + │ │ + └── PACKAGE_RESTRUCTURING┘ + └── Package changes +``` + +## Glossary + +- **@objectstack/runtime:** Base framework providing application lifecycle, metadata, and plugin system +- **@objectstack/spec:** Protocol specification defining DriverInterface and QueryAST +- **Plugin:** Modular component that extends runtime functionality +- **Driver:** Database adapter implementing DriverInterface +- **QueryAST:** Abstract syntax tree representing a query +- **Migration Wave:** Group of related migration tasks with dependencies +- **Quality Gate:** Success criteria that must be met before proceeding + +## Version History + +| Version | Date | Changes | +|---------|------|---------| +| 1.0.0 | 2026-01-21 | Initial comprehensive planning documentation | + +## Status + +**Current Phase:** Documentation Complete ✅ +**Next Phase:** Stakeholder Review & Approval +**Overall Status:** Ready for execution pending approval + +## Contact & Support + +For questions about this migration: +- **Technical Questions:** Review plugin architecture and implementation roadmap +- **Timeline Questions:** Review implementation roadmap +- **Resource Questions:** Review migration strategy +- **Scope Questions:** Review feature migration matrix + +## Getting Started + +**Recommended Reading Order:** + +1. **15 min:** [EXECUTIVE_SUMMARY.md](./EXECUTIVE_SUMMARY.md) - Get the big picture +2. **30 min:** [MIGRATION_STRATEGY.md](./MIGRATION_STRATEGY.md) - Understand the approach +3. **45 min:** [IMPLEMENTATION_ROADMAP.md](./IMPLEMENTATION_ROADMAP.md) - See the timeline +4. **60 min:** [PLUGIN_ARCHITECTURE.md](./PLUGIN_ARCHITECTURE.md) - Learn the architecture +5. **30 min:** [FEATURE_MIGRATION_MATRIX.md](./FEATURE_MIGRATION_MATRIX.md) - Review feature details +6. **30 min:** [PACKAGE_RESTRUCTURING.md](./PACKAGE_RESTRUCTURING.md) - Understand packages + +**Total:** ~3.5 hours for complete understanding + +--- + +**Last Updated:** 2026-01-21 +**Maintained By:** ObjectQL Migration Team +**Document Index Version:** 1.0.0 From a46967f1762394f2866953d1575c38fc8acdebcb Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Wed, 21 Jan 2026 11:40:16 +0000 Subject: [PATCH 5/9] Fix markdown formatting issues in migration documentation Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com> --- PACKAGE_RESTRUCTURING.md | 20 +++++++++----------- PLUGIN_ARCHITECTURE.md | 8 ++++---- 2 files changed, 13 insertions(+), 15 deletions(-) diff --git a/PACKAGE_RESTRUCTURING.md b/PACKAGE_RESTRUCTURING.md index fb969465..4925e0d1 100644 --- a/PACKAGE_RESTRUCTURING.md +++ b/PACKAGE_RESTRUCTURING.md @@ -76,17 +76,15 @@ packages/ - Thin type re-exports from @objectstack/spec **Changes:** -```diff -- Remove: Driver interface (use @objectstack/spec) -- Remove: MetadataRegistry types (use @objectstack/runtime) -- Remove: Context types (use @objectstack/runtime) -- Remove: Hook types (use @objectstack/runtime) -- Remove: Action types (use @objectstack/runtime) -+ Add: QueryPlugin interface -+ Add: RepositoryPlugin interface -+ Add: Plugin-specific types -+ Keep: Query-specific helper types -``` +- ❌ Remove: Driver interface (use @objectstack/spec) +- ❌ Remove: MetadataRegistry types (use @objectstack/runtime) +- ❌ Remove: Context types (use @objectstack/runtime) +- ❌ Remove: Hook types (use @objectstack/runtime) +- ❌ Remove: Action types (use @objectstack/runtime) +- ✅ Add: QueryPlugin interface +- ✅ Add: RepositoryPlugin interface +- ✅ Add: Plugin-specific types +- ✅ Keep: Query-specific helper types **Migration Steps:** 1. Create `/packages/core/types/` (new location) diff --git a/PLUGIN_ARCHITECTURE.md b/PLUGIN_ARCHITECTURE.md index 4f06d484..a3e14384 100644 --- a/PLUGIN_ARCHITECTURE.md +++ b/PLUGIN_ARCHITECTURE.md @@ -7,7 +7,7 @@ This document defines the plugin architecture for ObjectQL 4.0, where the reposi ## Design Principles ### 1. Plugin-First Philosophy -**Every feature is a plugin.** Even core query functionality is modular and composable. +**Every feature is a Plugin.** Even core query functionality is modular and composable. ### 2. Clear Separation of Concerns - **@objectstack/runtime**: Application lifecycle, metadata, security, transactions @@ -581,13 +581,13 @@ My awesome plugin for ObjectQL. ## Installation -\`\`\`bash +```bash npm install @objectql/my-plugin -\`\`\` +``` ## Usage -\`\`\`typescript +```typescript import { createRuntime } from '@objectstack/runtime'; import { myPlugin } from '@objectql/my-plugin'; From 1857b5ae55bdae667794c23159199b4290f8a31f Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Wed, 21 Jan 2026 12:23:29 +0000 Subject: [PATCH 6/9] Initial plan From 8694d3e71583b6813dcbed1c940314df0a44a333 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Wed, 21 Jan 2026 12:29:14 +0000 Subject: [PATCH 7/9] feat: Create @objectql/types v4.0.0 with plugin interfaces Co-authored-by: huangyiirene <7665279+huangyiirene@users.noreply.github.com> --- packages/core/types/.gitignore | 5 + packages/core/types/CHANGELOG.md | 67 +++++++ packages/core/types/README.md | 56 ++++++ packages/core/types/jest.config.js | 9 + packages/core/types/package-lock.json | 64 +++++++ packages/core/types/package.json | 28 +++ packages/core/types/src/index.ts | 61 +++++++ packages/core/types/src/plugin.ts | 67 +++++++ packages/core/types/src/query-processor.ts | 171 ++++++++++++++++++ packages/core/types/src/query.ts | 155 ++++++++++++++++ packages/core/types/src/repository.ts | 201 +++++++++++++++++++++ packages/core/types/tsconfig.json | 9 + pnpm-workspace.yaml | 2 + tsconfig.json | 1 + 14 files changed, 896 insertions(+) create mode 100644 packages/core/types/.gitignore create mode 100644 packages/core/types/CHANGELOG.md create mode 100644 packages/core/types/README.md create mode 100644 packages/core/types/jest.config.js create mode 100644 packages/core/types/package-lock.json create mode 100644 packages/core/types/package.json create mode 100644 packages/core/types/src/index.ts create mode 100644 packages/core/types/src/plugin.ts create mode 100644 packages/core/types/src/query-processor.ts create mode 100644 packages/core/types/src/query.ts create mode 100644 packages/core/types/src/repository.ts create mode 100644 packages/core/types/tsconfig.json diff --git a/packages/core/types/.gitignore b/packages/core/types/.gitignore new file mode 100644 index 00000000..0b50d775 --- /dev/null +++ b/packages/core/types/.gitignore @@ -0,0 +1,5 @@ +node_modules +dist +*.tsbuildinfo +coverage +.DS_Store diff --git a/packages/core/types/CHANGELOG.md b/packages/core/types/CHANGELOG.md new file mode 100644 index 00000000..30326f7e --- /dev/null +++ b/packages/core/types/CHANGELOG.md @@ -0,0 +1,67 @@ +# @objectql/types v4.0.0 + +## [4.0.0] - 2026-01-21 + +### Breaking Changes + +- Complete rewrite of type system for plugin-based architecture +- Removed types now in @objectstack/spec: + - `Driver` interface (use `DriverInterface` from @objectstack/spec) +- Removed types now in @objectstack/runtime: + - `MetadataRegistry` class + - `Context` types + - `Hook` types + - `Action` types + +### Added + +- New plugin interfaces: + - `QueryProcessorPlugin` - For query validation, optimization, and transformation + - `RepositoryPlugin` - For extending repository with batch operations, audit tracking + - `BasePlugin` - Base interface for all plugins +- Query-specific types: + - `UnifiedQuery` - Core query structure + - `FilterExpression` - Type-safe filter expressions + - `QueryResult` - Query result wrapper with pagination + - `QueryOptions` - Query execution options +- Runtime types: + - `RuntimeContext` - Context available to plugins during execution + - `ValidationResult` - Query validation result + - `ValidationError` - Validation error details + +### Dependencies + +- Added `@objectstack/spec` ^0.2.0 +- Added `@objectstack/runtime` ^0.2.0 + +### Migration Guide + +If you were using types that are now removed: + +```typescript +// Old (v3.x) +import { Driver, MetadataRegistry, Hook } from '@objectql/types'; + +// New (v4.x) +import { DriverInterface } from '@objectstack/spec'; +// MetadataRegistry, Hook, Action are now part of @objectstack/runtime +``` + +For plugin development: + +```typescript +// New in v4.x +import { QueryProcessorPlugin, RepositoryPlugin } from '@objectql/types'; + +export function myPlugin(): QueryProcessorPlugin { + return { + name: '@myorg/plugin', + version: '1.0.0', + type: 'query-processor', + async beforeQuery(ast, context) { + // Transform query + return ast; + } + }; +} +``` diff --git a/packages/core/types/README.md b/packages/core/types/README.md new file mode 100644 index 00000000..69684cde --- /dev/null +++ b/packages/core/types/README.md @@ -0,0 +1,56 @@ +# @objectql/types + +Type definitions and plugin interfaces for ObjectQL v4.0. + +## Overview + +This package provides the core TypeScript type definitions for ObjectQL's plugin architecture. It includes: + +- **Plugin Interfaces**: Define custom query plugins, repository extensions, and query processors +- **Query Types**: Type-safe query construction and manipulation +- **Helper Types**: Utility types for working with ObjectQL queries + +## Key Principles + +1. **Zero Dependencies on Internal Packages**: This package depends only on `@objectstack/spec` and `@objectstack/runtime` +2. **Protocol-Driven**: Types define the contract, implementation follows +3. **Strict Type Safety**: All types use TypeScript strict mode + +## Installation + +```bash +npm install @objectql/types +``` + +## Usage + +```typescript +import { QueryPlugin, RepositoryPlugin, UnifiedQuery } from '@objectql/types'; + +// Define a custom query plugin +export function myQueryPlugin(): QueryPlugin { + return { + name: '@myorg/query-plugin', + version: '1.0.0', + type: 'query-processor', + async beforeQuery(ast, context) { + // Transform query before execution + return ast; + } + }; +} +``` + +## Migration from v3.x + +In v4.0, ObjectQL has transitioned to a plugin-based architecture built on `@objectstack/runtime`: + +- **Driver Interface**: Now uses `DriverInterface` from `@objectstack/spec` +- **Metadata & Context**: Now managed by `@objectstack/runtime` +- **Hooks & Actions**: Now part of `@objectstack/runtime` + +This package focuses exclusively on query-specific types and plugin interfaces. + +## License + +MIT © ObjectStack Inc. diff --git a/packages/core/types/jest.config.js b/packages/core/types/jest.config.js new file mode 100644 index 00000000..96f9ee81 --- /dev/null +++ b/packages/core/types/jest.config.js @@ -0,0 +1,9 @@ +module.exports = { + preset: 'ts-jest', + testEnvironment: 'node', + testMatch: ['**/__tests__/**/*.test.ts'], + collectCoverageFrom: [ + 'src/**/*.ts', + '!src/**/*.d.ts', + ], +}; diff --git a/packages/core/types/package-lock.json b/packages/core/types/package-lock.json new file mode 100644 index 00000000..dc5d284e --- /dev/null +++ b/packages/core/types/package-lock.json @@ -0,0 +1,64 @@ +{ + "name": "@objectql/types", + "version": "4.0.0", + "lockfileVersion": 3, + "requires": true, + "packages": { + "": { + "name": "@objectql/types", + "version": "4.0.0", + "license": "MIT", + "dependencies": { + "@objectstack/runtime": "^0.2.0", + "@objectstack/spec": "^0.2.0" + } + }, + "node_modules/@objectstack/objectql": { + "version": "0.2.0", + "resolved": "https://registry.npmjs.org/@objectstack/objectql/-/objectql-0.2.0.tgz", + "integrity": "sha512-LCgCs+K7J4/rOwJZdXFZgm+zVD936ppRgRVwYn3Uc8xP/JZeFg/DDrhW+O1li3Enwbai2eKMCRB6XYinOhoCAg==", + "dependencies": { + "@objectstack/spec": "0.2.0" + } + }, + "node_modules/@objectstack/runtime": { + "version": "0.2.0", + "resolved": "https://registry.npmjs.org/@objectstack/runtime/-/runtime-0.2.0.tgz", + "integrity": "sha512-nKDm3HSbGDkpccGKXDXhOr3nvEOgz6cp1j/z74DoreVD/6gIH6PuWPldHJbOVdR/nhPHNV/ViK7tvGmLML2v1A==", + "dependencies": { + "@objectstack/objectql": "0.2.0", + "@objectstack/spec": "0.2.0", + "@objectstack/types": "0.2.0" + } + }, + "node_modules/@objectstack/spec": { + "version": "0.2.0", + "resolved": "https://registry.npmjs.org/@objectstack/spec/-/spec-0.2.0.tgz", + "integrity": "sha512-y4JALcsTgOeEE0xRJ7Co16beQTIO42c6KngFFcy7QQH0BeN4gKgMoqWGkRxPfOKZnnlUzYo/1hSLxmmUtJMisA==", + "license": "Apache-2.0", + "dependencies": { + "zod": "^3.22.4" + }, + "engines": { + "node": ">=18.0.0" + } + }, + "node_modules/@objectstack/types": { + "version": "0.2.0", + "resolved": "https://registry.npmjs.org/@objectstack/types/-/types-0.2.0.tgz", + "integrity": "sha512-zxZ4vMuVETKgCp19i2lPMP0767fJZ0hXasJbdVrc2RvBa06fCbMCmJWc4bsn21XbyGEMaE/BFlqEXw+F4Gyh9Q==", + "dependencies": { + "@objectstack/spec": "0.2.0" + } + }, + "node_modules/zod": { + "version": "3.25.76", + "resolved": "https://registry.npmjs.org/zod/-/zod-3.25.76.tgz", + "integrity": "sha512-gzUt/qt81nXsFGKIFcC3YnfEAx5NkunCfnDlvuBSSFS02bcXu4Lmea0AFIUwbLWxWPx3d9p8S5QoaujKcNQxcQ==", + "license": "MIT", + "funding": { + "url": "https://github.com/sponsors/colinhacks" + } + } + } +} diff --git a/packages/core/types/package.json b/packages/core/types/package.json new file mode 100644 index 00000000..c0ed8530 --- /dev/null +++ b/packages/core/types/package.json @@ -0,0 +1,28 @@ +{ + "name": "@objectql/types", + "version": "4.0.0", + "description": "Type definitions and plugin interfaces for ObjectQL - Query-specific types and plugin architecture", + "keywords": [ + "objectql", + "types", + "typescript", + "interfaces", + "plugin", + "query", + "ai-native" + ], + "license": "MIT", + "main": "dist/index.js", + "types": "dist/index.d.ts", + "files": [ + "dist" + ], + "scripts": { + "build": "tsc", + "test": "jest --passWithNoTests" + }, + "dependencies": { + "@objectstack/spec": "^0.2.0", + "@objectstack/runtime": "^0.2.0" + } +} diff --git a/packages/core/types/src/index.ts b/packages/core/types/src/index.ts new file mode 100644 index 00000000..a7825f06 --- /dev/null +++ b/packages/core/types/src/index.ts @@ -0,0 +1,61 @@ +/** + * ObjectQL + * Copyright (c) 2026-present ObjectStack Inc. + * + * This source code is licensed under the MIT license found in the + * LICENSE file in the root directory of this source tree. + */ + +/** + * @objectql/types - Type definitions for ObjectQL v4.0 + * + * This package provides the core TypeScript type definitions for ObjectQL's + * plugin-based architecture. It includes: + * + * - Plugin interfaces (QueryProcessorPlugin, RepositoryPlugin) + * - Query types (UnifiedQuery, FilterExpression) + * - Runtime context and options + * + * Key principles: + * - Zero dependencies on other @objectql packages + * - Depends only on @objectstack/spec and @objectstack/runtime + * - Strict type safety + * - Protocol-driven design + */ + +// ============================================================================ +// Plugin System +// ============================================================================ + +export * from './plugin'; +export * from './query-processor'; +export * from './repository'; + +// ============================================================================ +// Query System +// ============================================================================ + +export * from './query'; + +// ============================================================================ +// Re-exports from @objectstack packages +// ============================================================================ + +/** + * Re-export commonly used types from @objectstack/spec + * This provides a convenient import path for ObjectQL users. + */ +export type { + DriverInterface, + // Add other spec types as needed +} from '@objectstack/spec'; + +/** + * Re-export commonly used types from @objectstack/runtime + * This provides a convenient import path for ObjectQL users. + */ +export type { + // RuntimePlugin, + // Runtime, + // Add other runtime types as needed +} from '@objectstack/runtime'; diff --git a/packages/core/types/src/plugin.ts b/packages/core/types/src/plugin.ts new file mode 100644 index 00000000..73da984b --- /dev/null +++ b/packages/core/types/src/plugin.ts @@ -0,0 +1,67 @@ +/** + * ObjectQL + * Copyright (c) 2026-present ObjectStack Inc. + * + * This source code is licensed under the MIT license found in the + * LICENSE file in the root directory of this source tree. + */ + +/** + * Base plugin metadata required by all ObjectQL plugins. + * + * This extends the RuntimePlugin from @objectstack/runtime with + * ObjectQL-specific plugin types. + */ +export interface PluginMetadata { + /** Unique plugin identifier (e.g., '@objectql/query-validation') */ + name: string; + + /** Semantic version (e.g., '4.0.0') */ + version: string; + + /** Plugin type discriminator */ + type: PluginType; + + /** Required plugin dependencies */ + dependencies?: string[]; + + /** Optional plugin dependencies that enhance functionality */ + optionalDependencies?: string[]; + + /** Plugins that conflict with this one */ + conflicts?: string[]; +} + +/** + * Supported ObjectQL plugin types. + */ +export type PluginType = + | 'driver' + | 'query-processor' + | 'repository' + | 'feature' + | 'custom'; + +/** + * Plugin lifecycle methods. + */ +export interface PluginLifecycle { + /** + * Called when the plugin is registered with the runtime. + * Use this to initialize resources, register handlers, etc. + */ + setup?(runtime: any): Promise; + + /** + * Called when the runtime is shutting down. + * Use this to cleanup resources, close connections, etc. + */ + teardown?(runtime: any): Promise; +} + +/** + * Base plugin interface that all ObjectQL plugins must implement. + * + * This is a minimal interface that can be extended by specific plugin types. + */ +export interface BasePlugin extends PluginMetadata, PluginLifecycle {} diff --git a/packages/core/types/src/query-processor.ts b/packages/core/types/src/query-processor.ts new file mode 100644 index 00000000..db27d2d9 --- /dev/null +++ b/packages/core/types/src/query-processor.ts @@ -0,0 +1,171 @@ +/** + * ObjectQL + * Copyright (c) 2026-present ObjectStack Inc. + * + * This source code is licensed under the MIT license found in the + * LICENSE file in the root directory of this source tree. + */ + +import { BasePlugin } from './plugin'; + +/** + * Query AST validation result. + */ +export interface ValidationResult { + /** Whether the query is valid */ + valid: boolean; + + /** List of validation errors, if any */ + errors: ValidationError[]; +} + +/** + * Validation error details. + */ +export interface ValidationError { + /** Field or path that failed validation */ + field: string; + + /** Human-readable error message */ + message: string; + + /** Error code for programmatic handling */ + code?: string; + + /** Additional error metadata */ + metadata?: Record; +} + +/** + * Query processor plugin interface. + * + * Query processors enhance the query execution pipeline by: + * - Validating queries before execution + * - Optimizing query AST for performance + * - Transforming queries (e.g., adding filters, modifying fields) + * - Post-processing query results + * + * Execution order: + * 1. validateQuery (if provided) + * 2. beforeQuery (if provided) + * 3. optimizeQuery (if provided) + * 4. [Query Execution] + * 5. afterQuery (if provided) + */ +export interface QueryProcessorPlugin extends BasePlugin { + type: 'query-processor'; + + /** + * Validate a query AST before execution. + * + * @param ast - The query AST to validate + * @param context - Execution context with runtime and user info + * @returns Validation result with any errors + */ + validateQuery?( + ast: any, + context: RuntimeContext + ): Promise; + + /** + * Transform a query AST before execution. + * + * Use this to: + * - Add security filters + * - Inject computed fields + * - Modify query structure + * + * @param ast - The query AST to transform + * @param context - Execution context + * @returns Transformed query AST + * @throws If the query is invalid + */ + beforeQuery?( + ast: any, + context: RuntimeContext + ): Promise; + + /** + * Optimize a query AST for better performance. + * + * Use this to: + * - Remove redundant filters + * - Add index hints + * - Optimize join order + * - Prune unused fields + * + * @param ast - The query AST to optimize + * @param context - Execution context + * @returns Optimized query AST + */ + optimizeQuery?( + ast: any, + context: RuntimeContext + ): Promise; + + /** + * Transform query results after execution. + * + * Use this to: + * - Post-process data + * - Apply computed fields + * - Filter sensitive information + * + * @param result - The query result to transform + * @param context - Execution context + * @returns Transformed result + */ + afterQuery?( + result: any, + context: RuntimeContext + ): Promise; +} + +/** + * Runtime context available to plugins during query execution. + */ +export interface RuntimeContext { + /** Runtime instance (avoid circular dep by using 'any') */ + runtime: any; + + /** User ID who initiated the query */ + userId?: string | number; + + /** Session or authentication token */ + sessionId?: string; + + /** User roles for permission checking */ + roles?: string[]; + + /** Additional context metadata */ + metadata?: Record; + + /** Query being executed (set by beforeQuery) */ + query?: any; + + /** Flag to skip query execution (e.g., for cache hits) */ + skipExecution?: boolean; + + /** Cached result (if skipExecution is true) */ + cachedResult?: any; +} + +/** + * Configuration options for query validation plugins. + */ +export interface ValidationOptions { + /** Enable strict validation mode */ + strict?: boolean; + + /** Allow querying unknown fields */ + allowUnknownFields?: boolean; + + /** Custom validator functions */ + customValidators?: Record boolean>; + + /** Maximum query depth */ + maxDepth?: number; + + /** Maximum number of fields */ + maxFields?: number; +} diff --git a/packages/core/types/src/query.ts b/packages/core/types/src/query.ts new file mode 100644 index 00000000..b9ccdfd7 --- /dev/null +++ b/packages/core/types/src/query.ts @@ -0,0 +1,155 @@ +/** + * ObjectQL + * Copyright (c) 2026-present ObjectStack Inc. + * + * This source code is licensed under the MIT license found in the + * LICENSE file in the root directory of this source tree. + */ + +/** + * Filter criterion: [field, operator, value] + * + * Examples: + * - ['name', '=', 'John'] + * - ['age', '>', 18] + * - ['status', 'in', ['active', 'pending']] + */ +export type FilterCriterion = [string, string, any]; + +/** + * Filter expression supporting nested logical operators. + * + * Can be: + * - A single criterion: ['name', '=', 'John'] + * - A logical operator: 'and' | 'or' + * - A nested array: [criterion1, 'and', criterion2, 'or', criterion3] + */ +export type FilterExpression = FilterCriterion | 'and' | 'or' | FilterExpression[]; + +/** + * Aggregate function types. + */ +export type AggregateFunction = 'count' | 'sum' | 'avg' | 'min' | 'max'; + +/** + * Aggregate operation configuration. + */ +export interface AggregateOption { + /** Aggregate function to apply */ + func: AggregateFunction; + + /** Field to aggregate */ + field: string; + + /** Optional alias for the result field */ + alias?: string; +} + +/** + * Sort direction. + */ +export type SortDirection = 'asc' | 'desc'; + +/** + * Sort configuration: [field, direction] + */ +export type SortOption = [string, SortDirection]; + +/** + * Unified query interface for ObjectQL. + * + * This is the core query structure used throughout ObjectQL for: + * - Finding records + * - Filtering data + * - Sorting and pagination + * - Aggregations + * - Expanding related records + */ +export interface UnifiedQuery { + /** + * Fields to select. + * If not specified, all fields are returned. + */ + fields?: string[]; + + /** + * Filter expressions to apply. + * Multiple filters are combined with AND by default. + */ + filters?: FilterExpression[]; + + /** + * Sort options. + * Multiple sorts are applied in order. + */ + sort?: SortOption[]; + + /** + * Number of records to skip (for pagination). + */ + skip?: number; + + /** + * Maximum number of records to return. + */ + limit?: number; + + /** + * Expand related records (joins). + * Key is the relation field name, value is the query for that relation. + */ + expand?: Record; + + /** + * Group by fields (for aggregations). + */ + groupBy?: string[]; + + /** + * Aggregate operations to apply. + */ + aggregate?: AggregateOption[]; +} + +/** + * Query result wrapper with pagination metadata. + */ +export interface QueryResult { + /** Query results */ + data: T[]; + + /** Total count (ignoring skip/limit) */ + total?: number; + + /** Number of records skipped */ + skip?: number; + + /** Maximum number of records returned */ + limit?: number; + + /** Whether there are more records available */ + hasMore?: boolean; +} + +/** + * Query execution options. + */ +export interface QueryOptions { + /** Transaction to execute within */ + transaction?: any; + + /** User context for permission checking */ + userId?: string | number; + + /** Skip permission checks */ + skipPermissions?: boolean; + + /** Skip validation */ + skipValidation?: boolean; + + /** Skip hooks */ + skipHooks?: boolean; + + /** Additional metadata */ + metadata?: Record; +} diff --git a/packages/core/types/src/repository.ts b/packages/core/types/src/repository.ts new file mode 100644 index 00000000..bf7ef655 --- /dev/null +++ b/packages/core/types/src/repository.ts @@ -0,0 +1,201 @@ +/** + * ObjectQL + * Copyright (c) 2026-present ObjectStack Inc. + * + * This source code is licensed under the MIT license found in the + * LICENSE file in the root directory of this source tree. + */ + +import { BasePlugin } from './plugin'; +import { RuntimeContext } from './query-processor'; + +/** + * Repository plugin interface. + * + * Repository plugins extend the base repository with additional capabilities: + * - Batch operations (createMany, updateMany, deleteMany) + * - Upsert operations (create or update) + * - Soft delete functionality + * - Audit tracking (createdBy, updatedBy, etc.) + * - Custom CRUD methods + * + * Lifecycle hooks allow plugins to intercept and modify CRUD operations. + */ +export interface RepositoryPlugin extends BasePlugin { + type: 'repository'; + + /** + * Extend the base repository with custom methods. + * + * Use this to add new methods to the repository instance: + * - repo.createMany(records) + * - repo.upsert(id, data) + * - repo.softDelete(id) + * + * @param repository - Base repository instance (avoid circular dep) + * @param context - Execution context + */ + extendRepository?( + repository: any, + context: RuntimeContext + ): void; + + /** + * Hook called before creating a new record. + * + * Use this to: + * - Add audit fields (createdBy, createdAt) + * - Validate data + * - Set default values + * - Transform data + * + * @param data - Data to be created + * @param context - Execution context + * @returns Modified data + */ + beforeCreate?( + data: any, + context: RuntimeContext + ): Promise; + + /** + * Hook called after creating a new record. + * + * Use this to: + * - Emit events + * - Update related records + * - Clear caches + * + * @param result - Created record + * @param context - Execution context + * @returns Modified result + */ + afterCreate?( + result: any, + context: RuntimeContext + ): Promise; + + /** + * Hook called before updating a record. + * + * Use this to: + * - Add audit fields (updatedBy, updatedAt) + * - Validate changes + * - Transform data + * + * @param id - Record ID to update + * @param data - Update data + * @param context - Execution context + * @returns Modified data + */ + beforeUpdate?( + id: string | number, + data: any, + context: RuntimeContext + ): Promise; + + /** + * Hook called after updating a record. + * + * Use this to: + * - Emit events + * - Update related records + * - Clear caches + * + * @param result - Updated record + * @param context - Execution context + * @returns Modified result + */ + afterUpdate?( + result: any, + context: RuntimeContext + ): Promise; + + /** + * Hook called before deleting a record. + * + * Use this to: + * - Validate deletion + * - Check dependencies + * - Implement soft delete + * + * @param id - Record ID to delete + * @param context - Execution context + */ + beforeDelete?( + id: string | number, + context: RuntimeContext + ): Promise; + + /** + * Hook called after deleting a record. + * + * Use this to: + * - Emit events + * - Cleanup related records + * - Clear caches + * + * @param id - Deleted record ID + * @param context - Execution context + */ + afterDelete?( + id: string | number, + context: RuntimeContext + ): Promise; + + /** + * Hook called before finding records. + * + * Use this to: + * - Add security filters + * - Modify query + * + * @param query - Query AST + * @param context - Execution context + * @returns Modified query + */ + beforeFind?( + query: any, + context: RuntimeContext + ): Promise; + + /** + * Hook called after finding records. + * + * Use this to: + * - Filter results + * - Transform data + * - Add computed fields + * + * @param results - Query results + * @param context - Execution context + * @returns Modified results + */ + afterFind?( + results: any[], + context: RuntimeContext + ): Promise; +} + +/** + * Configuration options for repository plugins. + */ +export interface RepositoryOptions { + /** Enable automatic audit fields (_createdBy, _createdAt, etc.) */ + auditFields?: boolean; + + /** Enable soft delete functionality */ + softDelete?: boolean; + + /** Field name for soft delete flag */ + softDeleteField?: string; + + /** Field name for deletion timestamp */ + softDeleteTimestampField?: string; + + /** Enable batch operations */ + enableBatchOperations?: boolean; + + /** Maximum batch size for bulk operations */ + maxBatchSize?: number; +} diff --git a/packages/core/types/tsconfig.json b/packages/core/types/tsconfig.json new file mode 100644 index 00000000..5caba7d1 --- /dev/null +++ b/packages/core/types/tsconfig.json @@ -0,0 +1,9 @@ +{ + "extends": "../../../tsconfig.base.json", + "compilerOptions": { + "outDir": "./dist", + "rootDir": "./src" + }, + "include": ["src/**/*"], + "references": [] +} diff --git a/pnpm-workspace.yaml b/pnpm-workspace.yaml index 1bc9005c..c1b451e3 100644 --- a/pnpm-workspace.yaml +++ b/pnpm-workspace.yaml @@ -1,5 +1,7 @@ packages: - packages/foundation/* + - packages/core/* + - packages/plugins/* - packages/drivers/* - packages/runtime/* - packages/tools/* diff --git a/tsconfig.json b/tsconfig.json index 5c2ac2d0..38abb6fe 100644 --- a/tsconfig.json +++ b/tsconfig.json @@ -1,6 +1,7 @@ { "files": [], "references": [ + { "path": "./packages/core/types" }, { "path": "./packages/foundation/types" }, { "path": "./packages/foundation/core" }, { "path": "./packages/drivers/mongo" }, From d63ded2331f0d412fc4650d597d0b6502fafa419 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Wed, 21 Jan 2026 12:30:28 +0000 Subject: [PATCH 8/9] docs: Add Phase 1 completion documentation and plugin directory README Co-authored-by: huangyiirene <7665279+huangyiirene@users.noreply.github.com> --- PHASE_1_COMPLETE.md | 153 +++++++++++++++++++++++++++++++++++++ packages/plugins/README.md | 96 +++++++++++++++++++++++ 2 files changed, 249 insertions(+) create mode 100644 PHASE_1_COMPLETE.md create mode 100644 packages/plugins/README.md diff --git a/PHASE_1_COMPLETE.md b/PHASE_1_COMPLETE.md new file mode 100644 index 00000000..9d5b6638 --- /dev/null +++ b/PHASE_1_COMPLETE.md @@ -0,0 +1,153 @@ +# Phase 1 Migration Complete: Foundation & Core Types + +## Overview + +Phase 1 of the ObjectQL v4.0 migration has been completed. This phase establishes the foundation for the new plugin-based architecture. + +## What Changed + +### 1. New Directory Structure + +``` +packages/ +├── core/ +│ └── types/ # NEW: @objectql/types v4.0.0 +├── plugins/ # NEW: Plugin directory (empty, ready for plugins) +├── foundation/ # EXISTING: v3.x packages (unchanged) +│ ├── types/ # Will be deprecated in future +│ ├── core/ +│ └── platform-node/ +├── drivers/ # EXISTING: Will be migrated to plugins +└── tools/ # EXISTING: Will be updated +``` + +### 2. New Package: @objectql/types v4.0.0 + +A new types package has been created at `packages/core/types/` with: + +#### Plugin Interfaces + +- **`BasePlugin`** - Base interface for all plugins +- **`QueryProcessorPlugin`** - For query validation, optimization, transformation +- **`RepositoryPlugin`** - For extending repositories with batch ops, audit tracking +- **`PluginMetadata`** - Plugin metadata and dependencies +- **`PluginLifecycle`** - Setup and teardown hooks + +#### Query Types + +- **`UnifiedQuery`** - Core query structure +- **`FilterExpression`** - Type-safe filter expressions +- **`QueryResult`** - Query results with pagination metadata +- **`QueryOptions`** - Query execution options + +#### Runtime Types + +- **`RuntimeContext`** - Context available to plugins +- **`ValidationResult`** - Query validation results +- **`ValidationError`** - Validation error details + +### 3. Type Removals + +The following types have been removed from the new v4.0 package as they are now provided by `@objectstack/spec` or `@objectstack/runtime`: + +#### Removed (Now in @objectstack/spec): +- `Driver` interface → Use `DriverInterface` from `@objectstack/spec` + +#### Removed (Now in @objectstack/runtime): +- `MetadataRegistry` class +- `Context` types +- `Hook` types +- `Action` types + +### 4. Workspace Configuration + +- **pnpm-workspace.yaml**: Added `packages/core/*` and `packages/plugins/*` +- **tsconfig.json**: Added reference to `packages/core/types` + +## Migration Impact + +### For Plugin Developers + +If you're developing plugins, use the new interfaces: + +```typescript +// New v4.0 plugin +import { QueryProcessorPlugin } from '@objectql/types'; + +export function myPlugin(): QueryProcessorPlugin { + return { + name: '@myorg/plugin', + version: '1.0.0', + type: 'query-processor', + + async validateQuery(ast, context) { + return { valid: true, errors: [] }; + }, + + async beforeQuery(ast, context) { + return ast; + } + }; +} +``` + +### For Application Developers + +**No immediate action required.** The existing `@objectql/types` (v3.x) in `packages/foundation/types` remains unchanged and will continue to work. + +The new package at `packages/core/types` is for new plugin development and will be used by migrated packages in future phases. + +## Dependencies + +The new `@objectql/types` v4.0.0 depends on: + +- `@objectstack/spec` ^0.2.0 - Protocol specifications +- `@objectstack/runtime` ^0.2.0 - Runtime types + +These packages provide the foundation for driver interfaces, metadata management, and runtime context. + +## Build Verification + +The new package has been built and verified: + +✅ TypeScript compilation successful +✅ All type definitions generated +✅ No circular dependencies +✅ Strict type checking enabled + +## Next Steps + +### Phase 2: Core Plugin Migration (Week 3-4) + +The following will be migrated in the next phase: + +- Create `@objectql/query-validation` plugin +- Create `@objectql/advanced-repository` plugin +- Extract functionality from `@objectql/core` + +### Phase 3: Driver Migration (Week 5-6) + +Drivers will be migrated to the plugin architecture: + +- `@objectql/driver-sql` +- `@objectql/driver-memory` +- `@objectql/driver-mongo` +- `@objectql/driver-sdk` + +## Documentation + +- **Plugin Architecture**: See [PLUGIN_ARCHITECTURE.md](./PLUGIN_ARCHITECTURE.md) +- **Package Restructuring**: See [PACKAGE_RESTRUCTURING.md](./PACKAGE_RESTRUCTURING.md) +- **Implementation Roadmap**: See [IMPLEMENTATION_ROADMAP.md](./IMPLEMENTATION_ROADMAP.md) +- **Plugin Directory**: See [packages/plugins/README.md](./packages/plugins/README.md) +- **New Types Package**: See [packages/core/types/README.md](./packages/core/types/README.md) + +## Questions or Issues? + +If you encounter any issues with the new type definitions or have questions about plugin development, please open an issue on GitHub. + +--- + +**Phase 1 Status**: ✅ Complete +**Date**: 2026-01-21 +**Next Phase**: Week 3-4 (Core Plugin Migration) diff --git a/packages/plugins/README.md b/packages/plugins/README.md new file mode 100644 index 00000000..1c330355 --- /dev/null +++ b/packages/plugins/README.md @@ -0,0 +1,96 @@ +# ObjectQL Plugins + +This directory contains official ObjectQL plugins that extend the core query functionality. + +## Overview + +ObjectQL v4.0 is built on a plugin-based architecture. Each plugin provides specialized functionality: + +- **Query Processors**: Validate, optimize, and transform queries +- **Repository Extensions**: Add batch operations, audit tracking, soft deletes +- **Feature Plugins**: Formula engines, AI query generation, caching +- **Driver Plugins**: Database adapters (SQL, MongoDB, Redis, etc.) + +## Plugin Categories + +### Query Processing Plugins + +Enhance the query execution pipeline: + +- `query-validation` - Query AST validation +- `query-optimizer` - Query optimization and performance improvements +- `query-cache` - Query result caching + +### Repository Plugins + +Extend repository capabilities: + +- `advanced-repository` - Batch operations, upsert, soft delete, audit tracking + +### Feature Plugins + +Add specialized capabilities: + +- `formula-engine` - Formula parsing and evaluation +- `ai-query-generator` - Natural language to QueryAST conversion + +## Creating a Plugin + +All plugins must implement one of the plugin interfaces from `@objectql/types`: + +```typescript +import { QueryProcessorPlugin } from '@objectql/types'; + +export function myPlugin(): QueryProcessorPlugin { + return { + name: '@objectql/my-plugin', + version: '1.0.0', + type: 'query-processor', + + async setup(runtime) { + // Initialize plugin + }, + + async validateQuery(ast, context) { + // Validate query + return { valid: true, errors: [] }; + }, + + async beforeQuery(ast, context) { + // Transform query before execution + return ast; + } + }; +} +``` + +## Plugin Development + +See [PLUGIN_ARCHITECTURE.md](../../PLUGIN_ARCHITECTURE.md) for detailed documentation on: + +- Plugin types and interfaces +- Lifecycle hooks +- Plugin composition +- Best practices +- Testing strategies + +## Official Plugins + +Plugins will be added as they are migrated from the v3.x architecture: + +- [ ] `@objectql/query-validation` - Week 7-8 +- [ ] `@objectql/advanced-repository` - Week 9-10 +- [ ] `@objectql/formula-engine` - Week 11-12 +- [ ] `@objectql/query-optimizer` - Week 13-14 +- [ ] `@objectql/query-cache` - Week 15-16 +- [ ] `@objectql/ai-query-generator` - Week 17-18 + +See [IMPLEMENTATION_ROADMAP.md](../../IMPLEMENTATION_ROADMAP.md) for the full migration timeline. + +## Community Plugins + +Community-developed plugins will be listed here as they become available. + +## License + +MIT © ObjectStack Inc. From c9dabefa826c9f37d5a1ac2309d61fb0d56ba793 Mon Sep 17 00:00:00 2001 From: "copilot-swe-agent[bot]" <198982749+Copilot@users.noreply.github.com> Date: Wed, 21 Jan 2026 12:51:37 +0000 Subject: [PATCH 9/9] =?UTF-8?q?Add=20comprehensive=20protocol=20improvemen?= =?UTF-8?q?ts=20plan=20(=E5=8D=8F=E8=AE=AE=E6=94=B9=E8=BF=9B=E8=AE=A1?= =?UTF-8?q?=E5=88=92)?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: hotlong <50353452+hotlong@users.noreply.github.com> --- PROTOCOL_IMPROVEMENTS.md | 1143 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 1143 insertions(+) create mode 100644 PROTOCOL_IMPROVEMENTS.md diff --git a/PROTOCOL_IMPROVEMENTS.md b/PROTOCOL_IMPROVEMENTS.md new file mode 100644 index 00000000..c89b4599 --- /dev/null +++ b/PROTOCOL_IMPROVEMENTS.md @@ -0,0 +1,1143 @@ +# ObjectQL 协议改进计划 (Protocol Improvements Plan) + +## 文档概述 (Document Overview) + +本文档专门分析 ObjectQL 迁移至 @objectstack/runtime 架构过程中，协议层面需要改进的内容，并制定详细的工作计划。 + +**Protocol improvements needed for ObjectQL migration to @objectstack/runtime architecture, with detailed work plan.** + +--- + +## 一、当前协议状态分析 (Current Protocol State Analysis) + +### 1.1 现有协议体系 (Existing Protocol System) + +ObjectQL v3.x 当前使用以下协议： + +| 协议类型 (Protocol Type) | 来源 (Source) | 版本 (Version) | 使用状态 (Status) | +|-------------------------|--------------|----------------|-------------------| +| **DriverInterface** | @objectstack/spec | 0.1.2 | ✅ 已集成 (Integrated) | +| **QueryAST** | @objectstack/spec | 0.1.2 | ⚠️ 类型导出，未完全使用 (Type export only) | +| **RuntimeProtocol** | @objectstack/runtime | 0.1.1 | ❌ 未使用 (Not used) | +| **ObjectQLEngine** | @objectstack/objectql | 0.1.1 | ❌ 未使用 (Not used) | +| **内部 Query 协议** | @objectql/types | 3.0.1 | ✅ 当前使用 (Currently used) | + +### 1.2 协议问题识别 (Protocol Issues Identified) + +#### 问题 1: QueryAST 协议不统一 (Inconsistent QueryAST Protocol) + +**现状 (Current State):** +- ObjectQL 内部使用自定义的 Query 结构 +- @objectstack/spec 定义了标准 QueryAST +- 两者之间缺乏对齐 + +**影响 (Impact):** +- 驱动之间不可移植 (Drivers not portable) +- 插件无法跨项目共享 (Plugins not shareable) +- 增加维护成本 (Increased maintenance cost) + +#### 问题 2: DriverInterface 实现不完整 (Incomplete DriverInterface Implementation) + +**现状 (Current State):** +- 仅 8 个驱动中的 4 个完全实现了 @objectstack/spec.DriverInterface +- 部分驱动仍使用旧的 Driver 接口 +- 缺少统一的错误处理协议 + +**影响 (Impact):** +- 驱动行为不一致 (Inconsistent driver behavior) +- 错误处理不标准化 (Non-standard error handling) +- 难以实现跨驱动功能 (Cross-driver features difficult) + +#### 问题 3: 缺少插件通信协议 (Missing Plugin Communication Protocol) + +**现状 (Current State):** +- 插件之间没有标准化的通信机制 +- 事件系统未统一 +- 缺少插件间数据共享协议 + +**影响 (Impact):** +- 插件隔离，无法协作 (Plugins isolated, cannot collaborate) +- 功能组合受限 (Limited feature composition) +- 重复实现相同逻辑 (Duplicate implementations) + +#### 问题 4: 元数据协议版本化缺失 (Missing Metadata Protocol Versioning) + +**现状 (Current State):** +- .object.yml, .validation.yml 等文件没有版本标识 +- 协议变更无法追踪 +- 向后兼容性难以保证 + +**影响 (Impact):** +- 升级困难 (Difficult upgrades) +- 破坏性变更无预警 (Breaking changes without warning) +- 工具链兼容性问题 (Tooling compatibility issues) + +#### 问题 5: API 协议不完整 (Incomplete API Protocol) + +**现状 (Current State):** +- REST API 缺少规范化的错误响应格式 +- GraphQL 协议未完全定义 +- 缺少 API 版本控制机制 + +**影响 (Impact):** +- 客户端集成困难 (Client integration difficult) +- API 演进受阻 (API evolution blocked) +- 文档与实现不一致 (Documentation-implementation mismatch) + +--- + +## 二、协议改进需求 (Protocol Improvement Requirements) + +### 2.1 QueryAST 协议标准化 (QueryAST Protocol Standardization) + +#### 目标 (Objective) +完全对齐 @objectstack/spec.QueryAST，实现查询协议的标准化。 + +#### 改进内容 (Improvements) + +**1. 统一查询结构 (Unified Query Structure)** + +**当前 (Current):** +```typescript +// ObjectQL 内部查询结构 +interface InternalQuery { + filters?: any; + fields?: string[]; + sort?: any; + limit?: number; + skip?: number; +} +``` + +**目标 (Target):** +```typescript +// 使用标准 QueryAST +import { QueryAST } from '@objectstack/spec'; + +interface UnifiedQuery extends QueryAST { + // 扩展字段（如果需要） + // ObjectQL-specific extensions (if needed) +} +``` + +**2. 过滤器表达式标准化 (Standardized Filter Expressions)** + +**增强 (Enhancements):** +- 支持复杂逻辑运算符 (Complex logical operators) +- 标准化字段引用语法 (Standardized field reference syntax) +- 类型安全的值比较 (Type-safe value comparisons) +- 子查询支持 (Subquery support) + +**示例 (Example):** +```typescript +{ + filters: { + $and: [ + { status: { $in: ['active', 'pending'] } }, + { createdAt: { $gte: '2026-01-01' } }, + { + $or: [ + { priority: { $eq: 'high' } }, + { assignee: { $eq: 'user123' } } + ] + } + ] + } +} +``` + +**3. 聚合查询协议 (Aggregation Query Protocol)** + +**新增 (New):** +```typescript +interface AggregationAST extends QueryAST { + aggregations: { + groupBy?: string[]; + metrics?: { + [key: string]: { + function: 'sum' | 'avg' | 'count' | 'min' | 'max'; + field: string; + }; + }; + having?: FilterExpression; + }; +} +``` + +### 2.2 DriverInterface 协议完善 (DriverInterface Protocol Enhancement) + +#### 目标 (Objective) +所有驱动完全符合 @objectstack/spec.DriverInterface，并扩展必要的功能。 + +#### 改进内容 (Improvements) + +**1. 标准化错误协议 (Standardized Error Protocol)** + +```typescript +interface DriverError { + code: string; // 标准错误码 (Standard error code) + message: string; // 错误消息 (Error message) + driver: string; // 驱动名称 (Driver name) + operation: string; // 操作类型 (Operation type) + details?: any; // 详细信息 (Details) + retryable: boolean; // 是否可重试 (Retryable) + timestamp: Date; // 时间戳 (Timestamp) +} + +// 标准错误码 (Standard error codes) +enum DriverErrorCode { + CONNECTION_FAILED = 'DRIVER_CONNECTION_FAILED', + QUERY_FAILED = 'DRIVER_QUERY_FAILED', + VALIDATION_FAILED = 'DRIVER_VALIDATION_FAILED', + TIMEOUT = 'DRIVER_TIMEOUT', + PERMISSION_DENIED = 'DRIVER_PERMISSION_DENIED', + RESOURCE_NOT_FOUND = 'DRIVER_RESOURCE_NOT_FOUND', + CONFLICT = 'DRIVER_CONFLICT', + TRANSACTION_FAILED = 'DRIVER_TRANSACTION_FAILED' +} +``` + +**2. 事务协议 (Transaction Protocol)** + +```typescript +interface TransactionProtocol { + begin(options?: TransactionOptions): Promise; + commit(transaction: Transaction): Promise; + rollback(transaction: Transaction): Promise; + + // 嵌套事务支持 (Nested transaction support) + savepoint(transaction: Transaction, name: string): Promise; + rollbackToSavepoint(transaction: Transaction, name: string): Promise; +} + +interface TransactionOptions { + isolationLevel?: 'READ_UNCOMMITTED' | 'READ_COMMITTED' | 'REPEATABLE_READ' | 'SERIALIZABLE'; + timeout?: number; // 毫秒 (milliseconds) + readOnly?: boolean; +} +``` + +**3. 批量操作协议 (Batch Operations Protocol)** + +```typescript +interface BatchProtocol { + batchCreate( + object: string, + records: any[], + options?: BatchOptions + ): Promise; + + batchUpdate( + object: string, + updates: Array<{ id: string; data: any }>, + options?: BatchOptions + ): Promise; + + batchDelete( + object: string, + ids: string[], + options?: BatchOptions + ): Promise; +} + +interface BatchResult { + successful: number; // 成功数量 (Successful count) + failed: number; // 失败数量 (Failed count) + errors: Array<{ // 错误详情 (Error details) + index: number; + error: DriverError; + }>; + results: any[]; // 结果集 (Results) +} +``` + +**4. 流式查询协议 (Streaming Query Protocol)** + +```typescript +interface StreamingProtocol { + findStream( + object: string, + query: QueryAST, + options?: StreamOptions + ): AsyncIterable; + + // 游标支持 (Cursor support) + createCursor( + object: string, + query: QueryAST, + options?: CursorOptions + ): Promise; +} + +interface StreamOptions { + batchSize?: number; // 批次大小 (Batch size) + highWaterMark?: number; // 高水位标记 (High water mark) +} +``` + +### 2.3 插件通信协议 (Plugin Communication Protocol) + +#### 目标 (Objective) +建立标准化的插件间通信机制。 + +#### 改进内容 (Improvements) + +**1. 事件总线协议 (Event Bus Protocol)** + +```typescript +interface PluginEventBus { + // 发布事件 (Publish event) + emit(event: PluginEvent): Promise; + + // 订阅事件 (Subscribe to event) + on(eventType: string, handler: EventHandler): void; + + // 一次性订阅 (One-time subscription) + once(eventType: string, handler: EventHandler): void; + + // 取消订阅 (Unsubscribe) + off(eventType: string, handler: EventHandler): void; +} + +interface PluginEvent { + type: string; // 事件类型 (Event type) + source: string; // 来源插件 (Source plugin) + timestamp: Date; // 时间戳 (Timestamp) + data: any; // 事件数据 (Event data) + metadata?: { // 元数据 (Metadata) + correlationId?: string; + userId?: string; + traceId?: string; + }; +} + +// 标准事件类型 (Standard event types) +enum PluginEventType { + // 查询生命周期 (Query lifecycle) + QUERY_BEFORE = 'query.before', + QUERY_AFTER = 'query.after', + QUERY_ERROR = 'query.error', + + // 数据变更 (Data changes) + DATA_CREATED = 'data.created', + DATA_UPDATED = 'data.updated', + DATA_DELETED = 'data.deleted', + + // 验证 (Validation) + VALIDATION_BEFORE = 'validation.before', + VALIDATION_AFTER = 'validation.after', + + // 缓存 (Cache) + CACHE_HIT = 'cache.hit', + CACHE_MISS = 'cache.miss', + CACHE_INVALIDATE = 'cache.invalidate' +} +``` + +**2. 插件间数据共享协议 (Plugin Data Sharing Protocol)** + +```typescript +interface PluginContext { + // 设置共享数据 (Set shared data) + set(key: string, value: any, options?: ContextOptions): void; + + // 获取共享数据 (Get shared data) + get(key: string): T | undefined; + + // 检查是否存在 (Check existence) + has(key: string): boolean; + + // 删除数据 (Delete data) + delete(key: string): void; + + // 清空上下文 (Clear context) + clear(): void; +} + +interface ContextOptions { + scope?: 'request' | 'session' | 'global'; + ttl?: number; // 生存时间（毫秒）(Time to live in ms) + plugin?: string; // 所属插件 (Owning plugin) +} +``` + +**3. 插件依赖协议 (Plugin Dependency Protocol)** + +```typescript +interface PluginDependency { + name: string; // 依赖的插件名称 (Required plugin name) + version?: string; // 版本要求 (Version requirement) + optional?: boolean; // 是否可选 (Is optional) +} + +interface PluginMetadata { + name: string; + version: string; + dependencies: PluginDependency[]; + provides: string[]; // 提供的功能 (Provided features) + requires: string[]; // 需要的功能 (Required features) +} +``` + +### 2.4 元数据协议版本化 (Metadata Protocol Versioning) + +#### 目标 (Objective) +为所有元数据文件添加版本控制和演进机制。 + +#### 改进内容 (Improvements) + +**1. 元数据文件版本标识 (Metadata File Versioning)** + +```yaml +# object.object.yml +$schema: "https://objectql.org/schemas/object/v4.0.0" +version: "4.0.0" +name: project +fields: + # ... +``` + +**2. 向后兼容性协议 (Backward Compatibility Protocol)** + +```typescript +interface MetadataVersioning { + // 当前支持的版本 (Currently supported versions) + supportedVersions: string[]; + + // 版本迁移器 (Version migrator) + migrate( + metadata: any, + fromVersion: string, + toVersion: string + ): Promise; + + // 版本验证 (Version validation) + validateVersion(metadata: any): ValidationResult; +} + +// 版本策略 (Versioning strategy) +enum VersionStrategy { + STRICT = 'strict', // 严格模式：只接受指定版本 + COMPATIBLE = 'compatible', // 兼容模式：接受兼容版本 + MIGRATE = 'migrate' // 迁移模式：自动迁移旧版本 +} +``` + +**3. 模式注册表协议 (Schema Registry Protocol)** + +```typescript +interface SchemaRegistry { + // 注册模式 (Register schema) + register(type: string, version: string, schema: JSONSchema): void; + + // 获取模式 (Get schema) + getSchema(type: string, version: string): JSONSchema | undefined; + + // 验证数据 (Validate data) + validate(type: string, version: string, data: any): ValidationResult; + + // 列出所有模式 (List all schemas) + listSchemas(): SchemaInfo[]; +} + +interface SchemaInfo { + type: string; // object, validation, permission, etc. + version: string; + url: string; // Schema URL + deprecated?: boolean; + replacedBy?: string; // 替代版本 (Replacement version) +} +``` + +### 2.5 API 协议标准化 (API Protocol Standardization) + +#### 目标 (Objective) +标准化 REST 和 GraphQL API 协议。 + +#### 改进内容 (Improvements) + +**1. REST API 响应格式 (REST API Response Format)** + +```typescript +// 成功响应 (Success response) +interface APISuccessResponse { + success: true; + data: T; + metadata?: { + total?: number; + page?: number; + pageSize?: number; + hasMore?: boolean; + }; + timestamp: string; +} + +// 错误响应 (Error response) +interface APIErrorResponse { + success: false; + error: { + code: string; // 错误码 (Error code) + message: string; // 错误消息 (Error message) + details?: any; // 详细信息 (Details) + path?: string; // 错误路径 (Error path) + timestamp: string; // 时间戳 (Timestamp) + requestId?: string; // 请求ID (Request ID) + }; +} + +// 批量操作响应 (Batch operation response) +interface APIBatchResponse { + success: boolean; + results: Array<{ + success: boolean; + data?: T; + error?: { + code: string; + message: string; + }; + }>; + summary: { + total: number; + successful: number; + failed: number; + }; +} +``` + +**2. API 版本控制协议 (API Versioning Protocol)** + +```typescript +// URL 路径版本 (URL path versioning) +// /api/v1/objects/users +// /api/v2/objects/users + +// 请求头版本 (Header versioning) +// Accept: application/vnd.objectql.v1+json + +interface APIVersion { + version: string; // 版本号 (Version number) + deprecated?: boolean; // 是否废弃 (Is deprecated) + sunset?: Date; // 废弃日期 (Sunset date) + replacedBy?: string; // 替代版本 (Replacement version) +} +``` + +**3. GraphQL 协议增强 (GraphQL Protocol Enhancement)** + +```graphql +# 统一的错误类型 (Unified error type) +type Error { + code: String! + message: String! + path: [String!] + extensions: JSON +} + +# 分页协议 (Pagination protocol) +interface Connection { + edges: [Edge!]! + pageInfo: PageInfo! + totalCount: Int +} + +type PageInfo { + hasNextPage: Boolean! + hasPreviousPage: Boolean! + startCursor: String + endCursor: String +} + +# 变更响应 (Mutation response) +type MutationResult { + success: Boolean! + data: JSON + errors: [Error!] +} +``` + +--- + +## 三、工作计划 (Work Plan) + +### 3.1 总体时间安排 (Overall Timeline) + +| 阶段 (Phase) | 周数 (Weeks) | 工作量 (Hours) | 优先级 (Priority) | +|-------------|--------------|----------------|-------------------| +| Phase 1: QueryAST 标准化 | 2 | 80 | P0 - 关键 (Critical) | +| Phase 2: DriverInterface 完善 | 3 | 120 | P0 - 关键 (Critical) | +| Phase 3: 插件通信协议 | 2 | 80 | P1 - 高 (High) | +| Phase 4: 元数据版本化 | 2 | 80 | P1 - 高 (High) | +| Phase 5: API 协议标准化 | 2 | 80 | P2 - 中 (Medium) | +| Phase 6: 测试与文档 | 2 | 80 | P0 - 关键 (Critical) | +| **总计 (Total)** | **13** | **520** | - | + +### 3.2 Phase 1: QueryAST 标准化 (Weeks 1-2) + +#### 目标 (Objectives) +- 完全对齐 @objectstack/spec.QueryAST +- 统一查询过滤器语法 +- 实现聚合查询协议 + +#### 详细任务 (Detailed Tasks) + +**Week 1:** + +**1.1 QueryAST 类型定义 (16 hours)** +- [ ] 分析 @objectstack/spec.QueryAST 完整定义 +- [ ] 对比当前 ObjectQL 内部查询结构 +- [ ] 设计迁移路径和兼容层 +- [ ] 更新 @objectql/types 中的查询类型定义 + +**1.2 过滤器表达式实现 (24 hours)** +- [ ] 实现标准逻辑运算符 ($and, $or, $not) +- [ ] 实现比较运算符 ($eq, $ne, $gt, $gte, $lt, $lte) +- [ ] 实现集合运算符 ($in, $nin, $all) +- [ ] 实现字符串运算符 ($regex, $text) +- [ ] 实现存在性运算符 ($exists, $type) +- [ ] 添加类型验证和错误处理 + +**Week 2:** + +**2.1 聚合查询协议 (16 hours)** +- [ ] 设计 AggregationAST 接口 +- [ ] 实现 groupBy 功能 +- [ ] 实现聚合函数 (sum, avg, count, min, max) +- [ ] 实现 having 子句 +- [ ] 添加聚合查询测试 + +**2.2 驱动适配 (24 hours)** +- [ ] 更新 SQL 驱动以支持新 QueryAST +- [ ] 更新 MongoDB 驱动以支持新 QueryAST +- [ ] 更新 Memory 驱动以支持新 QueryAST +- [ ] 更新其他驱动 +- [ ] 添加兼容性测试 + +**交付物 (Deliverables):** +- ✅ @objectql/types v4.0.0 with unified QueryAST +- ✅ All drivers supporting new QueryAST +- ✅ Comprehensive test coverage (>90%) +- ✅ Migration guide for query syntax changes + +### 3.3 Phase 2: DriverInterface 完善 (Weeks 3-5) + +#### 目标 (Objectives) +- 所有驱动符合 @objectstack/spec.DriverInterface +- 实现标准化错误协议 +- 添加事务和批量操作支持 + +#### 详细任务 (Detailed Tasks) + +**Week 3: 错误协议标准化** + +**3.1 错误类型定义 (16 hours)** +- [ ] 定义 DriverError 接口 +- [ ] 定义标准错误码枚举 +- [ ] 实现错误工厂函数 +- [ ] 添加错误序列化/反序列化 +- [ ] 编写错误处理指南 + +**3.2 驱动错误迁移 (24 hours)** +- [ ] 更新 SQL 驱动错误处理 +- [ ] 更新 MongoDB 驱动错误处理 +- [ ] 更新 Memory 驱动错误处理 +- [ ] 更新其他驱动错误处理 +- [ ] 添加错误处理测试 + +**Week 4: 事务协议** + +**4.1 事务接口实现 (24 hours)** +- [ ] 定义 TransactionProtocol 接口 +- [ ] 在 SQL 驱动中实现事务支持 +- [ ] 在 MongoDB 驱动中实现会话支持 +- [ ] 实现嵌套事务/保存点 +- [ ] 添加事务隔离级别支持 + +**4.2 事务测试 (16 hours)** +- [ ] 编写事务提交测试 +- [ ] 编写事务回滚测试 +- [ ] 编写嵌套事务测试 +- [ ] 编写并发事务测试 +- [ ] 编写事务超时测试 + +**Week 5: 批量操作和流式查询** + +**5.1 批量操作协议 (20 hours)** +- [ ] 定义 BatchProtocol 接口 +- [ ] 实现批量创建 +- [ ] 实现批量更新 +- [ ] 实现批量删除 +- [ ] 添加批量操作优化 + +**5.2 流式查询协议 (20 hours)** +- [ ] 定义 StreamingProtocol 接口 +- [ ] 实现流式查询（SQL 驱动） +- [ ] 实现游标支持 +- [ ] 添加背压处理 +- [ ] 性能测试和优化 + +**交付物 (Deliverables):** +- ✅ Standardized error protocol across all drivers +- ✅ Transaction support in SQL and MongoDB drivers +- ✅ Batch operations in all drivers +- ✅ Streaming query support +- ✅ Driver compatibility test suite + +### 3.4 Phase 3: 插件通信协议 (Weeks 6-7) + +#### 目标 (Objectives) +- 实现事件总线系统 +- 建立插件间数据共享机制 +- 定义插件依赖管理协议 + +#### 详细任务 (Detailed Tasks) + +**Week 6: 事件总线** + +**6.1 事件总线实现 (24 hours)** +- [ ] 设计事件总线架构 +- [ ] 实现事件发布/订阅机制 +- [ ] 实现事件过滤和路由 +- [ ] 添加异步事件处理 +- [ ] 实现事件优先级队列 + +**6.2 标准事件类型 (16 hours)** +- [ ] 定义查询生命周期事件 +- [ ] 定义数据变更事件 +- [ ] 定义验证事件 +- [ ] 定义缓存事件 +- [ ] 编写事件使用指南 + +**Week 7: 插件上下文和依赖** + +**7.1 插件上下文实现 (20 hours)** +- [ ] 实现 PluginContext 接口 +- [ ] 添加作用域管理（request/session/global） +- [ ] 实现 TTL 和清理机制 +- [ ] 添加上下文隔离 +- [ ] 性能优化 + +**7.2 依赖管理协议 (20 hours)** +- [ ] 定义插件元数据格式 +- [ ] 实现依赖解析算法 +- [ ] 添加版本兼容性检查 +- [ ] 实现可选依赖处理 +- [ ] 创建依赖图可视化工具 + +**交付物 (Deliverables):** +- ✅ Event bus implementation +- ✅ Plugin context system +- ✅ Dependency management protocol +- ✅ Inter-plugin communication examples +- ✅ Plugin development best practices guide + +### 3.5 Phase 4: 元数据版本化 (Weeks 8-9) + +#### 目标 (Objectives) +- 为所有元数据文件添加版本标识 +- 实现版本迁移机制 +- 建立模式注册表 + +#### 详细任务 (Detailed Tasks) + +**Week 8: 版本控制** + +**8.1 元数据版本标识 (16 hours)** +- [ ] 更新 .object.yml schema 添加版本字段 +- [ ] 更新 .validation.yml schema 添加版本字段 +- [ ] 更新 .permission.yml schema 添加版本字段 +- [ ] 更新其他元数据文件 schema +- [ ] 更新 VSCode 扩展以支持版本提示 + +**8.2 版本验证 (24 hours)** +- [ ] 实现版本验证器 +- [ ] 添加严格/兼容/迁移模式 +- [ ] 实现版本比较逻辑 +- [ ] 添加版本废弃警告 +- [ ] 编写版本验证测试 + +**Week 9: 迁移和注册表** + +**9.1 版本迁移器 (24 hours)** +- [ ] 设计迁移框架 +- [ ] 实现 v3.x -> v4.0 迁移器 +- [ ] 添加迁移测试用例 +- [ ] 创建迁移验证工具 +- [ ] 编写迁移指南 + +**9.2 模式注册表 (16 hours)** +- [ ] 实现 SchemaRegistry 接口 +- [ ] 创建在线模式仓库 +- [ ] 实现模式缓存机制 +- [ ] 添加模式版本追踪 +- [ ] 集成到 CLI 工具 + +**交付物 (Deliverables):** +- ✅ Versioned metadata schemas +- ✅ Automatic migration tool +- ✅ Schema registry service +- ✅ Version validation in all tools +- ✅ Migration documentation + +### 3.6 Phase 5: API 协议标准化 (Weeks 10-11) + +#### 目标 (Objectives) +- 标准化 REST API 响应格式 +- 实现 API 版本控制 +- 增强 GraphQL 协议 + +#### 详细任务 (Detailed Tasks) + +**Week 10: REST API** + +**10.1 响应格式标准化 (20 hours)** +- [ ] 定义统一响应接口 +- [ ] 实现成功响应格式化 +- [ ] 实现错误响应格式化 +- [ ] 实现批量操作响应 +- [ ] 更新 @objectql/server + +**10.2 API 版本控制 (20 hours)** +- [ ] 设计版本控制策略 +- [ ] 实现 URL 路径版本 +- [ ] 实现请求头版本 +- [ ] 添加版本废弃机制 +- [ ] 创建 API 版本文档 + +**Week 11: GraphQL** + +**11.1 GraphQL 协议增强 (24 hours)** +- [ ] 标准化错误类型 +- [ ] 实现 Connection 分页协议 +- [ ] 实现统一的 Mutation 响应 +- [ ] 添加订阅（Subscription）支持 +- [ ] 性能优化（DataLoader） + +**11.2 GraphQL 工具 (16 hours)** +- [ ] 生成 GraphQL Schema +- [ ] 实现 GraphQL Playground +- [ ] 添加 GraphQL 查询复杂度分析 +- [ ] 实现查询深度限制 +- [ ] 编写 GraphQL 使用指南 + +**交付物 (Deliverables):** +- ✅ Standardized REST API format +- ✅ API versioning implementation +- ✅ Enhanced GraphQL protocol +- ✅ API documentation (OpenAPI/Swagger) +- ✅ GraphQL schema and playground + +### 3.7 Phase 6: 测试与文档 (Weeks 12-13) + +#### 目标 (Objectives) +- 全面测试所有协议改进 +- 编写完整的协议文档 +- 创建迁移指南和示例 + +#### 详细任务 (Detailed Tasks) + +**Week 12: 测试** + +**12.1 协议兼容性测试 (24 hours)** +- [ ] QueryAST 兼容性测试 +- [ ] DriverInterface 兼容性测试 +- [ ] 插件通信协议测试 +- [ ] API 协议测试 +- [ ] 跨版本兼容性测试 + +**12.2 集成测试 (16 hours)** +- [ ] 端到端场景测试 +- [ ] 多驱动集成测试 +- [ ] 插件组合测试 +- [ ] 性能基准测试 +- [ ] 负载测试 + +**Week 13: 文档** + +**13.1 协议规范文档 (24 hours)** +- [ ] QueryAST 协议规范 +- [ ] DriverInterface 协议规范 +- [ ] 插件通信协议规范 +- [ ] 元数据协议规范 +- [ ] API 协议规范 + +**13.2 开发者指南 (16 hours)** +- [ ] 协议迁移指南 +- [ ] 驱动开发指南 +- [ ] 插件开发指南 +- [ ] API 集成指南 +- [ ] 示例代码和最佳实践 + +**交付物 (Deliverables):** +- ✅ Comprehensive test suite (>95% coverage) +- ✅ Complete protocol specifications +- ✅ Developer guides and tutorials +- ✅ Migration tools and scripts +- ✅ Example implementations + +--- + +## 四、资源需求 (Resource Requirements) + +### 4.1 人力资源 (Human Resources) + +| 角色 (Role) | 工作量 (Effort) | 技能要求 (Skills Required) | +|------------|----------------|---------------------------| +| **协议架构师** (Protocol Architect) | 40% (全程) | TypeScript, 协议设计, 系统架构 | +| **驱动开发工程师** (Driver Developer) | 80% (Week 3-5) | SQL, MongoDB, TypeScript | +| **插件系统工程师** (Plugin Engineer) | 80% (Week 6-7) | 事件系统, 依赖管理 | +| **API 开发工程师** (API Developer) | 80% (Week 10-11) | REST, GraphQL, OpenAPI | +| **测试工程师** (QA Engineer) | 60% (Week 12-13) | 测试框架, 性能测试 | +| **技术文档工程师** (Tech Writer) | 40% (Week 13) | 技术写作, 示例代码 | + +**推荐团队配置 (Recommended Team):** +- 2 名高级工程师（全职 13 周） +- 1 名测试工程师（兼职，Week 12-13） +- 1 名技术文档工程师（兼职，Week 13） + +### 4.2 时间预算 (Time Budget) + +| Phase | 计划工作量 (Hours) | 实际预留 (Hours) | 缓冲 (Buffer) | +|-------|-------------------|------------------|--------------| +| Phase 1 | 80 | 88 | 10% | +| Phase 2 | 120 | 132 | 10% | +| Phase 3 | 80 | 88 | 10% | +| Phase 4 | 80 | 88 | 10% | +| Phase 5 | 80 | 88 | 10% | +| Phase 6 | 80 | 88 | 10% | +| **总计** | **520** | **572** | **10%** | + +### 4.3 成本估算 (Cost Estimation) + +假设高级工程师平均成本 $100/小时： + +| 项目 (Item) | 工作量 (Hours) | 成本 (Cost) | +|------------|----------------|------------| +| 核心开发 (Core Development) | 520 | $52,000 | +| 缓冲时间 (Buffer) | 52 | $5,200 | +| 测试与QA (Testing & QA) | 40 | $4,000 | +| 文档编写 (Documentation) | 32 | $3,200 | +| **总计 (Total)** | **644** | **$64,400** | + +--- + +## 五、风险与缓解 (Risks and Mitigation) + +### 5.1 技术风险 (Technical Risks) + +#### 风险 1: 协议变更导致破坏性影响 +**可能性 (Likelihood):** 高 (High) +**影响 (Impact):** 高 (High) +**缓解措施 (Mitigation):** +- 提供完整的兼容层 +- 实现自动迁移工具 +- 分阶段发布（Alpha → Beta → RC → GA） +- 保持 v3.x 维护 6 个月 + +#### 风险 2: 性能回退 +**可能性 (Likelihood):** 中 (Medium) +**影响 (Impact):** 高 (High) +**缓解措施 (Mitigation):** +- 持续性能基准测试 +- 在每个 Phase 进行性能验证 +- 性能目标：<5% 开销 +- 性能优化专项时间预留 + +#### 风险 3: 第三方驱动不兼容 +**可能性 (Likelihood):** 中 (Medium) +**影响 (Impact):** 中 (Medium) +**缓解措施 (Mitigation):** +- 提供驱动迁移指南 +- 创建驱动适配器工具 +- 社区支持计划 +- 官方驱动优先迁移示例 + +### 5.2 项目风险 (Project Risks) + +#### 风险 4: 时间延期 +**可能性 (Likelihood):** 中 (Medium) +**影响 (Impact):** 中 (Medium) +**缓解措施 (Mitigation):** +- 10% 时间缓冲 +- 每周进度评审 +- 关键路径监控 +- 调整优先级机制 + +#### 风险 5: 资源不足 +**可能性 (Likelihood):** 低 (Low) +**影响 (Impact):** 高 (High) +**缓解措施 (Mitigation):** +- 提前确认资源承诺 +- 关键角色备份计划 +- 外部顾问储备 +- 灵活的 Phase 排期 + +--- + +## 六、质量保证 (Quality Assurance) + +### 6.1 测试策略 (Testing Strategy) + +| 测试类型 (Test Type) | 覆盖率目标 (Coverage Target) | 工具 (Tools) | +|---------------------|----------------------------|-------------| +| 单元测试 (Unit Tests) | >90% | Jest, Vitest | +| 集成测试 (Integration Tests) | >80% | Jest, Supertest | +| 协议兼容性测试 (Protocol Compatibility) | 100% | 自定义测试套件 | +| 性能测试 (Performance Tests) | 关键路径 (Critical Paths) | Benchmark.js | +| 端到端测试 (E2E Tests) | 主要场景 (Main Scenarios) | Playwright | + +### 6.2 质量门禁 (Quality Gates) + +每个 Phase 结束必须通过： + +- ✅ 所有单元测试通过 +- ✅ 代码覆盖率 >90% +- ✅ 零高危安全漏洞 +- ✅ 性能基准测试通过（<5% 回退） +- ✅ TypeScript 类型检查通过 +- ✅ Lint 检查通过 +- ✅ 协议规范文档完成 + +### 6.3 代码审查标准 (Code Review Standards) + +所有协议相关代码必须： + +- 至少 2 名工程师审查 +- 包含完整的 TypeScript 类型定义 +- 包含 JSDoc 注释 +- 包含使用示例 +- 通过自动化测试 +- 符合协议规范 + +--- + +## 七、成功指标 (Success Metrics) + +### 7.1 技术指标 (Technical Metrics) + +| 指标 (Metric) | 目标 (Target) | 测量方法 (Measurement) | +|--------------|--------------|----------------------| +| 协议一致性 (Protocol Consistency) | 100% | 所有驱动符合 DriverInterface | +| 测试覆盖率 (Test Coverage) | >90% | Jest/Vitest 报告 | +| 性能开销 (Performance Overhead) | <5% | 基准测试对比 | +| 文档完整性 (Documentation Completeness) | 100% | 所有协议有规范文档 | +| 破坏性变更 (Breaking Changes) | 0 (with compat layer) | 兼容性测试 | + +### 7.2 用户指标 (User Metrics) + +| 指标 (Metric) | 目标 (Target) | 测量方法 (Measurement) | +|--------------|--------------|----------------------| +| 迁移成功率 (Migration Success Rate) | >95% | 用户反馈 | +| API 错误率 (API Error Rate) | <1% | 生产监控 | +| 开发者满意度 (Developer Satisfaction) | >4.5/5 | 问卷调查 | +| 文档有用性 (Documentation Usefulness) | >4.0/5 | 文档反馈 | + +--- + +## 八、后续行动 (Next Actions) + +### 立即行动 (Immediate Actions - Week 0) + +**1. 利益相关者审批 (Stakeholder Approval)** +- [ ] 技术负责人审查协议改进计划 +- [ ] 产品负责人批准时间和资源 +- [ ] 社区征求意见（GitHub Discussion） + +**2. 团队准备 (Team Preparation)** +- [ ] 分配角色和责任 +- [ ] 设置项目跟踪系统（Jira/Linear） +- [ ] 创建技术设计文档模板 + +**3. 环境准备 (Environment Setup)** +- [ ] 创建 protocol-improvements 分支 +- [ ] 设置 CI/CD 流水线 +- [ ] 准备测试数据集 + +### 第一周启动 (Week 1 Kickoff) + +**Day 1-2: QueryAST 分析** +- 深入研究 @objectstack/spec.QueryAST +- 对比分析现有实现 +- 确定迁移策略 + +**Day 3-4: 类型定义** +- 更新 @objectql/types +- 创建兼容层 +- 编写初步测试 + +**Day 5: 审查和调整** +- 团队代码审查 +- 调整设计（如需要） +- 准备下周任务 + +--- + +## 九、参考资料 (References) + +### 相关文档 (Related Documents) + +1. **MIGRATION_STRATEGY.md** - 整体迁移策略 +2. **PLUGIN_ARCHITECTURE.md** - 插件架构设计 +3. **FEATURE_MIGRATION_MATRIX.md** - 功能迁移矩阵 +4. **RUNTIME_INTEGRATION.md** - Runtime 集成文档 + +### 外部规范 (External Specifications) + +1. **@objectstack/spec** - ObjectStack 协议规范 + - https://github.com/objectstack-ai/spec + +2. **JSON Schema** - 元数据模式定义 + - https://json-schema.org/ + +3. **OpenAPI Specification** - REST API 规范 + - https://swagger.io/specification/ + +4. **GraphQL Specification** - GraphQL 协议 + - https://spec.graphql.org/ + +### 最佳实践 (Best Practices) + +1. **Semantic Versioning** - 版本控制 + - https://semver.org/ + +2. **JSON:API** - API 设计规范 + - https://jsonapi.org/ + +3. **Relay Cursor Connections** - GraphQL 分页 + - https://relay.dev/graphql/connections.htm + +--- + +## 十、总结 (Summary) + +本协议改进计划涵盖了 ObjectQL 迁移至 @objectstack/runtime 架构所需的所有协议层面改进： + +### 核心改进 (Core Improvements) + +1. **QueryAST 标准化** - 统一查询协议，支持复杂查询和聚合 +2. **DriverInterface 完善** - 标准化错误、事务、批量操作和流式查询 +3. **插件通信协议** - 事件总线、上下文共享、依赖管理 +4. **元数据版本化** - 版本控制、自动迁移、模式注册表 +5. **API 协议标准化** - REST 和 GraphQL 协议规范化 + +### 时间和资源 (Time and Resources) + +- **总时长 (Duration):** 13 周 +- **工作量 (Effort):** 520 小时（含缓冲 572 小时） +- **团队 (Team):** 2 名高级工程师 + 支持人员 +- **预算 (Budget):** ~$64,400 + +### 预期成果 (Expected Outcomes) + +- ✅ 100% 协议一致性 +- ✅ >90% 测试覆盖率 +- ✅ <5% 性能开销 +- ✅ 完整的协议文档和指南 +- ✅ 平滑的迁移路径 + +--- + +**文档版本 (Document Version):** 1.0.0 +**创建日期 (Created):** 2026-01-21 +**最后更新 (Last Updated):** 2026-01-21 +**状态 (Status):** 待审批 (Pending Approval) +**负责人 (Owner):** ObjectQL Migration Team