Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
36 changes: 33 additions & 3 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,11 +7,43 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0

## [Unreleased]

### Fixed

- **`applySyncResponse` frontier source** — now uses `_lastFrontier` (SHA map) instead of `observedFrontier` (VersionVector) as the 3rd arg to `applySyncResponseImpl`, eliminating a double-cast bug (B56).
- **`syncWith` infinite delegation guard** — `syncWith` now calls `this.createSyncRequest()` / `this.applySyncResponse()` directly instead of `this._host.*`, preventing infinite delegation when the host delegates back to the controller.
- **`BitmapIndexReader` strict default** — changed from `false` to `true`; shard OID validation errors now throw `ShardCorruptionError` by default instead of silently skipping. All existing callers already pass `strict: true` explicitly.
- **`mockServerGraph` asymmetry** — `syncAuth` test helper now mocks on `_syncController.processSyncRequest` (matching `mockClientGraph` pattern) instead of shadowing the prototype with an own-property mock.
- **Stale comment** — `BitmapIndexReader.test.js` comment corrected from "default" to "explicit override" for `strict: false` reader.
- **OID length standardization** — all 25 short 8-char OIDs in `BitmapIndexReader.test.js` extended to 40-char zero-padded hex, matching real Git SHA-1 length and eliminating non-hex characters (`eeffgghh` → `eeff00dd…`).

### Added

- **Frontier fix verification test** (`SyncController.test.js`) — confirms `applySyncResponse` passes `_lastFrontier` (SHA map), not `observedFrontier` (VersionVector), as 3rd arg.
- **Null-context guard tests** (`JoinReducer.test.js`) — 2 tests verifying `applyFast` handles `undefined` and `null` context gracefully via the `|| {}` fallback.
- **Auto-materialize path tests** (`SyncController.test.js`) — 2 tests for `syncWith`: calls `materialize()` when `_cachedState` is null; returns `state` when `materialize: true`.
- **HTTP sync path tests** (`SyncController.test.js`) — 9 tests covering success, 5xx/4xx status codes, invalid JSON, AbortError, TimeoutError, network error, `shouldRetry` predicate, and auth header forwarding.
- **`serve()` deeper tests** (`SyncController.test.js`) — 3 tests verifying `HttpSyncServer` constructor args, auth config enhancement (crypto + logger injection), and graph host passthrough.

## [11.5.1] — 2026-02-22 — M9 PARTITION: Architectural Decomposition

Breaks apart structural DRY violations and extracts encapsulated services
from the WarpGraph god class, without changing any public API surface.

### Added

- **Publication-quality SVG diagrams** — 8 Graphviz diagrams in `docs/diagrams/` covering the empty tree trick, two-plane state model, ref layout, patch anatomy, multi-writer convergence, materialization pipeline, checkpoint tree, and hexagonal architecture. Grayscale, transparent-background, serif-font styling matching the AION paper aesthetic.
- **Publication-quality SVG diagrams** — 8 Graphviz diagrams in `docs/diagrams/` covering data storage, two-plane state model, ref layout, patch anatomy, multi-writer convergence, materialization pipeline, checkpoint tree, and hexagonal architecture. Grayscale, transparent-background, serif-font styling matching the AION paper aesthetic.
- **`scripts/build-diagrams.sh`** — compiles all `.dot` files to SVG with transparent-background post-processing.

- **`SyncController`** (`src/domain/services/SyncController.js`) — new class encapsulating all 9 sync methods (`getFrontier`, `hasFrontierChanged`, `status`, `createSyncRequest`, `processSyncRequest`, `applySyncResponse`, `syncNeeded`, `syncWith`, `serve`) and 2 private helpers. Independently unit-testable with a mock host object. 16 new tests.
- **`applyFast()` / `applyWithReceipt()`** — named exported functions in `JoinReducer.js` replacing the duplicated fast/receipt code paths. `join()` is now a 3-line dispatcher. `reduceV5()` calls named functions directly. 4 new tests.
- **`isValidOid()`** (`src/domain/utils/validateShardOid.js`) — domain-local hex OID validator (4–64 chars). `BitmapIndexReader.setup()` validates each shard OID: strict mode throws `ShardCorruptionError`, non-strict skips with warning. 13 new tests.

### Changed

- **`sync.methods.js`** — deleted entirely; sync methods now wired via `defineProperty` delegation to `_syncController`.
- **`WarpGraph.js`** — added `_syncController` field instantiation in constructor (+4 LOC, now 422 LOC total — well under the 500 LOC M9 gate).
- **`JoinReducer.join()`** — refactored from inline dual-path to dispatcher over `applyFast` / `applyWithReceipt`. Shared frontier update logic extracted into `updateFrontierFromPatch()` helper.

## [11.5.0] — 2026-02-20 — Content Attachment (Paper I `Atom(p)`)

Implements content attachment — the ability to attach content-addressed blobs
Expand Down Expand Up @@ -153,8 +185,6 @@ to prevent regressions.
- **WarpPersistence types** — Added `IndexStorage` typedef (`BlobPort & TreePort & RefPort`).
- **Policy checker upgrade** — `ts-policy-check.js` now enforces 4 rules: (1) ban `@ts-ignore`, (2) ban `@type {*}`/`@type {any}`, (3) ban embedded wildcards in JSDoc generics, (4) ban `z.any()`.

## [Unreleased]

## [11.3.0] — 2026-02-17 — DX-HAMMER: Read-Path CLI Improvements

New CLI commands and improved output for graph inspection and debugging.
Expand Down
12 changes: 12 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,10 @@ const result = await graph.query()

## How It Works

<p align="center">
<img src="docs/diagrams/fig-data-storage.svg" alt="WARP data storage — invisible to normal Git workflows" width="700">
</p>

### The Multi-Writer Problem (and How It's Solved)

Multiple people (or machines, or processes) can write to the same graph **simultaneously, without any coordination**. There's no central server, no locking, no "wait your turn."
Expand All @@ -73,6 +77,10 @@ Every operation gets a unique **EventId** — `(lamport, writerId, patchSha, opI

## Multi-Writer Collaboration

<p align="center">
<img src="docs/diagrams/fig-multi-writer.svg" alt="Multi-writer convergence — independent chains, deterministic merge" width="700">
</p>

Writers operate independently on the same Git repository. Sync happens through standard Git transport (push/pull) or the built-in HTTP sync protocol.

```javascript
Expand Down Expand Up @@ -457,6 +465,10 @@ When a seek cursor is active, `query`, `info`, `materialize`, and `history` auto

## Architecture

<p align="center">
<img src="docs/diagrams/fig-architecture.svg" alt="Hexagonal architecture — dependency rule: arrows point inward only" width="700">
</p>

The codebase follows hexagonal architecture with ports and adapters:

**Ports** define abstract interfaces for infrastructure:
Expand Down
13 changes: 9 additions & 4 deletions ROADMAP.md
Original file line number Diff line number Diff line change
Expand Up @@ -619,23 +619,23 @@ No v2.0 tag until **every** gate passes. If any RG fails: no tag. Period.

### M9.T1 — SyncController Extraction

- **Status:** `PENDING`
- **Status:** `DONE`

**Items:**

- **B33** (WARPGRAPH SYNCCONTROLLER EXTRACTION) — move `syncWith`, `serve`, `processSyncRequest` from `WarpGraph.js` into `SyncController.js`; reduces god class by ~800 LOC; isolates network concerns from graph concerns.

### M9.T2 — JoinReducer Dual-Path Refactor

- **Status:** `PENDING`
- **Status:** `DONE`

**Items:**

- **B32** (JOINREDUCER DUAL-PATH REFACTOR) — split `join()` into `applyFast(state, patch)` and `applyWithReceipt(state, patch)` strategy pair; eliminates DRY violation where new op types in fast path can be missed in receipt path.

### M9.T3 — Bitmap OID Validation (Opportunistic)

- **Status:** `PENDING`
- **Status:** `DONE`

**Items:**

Expand Down Expand Up @@ -745,8 +745,13 @@ Items that can be picked up opportunistically without blocking anything. No mile
| B53 | **FIX JSR PUBLISH DRY-RUN DENO PANIC** — Deno 2.6.7 `deno_ast` panics on overlapping text changes from duplicate `roaring` import rewrites; either pin Deno version, vendor the import, or file upstream issue and add workaround |
| B54 | **`typedCustom()` ZOD HELPER** — `z.custom()` without a generic yields `unknown` in JS; a JSDoc-friendly wrapper (or `@typedef`-based pattern) would eliminate verbose `/** @type {z.ZodType<T>} */ (z.custom(...))` casts across HttpSyncServer and future Zod schemas |
| B55 | **UPGRADE `HttpServerPort` REQUEST/RESPONSE TYPES** — `createServer` callback uses `Object` for `headers` and `string|Buffer` for response body; tighten to `Record<string, string>` and extract shared request/response typedefs to avoid repeated inline casts in HttpSyncServer, NodeHttpAdapter, BunHttpAdapter, DenoHttpAdapter |
| B56 | **INVESTIGATE `observedFrontier` / FRONTIER SEMANTIC MISMATCH** — `sync.methods.js` line 261 double-casts `observedFrontier` (a version vector `Map<string, number>`) to `Map<string, string>` (writer frontier) before passing to `applySyncResponseImpl`; determine whether this is a latent correctness bug or an intentional coercion, and fix or document accordingly |
| ~~B56~~ | ~~**INVESTIGATE `observedFrontier` / FRONTIER SEMANTIC MISMATCH**~~DONE. Was a latent bug: `applySyncResponse()` passed `observedFrontier` (Lamport counters `Map<string, number>`) instead of `_lastFrontier` (SHA frontier `Map<string, string>`). Fixed to use `_lastFrontier` with `createFrontier()` fallback. |
| B57 | **CI: AUTO-VALIDATE `type-surface.m8.json` AGAINST `index.d.ts`** — add a CI gate or pre-push check that parses the manifest and confirms every declared method/property/return type matches the corresponding signature in `index.d.ts`; prevents drift like the missing `setSeekCache` and `syncWith.state` return found in review |
| ~~B58~~ | ~~**DEAD SYNC CONSTANTS IN `_internal.js`**~~ — DONE. Deleted orphaned sync constants and re-exports from `_internal.js`. |
| ~~B59~~ | ~~**ELIMINATE `wireWarpMethods` INDIRECTION FOR SYNC**~~ — DONE. Deleted `sync.methods.js`; 9 sync methods wired directly onto WarpGraph.prototype via `defineProperty` loop delegating to `_syncController`. |
| ~~B60~~ | ~~**TEST HELPER: `mockOid(tag)`**~~ — DONE then REVERTED. Created speculatively but never adopted; deleted as dead code. |
| ~~B61~~ | ~~**REMOVE DOUBLED `/** @type {any} */` ANNOTATIONS IN TESTS**~~ — DONE. Removed 66 doubled annotations across 13 test files. |
| ~~B62~~ | ~~**DEFINE `SyncHost` INTERFACE FOR SYNCCONTROLLER**~~ — DONE. Added `@typedef SyncHost` to `SyncController.js` documenting all 12 host fields + 3 methods. |

### Conformance Property Pack (B19 + B22)

Expand Down
28 changes: 28 additions & 0 deletions docs/GUIDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -87,6 +87,10 @@ const exists = await graph.hasNode('todo:1');

That's it. Your graph data is stored as Git commits — invisible to normal Git workflows but inheriting all of Git's properties.

<p align="center">
<img src="diagrams/fig-architecture.svg" alt="Hexagonal architecture overview" width="700">
</p>

---

## Writing Data
Expand Down Expand Up @@ -207,6 +211,10 @@ Before reading, you need to **materialize** — this replays all patches from al

### Materialization

<p align="center">
<img src="diagrams/fig-materialize-pipeline.svg" alt="Materialization pipeline — from refs to consistent state" width="700">
</p>

```javascript
const state = await graph.materialize();
```
Expand Down Expand Up @@ -464,6 +472,10 @@ WarpGraph's core strength is coordination-free multi-writer collaboration. Each

### How It Works

<p align="center">
<img src="diagrams/fig-multi-writer.svg" alt="Multi-writer convergence" width="700">
</p>

```javascript
// === Machine A ===
const graphA = await WarpGraph.open({
Expand Down Expand Up @@ -497,6 +509,10 @@ const stateB = await graphB.materialize();

### Conflict Resolution

<p align="center">
<img src="diagrams/fig-two-plane.svg" alt="Two-plane state model — skeleton topology + attachment fibres" width="600">
</p>

When two writers modify the same property concurrently, the conflict is resolved deterministically using **Last-Writer-Wins (LWW)** semantics. The winner is the operation with the higher priority, compared in this order:

1. Higher Lamport timestamp wins
Expand Down Expand Up @@ -572,6 +588,10 @@ if (changed) {

### Checkpoints

<p align="center">
<img src="diagrams/fig-checkpoint-tree.svg" alt="Checkpoint tree — snapshot for fast recovery" width="500">
</p>

A **checkpoint** is a snapshot of materialized state at a known point in history. Without checkpoints, materialization replays every patch from every writer. With a checkpoint, it loads the snapshot and only replays patches since then.

```javascript
Expand Down Expand Up @@ -1348,6 +1368,10 @@ Each patch carries its version vector as causal context. This allows the reducer

### Appendix B: Git Ref Layout

<p align="center">
<img src="diagrams/fig-ref-layout.svg" alt="Ref layout — the refs/warp/ namespace" width="600">
</p>

```text
refs/warp/<graphName>/
├── writers/
Expand All @@ -1364,6 +1388,10 @@ Each writer's ref points to the tip of their patch chain. Patches are Git commit

### Appendix C: Patch Format

<p align="center">
<img src="diagrams/fig-patch-anatomy.svg" alt="Patch commit anatomy" width="500">
</p>

Each patch is a Git commit containing:

- **CBOR-encoded operations** in a blob referenced from the commit message
Expand Down
20 changes: 12 additions & 8 deletions docs/diagrams/fig-architecture.dot
Original file line number Diff line number Diff line change
Expand Up @@ -93,21 +93,25 @@ digraph architecture {
}
}

// === Inward-pointing arrows (dependency rule) ===
// Adapters implement Ports
git_adapter -> persistence_port [dir=back, style=dashed, penwidth=0.5]
cbor_codec -> codec_port [dir=back, style=dashed, penwidth=0.5]
web_crypto -> crypto_port [dir=back, style=dashed, penwidth=0.5]
clock_adapter -> clock_port [dir=back, style=dashed, penwidth=0.5]
console_log -> logger_port [dir=back, style=dashed, penwidth=0.5]
cas_cache -> cache_port [dir=back, style=dashed, penwidth=0.5]
// === Outward-pointing arrows (adapter implements port) ===
persistence_port -> git_adapter [style=dashed, penwidth=0.5, label=<implements>]
codec_port -> cbor_codec [style=dashed, penwidth=0.5, label=<implements>]
crypto_port -> web_crypto [style=dashed, penwidth=0.5, label=<implements>]
clock_port -> clock_adapter [style=dashed, penwidth=0.5, label=<implements>]
logger_port -> console_log [style=dashed, penwidth=0.5, label=<implements>]
cache_port -> cas_cache [style=dashed, penwidth=0.5, label=<implements>]

// IndexStoragePort is implemented via GitGraphAdapter (shared adapter)
index_port -> git_adapter [style=dashed, penwidth=0.5, label=<via>]

// Domain uses Ports
warp_graph -> persistence_port [penwidth=0.5]
warp_graph -> codec_port [penwidth=0.5]
warp_graph -> crypto_port [penwidth=0.5]
warp_graph -> clock_port [penwidth=0.5]
warp_graph -> logger_port [penwidth=0.5]
warp_graph -> index_port [penwidth=0.5]
warp_graph -> cache_port [penwidth=0.5]

// Internal domain relationships
warp_graph -> reducer [penwidth=0.5, style=invis]
Expand Down
Loading
Loading