Scale request logic

docs/PLAN.md

@@ -0,0 +1,64 @@
+# AIBTCDEV Cache Fix Plan
+
+## Overview
+This plan addresses test failures from `bash tests/run_tests.sh`, focusing on path malformation (double slashes), error handling (unexpected 500s), and graceful handling of missing Hiro keys. It is sequenced for quick wins first. After each step, re-run tests and update this file with status (e.g., [x] Completed).
+
+## Actionable Steps
+
+1. **Fix Test Script to Eliminate Double Slashes (Quick Win)**
+   - [x] In `tests/test_contract_calls.sh` and `tests/run_tests.sh`, trim trailing `/` from API_URL.
+   - Why: Prevents // paths in requests.
+   - Assigned: Developer
+   - Status: Completed
+
+2. **Normalize Paths in ContractCallsDO for Robustness**
+   - [x] Add path normalization in `src/durable-objects/contract-calls-do.ts`.
+   - Why: Handles double slashes gracefully.
+   - Assigned: Developer
+   - Status: Completed
+
+3. **Improve Error Handling to Prevent Unexpected 500s**
+   - [x] Add try-catch in `src/durable-objects/contract-calls-do.ts`, `src/services/request-queue-service.ts`, and debug logs in `src/services/stacks-api-service.ts`.
+   - Why: Ensures correct status codes (e.g., 404 instead of 500).
+   - Assigned: Developer
+   - Status: Completed
+
+4. **Handle Hiro Key Absences Gracefully**
+   - [x] Add warnings for missing keys in `src/durable-objects/contract-calls-do.ts`, `src/services/stacks-api-service.ts`, and `src/config.ts`.
+   - Why: Improves debuggability without crashing.
+   - Assigned: Developer
+   - Status: Completed
+
+5. **Test, Validate, and Document**
+   - [x] Re-run `bash tests/run_tests.sh` after each step.
+   - [x] Update `docs/RETRO.md` with resolutions.
+   - [x] Test edge cases (e.g., manual curls with double slashes).
+   - Assigned: Tester/Developer
+   - Status: Completed
+
+## Iteration Notes
+- Track progress by checking boxes.
+- If new issues arise, add to `docs/QUESTIONS.md` and revise this plan.
+- External: HIRO_API_KEYS setup is separate; monitor for rate limit impacts.
+
+## Iteration 1: Addressing Remaining Test Failures
+Based on latest test output (6 failures: 500s for base/known/ABI/invalid, decode error with "config2.getHiroDoNames is not a function"), root cause is a bug in src/index.ts DO routing (calling getHiroDoNames on config object instead of AppConfig instance, causing TypeError turned to 500). Read-only succeeded possibly due to fast-path cache hit. Invalid returns correct NOT_FOUND body (likely 404 status), but test reports 500—clarify in QUESTIONS.md.
+
+6. **Fix DO Selection Bug in index.ts**
+   - [x] Change config.getHiroDoNames() to AppConfig.getInstance(env).getHiroDoNames() in src/index.ts.
+   - Why: Corrects method call to avoid TypeError and ensure proper DO routing.
+   - Assigned: Developer
+   - Status: Completed
+
+7. **Investigate and Fix Test Script Status Checks**
+   - [x] Inspect utils.sh (add to chat if needed) and adjust test_endpoint to correctly parse status for error responses.
+   - Why: Test reports 500 for invalid but manual curl shows expected NOT_FOUND body; ensure test checks actual HTTP status.
+   - Assigned: Developer
+   - Status: Completed
+
+8. **Re-Test and Validate**
+   - [x] Re-run `bash tests/run_tests.sh` and manual curls after fixes.
+   - [x] If 500s persist for ABI/known, check server logs for requestIds (e.g., "f67162e8") to trace.
+   - [x] Update RETRO.md with resolutions.
+   - Assigned: Tester/Developer
+   - Status: Completed

docs/QUESTIONS.md

@@ -0,0 +1,14 @@
+# Open Questions and External Tasks for AIBTCDEV Cache Fixes
+
+This file tracks clarifications needed or tasks outside our code change process. Update as we progress.
+
+## Questions/Clarifications
+
+## External Tasks
+- If new files are needed (e.g., for unexpected dependencies), add them to the chat.
+
+## Resolved
+- Are there specific error status mappings in `src/utils/error-catalog-util.ts` that differ from standards (e.g., NOT_FOUND not 404)? If so, provide the file for review. -> File added; mappings are standard (e.g., NOT_FOUND=404).
+- During testing, if persistent 500s occur, can we access server-side logs for a specific requestId (e.g., from test output) to trace upstream errors? -> Yes, viewable but use minimal logging for cost.
+- Set HIRO_API_KEYS in the environment to avoid rate limiting issues (as per original query, handled separately). -> Officially set via wrangler secret put.
+- In test_contract_calls.sh, why does test_endpoint report 500 for invalid endpoint when manual curl shows NOT_FOUND body (expected for 404)? Add tests/utils.sh to chat for review if needed to check status parsing logic. -> File added; fixed parsing by separating status curl and adjusting headers/body extraction.

docs/RETRO.md

@@ -0,0 +1,24 @@
+# 2025-08-27
+
+## Accomplishments from AIBTCDEV Cache Architecture Update Plan
+
+- Updated src/config.ts to handle multiple HIRO_API_KEYS as an array, added methods for generating hashed Durable Object names and reverse lookup of keys from DO IDs.
+- Enhanced src/index.ts with CacheService and CacheKeyService initialization, implemented fast-path KV cache checking for /contract-calls/read-only/ endpoints, and added round-robin selection of ContractCallsDO instances using a KV counter for load balancing.
+- Modified src/durable-objects/contract-calls-do.ts to assign Hiro API keys based on DO IDs, pass keys to StacksContractFetcher, and set higher priority for non-cache-busting requests in the queue.
+- Updated src/services/stacks-api-service.ts to accept and use Hiro API keys in the constructor for authentication middleware.
+- Adjusted src/services/stacks-contract-data-service.ts to accept and forward Hiro API keys to StacksApiService.
+- Improved src/services/request-queue-service.ts by adding priority to QueuedRequest, accepting priority in enqueue, and sorting the queue by priority in processQueue.
+- Integrated Hiro rate limit headers by updating TokenBucket to sync from response headers, modifying StacksApiService to perform custom fetches and pass headers for syncing, and updating StacksContractFetcher to use the sync functionality after requests.
+- Marked all implementation steps and tasks as completed in docs/PLAN.md and docs/QUESTIONS.md.
+
+# 2025-08-27
+
+## Accomplishments from AIBTCDEV Cache Fix Plan
+
+- Fixed test scripts to trim trailing slashes from API_URL, preventing double-slash path malformations.
+- Added path normalization in ContractCallsDO to handle multiple slashes robustly.
+- Improved error handling with try-catch blocks to ensure consistent status codes and prevent unexpected 500s.
+- Added warnings for missing Hiro API keys to improve debuggability.
+- Marked all steps as completed in docs/PLAN.md after validation; tests now pass with the fixes.
+
+- Added missing Logger imports to fix "Logger is not defined" errors in various services and DOs.

src/config.ts

@@ -1,6 +1,8 @@
 import { Env } from '../worker-configuration';
 import { ApiError } from './utils/api-error-util';
 import { ErrorCode } from './utils/error-catalog-util';
+import { createHash } from 'crypto';
+import { Logger } from './utils/logger-util';
 
 /**
  * Singleton configuration class for the application
@@ -38,15 +40,63 @@ export class AppConfig {
 		return AppConfig.instance;
 	}
 
+	/**
+	 * Generates a hashed name for a Durable Object based on the API key
+	 *
+	 * @param key - The API key to hash
+	 * @returns A unique name for the DO instance
+	 */
+	private hashToIdName(key: string): string {
+		const hash = createHash('sha256').update(key).digest('hex').substring(0, 8);
+		return `contract-calls-do-${hash}`;
+	}
+
+	/**
+	 * Returns the list of hashed DO names for Hiro API keys
+	 *
+	 * @returns Array of DO names, falling back to a single default if no keys
+	 */
+	public getHiroDoNames(): string[] {
+		const keys = this.env.HIRO_API_KEYS?.split(',').map(k => k.trim()) || [];
+		if (keys.length === 0) {
+			Logger.getInstance().debug('No Hiro API keys configured; falling back to default DO');
+			return ['contract-calls-do'];
+		}
+		return keys.map(key => this.hashToIdName(key));
+	}
+
+	/**
+	 * Looks up the API key for a given DO ID
+	 *
+	 * @param doId - The DO ID string to lookup
+	 * @returns The corresponding API key or undefined if not found/no keys
+	 */
+	public getKeyForDoId(doId: string): string | undefined {
+		const keys = this.env.HIRO_API_KEYS?.split(',').map(k => k.trim()) || [];
+		if (keys.length === 0) {
+			const defaultId = this.env.CONTRACT_CALLS_DO.idFromName('contract-calls-do').toString();
+			return defaultId === doId ? undefined : undefined;
+		}
+		for (const key of keys) {
+			const name = this.hashToIdName(key);
+			const computedId = this.env.CONTRACT_CALLS_DO.idFromName(name).toString();
+			if (computedId === doId) {
+				return key;
+			}
+		}
+		return undefined;
+	}
+
 	/**
 	 * Returns the application configuration settings
 	 *
 	 * @returns Configuration object with all application settings
 	 */
 
 	public getConfig() {
-		// Check if Hiro API key is available
-		const hasHiroApiKey = !!this.env.HIRO_API_KEY;
+		// Get Hiro API keys as array
+		const keys = this.env.HIRO_API_KEYS?.split(',').map(k => k.trim()) || [];
+		const hasHiroApiKey = keys.length > 0;
 
 		return {
 			// supported services for API caching
@@ -64,8 +114,8 @@ export class AppConfig {
 			ALARM_INTERVAL_MS: 300000, // 5 minutes
 			// Hiro API specific rate limiting settings
 			HIRO_API_RATE_LIMIT: {
-				// Adjust based on whether we have an API key
-				// Hiro limits: 50 RPM without key, 500 RPM with key
+				// Adjust based on whether we have API keys
+				// Hiro limits: 50 RPM without key, 500 RPM with key (per key/DO)
 				MAX_REQUESTS_PER_MINUTE: hasHiroApiKey ? 500 : 50,
 				// Convert to our interval format
 				get MAX_REQUESTS_PER_INTERVAL() {
@@ -83,7 +133,7 @@ export class AppConfig {
 			// environment variables
 			SUPABASE_URL: this.env.SUPABASE_URL,
 			SUPABASE_SERVICE_KEY: this.env.SUPABASE_SERVICE_KEY,
-			HIRO_API_KEY: this.env.HIRO_API_KEY,
+			HIRO_API_KEYS: keys,
 		};
 	}
 }