fix(performance): eliminate memory debugging overhead from benchmarks

The 3-4x performance regression was caused by memory optimization environment
variables (PYTHONMALLOC=debug, single-threading) that prevent segfaults but
severely impact benchmark performance.

Changes:
- Remove CI=true/GITHUB_ACTIONS=true from benchmark workflows to avoid memory debugging
- Set optimal performance environment for benchmarks (4 threads vs 1)
- Use direct pytest for benchmarks instead of run_tests.py wrapper
- Keep memory optimizations only for regular tests that need segfault protection
- Maintain consistent text size (100 repetitions) across all environments

This should restore benchmark performance to expected levels while maintaining
segfault protection for regular test runs.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>

Author: sidmohan0

.github/workflows/benchmark.yml

@@ -39,11 +39,14 @@ jobs:
 
       - name: Run benchmarks and save baseline
         env:
-          CI: true
-          GITHUB_ACTIONS: true
+          # DO NOT set CI=true or GITHUB_ACTIONS=true here to avoid memory optimization slowdowns
+          # Set optimal performance environment for benchmarks
+          OMP_NUM_THREADS: 4
+          MKL_NUM_THREADS: 4
+          OPENBLAS_NUM_THREADS: 4
         run: |
-          # Run benchmarks with segfault protection and save results
-          echo "Running benchmarks with memory optimizations..."
+          # Run benchmarks with optimal performance settings (no memory debugging)
+          echo "Running benchmarks with performance-optimized settings..."
           python -m pytest tests/benchmark_text_service.py -v --benchmark-autosave --benchmark-json=benchmark-results.json --tb=short
 
       - name: Check for performance regression

.github/workflows/beta-release.yml

@@ -126,9 +126,9 @@ jobs:
           echo "Running integration tests..."
           python run_tests.py -m integration --no-header
           
-          # Run benchmark tests with segfault protection
-          echo "Running benchmark tests with safeguards..."
-          python run_tests.py tests/benchmark_text_service.py --no-header
+          # Run benchmark tests with optimal performance (no memory debugging)
+          echo "Running benchmark tests with performance optimizations..."
+          OMP_NUM_THREADS=4 MKL_NUM_THREADS=4 OPENBLAS_NUM_THREADS=4 python -m pytest tests/benchmark_text_service.py -v --no-header
 
       - name: Build package
         run: |

tests/benchmark_text_service.py

@@ -26,12 +26,14 @@ def sample_text_10kb():
     # Check if running in CI environment
     import os
 
+    # For performance benchmarks, always use consistent moderate size for stable results
+    # regardless of environment to avoid performance variance from text size differences
     if os.environ.get("CI") or os.environ.get("GITHUB_ACTIONS"):
         # Use moderate sample in CI for stable benchmarks (not too small to avoid variance)
         repetitions = 100  # Increased from 50 for more stable results
     else:
-        # Use full size for local development
-        repetitions = 10000 // len(base_text) + 1
+        # Use moderate size for local development benchmarks too for consistency
+        repetitions = 100  # Keep consistent with CI for fair comparison
 
     return base_text * repetitions