Skip to content

Flask backend for server-side Python execution#250

Merged
milanofthe merged 27 commits intomainfrom
feature/flask-backend
Feb 11, 2026
Merged

Flask backend for server-side Python execution#250
milanofthe merged 27 commits intomainfrom
feature/flask-backend

Conversation

@milanofthe
Copy link
Member

@milanofthe milanofthe commented Feb 10, 2026

Adds a Flask/Waitress backend as an alternative to the Pyodide Web Worker for server-side Python execution. Useful for packages with native C extensions that aren't built for Emscripten.

Architecture

  • Worker subprocess (worker.py): REPL over stdin/stdout JSON lines. One per session. Handles exec, eval, and streaming with timeout watchdog (30s).
  • Flask server (app.py): Session management, routes translate HTTP into subprocess messages. Uses waitress in production, Flask dev server with --debug.
  • Venv isolation (venv.py): Workers run in a dedicated venv at ~/.pathview/venv. Simulation packages install there, not in the user's environment.
  • FlaskBackend (backend.ts): TypeScript client implementing the Backend interface. HTTP requests + long-polling for streaming.
  • Session sharing: Cross-tab coordination via localStorage + BroadcastChannel so multiple tabs share one worker.

Key details

  • Backend selected via URL param: ?backend=flask&host=http://localhost:5050
  • Package list comes from PYTHON_PACKAGES in dependencies.ts (same source of truth as Pyodide)
  • Pip-installable: pip install pathviewpathview serve
  • Worker crash recovery: auto-reinit on next request
  • Streaming: server-side generator loop with long-polling (100ms block), supports runtime parameter changes via stream-exec

Testing

  • 23 tests: integration tests (Flask routes + real worker subprocesses) and unit tests (worker JSON protocol)
  • CI: GitHub Actions matrix — ubuntu/windows × Python 3.11/3.12

milanofthe and others added 27 commits February 10, 2026 11:17
Co-Authored-By: KDW1 <KDW1@users.noreply.github.com>
Co-Authored-By: KDW1 <KDW1@users.noreply.github.com>
Co-Authored-By: KDW1 <KDW1@users.noreply.github.com>
Co-Authored-By: KDW1 <KDW1@users.noreply.github.com>
Co-Authored-By: KDW1 <KDW1@users.noreply.github.com>
Co-Authored-By: KDW1 <KDW1@users.noreply.github.com>
Co-Authored-By: KDW1 <KDW1@users.noreply.github.com>
Co-Authored-By: KDW1 <KDW1@users.noreply.github.com>
Co-Authored-By: KDW1 <KDW1@users.noreply.github.com>
Co-Authored-By: KDW1 <KDW1@users.noreply.github.com>
- Migrate streaming from SSE to polling (start+poll pattern)
- Fix stdin race condition with noop flush and leftover queue
- Fix logging capture with permanent _ProtocolWriter on sys.stdout/stderr
- Add recursion guard to _ProtocolWriter to prevent infinite loops
- Fix ensureServerInit to retry on failure instead of caching rejection
- Fix initBackendFromUrl to call init() for proper callback setup
- Add resp.ok checks and poll response validation in FlaskBackend
- Align stopStreaming to wait for server confirmation like Pyodide
- Validate X-Session-ID header, return 400 when missing
- Prevent orphan sessions in poll/exec/stop by using lookup instead of create
- Move cleanup thread start into create_app factory
- Fix requires-python to >=3.10 and add numpy dependency
- Remove dead code: _capture_output, REQUEST_TIMEOUT, unused imports

Co-Authored-By: KDW1 <KDW1@users.noreply.github.com>
@milanofthe milanofthe merged commit 8a33d24 into main Feb 11, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant