diff --git a/.github/workflows/format.yml b/.github/workflows/format.yml
new file mode 100644
index 0000000..a226ae0
--- /dev/null
+++ b/.github/workflows/format.yml
@@ -0,0 +1,23 @@
+name: pre-commit-format
+
+on:
+ push:
+ branches: [main]
+ pull_request:
+ branches: [main]
+
+jobs:
+ pre_commit_checks:
+ runs-on: ubuntu-latest
+ steps:
+ - uses: actions/checkout@v3
+ with:
+ fetch-depth: 0
+ ref: ${{ github.head_ref }}
+
+ - uses: actions/setup-python@v4
+ with:
+ python-version: '3.10'
+
+ - run: pip install pre-commit
+ - run: pre-commit run --all-files
diff --git a/.gitignore b/.gitignore
index 208ed2c..7c5b18d 100644
--- a/.gitignore
+++ b/.gitignore
@@ -1,10 +1,3 @@
-#####################
-### DLC Live Specific
-#####################
-
-*config*
-**test*
-
###################
### python standard
###################
@@ -116,3 +109,7 @@ venv.bak/
# ide files
.vscode
+
+!dlclivegui/config.py
+# uv package files
+uv.lock
diff --git a/.pre-commit-config.yaml b/.pre-commit-config.yaml
new file mode 100644
index 0000000..ceaed2d
--- /dev/null
+++ b/.pre-commit-config.yaml
@@ -0,0 +1,19 @@
+repos:
+ - repo: https://github.com/pre-commit/pre-commit-hooks
+ rev: v4.4.0
+ hooks:
+ - id: check-added-large-files
+ - id: check-yaml
+ - id: end-of-file-fixer
+ - id: name-tests-test
+ args: [--pytest-test-first]
+ - id: trailing-whitespace
+ - id: check-merge-conflict
+ - repo: https://github.com/astral-sh/ruff-pre-commit
+ rev: v0.14.10
+ hooks:
+ # Run the formatter.
+ - id: ruff-format
+ # Run the linter.
+ - id: ruff-check
+ args: [--fix,--unsafe-fixes]
diff --git a/README.md b/README.md
index acdadf2..b9a2785 100644
--- a/README.md
+++ b/README.md
@@ -1,69 +1,318 @@
-# DeepLabCut-Live! GUI 
+# DeepLabCut Live GUI
-
-
-
-
+A modern PySide6 GUI for running [DeepLabCut-live](https://github.com/DeepLabCut/DeepLabCut-live) experiments with real-time pose estimation. The application streams frames from industrial or consumer cameras, performs DLCLive inference, and records high-quality video with synchronized pose data.
-[](https://github.com/DeepLabCut/deeplabcutlive/raw/master/LICENSE)
-[](https://forum.image.sc/tags/deeplabcut)
-[](https://gitter.im/DeepLabCut/community?utm_source=badge&utm_medium=badge&utm_campaign=pr-badge)
-[](https://twitter.com/DeepLabCut)
+## Features
-GUI to run [DeepLabCut-live](https://github.com/DeepLabCut/DeepLabCut-live) on a video feed, record videos, and record external timestamps.
+### Core Functionality
+- **Modern Python Stack**: Python 3.10+ compatible codebase with PySide6 interface
+- **Multi-Backend Camera Support**: OpenCV, GenTL (Harvesters), Aravis, and Basler (pypylon)
+- **Real-Time Pose Estimation**: Live DLCLive inference with configurable models (TensorFlow, PyTorch)
+- **High-Performance Recording**: Hardware-accelerated video encoding via FFmpeg
+- **Flexible Configuration**: Single JSON file for all settings with GUI editing
-## [Installation Instructions](docs/install.md)
+### Camera Features
+- **Multiple Backends**:
+ - OpenCV - Universal webcam support
+ - GenTL - Industrial cameras via Harvesters (Windows/Linux)
+ - Aravis - GenICam/GigE cameras (Linux/macOS)
+ - Basler - Basler cameras via pypylon
+- **Smart Device Detection**: Automatic camera enumeration without unnecessary probing
+- **Camera Controls**: Exposure time, gain, frame rate, and ROI cropping
+- **Live Preview**: Real-time camera feed with rotation support (0°, 90°, 180°, 270°)
-## Getting Started
+### DLCLive Features
+- **Model Support**: Only PyTorch models! (in theory also tensorflow models work)
+- **Processor System**: Plugin architecture for custom pose processing
+- **Auto-Recording**: Automatic video recording triggered by processor commands
+- **Performance Metrics**: Real-time FPS, latency, and queue monitoring
+- **Pose Visualization**: Optional overlay of detected keypoints on live feed
-#### Open DeepLabCut-live-GUI
+### Recording Features
+- **Hardware Encoding**: NVENC (NVIDIA GPU) and software codecs (libx264, libx265)
+- **Configurable Quality**: CRF-based quality control
+- **Multiple Formats**: MP4, AVI, MOV containers
+- **Timestamp Support**: Frame-accurate timestamps for synchronization
+- **Performance Monitoring**: Write FPS, buffer status, and dropped frame tracking
-In a terminal, activate the conda or virtual environment where DeepLabCut-live-GUI is installed, then run:
+### User Interface
+- **Intuitive Layout**: Organized control panels with clear separation of concerns
+- **Configuration Management**: Load/save settings, support for multiple configurations
+- **Status Indicators**: Real-time feedback on camera, inference, and recording status
+- **Bounding Box Tool**: Visual overlay for ROI definition
+## Installation
+
+### Basic Installation
+
+```bash
+pip install deeplabcut-live-gui
+```
+
+This installs the core package with OpenCV camera support.
+
+### Full Installation with Optional Dependencies
+
+```bash
+# Install with gentl support
+pip install deeplabcut-live-gui[gentl]
+```
+
+### Platform-Specific Camera Backend Setup
+
+#### Windows (GenTL for Industrial Cameras)
+1. Install camera vendor drivers and SDK
+2. Ensure GenTL producer (.cti) files are accessible
+3. Common locations:
+ - `C:\Program Files\The Imaging Source Europe GmbH\IC4 GenTL Driver\bin\`
+ - Check vendor documentation for CTI file location
+
+#### Linux (Aravis for GenICam Cameras - Recommended)
+NOT tested
+```bash
+# Ubuntu/Debian
+sudo apt-get install gir1.2-aravis-0.8 python3-gi
+
+# Fedora
+sudo dnf install aravis python3-gobject
```
-dlclivegui
+
+#### macOS (Aravis)
+NOT tested
+```bash
+brew install aravis
+pip install pygobject
```
+#### Basler Cameras (All Platforms)
+NOT tested
+```bash
+# Install Pylon SDK from Basler website
+# Then install pypylon
+pip install pypylon
+```
-#### Configurations
+### Hardware Acceleration (Optional)
+For NVIDIA GPU encoding (highly recommended for high-resolution/high-FPS recording):
+```bash
+# Ensure NVIDIA drivers are installed
+# FFmpeg with NVENC support will be used automatically
+```
-First, create a configuration file: select the drop down menu labeled `Config`, and click `Create New Config`. All settings, such as details about cameras, DLC networks, and DLC-live Processors, will be saved into configuration files so that you can close and reopen the GUI without losing all of these details. You can create multiple configuration files on the same system, so that different users can save different camera options, etc on the same computer. To load previous settings from a configuration file, please just select the file from the drop-down menu. Configuration files are stored at `$HOME/Documents/DeepLabCut-live-GUI/config`. These files do not need to be edited manually, they can be entirely created and edited automatically within the GUI.
+## Quick Start
-#### Set Up Cameras 
+1. **Launch the GUI**:
+ ```bash
+ dlclivegui
+ ```
-To setup a new camera, select `Add Camera` from the dropdown menu, and then click `Init Cam`. This will be bring up a new window where you need to select the type of camera (see [Camera Support](docs/camera_support.md)), input a name for the camera, and click `Add Camera`. This will initialize a new `Camera` entry in the drop down menu. Now, select your camera from the dropdown menu and click`Edit Camera Settings` to setup your camera settings (i.e. set the serial number, exposure, cropping parameters, etc; the exact settings depend on the specific type of camera). Once you have set the camera settings, click `Init Cam` to start streaming. To stop streaming data, click `Close Camera`, and to remove a camera from the dropdown menu, click `Remove Camera`.
+2. **Select Camera Backend**: Choose from the dropdown (opencv, gentl, aravis, basler)
-#### Processor (optional)
+3. **Configure Camera**: Set FPS, exposure, gain, and other parameters
-To write custom `Processors`, please see [here](https://github.com/DeepLabCut/DeepLabCut-live/tree/master/dlclive/processor). The directory that contains your custom `Processor` should be a python module -- this directory must contain an `__init__.py` file that imports your custom `Processor`. For examples of how to structure a custom `Processor` directory, please see [here](https://github.com/DeepLabCut/DeepLabCut-live/tree/master/example_processors).
+4. **Start Preview**: Click "Start Preview" to begin camera streaming
-To use your processor in the GUI, you must first add your custom `Processor` directory to the dropdown menu: next to the `Processor Dir` label, click `Browse`, and select your custom `Processor` directory. Next, select the desired directory from the `Processor Dir` dropdown menu, then select the `Processor` you would like to use from the `Processor` menu. If you would like to edit the arguments for your processor, please select `Edit Proc Settings`, and finally, to use the processor, click `Set Proc`. If you have previously set a `Processor` and would like to clear it, click `Clear Proc`.
+5. **Optional - Load DLC Model**: Browse to your exported DLCLive model directory
-#### Configure DeepLabCut Network
+6. **Optional - Start Inference**: Click "Start pose inference" for real-time tracking
-
+7. **Optional - Record Video**: Configure output path and click "Start recording"
-Select the `DeepLabCut` dropdown menu, and click `Add DLC`. This will bring up a new window to choose a name for the DeepLabCut configuration, choose the path to the exported DeepLabCut model, and set DeepLabCut-live settings, such as the cropping or resize parameters. Once configured, click `Update` to add this DeepLabCut configuration to the dropdown menu. You can edit the settings at any time by clicking `Edit DLC Settings`. Once configured, you can load the network and start performing inference by clicking `Start DLC`. If you would like to view the DeepLabCut pose estimation in real-time, select `Display DLC Keypoints`. You can edit the keypoint display settings (the color scheme, size of points, and the likelihood threshold for display) by selecting `Edit DLC Display Settings`.
+## Configuration
-If you want to stop performing inference at any time, just click `Stop DLC`, and if you want to remove a DeepLabCut configuration from the dropdown menu, click `Remove DLC`.
+The GUI uses a single JSON configuration file containing all experiment settings:
+
+```json
+{
+ "camera": {
+ "name": "Camera 0",
+ "index": 0,
+ "fps": 60.0,
+ "backend": "gentl",
+ "exposure": 10000,
+ "gain": 5.0,
+ "crop_x0": 0,
+ "crop_y0": 0,
+ "crop_x1": 0,
+ "crop_y1": 0,
+ "max_devices": 3,
+ "properties": {}
+ },
+ "dlc": {
+ "model_path": "/path/to/exported-model",
+ "model_type": "pytorch",
+ },
+ "recording": {
+ "enabled": true,
+ "directory": "~/Videos/deeplabcut-live",
+ "filename": "session.mp4",
+ "container": "mp4",
+ "codec": "h264_nvenc",
+ "crf": 23
+ },
+ "bbox": {
+ "enabled": false,
+ "x0": 0,
+ "y0": 0,
+ "x1": 200,
+ "y1": 100
+ }
+}
+```
+
+### Configuration Management
+
+- **Load**: File → Load configuration… (or Ctrl+O)
+- **Save**: File → Save configuration (or Ctrl+S)
+- **Save As**: File → Save configuration as… (or Ctrl+Shift+S)
+
+All GUI fields are automatically synchronized with the configuration file.
+
+## Camera Backends
+
+### Backend Selection Guide
+
+| Backend | Platform | Use Case | Auto-Detection |
+|---------|----------|----------|----------------|
+| **opencv** | All | Webcams, simple USB cameras | Basic |
+| **gentl** | Windows, Linux | Industrial cameras via CTI files | Yes |
+| **aravis** | Linux, macOS | GenICam/GigE cameras | Yes |
+| **basler** | All | Basler cameras specifically | Yes |
+
+### Backend-Specific Configuration
+
+#### OpenCV
+```json
+{
+ "camera": {
+ "backend": "opencv",
+ "index": 0,
+ "fps": 30.0
+ }
+}
+```
+**Note**: Exposure and gain controls are disabled for OpenCV backend due to limited driver support.
+
+#### GenTL (Harvesters)
+```json
+{
+ "camera": {
+ "backend": "gentl",
+ "index": 0,
+ "fps": 60.0,
+ "exposure": 15000,
+ "gain": 8.0,
+ }
+}
+```
-#### Set Up Session
-Sessions are defined by the subject name, the date, and an attempt number. Within the GUI, select a `Subject` from the dropdown menu, or to add a new subject, type the new subject name in to the entry box and click `Add Subject`. Next, select an `Attempt` from the dropdown menu. Then, select the directory that you would like to save data to from the `Directory` dropdown menu. To add a new directory to the dropdown menu, click `Browse`. Finally, click `Set Up Session` to initiate a new recording. This will prepare the GUI to save data. Once you click `Set Up Session`, the `Ready` button should turn blue, indicating a session is ready to record.
+See [Camera Backend Documentation](docs/camera_support.md) for detailed setup instructions.
-#### Controlling Recording
+## DLCLive Integration
-If the `Ready` button is selected, you can now start a recording by clicking `On`. The `On` button will then turn green indicating a recording is active. To stop a recording, click `Off`. This will cause the `Ready` button to be selected again, as the GUI is prepared to restart the recording and to save the data to the same file. If you're session is complete, click `Save Video` to save all files: the video recording (as .avi file), a numpy file with timestamps for each recorded frame, the DeepLabCut poses as a pandas data frame (hdf5 file) that includes the time of each frame used for pose estimation and the time that each pose was obtained, and if applicable, files saved by the `Processor` in use. These files will be saved into a new directory at `{YOUR_SAVE_DIRECTORY}/{CAMERA NAME}_{SUBJECT}_{DATE}_{ATTEMPT}`
+### Model Types
-- YOUR_SAVE_DIRECTORY : the directory chosen from the `Directory` dropdown menu.
-- CAMERA NAME : the name of selected camera (from the `Camera` dropdown menu).
-- SUBJECT : the subject chosen from the `Subject` drowdown menu.
-- DATE : the current date of the experiment.
-- ATTEMPT : the attempt number chosen from the `Attempt` dropdown.
+The GUI supports PyTorch DLCLive models:
-If you would not like to save the data from the session, please click `Delete Video`, and all data will be discarded. After you click `Save Video` or `Delete Video`, the `Off` button will be selected, indicating you can now set up a new session.
+1. **PyTorch**: PyTorch-based models (requires PyTorch installation)
-#### References:
+Select the model type from the dropdown before starting inference.
-If you use this code we kindly ask you to you please [cite Kane et al, eLife 2020](https://elifesciences.org/articles/61909). The preprint is available here: https://www.biorxiv.org/content/10.1101/2020.08.04.236422v2
+### Processor System
+
+The GUI includes a plugin system for custom pose processing:
+
+```python
+# Example processor
+class MyProcessor:
+ def process(self, pose, timestamp):
+ # Custom processing logic
+ x, y = pose[0, :2] # First keypoint
+ print(f"Position: ({x}, {y})")
+ def save(self):
+ pass
+```
+
+Place processors in `dlclivegui/processors/` and refresh to load them.
+
+See [Processor Plugin Documentation](docs/PLUGIN_SYSTEM.md) for details.
+
+### Auto-Recording Feature
+
+Enable "Auto-record video on processor command" to automatically start/stop recording based on processor signals. Useful for event-triggered recording in behavioral experiments.
+
+## Performance Optimization
+
+### High-Speed Camera Tips
+
+1. **Use Hardware Encoding**: Select `h264_nvenc` codec for NVIDIA GPUs
+2. **Adjust Buffer Count**: Increase buffers for GenTL/Aravis backends
+ ```json
+ "properties": {"n_buffers": 20}
+ ```
+3. **Optimize CRF**: Lower CRF = higher quality but larger files (default: 23)
+4. **Disable Visualization**: Uncheck "Display pose predictions" during recording
+5. **Crop Region**: Use cropping to reduce frame size before inference
+
+### Project Structure
+
+```
+dlclivegui/
+├── __init__.py
+├── gui.py # Main PySide6 application
+├── config.py # Configuration dataclasses
+├── camera_controller.py # Camera capture thread
+├── dlc_processor.py # DLCLive inference thread
+├── video_recorder.py # Video encoding thread
+├── cameras/ # Camera backend modules
+│ ├── base.py # Abstract base class
+│ ├── factory.py # Backend registry and detection
+│ ├── opencv_backend.py
+│ ├── gentl_backend.py
+│ ├── aravis_backend.py
+│ └── basler_backend.py
+└── processors/ # Pose processor plugins
+ ├── processor_utils.py
+ └── dlc_processor_socket.py
+```
+
+
+## Documentation
+
+- [Camera Support](docs/camera_support.md) - All camera backends and setup
+- [Aravis Backend](docs/aravis_backend.md) - GenICam camera setup (Linux/macOS)
+- [Processor Plugins](docs/PLUGIN_SYSTEM.md) - Custom pose processing
+- [Installation Guide](docs/install.md) - Detailed setup instructions
+- [Timestamp Format](docs/timestamp_format.md) - Timestamp synchronization
+
+## System Requirements
+
+
+### Recommended
+- Python 3.10+
+- 8 GB RAM
+- NVIDIA GPU with CUDA support (for DLCLive inference and video encoding)
+- USB 3.0 or GigE network (for industrial cameras)
+- SSD storage (for high-speed recording)
+
+### Tested Platforms
+- Windows 11
+
+## License
+
+This project is licensed under the GNU Lesser General Public License v3.0. See the [LICENSE](LICENSE) file for more information.
+
+## Citation
+
+Cite the original DeepLabCut-live paper:
+```bibtex
+@article{Kane2020,
+ title={Real-time, low-latency closed-loop feedback using markerless posture tracking},
+ author={Kane, Gary A and Lopes, Gonçalo and Saunders, Jonny L and Mathis, Alexander and Mathis, Mackenzie W},
+ journal={eLife},
+ year={2020},
+ doi={10.7554/eLife.61909}
+}
+```
diff --git a/dlclivegui/__init__.py b/dlclivegui/__init__.py
index 1583156..98c82aa 100644
--- a/dlclivegui/__init__.py
+++ b/dlclivegui/__init__.py
@@ -1,4 +1,26 @@
-from dlclivegui.camera_process import CameraProcess
-from dlclivegui.pose_process import CameraPoseProcess
-from dlclivegui.video import create_labeled_video
-from dlclivegui.dlclivegui import DLCLiveGUI
+"""DeepLabCut Live GUI package."""
+
+from .config import (
+ ApplicationSettings,
+ CameraSettings,
+ DLCProcessorSettings,
+ MultiCameraSettings,
+ RecordingSettings,
+)
+from .gui.camera_config_dialog import CameraConfigDialog
+from .gui.main_window import DLCLiveMainWindow
+from .main import main
+from .services.multi_camera_controller import MultiCameraController, MultiFrameData
+
+__all__ = [
+ "ApplicationSettings",
+ "CameraSettings",
+ "DLCProcessorSettings",
+ "MultiCameraSettings",
+ "RecordingSettings",
+ "DLCLiveMainWindow",
+ "MultiCameraController",
+ "MultiFrameData",
+ "CameraConfigDialog",
+ "main",
+]
diff --git a/dlclivegui/assets/logo.png b/dlclivegui/assets/logo.png
new file mode 100644
index 0000000..ec77b4a
Binary files /dev/null and b/dlclivegui/assets/logo.png differ
diff --git a/dlclivegui/assets/logo_transparent.png b/dlclivegui/assets/logo_transparent.png
new file mode 100644
index 0000000..45dd2a3
Binary files /dev/null and b/dlclivegui/assets/logo_transparent.png differ
diff --git a/dlclivegui/assets/welcome.png b/dlclivegui/assets/welcome.png
new file mode 100644
index 0000000..9afebe0
Binary files /dev/null and b/dlclivegui/assets/welcome.png differ
diff --git a/dlclivegui/camera/__init__.py b/dlclivegui/camera/__init__.py
deleted file mode 100644
index 2368198..0000000
--- a/dlclivegui/camera/__init__.py
+++ /dev/null
@@ -1,43 +0,0 @@
-"""
-DeepLabCut Toolbox (deeplabcut.org)
-© A. & M. Mathis Labs
-
-Licensed under GNU Lesser General Public License v3.0
-"""
-
-
-import platform
-
-from dlclivegui.camera.camera import Camera, CameraError
-from dlclivegui.camera.opencv import OpenCVCam
-
-if platform.system() == "Windows":
- try:
- from dlclivegui.camera.tiscamera_windows import TISCam
- except Exception as e:
- pass
-
-if platform.system() == "Linux":
- try:
- from dlclivegui.camera.tiscamera_linux import TISCam
- except Exception as e:
- pass
- # print(f"Error importing TISCam on Linux: {e}")
-
-if platform.system() in ["Darwin", "Linux"]:
- try:
- from dlclivegui.camera.aravis import AravisCam
- except Exception as e:
- pass
- # print(f"Error importing AravisCam: f{e}")
-
-if platform.system() == "Darwin":
- try:
- from dlclivegui.camera.pseye import PSEyeCam
- except Exception as e:
- pass
-
-try:
- from dlclivegui.camera.basler import BaslerCam
-except Exception as e:
- pass
diff --git a/dlclivegui/camera/aravis.py b/dlclivegui/camera/aravis.py
deleted file mode 100644
index 92662e1..0000000
--- a/dlclivegui/camera/aravis.py
+++ /dev/null
@@ -1,128 +0,0 @@
-"""
-DeepLabCut Toolbox (deeplabcut.org)
-© A. & M. Mathis Labs
-
-Licensed under GNU Lesser General Public License v3.0
-"""
-
-
-import ctypes
-import numpy as np
-import time
-
-import gi
-
-gi.require_version("Aravis", "0.6")
-from gi.repository import Aravis
-import cv2
-
-from dlclivegui.camera import Camera
-
-
-class AravisCam(Camera):
- @staticmethod
- def arg_restrictions():
-
- Aravis.update_device_list()
- n_cams = Aravis.get_n_devices()
- ids = [Aravis.get_device_id(i) for i in range(n_cams)]
- return {"id": ids}
-
- def __init__(
- self,
- id="",
- resolution=[720, 540],
- exposure=0.005,
- gain=0,
- rotate=0,
- crop=None,
- fps=100,
- display=True,
- display_resize=1.0,
- ):
-
- super().__init__(
- id,
- resolution=resolution,
- exposure=exposure,
- gain=gain,
- rotate=rotate,
- crop=crop,
- fps=fps,
- use_tk_display=display,
- display_resize=display_resize,
- )
-
- def set_capture_device(self):
-
- self.cam = Aravis.Camera.new(self.id)
- self.no_auto()
- self.set_exposure(self.exposure)
- self.set_crop(self.crop)
- self.cam.set_frame_rate(self.fps)
-
- self.stream = self.cam.create_stream()
- self.stream.push_buffer(Aravis.Buffer.new_allocate(self.cam.get_payload()))
- self.cam.start_acquisition()
-
- return True
-
- def no_auto(self):
-
- self.cam.set_exposure_time_auto(0)
- self.cam.set_gain_auto(0)
-
- def set_exposure(self, val):
-
- val = 1 if val > 1 else val
- val = 0 if val < 0 else val
- self.cam.set_exposure_time(val * 1e6)
-
- def set_crop(self, crop):
-
- if crop:
- left = crop[0]
- width = crop[1] - left
- top = crop[3]
- height = top - crop[2]
- self.cam.set_region(left, top, width, height)
- self.im_size = (width, height)
-
- def get_image_on_time(self):
-
- buffer = None
- while buffer is None:
- buffer = self.stream.try_pop_buffer()
-
- frame = self._convert_image_to_numpy(buffer)
- self.stream.push_buffer(buffer)
-
- return frame, time.time()
-
- def _convert_image_to_numpy(self, buffer):
- """ from https://github.com/SintefManufacturing/python-aravis """
-
- pixel_format = buffer.get_image_pixel_format()
- bits_per_pixel = pixel_format >> 16 & 0xFF
-
- if bits_per_pixel == 8:
- INTP = ctypes.POINTER(ctypes.c_uint8)
- else:
- INTP = ctypes.POINTER(ctypes.c_uint16)
-
- addr = buffer.get_data()
- ptr = ctypes.cast(addr, INTP)
-
- frame = np.ctypeslib.as_array(
- ptr, (buffer.get_image_height(), buffer.get_image_width())
- )
- frame = frame.copy()
-
- if frame.ndim < 3:
- frame = cv2.cvtColor(frame, cv2.COLOR_GRAY2RGB)
-
- return frame
-
- def close_capture_device():
-
- self.cam.stop_acquisition()
diff --git a/dlclivegui/camera/basler.py b/dlclivegui/camera/basler.py
deleted file mode 100644
index 1706208..0000000
--- a/dlclivegui/camera/basler.py
+++ /dev/null
@@ -1,104 +0,0 @@
-"""
-DeepLabCut Toolbox (deeplabcut.org)
-© A. & M. Mathis Labs
-
-Licensed under GNU Lesser General Public License v3.0
-"""
-
-#import pypylon as pylon
-from pypylon import pylon
-from imutils import rotate_bound
-import time
-
-from dlclivegui.camera import Camera, CameraError
-TIMEOUT = 100
-
-def get_devices():
- tlFactory = pylon.TlFactory.GetInstance()
- devices = tlFactory.EnumerateDevices()
- return devices
-
-class BaslerCam(Camera):
- @staticmethod
- def arg_restrictions():
- """ Returns a dictionary of arguments restrictions for DLCLiveGUI
- """
- devices = get_devices()
- device_ids = list(range(len(devices)))
- return {"device": device_ids, "display": [True, False]}
-
- def __init__(
- self,
- device=0,
- resolution=[640, 480],
- exposure=15000,
- rotate=0,
- crop=None,
- gain=0.0,
- fps=30,
- display=True,
- display_resize=1.0,
- ):
-
- super().__init__(
- device,
- resolution=resolution,
- exposure=exposure,
- rotate=rotate,
- crop=crop,
- gain=gain,
- fps=fps,
- use_tk_display=display,
- display_resize=display_resize,
- )
-
- self.display = display
-
- def set_capture_device(self):
-
- devices = get_devices()
- self.cam = pylon.InstantCamera(
- pylon.TlFactory.GetInstance().CreateDevice(devices[self.id])
- )
- self.cam.Open()
-
- self.cam.Gain.SetValue(self.gain)
- self.cam.ExposureTime.SetValue(self.exposure)
- self.cam.Width.SetValue(self.im_size[0])
- self.cam.Height.SetValue(self.im_size[1])
-
- self.cam.StartGrabbing(pylon.GrabStrategy_LatestImageOnly)
- self.converter = pylon.ImageFormatConverter()
- self.converter.OutputPixelFormat = pylon.PixelType_BGR8packed
- self.converter.OutputBitAlignment = pylon.OutputBitAlignment_MsbAligned
-
- return True
-
- def get_image(self):
- grabResult = self.cam.RetrieveResult(
- TIMEOUT, pylon.TimeoutHandling_ThrowException)
-
- frame = None
-
- if grabResult.GrabSucceeded():
-
- image = self.converter.Convert(grabResult)
- frame = image.GetArray()
-
- if self.rotate:
- frame = rotate_bound(frame, self.rotate)
- if self.crop:
- frame = frame[self.crop[2]: self.crop[3],
- self.crop[0]: self.crop[1]]
-
- else:
-
- raise CameraError("Basler Camera did not return an image!")
-
- grabResult.Release()
-
- return frame
-
- def close_capture_device(self):
-
- self.cam.StopGrabbing()
diff --git a/dlclivegui/camera/camera.py b/dlclivegui/camera/camera.py
deleted file mode 100644
index e81442f..0000000
--- a/dlclivegui/camera/camera.py
+++ /dev/null
@@ -1,139 +0,0 @@
-"""
-DeepLabCut Toolbox (deeplabcut.org)
-© A. & M. Mathis Labs
-
-Licensed under GNU Lesser General Public License v3.0
-"""
-
-
-import cv2
-import time
-
-
-class CameraError(Exception):
- """
- Exception for incorrect use of cameras
- """
-
- pass
-
-
-class Camera(object):
- """ Base camera class. Controls image capture, writing images to video, pose estimation and image display.
-
- Parameters
- ----------
- id : [type]
- camera id
- exposure : int, optional
- exposure time in microseconds, by default None
- gain : int, optional
- gain value, by default None
- rotate : [type], optional
- [description], by default None
- crop : list, optional
- camera cropping parameters: [left, right, top, bottom], by default None
- fps : float, optional
- frame rate in frames per second, by default None
- use_tk_display : bool, optional
- flag to use tk image display (if using GUI), by default False
- display_resize : float, optional
- factor to resize images if using opencv display (display is very slow for large images), by default None
- """
-
- @staticmethod
- def arg_restrictions():
- """ Returns a dictionary of arguments restrictions for DLCLiveGUI
- """
-
- return {}
-
- def __init__(
- self,
- id,
- resolution=None,
- exposure=None,
- gain=None,
- rotate=None,
- crop=None,
- fps=None,
- use_tk_display=False,
- display_resize=1.0,
- ):
- """ Constructor method
- """
-
- self.id = id
- self.exposure = exposure
- self.gain = gain
- self.rotate = rotate
- self.crop = [int(c) for c in crop] if crop else None
- self.set_im_size(resolution)
- self.fps = fps
- self.use_tk_display = use_tk_display
- self.display_resize = display_resize if display_resize else 1.0
- self.next_frame = 0
-
- def set_im_size(self, res):
- """[summary]
-
- Parameters
- ----------
- default : [, optional
- [description], by default None
-
- Raises
- ------
- DLCLiveCameraError
- throws error if resolution is not set
- """
-
- if not res:
- raise CameraError("Resolution is not set!")
-
- self.im_size = (
- (int(res[0]), int(res[1]))
- if self.crop is None
- else (self.crop[3] - self.crop[2], self.crop[1] - self.crop[0])
- )
-
- def set_capture_device(self):
- """ Sets frame capture device with desired properties
- """
-
- raise NotImplementedError
-
- def get_image_on_time(self):
- """ Gets an image from frame capture device at the appropriate time (according to fps).
-
- Returns
- -------
- `np.ndarray`
- image as a numpy array
- float
- timestamp at which frame was taken, obtained from :func:`time.time`
- """
-
- frame = None
- while frame is None:
- cur_time = time.time()
- if cur_time > self.next_frame:
- frame = self.get_image()
- timestamp = cur_time
- self.next_frame = max(
- self.next_frame + 1.0 / self.fps, cur_time + 0.5 / self.fps
- )
-
- return frame, timestamp
-
- def get_image(self):
- """ Gets image from frame capture device
- """
-
- raise NotImplementedError
-
- def close_capture_device(self):
- """ Closes frame capture device
- """
-
- raise NotImplementedError
diff --git a/dlclivegui/camera/opencv.py b/dlclivegui/camera/opencv.py
deleted file mode 100644
index 7ac96b2..0000000
--- a/dlclivegui/camera/opencv.py
+++ /dev/null
@@ -1,155 +0,0 @@
-"""
-DeepLabCut Toolbox (deeplabcut.org)
-© A. & M. Mathis Labs
-
-Licensed under GNU Lesser General Public License v3.0
-"""
-
-
-import cv2
-from tkinter import filedialog
-from imutils import rotate_bound
-import time
-import platform
-
-from dlclivegui.camera import Camera, CameraError
-
-
-class OpenCVCam(Camera):
- @staticmethod
- def arg_restrictions():
- """ Returns a dictionary of arguments restrictions for DLCLiveGUI
- """
-
- cap = cv2.VideoCapture()
- devs = [-1]
- avail = True
- while avail:
- cur_index = devs[-1] + 1
- avail = cap.open(cur_index)
- if avail:
- devs.append(cur_index)
- cap.release()
-
- return {"device": devs, "display": [True, False]}
-
- def __init__(
- self,
- device=-1,
- file="",
- resolution=[640, 480],
- auto_exposure=0,
- exposure=0,
- gain=0,
- rotate=0,
- crop=None,
- fps=30,
- display=True,
- display_resize=1.0,
- ):
-
- if device != -1:
- if file:
- raise DLCLiveCameraError(
- "A device and file were provided to OpenCVCam. Must initialize an OpenCVCam with either a device id or a video file."
- )
-
- self.video = False
- id = int(device)
-
- else:
- if not file:
- file = filedialog.askopenfilename(
- title="Select video file for DLC-live-GUI"
- )
- if not file:
- raise DLCLiveCameraError(
- "Neither a device nor file were provided to OpenCVCam. Must initialize an OpenCVCam with either a device id or a video file."
- )
-
- self.video = True
- cap = cv2.VideoCapture(file)
- resolution = (
- cap.get(cv2.CAP_PROP_FRAME_WIDTH),
- cap.get(cv2.CAP_PROP_FRAME_HEIGHT),
- )
- fps = cap.get(cv2.CAP_PROP_FPS)
- del cap
- id = file
-
- super().__init__(
- id,
- resolution=resolution,
- exposure=exposure,
- rotate=rotate,
- crop=crop,
- fps=fps,
- use_tk_display=display,
- display_resize=display_resize,
- )
- self.auto_exposure = auto_exposure
- self.gain = gain
-
- def set_capture_device(self):
-
- if not self.video:
-
- self.cap = (
- cv2.VideoCapture(self.id, cv2.CAP_V4L)
- if platform.system() == "Linux"
- else cv2.VideoCapture(self.id)
- )
- ret, frame = self.cap.read()
-
- if self.im_size:
- self.cap.set(cv2.CAP_PROP_FRAME_WIDTH, self.im_size[0])
- self.cap.set(cv2.CAP_PROP_FRAME_HEIGHT, self.im_size[1])
- if self.auto_exposure:
- self.cap.set(cv2.CAP_PROP_AUTO_EXPOSURE, self.auto_exposure)
- if self.exposure:
- self.cap.set(cv2.CAP_PROP_EXPOSURE, self.exposure)
- if self.gain:
- self.cap.set(cv2.CAP_PROP_GAIN, self.gain)
- if self.fps:
- self.cap.set(cv2.CAP_PROP_FPS, self.fps)
-
- else:
-
- self.cap = cv2.VideoCapture(self.id)
-
- # self.im_size = (self.cap.get(cv2.CAP_PROP_FRAME_WIDTH), self.cap.get(cv2.CAP_PROP_FRAME_HEIGHT))
- # self.fps = self.cap.get(cv2.CAP_PROP_FPS)
- self.last_cap_read = 0
-
- self.cv2_color = self.cap.get(cv2.CAP_PROP_MODE)
-
- return True
-
- def get_image_on_time(self):
-
- # if video, wait...
- if self.video:
- while time.time() - self.last_cap_read < (1.0 / self.fps):
- pass
-
- ret, frame = self.cap.read()
-
- if ret:
- if self.rotate:
- frame = rotate_bound(frame, self.rotate)
- if self.crop:
- frame = frame[self.crop[2] : self.crop[3], self.crop[0] : self.crop[1]]
-
- if frame.ndim == 3:
- if self.cv2_color == 1:
- frame = cv2.cvtColor(frame, cv2.COLOR_RGB2BGR)
-
- self.last_cap_read = time.time()
-
- return frame, self.last_cap_read
- else:
- raise CameraError("OpenCV VideoCapture.read did not return an image!")
-
- def close_capture_device(self):
-
- self.cap.release()
diff --git a/dlclivegui/camera/pseye.py b/dlclivegui/camera/pseye.py
deleted file mode 100644
index 4c79065..0000000
--- a/dlclivegui/camera/pseye.py
+++ /dev/null
@@ -1,101 +0,0 @@
-"""
-DeepLabCut Toolbox (deeplabcut.org)
-© A. & M. Mathis Labs
-
-Licensed under GNU Lesser General Public License v3.0
-"""
-
-
-import cv2
-from imutils import rotate_bound
-import numpy as np
-import pseyepy
-
-from dlclivegui.camera import Camera, CameraError
-
-
-class PSEyeCam(Camera):
- @staticmethod
- def arg_restrictions():
-
- return {
- "device": [i for i in range(pseyepy.cam_count())],
- "resolution": [[320, 240], [640, 480]],
- "fps": [30, 40, 50, 60, 75, 100, 125],
- "colour": [True, False],
- "auto_whitebalance": [True, False],
- }
-
- def __init__(
- self,
- device=0,
- resolution=[320, 240],
- exposure=100,
- gain=20,
- rotate=0,
- crop=None,
- fps=60,
- colour=False,
- auto_whitebalance=False,
- red_balance=125,
- blue_balance=125,
- green_balance=125,
- display=True,
- display_resize=1.0,
- ):
-
- super().__init__(
- device,
- resolution=resolution,
- exposure=exposure,
- gain=gain,
- rotate=rotate,
- crop=crop,
- fps=fps,
- use_tk_display=display,
- display_resize=display_resize,
- )
- self.colour = colour
- self.auto_whitebalance = auto_whitebalance
- self.red_balance = red_balance
- self.blue_balance = blue_balance
- self.green_balance = green_balance
-
- def set_capture_device(self):
-
- if self.im_size[0] == 320:
- res = pseyepy.Camera.RES_SMALL
- elif self.im_size[0] == 640:
- res = pseyepy.Camera.RES_LARGE
- else:
- raise CameraError(f"pseye resolution {self.im_size} not supported")
-
- self.cap = pseyepy.Camera(
- self.id,
- fps=self.fps,
- resolution=res,
- exposure=self.exposure,
- gain=self.gain,
- colour=self.colour,
- auto_whitebalance=self.auto_whitebalance,
- red_balance=self.red_balance,
- blue_balance=self.blue_balance,
- green_balance=self.green_balance,
- )
-
- return True
-
- def get_image_on_time(self):
-
- frame, _ = self.cap.read()
-
- if self.rotate != 0:
- frame = rotate_bound(frame, self.rotate)
- if self.crop:
- frame = frame[self.crop[2] : self.crop[3], self.crop[0] : self.crop[1]]
-
- return frame
-
- def close_capture_device(self):
-
- self.cap.end()
diff --git a/dlclivegui/camera/tiscamera_linux.py b/dlclivegui/camera/tiscamera_linux.py
deleted file mode 100644
index 5f0afd8..0000000
--- a/dlclivegui/camera/tiscamera_linux.py
+++ /dev/null
@@ -1,204 +0,0 @@
-"""
-DeepLabCut Toolbox (deeplabcut.org)
-© A. & M. Mathis Labs
-
-Licensed under GNU Lesser General Public License v3.0
-"""
-
-
-import warnings
-import numpy as np
-import time
-
-import gi
-
-gi.require_version("Tcam", "0.1")
-gi.require_version("Gst", "1.0")
-from gi.repository import Tcam, Gst, GLib, GObject
-
-from dlclivegui.camera import Camera
-
-
-class TISCam(Camera):
-
- FRAME_RATE_OPTIONS = [15, 30, 60, 120, 240, 480]
- FRAME_RATE_FRACTIONS = ["15/1", "30/1", "60/1", "120/1", "5000000/20833", "480/1"]
- IM_FORMAT = (720, 540)
- ROTATE_OPTIONS = ["identity", "90r", "180", "90l", "horiz", "vert"]
-
- @staticmethod
- def arg_restrictions():
-
- if not Gst.is_initialized():
- Gst.init()
-
- source = Gst.ElementFactory.make("tcambin")
- return {
- "serial_number": source.get_device_serials(),
- "fps": TISCam.FRAME_RATE_OPTIONS,
- "rotate": TISCam.ROTATE_OPTIONS,
- "color": [True, False],
- "display": [True, False],
- }
-
- def __init__(
- self,
- serial_number="",
- resolution=[720, 540],
- exposure=0.005,
- rotate="identity",
- crop=None,
- fps=120,
- color=False,
- display=True,
- tk_resize=1.0,
- ):
-
- super().__init__(
- serial_number,
- resolution=resolution,
- exposure=exposure,
- rotate=rotate,
- crop=crop,
- fps=fps,
- use_tk_display=(not display),
- display_resize=tk_resize,
- )
- self.color = color
- self.display = display
- self.sample_locked = False
- self.new_sample = False
-
- def no_auto(self):
-
- self.cam.set_tcam_property("Exposure Auto", GObject.Value(bool, False))
-
- def set_exposure(self, val):
-
- val = 1 if val > 1 else val
- val = 0 if val < 0 else val
- self.cam.set_tcam_property("Exposure", val * 1e6)
-
- def set_crop(self, crop):
-
- if crop:
- self.gst_crop = self.gst_pipeline.get_by_name("crop")
- self.gst_crop.set_property("left", crop[0])
- self.gst_crop.set_property("right", TISCam.IM_FORMAT[0] - crop[1])
- self.gst_crop.set_property("top", crop[2])
- self.gst_crop.set_property("bottom", TISCam.IM_FORMAT[1] - crop[3])
- self.im_size = (crop[3] - crop[2], crop[1] - crop[0])
-
- def set_rotation(self, val):
-
- if val:
- self.gst_rotate = self.gst_pipeline.get_by_name("rotate")
- self.gst_rotate.set_property("video-direction", val)
-
- def set_sink(self):
-
- self.gst_sink = self.gst_pipeline.get_by_name("sink")
- self.gst_sink.set_property("max-buffers", 1)
- self.gst_sink.set_property("drop", 1)
- self.gst_sink.set_property("emit-signals", True)
- self.gst_sink.connect("new-sample", self.get_image)
-
- def setup_gst(self, serial_number, fps):
-
- if not Gst.is_initialized():
- Gst.init()
-
- fps_index = np.where(
- [int(fps) == int(opt) for opt in TISCam.FRAME_RATE_OPTIONS]
- )[0][0]
- fps_frac = TISCam.FRAME_RATE_FRACTIONS[fps_index]
- fmat = "BGRx" if self.color else "GRAY8"
-
- pipeline = (
- "tcambin name=cam "
- "! videocrop name=crop "
- "! videoflip name=rotate "
- "! video/x-raw,format={},framerate={} ".format(fmat, fps_frac)
- )
-
- if self.display:
- pipe_sink = (
- "! tee name=t "
- "t. ! queue ! videoconvert ! ximagesink "
- "t. ! queue ! appsink name=sink"
- )
- else:
- pipe_sink = "! appsink name=sink"
-
- pipeline += pipe_sink
-
- self.gst_pipeline = Gst.parse_launch(pipeline)
-
- self.cam = self.gst_pipeline.get_by_name("cam")
- self.cam.set_property("serial", serial_number)
-
- self.set_exposure(self.exposure)
- self.set_crop(self.crop)
- self.set_rotation(self.rotate)
- self.set_sink()
-
- def set_capture_device(self):
-
- self.setup_gst(self.id, self.fps)
- self.gst_pipeline.set_state(Gst.State.PLAYING)
-
- return True
-
- def get_image(self, sink):
-
- # wait for sample to unlock
- while self.sample_locked:
- pass
-
- try:
-
- self.sample = sink.get_property("last-sample")
- self._convert_image_to_numpy()
-
- except GLib.Error as e:
-
- warnings.warn("Error reading image :: {}".format(e))
-
- finally:
-
- return 0
-
- def _convert_image_to_numpy(self):
-
- self.sample_locked = True
-
- buffer = self.sample.get_buffer()
- struct = self.sample.get_caps().get_structure(0)
-
- height = struct.get_value("height")
- width = struct.get_value("width")
- fmat = struct.get_value("format")
- dtype = np.uint16 if fmat == "GRAY16_LE" else np.uint8
- ncolors = 1 if "GRAY" in fmat else 4
-
- self.frame = np.ndarray(
- shape=(height, width, ncolors),
- buffer=buffer.extract_dup(0, buffer.get_size()),
- dtype=dtype,
- )
-
- self.sample_locked = False
- self.new_sample = True
-
- def get_image_on_time(self):
-
- # wait for new sample
- while not self.new_sample:
- pass
- self.new_sample = False
-
- return self.frame, time.time()
-
- def close_capture_device(self):
-
- self.gst_pipeline.set_state(Gst.State.NULL)
diff --git a/dlclivegui/camera/tiscamera_windows.py b/dlclivegui/camera/tiscamera_windows.py
deleted file mode 100644
index bac6359..0000000
--- a/dlclivegui/camera/tiscamera_windows.py
+++ /dev/null
@@ -1,129 +0,0 @@
-"""
-DeepLabCut Toolbox (deeplabcut.org)
-© A. & M. Mathis Labs
-
-Licensed under GNU Lesser General Public License v3.0
-"""
-
-import time
-import cv2
-
-from dlclivegui.camera import Camera, CameraError
-from dlclivegui.camera.tisgrabber_windows import TIS_CAM
-
-
-class TISCam(Camera):
- @staticmethod
- def arg_restrictions():
-
- return {"serial_number": TIS_CAM().GetDevices(), "rotate": [0, 90, 180, 270]}
-
- def __init__(
- self,
- serial_number="",
- resolution=[720, 540],
- exposure=0.005,
- rotate=0,
- crop=None,
- fps=100,
- display=True,
- display_resize=1.0,
- ):
- """
- Params
- ------
- serial_number = string; serial number for imaging source camera
- crop = dict; contains ints named top, left, height, width for cropping
- default = None, uses default parameters specific to camera
- """
-
- if (rotate == 90) or (rotate == 270):
- resolution = [resolution[1], resolution[0]]
-
- super().__init__(
- serial_number,
- resolution=resolution,
- exposure=exposure,
- rotate=rotate,
- crop=crop,
- fps=fps,
- use_tk_display=display,
- display_resize=display_resize,
- )
- self.display = display
-
- def set_exposure(self):
-
- val = self.exposure
- val = 1 if val > 1 else val
- val = 0 if val < 0 else val
- self.cam.SetPropertyAbsoluteValue("Exposure", "Value", val)
-
- def get_exposure(self):
-
- exposure = [0]
- self.cam.GetPropertyAbsoluteValue("Exposure", "Value", exposure)
- return round(exposure[0], 3)
-
- # def set_crop(self):
-
- # crop = self.crop
-
- # if crop:
- # top = int(crop[0])
- # left = int(crop[2])
- # height = int(crop[1]-top)
- # width = int(crop[3]-left)
-
- # if not self.crop_filter:
- # self.crop_filter = self.cam.CreateFrameFilter(b'ROI')
- # self.cam.AddFrameFilter(self.crop_filter)
-
- # self.cam.FilterSetParameter(self.crop_filter, b'Top', top)
- # self.cam.FilterSetParameter(self.crop_filter, b'Left', left)
- # self.cam.FilterSetParameter(self.crop_filter, b'Height', height)
- # self.cam.FilterSetParameter(self.crop_filter, b'Width', width)
-
- def set_rotation(self):
-
- if not self.rotation_filter:
- self.rotation_filter = self.cam.CreateFrameFilter(b"Rotate Flip")
- self.cam.AddFrameFilter(self.rotation_filter)
- self.cam.FilterSetParameter(
- self.rotation_filter, b"Rotation Angle", self.rotate
- )
-
- def set_fps(self):
-
- self.cam.SetFrameRate(self.fps)
-
- def set_capture_device(self):
-
- self.cam = TIS_CAM()
- self.crop_filter = None
- self.rotation_filter = None
- self.set_rotation()
- # self.set_crop()
- self.set_fps()
- self.next_frame = time.time()
-
- self.cam.open(self.id)
- self.cam.SetContinuousMode(0)
- self.cam.StartLive(0)
-
- self.set_exposure()
-
- return True
-
- def get_image(self):
-
- self.cam.SnapImage()
- frame = self.cam.GetImageEx()
- frame = cv2.flip(frame, 0)
- if self.crop is not None:
- frame = frame[self.crop[0] : self.crop[1], self.crop[2] : self.crop[3]]
- return frame
-
- def close_capture_device(self):
-
- self.cam.StopLive()
diff --git a/dlclivegui/camera/tisgrabber_windows.py b/dlclivegui/camera/tisgrabber_windows.py
deleted file mode 100644
index 194e18e..0000000
--- a/dlclivegui/camera/tisgrabber_windows.py
+++ /dev/null
@@ -1,781 +0,0 @@
-"""
-Created on Mon Nov 21 09:44:40 2016
-
-@author: Daniel Vassmer, Stefan_Geissler
-From: https://github.com/TheImagingSource/IC-Imaging-Control-Samples/tree/master/Python
-
-modified 10/3/2019 by Gary Kane - https://github.com/gkane26
-"""
-
-"""
-DeepLabCut Toolbox (deeplabcut.org)
-© A. & M. Mathis Labs
-
-Licensed under GNU Lesser General Public License v3.0
-"""
-
-
-from enum import Enum
-
-import ctypes as C
-import os
-import sys
-import numpy as np
-
-
-class SinkFormats(Enum):
- Y800 = 0
- RGB24 = 1
- RGB32 = 2
- UYVY = 3
- Y16 = 4
-
-
-ImageFileTypes = {"BMP": 0, "JPEG": 1}
-
-
-class GrabberHandle(C.Structure):
- pass
-
-
-GrabberHandle._fields_ = [("unused", C.c_int)]
-
-##############################################################################
-
-### GK Additions from: https://github.com/morefigs/py-ic-imaging-control
-
-
-class FilterParameter(C.Structure):
- pass
-
-
-FilterParameter._fields_ = [("Name", C.c_char * 30), ("Type", C.c_int)]
-
-
-class FrameFilterHandle(C.Structure):
- pass
-
-
-FrameFilterHandle._fields_ = [
- ("pFilter", C.c_void_p),
- ("bHasDialog", C.c_int),
- ("ParameterCount", C.c_int),
- ("Parameters", C.POINTER(FilterParameter)),
-]
-
-##############################################################################
-
-
-class TIS_GrabberDLL(object):
- if sys.maxsize > 2 ** 32:
- __tisgrabber = C.windll.LoadLibrary("tisgrabber_x64.dll")
- else:
- __tisgrabber = C.windll.LoadLibrary("tisgrabber.dll")
-
- def __init__(self, **keyargs):
- """Initialize the Albatross from the keyword arguments."""
- self.__dict__.update(keyargs)
-
- GrabberHandlePtr = C.POINTER(GrabberHandle)
-
- ####################################
-
- # Initialize the ICImagingControl class library. This function must be called
- # only once before any other functions of this library are called.
- # @param szLicenseKey IC Imaging Control license key or NULL if only a trial version is available.
- # @retval IC_SUCCESS on success.
- # @retval IC_ERROR on wrong license key or other errors.
- # @sa IC_CloseLibrary
- InitLibrary = __tisgrabber.IC_InitLibrary(None)
-
- # Get the number of the currently available devices. This function creates an
- # internal array of all connected video capture devices. With each call to this
- # function, this array is rebuild. The name and the unique name can be retrieved
- # from the internal array using the functions IC_GetDevice() and IC_GetUniqueNamefromList.
- # They are usefull for retrieving device names for opening devices.
- #
- # @retval >= 0 Success, count of found devices.
- # @retval IC_NO_HANDLE Internal Error.
- #
- # @sa IC_GetDevice
- # @sa IC_GetUniqueNamefromList
- get_devicecount = __tisgrabber.IC_GetDeviceCount
- get_devicecount.restype = C.c_int
- get_devicecount.argtypes = None
-
- # Get unique device name of a device specified by iIndex. The unique device name
- # consist from the device name and its serial number. It allows to differ between
- # more then one device of the same type connected to the computer. The unique device name
- # is passed to the function IC_OpenDevByUniqueName
- #
- # @param iIndex The number of the device whose name is to be returned. It must be
- # in the range from 0 to IC_GetDeviceCount(),
- # @return Returns the string representation of the device on success, NULL
- # otherwise.
- #
- # @sa IC_GetDeviceCount
- # @sa IC_GetUniqueNamefromList
- # @sa IC_OpenDevByUniqueName
-
- get_unique_name_from_list = __tisgrabber.IC_GetUniqueNamefromList
- get_unique_name_from_list.restype = C.c_char_p
- get_unique_name_from_list.argtypes = (C.c_int,)
-
- # Creates a new grabber handle and returns it. A new created grabber should be
- # release with a call to IC_ReleaseGrabber if it is no longer needed.
- # @sa IC_ReleaseGrabber
- create_grabber = __tisgrabber.IC_CreateGrabber
- create_grabber.restype = GrabberHandlePtr
- create_grabber.argtypes = None
-
- # Open a video capture by using its UniqueName. Use IC_GetUniqueName() to
- # retrieve the unique name of a camera.
- #
- # @param hGrabber Handle to a grabber object
- # @param szDisplayName Memory that will take the display name.
- #
- # @sa IC_GetUniqueName
- # @sa IC_ReleaseGrabber
- open_device_by_unique_name = __tisgrabber.IC_OpenDevByUniqueName
- open_device_by_unique_name.restype = C.c_int
- open_device_by_unique_name.argtypes = (GrabberHandlePtr, C.c_char_p)
-
- set_videoformat = __tisgrabber.IC_SetVideoFormat
- set_videoformat.restype = C.c_int
- set_videoformat.argtypes = (GrabberHandlePtr, C.c_char_p)
-
- set_framerate = __tisgrabber.IC_SetFrameRate
- set_framerate.restype = C.c_int
- set_framerate.argtypes = (GrabberHandlePtr, C.c_float)
-
- # Returns the width of the video format.
- get_video_format_width = __tisgrabber.IC_GetVideoFormatWidth
- get_video_format_width.restype = C.c_int
- get_video_format_width.argtypes = (GrabberHandlePtr,)
-
- # returns the height of the video format.
- get_video_format_height = __tisgrabber.IC_GetVideoFormatHeight
- get_video_format_height.restype = C.c_int
- get_video_format_height.argtypes = (GrabberHandlePtr,)
-
- # Get the number of the available video formats for the current device.
- # A video capture device must have been opened before this call.
- #
- # @param hGrabber The handle to the grabber object.
- #
- # @retval >= 0 Success
- # @retval IC_NO_DEVICE No video capture device selected.
- # @retval IC_NO_HANDLE No handle to the grabber object.
- #
- # @sa IC_GetVideoFormat
- GetVideoFormatCount = __tisgrabber.IC_GetVideoFormatCount
- GetVideoFormatCount.restype = C.c_int
- GetVideoFormatCount.argtypes = (GrabberHandlePtr,)
-
- # Get a string representation of the video format specified by iIndex.
- # iIndex must be between 0 and IC_GetVideoFormatCount().
- # IC_GetVideoFormatCount() must have been called before this function,
- # otherwise it will always fail.
- #
- # @param hGrabber The handle to the grabber object.
- # @param iIndex Number of the video format to be used.
- #
- # @retval Nonnull The name of the specified video format.
- # @retval NULL An error occured.
- # @sa IC_GetVideoFormatCount
- GetVideoFormat = __tisgrabber.IC_GetVideoFormat
- GetVideoFormat.restype = C.c_char_p
- GetVideoFormat.argtypes = (GrabberHandlePtr, C.c_int)
-
- # Get the number of the available input channels for the current device.
- # A video capture device must have been opened before this call.
- #
- # @param hGrabber The handle to the grabber object.
- #
- # @retval >= 0 Success
- # @retval IC_NO_DEVICE No video capture device selected.
- # @retval IC_NO_HANDLE No handle to the grabber object.
- #
- # @sa IC_GetInputChannel
- GetInputChannelCount = __tisgrabber.IC_GetInputChannelCount
- GetInputChannelCount.restype = C.c_int
- GetInputChannelCount.argtypes = (GrabberHandlePtr,)
-
- # Get a string representation of the input channel specified by iIndex.
- # iIndex must be between 0 and IC_GetInputChannelCount().
- # IC_GetInputChannelCount() must have been called before this function,
- # otherwise it will always fail.
- # @param hGrabber The handle to the grabber object.
- # @param iIndex Number of the input channel to be used..
- #
- # @retval Nonnull The name of the specified input channel
- # @retval NULL An error occured.
- # @sa IC_GetInputChannelCount
- GetInputChannel = __tisgrabber.IC_GetInputChannel
- GetInputChannel.restype = C.c_char_p
- GetInputChannel.argtypes = (GrabberHandlePtr, C.c_int)
-
- # Get the number of the available video norms for the current device.
- # A video capture device must have been opened before this call.
- #
- # @param hGrabber The handle to the grabber object.
- #
- # @retval >= 0 Success
- # @retval IC_NO_DEVICE No video capture device selected.
- # @retval IC_NO_HANDLE No handle to the grabber object.
- #
- # @sa IC_GetVideoNorm
- GetVideoNormCount = __tisgrabber.IC_GetVideoNormCount
- GetVideoNormCount.restype = C.c_int
- GetVideoNormCount.argtypes = (GrabberHandlePtr,)
-
- # Get a string representation of the video norm specified by iIndex.
- # iIndex must be between 0 and IC_GetVideoNormCount().
- # IC_GetVideoNormCount() must have been called before this function,
- # otherwise it will always fail.
- #
- # @param hGrabber The handle to the grabber object.
- # @param iIndex Number of the video norm to be used.
- #
- # @retval Nonnull The name of the specified video norm.
- # @retval NULL An error occured.
- # @sa IC_GetVideoNormCount
- GetVideoNorm = __tisgrabber.IC_GetVideoNorm
- GetVideoNorm.restype = C.c_char_p
- GetVideoNorm.argtypes = (GrabberHandlePtr, C.c_int)
-
- SetFormat = __tisgrabber.IC_SetFormat
- SetFormat.restype = C.c_int
- SetFormat.argtypes = (GrabberHandlePtr, C.c_int)
- GetFormat = __tisgrabber.IC_GetFormat
- GetFormat.restype = C.c_int
- GetFormat.argtypes = (GrabberHandlePtr,)
-
- # Start the live video.
- # @param hGrabber The handle to the grabber object.
- # @param iShow The parameter indicates: @li 1 : Show the video @li 0 : Do not show the video, but deliver frames. (For callbacks etc.)
- # @retval IC_SUCCESS on success
- # @retval IC_ERROR if something went wrong.
- # @sa IC_StopLive
-
- StartLive = __tisgrabber.IC_StartLive
- StartLive.restype = C.c_int
- StartLive.argtypes = (GrabberHandlePtr, C.c_int)
-
- StopLive = __tisgrabber.IC_StopLive
- StopLive.restype = C.c_int
- StopLive.argtypes = (GrabberHandlePtr,)
-
- SetHWND = __tisgrabber.IC_SetHWnd
- SetHWND.restype = C.c_int
- SetHWND.argtypes = (GrabberHandlePtr, C.c_int)
-
- # Snaps an image. The video capture device must be set to live mode and a
- # sink type has to be set before this call. The format of the snapped images depend on
- # the selected sink type.
- #
- # @param hGrabber The handle to the grabber object.
- # @param iTimeOutMillisek The Timeout time is passed in milli seconds. A value of -1 indicates, that
- # no time out is set.
- #
- #
- # @retval IC_SUCCESS if an image has been snapped
- # @retval IC_ERROR if something went wrong.
- # @retval IC_NOT_IN_LIVEMODE if the live video has not been started.
- #
- # @sa IC_StartLive
- # @sa IC_SetFormat
-
- SnapImage = __tisgrabber.IC_SnapImage
- SnapImage.restype = C.c_int
- SnapImage.argtypes = (GrabberHandlePtr, C.c_int)
-
- # Retrieve the properties of the current video format and sink type
- # @param hGrabber The handle to the grabber object.
- # @param *lWidth This recieves the width of the image buffer.
- # @param *lHeight This recieves the height of the image buffer.
- # @param *iBitsPerPixel This recieves the count of bits per pixel.
- # @param *format This recieves the current color format.
- # @retval IC_SUCCESS on success
- # @retval IC_ERROR if something went wrong.
-
- GetImageDescription = __tisgrabber.IC_GetImageDescription
- GetImageDescription.restype = C.c_int
- GetImageDescription.argtypes = (
- GrabberHandlePtr,
- C.POINTER(C.c_long),
- C.POINTER(C.c_long),
- C.POINTER(C.c_int),
- C.POINTER(C.c_int),
- )
-
- GetImagePtr = __tisgrabber.IC_GetImagePtr
- GetImagePtr.restype = C.c_void_p
- GetImagePtr.argtypes = (GrabberHandlePtr,)
-
- # ############################################################################
- ShowDeviceSelectionDialog = __tisgrabber.IC_ShowDeviceSelectionDialog
- ShowDeviceSelectionDialog.restype = GrabberHandlePtr
- ShowDeviceSelectionDialog.argtypes = (GrabberHandlePtr,)
-
- # ############################################################################
-
- ShowPropertyDialog = __tisgrabber.IC_ShowPropertyDialog
- ShowPropertyDialog.restype = GrabberHandlePtr
- ShowPropertyDialog.argtypes = (GrabberHandlePtr,)
-
- # ############################################################################
- IsDevValid = __tisgrabber.IC_IsDevValid
- IsDevValid.restype = C.c_int
- IsDevValid.argtypes = (GrabberHandlePtr,)
-
- # ############################################################################
-
- LoadDeviceStateFromFile = __tisgrabber.IC_LoadDeviceStateFromFile
- LoadDeviceStateFromFile.restype = GrabberHandlePtr
- LoadDeviceStateFromFile.argtypes = (GrabberHandlePtr, C.c_char_p)
-
- # ############################################################################
- SaveDeviceStateToFile = __tisgrabber.IC_SaveDeviceStateToFile
- SaveDeviceStateToFile.restype = C.c_int
- SaveDeviceStateToFile.argtypes = (GrabberHandlePtr, C.c_char_p)
-
- GetCameraProperty = __tisgrabber.IC_GetCameraProperty
- GetCameraProperty.restype = C.c_int
- GetCameraProperty.argtypes = (GrabberHandlePtr, C.c_int, C.POINTER(C.c_long))
-
- SetCameraProperty = __tisgrabber.IC_SetCameraProperty
- SetCameraProperty.restype = C.c_int
- SetCameraProperty.argtypes = (GrabberHandlePtr, C.c_int, C.c_long)
-
- SetPropertyValue = __tisgrabber.IC_SetPropertyValue
- SetPropertyValue.restype = C.c_int
- SetPropertyValue.argtypes = (GrabberHandlePtr, C.c_char_p, C.c_char_p, C.c_int)
-
- GetPropertyValue = __tisgrabber.IC_GetPropertyValue
- GetPropertyValue.restype = C.c_int
- GetPropertyValue.argtypes = (
- GrabberHandlePtr,
- C.c_char_p,
- C.c_char_p,
- C.POINTER(C.c_long),
- )
-
- # ############################################################################
- SetPropertySwitch = __tisgrabber.IC_SetPropertySwitch
- SetPropertySwitch.restype = C.c_int
- SetPropertySwitch.argtypes = (GrabberHandlePtr, C.c_char_p, C.c_char_p, C.c_int)
-
- GetPropertySwitch = __tisgrabber.IC_GetPropertySwitch
- GetPropertySwitch.restype = C.c_int
- GetPropertySwitch.argtypes = (
- GrabberHandlePtr,
- C.c_char_p,
- C.c_char_p,
- C.POINTER(C.c_long),
- )
- # ############################################################################
-
- IsPropertyAvailable = __tisgrabber.IC_IsPropertyAvailable
- IsPropertyAvailable.restype = C.c_int
- IsPropertyAvailable.argtypes = (GrabberHandlePtr, C.c_char_p, C.c_char_p)
-
- PropertyOnePush = __tisgrabber.IC_PropertyOnePush
- PropertyOnePush.restype = C.c_int
- PropertyOnePush.argtypes = (GrabberHandlePtr, C.c_char_p, C.c_char_p)
-
- SetPropertyAbsoluteValue = __tisgrabber.IC_SetPropertyAbsoluteValue
- SetPropertyAbsoluteValue.restype = C.c_int
- SetPropertyAbsoluteValue.argtypes = (
- GrabberHandlePtr,
- C.c_char_p,
- C.c_char_p,
- C.c_float,
- )
-
- GetPropertyAbsoluteValue = __tisgrabber.IC_GetPropertyAbsoluteValue
- GetPropertyAbsoluteValue.restype = C.c_int
- GetPropertyAbsoluteValue.argtypes = (
- GrabberHandlePtr,
- C.c_char_p,
- C.c_char_p,
- C.POINTER(C.c_float),
- )
-
- # definition of the frameready callback
- FRAMEREADYCALLBACK = C.CFUNCTYPE(
- C.c_void_p, C.c_int, C.POINTER(C.c_ubyte), C.c_ulong, C.py_object
- )
-
- # set callback function
- SetFrameReadyCallback = __tisgrabber.IC_SetFrameReadyCallback
- SetFrameReadyCallback.restype = C.c_int
- SetFrameReadyCallback.argtypes = [GrabberHandlePtr, FRAMEREADYCALLBACK, C.py_object]
-
- SetContinuousMode = __tisgrabber.IC_SetContinuousMode
-
- SaveImage = __tisgrabber.IC_SaveImage
- SaveImage.restype = C.c_int
- SaveImage.argtypes = [C.c_void_p, C.c_char_p, C.c_int, C.c_int]
-
- OpenVideoCaptureDevice = __tisgrabber.IC_OpenVideoCaptureDevice
- OpenVideoCaptureDevice.restype = C.c_int
- OpenVideoCaptureDevice.argtypes = [C.c_void_p, C.c_char_p]
-
- # ############################################################################
-
- ### GK Additions - adding frame filters. Pieces copied from: https://github.com/morefigs/py-ic-imaging-control
-
- CreateFrameFilter = __tisgrabber.IC_CreateFrameFilter
- CreateFrameFilter.restype = C.c_int
- CreateFrameFilter.argtypes = (C.c_char_p, C.POINTER(FrameFilterHandle))
-
- AddFrameFilter = __tisgrabber.IC_AddFrameFilterToDevice
- AddFrameFilter.restype = C.c_int
- AddFrameFilter.argtypes = (GrabberHandlePtr, C.POINTER(FrameFilterHandle))
-
- FilterGetParameter = __tisgrabber.IC_FrameFilterGetParameter
- FilterGetParameter.restype = C.c_int
- FilterGetParameter.argtypes = (C.POINTER(FrameFilterHandle), C.c_char_p, C.c_void_p)
-
- FilterSetParameter = __tisgrabber.IC_FrameFilterSetParameterInt
- FilterSetParameter.restype = C.c_int
- FilterSetParameter.argtypes = (C.POINTER(FrameFilterHandle), C.c_char_p, C.c_int)
-
-
-# ############################################################################
-
-
-class TIS_CAM(object):
- @property
- def callback_registered(self):
- return self._callback_registered
-
- def __init__(self):
-
- self._handle = C.POINTER(GrabberHandle)
- self._handle = TIS_GrabberDLL.create_grabber()
- self._callback_registered = False
- self._frame = {"num": -1, "ready": False}
-
- def s(self, strin):
- if sys.version[0] == "2":
- return strin
- if type(strin) == "byte":
- return strin
- return strin.encode("utf-8")
-
- def SetFrameReadyCallback(self, CallbackFunction, data):
- """ Set a callback function, which is called, when a new frame arrives.
-
- CallbackFunction : The callback function
-
- data : a self defined class with user data.
- """
- return TIS_GrabberDLL.SetFrameReadyCallback(
- self._handle, CallbackFunction, data
- )
-
- def SetContinuousMode(self, Mode):
- """ Determines, whether new frames are automatically copied into memory.
-
- :param Mode: If 0, all frames are copied automatically into memory. This is recommened, if the camera runs in trigger mode.
- If 1, then snapImages must be called to get a frame into memory.
- :return: None
- """
- return TIS_GrabberDLL.SetContinuousMode(self._handle, Mode)
-
- def open(self, unique_device_name):
- """ Open a device
-
- unique_device_name : The name and serial number of the device to be opened. The device name and serial number are separated by a space.
- """
- test = TIS_GrabberDLL.open_device_by_unique_name(
- self._handle, self.s(unique_device_name)
- )
-
- return test
-
- def close(self):
- TIS_GrabberDLL.close_device(self._handle)
-
- def ShowDeviceSelectionDialog(self):
- self._handle = TIS_GrabberDLL.ShowDeviceSelectionDialog(self._handle)
-
- def ShowPropertyDialog(self):
- self._handle = TIS_GrabberDLL.ShowPropertyDialog(self._handle)
-
- def IsDevValid(self):
- return TIS_GrabberDLL.IsDevValid(self._handle)
-
- def SetHWND(self, Hwnd):
- return TIS_GrabberDLL.SetHWND(self._handle, Hwnd)
-
- def SaveDeviceStateToFile(self, FileName):
- return TIS_GrabberDLL.SaveDeviceStateToFile(self._handle, self.s(FileName))
-
- def LoadDeviceStateFromFile(self, FileName):
- self._handle = TIS_GrabberDLL.LoadDeviceStateFromFile(
- self._handle, self.s(FileName)
- )
-
- def SetVideoFormat(self, Format):
- return TIS_GrabberDLL.set_videoformat(self._handle, self.s(Format))
-
- def SetFrameRate(self, FPS):
- return TIS_GrabberDLL.set_framerate(self._handle, FPS)
-
- def get_video_format_width(self):
- return TIS_GrabberDLL.get_video_format_width(self._handle)
-
- def get_video_format_height(self):
- return TIS_GrabberDLL.get_video_format_height(self._handle)
-
- def GetDevices(self):
- self._Devices = []
- iDevices = TIS_GrabberDLL.get_devicecount()
- for i in range(iDevices):
- self._Devices.append(TIS_GrabberDLL.get_unique_name_from_list(i))
- return self._Devices
-
- def GetVideoFormats(self):
- self._Properties = []
- iVideoFormats = TIS_GrabberDLL.GetVideoFormatCount(self._handle)
- for i in range(iVideoFormats):
- self._Properties.append(TIS_GrabberDLL.GetVideoFormat(self._handle, i))
- return self._Properties
-
- def GetInputChannels(self):
- self.InputChannels = []
- InputChannelscount = TIS_GrabberDLL.GetInputChannelCount(self._handle)
- for i in range(InputChannelscount):
- self.InputChannels.append(TIS_GrabberDLL.GetInputChannel(self._handle, i))
- return self.InputChannels
-
- def GetVideoNormCount(self):
- self.GetVideoNorm = []
- GetVideoNorm_Count = TIS_GrabberDLL.GetVideoNormCount(self._handle)
- for i in range(GetVideoNorm_Count):
- self.GetVideoNorm.append(TIS_GrabberDLL.GetVideoNorm(self._handle, i))
- return self.GetVideoNorm
-
- def SetFormat(self, Format):
- """ SetFormat
- Sets the pixel format in memory
- @param Format Sinkformat enumeration
- """
- TIS_GrabberDLL.SetFormat(self._handle, Format.value)
-
- def GetFormat(self):
- val = TIS_GrabberDLL.GetFormat(self._handle)
- if val == 0:
- return SinkFormats.Y800
- if val == 2:
- return SinkFormats.RGB32
- if val == 1:
- return SinkFormats.RGB24
- if val == 3:
- return SinkFormats.UYVY
- if val == 4:
- return SinkFormats.Y16
- return SinkFormats.RGB24
-
- def StartLive(self, showlive=1):
- """
- Start the live video stream.
-
- showlive: 1 : a live video is shown, 0 : the live video is not shown.
- """
- Error = TIS_GrabberDLL.StartLive(self._handle, showlive)
- return Error
-
- def StopLive(self):
- """
- Stop the live video.
- """
- Error = TIS_GrabberDLL.StopLive(self._handle)
- return Error
-
- def SnapImage(self):
- Error = TIS_GrabberDLL.SnapImage(self._handle, 2000)
- return Error
-
- def GetImageDescription(self):
- lWidth = C.c_long()
- lHeight = C.c_long()
- iBitsPerPixel = C.c_int()
- COLORFORMAT = C.c_int()
-
- Error = TIS_GrabberDLL.GetImageDescription(
- self._handle, lWidth, lHeight, iBitsPerPixel, COLORFORMAT
- )
- return (lWidth.value, lHeight.value, iBitsPerPixel.value, COLORFORMAT.value)
-
- def GetImagePtr(self):
- ImagePtr = TIS_GrabberDLL.GetImagePtr(self._handle)
-
- return ImagePtr
-
- def GetImage(self):
- BildDaten = self.GetImageDescription()[:4]
- lWidth = BildDaten[0]
- lHeight = BildDaten[1]
- iBitsPerPixel = BildDaten[2] // 8
-
- buffer_size = lWidth * lHeight * iBitsPerPixel * C.sizeof(C.c_uint8)
- img_ptr = self.GetImagePtr()
-
- Bild = C.cast(img_ptr, C.POINTER(C.c_ubyte * buffer_size))
-
- img = np.ndarray(
- buffer=Bild.contents, dtype=np.uint8, shape=(lHeight, lWidth, iBitsPerPixel)
- )
- return img
-
- def GetImageEx(self):
- """ Return a numpy array with the image data tyes
- If the sink is Y16 or RGB64 (not supported yet), the dtype in the array is uint16, othereise it is uint8
- """
- BildDaten = self.GetImageDescription()[:4]
- lWidth = BildDaten[0]
- lHeight = BildDaten[1]
- iBytesPerPixel = BildDaten[2] // 8
-
- buffer_size = lWidth * lHeight * iBytesPerPixel * C.sizeof(C.c_uint8)
- img_ptr = self.GetImagePtr()
-
- Bild = C.cast(img_ptr, C.POINTER(C.c_ubyte * buffer_size))
-
- pixeltype = np.uint8
-
- if BildDaten[3] == 4: # SinkFormats.Y16:
- pixeltype = np.uint16
- iBytesPerPixel = 1
-
- img = np.ndarray(
- buffer=Bild.contents,
- dtype=pixeltype,
- shape=(lHeight, lWidth, iBytesPerPixel),
- )
- return img
-
- def GetCameraProperty(self, iProperty):
- lFocusPos = C.c_long()
- Error = TIS_GrabberDLL.GetCameraProperty(self._handle, iProperty, lFocusPos)
- return lFocusPos.value
-
- def SetCameraProperty(self, iProperty, iValue):
- Error = TIS_GrabberDLL.SetCameraProperty(self._handle, iProperty, iValue)
- return Error
-
- def SetPropertyValue(self, Property, Element, Value):
- error = TIS_GrabberDLL.SetPropertyValue(
- self._handle, self.s(Property), self.s(Element), Value
- )
- return error
-
- def GetPropertyValue(self, Property, Element):
- Value = C.c_long()
- error = TIS_GrabberDLL.GetPropertyValue(
- self._handle, self.s(Property), self.s(Element), Value
- )
- return Value.value
-
- def PropertyAvailable(self, Property):
- Null = None
- error = TIS_GrabberDLL.IsPropertyAvailable(self._handle, self.s(Property), Null)
- return error
-
- def SetPropertySwitch(self, Property, Element, Value):
- error = TIS_GrabberDLL.SetPropertySwitch(
- self._handle, self.s(Property), self.s(Element), Value
- )
- return error
-
- def GetPropertySwitch(self, Property, Element, Value):
- lValue = C.c_long()
- error = TIS_GrabberDLL.GetPropertySwitch(
- self._handle, self.s(Property), self.s(Element), lValue
- )
- Value[0] = lValue.value
- return error
-
- def PropertyOnePush(self, Property, Element):
- error = TIS_GrabberDLL.PropertyOnePush(
- self._handle, self.s(Property), self.s(Element)
- )
- return error
-
- def SetPropertyAbsoluteValue(self, Property, Element, Value):
- error = TIS_GrabberDLL.SetPropertyAbsoluteValue(
- self._handle, self.s(Property), self.s(Element), Value
- )
- return error
-
- def GetPropertyAbsoluteValue(self, Property, Element, Value):
- """ Get a property value of absolute values interface, e.g. seconds or dB.
- Example code:
- ExposureTime=[0]
- Camera.GetPropertyAbsoluteValue("Exposure","Value", ExposureTime)
- print("Exposure time in secods: ", ExposureTime[0])
-
- :param Property: Name of the property, e.g. Gain, Exposure
- :param Element: Name of the element, e.g. "Value"
- :param Value: Object, that receives the value of the property
- :returns: 0 on success
- """
- lValue = C.c_float()
- error = TIS_GrabberDLL.GetPropertyAbsoluteValue(
- self._handle, self.s(Property), self.s(Element), lValue
- )
- Value[0] = lValue.value
- return error
-
- def SaveImage(self, FileName, FileType, Quality=75):
- """ Saves the last snapped image. Can by of type BMP or JPEG.
- :param FileName : Name of the mage file
- :param FileType : Determines file type, can be "JPEG" or "BMP"
- :param Quality : If file typ is JPEG, the qualitly can be given from 1 to 100.
- :return: Error code
- """
- return TIS_GrabberDLL.SaveImage(
- self._handle, self.s(FileName), IC.ImageFileTypes[self.s(FileType)], Quality
- )
-
- def openVideoCaptureDevice(self, DeviceName):
- """ Open the device specified by DeviceName
- :param DeviceName: Name of the device , e.g. "DFK 72AUC02"
- :returns: 1 on success, 0 otherwise.
- """
- return TIS_GrabberDLL.OpenVideoCaptureDevice(self._handle, self.s(DeviceName))
-
- def CreateFrameFilter(self, name):
- frame_filter_handle = FrameFilterHandle()
-
- err = TIS_GrabberDLL.CreateFrameFilter(
- C.c_char_p(name), C.byref(frame_filter_handle)
- )
- if err != 1:
- raise Exception("ERROR CREATING FILTER")
- return frame_filter_handle
-
- def AddFrameFilter(self, frame_filter_handle):
- err = TIS_GrabberDLL.AddFrameFilter(self._handle, frame_filter_handle)
- return err
-
- def FilterGetParameter(self, frame_filter_handle, parameter_name):
- data = C.c_int()
-
- err = TIS_GrabberDLL.FilterGetParameter(
- frame_filter_handle, parameter_name, C.byref(data)
- )
- return data.value
-
- def FilterSetParameter(self, frame_filter_handle, parameter_name, data):
- if type(data) is int:
- err = TIS_GrabberDLL.FilterSetParameter(
- frame_filter_handle, C.c_char_p(parameter_name), C.c_int(data)
- )
- return err
- else:
- raise Exception("Unknown set parameter type")
diff --git a/dlclivegui/camera_process.py b/dlclivegui/camera_process.py
deleted file mode 100644
index 324c8e7..0000000
--- a/dlclivegui/camera_process.py
+++ /dev/null
@@ -1,338 +0,0 @@
-"""
-DeepLabCut Toolbox (deeplabcut.org)
-© A. & M. Mathis Labs
-
-Licensed under GNU Lesser General Public License v3.0
-"""
-
-
-import time
-import multiprocess as mp
-import ctypes
-from dlclivegui.queue import ClearableQueue, ClearableMPQueue
-import threading
-import cv2
-import numpy as np
-import os
-
-
-class CameraProcessError(Exception):
- """
- Exception for incorrect use of Cameras
- """
-
- pass
-
-
-class CameraProcess(object):
- """ Camera Process Manager class. Controls image capture and writing images to a video file in a background process.
-
- Parameters
- ----------
- device : :class:`cameracontrol.Camera`
- a camera object
- ctx : :class:`multiprocess.Context`
- multiprocessing context
- """
-
- def __init__(self, device, ctx=mp.get_context("spawn")):
- """ Constructor method
- """
-
- self.device = device
- self.ctx = ctx
-
- res = self.device.im_size
- self.frame_shared = mp.Array(ctypes.c_uint8, res[1] * res[0] * 3)
- self.frame = np.frombuffer(self.frame_shared.get_obj(), dtype="uint8").reshape(
- res[1], res[0], 3
- )
- self.frame_time_shared = mp.Array(ctypes.c_double, 1)
- self.frame_time = np.frombuffer(self.frame_time_shared.get_obj(), dtype="d")
-
- self.q_to_process = ClearableMPQueue(ctx=self.ctx)
- self.q_from_process = ClearableMPQueue(ctx=self.ctx)
- self.write_frame_queue = ClearableMPQueue(ctx=self.ctx)
-
- self.capture_process = None
- self.writer_process = None
-
- def start_capture_process(self, timeout=60):
-
- cmds = self.q_to_process.read(clear=True, position="all")
- if cmds is not None:
- for c in cmds:
- if c[1] != "capture":
- self.q_to_process.write(c)
-
- self.capture_process = self.ctx.Process(
- target=self._run_capture,
- args=(self.frame_shared, self.frame_time_shared),
- daemon=True,
- )
- self.capture_process.start()
-
- stime = time.time()
- while time.time() - stime < timeout:
- cmd = self.q_from_process.read()
- if cmd is not None:
- if (cmd[0] == "capture") and (cmd[1] == "start"):
- return cmd[2]
- else:
- self.q_to_process.write(cmd)
-
- return True
-
- def _run_capture(self, frame_shared, frame_time):
-
- res = self.device.im_size
- self.frame = np.frombuffer(frame_shared.get_obj(), dtype="uint8").reshape(
- res[1], res[0], 3
- )
- self.frame_time = np.frombuffer(frame_time.get_obj(), dtype="d")
-
- ret = self.device.set_capture_device()
- if not ret:
- raise CameraProcessError("Could not start capture device.")
- self.q_from_process.write(("capture", "start", ret))
-
- self._capture_loop()
-
- self.device.close_capture_device()
- self.q_from_process.write(("capture", "end", True))
-
- def _capture_loop(self):
- """ Acquires frames from frame capture device in a loop
- """
-
- run = True
- write = False
- last_frame_time = time.time()
-
- while run:
-
- start_capture = time.time()
-
- frame, frame_time = self.device.get_image_on_time()
-
- write_capture = time.time()
-
- np.copyto(self.frame, frame)
- self.frame_time[0] = frame_time
-
- if write:
- ret = self.write_frame_queue.write((frame, frame_time))
-
- end_capture = time.time()
-
- # print("read frame = %0.6f // write to queues = %0.6f" % (write_capture-start_capture, end_capture-write_capture))
- # print("capture rate = %d" % (int(1 / (time.time()-last_frame_time))))
- # print("\n")
-
- last_frame_time = time.time()
-
- ### read commands
- cmd = self.q_to_process.read()
- if cmd is not None:
- if cmd[0] == "capture":
- if cmd[1] == "write":
- write = cmd[2]
- self.q_from_process.write(cmd)
- elif cmd[1] == "end":
- run = False
- else:
- self.q_to_process.write(cmd)
-
- def stop_capture_process(self):
-
- ret = True
- if self.capture_process is not None:
- if self.capture_process.is_alive():
- self.q_to_process.write(("capture", "end"))
-
- while True:
- cmd = self.q_from_process.read()
- if cmd is not None:
- if cmd[0] == "capture":
- if cmd[1] == "end":
- break
- else:
- self.q_from_process.write(cmd)
-
- self.capture_process.join(5)
- if self.capture_process.is_alive():
- self.capture_process.terminate()
-
- return True
-
- def start_writer_process(self, filename, timeout=60):
-
- cmds = self.q_to_process.read(clear=True, position="all")
- if cmds is not None:
- for c in cmds:
- if c[1] != "writer":
- self.q_to_process.write(c)
-
- self.writer_process = self.ctx.Process(
- target=self._run_writer, args=(filename,), daemon=True
- )
- self.writer_process.start()
-
- stime = time.time()
- while time.time() - stime < timeout:
- cmd = self.q_from_process.read()
- if cmd is not None:
- if (cmd[0] == "writer") and (cmd[1] == "start"):
- return cmd[2]
- else:
- self.q_to_process.write(cmd)
-
- return True
-
- def _run_writer(self, filename):
-
- ret = self._create_writer(filename)
- self.q_from_process.write(("writer", "start", ret))
-
- save = self._write_loop()
-
- ret = self._save_video(not save)
- self.q_from_process.write(("writer", "end", ret))
-
- def _create_writer(self, filename):
-
- self.filename = filename
- self.video_file = f"{self.filename}_VIDEO.avi"
- self.timestamp_file = f"{self.filename}_TS.npy"
-
- self.video_writer = cv2.VideoWriter(
- self.video_file,
- cv2.VideoWriter_fourcc(*"DIVX"),
- self.device.fps,
- self.device.im_size,
- )
- self.write_frame_ts = []
-
- return True
-
- def _write_loop(self):
- """ read frames from write_frame_queue and write to file
- """
-
- run = True
- new_frame = None
-
- while run or (new_frame is not None):
-
- new_frame = self.write_frame_queue.read()
- if new_frame is not None:
- frame, ts = new_frame
- if frame.shape[2] == 1:
- frame = cv2.cvtColor(frame, cv2.COLOR_GRAY2RGB)
- self.video_writer.write(frame)
- self.write_frame_ts.append(ts)
-
- cmd = self.q_to_process.read()
- if cmd is not None:
- if cmd[0] == "writer":
- if cmd[1] == "end":
- run = False
- save = cmd[2]
- else:
- self.q_to_process.write(cmd)
-
- return save
-
- def _save_video(self, delete=False):
-
- ret = False
-
- self.video_writer.release()
-
- if (not delete) and (len(self.write_frame_ts) > 0):
- np.save(self.timestamp_file, self.write_frame_ts)
- ret = True
- else:
- os.remove(self.video_file)
- if os.path.isfile(self.timestamp_file):
- os.remove(self.timestamp_file)
-
- return ret
-
- def stop_writer_process(self, save=True):
-
- ret = False
- if self.writer_process is not None:
- if self.writer_process.is_alive():
- self.q_to_process.write(("writer", "end", save))
-
- while True:
- cmd = self.q_from_process.read()
- if cmd is not None:
- if cmd[0] == "writer":
- if cmd[1] == "end":
- ret = cmd[2]
- break
- else:
- self.q_from_process.write(cmd)
-
- self.writer_process.join(5)
- if self.writer_process.is_alive():
- self.writer_process.terminate()
-
- return ret
-
- def start_record(self, timeout=5):
-
- ret = False
-
- if (self.capture_process is not None) and (self.writer_process is not None):
- if self.capture_process.is_alive() and self.writer_process.is_alive():
- self.q_to_process.write(("capture", "write", True))
-
- stime = time.time()
- while time.time() - stime < timeout:
- cmd = self.q_from_process.read()
- if cmd is not None:
- if (cmd[0] == "capture") and (cmd[1] == "write"):
- ret = cmd[2]
- break
- else:
- self.q_from_process.write(cmd)
-
- return ret
-
- def stop_record(self, timeout=5):
-
- ret = False
-
- if (self.capture_process is not None) and (self.writer_process is not None):
- if (self.capture_process.is_alive()) and (self.writer_process.is_alive()):
- self.q_to_process.write(("capture", "write", False))
-
- stime = time.time()
- while time.time() - stime < timeout:
- cmd = self.q_from_process.read()
- if cmd is not None:
- if (cmd[0] == "capture") and (cmd[1] == "write"):
- ret = not cmd[2]
- break
- else:
- self.q_from_process.write(cmd)
-
- return ret
-
- def get_display_frame(self):
-
- frame = self.frame.copy()
- if frame is not None:
- if self.device.display_resize != 1:
- frame = cv2.resize(
- frame,
- (
- int(frame.shape[1] * self.device.display_resize),
- int(frame.shape[0] * self.device.display_resize),
- ),
- )
-
- return frame
diff --git a/dlclivegui/cameras/__init__.py b/dlclivegui/cameras/__init__.py
new file mode 100644
index 0000000..d7290a9
--- /dev/null
+++ b/dlclivegui/cameras/__init__.py
@@ -0,0 +1,16 @@
+"""Camera backend implementations and factory helpers."""
+
+from __future__ import annotations
+
+from ..config import CameraSettings
+from .base import _BACKEND_REGISTRY as _BACKEND_REGISTRY
+from .base import CameraBackend
+from .factory import CameraFactory, DetectedCamera
+
+__all__ = [
+ "CameraSettings",
+ "CameraBackend",
+ "CameraFactory",
+ "DetectedCamera",
+ "_BACKEND_REGISTRY",
+]
diff --git a/dlclivegui/cameras/backends/__init__.py b/dlclivegui/cameras/backends/__init__.py
new file mode 100644
index 0000000..d14e764
--- /dev/null
+++ b/dlclivegui/cameras/backends/__init__.py
@@ -0,0 +1,11 @@
+from .aravis_backend import AravisCameraBackend
+from .basler_backend import BaslerCameraBackend
+from .gentl_backend import GenTLCameraBackend
+from .opencv_backend import OpenCVCameraBackend
+
+__all__ = [
+ "AravisCameraBackend",
+ "BaslerCameraBackend",
+ "GenTLCameraBackend",
+ "OpenCVCameraBackend",
+]
diff --git a/dlclivegui/cameras/backends/aravis_backend.py b/dlclivegui/cameras/backends/aravis_backend.py
new file mode 100644
index 0000000..d3512ea
--- /dev/null
+++ b/dlclivegui/cameras/backends/aravis_backend.py
@@ -0,0 +1,342 @@
+"""Aravis backend for GenICam cameras."""
+
+from __future__ import annotations
+
+import logging
+import time
+
+import cv2
+import numpy as np
+
+from ..base import CameraBackend, register_backend
+
+LOG = logging.getLogger(__name__)
+
+try: # pragma: no cover - optional dependency
+ import gi
+
+ gi.require_version("Aravis", "0.8")
+ from gi.repository import Aravis
+
+ ARAVIS_AVAILABLE = True
+except Exception: # pragma: no cover - optional dependency
+ Aravis = None # type: ignore
+ ARAVIS_AVAILABLE = False
+
+
+@register_backend("aravis")
+class AravisCameraBackend(CameraBackend):
+ """Capture frames from GenICam-compatible devices via Aravis."""
+
+ def __init__(self, settings):
+ super().__init__(settings)
+ props = settings.properties
+ self._camera_id: str | None = props.get("camera_id")
+ self._pixel_format: str = props.get("pixel_format", "Mono8")
+ self._timeout: int = int(props.get("timeout", 2000000)) # microseconds
+ self._n_buffers: int = int(props.get("n_buffers", 10))
+
+ self._camera = None
+ self._stream = None
+ self._device_label: str | None = None
+
+ @classmethod
+ def is_available(cls) -> bool:
+ """Check if Aravis is available on this system."""
+ return ARAVIS_AVAILABLE
+
+ @classmethod
+ def get_device_count(cls) -> int:
+ """Get the actual number of Aravis devices detected.
+
+ Returns the number of devices found, or -1 if detection fails.
+ """
+ if not ARAVIS_AVAILABLE:
+ return -1
+
+ try:
+ Aravis.update_device_list()
+ return Aravis.get_n_devices()
+ except Exception:
+ return -1
+
+ def open(self) -> None:
+ """Open the Aravis camera device."""
+ if not ARAVIS_AVAILABLE: # pragma: no cover - optional dependency
+ raise RuntimeError(
+ "The 'aravis' library is required for the Aravis backend. "
+ "Install it via your system package manager (e.g., 'sudo apt install gir1.2-aravis-0.8' on Ubuntu)."
+ )
+
+ # Update device list
+ Aravis.update_device_list()
+ n_devices = Aravis.get_n_devices()
+
+ if n_devices == 0:
+ raise RuntimeError("No Aravis cameras detected")
+
+ # Open camera by ID or index
+ if self._camera_id:
+ self._camera = Aravis.Camera.new(self._camera_id)
+ if self._camera is None:
+ raise RuntimeError(f"Failed to open camera with ID '{self._camera_id}'")
+ else:
+ index = int(self.settings.index or 0)
+ if index < 0 or index >= n_devices:
+ raise RuntimeError(f"Camera index {index} out of range for {n_devices} Aravis device(s)")
+ camera_id = Aravis.get_device_id(index)
+ self._camera = Aravis.Camera.new(camera_id)
+ if self._camera is None:
+ raise RuntimeError(f"Failed to open camera at index {index}")
+
+ # Get device information for label
+ self._device_label = self._resolve_device_label()
+
+ # Configure camera
+ self._configure_pixel_format()
+ self._configure_exposure()
+ self._configure_gain()
+ self._configure_frame_rate()
+
+ # Create stream
+ self._stream = self._camera.create_stream(None, None)
+ if self._stream is None:
+ raise RuntimeError("Failed to create Aravis stream")
+
+ # Push buffers to stream
+ payload_size = self._camera.get_payload()
+ for _ in range(self._n_buffers):
+ self._stream.push_buffer(Aravis.Buffer.new_allocate(payload_size))
+
+ # Start acquisition
+ self._camera.start_acquisition()
+
+ def read(self) -> tuple[np.ndarray, float]:
+ """Read a frame from the camera."""
+ if self._camera is None or self._stream is None:
+ raise RuntimeError("Aravis camera not initialized")
+
+ # Pop buffer from stream
+ buffer = self._stream.timeout_pop_buffer(self._timeout)
+
+ if buffer is None:
+ raise TimeoutError("Failed to grab frame from Aravis camera (timeout)")
+
+ # Check buffer status
+ status = buffer.get_status()
+ if status != Aravis.BufferStatus.SUCCESS:
+ self._stream.push_buffer(buffer)
+ raise TimeoutError(f"Aravis buffer status error: {status}")
+
+ # Get image data
+ try:
+ # Get buffer data as numpy array
+ data = buffer.get_data()
+ width = buffer.get_image_width()
+ height = buffer.get_image_height()
+ pixel_format = buffer.get_image_pixel_format()
+
+ # Convert to numpy array
+ if pixel_format == Aravis.PIXEL_FORMAT_MONO_8:
+ frame = np.frombuffer(data, dtype=np.uint8).reshape((height, width))
+ frame = cv2.cvtColor(frame, cv2.COLOR_GRAY2BGR)
+ elif pixel_format == Aravis.PIXEL_FORMAT_RGB_8_PACKED:
+ frame = np.frombuffer(data, dtype=np.uint8).reshape((height, width, 3))
+ frame = cv2.cvtColor(frame, cv2.COLOR_RGB2BGR)
+ elif pixel_format == Aravis.PIXEL_FORMAT_BGR_8_PACKED:
+ frame = np.frombuffer(data, dtype=np.uint8).reshape((height, width, 3))
+ elif pixel_format in (Aravis.PIXEL_FORMAT_MONO_12, Aravis.PIXEL_FORMAT_MONO_16):
+ # Handle 12-bit and 16-bit mono
+ frame = np.frombuffer(data, dtype=np.uint16).reshape((height, width))
+ # Scale to 8-bit
+ max_val = float(frame.max()) if frame.size else 0.0
+ scale = 255.0 / max_val if max_val > 0.0 else 1.0
+ frame = np.clip(frame * scale, 0, 255).astype(np.uint8)
+ frame = cv2.cvtColor(frame, cv2.COLOR_GRAY2BGR)
+ else:
+ # Fallback for unknown formats - try to interpret as mono8
+ frame = np.frombuffer(data, dtype=np.uint8).reshape((height, width))
+ frame = cv2.cvtColor(frame, cv2.COLOR_GRAY2BGR)
+
+ frame = frame.copy()
+ timestamp = time.time()
+
+ finally:
+ # Always push buffer back to stream
+ self._stream.push_buffer(buffer)
+
+ return frame, timestamp
+
+ def stop(self) -> None:
+ """Stop camera acquisition."""
+ if self._camera is not None:
+ try:
+ self._camera.stop_acquisition()
+ except Exception:
+ pass
+
+ def close(self) -> None:
+ """Release the camera and stream."""
+ if self._camera is not None:
+ try:
+ self._camera.stop_acquisition()
+ except Exception:
+ pass
+
+ # Clear stream buffers
+ if self._stream is not None:
+ try:
+ # Flush remaining buffers
+ while True:
+ buffer = self._stream.try_pop_buffer()
+ if buffer is None:
+ break
+ except Exception:
+ pass
+ self._stream = None
+
+ # Release camera
+ try:
+ del self._camera
+ except Exception:
+ pass
+ finally:
+ self._camera = None
+
+ self._device_label = None
+
+ def device_name(self) -> str:
+ """Return a human-readable device name."""
+ if self._device_label:
+ return self._device_label
+ return super().device_name()
+
+ # ------------------------------------------------------------------
+ # Configuration helpers
+ # ------------------------------------------------------------------
+
+ def _configure_pixel_format(self) -> None:
+ """Configure the camera pixel format."""
+ if self._camera is None:
+ return
+
+ try:
+ # Map common format names to Aravis pixel formats
+ format_map = {
+ "Mono8": Aravis.PIXEL_FORMAT_MONO_8,
+ "Mono12": Aravis.PIXEL_FORMAT_MONO_12,
+ "Mono16": Aravis.PIXEL_FORMAT_MONO_16,
+ "RGB8": Aravis.PIXEL_FORMAT_RGB_8_PACKED,
+ "BGR8": Aravis.PIXEL_FORMAT_BGR_8_PACKED,
+ }
+
+ if self._pixel_format in format_map:
+ self._camera.set_pixel_format(format_map[self._pixel_format])
+ LOG.info(f"Pixel format set to '{self._pixel_format}'")
+ else:
+ # Try setting as string
+ self._camera.set_pixel_format_from_string(self._pixel_format)
+ LOG.info(f"Pixel format set to '{self._pixel_format}' (from string)")
+ except Exception as e:
+ LOG.warning(f"Failed to set pixel format '{self._pixel_format}': {e}")
+
+ def _configure_exposure(self) -> None:
+ """Configure camera exposure time."""
+ if self._camera is None:
+ return
+
+ # Get exposure from settings
+ exposure = None
+ if hasattr(self.settings, "exposure") and self.settings.exposure > 0:
+ exposure = float(self.settings.exposure)
+
+ if exposure is None:
+ return
+
+ try:
+ # Disable auto exposure
+ try:
+ self._camera.set_exposure_time_auto(Aravis.Auto.OFF)
+ LOG.info("Auto exposure disabled")
+ except Exception as e:
+ LOG.warning(f"Failed to disable auto exposure: {e}")
+
+ # Set exposure time (in microseconds)
+ self._camera.set_exposure_time(exposure)
+ actual = self._camera.get_exposure_time()
+ if abs(actual - exposure) > 1.0: # Allow 1μs tolerance
+ LOG.warning(f"Exposure mismatch: requested {exposure}μs, got {actual}μs")
+ else:
+ LOG.info(f"Exposure set to {actual}μs")
+ except Exception as e:
+ LOG.warning(f"Failed to set exposure to {exposure}μs: {e}")
+
+ def _configure_gain(self) -> None:
+ """Configure camera gain."""
+ if self._camera is None:
+ return
+
+ # Get gain from settings
+ gain = None
+ if hasattr(self.settings, "gain") and self.settings.gain > 0.0:
+ gain = float(self.settings.gain)
+
+ if gain is None:
+ return
+
+ try:
+ # Disable auto gain
+ try:
+ self._camera.set_gain_auto(Aravis.Auto.OFF)
+ LOG.info("Auto gain disabled")
+ except Exception as e:
+ LOG.warning(f"Failed to disable auto gain: {e}")
+
+ # Set gain value
+ self._camera.set_gain(gain)
+ actual = self._camera.get_gain()
+ if abs(actual - gain) > 0.1: # Allow 0.1 tolerance
+ LOG.warning(f"Gain mismatch: requested {gain}, got {actual}")
+ else:
+ LOG.info(f"Gain set to {actual}")
+ except Exception as e:
+ LOG.warning(f"Failed to set gain to {gain}: {e}")
+
+ def _configure_frame_rate(self) -> None:
+ """Configure camera frame rate."""
+ if self._camera is None or not self.settings.fps:
+ return
+
+ try:
+ target_fps = float(self.settings.fps)
+ self._camera.set_frame_rate(target_fps)
+ actual_fps = self._camera.get_frame_rate()
+ if abs(actual_fps - target_fps) > 0.1:
+ LOG.warning(f"FPS mismatch: requested {target_fps:.2f}, got {actual_fps:.2f}")
+ else:
+ LOG.info(f"Frame rate set to {actual_fps:.2f} FPS")
+ except Exception as e:
+ LOG.warning(f"Failed to set frame rate to {self.settings.fps}: {e}")
+
+ def _resolve_device_label(self) -> str | None:
+ """Get a human-readable device label."""
+ if self._camera is None:
+ return None
+
+ try:
+ model = self._camera.get_model_name()
+ vendor = self._camera.get_vendor_name()
+ serial = self._camera.get_device_serial_number()
+
+ if model and serial:
+ if vendor:
+ return f"{vendor} {model} ({serial})"
+ return f"{model} ({serial})"
+ elif model:
+ return model
+ elif serial:
+ return f"Camera {serial}"
+ except Exception:
+ pass
+
+ return None
diff --git a/dlclivegui/cameras/backends/basler_backend.py b/dlclivegui/cameras/backends/basler_backend.py
new file mode 100644
index 0000000..31ab2c7
--- /dev/null
+++ b/dlclivegui/cameras/backends/basler_backend.py
@@ -0,0 +1,202 @@
+"""Basler camera backend implemented with :mod:`pypylon`."""
+
+from __future__ import annotations
+
+import logging
+import time
+
+import numpy as np
+
+from ..base import CameraBackend, register_backend
+
+LOG = logging.getLogger(__name__)
+
+try: # pragma: no cover - optional dependency
+ from pypylon import pylon
+except Exception: # pragma: no cover - optional dependency
+ pylon = None # type: ignore
+
+
+@register_backend("basler")
+class BaslerCameraBackend(CameraBackend):
+ """Capture frames from Basler cameras using the Pylon SDK."""
+
+ def __init__(self, settings):
+ super().__init__(settings)
+ self._camera: pylon.InstantCamera | None = None
+ self._converter: pylon.ImageFormatConverter | None = None
+ # Parse resolution with defaults (720x540)
+ self._resolution: tuple[int, int] = self._parse_resolution(settings.properties.get("resolution"))
+
+ @classmethod
+ def is_available(cls) -> bool:
+ return pylon is not None
+
+ def open(self) -> None:
+ if pylon is None: # pragma: no cover - optional dependency
+ raise RuntimeError("pypylon is required for the Basler backend but is not installed")
+ devices = self._enumerate_devices()
+ if not devices:
+ raise RuntimeError("No Basler cameras detected")
+ device = self._select_device(devices)
+ self._camera = pylon.InstantCamera(pylon.TlFactory.GetInstance().CreateDevice(device))
+ self._camera.Open()
+
+ # Configure exposure
+ exposure = self._settings_value("exposure", self.settings.properties)
+ if exposure is not None:
+ try:
+ self._camera.ExposureTime.SetValue(float(exposure))
+ actual = self._camera.ExposureTime.GetValue()
+ if abs(actual - float(exposure)) > 1.0: # Allow 1μs tolerance
+ LOG.warning(f"Exposure mismatch: requested {exposure}μs, got {actual}μs")
+ else:
+ LOG.info(f"Exposure set to {actual}μs")
+ except Exception as e:
+ LOG.warning(f"Failed to set exposure to {exposure}μs: {e}")
+
+ # Configure gain
+ gain = self._settings_value("gain", self.settings.properties)
+ if gain is not None:
+ try:
+ self._camera.Gain.SetValue(float(gain))
+ actual = self._camera.Gain.GetValue()
+ if abs(actual - float(gain)) > 0.1: # Allow 0.1 tolerance
+ LOG.warning(f"Gain mismatch: requested {gain}, got {actual}")
+ else:
+ LOG.info(f"Gain set to {actual}")
+ except Exception as e:
+ LOG.warning(f"Failed to set gain to {gain}: {e}")
+
+ # Configure resolution
+ requested_width, requested_height = self._resolution
+ try:
+ self._camera.Width.SetValue(requested_width)
+ self._camera.Height.SetValue(requested_height)
+ actual_width = self._camera.Width.GetValue()
+ actual_height = self._camera.Height.GetValue()
+ if actual_width != requested_width or actual_height != requested_height:
+ LOG.warning(
+ f"Resolution mismatch: requested {requested_width}x{requested_height}, "
+ f"got {actual_width}x{actual_height}"
+ )
+ else:
+ LOG.info(f"Resolution set to {actual_width}x{actual_height}")
+ except Exception as e:
+ LOG.warning(f"Failed to set resolution to {requested_width}x{requested_height}: {e}")
+
+ # Configure frame rate
+ fps = self._settings_value("fps", self.settings.properties, fallback=self.settings.fps)
+ if fps is not None:
+ try:
+ self._camera.AcquisitionFrameRateEnable.SetValue(True)
+ self._camera.AcquisitionFrameRate.SetValue(float(fps))
+ actual_fps = self._camera.AcquisitionFrameRate.GetValue()
+ if abs(actual_fps - float(fps)) > 0.1:
+ LOG.warning(f"FPS mismatch: requested {fps:.2f}, got {actual_fps:.2f}")
+ else:
+ LOG.info(f"Frame rate set to {actual_fps:.2f} FPS")
+ except Exception as e:
+ LOG.warning(f"Failed to set frame rate to {fps}: {e}")
+
+ self._camera.StartGrabbing(pylon.GrabStrategy_LatestImageOnly)
+ self._converter = pylon.ImageFormatConverter()
+ self._converter.OutputPixelFormat = pylon.PixelType_BGR8packed
+ self._converter.OutputBitAlignment = pylon.OutputBitAlignment_MsbAligned
+
+ # Read back final settings
+ try:
+ self.settings.width = int(self._camera.Width.GetValue())
+ self.settings.height = int(self._camera.Height.GetValue())
+ except Exception:
+ pass
+ try:
+ self.settings.fps = float(self._camera.ResultingFrameRateAbs.GetValue())
+ LOG.info(f"Camera configured with resulting FPS: {self.settings.fps:.2f}")
+ except Exception:
+ pass
+
+ def read(self) -> tuple[np.ndarray, float]:
+ if self._camera is None or self._converter is None:
+ raise RuntimeError("Basler camera not opened")
+ try:
+ grab_result = self._camera.RetrieveResult(100, pylon.TimeoutHandling_ThrowException)
+ except Exception as exc:
+ raise RuntimeError("Failed to retrieve image from Basler camera.") from exc
+ if not grab_result.GrabSucceeded():
+ grab_result.Release()
+ raise RuntimeError("Basler camera did not return an image")
+ image = self._converter.Convert(grab_result)
+ frame = image.GetArray()
+ grab_result.Release()
+ rotate = self._settings_value("rotate", self.settings.properties)
+ if rotate:
+ frame = self._rotate(frame, float(rotate))
+ crop = self.settings.properties.get("crop")
+ if isinstance(crop, (list, tuple)) and len(crop) == 4:
+ left, right, top, bottom = map(int, crop)
+ frame = frame[top:bottom, left:right]
+ return frame, time.time()
+
+ def close(self) -> None:
+ if self._camera is not None:
+ if self._camera.IsGrabbing():
+ self._camera.StopGrabbing()
+ if self._camera.IsOpen():
+ self._camera.Close()
+ self._camera = None
+ self._converter = None
+
+ def stop(self) -> None:
+ if self._camera is not None and self._camera.IsGrabbing():
+ try:
+ self._camera.StopGrabbing()
+ except Exception:
+ pass
+
+ def _enumerate_devices(self):
+ factory = pylon.TlFactory.GetInstance()
+ return factory.EnumerateDevices()
+
+ def _select_device(self, devices):
+ serial = self.settings.properties.get("serial") or self.settings.properties.get("serial_number")
+ if serial:
+ for device in devices:
+ if getattr(device, "GetSerialNumber", None) and device.GetSerialNumber() == serial:
+ return device
+ index = int(self.settings.index)
+ if index < 0 or index >= len(devices):
+ raise RuntimeError(f"Camera index {index} out of range for {len(devices)} Basler device(s)")
+ return devices[index]
+
+ def _rotate(self, frame: np.ndarray, angle: float) -> np.ndarray:
+ try:
+ from imutils import rotate_bound # pragma: no cover - optional
+ except Exception as exc: # pragma: no cover - optional dependency
+ raise RuntimeError("Rotation requested for Basler camera but imutils is not installed") from exc
+ return rotate_bound(frame, angle)
+
+ def _parse_resolution(self, resolution) -> tuple[int, int]:
+ """Parse resolution setting.
+
+ Args:
+ resolution: Can be a tuple/list [width, height], or None
+
+ Returns:
+ Tuple of (width, height), defaults to (720, 540)
+ """
+ if resolution is None:
+ return (720, 540) # Default resolution
+
+ if isinstance(resolution, (list, tuple)) and len(resolution) == 2:
+ try:
+ return (int(resolution[0]), int(resolution[1]))
+ except (ValueError, TypeError):
+ return (720, 540)
+
+ return (720, 540)
+
+ @staticmethod
+ def _settings_value(key: str, source: dict, fallback: float | None = None):
+ value = source.get(key, fallback)
+ return None if value is None else value
diff --git a/dlclivegui/cameras/backends/gentl_backend.py b/dlclivegui/cameras/backends/gentl_backend.py
new file mode 100644
index 0000000..e9b7c0d
--- /dev/null
+++ b/dlclivegui/cameras/backends/gentl_backend.py
@@ -0,0 +1,604 @@
+"""GenTL backend implemented using the Harvesters library."""
+
+from __future__ import annotations
+
+import glob
+import logging
+import os
+import time
+from collections.abc import Iterable
+
+import cv2
+import numpy as np
+
+from ..base import CameraBackend, register_backend
+
+LOG = logging.getLogger(__name__)
+
+try: # pragma: no cover - optional dependency
+ from harvesters.core import Harvester # type: ignore
+
+ try:
+ from harvesters.core import HarvesterTimeoutError # type: ignore
+ except Exception: # pragma: no cover - optional dependency
+ HarvesterTimeoutError = TimeoutError # type: ignore
+except Exception: # pragma: no cover - optional dependency
+ Harvester = None # type: ignore
+ HarvesterTimeoutError = TimeoutError # type: ignore
+
+
+@register_backend("gentl")
+class GenTLCameraBackend(CameraBackend):
+ """Capture frames from GenTL-compatible devices via Harvesters."""
+
+ _DEFAULT_CTI_PATTERNS: tuple[str, ...] = (
+ r"C:\\Program Files\\The Imaging Source Europe GmbH\\IC4 GenTL Driver for USB3Vision Devices *\\bin\\*.cti",
+ r"C:\\Program Files\\The Imaging Source Europe GmbH\\TIS Grabber\\bin\\win64_x64\\*.cti",
+ r"C:\\Program Files\\The Imaging Source Europe GmbH\\TIS Camera SDK\\bin\\win64_x64\\*.cti",
+ r"C:\\Program Files (x86)\\The Imaging Source Europe GmbH\\TIS Grabber\\bin\\win64_x64\\*.cti",
+ )
+
+ def __init__(self, settings):
+ super().__init__(settings)
+ props = settings.properties
+ self._cti_file: str | None = props.get("cti_file")
+ self._serial_number: str | None = props.get("serial_number") or props.get("serial")
+ self._pixel_format: str = props.get("pixel_format", "Mono8")
+ self._rotate: int = int(props.get("rotate", 0)) % 360
+ self._crop: tuple[int, int, int, int] | None = self._parse_crop(props.get("crop"))
+ # Check settings first (from config), then properties (for backward compatibility)
+ self._exposure: float | None = settings.exposure if settings.exposure else props.get("exposure")
+ self._gain: float | None = settings.gain if settings.gain else props.get("gain")
+ self._timeout: float = float(props.get("timeout", 2.0))
+ self._cti_search_paths: tuple[str, ...] = self._parse_cti_paths(props.get("cti_search_paths"))
+ # Parse resolution (width, height) with defaults
+ self._resolution: tuple[int, int] | None = self._parse_resolution(props.get("resolution"))
+
+ self._harvester = None
+ self._acquirer = None
+ self._device_label: str | None = None
+
+ @classmethod
+ def is_available(cls) -> bool:
+ return Harvester is not None
+
+ @classmethod
+ def get_device_count(cls) -> int:
+ """Get the actual number of GenTL devices detected by Harvester.
+
+ Returns the number of devices found, or -1 if detection fails.
+ """
+ if Harvester is None:
+ return -1
+
+ harvester = None
+ try:
+ harvester = Harvester()
+ # Use the static helper to find CTI file with default patterns
+ cti_file = cls._search_cti_file(cls._DEFAULT_CTI_PATTERNS)
+
+ if not cti_file:
+ return -1
+
+ harvester.add_file(cti_file)
+ harvester.update()
+ return len(harvester.device_info_list)
+ except Exception:
+ return -1
+ finally:
+ if harvester is not None:
+ try:
+ harvester.reset()
+ except Exception:
+ pass
+
+ def open(self) -> None:
+ if Harvester is None: # pragma: no cover - optional dependency
+ raise RuntimeError(
+ "The 'harvesters' package is required for the GenTL backend. Install it via 'pip install harvesters'."
+ )
+
+ self._harvester = Harvester()
+ cti_file = self._cti_file or self._find_cti_file()
+ self._harvester.add_file(cti_file)
+ self._harvester.update()
+
+ if not self._harvester.device_info_list:
+ raise RuntimeError("No GenTL cameras detected via Harvesters")
+
+ serial = self._serial_number
+ index = int(self.settings.index or 0)
+ if serial:
+ available = self._available_serials()
+ matches = [s for s in available if serial in s]
+ if not matches:
+ raise RuntimeError(f"Camera with serial '{serial}' not found. Available cameras: {available}")
+ serial = matches[0]
+ else:
+ device_count = len(self._harvester.device_info_list)
+ if index < 0 or index >= device_count:
+ raise RuntimeError(f"Camera index {index} out of range for {device_count} GenTL device(s)")
+
+ self._acquirer = self._create_acquirer(serial, index)
+
+ remote = self._acquirer.remote_device
+ node_map = remote.node_map
+
+ # print(dir(node_map))
+ """
+ ['AcquisitionBurstFrameCount', 'AcquisitionControl', 'AcquisitionFrameRate', 'AcquisitionMode',
+ 'AcquisitionStart', 'AcquisitionStop', 'AnalogControl', 'AutoFunctionsROI', 'AutoFunctionsROIEnable',
+ 'AutoFunctionsROIHeight', 'AutoFunctionsROILeft', 'AutoFunctionsROIPreset', 'AutoFunctionsROITop',
+ 'AutoFunctionsROIWidth', 'BinningHorizontal', 'BinningVertical', 'BlackLevel', 'CameraRegisterAddress',
+ 'CameraRegisterAddressSpace', 'CameraRegisterControl', 'CameraRegisterRead', 'CameraRegisterValue',
+ 'CameraRegisterWrite', 'Contrast', 'DecimationHorizontal', 'DecimationVertical', 'Denoise',
+ 'DeviceControl', 'DeviceFirmwareVersion', 'DeviceModelName', 'DeviceReset', 'DeviceSFNCVersionMajor',
+ 'DeviceSFNCVersionMinor', 'DeviceSFNCVersionSubMinor', 'DeviceScanType', 'DeviceSerialNumber',
+ 'DeviceTLType', 'DeviceTLVersionMajor', 'DeviceTLVersionMinor', 'DeviceTLVersionSubMinor',
+ 'DeviceTemperature', 'DeviceTemperatureSelector', 'DeviceType', 'DeviceUserID', 'DeviceVendorName',
+ 'DigitalIO', 'ExposureAuto', 'ExposureAutoHighlightReduction', 'ExposureAutoLowerLimit',
+ 'ExposureAutoReference', 'ExposureAutoUpperLimit', 'ExposureAutoUpperLimitAuto', 'ExposureTime',
+ 'GPIn', 'GPOut', 'Gain', 'GainAuto', 'GainAutoLowerLimit', 'GainAutoUpperLimit', 'Gamma', 'Height',
+ 'HeightMax', 'IMXLowLatencyTriggerMode', 'ImageFormatControl', 'OffsetAutoCenter', 'OffsetX', 'OffsetY',
+ 'PayloadSize', 'PixelFormat', 'ReverseX', 'ReverseY', 'Root', 'SensorHeight', 'SensorWidth', 'Sharpness',
+ 'ShowOverlay', 'SoftwareAnalogControl', 'SoftwareTransformControl', 'SoftwareTransformEnable',
+ 'StrobeDelay', 'StrobeDuration', 'StrobeEnable', 'StrobeOperation', 'StrobePolarity', 'TLParamsLocked',
+ 'TestControl', 'TestPendingAck', 'TimestampLatch', 'TimestampLatchValue', 'TimestampReset', 'ToneMappingAuto',
+ 'ToneMappingControl', 'ToneMappingEnable', 'ToneMappingGlobalBrightness', 'ToneMappingIntensity',
+ 'TransportLayerControl', 'TriggerActivation', 'TriggerDebouncer', 'TriggerDelay', 'TriggerDenoise',
+ 'TriggerMask', 'TriggerMode', 'TriggerOverlap', 'TriggerSelector', 'TriggerSoftware', 'TriggerSource',
+ 'UserSetControl', 'UserSetDefault', 'UserSetLoad', 'UserSetSave', 'UserSetSelector', 'Width', 'WidthMax']
+ """
+
+ self._device_label = self._resolve_device_label(node_map)
+
+ self._configure_pixel_format(node_map)
+ self._configure_resolution(node_map)
+ self._configure_exposure(node_map)
+ self._configure_gain(node_map)
+ self._configure_frame_rate(node_map)
+
+ self._acquirer.start()
+
+ def read(self) -> tuple[np.ndarray, float]:
+ if self._acquirer is None:
+ raise RuntimeError("GenTL image acquirer not initialised")
+
+ try:
+ with self._acquirer.fetch(timeout=self._timeout) as buffer:
+ component = buffer.payload.components[0]
+ channels = 3 if self._pixel_format in {"RGB8", "BGR8"} else 1
+ array = np.asarray(component.data)
+ expected = component.height * component.width * channels
+ if array.size != expected:
+ array = np.frombuffer(bytes(component.data), dtype=array.dtype)
+ try:
+ if channels > 1:
+ frame = array.reshape(component.height, component.width, channels).copy()
+ else:
+ frame = array.reshape(component.height, component.width).copy()
+ except ValueError:
+ frame = array.copy()
+ except HarvesterTimeoutError as exc:
+ raise TimeoutError(str(exc) + " (GenTL timeout)") from exc
+
+ frame = self._convert_frame(frame)
+ timestamp = time.time()
+ return frame, timestamp
+
+ def stop(self) -> None:
+ if self._acquirer is not None:
+ try:
+ self._acquirer.stop()
+ except Exception:
+ pass
+
+ def close(self) -> None:
+ if self._acquirer is not None:
+ try:
+ self._acquirer.stop()
+ except Exception:
+ pass
+ try:
+ destroy = getattr(self._acquirer, "destroy", None)
+ if destroy is not None:
+ destroy()
+ finally:
+ self._acquirer = None
+
+ if self._harvester is not None:
+ try:
+ self._harvester.reset()
+ finally:
+ self._harvester = None
+
+ self._device_label = None
+
+ # ------------------------------------------------------------------
+ # Helpers
+ # ------------------------------------------------------------------
+
+ def _parse_cti_paths(self, value) -> tuple[str, ...]:
+ if value is None:
+ return self._DEFAULT_CTI_PATTERNS
+ if isinstance(value, str):
+ return (value,)
+ if isinstance(value, Iterable):
+ return tuple(str(item) for item in value)
+ return self._DEFAULT_CTI_PATTERNS
+
+ def _parse_crop(self, crop) -> tuple[int, int, int, int] | None:
+ if isinstance(crop, (list, tuple)) and len(crop) == 4:
+ return tuple(int(v) for v in crop)
+ return None
+
+ def _parse_resolution(self, resolution) -> tuple[int, int] | None:
+ """Parse resolution setting.
+
+ Args:
+ resolution: Can be a tuple/list [width, height], or None
+
+ Returns:
+ Tuple of (width, height) or None if not specified
+ Default is (720, 540) if parsing fails but value is provided
+ """
+ if resolution is None:
+ return (720, 540) # Default resolution
+
+ if isinstance(resolution, (list, tuple)) and len(resolution) == 2:
+ try:
+ return (int(resolution[0]), int(resolution[1]))
+ except (ValueError, TypeError):
+ return (720, 540)
+
+ return (720, 540)
+
+ @staticmethod
+ def _search_cti_file(patterns: tuple[str, ...]) -> str | None:
+ """Search for a CTI file using the given patterns.
+
+ Returns the first CTI file found, or None if none found.
+ """
+ for pattern in patterns:
+ for file_path in glob.glob(pattern):
+ if os.path.isfile(file_path):
+ return file_path
+ return None
+
+ def _find_cti_file(self) -> str:
+ """Find a CTI file using configured or default search paths.
+
+ Raises RuntimeError if no CTI file is found.
+ """
+ cti_file = self._search_cti_file(self._cti_search_paths)
+ if cti_file is None:
+ raise RuntimeError(
+ "Could not locate a GenTL producer (.cti) file. Set 'cti_file' in "
+ "camera.properties or provide search paths via 'cti_search_paths'."
+ )
+ return cti_file
+
+ def _available_serials(self) -> list[str]:
+ assert self._harvester is not None
+ serials: list[str] = []
+ for info in self._harvester.device_info_list:
+ serial = getattr(info, "serial_number", "")
+ if serial:
+ serials.append(serial)
+ return serials
+
+ def _create_acquirer(self, serial: str | None, index: int):
+ assert self._harvester is not None
+ methods = [
+ getattr(self._harvester, "create", None),
+ getattr(self._harvester, "create_image_acquirer", None),
+ ]
+ methods = [m for m in methods if m is not None]
+ errors: list[str] = []
+ device_info = None
+ if not serial:
+ device_list = self._harvester.device_info_list
+ if 0 <= index < len(device_list):
+ device_info = device_list[index]
+ for create in methods:
+ try:
+ if serial:
+ return create({"serial_number": serial})
+ except Exception as exc:
+ errors.append(f"{create.__name__} serial: {exc}")
+ for create in methods:
+ try:
+ return create(index=index)
+ except TypeError:
+ try:
+ return create(index)
+ except Exception as exc:
+ errors.append(f"{create.__name__} index positional: {exc}")
+ except Exception as exc:
+ errors.append(f"{create.__name__} index: {exc}")
+ if device_info is not None:
+ for create in methods:
+ try:
+ return create(device_info)
+ except Exception as exc:
+ errors.append(f"{create.__name__} device_info: {exc}")
+ if not serial and index == 0:
+ for create in methods:
+ try:
+ return create()
+ except Exception as exc:
+ errors.append(f"{create.__name__} default: {exc}")
+ joined = "; ".join(errors) or "no creation methods available"
+ raise RuntimeError(f"Failed to initialise GenTL image acquirer ({joined})")
+
+ def _configure_pixel_format(self, node_map) -> None:
+ try:
+ if self._pixel_format in node_map.PixelFormat.symbolics:
+ node_map.PixelFormat.value = self._pixel_format
+ actual = node_map.PixelFormat.value
+ if actual != self._pixel_format:
+ LOG.warning(f"Pixel format mismatch: requested '{self._pixel_format}', got '{actual}'")
+ else:
+ LOG.info(f"Pixel format set to '{actual}'")
+ else:
+ LOG.warning(
+ f"Pixel format '{self._pixel_format}' not in available formats: {node_map.PixelFormat.symbolics}"
+ )
+ except Exception as e:
+ LOG.warning(f"Failed to set pixel format '{self._pixel_format}': {e}")
+
+ def _configure_resolution(self, node_map) -> None:
+ """Configure camera resolution (width and height)."""
+ if self._resolution is None:
+ return
+
+ requested_width, requested_height = self._resolution
+ actual_width, actual_height = None, None
+
+ # Try to set width
+ for width_attr in ("Width", "WidthMax"):
+ try:
+ node = getattr(node_map, width_attr)
+ if width_attr == "Width":
+ # Get constraints
+ try:
+ min_w = node.min
+ max_w = node.max
+ inc_w = getattr(node, "inc", 1)
+ # Adjust to valid value
+ width = self._adjust_to_increment(requested_width, min_w, max_w, inc_w)
+ if width != requested_width:
+ LOG.info(
+ f"Width adjusted from {requested_width} to {width} "
+ f"(min={min_w}, max={max_w}, inc={inc_w})"
+ )
+ node.value = int(width)
+ actual_width = node.value
+ break
+ except Exception as e:
+ # Try setting without adjustment
+ try:
+ node.value = int(requested_width)
+ actual_width = node.value
+ break
+ except Exception:
+ LOG.warning(f"Failed to set width via {width_attr}: {e}")
+ continue
+ except AttributeError:
+ continue
+
+ # Try to set height
+ for height_attr in ("Height", "HeightMax"):
+ try:
+ node = getattr(node_map, height_attr)
+ if height_attr == "Height":
+ # Get constraints
+ try:
+ min_h = node.min
+ max_h = node.max
+ inc_h = getattr(node, "inc", 1)
+ # Adjust to valid value
+ height = self._adjust_to_increment(requested_height, min_h, max_h, inc_h)
+ if height != requested_height:
+ LOG.info(
+ f"Height adjusted from {requested_height} to {height} "
+ f"(min={min_h}, max={max_h}, inc={inc_h})"
+ )
+ node.value = int(height)
+ actual_height = node.value
+ break
+ except Exception as e:
+ # Try setting without adjustment
+ try:
+ node.value = int(requested_height)
+ actual_height = node.value
+ break
+ except Exception:
+ LOG.warning(f"Failed to set height via {height_attr}: {e}")
+ continue
+ except AttributeError:
+ continue
+
+ # Log final resolution
+ if actual_width is not None and actual_height is not None:
+ if actual_width != requested_width or actual_height != requested_height:
+ LOG.warning(
+ f"Resolution mismatch: requested {requested_width}x{requested_height}, "
+ f"got {actual_width}x{actual_height}"
+ )
+ else:
+ LOG.info(f"Resolution set to {actual_width}x{actual_height}")
+ else:
+ LOG.warning(f"Could not verify resolution setting (width={actual_width}, height={actual_height})")
+
+ def _configure_exposure(self, node_map) -> None:
+ if self._exposure is None:
+ return
+
+ # Try to disable auto exposure first
+ for attr in ("ExposureAuto",):
+ try:
+ node = getattr(node_map, attr)
+ node.value = "Off"
+ LOG.info("Auto exposure disabled")
+ break
+ except AttributeError:
+ continue
+ except Exception as e:
+ LOG.warning(f"Failed to disable auto exposure: {e}")
+
+ # Set exposure value
+ for attr in ("ExposureTime", "Exposure"):
+ try:
+ node = getattr(node_map, attr)
+ except AttributeError:
+ continue
+ try:
+ node.value = float(self._exposure)
+ actual = node.value
+ if abs(actual - self._exposure) > 1.0: # Allow 1μs tolerance
+ LOG.warning(f"Exposure mismatch: requested {self._exposure}μs, got {actual}μs")
+ else:
+ LOG.info(f"Exposure set to {actual}μs")
+ return
+ except Exception as e:
+ LOG.warning(f"Failed to set exposure via {attr}: {e}")
+ continue
+
+ LOG.warning(f"Could not set exposure to {self._exposure}μs (no compatible attribute found)")
+
+ def _configure_gain(self, node_map) -> None:
+ if self._gain is None:
+ return
+
+ # Try to disable auto gain first
+ for attr in ("GainAuto",):
+ try:
+ node = getattr(node_map, attr)
+ node.value = "Off"
+ LOG.info("Auto gain disabled")
+ break
+ except AttributeError:
+ continue
+ except Exception as e:
+ LOG.warning(f"Failed to disable auto gain: {e}")
+
+ # Set gain value
+ for attr in ("Gain",):
+ try:
+ node = getattr(node_map, attr)
+ except AttributeError:
+ continue
+ try:
+ node.value = float(self._gain)
+ actual = node.value
+ if abs(actual - self._gain) > 0.1: # Allow 0.1 tolerance
+ LOG.warning(f"Gain mismatch: requested {self._gain}, got {actual}")
+ else:
+ LOG.info(f"Gain set to {actual}")
+ return
+ except Exception as e:
+ LOG.warning(f"Failed to set gain via {attr}: {e}")
+ continue
+
+ LOG.warning(f"Could not set gain to {self._gain} (no compatible attribute found)")
+
+ def _configure_frame_rate(self, node_map) -> None:
+ if not self.settings.fps:
+ return
+
+ target = float(self.settings.fps)
+
+ # Try to enable frame rate control
+ for attr in ("AcquisitionFrameRateEnable", "AcquisitionFrameRateControlEnable"):
+ try:
+ getattr(node_map, attr).value = True
+ LOG.info(f"Frame rate control enabled via {attr}")
+ break
+ except Exception:
+ continue
+
+ # Set frame rate value
+ for attr in ("AcquisitionFrameRate", "ResultingFrameRate", "AcquisitionFrameRateAbs"):
+ try:
+ node = getattr(node_map, attr)
+ except AttributeError:
+ continue
+ try:
+ node.value = target
+ actual = node.value
+ if abs(actual - target) > 0.1:
+ LOG.warning(f"FPS mismatch: requested {target:.2f}, got {actual:.2f}")
+ else:
+ LOG.info(f"Frame rate set to {actual:.2f} FPS")
+ return
+ except Exception as e:
+ LOG.warning(f"Failed to set frame rate via {attr}: {e}")
+ continue
+
+ LOG.warning(f"Could not set frame rate to {target} FPS (no compatible attribute found)")
+
+ def _convert_frame(self, frame: np.ndarray) -> np.ndarray:
+ if frame.dtype != np.uint8:
+ max_val = float(frame.max()) if frame.size else 0.0
+ scale = 255.0 / max_val if max_val > 0.0 else 1.0
+ frame = np.clip(frame * scale, 0, 255).astype(np.uint8)
+
+ if frame.ndim == 2:
+ frame = cv2.cvtColor(frame, cv2.COLOR_GRAY2BGR)
+ elif frame.ndim == 3 and frame.shape[2] == 3 and self._pixel_format == "RGB8":
+ frame = cv2.cvtColor(frame, cv2.COLOR_RGB2BGR)
+
+ if self._crop is not None:
+ top, bottom, left, right = (int(v) for v in self._crop)
+ top = max(0, top)
+ left = max(0, left)
+ bottom = bottom if bottom > 0 else frame.shape[0]
+ right = right if right > 0 else frame.shape[1]
+ bottom = min(frame.shape[0], bottom)
+ right = min(frame.shape[1], right)
+ frame = frame[top:bottom, left:right]
+
+ if self._rotate in (90, 180, 270):
+ rotations = {
+ 90: cv2.ROTATE_90_CLOCKWISE,
+ 180: cv2.ROTATE_180,
+ 270: cv2.ROTATE_90_COUNTERCLOCKWISE,
+ }
+ frame = cv2.rotate(frame, rotations[self._rotate])
+
+ return frame.copy()
+
+ def _resolve_device_label(self, node_map) -> str | None:
+ candidates = [
+ ("DeviceModelName", "DeviceSerialNumber"),
+ ("DeviceDisplayName", "DeviceSerialNumber"),
+ ]
+ for name_attr, serial_attr in candidates:
+ try:
+ model = getattr(node_map, name_attr).value
+ except AttributeError:
+ continue
+ serial = None
+ try:
+ serial = getattr(node_map, serial_attr).value
+ except AttributeError:
+ pass
+ if model:
+ model_str = str(model)
+ serial_str = str(serial) if serial else None
+ return f"{model_str} ({serial_str})" if serial_str else model_str
+ return None
+
+ def _adjust_to_increment(self, value: int, minimum: int, maximum: int, increment: int) -> int:
+ value = max(minimum, min(maximum, int(value)))
+ if increment <= 0:
+ return value
+ offset = value - minimum
+ steps = offset // increment
+ return minimum + steps * increment
+
+ def device_name(self) -> str:
+ if self._device_label:
+ return self._device_label
+ return super().device_name()
diff --git a/dlclivegui/cameras/backends/opencv_backend.py b/dlclivegui/cameras/backends/opencv_backend.py
new file mode 100644
index 0000000..2de4f25
--- /dev/null
+++ b/dlclivegui/cameras/backends/opencv_backend.py
@@ -0,0 +1,396 @@
+"""OpenCV-based camera backend (platform-optimized, fast startup, robust read)."""
+
+from __future__ import annotations
+
+import logging
+import os
+import platform
+import time
+
+import cv2
+import numpy as np
+
+from ..base import CameraBackend, register_backend
+
+logger = logging.getLogger(__name__)
+logger.setLevel(logging.DEBUG) # FIXME @C-Achard remove before release
+
+
+@register_backend("opencv")
+class OpenCVCameraBackend(CameraBackend):
+ """
+ Platform-aware OpenCV backend:
+
+ - Windows: prefer DSHOW, fall back to MSMF/ANY.
+ Order: FOURCC -> resolution -> FPS. Try standard UVC modes if request fails.
+ Optional alt-index probe (index+1) for Logitech-like endpoints: properties["alt_index_probe"]=True
+ Optional fast-start: properties["fast_start"]=True
+
+ - macOS: prefer AVFOUNDATION, fall back to ANY.
+
+ - Linux: prefer V4L2, fall back to GStreamer (if explicitly requested) or ANY.
+ Discovery can use /dev/video* to avoid blind opens (via quick_ping()).
+
+ Robust read(): returns (None, ts) on transient failures (never raises).
+ """
+
+ SAFE_PROP_IDS = {
+ int(getattr(cv2, "CAP_PROP_EXPOSURE", 15)),
+ int(getattr(cv2, "CAP_PROP_AUTO_EXPOSURE", 21)),
+ int(getattr(cv2, "CAP_PROP_GAIN", 14)),
+ int(getattr(cv2, "CAP_PROP_FPS", 5)),
+ int(getattr(cv2, "CAP_PROP_BRIGHTNESS", 10)),
+ int(getattr(cv2, "CAP_PROP_CONTRAST", 11)),
+ int(getattr(cv2, "CAP_PROP_SATURATION", 12)),
+ int(getattr(cv2, "CAP_PROP_HUE", 13)),
+ int(getattr(cv2, "CAP_PROP_CONVERT_RGB", 17)),
+ }
+
+ # Standard UVC modes that commonly succeed fast on Windows/Logitech
+ UVC_FALLBACK_MODES = [(1280, 720), (1920, 1080), (640, 480)]
+
+ def __init__(self, settings):
+ super().__init__(settings)
+ self._capture: cv2.VideoCapture | None = None
+ self._resolution: tuple[int, int] = self._parse_resolution(settings.properties.get("resolution"))
+ self._fast_start: bool = bool(self.settings.properties.get("fast_start", False))
+ self._alt_index_probe: bool = bool(self.settings.properties.get("alt_index_probe", False))
+ self._actual_width: int | None = None
+ self._actual_height: int | None = None
+ self._actual_fps: float | None = None
+ self._codec_str: str = ""
+ self._mjpg_attempted: bool = False
+
+ # ----------------------------
+ # Public API
+ # ----------------------------
+
+ def open(self) -> None:
+ backend_flag = self._preferred_backend_flag(self.settings.properties.get("api"))
+ index = int(self.settings.index)
+
+ # 1) Preferred backend
+ self._capture = self._try_open(index, backend_flag)
+
+ # 2) Optional Logitech endpoint trick (Windows only)
+ if (
+ (not self._capture or not self._capture.isOpened())
+ and platform.system() == "Windows"
+ and self._alt_index_probe
+ ):
+ logger.debug("Primary index failed; trying alternate endpoint (index+1) with same backend.")
+ self._capture = self._try_open(index + 1, backend_flag)
+
+ if not self._capture or not self._capture.isOpened():
+ raise RuntimeError(
+ f"Unable to open camera index {self.settings.index} with OpenCV (backend {backend_flag})"
+ )
+
+ # MSMF hint for slow systems
+ if platform.system() == "Windows" and backend_flag == getattr(cv2, "CAP_MSMF", cv2.CAP_ANY):
+ if os.environ.get("OPENCV_VIDEOIO_MSMF_ENABLE_HW_TRANSFORMS") is None:
+ logger.debug(
+ "MSMF selected. If open is slow, consider setting "
+ "OPENCV_VIDEOIO_MSMF_ENABLE_HW_TRANSFORMS=0 before importing cv2."
+ )
+
+ self._configure_capture()
+
+ def read(self) -> tuple[np.ndarray | None, float]:
+ """Robust frame read: return (None, ts) on transient failures; never raises."""
+ if self._capture is None:
+ logger.warning("OpenCVCameraBackend.read() called before open()")
+ return None, time.time()
+ try:
+ if not self._capture.grab():
+ return None, time.time()
+ success, frame = self._capture.retrieve()
+ if not success or frame is None or frame.size == 0:
+ return None, time.time()
+ return frame, time.time()
+ except Exception as exc:
+ logger.debug(f"OpenCV read transient error: {exc}")
+ return None, time.time()
+
+ def close(self) -> None:
+ self._release_capture()
+
+ def stop(self) -> None:
+ self._release_capture()
+
+ def device_name(self) -> str:
+ base_name = "OpenCV"
+ if self._capture and hasattr(self._capture, "getBackendName"):
+ try:
+ backend_name = self._capture.getBackendName()
+ except Exception:
+ backend_name = ""
+ if backend_name:
+ base_name = backend_name
+ return f"{base_name} camera #{self.settings.index}"
+
+ @property
+ def actual_fps(self) -> float | None:
+ """Return the actual configured FPS, if known."""
+ return self._actual_fps
+
+ @property
+ def actual_resolution(self) -> tuple[int, int] | None:
+ """Return the actual configured resolution, if known."""
+ if self._actual_width and self._actual_height:
+ return (self._actual_width, self._actual_height)
+ return None
+
+ # ----------------------------
+ # Internal helpers
+ # ----------------------------
+
+ def _release_capture(self) -> None:
+ if self._capture:
+ try:
+ self._capture.release()
+ except Exception:
+ pass
+ finally:
+ self._capture = None
+ time.sleep(0.02 if platform.system() == "Windows" else 0.0)
+
+ def _parse_resolution(self, resolution) -> tuple[int, int]:
+ if resolution is None:
+ return (720, 540) # normalized later where needed
+ if isinstance(resolution, (list, tuple)) and len(resolution) == 2:
+ try:
+ return (int(resolution[0]), int(resolution[1]))
+ except (ValueError, TypeError):
+ logger.debug(f"Invalid resolution values: {resolution}, defaulting to 720x540")
+ return (720, 540)
+ return (720, 540)
+
+ def _normalize_resolution(self, width: int, height: int) -> tuple[int, int]:
+ """On Windows, map non-standard requests to UVC-friendly modes for fast acceptance."""
+ if platform.system() == "Windows":
+ if (width, height) in self.UVC_FALLBACK_MODES:
+ return (width, height)
+ logger.debug(f"Normalizing unsupported resolution {width}x{height} to 1280x720 on Windows.")
+ return self.UVC_FALLBACK_MODES[0]
+ return (width, height)
+
+ def _preferred_backend_flag(self, backend: str | None) -> int:
+ """Resolve preferred backend by platform."""
+ if backend: # user override
+ return self._resolve_backend(backend)
+
+ sys = platform.system()
+ if sys == "Windows":
+ # Prefer DSHOW (faster on many Logitech cams), then MSMF, then ANY.
+ return getattr(cv2, "CAP_DSHOW", cv2.CAP_ANY)
+ if sys == "Darwin":
+ return getattr(cv2, "CAP_AVFOUNDATION", cv2.CAP_ANY)
+ # Linux and others
+ return getattr(cv2, "CAP_V4L2", cv2.CAP_ANY)
+
+ def _try_open(self, index: int, preferred_flag: int) -> cv2.VideoCapture | None:
+ """Try opening with preferred backend, then platform-appropriate fallbacks."""
+ # 1) preferred
+ cap = cv2.VideoCapture(index, preferred_flag)
+ if cap.isOpened():
+ return cap
+
+ sys = platform.system()
+
+ # Windows: try MSMF then ANY
+ if sys == "Windows":
+ ms = getattr(cv2, "CAP_MSMF", cv2.CAP_ANY)
+ if preferred_flag != ms:
+ cap = cv2.VideoCapture(index, ms)
+ if cap.isOpened():
+ return cap
+
+ # macOS: ANY fallback
+ if sys == "Darwin":
+ cap = cv2.VideoCapture(index, cv2.CAP_ANY)
+ if cap.isOpened():
+ return cap
+
+ # Linux: try ANY as final fallback
+ cap = cv2.VideoCapture(index, cv2.CAP_ANY)
+ if cap.isOpened():
+ return cap
+ return None
+
+ def _configure_capture(self) -> None:
+ if not self._capture:
+ return
+
+ # --- FOURCC (Windows benefits from setting this first) ---
+ self._codec_str = self._read_codec_string()
+ logger.info(f"Camera using codec: {self._codec_str}")
+
+ if platform.system() == "Windows" and not self._mjpg_attempted:
+ self._maybe_enable_mjpg()
+ self._mjpg_attempted = True
+ self._codec_str = self._read_codec_string()
+ logger.info(f"Camera codec after MJPG attempt: {self._codec_str}")
+
+ # --- Resolution (normalize non-standard on Windows) ---
+ req_w, req_h = self._resolution
+ req_w, req_h = self._normalize_resolution(req_w, req_h)
+
+ if not self._fast_start:
+ self._set_resolution_if_needed(req_w, req_h)
+ else:
+ self._actual_width = int(self._capture.get(cv2.CAP_PROP_FRAME_WIDTH) or 0)
+ self._actual_height = int(self._capture.get(cv2.CAP_PROP_FRAME_HEIGHT) or 0)
+ if self._actual_width and self._actual_height:
+ self.settings.properties["resolution"] = (self._actual_width, self._actual_height)
+
+ # Handle mismatch quickly with a few known-good UVC fallbacks (Windows only)
+ if platform.system() == "Windows" and self._actual_width and self._actual_height:
+ if (self._actual_width, self._actual_height) != (req_w, req_h) and not self._fast_start:
+ logger.warning(
+ f"Resolution mismatch: requested {req_w}x{req_h}, got {self._actual_width}x{self._actual_height}"
+ )
+ for fw, fh in self.UVC_FALLBACK_MODES:
+ if (fw, fh) == (self._actual_width, self._actual_height):
+ break # already at a fallback
+ if self._set_resolution_if_needed(fw, fh, reconfigure_only=True):
+ logger.info(f"Switched to supported resolution {fw}x{fh}")
+ self._actual_width, self._actual_height = fw, fh
+ break
+ self._resolution = (self._actual_width or req_w, self._actual_height or req_h)
+ else:
+ # Non-Windows: accept actual as-is
+ self._resolution = (self._actual_width or req_w, self._actual_height or req_h)
+
+ logger.info(f"Camera configured with resolution: {self._resolution[0]}x{self._resolution[1]}")
+
+ # --- FPS ---
+ requested_fps = float(self.settings.fps or 0.0)
+ if not self._fast_start and requested_fps > 0.0:
+ current_fps = float(self._capture.get(cv2.CAP_PROP_FPS) or 0.0)
+ if current_fps <= 0.0 or abs(current_fps - requested_fps) > 0.1:
+ if not self._capture.set(cv2.CAP_PROP_FPS, requested_fps):
+ logger.debug(f"Device ignored FPS set to {requested_fps:.2f}")
+ self._actual_fps = float(self._capture.get(cv2.CAP_PROP_FPS) or 0.0)
+ else:
+ self._actual_fps = float(self._capture.get(cv2.CAP_PROP_FPS) or 0.0)
+
+ # Log any mismatch
+ if self._actual_fps and requested_fps and abs(self._actual_fps - requested_fps) > 0.1:
+ logger.warning(f"FPS mismatch: requested {requested_fps:.2f}, got {self._actual_fps:.2f}")
+
+ # Always reconcile the settings with what we measured/obtained
+ if self._actual_fps:
+ self.settings.fps = float(self._actual_fps)
+ logger.info(f"Camera configured with FPS: {self._actual_fps:.2f}")
+ logger.debug(
+ "CAP_PROP_FPS requested=%s set_ok=%s get=%s",
+ self.settings.fps,
+ self._capture.set(cv2.CAP_PROP_FPS, float(self.settings.fps)),
+ self._capture.get(cv2.CAP_PROP_FPS),
+ )
+
+ # --- Extra properties (safe whitelist) ---
+ for prop, value in self.settings.properties.items():
+ if prop in ("api", "resolution", "fast_start", "alt_index_probe"):
+ continue
+ try:
+ prop_id = int(prop)
+ except (TypeError, ValueError):
+ logger.debug(f"Ignoring non-numeric property ID: {prop}")
+ continue
+ if prop_id not in self.SAFE_PROP_IDS:
+ logger.debug(f"Skipping unsupported/unsafe property {prop_id}")
+ continue
+ try:
+ if not self._capture.set(prop_id, float(value)):
+ logger.debug(f"Device ignored property {prop_id} -> {value}")
+ except Exception as exc:
+ logger.debug(f"Failed to set property {prop_id} -> {value}: {exc}")
+
+ # ----------------------------
+ # Lower-level helpers
+ # ----------------------------
+
+ def _read_codec_string(self) -> str:
+ try:
+ fourcc = int(self._capture.get(cv2.CAP_PROP_FOURCC) or 0)
+ except Exception:
+ fourcc = 0
+ if fourcc <= 0:
+ return ""
+ return "".join([chr((fourcc >> (8 * i)) & 0xFF) for i in range(4)])
+
+ def _maybe_enable_mjpg(self) -> None:
+ """Attempt to enable MJPG on Windows devices; verify once."""
+ if platform.system() != "Windows":
+ return
+ try:
+ fourcc_mjpg = cv2.VideoWriter_fourcc(*"MJPG")
+ if self._capture.set(cv2.CAP_PROP_FOURCC, fourcc_mjpg):
+ verify = self._read_codec_string()
+ if verify and verify.upper().startswith("MJPG"):
+ logger.info("MJPG enabled successfully.")
+ else:
+ logger.debug(f"MJPG set reported success, but codec is '{verify}'")
+ else:
+ logger.debug("Device rejected MJPG FourCC set.")
+ except Exception as exc:
+ logger.debug(f"MJPG enable attempt raised: {exc}")
+
+ def _set_resolution_if_needed(self, width: int, height: int, reconfigure_only: bool = False) -> bool:
+ """Set width/height only if different.
+ Returns True if the device ends up at the requested size.
+ """
+ try:
+ cur_w = int(self._capture.get(cv2.CAP_PROP_FRAME_WIDTH) or 0)
+ cur_h = int(self._capture.get(cv2.CAP_PROP_FRAME_HEIGHT) or 0)
+ except Exception:
+ cur_w, cur_h = 0, 0
+
+ if (cur_w != width) or (cur_h != height):
+ set_w_ok = self._capture.set(cv2.CAP_PROP_FRAME_WIDTH, float(width))
+ set_h_ok = self._capture.set(cv2.CAP_PROP_FRAME_HEIGHT, float(height))
+ if not set_w_ok:
+ logger.debug(f"Failed to set frame width to {width}")
+ if not set_h_ok:
+ logger.debug(f"Failed to set frame height to {height}")
+
+ try:
+ self._actual_width = int(self._capture.get(cv2.CAP_PROP_FRAME_WIDTH) or 0)
+ self._actual_height = int(self._capture.get(cv2.CAP_PROP_FRAME_HEIGHT) or 0)
+ except Exception:
+ self._actual_width, self._actual_height = 0, 0
+
+ return (self._actual_width, self._actual_height) == (width, height)
+
+ def _resolve_backend(self, backend: str | None) -> int:
+ if backend is None:
+ return cv2.CAP_ANY
+ key = backend.upper()
+ return getattr(cv2, f"CAP_{key}", cv2.CAP_ANY)
+
+ # ----------------------------
+ # Discovery helper (optional use by factory)
+ # ----------------------------
+ @staticmethod
+ def quick_ping(index: int, backend_flag: int | None = None) -> bool:
+ """Cheap 'is-present' check to avoid expensive blind opens during discovery."""
+ sys = platform.system()
+ if sys == "Linux":
+ # /dev/videoN present? That's a cheap, reliable hint.
+ return os.path.exists(f"/dev/video{index}")
+ if backend_flag is None:
+ if sys == "Windows":
+ backend_flag = getattr(cv2, "CAP_DSHOW", cv2.CAP_ANY)
+ elif sys == "Darwin":
+ backend_flag = getattr(cv2, "CAP_AVFOUNDATION", cv2.CAP_ANY)
+ else:
+ backend_flag = getattr(cv2, "CAP_V4L2", cv2.CAP_ANY)
+ cap = cv2.VideoCapture(index, backend_flag)
+ ok = cap.isOpened()
+ try:
+ cap.release()
+ except Exception:
+ pass
+ return ok
diff --git a/dlclivegui/cameras/base.py b/dlclivegui/cameras/base.py
new file mode 100644
index 0000000..6cb2fbe
--- /dev/null
+++ b/dlclivegui/cameras/base.py
@@ -0,0 +1,88 @@
+# dlclivegui/cameras/base.py
+from __future__ import annotations
+
+from abc import ABC, abstractmethod
+
+import numpy as np
+
+from ..utils.config_models import CameraSettingsModel
+
+_BACKEND_REGISTRY: dict[str, type[CameraBackend]] = {}
+
+
+def register_backend(name: str):
+ """
+ Decorator to register a camera backend class.
+
+ Usage:
+ @register_backend("opencv")
+ class OpenCVCameraBackend(CameraBackend):
+ ...
+ """
+
+ def decorator(cls: type[CameraBackend]):
+ if not issubclass(cls, CameraBackend):
+ raise TypeError(f"Backend '{name}' must subclass CameraBackend")
+ _BACKEND_REGISTRY[name.lower()] = cls
+ return cls
+
+ return decorator
+
+
+def register_backend_direct(name: str, cls: type[CameraBackend]):
+ """Allow tests or dynamic plugins to register backends programmatically."""
+ if not issubclass(cls, CameraBackend):
+ raise TypeError(f"Backend '{name}' must subclass CameraBackend")
+ _BACKEND_REGISTRY[name.lower()] = cls
+
+
+def unregister_backend(name: str):
+ """Remove a backend from the registry. Useful for tests."""
+ _BACKEND_REGISTRY.pop(name.lower(), None)
+
+
+def reset_backends():
+ """Clear registry (useful for isolated unit tests)."""
+ _BACKEND_REGISTRY.clear()
+
+
+class CameraBackend(ABC):
+ """Abstract base class for camera backends."""
+
+ def __init__(self, settings: CameraSettingsModel):
+ # Normalize to dataclass so all backends stay unchanged
+ self.settings: CameraSettingsModel = settings
+
+ @classmethod
+ def name(cls) -> str:
+ """Return the backend identifier."""
+ return cls.__name__.lower()
+
+ @classmethod
+ def is_available(cls) -> bool:
+ """Return whether the backend can be used on this system."""
+ return True
+
+ def stop(self) -> None: # noqa B027
+ """Optional: Request a graceful stop. No-op by default."""
+ # Subclasses may override when they need to interrupt blocking reads.
+ pass
+
+ def device_name(self) -> str:
+ """Return a human readable name for the device currently in use."""
+ return self.settings.name
+
+ @abstractmethod
+ def open(self) -> None:
+ """Open the capture device."""
+ raise NotImplementedError
+
+ @abstractmethod
+ def read(self) -> tuple[np.ndarray, float]:
+ """Read a frame and return the image with a timestamp."""
+ raise NotImplementedError
+
+ @abstractmethod
+ def close(self) -> None:
+ """Release the capture device."""
+ raise NotImplementedError
diff --git a/dlclivegui/cameras/config_adapters.py b/dlclivegui/cameras/config_adapters.py
new file mode 100644
index 0000000..2e0bff7
--- /dev/null
+++ b/dlclivegui/cameras/config_adapters.py
@@ -0,0 +1,42 @@
+# dlclivegui/cameras/adapters.py
+from __future__ import annotations
+
+import copy
+from typing import TYPE_CHECKING, Any, Union
+
+if TYPE_CHECKING:
+ from dlclivegui.config import CameraSettingsModel
+
+from dlclivegui.config import CameraSettings
+from dlclivegui.utils.config_models import CameraSettingsModel
+
+CameraSettingsLike = Union[CameraSettings, "CameraSettingsModel", dict[str, Any]]
+
+
+def ensure_dc_camera(settings: CameraSettingsLike) -> CameraSettings:
+ """
+ Normalize any supported camera settings payload to the legacy dataclass CameraSettings.
+ - If already a dataclass: deep-copy and return.
+ - If it's a Pydantic CameraSettingsModel: convert via model_dump().
+ - If it's a dict: unpack into CameraSettings.
+ Ensures default application and type coercions via dataclass.apply_defaults().
+ """
+ # Case 1: Already the dataclass
+ if isinstance(settings, CameraSettings):
+ dc = copy.deepcopy(settings)
+ return dc.apply_defaults()
+
+ # Case 2: Pydantic model (if available in this environment)
+ if CameraSettingsModel is not None and isinstance(settings, CameraSettingsModel):
+ data = settings.model_dump()
+ dc = CameraSettings(**data)
+ return dc.apply_defaults()
+
+ # Case 3: Plain dict (best-effort flexibility)
+ if isinstance(settings, dict):
+ dc = CameraSettings(**settings)
+ return dc.apply_defaults()
+
+ raise TypeError(
+ "Unsupported camera settings type. Expected CameraSettings dataclass, CameraSettingsModel (Pydantic), or dict."
+ )
diff --git a/dlclivegui/cameras/factory.py b/dlclivegui/cameras/factory.py
new file mode 100644
index 0000000..695159a
--- /dev/null
+++ b/dlclivegui/cameras/factory.py
@@ -0,0 +1,329 @@
+"""Backend discovery and construction utilities."""
+
+from __future__ import annotations
+
+import copy
+import importlib
+import pkgutil
+from collections.abc import Callable, Iterable
+from contextlib import contextmanager
+from dataclasses import dataclass
+
+from ..utils.config_models import CameraSettingsModel
+from .base import _BACKEND_REGISTRY, CameraBackend
+
+
+@dataclass
+class DetectedCamera:
+ """Information about a camera discovered during probing."""
+
+ index: int
+ label: str
+
+
+def _opencv_get_log_level(cv2):
+ """Return OpenCV log level using new utils.logging API when available, else legacy."""
+ # Preferred (OpenCV ≥ 4.x): cv2.utils.logging.getLogLevel()
+ try:
+ return cv2.utils.logging.getLogLevel()
+ except Exception:
+ # Legacy (older OpenCV): cv2.getLogLevel()
+ try:
+ return cv2.getLogLevel()
+ except Exception:
+ return None # unknown / not supported
+
+
+def _opencv_set_log_level(cv2, level: int):
+ """Set OpenCV log level using new utils.logging API when available, else legacy."""
+ # Preferred (OpenCV ≥ 4.x): cv2.utils.logging.setLogLevel(level)
+ try:
+ cv2.utils.logging.setLogLevel(level)
+ return
+ except Exception:
+ # Legacy (older OpenCV): cv2.setLogLevel(level)
+ try:
+ cv2.setLogLevel(level)
+ except Exception:
+ pass # not supported on this build
+
+
+@contextmanager
+def _suppress_opencv_logging():
+ """Temporarily suppress OpenCV logging during camera probing (backwards compatible)."""
+ try:
+ import cv2
+
+ # Resolve a 'silent' level cross-version.
+ # In newer OpenCV it's 0 (LOG_LEVEL_SILENT).
+ SILENT = 0
+ old_level = _opencv_get_log_level(cv2)
+
+ _opencv_set_log_level(cv2, SILENT)
+ try:
+ yield
+ finally:
+ # Restore if we were able to read it
+ if old_level is not None:
+ _opencv_set_log_level(cv2, int(old_level))
+ except ImportError:
+ # OpenCV not installed; nothing to suppress
+ yield
+
+
+# Lazy loader for backends (ensures @register_backend runs)
+_BUILTIN_BACKEND_PACKAGES = (
+ "dlclivegui.cameras.backends", # import every submodule once
+)
+_BACKENDS_IMPORTED = False
+
+
+def _ensure_backends_loaded() -> None:
+ """Import all built-in backend modules once so their decorators run."""
+ global _BACKENDS_IMPORTED
+ if _BACKENDS_IMPORTED:
+ return
+
+ for pkg_name in _BUILTIN_BACKEND_PACKAGES:
+ try:
+ pkg = importlib.import_module(pkg_name)
+ except Exception:
+ # Package might not exist (fine if all backends are third-party via tests/plugins)
+ continue
+
+ # Import every submodule of the package (triggers decorator side-effects)
+ pkg_path = getattr(pkg, "__path__", None)
+ if not pkg_path:
+ continue
+
+ for _finder, mod_name, _is_pkg in pkgutil.iter_modules(pkg_path, prefix=pkg_name + "."):
+ try:
+ importlib.import_module(mod_name)
+ except Exception:
+ # Ignore misconfigured/optional backends; they just won't register
+ continue
+
+ _BACKENDS_IMPORTED = True
+
+
+def _sanitize_for_probe(settings: CameraSettingsModel) -> CameraSettingsModel:
+ """
+ Return a light, side-effect-minimized dataclass copy for availability probes.
+ - Zero FPS (let driver pick default)
+ - Keep only 'api' hint in properties, force fast_start=True
+ - Do not change 'enabled'
+ """
+ dc = settings
+ probe = copy.deepcopy(dc)
+ probe.fps = 0.0 # don't force FPS during probe
+ props = probe.properties if isinstance(probe.properties, dict) else {}
+ api = props.get("api")
+ probe.properties = {}
+ if api is not None:
+ probe.properties["api"] = api
+ probe.properties["fast_start"] = True
+ return probe
+
+
+class CameraFactory:
+ """Create camera backend instances based on configuration."""
+
+ @staticmethod
+ def backend_names() -> Iterable[str]:
+ """Return the identifiers of all known backends."""
+ _ensure_backends_loaded()
+ return tuple(_BACKEND_REGISTRY.keys())
+
+ @staticmethod
+ def available_backends() -> dict[str, bool]:
+ """Return a mapping of backend names to availability flags."""
+ _ensure_backends_loaded()
+ availability: dict[str, bool] = {}
+ for name in _BACKEND_REGISTRY:
+ try:
+ backend_cls = CameraFactory._resolve_backend(name)
+ except RuntimeError:
+ availability[name] = False
+ continue
+ availability[name] = backend_cls.is_available()
+ return availability
+
+ @staticmethod
+ def detect_cameras(
+ backend: str,
+ max_devices: int = 10,
+ *,
+ should_cancel: Callable[[], bool] | None = None,
+ progress_cb: Callable[[str], None] | None = None,
+ ) -> list[DetectedCamera]:
+ """Probe ``backend`` for available cameras.
+
+ Parameters
+ ----------
+ backend:
+ The backend identifier, e.g. ``"opencv"``.
+ max_devices:
+ Upper bound for the indices that should be probed.
+ For backends with get_device_count (GenTL, Aravis), the actual device count is queried.
+ should_cancel:
+ Optional callable that returns True if discovery should be canceled.
+ When cancellation is requested, the function returns the cameras found so far.
+ progress_cb:
+ Optional callable to receive human-readable progress messages.
+
+ Returns
+ -------
+ list of :class:`DetectedCamera`
+ Sorted list of detected cameras with human readable labels (partial if canceled).
+ """
+ _ensure_backends_loaded()
+
+ def _canceled() -> bool:
+ return bool(should_cancel and should_cancel())
+
+ try:
+ backend_cls = CameraFactory._resolve_backend(backend)
+ except RuntimeError:
+ return []
+ if not backend_cls.is_available():
+ return []
+
+ # Resolve device count if possible
+ num_devices = max_devices
+ if hasattr(backend_cls, "get_device_count"):
+ try:
+ if _canceled():
+ return []
+ actual_count = backend_cls.get_device_count()
+ if actual_count >= 0:
+ num_devices = actual_count
+ except Exception:
+ pass
+
+ detected: list[DetectedCamera] = []
+ # Suppress OpenCV warnings/errors during probing (e.g., "can't open camera by index")
+ with _suppress_opencv_logging():
+ try:
+ for index in range(num_devices):
+ if _canceled():
+ # return partial results immediately
+ break
+
+ if progress_cb:
+ progress_cb(f"Probing {backend}:{index}…")
+
+ # Prefer quick presence check first
+ quick_ok = None
+ if hasattr(backend_cls, "quick_ping"):
+ try:
+ quick_ok = bool(backend_cls.quick_ping(index)) # type: ignore[attr-defined]
+ except TypeError:
+ quick_ok = bool(backend_cls.quick_ping(index, None)) # type: ignore[attr-defined]
+ except Exception:
+ quick_ok = None
+ if quick_ok is False:
+ # Definitely not present, skip heavy open
+ continue
+
+ settings = CameraSettingsModel(
+ name=f"Probe {index}",
+ index=index,
+ fps=30.0,
+ backend=backend,
+ properties={},
+ )
+ backend_instance = backend_cls(settings)
+
+ try:
+ # This open() may block for a short time depending on driver/backend.
+ backend_instance.open()
+ except Exception:
+ # Not available → continue probing next index
+ pass
+ else:
+ label = backend_instance.device_name() or f"{backend.title()} #{index}"
+ detected.append(DetectedCamera(index=index, label=label))
+ if progress_cb:
+ progress_cb(f"Found {label}")
+ finally:
+ try:
+ backend_instance.close()
+ except Exception:
+ pass
+
+ # Check cancel again between indices
+ if _canceled():
+ break
+
+ except KeyboardInterrupt:
+ # Graceful early exit with partial results
+ if progress_cb:
+ progress_cb("Discovery interrupted.")
+ # any other exception bubbles up to caller
+
+ detected.sort(key=lambda camera: camera.index)
+ return detected
+
+ @staticmethod
+ def create(settings: CameraSettingsModel) -> CameraBackend:
+ """Instantiate a backend for ``settings``."""
+ dc = settings
+ backend_name = (dc.backend or "opencv").lower()
+ try:
+ backend_cls = CameraFactory._resolve_backend(backend_name)
+ except RuntimeError as exc: # pragma: no cover - runtime configuration
+ raise RuntimeError(f"Unknown camera backend '{backend_name}': {exc}") from exc
+ if not backend_cls.is_available():
+ raise RuntimeError(
+ f"Camera backend '{backend_name}' is not available. "
+ "Ensure the required drivers and Python packages are installed."
+ )
+ return backend_cls(dc)
+
+ @staticmethod
+ def check_camera_available(settings: CameraSettingsModel) -> tuple[bool, str]:
+ """Check if a camera is present/accessible without pushing heavy settings like FPS."""
+ dc = settings
+ backend_name = (dc.backend or "opencv").lower()
+
+ try:
+ backend_cls = CameraFactory._resolve_backend(backend_name)
+ except RuntimeError as exc:
+ return False, f"Backend '{backend_name}' not installed: {exc}"
+
+ if not backend_cls.is_available():
+ return False, f"Backend '{backend_name}' is not available (missing drivers/packages)"
+
+ # Prefer quick presence test
+ if hasattr(backend_cls, "quick_ping"):
+ try:
+ with _suppress_opencv_logging():
+ idx = int(dc.index)
+ ok = False
+ try:
+ ok = backend_cls.quick_ping(idx) # type: ignore[attr-defined]
+ except TypeError:
+ ok = backend_cls.quick_ping(idx, None) # type: ignore[attr-defined]
+ if ok:
+ return True, ""
+ return False, "Device not present"
+ except Exception as exc:
+ return False, f"Quick probe failed: {exc}"
+
+ # Fallback: lightweight open/close with sanitized settings
+ try:
+ probe_settings = _sanitize_for_probe(dc)
+ backend_instance = backend_cls(probe_settings)
+ with _suppress_opencv_logging():
+ backend_instance.open()
+ backend_instance.close()
+ return True, ""
+ except Exception as exc:
+ return False, f"Camera not accessible: {exc}"
+
+ @staticmethod
+ def _resolve_backend(name: str) -> type[CameraBackend]:
+ try:
+ return _BACKEND_REGISTRY[name.lower()]
+ except KeyError as exc:
+ raise RuntimeError("Backend %s not registered", name) from exc
diff --git a/dlclivegui/config.py b/dlclivegui/config.py
new file mode 100644
index 0000000..c7f862c
--- /dev/null
+++ b/dlclivegui/config.py
@@ -0,0 +1,318 @@
+"""Configuration helpers for the DLC Live GUI."""
+
+from __future__ import annotations
+
+import json
+from dataclasses import asdict, dataclass, field
+from pathlib import Path
+from typing import Any
+
+from PySide6.QtCore import QSettings
+
+from dlclivegui.utils.utils import is_model_file
+
+
+@dataclass
+class CameraSettings:
+ """Configuration for a single camera device."""
+
+ name: str = "Camera 0"
+ index: int = 0
+ fps: float = 25.0
+ backend: str = "gentl"
+ exposure: int = 500 # 0 = auto, otherwise microseconds
+ gain: float = 10 # 0.0 = auto, otherwise gain value
+ crop_x0: int = 0 # Left edge of crop region (0 = no crop)
+ crop_y0: int = 0 # Top edge of crop region (0 = no crop)
+ crop_x1: int = 0 # Right edge of crop region (0 = no crop)
+ crop_y1: int = 0 # Bottom edge of crop region (0 = no crop)
+ max_devices: int = 3 # Maximum number of devices to probe during detection
+ rotation: int = 0 # Rotation degrees (0, 90, 180, 270)
+ enabled: bool = True # Whether this camera is active in multi-camera mode
+ properties: dict[str, Any] = field(default_factory=dict)
+
+ def apply_defaults(self) -> CameraSettings:
+ """Ensure fps is a positive number and validate crop settings."""
+
+ self.fps = float(self.fps) if self.fps else 30.0
+ self.exposure = int(self.exposure) if self.exposure else 0
+ self.gain = float(self.gain) if self.gain else 0.0
+ self.crop_x0 = max(0, int(self.crop_x0)) if hasattr(self, "crop_x0") else 0
+ self.crop_y0 = max(0, int(self.crop_y0)) if hasattr(self, "crop_y0") else 0
+ self.crop_x1 = max(0, int(self.crop_x1)) if hasattr(self, "crop_x1") else 0
+ self.crop_y1 = max(0, int(self.crop_y1)) if hasattr(self, "crop_y1") else 0
+ return self
+
+ def get_crop_region(self) -> tuple[int, int, int, int] | None:
+ """Get crop region as (x0, y0, x1, y1) or None if no cropping."""
+ if self.crop_x0 == 0 and self.crop_y0 == 0 and self.crop_x1 == 0 and self.crop_y1 == 0:
+ return None
+ return (self.crop_x0, self.crop_y0, self.crop_x1, self.crop_y1)
+
+ def copy(self) -> CameraSettings:
+ """Create a copy of this settings object."""
+ return CameraSettings(
+ name=self.name,
+ index=self.index,
+ fps=self.fps,
+ backend=self.backend,
+ exposure=self.exposure,
+ gain=self.gain,
+ crop_x0=self.crop_x0,
+ crop_y0=self.crop_y0,
+ crop_x1=self.crop_x1,
+ crop_y1=self.crop_y1,
+ max_devices=self.max_devices,
+ rotation=self.rotation,
+ enabled=self.enabled,
+ properties=dict(self.properties),
+ )
+
+
+@dataclass
+class MultiCameraSettings:
+ """Configuration for multiple cameras."""
+
+ cameras: list = field(default_factory=list) # List of CameraSettings
+ max_cameras: int = 4 # Maximum number of cameras that can be active
+ tile_layout: str = "auto" # "auto", "2x2", "1x4", "4x1"
+
+ def get_active_cameras(self) -> list:
+ """Get list of enabled cameras."""
+ return [cam for cam in self.cameras if cam.enabled]
+
+ def add_camera(self, settings: CameraSettings) -> bool:
+ """Add a camera to the configuration. Returns True if successful."""
+ if len(self.get_active_cameras()) >= self.max_cameras and settings.enabled:
+ return False
+ self.cameras.append(settings)
+ return True
+
+ def remove_camera(self, index: int) -> bool:
+ """Remove camera at the given list index."""
+ if 0 <= index < len(self.cameras):
+ del self.cameras[index]
+ return True
+ return False
+
+ @classmethod
+ def from_dict(cls, data: dict[str, Any]) -> MultiCameraSettings:
+ """Create MultiCameraSettings from a dictionary."""
+ cameras = []
+ for cam_data in data.get("cameras", []):
+ cam = CameraSettings(**cam_data)
+ cam.apply_defaults()
+ cameras.append(cam)
+ return cls(
+ cameras=cameras,
+ max_cameras=data.get("max_cameras", 4),
+ tile_layout=data.get("tile_layout", "auto"),
+ )
+
+ def to_dict(self) -> dict[str, Any]:
+ """Convert to dictionary for serialization."""
+ return {
+ "cameras": [asdict(cam) for cam in self.cameras],
+ "max_cameras": self.max_cameras,
+ "tile_layout": self.tile_layout,
+ }
+
+
+@dataclass
+class DLCProcessorSettings:
+ """Configuration for DLCLive processing."""
+
+ model_path: str = ""
+ model_directory: str = "." # Default directory for model browser (current dir if not set)
+ device: str | None = (
+ "auto" # Device for inference (e.g., "cuda:0", "cpu"). None should be auto, but might default to cpu
+ )
+ dynamic: tuple = (False, 0.5, 10) # Dynamic cropping: (enabled, margin, max_missing_frames)
+ resize: float = 1.0 # Resize factor for input frames
+ precision: str = "FP32" # Inference precision ("FP32", "FP16")
+ additional_options: dict[str, Any] = field(default_factory=dict)
+ model_type: str = "pytorch" # Only PyTorch models are supported
+ single_animal: bool = True # Only single-animal models are supported
+
+
+@dataclass
+class BoundingBoxSettings:
+ """Configuration for bounding box visualization."""
+
+ enabled: bool = False
+ x0: int = 0
+ y0: int = 0
+ x1: int = 200
+ y1: int = 100
+
+
+@dataclass
+class VisualizationSettings:
+ """Configuration for pose visualization."""
+
+ p_cutoff: float = 0.6 # Confidence threshold for displaying keypoints
+ colormap: str = "hot" # Matplotlib colormap for keypoints
+ bbox_color: tuple[int, int, int] = (0, 0, 255) # BGR color for bounding box (default: red)
+
+ def get_bbox_color_bgr(self) -> tuple[int, int, int]:
+ """Get bounding box color in BGR format."""
+ if isinstance(self.bbox_color, (list, tuple)) and len(self.bbox_color) == 3:
+ return tuple(int(c) for c in self.bbox_color)
+ return (0, 0, 255) # Default to red
+
+
+@dataclass
+class RecordingSettings:
+ """Configuration for video recording."""
+
+ enabled: bool = False
+ directory: str = str(Path.home() / "Videos" / "deeplabcut-live")
+ filename: str = "session.mp4"
+ container: str = "mp4"
+ codec: str = "libx264"
+ crf: int = 23
+
+ def output_path(self) -> Path:
+ """Return the absolute output path for recordings."""
+
+ directory = Path(self.directory).expanduser().resolve()
+ directory.mkdir(parents=True, exist_ok=True)
+ name = Path(self.filename)
+ if name.suffix:
+ filename = name
+ else:
+ filename = name.with_suffix(f".{self.container}")
+ return directory / filename
+
+ def writegear_options(self, fps: float) -> dict[str, Any]:
+ """Return compression parameters for WriteGear."""
+
+ fps_value = float(fps) if fps else 30.0
+ codec_value = (self.codec or "libx264").strip() or "libx264"
+ crf_value = int(self.crf) if self.crf is not None else 23
+ return {
+ "-input_framerate": f"{fps_value:.6f}",
+ "-vcodec": codec_value,
+ "-crf": str(crf_value),
+ }
+
+
+@dataclass
+class ApplicationSettings:
+ """Top level application configuration."""
+
+ camera: CameraSettings = field(default_factory=CameraSettings)
+ multi_camera: MultiCameraSettings = field(default_factory=MultiCameraSettings)
+ dlc: DLCProcessorSettings = field(default_factory=DLCProcessorSettings)
+ recording: RecordingSettings = field(default_factory=RecordingSettings)
+ bbox: BoundingBoxSettings = field(default_factory=BoundingBoxSettings)
+ visualization: VisualizationSettings = field(default_factory=VisualizationSettings)
+
+ @classmethod
+ def from_dict(cls, data: dict[str, Any]) -> ApplicationSettings:
+ """Create an :class:`ApplicationSettings` from a dictionary."""
+
+ camera = CameraSettings(**data.get("camera", {})).apply_defaults()
+
+ # Parse multi-camera settings
+ multi_camera_data = data.get("multi_camera", {})
+ if multi_camera_data:
+ multi_camera = MultiCameraSettings.from_dict(multi_camera_data)
+ else:
+ multi_camera = MultiCameraSettings()
+
+ dlc_data = dict(data.get("dlc", {}))
+ # Parse dynamic parameter - can be list or tuple in JSON
+ dynamic_raw = dlc_data.get("dynamic", [False, 0.5, 10])
+ if isinstance(dynamic_raw, (list, tuple)) and len(dynamic_raw) == 3:
+ dynamic = tuple(dynamic_raw)
+ else:
+ dynamic = (False, 0.5, 10)
+ dlc = DLCProcessorSettings(
+ model_path=str(dlc_data.get("model_path", "")),
+ model_directory=str(dlc_data.get("model_directory", ".")),
+ device=dlc_data.get("device"), # None if not specified
+ dynamic=dynamic,
+ resize=float(dlc_data.get("resize", 1.0)),
+ precision=str(dlc_data.get("precision", "FP32")),
+ additional_options=dict(dlc_data.get("additional_options", {})),
+ )
+ recording_data = dict(data.get("recording", {}))
+ recording_data.pop("options", None)
+ recording = RecordingSettings(**recording_data)
+ bbox = BoundingBoxSettings(**data.get("bbox", {}))
+ visualization = VisualizationSettings(**data.get("visualization", {}))
+ return cls(
+ camera=camera,
+ multi_camera=multi_camera,
+ dlc=dlc,
+ recording=recording,
+ bbox=bbox,
+ visualization=visualization,
+ )
+
+ def to_dict(self) -> dict[str, Any]:
+ """Serialise the configuration to a dictionary."""
+
+ return {
+ "camera": asdict(self.camera),
+ "multi_camera": self.multi_camera.to_dict(),
+ "dlc": asdict(self.dlc),
+ "recording": asdict(self.recording),
+ "bbox": asdict(self.bbox),
+ "visualization": asdict(self.visualization),
+ }
+
+ @classmethod
+ def load(cls, path: Path | str) -> ApplicationSettings:
+ """Load configuration from ``path``."""
+
+ file_path = Path(path).expanduser()
+ if not file_path.exists():
+ raise FileNotFoundError(f"Configuration file not found: {file_path}")
+ with file_path.open("r", encoding="utf-8") as handle:
+ data = json.load(handle)
+ return cls.from_dict(data)
+
+ def save(self, path: Path | str) -> None:
+ """Persist configuration to ``path``."""
+
+ file_path = Path(path).expanduser()
+ file_path.parent.mkdir(parents=True, exist_ok=True)
+ with file_path.open("w", encoding="utf-8") as handle:
+ json.dump(self.to_dict(), handle, indent=2)
+
+
+DEFAULT_CONFIG = ApplicationSettings()
+
+
+class ModelPathStore:
+ """Persist and resolve the last model path via QSettings."""
+
+ def __init__(self, settings: QSettings | None = None):
+ self._settings = settings or QSettings("DeepLabCut", "DLCLiveGUI")
+
+ def load_last(self) -> str | None:
+ val = self._settings.value("dlc/last_model_path")
+ if not val:
+ return None
+ path = str(val)
+ try:
+ return path if is_model_file(path) else None
+ except Exception:
+ return None
+
+ def save_if_valid(self, path: str) -> None:
+ try:
+ if path and is_model_file(path):
+ self._settings.setValue("dlc/last_model_path", str(Path(path)))
+ except Exception:
+ pass
+
+ def resolve(self, config_path: str | None) -> str:
+ if config_path and is_model_file(config_path):
+ return config_path
+ persisted = self.load_last()
+ if persisted and is_model_file(persisted):
+ return persisted
+ return ""
diff --git a/dlclivegui/dlclivegui.py b/dlclivegui/dlclivegui.py
deleted file mode 100644
index 2a29d7d..0000000
--- a/dlclivegui/dlclivegui.py
+++ /dev/null
@@ -1,1498 +0,0 @@
-"""
-DeepLabCut Toolbox (deeplabcut.org)
-© A. & M. Mathis Labs
-
-Licensed under GNU Lesser General Public License v3.0
-"""
-
-
-from tkinter import (
- Tk,
- Toplevel,
- Label,
- Entry,
- Button,
- Radiobutton,
- Checkbutton,
- StringVar,
- IntVar,
- BooleanVar,
- filedialog,
- messagebox,
- simpledialog,
-)
-from tkinter.ttk import Combobox
-import os
-import sys
-import glob
-import json
-import datetime
-import inspect
-import importlib
-
-from PIL import Image, ImageTk, ImageDraw
-import colorcet as cc
-
-from dlclivegui import CameraPoseProcess
-from dlclivegui import processor
-from dlclivegui import camera
-from dlclivegui.tkutil import SettingsWindow
-
-
-class DLCLiveGUI(object):
- """ GUI to run DLC Live experiment
- """
-
- def __init__(self):
- """ Constructor method
- """
-
- ### check if documents path exists
-
- if not os.path.isdir(self.get_docs_path()):
- os.mkdir(self.get_docs_path())
- if not os.path.isdir(os.path.dirname(self.get_config_path(""))):
- os.mkdir(os.path.dirname(self.get_config_path("")))
-
- ### get configuration ###
-
- self.cfg_list = [
- os.path.splitext(os.path.basename(f))[0]
- for f in glob.glob(os.path.dirname(self.get_config_path("")) + "/*.json")
- ]
-
- ### initialize variables
-
- self.cam_pose_proc = None
- self.dlc_proc_params = None
-
- self.display_window = None
- self.display_cmap = None
- self.display_colors = None
- self.display_radius = None
- self.display_lik_thresh = None
-
- ### create GUI window ###
-
- self.createGUI()
-
- def get_docs_path(self):
- """ Get path to documents folder
-
- Returns
- -------
- str
- path to documents folder
- """
-
- return os.path.normpath(os.path.expanduser("~/Documents/DeepLabCut-live-GUI"))
-
- def get_config_path(self, cfg_name):
- """ Get path to configuration foler
-
- Parameters
- ----------
- cfg_name : str
- name of config file
-
- Returns
- -------
- str
- path to configuration file
- """
-
- return os.path.normpath(self.get_docs_path() + "/config/" + cfg_name + ".json")
-
- def get_config(self, cfg_name):
- """ Read configuration
-
- Parameters
- ----------
- cfg_name : str
- name of configuration
- """
-
- ### read configuration file ###
-
- self.cfg_file = self.get_config_path(cfg_name)
- if os.path.isfile(self.cfg_file):
- cfg = json.load(open(self.cfg_file))
- else:
- cfg = {}
-
- ### check config ###
-
- cfg["cameras"] = {} if "cameras" not in cfg else cfg["cameras"]
- cfg["processor_dir"] = (
- [] if "processor_dir" not in cfg else cfg["processor_dir"]
- )
- cfg["processor_args"] = (
- {} if "processor_args" not in cfg else cfg["processor_args"]
- )
- cfg["dlc_options"] = {} if "dlc_options" not in cfg else cfg["dlc_options"]
- cfg["dlc_display_options"] = (
- {} if "dlc_display_options" not in cfg else cfg["dlc_display_options"]
- )
- cfg["subjects"] = [] if "subjects" not in cfg else cfg["subjects"]
- cfg["directories"] = [] if "directories" not in cfg else cfg["directories"]
-
- self.cfg = cfg
-
- def change_config(self, event=None):
- """ Change configuration, update GUI menus
-
- Parameters
- ----------
- event : tkinter event, optional
- event , by default None
- """
-
- if self.cfg_name.get() == "Create New Config":
- new_name = simpledialog.askstring(
- "", "Please enter a name (no special characters).", parent=self.window
- )
- self.cfg_name.set(new_name)
- self.get_config(self.cfg_name.get())
-
- self.camera_entry["values"] = tuple(self.cfg["cameras"].keys()) + (
- "Add Camera",
- )
- self.camera_name.set("")
- self.dlc_proc_dir_entry["values"] = tuple(self.cfg["processor_dir"])
- self.dlc_proc_dir.set("")
- self.dlc_proc_name_entry["values"] = tuple()
- self.dlc_proc_name.set("")
- self.dlc_options_entry["values"] = tuple(self.cfg["dlc_options"].keys()) + (
- "Add DLC",
- )
- self.dlc_option.set("")
- self.subject_entry["values"] = tuple(self.cfg["subjects"])
- self.subject.set("")
- self.directory_entry["values"] = tuple(self.cfg["directories"])
- self.directory.set("")
-
- def remove_config(self):
- """ Remove configuration
- """
-
- cfg_name = self.cfg_name.get()
- delete_setup = messagebox.askyesnocancel(
- "Delete Config Permanently?",
- "Would you like to delete the configuration {} permanently (yes),\nremove the setup from the list for this session (no),\nor neither (cancel).".format(
- cfg_name
- ),
- parent=self.window,
- )
- if delete_setup is not None:
- if delete_setup:
- os.remove(self.get_config_path(cfg_name))
- self.cfg_list.remove(cfg_name)
- self.cfg_entry["values"] = tuple(self.cfg_list) + ("Create New Setup",)
- self.cfg_name.set("")
-
- def get_camera_names(self):
- """ Get camera names from configuration as a tuple
- """
-
- return tuple(self.cfg["cameras"].keys())
-
- def init_cam(self):
- """ Initialize camera
- """
-
- if self.cam_pose_proc is not None:
- messagebox.showerror(
- "Camera Exists",
- "Camera already exists! Please close current camera before initializing a new one.",
- )
- return
-
- this_cam = self.get_current_camera()
-
- if not this_cam:
-
- messagebox.showerror(
- "No Camera",
- "No camera selected. Please select a camera before initializing.",
- parent=self.window,
- )
-
- else:
-
- if this_cam["type"] == "Add Camera":
-
- self.add_camera_window()
- return
-
- else:
-
- self.cam_setup_window = Toplevel(self.window)
- self.cam_setup_window.title("Setting up camera...")
- Label(
- self.cam_setup_window, text="Setting up camera, please wait..."
- ).pack()
- self.cam_setup_window.update()
-
- cam_obj = getattr(camera, this_cam["type"])
- cam = cam_obj(**this_cam["params"])
- self.cam_pose_proc = CameraPoseProcess(cam)
- ret = self.cam_pose_proc.start_capture_process()
-
- if cam.use_tk_display:
- self.set_display_window()
-
- self.cam_setup_window.destroy()
-
- def get_current_camera(self):
- """ Get dictionary of the current camera
- """
-
- if self.camera_name.get():
- if self.camera_name.get() == "Add Camera":
- return {"type": "Add Camera"}
- else:
- return self.cfg["cameras"][self.camera_name.get()]
-
- def set_camera_param(self, key, value):
- """ Set a camera parameter
- """
-
- self.cfg["cameras"][self.camera_name.get()]["params"][key] = value
-
- def add_camera_window(self):
- """ Create gui to add a camera
- """
-
- add_cam = Tk()
- cur_row = 0
-
- Label(add_cam, text="Type: ").grid(sticky="w", row=cur_row, column=0)
- self.cam_type = StringVar(add_cam)
-
- cam_types = [c[0] for c in inspect.getmembers(camera, inspect.isclass)]
- cam_types = [c for c in cam_types if (c != "Camera") & ("Error" not in c)]
-
- type_entry = Combobox(add_cam, textvariable=self.cam_type, state="readonly")
- type_entry["values"] = tuple(cam_types)
- type_entry.current(0)
- type_entry.grid(sticky="nsew", row=cur_row, column=1)
- cur_row += 1
-
- Label(add_cam, text="Name: ").grid(sticky="w", row=cur_row, column=0)
- self.new_cam_name = StringVar(add_cam)
- Entry(add_cam, textvariable=self.new_cam_name).grid(
- sticky="nsew", row=cur_row, column=1
- )
- cur_row += 1
-
- Button(
- add_cam, text="Add Camera", command=lambda: self.add_cam_to_list(add_cam)
- ).grid(sticky="nsew", row=cur_row, column=1)
- cur_row += 1
-
- Button(add_cam, text="Cancel", command=add_cam.destroy).grid(
- sticky="nsew", row=cur_row, column=1
- )
-
- add_cam.mainloop()
-
- def add_cam_to_list(self, gui):
- """ Add new camera to the camera list
- """
-
- self.cfg["cameras"][self.new_cam_name.get()] = {
- "type": self.cam_type.get(),
- "params": {},
- }
- self.camera_name.set(self.new_cam_name.get())
- self.camera_entry["values"] = self.get_camera_names() + ("Add Camera",)
- self.save_config()
- # messagebox.showinfo("Camera Added", "Camera has been added to the dropdown menu. Please edit camera settings before initializing the new camera.", parent=gui)
- gui.destroy()
-
- def edit_cam_settings(self):
- """ GUI window to edit camera settings
- """
-
- arg_names, arg_vals, arg_dtypes, arg_restrict = self.get_cam_args()
-
- settings_window = Toplevel(self.window)
- settings_window.title("Camera Settings")
- cur_row = 0
- combobox_width = 15
-
- entry_vars = []
- for n, v in zip(arg_names, arg_vals):
-
- Label(settings_window, text=n + ": ").grid(row=cur_row, column=0)
-
- if type(v) is list:
- v = [str(x) if x is not None else "" for x in v]
- v = ", ".join(v)
- else:
- v = v if v is not None else ""
- entry_vars.append(StringVar(settings_window, value=str(v)))
-
- if n in arg_restrict.keys():
- restrict_vals = arg_restrict[n]
- if type(restrict_vals[0]) is list:
- restrict_vals = [
- ", ".join([str(i) for i in rv]) for rv in restrict_vals
- ]
- Combobox(
- settings_window,
- textvariable=entry_vars[-1],
- values=restrict_vals,
- state="readonly",
- width=combobox_width,
- ).grid(sticky="nsew", row=cur_row, column=1)
- else:
- Entry(settings_window, textvariable=entry_vars[-1]).grid(
- sticky="nsew", row=cur_row, column=1
- )
-
- cur_row += 1
-
- cur_row += 1
- Button(
- settings_window,
- text="Update",
- command=lambda: self.update_camera_settings(
- arg_names, entry_vars, arg_dtypes, settings_window
- ),
- ).grid(sticky="nsew", row=cur_row, column=1)
- cur_row += 1
- Button(settings_window, text="Cancel", command=settings_window.destroy).grid(
- sticky="nsew", row=cur_row, column=1
- )
-
- _, row_count = settings_window.grid_size()
- for r in range(row_count):
- settings_window.grid_rowconfigure(r, minsize=20)
-
- settings_window.mainloop()
-
- def get_cam_args(self):
- """ Get arguments for the new camera
- """
-
- this_cam = self.get_current_camera()
- cam_obj = getattr(camera, this_cam["type"])
- arg_restrict = cam_obj.arg_restrictions()
-
- cam_args = inspect.getfullargspec(cam_obj)
- n_args = len(cam_args[0][1:])
- n_vals = len(cam_args[3])
- arg_names = []
- arg_vals = []
- arg_dtype = []
- for i in range(n_args):
- arg_names.append(cam_args[0][i + 1])
-
- if arg_names[i] in this_cam["params"].keys():
- val = this_cam["params"][arg_names[i]]
- else:
- val = None if i < n_args - n_vals else cam_args[3][n_vals - n_args + i]
- arg_vals.append(val)
-
- dt_val = val if i < n_args - n_vals else cam_args[3][n_vals - n_args + i]
- dt = type(dt_val) if type(dt_val) is not list else type(dt_val[0])
- arg_dtype.append(dt)
-
- return arg_names, arg_vals, arg_dtype, arg_restrict
-
- def update_camera_settings(self, names, entries, dtypes, gui):
- """ Update camera settings from values input in settings GUI
- """
-
- gui.destroy()
-
- for name, entry, dt in zip(names, entries, dtypes):
- val = entry.get()
- val = val.split(",")
- val = [v.strip() for v in val]
- try:
- if dt is bool:
- val = [True if v == "True" else False for v in val]
- else:
- val = [dt(v) if v else None for v in val]
- except TypeError:
- pass
- val = val if len(val) > 1 else val[0]
- self.set_camera_param(name, val)
-
- self.save_config()
-
- def set_display_window(self):
- """ Create a video display window
- """
-
- self.display_window = Toplevel(self.window)
- self.display_frame_label = Label(self.display_window)
- self.display_frame_label.pack()
- self.display_frame()
-
- def set_display_colors(self, bodyparts):
- """ Set colors for keypoints
-
- Parameters
- ----------
- bodyparts : int
- the number of keypoints
- """
-
- all_colors = getattr(cc, self.display_cmap)
- self.display_colors = all_colors[:: int(len(all_colors) / bodyparts)]
-
- def display_frame(self):
- """ Display a frame in display window
- """
-
- if self.cam_pose_proc and self.display_window:
-
- frame = self.cam_pose_proc.get_display_frame()
-
- if frame is not None:
-
- img = Image.fromarray(frame)
- if frame.ndim == 3:
- b, g, r = img.split()
- img = Image.merge("RGB", (r, g, b))
-
- pose = (
- self.cam_pose_proc.get_display_pose()
- if self.display_keypoints.get()
- else None
- )
-
- if pose is not None:
-
- im_size = (frame.shape[1], frame.shape[0])
-
- if not self.display_colors:
- self.set_display_colors(pose.shape[0])
-
- img_draw = ImageDraw.Draw(img)
-
- for i in range(pose.shape[0]):
- if pose[i, 2] > self.display_lik_thresh:
- try:
- x0 = (
- pose[i, 0] - self.display_radius
- if pose[i, 0] - self.display_radius > 0
- else 0
- )
- x1 = (
- pose[i, 0] + self.display_radius
- if pose[i, 0] + self.display_radius < im_size[1]
- else im_size[1]
- )
- y0 = (
- pose[i, 1] - self.display_radius
- if pose[i, 1] - self.display_radius > 0
- else 0
- )
- y1 = (
- pose[i, 1] + self.display_radius
- if pose[i, 1] + self.display_radius < im_size[0]
- else im_size[0]
- )
- coords = [x0, y0, x1, y1]
- img_draw.ellipse(
- coords,
- fill=self.display_colors[i],
- outline=self.display_colors[i],
- )
- except Exception as e:
- print(e)
-
- imgtk = ImageTk.PhotoImage(image=img)
- self.display_frame_label.imgtk = imgtk
- self.display_frame_label.configure(image=imgtk)
-
- self.display_frame_label.after(10, self.display_frame)
-
- def change_display_keypoints(self):
- """ Toggle display keypoints. If turning on, set display options. If turning off, destroy display window
- """
-
- if self.display_keypoints.get():
-
- display_options = self.cfg["dlc_display_options"][
- self.dlc_option.get()
- ].copy()
- self.display_cmap = display_options["cmap"]
- self.display_radius = display_options["radius"]
- self.display_lik_thresh = display_options["lik_thresh"]
-
- if not self.display_window:
- self.set_display_window()
-
- else:
-
- if self.cam_pose_proc is not None:
- if not self.cam_pose_proc.device.use_tk_display:
- if self.display_window:
- self.display_window.destroy()
- self.display_window = None
- self.display_colors = None
-
- def edit_dlc_display(self):
-
- display_options = self.cfg["dlc_display_options"][self.dlc_option.get()]
-
- dlc_display_settings = {
- "color map": {
- "value": display_options["cmap"],
- "dtype": str,
- "restriction": ["bgy", "kbc", "bmw", "bmy", "kgy", "fire"],
- },
- "radius": {"value": display_options["radius"], "dtype": int},
- "likelihood threshold": {
- "value": display_options["lik_thresh"],
- "dtype": float,
- },
- }
-
- dlc_display_gui = SettingsWindow(
- title="Edit DLC Display Settings",
- settings=dlc_display_settings,
- parent=self.window,
- )
-
- dlc_display_gui.mainloop()
- display_settings = dlc_display_gui.get_values()
-
- display_options["cmap"] = display_settings["color map"]
- display_options["radius"] = display_settings["radius"]
- display_options["lik_thresh"] = display_settings["likelihood threshold"]
-
- self.display_cmap = display_options["cmap"]
- self.display_radius = display_options["radius"]
- self.display_lik_thresh = display_options["lik_thresh"]
-
- self.cfg["dlc_display_options"][self.dlc_option.get()] = display_options
- self.save_config()
-
- def close_camera(self):
- """ Close capture process and display
- """
-
- if self.cam_pose_proc:
- if self.display_window is not None:
- self.display_window.destroy()
- self.display_window = None
- ret = self.cam_pose_proc.stop_capture_process()
-
- self.cam_pose_proc = None
-
- def change_dlc_option(self, event=None):
-
- if self.dlc_option.get() == "Add DLC":
- self.edit_dlc_settings(True)
-
- def edit_dlc_settings(self, new=False):
-
- if new:
- cur_set = self.empty_dlc_settings()
- else:
- cur_set = self.cfg["dlc_options"][self.dlc_option.get()].copy()
- cur_set["name"] = self.dlc_option.get()
- cur_set["cropping"] = (
- ", ".join([str(c) for c in cur_set["cropping"]])
- if cur_set["cropping"]
- else ""
- )
- cur_set["dynamic"] = ", ".join([str(d) for d in cur_set["dynamic"]])
- cur_set["mode"] = (
- "Optimize Latency" if "mode" not in cur_set else cur_set["mode"]
- )
-
- self.dlc_settings_window = Toplevel(self.window)
- self.dlc_settings_window.title("DLC Settings")
- cur_row = 0
-
- Label(self.dlc_settings_window, text="Name: ").grid(
- sticky="w", row=cur_row, column=0
- )
- self.dlc_settings_name = StringVar(
- self.dlc_settings_window, value=cur_set["name"]
- )
- Entry(self.dlc_settings_window, textvariable=self.dlc_settings_name).grid(
- sticky="nsew", row=cur_row, column=1
- )
- cur_row += 1
-
- Label(self.dlc_settings_window, text="Model Path: ").grid(
- sticky="w", row=cur_row, column=0
- )
- self.dlc_settings_model_path = StringVar(
- self.dlc_settings_window, value=cur_set["model_path"]
- )
- Entry(self.dlc_settings_window, textvariable=self.dlc_settings_model_path).grid(
- sticky="nsew", row=cur_row, column=1
- )
- Button(
- self.dlc_settings_window, text="Browse", command=self.browse_dlc_path
- ).grid(sticky="nsew", row=cur_row, column=2)
- cur_row += 1
-
- Label(self.dlc_settings_window, text="Model Type: ").grid(
- sticky="w", row=cur_row, column=0
- )
- self.dlc_settings_model_type = StringVar(
- self.dlc_settings_window, value=cur_set["model_type"]
- )
- Combobox(
- self.dlc_settings_window,
- textvariable=self.dlc_settings_model_type,
- value=["base", "tensorrt", "tflite"],
- state="readonly",
- ).grid(sticky="nsew", row=cur_row, column=1)
- cur_row += 1
-
- Label(self.dlc_settings_window, text="Precision: ").grid(
- sticky="w", row=cur_row, column=0
- )
- self.dlc_settings_precision = StringVar(
- self.dlc_settings_window, value=cur_set["precision"]
- )
- Combobox(
- self.dlc_settings_window,
- textvariable=self.dlc_settings_precision,
- value=["FP32", "FP16", "INT8"],
- state="readonly",
- ).grid(sticky="nsew", row=cur_row, column=1)
- cur_row += 1
-
- Label(self.dlc_settings_window, text="Cropping: ").grid(
- sticky="w", row=cur_row, column=0
- )
- self.dlc_settings_cropping = StringVar(
- self.dlc_settings_window, value=cur_set["cropping"]
- )
- Entry(self.dlc_settings_window, textvariable=self.dlc_settings_cropping).grid(
- sticky="nsew", row=cur_row, column=1
- )
- cur_row += 1
-
- Label(self.dlc_settings_window, text="Dynamic: ").grid(
- sticky="w", row=cur_row, column=0
- )
- self.dlc_settings_dynamic = StringVar(
- self.dlc_settings_window, value=cur_set["dynamic"]
- )
- Entry(self.dlc_settings_window, textvariable=self.dlc_settings_dynamic).grid(
- sticky="nsew", row=cur_row, column=1
- )
- cur_row += 1
-
- Label(self.dlc_settings_window, text="Resize: ").grid(
- sticky="w", row=cur_row, column=0
- )
- self.dlc_settings_resize = StringVar(
- self.dlc_settings_window, value=cur_set["resize"]
- )
- Entry(self.dlc_settings_window, textvariable=self.dlc_settings_resize).grid(
- sticky="nsew", row=cur_row, column=1
- )
- cur_row += 1
-
- Label(self.dlc_settings_window, text="Mode: ").grid(
- sticky="w", row=cur_row, column=0
- )
- self.dlc_settings_mode = StringVar(
- self.dlc_settings_window, value=cur_set["mode"]
- )
- Combobox(
- self.dlc_settings_window,
- textvariable=self.dlc_settings_mode,
- state="readonly",
- values=["Optimize Latency", "Optimize Rate"],
- ).grid(sticky="nsew", row=cur_row, column=1)
- cur_row += 1
-
- Button(
- self.dlc_settings_window, text="Update", command=self.update_dlc_settings
- ).grid(sticky="nsew", row=cur_row, column=1)
- Button(
- self.dlc_settings_window,
- text="Cancel",
- command=self.dlc_settings_window.destroy,
- ).grid(sticky="nsew", row=cur_row, column=2)
-
- def empty_dlc_settings(self):
-
- return {
- "name": "",
- "model_path": "",
- "model_type": "base",
- "precision": "FP32",
- "cropping": "",
- "dynamic": "False, 0.5, 10",
- "resize": "1.0",
- "mode": "Optimize Latency",
- }
-
- def browse_dlc_path(self):
- """ Open file browser to select DLC exported model directory
- """
-
- new_dlc_path = filedialog.askdirectory(parent=self.dlc_settings_window)
- if new_dlc_path:
- self.dlc_settings_model_path.set(new_dlc_path)
-
- def update_dlc_settings(self):
- """ Update DLC settings for the current dlc option from DLC Settings GUI
- """
-
- precision = (
- self.dlc_settings_precision.get()
- if self.dlc_settings_precision.get()
- else "FP32"
- )
-
- crop_warn = False
- dlc_crop = self.dlc_settings_cropping.get()
- if dlc_crop:
- try:
- dlc_crop = dlc_crop.split(",")
- assert len(dlc_crop) == 4
- dlc_crop = [int(c) for c in dlc_crop]
- except Exception:
- crop_warn = True
- dlc_crop = None
- else:
- dlc_crop = None
-
- try:
- dlc_dynamic = self.dlc_settings_dynamic.get().replace(" ", "")
- dlc_dynamic = dlc_dynamic.split(",")
- dlc_dynamic[0] = True if dlc_dynamic[0] == "True" else False
- dlc_dynamic[1] = float(dlc_dynamic[1])
- dlc_dynamic[2] = int(dlc_dynamic[2])
- dlc_dynamic = tuple(dlc_dynamic)
- dyn_warn = False
- except Exception:
- dyn_warn = True
- dlc_dynamic = (False, 0.5, 10)
-
- dlc_resize = (
- float(self.dlc_settings_resize.get())
- if self.dlc_settings_resize.get()
- else None
- )
- dlc_mode = self.dlc_settings_mode.get()
-
- warn_msg = ""
- if crop_warn:
- warn_msg += "DLC Cropping was not set properly. Using default cropping parameters...\n"
- if dyn_warn:
- warn_msg += "DLC Dynamic Cropping was not set properly. Using default dynamic cropping parameters..."
- if warn_msg:
- messagebox.showerror(
- "DLC Settings Error", warn_msg, parent=self.dlc_settings_window
- )
-
- self.cfg["dlc_options"][self.dlc_settings_name.get()] = {
- "model_path": self.dlc_settings_model_path.get(),
- "model_type": self.dlc_settings_model_type.get(),
- "precision": precision,
- "cropping": dlc_crop,
- "dynamic": dlc_dynamic,
- "resize": dlc_resize,
- "mode": dlc_mode,
- }
-
- if self.dlc_settings_name.get() not in self.cfg["dlc_display_options"]:
- self.cfg["dlc_display_options"][self.dlc_settings_name.get()] = {
- "cmap": "bgy",
- "radius": 3,
- "lik_thresh": 0.5,
- }
-
- self.save_config()
- self.dlc_options_entry["values"] = tuple(self.cfg["dlc_options"].keys()) + (
- "Add DLC",
- )
- self.dlc_option.set(self.dlc_settings_name.get())
- self.dlc_settings_window.destroy()
-
- def remove_dlc_option(self):
- """ Delete DLC Option from config
- """
-
- del self.cfg["dlc_options"][self.dlc_option.get()]
- del self.cfg["dlc_display_options"][self.dlc_option.get()]
- self.dlc_options_entry["values"] = tuple(self.cfg["dlc_options"].keys()) + (
- "Add DLC",
- )
- self.dlc_option.set("")
- self.save_config()
-
- def init_dlc(self):
- """ Initialize DLC Live object
- """
-
- self.stop_pose()
-
- self.dlc_setup_window = Toplevel(self.window)
- self.dlc_setup_window.title("Setting up DLC...")
- Label(self.dlc_setup_window, text="Setting up DLC, please wait...").pack()
- self.dlc_setup_window.after(10, self.start_pose)
- self.dlc_setup_window.mainloop()
-
- def start_pose(self):
-
- dlc_params = self.cfg["dlc_options"][self.dlc_option.get()].copy()
- dlc_params["processor"] = self.dlc_proc_params
- ret = self.cam_pose_proc.start_pose_process(dlc_params)
- self.dlc_setup_window.destroy()
-
- def stop_pose(self):
- """ Stop pose process
- """
-
- if self.cam_pose_proc:
- ret = self.cam_pose_proc.stop_pose_process()
-
- def add_subject(self):
- new_sub = self.subject.get()
- if new_sub:
- if new_sub not in self.cfg["subjects"]:
- self.cfg["subjects"].append(new_sub)
- self.subject_entry["values"] = tuple(self.cfg["subjects"])
- self.save_config()
-
- def remove_subject(self):
-
- self.cfg["subjects"].remove(self.subject.get())
- self.subject_entry["values"] = self.cfg["subjects"]
- self.save_config()
- self.subject.set("")
-
- def browse_directory(self):
-
- new_dir = filedialog.askdirectory(parent=self.window)
- if new_dir:
- self.directory.set(new_dir)
- ask_add_dir = Tk()
- Label(
- ask_add_dir,
- text="Would you like to add this directory to dropdown list?",
- ).pack()
- Button(
- ask_add_dir, text="Yes", command=lambda: self.add_directory(ask_add_dir)
- ).pack()
- Button(ask_add_dir, text="No", command=ask_add_dir.destroy).pack()
-
- def add_directory(self, window):
-
- window.destroy()
- if self.directory.get() not in self.cfg["directories"]:
- self.cfg["directories"].append(self.directory.get())
- self.directory_entry["values"] = self.cfg["directories"]
- self.save_config()
-
- def save_config(self, notify=False):
-
- json.dump(self.cfg, open(self.cfg_file, "w"))
- if notify:
- messagebox.showinfo(
- title="Config file saved",
- message="Configuration file has been saved...",
- parent=self.window,
- )
-
- def remove_cam_cfg(self):
-
- if self.camera_name.get() != "Add Camera":
- delete = messagebox.askyesno(
- title="Delete Camera?",
- message="Are you sure you want to delete '%s'?"
- % self.camera_name.get(),
- parent=self.window,
- )
- if delete:
- del self.cfg["cameras"][self.camera_name.get()]
- self.camera_entry["values"] = self.get_camera_names() + ("Add Camera",)
- self.camera_name.set("")
- self.save_config()
-
- def browse_dlc_processor(self):
-
- new_dir = filedialog.askdirectory(parent=self.window)
- if new_dir:
- self.dlc_proc_dir.set(new_dir)
- self.update_dlc_proc_list()
-
- if new_dir not in self.cfg["processor_dir"]:
- if messagebox.askyesno(
- "Add to dropdown",
- "Would you like to add this directory to dropdown list?",
- ):
- self.cfg["processor_dir"].append(new_dir)
- self.dlc_proc_dir_entry["values"] = tuple(self.cfg["processor_dir"])
- self.save_config()
-
- def rem_dlc_proc_dir(self):
-
- if self.dlc_proc_dir.get() in self.cfg["processor_dir"]:
- self.cfg["processor_dir"].remove(self.dlc_proc_dir.get())
- self.save_config()
- self.dlc_proc_dir_entry["values"] = tuple(self.cfg["processor_dir"])
- self.dlc_proc_dir.set("")
-
- def update_dlc_proc_list(self, event=None):
-
- ### if dlc proc module already initialized, delete module and remove from path ###
-
- self.processor_list = []
-
- if self.dlc_proc_dir.get():
-
- if hasattr(self, "dlc_proc_module"):
- sys.path.remove(sys.path[0])
-
- new_path = os.path.normpath(os.path.dirname(self.dlc_proc_dir.get()))
- if new_path not in sys.path:
- sys.path.insert(0, new_path)
-
- new_mod = os.path.basename(self.dlc_proc_dir.get())
- if new_mod in sys.modules:
- del sys.modules[new_mod]
-
- ### load new module ###
-
- processor_spec = importlib.util.find_spec(
- os.path.basename(self.dlc_proc_dir.get())
- )
- try:
- self.dlc_proc_module = importlib.util.module_from_spec(processor_spec)
- processor_spec.loader.exec_module(self.dlc_proc_module)
- # self.processor_list = inspect.getmembers(self.dlc_proc_module, inspect.isclass)
- self.processor_list = [
- proc for proc in dir(self.dlc_proc_module) if "__" not in proc
- ]
- except AttributeError:
- if hasattr(self, "window"):
- messagebox.showerror(
- "Failed to load processors!",
- "Failed to load processors from directory = "
- + self.dlc_proc_dir.get()
- + ".\nPlease select a different directory.",
- parent=self.window,
- )
-
- self.dlc_proc_name_entry["values"] = tuple(self.processor_list)
-
- def set_proc(self):
-
- # proc_param_dict = {}
- # for i in range(1, len(self.proc_param_names)):
- # proc_param_dict[self.proc_param_names[i]] = self.proc_param_default_types[i](self.proc_param_values[i-1].get())
-
- # if self.dlc_proc_dir.get() not in self.cfg['processor_args']:
- # self.cfg['processor_args'][self.dlc_proc_dir.get()] = {}
- # self.cfg['processor_args'][self.dlc_proc_dir.get()][self.dlc_proc_name.get()] = proc_param_dict
- # self.save_config()
-
- # self.dlc_proc = self.proc_object(**proc_param_dict)
- proc_object = getattr(self.dlc_proc_module, self.dlc_proc_name.get())
- self.dlc_proc_params = {"object": proc_object}
- self.dlc_proc_params.update(
- self.cfg["processor_args"][self.dlc_proc_dir.get()][
- self.dlc_proc_name.get()
- ]
- )
-
- def clear_proc(self):
-
- self.dlc_proc_params = None
-
- def edit_proc(self):
-
- ### get default args: load module and read arguments ###
-
- self.proc_object = getattr(self.dlc_proc_module, self.dlc_proc_name.get())
- def_args = inspect.getargspec(self.proc_object)
- self.proc_param_names = def_args[0]
- self.proc_param_default_values = def_args[3]
- self.proc_param_default_types = [
- type(v) if type(v) is not list else [type(v[0])] for v in def_args[3]
- ]
- for i in range(len(def_args[0]) - len(def_args[3])):
- self.proc_param_default_values = ("",) + self.proc_param_default_values
- self.proc_param_default_types = [str] + self.proc_param_default_types
-
- ### check for existing settings in config ###
-
- old_args = {}
- if self.dlc_proc_dir.get() in self.cfg["processor_args"]:
- if (
- self.dlc_proc_name.get()
- in self.cfg["processor_args"][self.dlc_proc_dir.get()]
- ):
- old_args = self.cfg["processor_args"][self.dlc_proc_dir.get()][
- self.dlc_proc_name.get()
- ].copy()
- else:
- self.cfg["processor_args"][self.dlc_proc_dir.get()] = {}
-
- ### get dictionary of arguments ###
-
- proc_args_dict = {}
- for i in range(1, len(self.proc_param_names)):
-
- if self.proc_param_names[i] in old_args:
- this_value = old_args[self.proc_param_names[i]]
- else:
- this_value = self.proc_param_default_values[i]
-
- proc_args_dict[self.proc_param_names[i]] = {
- "value": this_value,
- "dtype": self.proc_param_default_types[i],
- }
-
- proc_args_gui = SettingsWindow(
- title="DLC Processor Settings", settings=proc_args_dict, parent=self.window
- )
- proc_args_gui.mainloop()
-
- self.cfg["processor_args"][self.dlc_proc_dir.get()][
- self.dlc_proc_name.get()
- ] = proc_args_gui.get_values()
- self.save_config()
-
- def init_session(self):
-
- ### check if video is currently open ###
-
- if self.record_on.get() > -1:
- messagebox.showerror(
- "Session Open",
- "Session is currently open! Please release the current video (click 'Save Video' of 'Delete Video', even if no frames have been recorded) before setting up a new one.",
- parent=self.window,
- )
- return
-
- ### check if camera is already set up ###
-
- if not self.cam_pose_proc:
- messagebox.showerror(
- "No Camera",
- "No camera is found! Please initialize a camera before setting up the video.",
- parent=self.window,
- )
- return
-
- ### set up session window
-
- self.session_setup_window = Toplevel(self.window)
- self.session_setup_window.title("Setting up session...")
- Label(
- self.session_setup_window, text="Setting up session, please wait..."
- ).pack()
- self.session_setup_window.after(10, self.start_writer)
- self.session_setup_window.mainloop()
-
- def start_writer(self):
-
- ### set up file name (get date and create directory)
-
- dt = datetime.datetime.now()
- date = f"{dt.year:04d}-{dt.month:02d}-{dt.day:02d}"
- self.out_dir = self.directory.get()
- if not os.path.isdir(os.path.normpath(self.out_dir)):
- os.makedirs(os.path.normpath(self.out_dir))
-
- ### create output file names
-
- self.base_name = os.path.normpath(
- f"{self.out_dir}/{self.camera_name.get().replace(' ', '')}_{self.subject.get()}_{date}_{self.attempt.get()}"
- )
- # self.vid_file = os.path.normpath(self.out_dir + '/VIDEO_' + self.base_name + '.avi')
- # self.ts_file = os.path.normpath(self.out_dir + '/TIMESTAMPS_' + self.base_name + '.pickle')
- # self.dlc_file = os.path.normpath(self.out_dir + '/DLC_' + self.base_name + '.h5')
- # self.proc_file = os.path.normpath(self.out_dir + '/PROC_' + self.base_name + '.pickle')
-
- ### check if files already exist
-
- fs = glob.glob(f"{self.base_name}*")
- if len(fs) > 0:
- overwrite = messagebox.askyesno(
- "Files Exist",
- "Files already exist with attempt number = {}. Would you like to overwrite the file?".format(
- self.attempt.get()
- ),
- parent=self.session_setup_window,
- )
- if not overwrite:
- return
-
- ### start writer
-
- ret = self.cam_pose_proc.start_writer_process(self.base_name)
-
- self.session_setup_window.destroy()
-
- ### set GUI to Ready
-
- self.record_on.set(0)
-
- def start_record(self):
- """ Issues command to start recording frames and poses
- """
-
- ret = False
- if self.cam_pose_proc is not None:
- ret = self.cam_pose_proc.start_record()
-
- if not ret:
- messagebox.showerror(
- "Recording Not Ready",
- "Recording has not been set up. Please make sure a camera and session have been initialized.",
- parent=self.window,
- )
- self.record_on.set(-1)
-
- def stop_record(self):
- """ Issues command to stop recording frames and poses
- """
-
- if self.cam_pose_proc is not None:
- ret = self.cam_pose_proc.stop_record()
- self.record_on.set(0)
-
- def save_vid(self, delete=False):
- """ Saves video, timestamp, and DLC files
-
- Parameters
- ----------
- delete : bool, optional
- flag to delete created files, by default False
- """
-
- ### perform checks ###
-
- if self.cam_pose_proc is None:
- messagebox.showwarning(
- "No Camera",
- "Camera has not yet been initialized, no video recorded.",
- parent=self.window,
- )
- return
-
- elif self.record_on.get() == -1:
- messagebox.showwarning(
- "No Video File",
- "Video was not set up, no video recorded.",
- parent=self.window,
- )
- return
-
- elif self.record_on.get() == 1:
- messagebox.showwarning(
- "Active Recording",
- "You are currently recording a video. Please stop the video before saving.",
- parent=self.window,
- )
- return
-
- elif delete:
- delete = messagebox.askokcancel(
- "Delete Video?", "Do you wish to delete the video?", parent=self.window
- )
-
- ### save or delete video ###
-
- if delete:
- ret = self.cam_pose_proc.stop_writer_process(save=False)
- messagebox.showinfo(
- "Video Deleted",
- "Video and timestamp files have been deleted.",
- parent=self.window,
- )
- else:
- ret = self.cam_pose_proc.stop_writer_process(save=True)
- ret_pose = self.cam_pose_proc.save_pose(self.base_name)
- if ret:
- if ret_pose:
- messagebox.showinfo(
- "Files Saved",
- "Video, timestamp, and DLC Files have been saved.",
- )
- else:
- messagebox.showinfo(
- "Files Saved", "Video and timestamp files have been saved."
- )
- else:
- messagebox.showwarning(
- "No Frames Recorded",
- "No frames were recorded, video was deleted",
- parent=self.window,
- )
-
- self.record_on.set(-1)
-
- def closeGUI(self):
-
- if self.cam_pose_proc:
- ret = self.cam_pose_proc.stop_writer_process()
- ret = self.cam_pose_proc.stop_pose_process()
- ret = self.cam_pose_proc.stop_capture_process()
-
- self.window.destroy()
-
- def createGUI(self):
-
- ### initialize window ###
-
- self.window = Tk()
- self.window.title("DeepLabCut Live")
- cur_row = 0
- combobox_width = 15
-
- ### select cfg file
- if len(self.cfg_list) > 0:
- initial_cfg = self.cfg_list[0]
- else:
- initial_cfg = ""
-
- Label(self.window, text="Config: ").grid(sticky="w", row=cur_row, column=0)
- self.cfg_name = StringVar(self.window, value=initial_cfg)
- self.cfg_entry = Combobox(
- self.window, textvariable=self.cfg_name, width=combobox_width
- )
- self.cfg_entry["values"] = tuple(self.cfg_list) + ("Create New Config",)
- self.cfg_entry.bind("<>", self.change_config)
- self.cfg_entry.grid(sticky="nsew", row=cur_row, column=1)
- Button(self.window, text="Remove Config", command=self.remove_config).grid(
- sticky="nsew", row=cur_row, column=2
- )
-
- self.get_config(initial_cfg)
-
- cur_row += 2
-
- ### select camera ###
-
- # camera entry
- Label(self.window, text="Camera: ").grid(sticky="w", row=cur_row, column=0)
- self.camera_name = StringVar(self.window)
- self.camera_entry = Combobox(self.window, textvariable=self.camera_name)
- cam_names = self.get_camera_names()
- self.camera_entry["values"] = cam_names + ("Add Camera",)
- if cam_names:
- self.camera_entry.current(0)
- self.camera_entry.grid(sticky="nsew", row=cur_row, column=1)
- Button(self.window, text="Init Cam", command=self.init_cam).grid(
- sticky="nsew", row=cur_row, column=2
- )
- cur_row += 1
-
- Button(
- self.window, text="Edit Camera Settings", command=self.edit_cam_settings
- ).grid(sticky="nsew", row=cur_row, column=1)
- Button(self.window, text="Close Camera", command=self.close_camera).grid(
- sticky="nsew", row=cur_row, column=2
- )
- cur_row += 1
- Button(self.window, text="Remove Camera", command=self.remove_cam_cfg).grid(
- sticky="nsew", row=cur_row, column=2
- )
-
- cur_row += 2
-
- ### set up proc ###
-
- Label(self.window, text="Processor Dir: ").grid(
- sticky="w", row=cur_row, column=0
- )
- self.dlc_proc_dir = StringVar(self.window)
- self.dlc_proc_dir_entry = Combobox(self.window, textvariable=self.dlc_proc_dir)
- self.dlc_proc_dir_entry["values"] = tuple(self.cfg["processor_dir"])
- if len(self.cfg["processor_dir"]) > 0:
- self.dlc_proc_dir_entry.current(0)
- self.dlc_proc_dir_entry.bind("<>", self.update_dlc_proc_list)
- self.dlc_proc_dir_entry.grid(sticky="nsew", row=cur_row, column=1)
- Button(self.window, text="Browse", command=self.browse_dlc_processor).grid(
- sticky="nsew", row=cur_row, column=2
- )
- Button(self.window, text="Remove Proc Dir", command=self.rem_dlc_proc_dir).grid(
- sticky="nsew", row=cur_row + 1, column=2
- )
- cur_row += 2
-
- Label(self.window, text="Processor: ").grid(sticky="w", row=cur_row, column=0)
- self.dlc_proc_name = StringVar(self.window)
- self.dlc_proc_name_entry = Combobox(
- self.window, textvariable=self.dlc_proc_name
- )
- self.update_dlc_proc_list()
- # self.dlc_proc_name_entry['values'] = tuple(self.processor_list) # tuple([c[0] for c in inspect.getmembers(processor, inspect.isclass)])
- if len(self.processor_list) > 0:
- self.dlc_proc_name_entry.current(0)
- self.dlc_proc_name_entry.grid(sticky="nsew", row=cur_row, column=1)
- Button(self.window, text="Set Proc", command=self.set_proc).grid(
- sticky="nsew", row=cur_row, column=2
- )
- Button(self.window, text="Edit Proc Settings", command=self.edit_proc).grid(
- sticky="nsew", row=cur_row + 1, column=1
- )
- Button(self.window, text="Clear Proc", command=self.clear_proc).grid(
- sticky="nsew", row=cur_row + 1, column=2
- )
-
- cur_row += 3
-
- ### set up dlc live ###
-
- Label(self.window, text="DeepLabCut: ").grid(sticky="w", row=cur_row, column=0)
- self.dlc_option = StringVar(self.window)
- self.dlc_options_entry = Combobox(self.window, textvariable=self.dlc_option)
- self.dlc_options_entry["values"] = tuple(self.cfg["dlc_options"].keys()) + (
- "Add DLC",
- )
- self.dlc_options_entry.bind("<>", self.change_dlc_option)
- self.dlc_options_entry.grid(sticky="nsew", row=cur_row, column=1)
- Button(self.window, text="Init DLC", command=self.init_dlc).grid(
- sticky="nsew", row=cur_row, column=2
- )
- cur_row += 1
-
- Button(
- self.window, text="Edit DLC Settings", command=self.edit_dlc_settings
- ).grid(sticky="nsew", row=cur_row, column=1)
- Button(self.window, text="Stop DLC", command=self.stop_pose).grid(
- sticky="nsew", row=cur_row, column=2
- )
- cur_row += 1
-
- self.display_keypoints = BooleanVar(self.window, value=False)
- Checkbutton(
- self.window,
- text="Display DLC Keypoints",
- variable=self.display_keypoints,
- indicatoron=0,
- command=self.change_display_keypoints,
- ).grid(sticky="nsew", row=cur_row, column=1)
- Button(self.window, text="Remove DLC", command=self.remove_dlc_option).grid(
- sticky="nsew", row=cur_row, column=2
- )
- cur_row += 1
-
- Button(
- self.window, text="Edit DLC Display Settings", command=self.edit_dlc_display
- ).grid(sticky="nsew", row=cur_row, column=1)
-
- cur_row += 2
-
- ### set up session ###
-
- # subject
- Label(self.window, text="Subject: ").grid(sticky="w", row=cur_row, column=0)
- self.subject = StringVar(self.window)
- self.subject_entry = Combobox(self.window, textvariable=self.subject)
- self.subject_entry["values"] = self.cfg["subjects"]
- self.subject_entry.grid(sticky="nsew", row=cur_row, column=1)
- Button(self.window, text="Add Subject", command=self.add_subject).grid(
- sticky="nsew", row=cur_row, column=2
- )
- cur_row += 1
-
- # attempt
- Label(self.window, text="Attempt: ").grid(sticky="w", row=cur_row, column=0)
- self.attempt = StringVar(self.window)
- self.attempt_entry = Combobox(self.window, textvariable=self.attempt)
- self.attempt_entry["values"] = tuple(range(1, 10))
- self.attempt_entry.current(0)
- self.attempt_entry.grid(sticky="nsew", row=cur_row, column=1)
- Button(self.window, text="Remove Subject", command=self.remove_subject).grid(
- sticky="nsew", row=cur_row, column=2
- )
- cur_row += 1
-
- # out directory
- Label(self.window, text="Directory: ").grid(sticky="w", row=cur_row, column=0)
- self.directory = StringVar(self.window)
- self.directory_entry = Combobox(self.window, textvariable=self.directory)
- if self.cfg["directories"]:
- self.directory_entry["values"] = self.cfg["directories"]
- self.directory_entry.current(0)
- self.directory_entry.grid(sticky="nsew", row=cur_row, column=1)
- Button(self.window, text="Browse", command=self.browse_directory).grid(
- sticky="nsew", row=cur_row, column=2
- )
- cur_row += 1
-
- # set up session
- Button(self.window, text="Set Up Session", command=self.init_session).grid(
- sticky="nsew", row=cur_row, column=1
- )
- cur_row += 2
-
- ### control recording ###
-
- Label(self.window, text="Record: ").grid(sticky="w", row=cur_row, column=0)
- self.record_on = IntVar(value=-1)
- Radiobutton(
- self.window,
- text="Ready",
- selectcolor="blue",
- indicatoron=0,
- variable=self.record_on,
- value=0,
- state="disabled",
- ).grid(stick="nsew", row=cur_row, column=1)
- Radiobutton(
- self.window,
- text="On",
- selectcolor="green",
- indicatoron=0,
- variable=self.record_on,
- value=1,
- command=self.start_record,
- ).grid(sticky="nsew", row=cur_row + 1, column=1)
- Radiobutton(
- self.window,
- text="Off",
- selectcolor="red",
- indicatoron=0,
- variable=self.record_on,
- value=-1,
- command=self.stop_record,
- ).grid(sticky="nsew", row=cur_row + 2, column=1)
- Button(self.window, text="Save Video", command=lambda: self.save_vid()).grid(
- sticky="nsew", row=cur_row + 1, column=2
- )
- Button(
- self.window, text="Delete Video", command=lambda: self.save_vid(delete=True)
- ).grid(sticky="nsew", row=cur_row + 2, column=2)
-
- cur_row += 4
-
- ### close program ###
-
- Button(self.window, text="Close", command=self.closeGUI).grid(
- sticky="nsew", row=cur_row, column=0, columnspan=2
- )
-
- ### configure size of empty rows
-
- _, row_count = self.window.grid_size()
- for r in range(row_count):
- self.window.grid_rowconfigure(r, minsize=20)
-
- def run(self):
-
- self.window.mainloop()
-
-
-def main():
-
- # import multiprocess as mp
- # mp.set_start_method("spawn")
-
- dlc_live_gui = DLCLiveGUI()
- dlc_live_gui.run()
diff --git a/dlclivegui/gui/camera_config_dialog.py b/dlclivegui/gui/camera_config_dialog.py
new file mode 100644
index 0000000..d693d03
--- /dev/null
+++ b/dlclivegui/gui/camera_config_dialog.py
@@ -0,0 +1,1211 @@
+"""Camera configuration dialog for multi-camera setup (with async preview loading)."""
+
+from __future__ import annotations
+
+import copy
+import logging
+
+import cv2
+from PySide6.QtCore import QEvent, Qt, QThread, QTimer, Signal
+from PySide6.QtGui import QFont, QImage, QKeyEvent, QPixmap, QTextCursor
+from PySide6.QtWidgets import (
+ QCheckBox,
+ QComboBox,
+ QDialog,
+ QDoubleSpinBox,
+ QFormLayout,
+ QGroupBox,
+ QHBoxLayout,
+ QLabel,
+ QListWidget,
+ QListWidgetItem,
+ QMessageBox,
+ QProgressBar,
+ QPushButton,
+ QSpinBox,
+ QStyle,
+ QTextEdit,
+ QVBoxLayout,
+ QWidget,
+)
+
+from dlclivegui.cameras import CameraFactory
+from dlclivegui.cameras.base import CameraBackend
+from dlclivegui.cameras.factory import DetectedCamera
+from dlclivegui.utils.config_models import CameraSettingsModel, MultiCameraSettingsModel
+
+LOGGER = logging.getLogger(__name__)
+
+
+# -------------------------------
+# Background worker to detect cameras
+# -------------------------------
+class DetectCamerasWorker(QThread):
+ """Background worker to detect cameras for the selected backend."""
+
+ progress = Signal(str) # human-readable text
+ result = Signal(list) # list[DetectedCamera]
+ error = Signal(str)
+ finished = Signal()
+
+ def __init__(self, backend: str, max_devices: int = 10, parent: QWidget | None = None):
+ super().__init__(parent)
+ self.backend = backend
+ self.max_devices = max_devices
+
+ def run(self):
+ try:
+ # Initial message
+ self.progress.emit(f"Scanning {self.backend} cameras…")
+
+ cams = CameraFactory.detect_cameras(
+ self.backend,
+ max_devices=self.max_devices,
+ should_cancel=self.isInterruptionRequested,
+ progress_cb=self.progress.emit,
+ )
+ self.result.emit(cams)
+ except Exception as exc:
+ self.error.emit(f"{type(exc).__name__}: {exc}")
+ finally:
+ self.finished.emit()
+
+
+# -------------------------------
+# Singleton camera preview loader worker
+# -------------------------------
+class CameraLoadWorker(QThread):
+ """Open/configure a camera backend off the UI thread with progress and cancel support."""
+
+ progress = Signal(str) # Human-readable status updates
+ success = Signal(object) # Emits the ready backend (CameraBackend)
+ error = Signal(str) # Emits error message
+ canceled = Signal() # Emits when canceled before success
+
+ def __init__(self, cam: CameraSettingsModel, parent: QWidget | None = None):
+ super().__init__(parent)
+ # Work on a defensive copy so we never mutate the original settings
+ self._cam = copy.deepcopy(cam)
+ # Make first-time opening snappier by allowing backend fast-path if supported
+ if isinstance(self._cam.properties, dict):
+ self._cam.properties.setdefault("fast_start", True)
+ self._cancel = False
+ self._backend: CameraBackend | None = None
+
+ def request_cancel(self):
+ self._cancel = True
+
+ def _check_cancel(self) -> bool:
+ if self._cancel:
+ self.progress.emit("Canceled by user.")
+ return True
+ return False
+
+ def run(self):
+ try:
+ self.progress.emit("Creating backend…")
+ if self._check_cancel():
+ self.canceled.emit()
+ return
+
+ LOGGER.debug("Creating camera backend for %s:%d", self._cam.backend, self._cam.index)
+ self.progress.emit("Opening device…")
+ # Open only in GUI thread to avoid simultaneous opens
+ self.success.emit(self._cam)
+
+ except Exception as exc:
+ msg = f"{type(exc).__name__}: {exc}"
+ try:
+ if self._backend:
+ self._backend.close()
+ except Exception:
+ pass
+ self.error.emit(msg)
+
+
+class CameraConfigDialog(QDialog):
+ """Dialog for configuring multiple cameras with async preview loading."""
+
+ MAX_CAMERAS = 4
+ settings_changed = Signal(object) # MultiCameraSettingsModel
+ # Camera discovery signals
+ scan_started = Signal(str)
+ scan_finished = Signal()
+
+ def __init__(
+ self,
+ parent: QWidget | None = None,
+ multi_camera_settings: MultiCameraSettingsModel | None = None,
+ ):
+ super().__init__(parent)
+ self.setWindowTitle("Configure Cameras")
+ self.setMinimumSize(960, 720)
+
+ self._dlc_camera_id = None
+ self.dlc_camera_id: str | None = None
+ # Actual/working camera settings
+ self._multi_camera_settings = multi_camera_settings
+ self._working_settings = self._multi_camera_settings.model_copy(deep=True)
+ self._detected_cameras: list[DetectedCamera] = []
+ self._current_edit_index: int | None = None
+
+ # Preview state
+ self._preview_backend: CameraBackend | None = None
+ self._preview_timer: QTimer | None = None
+ self._preview_active: bool = False
+
+ # Camera detection worker
+ self._scan_worker: DetectCamerasWorker | None = None
+
+ # Singleton loader per dialog
+ self._loader: CameraLoadWorker | None = None
+ self._loading_active: bool = False
+
+ self._setup_ui()
+ self._populate_from_settings()
+ self._connect_signals()
+
+ @property
+ def dlc_camera_id(self) -> str | None:
+ """Get the currently selected DLC camera ID."""
+ return self._dlc_camera_id
+
+ @dlc_camera_id.setter
+ def dlc_camera_id(self, value: str | None) -> None:
+ """Set the currently selected DLC camera ID."""
+ self._dlc_camera_id = value
+ self._refresh_camera_labels()
+
+ # -------------------------------
+ # Config helpers
+ # ------------------------------
+
+ def _build_model_from_form(self, base: CameraSettingsModel) -> CameraSettingsModel:
+ # construct a dict from form widgets; Pydantic will coerce/validate
+ payload = base.model_dump()
+ payload.update(
+ {
+ "enabled": bool(self.cam_enabled_checkbox.isChecked()),
+ "fps": float(self.cam_fps.value()),
+ "exposure": int(self.cam_exposure.value()),
+ "gain": float(self.cam_gain.value()),
+ "rotation": int(self.cam_rotation.currentData() or 0),
+ "crop_x0": int(self.cam_crop_x0.value()),
+ "crop_y0": int(self.cam_crop_y0.value()),
+ "crop_x1": int(self.cam_crop_x1.value()),
+ "crop_y1": int(self.cam_crop_y1.value()),
+ }
+ )
+ # Validate and coerce; if invalid, Pydantic will raise
+ return CameraSettingsModel.model_validate(payload)
+
+ # -------------------------------
+ # UI setup
+ # -------------------------------
+ def _setup_ui(self) -> None:
+ # Main layout for the dialog
+ main_layout = QVBoxLayout(self)
+
+ # Horizontal layout for left and right panels
+ panels_layout = QHBoxLayout()
+
+ # Left panel: Camera list and controls
+ left_panel = QWidget()
+ left_layout = QVBoxLayout(left_panel)
+
+ # Active cameras list
+ active_group = QGroupBox("Active Cameras")
+ active_layout = QVBoxLayout(active_group)
+
+ self.active_cameras_list = QListWidget()
+ self.active_cameras_list.setMinimumWidth(250)
+ active_layout.addWidget(self.active_cameras_list)
+
+ # Buttons for managing active cameras
+ list_buttons = QHBoxLayout()
+ self.remove_camera_btn = QPushButton("Remove")
+ self.remove_camera_btn.setIcon(self.style().standardIcon(QStyle.StandardPixmap.SP_TrashIcon))
+ self.remove_camera_btn.setEnabled(False)
+ self.move_up_btn = QPushButton("↑")
+ self.move_up_btn.setIcon(self.style().standardIcon(QStyle.StandardPixmap.SP_ArrowUp))
+ self.move_up_btn.setEnabled(False)
+ self.move_down_btn = QPushButton("↓")
+ self.move_down_btn.setIcon(self.style().standardIcon(QStyle.StandardPixmap.SP_ArrowDown))
+ self.move_down_btn.setEnabled(False)
+ list_buttons.addWidget(self.remove_camera_btn)
+ list_buttons.addWidget(self.move_up_btn)
+ list_buttons.addWidget(self.move_down_btn)
+ active_layout.addLayout(list_buttons)
+
+ left_layout.addWidget(active_group)
+
+ # Available cameras section
+ available_group = QGroupBox("Available Cameras")
+ available_layout = QVBoxLayout(available_group)
+
+ # Backend selection
+ backend_layout = QHBoxLayout()
+ backend_layout.addWidget(QLabel("Backend:"))
+ self.backend_combo = QComboBox()
+ availability = CameraFactory.available_backends()
+ for backend in CameraFactory.backend_names():
+ label = backend
+ if not availability.get(backend, True):
+ label = f"{backend} (unavailable)"
+ self.backend_combo.addItem(label, backend)
+ if self.backend_combo.count() == 0:
+ raise RuntimeError("No camera backends are registered!")
+ # Switch to first available backend
+ for i in range(self.backend_combo.count()):
+ backend = self.backend_combo.itemData(i)
+ if availability.get(backend, False):
+ self.backend_combo.setCurrentIndex(i)
+ break
+ backend_layout.addWidget(self.backend_combo)
+ self.refresh_btn = QPushButton("Refresh")
+ self.refresh_btn.setIcon(self.style().standardIcon(QStyle.StandardPixmap.SP_BrowserReload))
+ backend_layout.addWidget(self.refresh_btn)
+ available_layout.addLayout(backend_layout)
+
+ self.available_cameras_list = QListWidget()
+ available_layout.addWidget(self.available_cameras_list)
+
+ # Show status overlay during scan
+ self._scan_overlay = QLabel(available_group)
+ self._scan_overlay.setVisible(False)
+ self._scan_overlay.setAlignment(Qt.AlignCenter)
+ self._scan_overlay.setWordWrap(True)
+ self._scan_overlay.setStyleSheet(
+ "background-color: rgba(0, 0, 0, 140);color: white;padding: 12px;border: 1px solid #333;font-size: 12px;"
+ )
+ self._scan_overlay.setText("Discovering cameras…")
+ self.available_cameras_list.installEventFilter(self)
+
+ # Indeterminate progress bar + status text for async scan
+ self.scan_progress = QProgressBar()
+ self.scan_progress.setRange(0, 0)
+ self.scan_progress.setVisible(False)
+
+ available_layout.addWidget(self.scan_progress)
+
+ self.scan_cancel_btn = QPushButton("Cancel Scan")
+ self.scan_cancel_btn.setIcon(self.style().standardIcon(QStyle.StandardPixmap.SP_BrowserStop))
+ self.scan_cancel_btn.setVisible(False)
+ self.scan_cancel_btn.clicked.connect(self._on_scan_cancel)
+ available_layout.addWidget(self.scan_cancel_btn)
+
+ self.add_camera_btn = QPushButton("Add Selected Camera →")
+ self.add_camera_btn.setIcon(self.style().standardIcon(QStyle.StandardPixmap.SP_ArrowRight))
+ self.add_camera_btn.setEnabled(False)
+ available_layout.addWidget(self.add_camera_btn)
+
+ left_layout.addWidget(available_group)
+
+ # Right panel: Camera settings editor
+ right_panel = QWidget()
+ right_layout = QVBoxLayout(right_panel)
+
+ settings_group = QGroupBox("Camera Settings")
+ self.settings_form = QFormLayout(settings_group)
+
+ self.cam_enabled_checkbox = QCheckBox("Enabled")
+ self.cam_enabled_checkbox.setChecked(True)
+ self.settings_form.addRow(self.cam_enabled_checkbox)
+
+ self.cam_name_label = QLabel("Camera 0")
+ self.cam_name_label.setStyleSheet("font-weight: bold; font-size: 14px;")
+ self.settings_form.addRow("Name:", self.cam_name_label)
+
+ self.cam_index_label = QLabel("0")
+ self.settings_form.addRow("Index:", self.cam_index_label)
+
+ self.cam_backend_label = QLabel("opencv")
+ self.settings_form.addRow("Backend:", self.cam_backend_label)
+
+ self.cam_fps = QDoubleSpinBox()
+ self.cam_fps.setRange(1.0, 240.0)
+ self.cam_fps.setDecimals(2)
+ self.cam_fps.setValue(30.0)
+ self.settings_form.addRow("Frame Rate:", self.cam_fps)
+
+ self.cam_exposure = QSpinBox()
+ self.cam_exposure.setRange(0, 1000000)
+ self.cam_exposure.setValue(0)
+ self.cam_exposure.setSpecialValueText("Auto")
+ self.cam_exposure.setSuffix(" μs")
+ self.settings_form.addRow("Exposure:", self.cam_exposure)
+
+ self.cam_gain = QDoubleSpinBox()
+ self.cam_gain.setRange(0.0, 100.0)
+ self.cam_gain.setValue(0.0)
+ self.cam_gain.setSpecialValueText("Auto")
+ self.cam_gain.setDecimals(2)
+ self.settings_form.addRow("Gain:", self.cam_gain)
+
+ # Rotation
+ self.cam_rotation = QComboBox()
+ self.cam_rotation.addItem("0° (default)", 0)
+ self.cam_rotation.addItem("90°", 90)
+ self.cam_rotation.addItem("180°", 180)
+ self.cam_rotation.addItem("270°", 270)
+ self.settings_form.addRow("Rotation:", self.cam_rotation)
+
+ # Crop settings
+ crop_widget = QWidget()
+ crop_layout = QHBoxLayout(crop_widget)
+ crop_layout.setContentsMargins(0, 0, 0, 0)
+
+ self.cam_crop_x0 = QSpinBox()
+ self.cam_crop_x0.setRange(0, 7680)
+ self.cam_crop_x0.setPrefix("x0:")
+ self.cam_crop_x0.setSpecialValueText("x0:None")
+ crop_layout.addWidget(self.cam_crop_x0)
+
+ self.cam_crop_y0 = QSpinBox()
+ self.cam_crop_y0.setRange(0, 4320)
+ self.cam_crop_y0.setPrefix("y0:")
+ self.cam_crop_y0.setSpecialValueText("y0:None")
+ crop_layout.addWidget(self.cam_crop_y0)
+
+ self.cam_crop_x1 = QSpinBox()
+ self.cam_crop_x1.setRange(0, 7680)
+ self.cam_crop_x1.setPrefix("x1:")
+ self.cam_crop_x1.setSpecialValueText("x1:None")
+ crop_layout.addWidget(self.cam_crop_x1)
+
+ self.cam_crop_y1 = QSpinBox()
+ self.cam_crop_y1.setRange(0, 4320)
+ self.cam_crop_y1.setPrefix("y1:")
+ self.cam_crop_y1.setSpecialValueText("y1:None")
+ crop_layout.addWidget(self.cam_crop_y1)
+
+ self.settings_form.addRow("Crop (x0,y0,x1,y1):", crop_widget)
+
+ self.apply_settings_btn = QPushButton("Apply Settings")
+ self.apply_settings_btn.setIcon(self.style().standardIcon(QStyle.StandardPixmap.SP_DialogApplyButton))
+ self.apply_settings_btn.setEnabled(False)
+ self.settings_form.addRow(self.apply_settings_btn)
+
+ # Preview button
+ self.preview_btn = QPushButton("Start Preview")
+ self.preview_btn.setIcon(self.style().standardIcon(QStyle.StandardPixmap.SP_MediaPlay))
+ self.preview_btn.setEnabled(False)
+ self.settings_form.addRow(self.preview_btn)
+
+ right_layout.addWidget(settings_group)
+
+ # Preview widget
+ self.preview_group = QGroupBox("Camera Preview")
+ preview_layout = QVBoxLayout(self.preview_group)
+
+ self.preview_label = QLabel("No preview")
+ self.preview_label.setAlignment(Qt.AlignmentFlag.AlignCenter)
+ self.preview_label.setMinimumSize(320, 240)
+ self.preview_label.setMaximumSize(400, 300)
+ self.preview_label.setStyleSheet("background-color: #1a1a1a; color: #888;")
+ preview_layout.addWidget(self.preview_label)
+ self.preview_label.installEventFilter(self) # For resize events
+
+ # Small, read-only status console for loader messages
+ self.preview_status = QTextEdit()
+ self.preview_status.setReadOnly(True)
+ self.preview_status.setFixedHeight(45)
+ self.preview_status.setStyleSheet(
+ "QTextEdit { background: #141414; color: #bdbdbd; border: 1px solid #2a2a2a; }"
+ )
+ font = QFont("Consolas")
+ font.setPointSize(9)
+ self.preview_status.setFont(font)
+ preview_layout.addWidget(self.preview_status)
+
+ # Overlay label for loading glass pane
+ self._loading_overlay = QLabel(self.preview_group)
+ self._loading_overlay.setVisible(False)
+ self._loading_overlay.setAlignment(Qt.AlignCenter)
+ self._loading_overlay.setStyleSheet("background-color: rgba(0,0,0,140); color: white; border: 1px solid #333;")
+ self._loading_overlay.setText("Loading camera…")
+
+ self.preview_group.setVisible(False)
+ right_layout.addWidget(self.preview_group)
+
+ right_layout.addStretch(1)
+
+ # Dialog buttons
+ button_layout = QHBoxLayout()
+ self.ok_btn = QPushButton("OK")
+ self.ok_btn.setAutoDefault(False)
+ self.ok_btn.setDefault(False)
+ self.ok_btn.setIcon(self.style().standardIcon(QStyle.StandardPixmap.SP_DialogOkButton))
+ self.cancel_btn = QPushButton("Cancel")
+ self.cancel_btn.setAutoDefault(False)
+ self.cancel_btn.setDefault(False)
+ self.cancel_btn.setIcon(self.style().standardIcon(QStyle.StandardPixmap.SP_DialogCancelButton))
+ button_layout.addStretch(1)
+ button_layout.addWidget(self.ok_btn)
+ button_layout.addWidget(self.cancel_btn)
+
+ # Add panels to horizontal layout
+ panels_layout.addWidget(left_panel, stretch=1)
+ panels_layout.addWidget(right_panel, stretch=1)
+
+ # Add everything to main layout
+ main_layout.addLayout(panels_layout)
+ main_layout.addLayout(button_layout)
+
+ # Pressing enter on any settings field applies settings
+ self.cam_fps.setKeyboardTracking(False)
+ fields = [
+ self.cam_enabled_checkbox,
+ self.cam_fps,
+ self.cam_exposure,
+ self.cam_gain,
+ self.cam_crop_x0,
+ self.cam_crop_y0,
+ self.cam_crop_x1,
+ self.cam_crop_y1,
+ ]
+ for field in fields:
+ if hasattr(field, "lineEdit"):
+ if hasattr(field.lineEdit(), "returnPressed"):
+ field.lineEdit().returnPressed.connect(self._apply_camera_settings)
+ if hasattr(field, "installEventFilter"):
+ field.installEventFilter(self)
+
+ # Maintain overlay geometry when resizing
+ def resizeEvent(self, event):
+ super().resizeEvent(event)
+ if hasattr(self, "_loading_overlay") and self._loading_overlay.isVisible():
+ self._position_loading_overlay()
+
+ def eventFilter(self, obj, event):
+ # Keep your existing overlay resize handling
+ if obj is self.available_cameras_list and event.type() == event.Type.Resize:
+ if self._scan_overlay and self._scan_overlay.isVisible():
+ self._position_scan_overlay()
+ return super().eventFilter(obj, event)
+
+ # Intercept Enter in FPS and crop spinboxes
+ if event.type() == QEvent.KeyPress and isinstance(event, QKeyEvent):
+ if event.key() in (Qt.Key_Return, Qt.Key_Enter):
+ if obj in (self.cam_fps, self.cam_crop_x0, self.cam_crop_y0, self.cam_crop_x1, self.cam_crop_y1):
+ # Commit any pending text → value
+ try:
+ obj.interpretText()
+ except Exception:
+ pass
+ # Apply settings to persist crop/FPS to CameraSettings
+ self._apply_camera_settings()
+ # Consume so OK isn't triggered
+ return True
+
+ return super().eventFilter(obj, event)
+
+ def _position_scan_overlay(self) -> None:
+ """Position scan overlay to cover the available_cameras_list area."""
+ if not self._scan_overlay or not self.available_cameras_list:
+ return
+ parent = self._scan_overlay.parent() # available_group
+ top_left = self.available_cameras_list.mapTo(parent, self.available_cameras_list.rect().topLeft())
+ rect = self.available_cameras_list.rect()
+ self._scan_overlay.setGeometry(top_left.x(), top_left.y(), rect.width(), rect.height())
+
+ def _show_scan_overlay(self, message: str = "Discovering cameras…") -> None:
+ self._scan_overlay.setText(message)
+ self._scan_overlay.setVisible(True)
+ self._position_scan_overlay()
+
+ def _hide_scan_overlay(self) -> None:
+ self._scan_overlay.setVisible(False)
+
+ def _position_loading_overlay(self):
+ # Cover just the preview image area (label), not the whole group
+ if not self.preview_label:
+ return
+ gp = self.preview_label.mapTo(self.preview_group, self.preview_label.rect().topLeft())
+ rect = self.preview_label.rect()
+ self._loading_overlay.setGeometry(gp.x(), gp.y(), rect.width(), rect.height())
+
+ # -------------------------------
+ # Signals / population
+ # -------------------------------
+ def _connect_signals(self) -> None:
+ self.backend_combo.currentIndexChanged.connect(self._on_backend_changed)
+ self.refresh_btn.clicked.connect(self._refresh_available_cameras)
+ self.add_camera_btn.clicked.connect(self._add_selected_camera)
+ self.remove_camera_btn.clicked.connect(self._remove_selected_camera)
+ self.move_up_btn.clicked.connect(self._move_camera_up)
+ self.move_down_btn.clicked.connect(self._move_camera_down)
+ self.active_cameras_list.currentRowChanged.connect(self._on_active_camera_selected)
+ self.available_cameras_list.currentRowChanged.connect(self._on_available_camera_selected)
+ self.available_cameras_list.itemDoubleClicked.connect(self._on_available_camera_double_clicked)
+ self.apply_settings_btn.clicked.connect(self._apply_camera_settings)
+ self.preview_btn.clicked.connect(self._toggle_preview)
+ self.ok_btn.clicked.connect(self._on_ok_clicked)
+ self.cancel_btn.clicked.connect(self.reject)
+ self.scan_started.connect(lambda _: setattr(self, "_dialog_active", True))
+ self.scan_finished.connect(lambda: setattr(self, "_dialog_active", False))
+ for sb in (self.cam_fps, self.cam_crop_x0, self.cam_crop_y0, self.cam_crop_x1, self.cam_crop_y1):
+ if hasattr(sb, "valueChanged"):
+ sb.valueChanged.connect(lambda _=None: self.apply_settings_btn.setEnabled(True))
+ self.cam_rotation.currentIndexChanged.connect(lambda _: self.apply_settings_btn.setEnabled(True))
+
+ def _populate_from_settings(self) -> None:
+ """Populate the dialog from existing settings."""
+ self.active_cameras_list.clear()
+ for i, cam in enumerate(self._working_settings.cameras):
+ item = QListWidgetItem(self._format_camera_label(cam, i))
+ item.setData(Qt.ItemDataRole.UserRole, cam)
+ if not cam.enabled:
+ item.setForeground(Qt.GlobalColor.gray)
+ self.active_cameras_list.addItem(item)
+
+ self._refresh_available_cameras()
+ self._update_button_states()
+
+ def _format_camera_label(self, cam: CameraSettingsModel, index: int = -1) -> str:
+ status = "✓" if cam.enabled else "○"
+ this_id = f"{cam.backend}:{cam.index}"
+ dlc_indicator = " [DLC]" if this_id == self._dlc_camera_id and cam.enabled else ""
+ return f"{status} {cam.name} [{cam.backend}:{cam.index}]{dlc_indicator}"
+
+ def _refresh_camera_labels(self) -> None:
+ cam_list = getattr(self, "active_cameras_list", None)
+ if cam_list:
+ for i in range(cam_list.count()):
+ item = cam_list.item(i)
+ cam = item.data(Qt.ItemDataRole.UserRole)
+ if cam:
+ item.setText(self._format_camera_label(cam, i))
+
+ def _on_backend_changed(self, _index: int) -> None:
+ self._refresh_available_cameras()
+
+ def _is_backend_opencv(self, backend_name: str) -> bool:
+ return backend_name.lower() == "opencv"
+
+ def _update_controls_for_backend(self, backend_name: str) -> None:
+ # FIXME in camera backend ABC, we should have a method to query supported features
+ is_opencv = self._is_backend_opencv(backend_name)
+ self.cam_exposure.setEnabled(not is_opencv)
+ self.cam_gain.setEnabled(not is_opencv)
+
+ tip = ""
+ if is_opencv:
+ tip = (
+ "Exposure/Gain are not configurable via the generic OpenCV backend and "
+ "will be ignored by most UVC devices."
+ )
+ self.cam_exposure.setToolTip(tip)
+ self.cam_gain.setToolTip(tip)
+
+ def _refresh_available_cameras(self) -> None:
+ """Refresh the list of available cameras asynchronously."""
+ backend = self.backend_combo.currentData()
+ if not backend:
+ backend = self.backend_combo.currentText().split()[0]
+
+ # If already scanning, ignore new requests to avoid races
+ if getattr(self, "_scan_worker", None) and self._scan_worker.isRunning():
+ self._show_scan_overlay("Already discovering cameras…")
+ return
+
+ # Reset list UI and show progress
+ self.available_cameras_list.clear()
+ self._detected_cameras = []
+ msg = f"Discovering {backend} cameras…"
+ self._show_scan_overlay(msg)
+ self.scan_progress.setRange(0, 0)
+ self.scan_progress.setVisible(True)
+ self.scan_cancel_btn.setVisible(True)
+ self.add_camera_btn.setEnabled(False)
+ self.refresh_btn.setEnabled(False)
+ self.backend_combo.setEnabled(False)
+
+ # Start worker
+ self._scan_worker = DetectCamerasWorker(backend, max_devices=10, parent=self)
+ self._scan_worker.progress.connect(self._on_scan_progress)
+ self._scan_worker.result.connect(self._on_scan_result)
+ self._scan_worker.error.connect(self._on_scan_error)
+ self._scan_worker.finished.connect(self._on_scan_finished)
+ self.scan_started.emit(f"Scanning {backend} cameras…")
+ self._scan_worker.start()
+
+ def _on_scan_progress(self, msg: str) -> None:
+ self._show_scan_overlay(msg or "Discovering cameras…")
+
+ def _on_scan_result(self, cams: list) -> None:
+ self._detected_cameras = cams or []
+ for cam in self._detected_cameras:
+ item = QListWidgetItem(f"{cam.label} (index {cam.index})")
+ item.setData(Qt.ItemDataRole.UserRole, cam)
+ self.available_cameras_list.addItem(item)
+
+ def _on_scan_error(self, msg: str) -> None:
+ QMessageBox.warning(self, "Camera Scan", f"Failed to detect cameras:\n{msg}")
+
+ def _on_scan_finished(self) -> None:
+ self._hide_scan_overlay()
+ self.scan_progress.setVisible(False)
+ self._scan_worker = None
+
+ self.scan_cancel_btn.setVisible(False)
+ self.scan_cancel_btn.setEnabled(True)
+ self.refresh_btn.setEnabled(True)
+ self.backend_combo.setEnabled(True)
+
+ self._update_button_states()
+ self.scan_finished.emit()
+
+ def _on_scan_cancel(self) -> None:
+ """User requested to cancel discovery."""
+ if self._scan_worker and self._scan_worker.isRunning():
+ try:
+ self._scan_worker.requestInterruption()
+ except Exception:
+ pass
+ # Keep the busy bar, update texts
+ self._show_scan_overlay("Canceling discovery…")
+ self.scan_progress.setVisible(True) # stay visible as indeterminate
+ self.scan_cancel_btn.setEnabled(False)
+
+ def _on_available_camera_selected(self, row: int) -> None:
+ self.add_camera_btn.setEnabled(row >= 0)
+
+ def _on_available_camera_double_clicked(self, item: QListWidgetItem) -> None:
+ self._add_selected_camera()
+
+ def _on_active_camera_selected(self, row: int) -> None:
+ # Stop any running preview when selection changes
+ if self._preview_active:
+ self._stop_preview()
+ self._current_edit_index = row
+ self._update_button_states()
+ if row < 0 or row >= self.active_cameras_list.count():
+ self._clear_settings_form()
+ return
+ item = self.active_cameras_list.item(row)
+ cam = item.data(Qt.ItemDataRole.UserRole)
+ if cam:
+ self.apply_settings_btn.setEnabled(True)
+ self._load_camera_to_form(cam)
+
+ # -------------------------------
+ # UI helpers/actions
+ # -------------------------------
+
+ def _needs_preview_reopen(self, cam: CameraSettingsModel) -> bool:
+ if not (self._preview_active and self._preview_backend):
+ return False
+
+ # FPS: for OpenCV, treat FPS changes as requiring reopen.
+ if self._is_backend_opencv(cam.backend):
+ prev_fps = getattr(self._preview_backend.settings, "fps", None)
+ if isinstance(prev_fps, (int, float)) and abs(cam.fps - float(prev_fps)) > 0.1:
+ return True
+
+ return any(
+ [
+ cam.exposure != getattr(self._preview_backend.settings, "exposure", cam.exposure),
+ cam.gain != getattr(self._preview_backend.settings, "gain", cam.gain),
+ cam.rotation != getattr(self._preview_backend.settings, "rotation", cam.rotation),
+ (cam.crop_x0, cam.crop_y0, cam.crop_x1, cam.crop_y1)
+ != (
+ getattr(self._preview_backend.settings, "crop_x0", cam.crop_x0),
+ getattr(self._preview_backend.settings, "crop_y0", cam.crop_y0),
+ getattr(self._preview_backend.settings, "crop_x1", cam.crop_x1),
+ getattr(self._preview_backend.settings, "crop_y1", cam.crop_y1),
+ ),
+ ]
+ )
+
+ def _backend_actual_fps(self) -> float | None:
+ """Return backend's actual FPS if known; for OpenCV do NOT fall back to settings.fps."""
+ if not self._preview_backend:
+ return None
+ try:
+ actual = getattr(self._preview_backend, "actual_fps", None)
+ if isinstance(actual, (int, float)) and actual > 0:
+ return float(actual)
+ return None
+ except Exception:
+ return None
+
+ def _adjust_preview_timer_for_fps(self, fps: float | None) -> None:
+ """Adjust preview cadence to match actual FPS (bounded for CPU)."""
+ if not self._preview_timer or not fps or fps <= 0:
+ return
+ interval_ms = max(15, int(1000.0 / min(max(fps, 1.0), 60.0)))
+ self._preview_timer.start(interval_ms)
+
+ def _reconcile_fps_from_backend(self, cam: CameraSettingsModel) -> None:
+ """Clamp UI/settings to measured device FPS when we can actually measure it."""
+ if not self._is_backend_opencv(cam.backend):
+ return
+
+ actual = self._backend_actual_fps()
+ if actual is None:
+ # OpenCV can't reliably report FPS; do not overwrite user's requested value.
+ self._append_status("[Info] OpenCV can't reliably report actual FPS; keeping requested value.")
+ return
+
+ if abs(cam.fps - actual) > 0.5:
+ cam.fps = actual
+ self.cam_fps.setValue(actual)
+ self._append_status(f"[Info] FPS adjusted to device-supported ~{actual:.2f}.")
+ self._adjust_preview_timer_for_fps(actual)
+
+ def _update_active_list_item(self, row: int, cam: CameraSettingsModel) -> None:
+ """Refresh the active camera list row text and color."""
+ item = self.active_cameras_list.item(row)
+ if not item:
+ return
+ item.setText(self._format_camera_label(cam, row))
+ item.setData(Qt.ItemDataRole.UserRole, cam)
+ item.setForeground(Qt.GlobalColor.gray if not cam.enabled else Qt.GlobalColor.black)
+ self._refresh_camera_labels()
+ self._update_button_states()
+
+ def _load_camera_to_form(self, cam: CameraSettingsModel) -> None:
+ self.cam_enabled_checkbox.setChecked(cam.enabled)
+ self.cam_name_label.setText(cam.name)
+ self.cam_index_label.setText(str(cam.index))
+ self.cam_backend_label.setText(cam.backend)
+ self._update_controls_for_backend(cam.backend)
+ self.cam_fps.setValue(cam.fps)
+ self.cam_exposure.setValue(cam.exposure)
+ self.cam_gain.setValue(cam.gain)
+ rot_index = self.cam_rotation.findData(cam.rotation)
+ if rot_index >= 0:
+ self.cam_rotation.setCurrentIndex(rot_index)
+ self.cam_crop_x0.setValue(cam.crop_x0)
+ self.cam_crop_y0.setValue(cam.crop_y0)
+ self.cam_crop_x1.setValue(cam.crop_x1)
+ self.cam_crop_y1.setValue(cam.crop_y1)
+ self.apply_settings_btn.setEnabled(True)
+
+ def _write_form_to_cam(self, cam: CameraSettingsModel) -> None:
+ cam.enabled = bool(self.cam_enabled_checkbox.isChecked())
+ cam.fps = float(self.cam_fps.value())
+ cam.exposure = int(self.cam_exposure.value())
+ cam.gain = float(self.cam_gain.value())
+ cam.rotation = int(self.cam_rotation.currentData() or 0)
+ cam.crop_x0 = int(self.cam_crop_x0.value())
+ cam.crop_y0 = int(self.cam_crop_y0.value())
+ cam.crop_x1 = int(self.cam_crop_x1.value())
+ cam.crop_y1 = int(self.cam_crop_y1.value())
+
+ def _clear_settings_form(self) -> None:
+ self.cam_enabled_checkbox.setChecked(True)
+ self.cam_name_label.setText("")
+ self.cam_index_label.setText("")
+ self.cam_backend_label.setText("")
+ self.cam_fps.setValue(30.0)
+ self.cam_exposure.setValue(0)
+ self.cam_gain.setValue(0.0)
+ self.cam_rotation.setCurrentIndex(0)
+ self.cam_crop_x0.setValue(0)
+ self.cam_crop_y0.setValue(0)
+ self.cam_crop_x1.setValue(0)
+ self.cam_crop_y1.setValue(0)
+ self.apply_settings_btn.setEnabled(False)
+
+ def _add_selected_camera(self) -> None:
+ row = self.available_cameras_list.currentRow()
+ if row < 0:
+ return
+ # limit check
+ active_count = len(
+ [
+ i
+ for i in range(self.active_cameras_list.count())
+ if self.active_cameras_list.item(i).data(Qt.ItemDataRole.UserRole).enabled
+ ]
+ )
+ if active_count >= self.MAX_CAMERAS:
+ QMessageBox.warning(self, "Maximum Cameras", f"Maximum of {self.MAX_CAMERAS} active cameras allowed.")
+ return
+ item = self.available_cameras_list.item(row)
+ detected = item.data(Qt.ItemDataRole.UserRole)
+ backend = self.backend_combo.currentData() or "opencv"
+
+ for i in range(self.active_cameras_list.count()):
+ existing_cam = self.active_cameras_list.item(i).data(Qt.ItemDataRole.UserRole)
+ if existing_cam.backend == backend and existing_cam.index == detected.index:
+ QMessageBox.warning(
+ self, "Duplicate Camera", f"Camera '{backend}:{detected.index}' is already in the active list."
+ )
+ return
+
+ new_cam = CameraSettingsModel(
+ name=detected.label,
+ index=detected.index,
+ fps=30.0,
+ backend=backend,
+ exposure=0,
+ gain=0.0,
+ enabled=True,
+ )
+ self._working_settings.cameras.append(new_cam)
+ new_index = len(self._working_settings.cameras) - 1
+ new_item = QListWidgetItem(self._format_camera_label(new_cam, new_index))
+ new_item.setData(Qt.ItemDataRole.UserRole, new_cam)
+ self.active_cameras_list.addItem(new_item)
+ self.active_cameras_list.setCurrentItem(new_item)
+ self._refresh_camera_labels()
+ self._update_button_states()
+
+ def _remove_selected_camera(self) -> None:
+ row = self.active_cameras_list.currentRow()
+ if row < 0:
+ return
+ self.active_cameras_list.takeItem(row)
+ if row < len(self._working_settings.cameras):
+ del self._working_settings.cameras[row]
+ self._current_edit_index = None
+ self._clear_settings_form()
+ self._refresh_camera_labels()
+ self._update_button_states()
+
+ def _move_camera_up(self) -> None:
+ row = self.active_cameras_list.currentRow()
+ if row <= 0:
+ return
+ item = self.active_cameras_list.takeItem(row)
+ self.active_cameras_list.insertItem(row - 1, item)
+ self.active_cameras_list.setCurrentRow(row - 1)
+ cams = self._working_settings.cameras
+ cams[row], cams[row - 1] = cams[row - 1], cams[row]
+ self._refresh_camera_labels()
+
+ def _move_camera_down(self) -> None:
+ row = self.active_cameras_list.currentRow()
+ if row < 0 or row >= self.active_cameras_list.count() - 1:
+ return
+ item = self.active_cameras_list.takeItem(row)
+ self.active_cameras_list.insertItem(row + 1, item)
+ self.active_cameras_list.setCurrentRow(row + 1)
+ cams = self._working_settings.cameras
+ cams[row], cams[row + 1] = cams[row + 1], cams[row]
+ self._refresh_camera_labels()
+
+ def _apply_camera_settings(self) -> None:
+ try:
+ for sb in (self.cam_fps, self.cam_crop_x0, self.cam_crop_y0, self.cam_crop_x1, self.cam_crop_y1):
+ try:
+ sb.interpretText()
+ except Exception:
+ pass
+ if self._current_edit_index is None:
+ return
+ row = self._current_edit_index
+ if row < 0 or row >= len(self._working_settings.cameras):
+ return
+
+ current_model = self._working_settings.cameras[row]
+ new_model = self._build_model_from_form(current_model)
+
+ cam = self._working_settings.cameras[row]
+ self._write_form_to_cam(cam)
+
+ must_reopen = False
+ if self._preview_active and self._preview_backend:
+ prev_model = getattr(self._preview_backend, "settings", None)
+ if prev_model:
+ must_reopen = self._needs_preview_reopen(new_model)
+
+ if self._preview_active:
+ if must_reopen:
+ self._stop_preview()
+ self._start_preview()
+ else:
+ self._reconcile_fps_from_backend(new_model)
+ if not self._backend_actual_fps():
+ self._append_status("[Info] FPS will reconcile automatically during preview.")
+
+ # Persist validated model back
+ self._working_settings.cameras[row] = new_model
+ self._update_active_list_item(row, new_model)
+
+ except Exception as exc:
+ LOGGER.exception("Apply camera settings failed")
+ QMessageBox.warning(self, "Apply Settings Error", str(exc))
+
+ def _update_button_states(self) -> None:
+ active_row = self.active_cameras_list.currentRow()
+ has_active_selection = active_row >= 0
+ self.remove_camera_btn.setEnabled(has_active_selection)
+ self.move_up_btn.setEnabled(has_active_selection and active_row > 0)
+ self.move_down_btn.setEnabled(has_active_selection and active_row < self.active_cameras_list.count() - 1)
+ # During loading, preview button becomes "Cancel Loading"
+ self.preview_btn.setEnabled(has_active_selection or self._loading_active)
+ available_row = self.available_cameras_list.currentRow()
+ self.add_camera_btn.setEnabled(available_row >= 0)
+
+ def _on_ok_clicked(self) -> None:
+ self._stop_preview()
+ active = self._working_settings.get_active_cameras()
+ if self._working_settings.cameras and not active:
+ QMessageBox.warning(self, "No Active Cameras", "Please enable at least one camera or remove all cameras.")
+ return
+ self.settings_changed.emit(copy.deepcopy(self._working_settings))
+ self.accept()
+
+ def reject(self) -> None:
+ """Handle dialog rejection (Cancel or close)."""
+ self._stop_preview()
+
+ if getattr(self, "_scan_worker", None) and self._scan_worker.isRunning():
+ try:
+ self._scan_worker.requestInterruption()
+ except Exception:
+ pass
+ self._scan_worker.wait(1500)
+ self._scan_worker = None
+
+ self._hide_scan_overlay()
+ self.scan_progress.setVisible(False)
+ self.scan_cancel_btn.setVisible(False)
+ self.scan_cancel_btn.setEnabled(True)
+ self.refresh_btn.setEnabled(True)
+ self.backend_combo.setEnabled(True)
+
+ super().reject()
+
+ # -------------------------------
+ # Preview start/stop (ASYNC)
+ # -------------------------------
+ def _toggle_preview(self) -> None:
+ if self._loading_active:
+ self._cancel_loading()
+ return
+ if self._preview_active:
+ self._stop_preview()
+ else:
+ self._start_preview()
+
+ def _start_preview(self) -> None:
+ """Start camera preview asynchronously (no UI freeze)."""
+ if self._current_edit_index is None or self._current_edit_index < 0:
+ return
+ item = self.active_cameras_list.item(self._current_edit_index)
+ if not item:
+ return
+ cam = item.data(Qt.ItemDataRole.UserRole)
+ if not cam:
+ return
+
+ # Ensure any existing preview or loader is stopped/canceled
+ self._stop_preview()
+ # if self._loader and self._loader.isRunning():
+ # self._loader.request_cancel()
+ # Create worker
+ self._loader = CameraLoadWorker(cam, self)
+ self._loader.progress.connect(self._on_loader_progress)
+ self._loader.success.connect(self._on_loader_success)
+ self._loader.error.connect(self._on_loader_error)
+ self._loader.canceled.connect(self._on_loader_canceled)
+ self._loader.finished.connect(self._on_loader_finished)
+ self._loading_active = True
+ self._update_button_states()
+
+ # Prepare UI
+ self.preview_group.setVisible(True)
+ self.preview_label.setText("No preview")
+ self.preview_status.clear()
+ self._show_loading_overlay("Loading camera…")
+ self._set_preview_button_loading(True)
+
+ self._loader.start()
+
+ def _stop_preview(self) -> None:
+ """Stop camera preview and cancel any ongoing loading."""
+ # Cancel loader if running
+ if self._loader and self._loader.isRunning():
+ self._loader.request_cancel()
+ self._loader.wait(1500)
+ self._loader = None
+ # Stop timer
+ if self._preview_timer:
+ self._preview_timer.stop()
+ self._preview_timer = None
+ # Close backend
+ if self._preview_backend:
+ try:
+ self._preview_backend.close()
+ except Exception:
+ pass
+ self._preview_backend = None
+ # Reset UI
+ self._loading_active = False
+ self._preview_active = False
+ self._set_preview_button_loading(False)
+ self.preview_btn.setText("Start Preview")
+ self.preview_btn.setIcon(self.style().standardIcon(QStyle.StandardPixmap.SP_MediaPlay))
+ self.preview_group.setVisible(False)
+ self.preview_label.setText("No preview")
+ self.preview_label.setPixmap(QPixmap())
+ self._hide_loading_overlay()
+ self._update_button_states()
+
+ # -------------------------------
+ # Loader UI helpers / slots
+ # -------------------------------
+ def _set_preview_button_loading(self, loading: bool) -> None:
+ if loading:
+ self.preview_btn.setText("Cancel Loading")
+ self.preview_btn.setIcon(self.style().standardIcon(QStyle.StandardPixmap.SP_BrowserStop))
+ else:
+ self.preview_btn.setText("Start Preview")
+ self.preview_btn.setIcon(self.style().standardIcon(QStyle.StandardPixmap.SP_MediaPlay))
+
+ def _show_loading_overlay(self, message: str) -> None:
+ self._loading_overlay.setText(message)
+ self._loading_overlay.setVisible(True)
+ self._position_loading_overlay()
+
+ def _hide_loading_overlay(self) -> None:
+ self._loading_overlay.setVisible(False)
+
+ def _append_status(self, text: str) -> None:
+ self.preview_status.append(text)
+ self.preview_status.moveCursor(QTextCursor.End)
+ self.preview_status.ensureCursorVisible()
+
+ def _cancel_loading(self) -> None:
+ if self._loader and self._loader.isRunning():
+ self._append_status("Cancel requested…")
+ self._loader.request_cancel()
+ # UI will flip back on finished -> _on_loader_finished
+ else:
+ self._loading_active = False
+ self._set_preview_button_loading(False)
+ self._hide_loading_overlay()
+ self._update_button_states()
+
+ # Loader signal handlers
+ def _on_loader_progress(self, message: str) -> None:
+ self._show_loading_overlay(message)
+ self._append_status(message)
+
+ def _on_loader_success(self, payload) -> None:
+ try:
+ if isinstance(payload, CameraSettingsModel):
+ cam_settings = payload
+ self._append_status("Opening camera…")
+ self._preview_backend = CameraFactory.create(cam_settings)
+ self._preview_backend.open()
+ else:
+ raise TypeError(f"Unexpected success payload type: {type(payload)}")
+
+ # Start preview UX
+ self._append_status("Starting preview…")
+ self._preview_active = True
+ self.preview_btn.setText("Stop Preview")
+ self.preview_btn.setIcon(self.style().standardIcon(QStyle.StandardPixmap.SP_MediaStop))
+ self.preview_group.setVisible(True)
+ self.preview_label.setText("Starting…")
+ self._hide_loading_overlay()
+
+ # Timer @ ~25 fps default; cadence may be overridden above
+ self._preview_timer = QTimer(self)
+ self._preview_timer.timeout.connect(self._update_preview)
+ self._preview_timer.start(40)
+
+ # FPS reconciliation + cadence (single source of truth)
+ actual_fps = self._backend_actual_fps()
+ if actual_fps:
+ self._adjust_preview_timer_for_fps(actual_fps)
+
+ self.apply_settings_btn.setEnabled(True)
+ except Exception as exc:
+ self._on_loader_error(str(exc))
+
+ def _on_loader_error(self, error: str) -> None:
+ self._append_status(f"Error: {error}")
+ LOGGER.exception("Failed to start preview")
+ self._preview_active = False
+ self._loading_active = False
+ self._hide_loading_overlay()
+ self.preview_group.setVisible(False)
+ self._set_preview_button_loading(False)
+ self._update_button_states()
+ QMessageBox.warning(self, "Preview Error", f"Failed to start camera preview:\n{error}")
+
+ def _on_loader_canceled(self) -> None:
+ self._append_status("Loading canceled.")
+ self._hide_loading_overlay()
+
+ def _on_loader_finished(self):
+ self._loading_active = False
+ self._loader = None
+
+ # If preview ended successfully, ensure Stop Preview is shown
+ if self._preview_active:
+ self.preview_btn.setText("Stop Preview")
+ self.preview_btn.setIcon(self.style().standardIcon(QStyle.StandardPixmap.SP_MediaStop))
+ else:
+ # Otherwise show Start Preview
+ self.preview_btn.setText("Start Preview")
+ self.preview_btn.setIcon(self.style().standardIcon(QStyle.StandardPixmap.SP_MediaPlay))
+
+ # ALWAYS refresh button states
+ self._update_button_states()
+
+ # -------------------------------
+ # Preview frame update
+ # -------------------------------
+ def _update_preview(self) -> None:
+ """Update preview frame."""
+ if not self._preview_backend or not self._preview_active:
+ return
+
+ try:
+ frame, _ = self._preview_backend.read()
+ if frame is None or frame.size == 0:
+ return
+
+ # Apply rotation if set in the form (real-time from UI)
+ rotation = self.cam_rotation.currentData()
+ if rotation == 90:
+ frame = cv2.rotate(frame, cv2.ROTATE_90_CLOCKWISE)
+ elif rotation == 180:
+ frame = cv2.rotate(frame, cv2.ROTATE_180)
+ elif rotation == 270:
+ frame = cv2.rotate(frame, cv2.ROTATE_90_COUNTERCLOCKWISE)
+
+ # Apply crop if set in the form (real-time from UI)
+ h, w = frame.shape[:2]
+ x0 = self.cam_crop_x0.value()
+ y0 = self.cam_crop_y0.value()
+ x1 = self.cam_crop_x1.value() or w
+ y1 = self.cam_crop_y1.value() or h
+ # Clamp to frame bounds
+ x0 = max(0, min(x0, w))
+ y0 = max(0, min(y0, h))
+ x1 = max(x0, min(x1, w))
+ y1 = max(y0, min(y1, h))
+ if x1 > x0 and y1 > y0:
+ frame = frame[y0:y1, x0:x1]
+
+ # Resize to fit preview label
+ h, w = frame.shape[:2]
+ max_w, max_h = 400, 300
+ scale = min(max_w / w, max_h / h)
+ new_w, new_h = int(w * scale), int(h * scale)
+ frame = cv2.resize(frame, (new_w, new_h))
+
+ # Convert to QImage and display
+ if frame.ndim == 2:
+ frame = cv2.cvtColor(frame, cv2.COLOR_GRAY2RGB)
+ elif frame.shape[2] == 4:
+ frame = cv2.cvtColor(frame, cv2.COLOR_BGRA2RGB)
+ else:
+ frame = cv2.cvtColor(frame, cv2.COLOR_BGR2RGB)
+
+ h, w, ch = frame.shape
+ bytes_per_line = ch * w
+ q_img = QImage(frame.data, w, h, bytes_per_line, QImage.Format.Format_RGB888)
+ self.preview_label.setPixmap(QPixmap.fromImage(q_img))
+
+ except Exception as exc:
+ LOGGER.debug(f"Preview frame skipped: {exc}")
diff --git a/dlclivegui/gui/main_window.py b/dlclivegui/gui/main_window.py
new file mode 100644
index 0000000..d6c5f96
--- /dev/null
+++ b/dlclivegui/gui/main_window.py
@@ -0,0 +1,1566 @@
+"""PySide6 based GUI for DeepLabCut Live."""
+
+from __future__ import annotations
+
+import importlib.metadata
+import json
+import logging
+import os
+import time
+from pathlib import Path
+
+os.environ["PYLON_CAMEMU"] = "2"
+
+import cv2
+import numpy as np
+from PySide6.QtCore import QSettings, Qt, QTimer
+from PySide6.QtGui import QAction, QActionGroup, QCloseEvent, QFont, QIcon, QImage, QPainter, QPixmap
+from PySide6.QtWidgets import (
+ QCheckBox,
+ QComboBox,
+ QFileDialog,
+ QFormLayout,
+ QGridLayout,
+ QGroupBox,
+ QHBoxLayout,
+ QLabel,
+ QLineEdit,
+ QMainWindow,
+ QMessageBox,
+ QPushButton,
+ QSizePolicy,
+ QSpinBox,
+ QStatusBar,
+ QStyle,
+ QVBoxLayout,
+ QWidget,
+)
+
+from dlclivegui.cameras import CameraFactory
+from dlclivegui.config import (
+ # DEFAULT_CONFIG,
+ # ApplicationSettings,
+ # BoundingBoxSettings,
+ # CameraSettings,
+ # DLCProcessorSettings,
+ ModelPathStore,
+ # MultiCameraSettings,
+ # RecordingSettings,
+ # VisualizationSettings,
+)
+from dlclivegui.gui.camera_config_dialog import CameraConfigDialog
+from dlclivegui.gui.recording_manager import RecordingManager
+from dlclivegui.gui.theme import LOGO, LOGO_ALPHA, AppStyle, apply_theme
+from dlclivegui.processors.processor_utils import instantiate_from_scan, scan_processor_folder
+from dlclivegui.services.dlc_processor import DLCLiveProcessor, PoseResult, ProcessorStats
+from dlclivegui.services.multi_camera_controller import MultiCameraController, MultiFrameData, get_camera_id
+from dlclivegui.services.video_recorder import RecorderStats
+from dlclivegui.utils.config_models import (
+ DEFAULT_CONFIG,
+ ApplicationSettingsModel,
+ BoundingBoxSettingsModel,
+ CameraSettingsModel,
+ DLCProcessorSettingsModel,
+ MultiCameraSettingsModel,
+ RecordingSettingsModel,
+ VisualizationSettingsModel,
+)
+from dlclivegui.utils.display import compute_tile_info, create_tiled_frame, draw_bbox, draw_pose
+from dlclivegui.utils.utils import FPSTracker
+
+# logging.basicConfig(level=logging.INFO)
+logging.basicConfig(level=logging.DEBUG) # FIXME @C-Achard set back to INFO for release
+logger = logging.getLogger("DLCLiveGUI")
+
+
+class DLCLiveMainWindow(QMainWindow):
+ """Main application window."""
+
+ def __init__(self, config: ApplicationSettingsModel | None = None):
+ super().__init__()
+ self.setWindowTitle("DeepLabCut Live GUI")
+
+ # Try to load myconfig.json from the application directory if no config provided
+ # FIXME @C-Achard change this behavior for release
+ if config is None:
+ myconfig_path = Path(__file__).parent.parent / "myconfig.json"
+ if myconfig_path.exists():
+ try:
+ config = ApplicationSettingsModel.load(str(myconfig_path))
+ self._config_path = myconfig_path
+ logger.info(f"Loaded configuration from {myconfig_path}")
+ except Exception as exc:
+ logger.warning(f"Failed to load myconfig.json: {exc}. Using default config.")
+ config = DEFAULT_CONFIG
+ self._config_path = None
+ else:
+ config = DEFAULT_CONFIG
+ self._config_path = None
+ else:
+ self._config_path = None
+
+ self.settings = QSettings("DeepLabCut", "DLCLiveGUI")
+ self._model_path_store = ModelPathStore(self.settings)
+ self._fps_tracker = FPSTracker()
+ self._rec_manager = RecordingManager()
+ self._dlc = DLCLiveProcessor()
+ self.multi_camera_controller = MultiCameraController()
+
+ self._config = config
+ self._inference_camera_id: str | None = None # Camera ID used for inference
+ self._running_cams_ids: set[str] = set()
+ self._current_frame: np.ndarray | None = None
+ self._raw_frame: np.ndarray | None = None
+ self._last_pose: PoseResult | None = None
+ self._dlc_active: bool = False
+ self._active_camera_settings: CameraSettingsModel | None = None
+ self._last_drop_warning = 0.0
+ self._last_recorder_summary = "Recorder idle"
+ self._display_interval = 1.0 / 25.0
+ self._last_display_time = 0.0
+ self._dlc_initialized = False
+ self._scanned_processors: dict = {}
+ self._processor_keys: list = []
+ self._last_processor_vid_recording = False
+ self._auto_record_session_name: str | None = None
+ self._bbox_x0 = 0
+ self._bbox_y0 = 0
+ self._bbox_x1 = 0
+ self._bbox_y1 = 0
+ self._bbox_enabled = False
+ # UI elements
+ self._current_style: AppStyle = AppStyle.DARK
+ self._cam_dialog: CameraConfigDialog | None = None
+
+ # Visualization settings (will be updated from config)
+ self._p_cutoff = 0.6
+ self._colormap = "hot"
+ self._bbox_color = (0, 0, 255) # BGR: red
+
+ # Multi-camera state
+ self._multi_camera_mode = False
+ self._multi_camera_frames: dict[str, np.ndarray] = {}
+ # DLC pose rendering info for tiled view
+ self._dlc_tile_offset: tuple[int, int] = (0, 0) # (x, y) offset in tiled frame
+ self._dlc_tile_scale: tuple[float, float] = (1.0, 1.0) # (scale_x, scale_y)
+ # Display flag (decoupled from frame capture for performance)
+ self._display_dirty: bool = False
+
+ self._load_icons()
+ self._preview_pixmap = QPixmap(LOGO_ALPHA)
+ self._setup_ui()
+ self._connect_signals()
+ self._apply_config(self._config)
+ self._refresh_processors() # Scan and populate processor dropdown
+ self._update_inference_buttons()
+ self._update_camera_controls_enabled()
+ self._metrics_timer = QTimer(self)
+ self._metrics_timer.setInterval(500)
+ self._metrics_timer.timeout.connect(self._update_metrics)
+ self._metrics_timer.start()
+ self._update_metrics()
+
+ # Display timer - decoupled from frame capture for performance
+ self._display_timer = QTimer(self)
+ self._display_timer.setInterval(33) # ~30 fps display rate
+ self._display_timer.timeout.connect(self._update_display_from_pending)
+ self._display_timer.start()
+
+ # Show status message if myconfig.json was loaded
+ if self._config_path and self._config_path.name == "myconfig.json":
+ self.statusBar().showMessage(f"Auto-loaded configuration from {self._config_path}", 5000)
+
+ # Validate cameras from loaded config (deferred to allow window to show first)
+ QTimer.singleShot(100, self._validate_configured_cameras)
+
+ def resizeEvent(self, event):
+ super().resizeEvent(event)
+ if not self.multi_camera_controller.is_running():
+ self._show_logo_and_text()
+
+ # ------------------------------------------------------------------ UI
+ def _init_theme_actions(self) -> None:
+ """Set initial checked state for theme actions based on current app stylesheet."""
+ self.action_dark_mode.setChecked(self._current_style == AppStyle.DARK)
+ self.action_light_mode.setChecked(self._current_style == AppStyle.SYS_DEFAULT)
+
+ def _apply_theme(self, mode: AppStyle) -> None:
+ """Apply the selected theme and update menu action states."""
+ apply_theme(mode, self.action_dark_mode, self.action_light_mode)
+ self._current_style = mode
+
+ def _load_icons(self):
+ self.setWindowIcon(QIcon(LOGO))
+
+ def _setup_ui(self) -> None:
+ central = QWidget()
+ layout = QHBoxLayout(central)
+
+ # Video panel with display and performance stats
+ video_panel = QWidget()
+ video_layout = QVBoxLayout(video_panel)
+ video_layout.setContentsMargins(0, 0, 0, 0)
+
+ # Video display widget
+ self.video_label = QLabel()
+ self.video_label.setAlignment(Qt.AlignmentFlag.AlignCenter)
+ self.video_label.setMinimumSize(640, 360)
+ self.video_label.setSizePolicy(QSizePolicy.Policy.Expanding, QSizePolicy.Policy.Expanding)
+ video_layout.addWidget(self.video_label, stretch=1)
+
+ # Stats panel below video with clear labels
+ stats_widget = QWidget()
+ stats_widget.setStyleSheet("padding: 5px;")
+ # stats_widget.setMinimumWidth(800) # Prevent excessive line breaks
+ stats_widget.setSizePolicy(QSizePolicy.Policy.Expanding, QSizePolicy.Policy.Fixed)
+ stats_widget.setMinimumHeight(80)
+
+ stats_layout = QGridLayout(stats_widget)
+ stats_layout.setContentsMargins(5, 5, 5, 5)
+ stats_layout.setHorizontalSpacing(8) # tighten horizontal gap between title and value
+ stats_layout.setVerticalSpacing(3)
+
+ row = 0
+
+ # Camera
+ title_camera = QLabel("Camera:")
+ title_camera.setSizePolicy(QSizePolicy.Minimum, QSizePolicy.Preferred)
+ stats_layout.addWidget(title_camera, row, 0, alignment=Qt.AlignTop)
+
+ self.camera_stats_label = QLabel("Camera idle")
+ self.camera_stats_label.setWordWrap(True)
+ self.camera_stats_label.setSizePolicy(QSizePolicy.Expanding, QSizePolicy.Preferred)
+ stats_layout.addWidget(self.camera_stats_label, row, 1, alignment=Qt.AlignTop)
+ row += 1
+
+ # DLC
+ title_dlc = QLabel("DLC Processor:")
+ title_dlc.setSizePolicy(QSizePolicy.Minimum, QSizePolicy.Preferred)
+ stats_layout.addWidget(title_dlc, row, 0, alignment=Qt.AlignTop)
+
+ self.dlc_stats_label = QLabel("DLC processor idle")
+ self.dlc_stats_label.setWordWrap(True)
+ self.dlc_stats_label.setSizePolicy(QSizePolicy.Expanding, QSizePolicy.Preferred)
+ stats_layout.addWidget(self.dlc_stats_label, row, 1, alignment=Qt.AlignTop)
+ row += 1
+
+ # Recorder
+ title_rec = QLabel("Recorder:")
+ title_rec.setSizePolicy(QSizePolicy.Minimum, QSizePolicy.Preferred)
+ stats_layout.addWidget(title_rec, row, 0, alignment=Qt.AlignTop)
+
+ self.recording_stats_label = QLabel("Recorder idle")
+ self.recording_stats_label.setWordWrap(True)
+ self.recording_stats_label.setSizePolicy(QSizePolicy.Expanding, QSizePolicy.Preferred)
+ stats_layout.addWidget(self.recording_stats_label, row, 1, alignment=Qt.AlignTop)
+
+ # Critical: make column 1 (values) eat the width, keep column 0 tight
+ stats_layout.setColumnStretch(0, 0)
+ stats_layout.setColumnStretch(1, 1)
+ video_layout.addWidget(stats_widget, stretch=0)
+
+ # Allow user to select stats text
+ for lbl in (self.camera_stats_label, self.dlc_stats_label, self.recording_stats_label):
+ lbl.setTextInteractionFlags(Qt.TextSelectableByMouse)
+
+ # Controls panel with fixed width to prevent shifting
+ controls_widget = QWidget()
+ controls_widget.setMaximumWidth(500)
+ controls_widget.setSizePolicy(QSizePolicy.Policy.Fixed, QSizePolicy.Policy.Expanding)
+ controls_layout = QVBoxLayout(controls_widget)
+ controls_layout.setContentsMargins(5, 5, 5, 5)
+ controls_layout.addWidget(self._build_camera_group())
+ controls_layout.addWidget(self._build_dlc_group())
+ controls_layout.addWidget(self._build_recording_group())
+ controls_layout.addWidget(self._build_bbox_group())
+
+ # Preview/Stop buttons at bottom of controls - wrap in widget
+ button_bar_widget = QWidget()
+ button_bar = QHBoxLayout(button_bar_widget)
+ button_bar.setContentsMargins(0, 5, 0, 5)
+ self.preview_button = QPushButton("Start Preview")
+ self.preview_button.setIcon(self.style().standardIcon(QStyle.StandardPixmap.SP_MediaPlay))
+ self.preview_button.setMinimumWidth(150)
+ self.stop_preview_button = QPushButton("Stop Preview")
+ self.stop_preview_button.setIcon(self.style().standardIcon(QStyle.StandardPixmap.SP_MediaStop))
+ self.stop_preview_button.setEnabled(False)
+ self.stop_preview_button.setMinimumWidth(150)
+ button_bar.addWidget(self.preview_button)
+ button_bar.addWidget(self.stop_preview_button)
+ controls_layout.addWidget(button_bar_widget)
+ controls_layout.addStretch(1)
+
+ # Add controls and video panel to main layout
+ layout.addWidget(controls_widget, stretch=0)
+ layout.addWidget(video_panel, stretch=1)
+ layout.setStretch(0, 0)
+ layout.setStretch(1, 1)
+
+ self.setCentralWidget(central)
+ self.setStatusBar(QStatusBar())
+ self._build_menus()
+ QTimer.singleShot(0, self._show_logo_and_text)
+
+ def _build_menus(self) -> None:
+ # File menu
+ file_menu = self.menuBar().addMenu("&File")
+
+ ## Save/Load config
+ self.load_config_action = QAction("Load configuration…", self)
+ self.load_config_action.triggered.connect(self._action_load_config)
+ file_menu.addAction(self.load_config_action)
+ save_action = QAction("Save configuration", self)
+ save_action.triggered.connect(self._action_save_config)
+ file_menu.addAction(save_action)
+ save_as_action = QAction("Save configuration as…", self)
+ save_as_action.triggered.connect(self._action_save_config_as)
+ file_menu.addAction(save_as_action)
+ ## Close
+ file_menu.addSeparator()
+ exit_action = QAction("Close window", self)
+ exit_action.triggered.connect(self.close)
+ file_menu.addAction(exit_action)
+
+ # View menu
+ view_menu = self.menuBar().addMenu("&View")
+ appearance_menu = view_menu.addMenu("Appearance")
+ ## Style actions
+ self.action_dark_mode = QAction("Dark theme", self, checkable=True)
+ self.action_light_mode = QAction("System theme", self, checkable=True)
+ theme_group = QActionGroup(self)
+ theme_group.setExclusive(True)
+ theme_group.addAction(self.action_dark_mode)
+ theme_group.addAction(self.action_light_mode)
+ self.action_dark_mode.triggered.connect(lambda: self._apply_theme(AppStyle.DARK))
+ self.action_light_mode.triggered.connect(lambda: self._apply_theme(AppStyle.SYS_DEFAULT))
+
+ appearance_menu.addAction(self.action_light_mode)
+ appearance_menu.addAction(self.action_dark_mode)
+ self._apply_theme(self._current_style)
+ self._init_theme_actions()
+
+ def _build_camera_group(self) -> QGroupBox:
+ group = QGroupBox("Camera settings")
+ form = QFormLayout(group)
+
+ # Camera config button - opens dialog for all camera configuration
+ config_layout = QHBoxLayout()
+ self.config_cameras_button = QPushButton("Configure Cameras...")
+ self.config_cameras_button.setIcon(self.style().standardIcon(QStyle.StandardPixmap.SP_ComputerIcon))
+ self.config_cameras_button.setToolTip("Configure camera settings (single or multi-camera)")
+ config_layout.addWidget(self.config_cameras_button)
+ form.addRow(config_layout)
+
+ # Active cameras display label
+ self.active_cameras_label = QLabel("No cameras configured")
+ self.active_cameras_label.setWordWrap(True)
+ form.addRow("Active:", self.active_cameras_label)
+
+ return group
+
+ def _build_dlc_group(self) -> QGroupBox:
+ group = QGroupBox("DLCLive settings")
+ form = QFormLayout(group)
+
+ path_layout = QHBoxLayout()
+ self.model_path_edit = QLineEdit()
+ self.model_path_edit.setPlaceholderText("/path/to/exported/model")
+ path_layout.addWidget(self.model_path_edit)
+ self.browse_model_button = QPushButton("Browse…")
+ self.browse_model_button.setIcon(self.style().standardIcon(QStyle.StandardPixmap.SP_DirOpenIcon))
+ self.browse_model_button.clicked.connect(self._action_browse_model)
+ path_layout.addWidget(self.browse_model_button)
+ form.addRow("Model file", path_layout)
+
+ # Processor selection
+ processor_path_layout = QHBoxLayout()
+ self.processor_folder_edit = QLineEdit()
+ self.processor_folder_edit.setText(str(Path(__file__).parent.joinpath("processors")))
+ processor_path_layout.addWidget(self.processor_folder_edit)
+
+ self.browse_processor_folder_button = QPushButton("Browse...")
+ self.browse_processor_folder_button.setIcon(self.style().standardIcon(QStyle.StandardPixmap.SP_DirOpenIcon))
+ self.browse_processor_folder_button.clicked.connect(self._action_browse_processor_folder)
+ processor_path_layout.addWidget(self.browse_processor_folder_button)
+
+ self.refresh_processors_button = QPushButton("Refresh")
+ self.refresh_processors_button.setIcon(self.style().standardIcon(QStyle.StandardPixmap.SP_BrowserReload))
+ self.refresh_processors_button.clicked.connect(self._refresh_processors)
+ processor_path_layout.addWidget(self.refresh_processors_button)
+ form.addRow("Processor folder", processor_path_layout)
+
+ self.processor_combo = QComboBox()
+ self.processor_combo.addItem("No Processor", None)
+ form.addRow("Processor", self.processor_combo)
+
+ # self.additional_options_edit = QPlainTextEdit()
+ # self.additional_options_edit.setPlaceholderText("")
+ # self.additional_options_edit.setFixedHeight(40)
+ # form.addRow("Additional options", self.additional_options_edit)
+ self.dlc_camera_combo = QComboBox()
+ self.dlc_camera_combo.setToolTip("Select which camera to use for pose inference")
+ form.addRow("Inference Camera", self.dlc_camera_combo)
+
+ # Wrap inference buttons in a widget to prevent shifting
+ inference_button_widget = QWidget()
+ inference_buttons = QHBoxLayout(inference_button_widget)
+ inference_buttons.setContentsMargins(0, 0, 0, 0)
+ self.start_inference_button = QPushButton("Start pose inference")
+ self.start_inference_button.setIcon(self.style().standardIcon(QStyle.StandardPixmap.SP_ArrowRight))
+ self.start_inference_button.setEnabled(False)
+ self.start_inference_button.setMinimumWidth(150)
+ inference_buttons.addWidget(self.start_inference_button)
+ self.stop_inference_button = QPushButton("Stop pose inference")
+ self.stop_inference_button.setIcon(self.style().standardIcon(QStyle.StandardPixmap.SP_BrowserStop))
+ self.stop_inference_button.setEnabled(False)
+ self.stop_inference_button.setMinimumWidth(150)
+ inference_buttons.addWidget(self.stop_inference_button)
+ form.addRow(inference_button_widget)
+
+ self.show_predictions_checkbox = QCheckBox("Display pose predictions")
+ self.show_predictions_checkbox.setChecked(True)
+ form.addRow(self.show_predictions_checkbox)
+
+ self.auto_record_checkbox = QCheckBox("Auto-record video on processor command")
+ self.auto_record_checkbox.setChecked(False)
+ self.auto_record_checkbox.setToolTip(
+ "Automatically start/stop video recording when processor receives video recording commands"
+ )
+ form.addRow(self.auto_record_checkbox)
+
+ self.processor_status_label = QLabel("Processor: No clients | Recording: No")
+ self.processor_status_label.setWordWrap(True)
+ form.addRow("Processor Status", self.processor_status_label)
+
+ return group
+
+ def _build_recording_group(self) -> QGroupBox:
+ group = QGroupBox("Recording")
+ form = QFormLayout(group)
+
+ dir_layout = QHBoxLayout()
+ self.output_directory_edit = QLineEdit()
+ dir_layout.addWidget(self.output_directory_edit)
+ browse_dir = QPushButton("Browse…")
+ browse_dir.setIcon(self.style().standardIcon(QStyle.StandardPixmap.SP_DirOpenIcon))
+ browse_dir.clicked.connect(self._action_browse_directory)
+ dir_layout.addWidget(browse_dir)
+ form.addRow("Output directory", dir_layout)
+
+ self.filename_edit = QLineEdit()
+ form.addRow("Filename", self.filename_edit)
+
+ self.container_combo = QComboBox()
+ self.container_combo.setEditable(True)
+ self.container_combo.addItems(["mp4", "avi", "mov"])
+ form.addRow("Container", self.container_combo)
+
+ self.codec_combo = QComboBox()
+ if os.sys.platform == "darwin":
+ self.codec_combo.addItems(["h264_videotoolbox", "libx264", "hevc_videotoolbox"])
+ else:
+ self.codec_combo.addItems(["h264_nvenc", "libx264", "hevc_nvenc"])
+ self.codec_combo.setCurrentText("libx264")
+ form.addRow("Codec", self.codec_combo)
+
+ self.crf_spin = QSpinBox()
+ self.crf_spin.setRange(0, 51)
+ self.crf_spin.setValue(23)
+ form.addRow("CRF", self.crf_spin)
+
+ # Wrap recording buttons in a widget to prevent shifting
+ recording_button_widget = QWidget()
+ buttons = QHBoxLayout(recording_button_widget)
+ buttons.setContentsMargins(0, 0, 0, 0)
+ self.start_record_button = QPushButton("Start recording")
+ self.start_record_button.setIcon(self.style().standardIcon(QStyle.StandardPixmap.SP_DialogYesButton))
+ self.start_record_button.setMinimumWidth(150)
+ buttons.addWidget(self.start_record_button)
+ self.stop_record_button = QPushButton("Stop recording")
+ self.stop_record_button.setIcon(self.style().standardIcon(QStyle.StandardPixmap.SP_DialogNoButton))
+ self.stop_record_button.setEnabled(False)
+ self.stop_record_button.setMinimumWidth(150)
+ buttons.addWidget(self.stop_record_button)
+ form.addRow(recording_button_widget)
+
+ return group
+
+ def _build_bbox_group(self) -> QGroupBox:
+ """Build bounding box visualization controls."""
+ group = QGroupBox("Bounding Box Visualization")
+ form = QFormLayout(group)
+
+ self.bbox_enabled_checkbox = QCheckBox("Show bounding box")
+ self.bbox_enabled_checkbox.setChecked(False)
+ form.addRow(self.bbox_enabled_checkbox)
+
+ bbox_layout = QHBoxLayout()
+
+ self.bbox_x0_spin = QSpinBox()
+ self.bbox_x0_spin.setRange(0, 7680)
+ self.bbox_x0_spin.setPrefix("x0:")
+ self.bbox_x0_spin.setValue(0)
+ bbox_layout.addWidget(self.bbox_x0_spin)
+
+ self.bbox_y0_spin = QSpinBox()
+ self.bbox_y0_spin.setRange(0, 4320)
+ self.bbox_y0_spin.setPrefix("y0:")
+ self.bbox_y0_spin.setValue(0)
+ bbox_layout.addWidget(self.bbox_y0_spin)
+
+ self.bbox_x1_spin = QSpinBox()
+ self.bbox_x1_spin.setRange(0, 7680)
+ self.bbox_x1_spin.setPrefix("x1:")
+ self.bbox_x1_spin.setValue(100)
+ bbox_layout.addWidget(self.bbox_x1_spin)
+
+ self.bbox_y1_spin = QSpinBox()
+ self.bbox_y1_spin.setRange(0, 4320)
+ self.bbox_y1_spin.setPrefix("y1:")
+ self.bbox_y1_spin.setValue(100)
+ bbox_layout.addWidget(self.bbox_y1_spin)
+
+ form.addRow("Coordinates", bbox_layout)
+
+ return group
+
+ # ------------------------------------------------------------------ signals
+ def _connect_signals(self) -> None:
+ self.preview_button.clicked.connect(self._start_preview)
+ self.stop_preview_button.clicked.connect(self._stop_preview)
+ self.start_record_button.clicked.connect(self._start_recording)
+ self.stop_record_button.clicked.connect(self._stop_recording)
+ self.start_inference_button.clicked.connect(self._start_inference)
+ self.stop_inference_button.clicked.connect(lambda: self._stop_inference())
+ self.show_predictions_checkbox.stateChanged.connect(self._on_show_predictions_changed)
+
+ # Camera config dialog
+ self.config_cameras_button.clicked.connect(self._open_camera_config_dialog)
+
+ # Connect bounding box controls
+ self.bbox_enabled_checkbox.stateChanged.connect(self._on_bbox_changed)
+ self.bbox_x0_spin.valueChanged.connect(self._on_bbox_changed)
+ self.bbox_y0_spin.valueChanged.connect(self._on_bbox_changed)
+ self.bbox_x1_spin.valueChanged.connect(self._on_bbox_changed)
+ self.bbox_y1_spin.valueChanged.connect(self._on_bbox_changed)
+
+ # Multi-camera controller signals (used for both single and multi-camera modes)
+ self.multi_camera_controller.frame_ready.connect(self._on_multi_frame_ready)
+ self.multi_camera_controller.all_started.connect(self._on_multi_camera_started)
+ self.multi_camera_controller.all_stopped.connect(self._on_multi_camera_stopped)
+ self.multi_camera_controller.camera_error.connect(self._on_multi_camera_error)
+ self.multi_camera_controller.initialization_failed.connect(self._on_multi_camera_initialization_failed)
+
+ self._dlc.pose_ready.connect(self._on_pose_ready)
+ self._dlc.error.connect(self._on_dlc_error)
+ self._dlc.initialized.connect(self._on_dlc_initialised)
+ self.dlc_camera_combo.currentIndexChanged.connect(self._on_dlc_camera_changed)
+
+ # ------------------------------------------------------------------ config
+ def _apply_config(self, config: ApplicationSettingsModel) -> None:
+ # Update active cameras label
+ self._update_active_cameras_label()
+
+ dlc = config.dlc
+ resolved_model_path = self._model_path_store.resolve(dlc.model_path)
+ self.model_path_edit.setText(resolved_model_path)
+
+ # self.additional_options_edit.setPlainText(json.dumps(dlc.additional_options, indent=2))
+
+ recording = config.recording
+ self.output_directory_edit.setText(recording.directory)
+ self.filename_edit.setText(recording.filename)
+ self.container_combo.setCurrentText(recording.container)
+ codec_index = self.codec_combo.findText(recording.codec)
+ if codec_index >= 0:
+ self.codec_combo.setCurrentIndex(codec_index)
+ else:
+ self.codec_combo.addItem(recording.codec)
+ self.codec_combo.setCurrentIndex(self.codec_combo.count() - 1)
+ self.crf_spin.setValue(int(recording.crf))
+
+ # Set bounding box settings from config
+ bbox = config.bbox
+ self.bbox_enabled_checkbox.setChecked(bbox.enabled)
+ self.bbox_x0_spin.setValue(bbox.x0)
+ self.bbox_y0_spin.setValue(bbox.y0)
+ self.bbox_x1_spin.setValue(bbox.x1)
+ self.bbox_y1_spin.setValue(bbox.y1)
+
+ # Set visualization settings from config
+ viz = config.visualization
+ self._p_cutoff = viz.p_cutoff
+ self._colormap = viz.colormap
+ self._bbox_color = viz.get_bbox_color_bgr()
+ # Update DLC camera list
+ self._refresh_dlc_camera_list()
+
+ def _current_config(self) -> ApplicationSettingsModel:
+ # Get the first camera from multi-camera config for backward compatibility
+ active_cameras = self._config.multi_camera.get_active_cameras()
+ camera = active_cameras[0] if active_cameras else CameraSettingsModel()
+
+ return ApplicationSettingsModel(
+ camera=camera,
+ multi_camera=self._config.multi_camera,
+ dlc=self._dlc_settings_from_ui(),
+ recording=self._recording_settings_from_ui(),
+ bbox=self._bbox_settings_from_ui(),
+ visualization=self._visualization_settings_from_ui(),
+ )
+
+ def _parse_json(self, value: str) -> dict:
+ text = value.strip()
+ if not text:
+ return {}
+ return json.loads(text)
+
+ def _dlc_settings_from_ui(self) -> DLCProcessorSettingsModel:
+ return DLCProcessorSettingsModel(
+ model_path=self.model_path_edit.text().strip(),
+ model_directory=self._config.dlc.model_directory, # Preserve from config
+ device=self._config.dlc.device, # Preserve from config
+ dynamic=self._config.dlc.dynamic, # Preserve from config
+ resize=self._config.dlc.resize, # Preserve from config
+ precision=self._config.dlc.precision, # Preserve from config
+ model_type="pytorch", # FIXME @C-Achard hardcoded for now, we should allow tf models too
+ # additional_options=self._parse_json(self.additional_options_edit.toPlainText()),
+ )
+
+ def _recording_settings_from_ui(self) -> RecordingSettingsModel:
+ return RecordingSettingsModel(
+ enabled=True, # Always enabled - recording controlled by button
+ directory=self.output_directory_edit.text().strip(),
+ filename=self.filename_edit.text().strip() or "session.mp4",
+ container=self.container_combo.currentText().strip() or "mp4",
+ codec=self.codec_combo.currentText().strip() or "libx264",
+ crf=int(self.crf_spin.value()),
+ )
+
+ def _bbox_settings_from_ui(self) -> BoundingBoxSettingsModel:
+ return BoundingBoxSettingsModel(
+ enabled=self.bbox_enabled_checkbox.isChecked(),
+ x0=self.bbox_x0_spin.value(),
+ y0=self.bbox_y0_spin.value(),
+ x1=self.bbox_x1_spin.value(),
+ y1=self.bbox_y1_spin.value(),
+ )
+
+ def _visualization_settings_from_ui(self) -> VisualizationSettingsModel:
+ return VisualizationSettingsModel(
+ p_cutoff=self._p_cutoff,
+ colormap=self._colormap,
+ bbox_color=self._bbox_color,
+ )
+
+ # ------------------------------------------------------------------ actions
+ def _action_load_config(self) -> None:
+ file_name, _ = QFileDialog.getOpenFileName(self, "Load configuration", str(Path.home()), "JSON files (*.json)")
+ if not file_name:
+ return
+ try:
+ config = ApplicationSettingsModel.load(file_name)
+ except Exception as exc: # pragma: no cover - GUI interaction
+ self._show_error(str(exc))
+ return
+ self._config = config
+ self._config_path = Path(file_name)
+ self._apply_config(config)
+ self.statusBar().showMessage(f"Loaded configuration: {file_name}", 5000)
+ # Validate cameras after loading
+ self._validate_configured_cameras()
+
+ def _action_save_config(self) -> None:
+ if self._config_path is None:
+ self._action_save_config_as()
+ return
+ self._save_config_to_path(self._config_path)
+
+ def _action_save_config_as(self) -> None:
+ file_name, _ = QFileDialog.getSaveFileName(self, "Save configuration", str(Path.home()), "JSON files (*.json)")
+ if not file_name:
+ return
+ path = Path(file_name)
+ if path.suffix.lower() != ".json":
+ path = path.with_suffix(".json")
+ self._config_path = path
+ self._save_config_to_path(path)
+
+ def _save_config_to_path(self, path: Path) -> None:
+ try:
+ config = self._current_config()
+ config.save(path)
+ except Exception as exc: # pragma: no cover - GUI interaction
+ self._show_error(str(exc))
+ return
+ self.statusBar().showMessage(f"Saved configuration to {path}", 5000)
+
+ def _action_browse_model(self) -> None:
+ # Use model_directory from config, default to current directory
+ start_dir = self._config.dlc.model_directory or "."
+ file_path, _ = QFileDialog.getOpenFileName(
+ self,
+ "Select DLCLive model file",
+ start_dir,
+ "Model files (*.pt *.pth *.pb);;All files (*.*)",
+ )
+ if file_path:
+ self.model_path_edit.setText(file_path)
+ self._model_path_store.save_if_valid(file_path)
+
+ def _action_browse_directory(self) -> None:
+ directory = QFileDialog.getExistingDirectory(self, "Select output directory", str(Path.home()))
+ if directory:
+ self.output_directory_edit.setText(directory)
+
+ def _action_browse_processor_folder(self) -> None:
+ """Browse for processor folder."""
+ current_path = self.processor_folder_edit.text() or "./processors"
+ directory = QFileDialog.getExistingDirectory(self, "Select processor folder", current_path)
+ if directory:
+ self.processor_folder_edit.setText(directory)
+ self._refresh_processors()
+
+ def _refresh_processors(self) -> None:
+ """Scan processor folder and populate dropdown."""
+ folder_path = self.processor_folder_edit.text() or "./processors"
+
+ # Clear existing items (keep "No Processor")
+ self.processor_combo.clear()
+ self.processor_combo.addItem("No Processor", None)
+
+ # Scan folder
+ try:
+ self._scanned_processors = scan_processor_folder(folder_path)
+ self._processor_keys = list(self._scanned_processors.keys())
+
+ # Populate dropdown
+ for key in self._processor_keys:
+ info = self._scanned_processors[key]
+ display_name = f"{info['name']} ({info['file']})"
+ self.processor_combo.addItem(display_name, key)
+
+ status_msg = f"Found {len(self._processor_keys)} processor(s) in {folder_path}"
+ self.statusBar().showMessage(status_msg, 3000)
+
+ except Exception as e:
+ error_msg = f"Error scanning processors: {e}"
+ self.statusBar().showMessage(error_msg, 5000)
+ logger.error(error_msg)
+ self._scanned_processors = {}
+ self._processor_keys = []
+
+ # ------------------------------------------------------------------ multi-camera
+ def _open_camera_config_dialog(self) -> None:
+ """Open the camera configuration dialog (non-modal, async inside)."""
+ if self.multi_camera_controller.is_running():
+ self._show_warning("Stop the main preview before configuring cameras.")
+ return
+
+ if self._cam_dialog is None:
+ self._cam_dialog = CameraConfigDialog(self, self._config.multi_camera)
+ self._cam_dialog.settings_changed.connect(self._on_multi_camera_settings_changed)
+ else:
+ # Refresh its UI from current settings when reopened
+ self._cam_dialog._populate_from_settings()
+ self._cam_dialog.dlc_camera_id = self._inference_camera_id
+
+ self._cam_dialog.show()
+ self._cam_dialog.raise_()
+ self._cam_dialog.activateWindow()
+
+ def _on_multi_camera_settings_changed(self, settings: MultiCameraSettingsModel) -> None:
+ """Handle changes to multi-camera settings."""
+ self._config.multi_camera = settings
+ self._update_active_cameras_label()
+ self._refresh_dlc_camera_list()
+ active_count = len(settings.get_active_cameras())
+ self.statusBar().showMessage(f"Camera configuration updated: {active_count} active camera(s)", 3000)
+
+ def _update_active_cameras_label(self) -> None:
+ """Update the label showing active cameras."""
+ active_cams = self._config.multi_camera.get_active_cameras()
+ if not active_cams:
+ self.active_cameras_label.setText("No cameras configured")
+ elif len(active_cams) == 1:
+ cam = active_cams[0]
+ self.active_cameras_label.setText(f"{cam.name} [{cam.backend}:{cam.index}] @ {cam.fps:.1f} fps")
+ else:
+ cam_names = [f"{c.name}" for c in active_cams]
+ self.active_cameras_label.setText(f"{len(active_cams)} cameras: {', '.join(cam_names)}")
+
+ def _validate_configured_cameras(self) -> None:
+ """Validate that configured cameras are available.
+
+ Disables unavailable cameras and shows a warning dialog.
+ """
+ if getattr(self._cam_dialog, "_dialog_active", False):
+ # Skip validation if camera config dialog is open
+ return
+
+ active_cams = self._config.multi_camera.get_active_cameras()
+ if not active_cams:
+ return
+
+ unavailable: list[tuple[str, str, CameraSettingsModel]] = []
+ for cam in active_cams:
+ cam_id = f"{cam.backend}:{cam.index}"
+ available, error = CameraFactory.check_camera_available(cam)
+ if not available:
+ unavailable.append((cam.name or cam_id, error, cam))
+
+ if unavailable:
+ # Disable unavailable cameras
+ for _, _, cam in unavailable:
+ cam.enabled = False
+
+ # Update the active cameras label
+ self._update_active_cameras_label()
+
+ # Build warning message
+ error_lines = ["The following camera(s) are not available and have been disabled:"]
+ for cam_name, error_msg, _ in unavailable:
+ error_lines.append(f" • {cam_name}: {error_msg}")
+ error_lines.append("")
+ error_lines.append("Please check camera connections or re-enable in camera settings.")
+ self._show_warning("\n".join(error_lines))
+ logger.warning("\n".join(error_lines))
+
+ def _label_for_cam_id(self, cam_id: str) -> str:
+ for cam in self._config.multi_camera.get_active_cameras():
+ if get_camera_id(cam) == cam_id:
+ return f"{cam.name} [{cam.backend}:{cam.index}]"
+ return cam_id
+
+ def _refresh_dlc_camera_list_running(self) -> None:
+ """Populate the inference camera dropdown from currently running cameras."""
+ self.dlc_camera_combo.blockSignals(True)
+ self.dlc_camera_combo.clear()
+ for cam_id in sorted(self._running_cams_ids):
+ self.dlc_camera_combo.addItem(self._label_for_cam_id(cam_id), cam_id)
+
+ # Keep current selection if still present, else select first running
+ if self._inference_camera_id in self._running_cams_ids:
+ idx = self.dlc_camera_combo.findData(self._inference_camera_id)
+ if idx >= 0:
+ self.dlc_camera_combo.setCurrentIndex(idx)
+ elif self.dlc_camera_combo.count() > 0:
+ self.dlc_camera_combo.setCurrentIndex(0)
+ self._inference_camera_id = self.dlc_camera_combo.currentData()
+ self.dlc_camera_combo.blockSignals(False)
+
+ def _set_dlc_combo_to_id(self, cam_id: str) -> None:
+ """Update combo selection to a given ID without firing signals."""
+ self.dlc_camera_combo.blockSignals(True)
+ idx = self.dlc_camera_combo.findData(cam_id)
+ if idx >= 0:
+ self.dlc_camera_combo.setCurrentIndex(idx)
+ self.dlc_camera_combo.blockSignals(False)
+
+ def _refresh_dlc_camera_list(self) -> None:
+ """Populate the inference camera dropdown from active cameras."""
+ self.dlc_camera_combo.blockSignals(True)
+ self.dlc_camera_combo.clear()
+
+ active_cams = self._config.multi_camera.get_active_cameras()
+ for cam in active_cams:
+ cam_id = get_camera_id(cam) # e.g., "opencv:0" or "pylon:1"
+ label = f"{cam.name} [{cam.backend}:{cam.index}]"
+ self.dlc_camera_combo.addItem(label, cam_id)
+
+ # Keep previous selection if still present, else default to first
+ if self._inference_camera_id is not None:
+ idx = self.dlc_camera_combo.findData(self._inference_camera_id)
+ if idx >= 0:
+ self.dlc_camera_combo.setCurrentIndex(idx)
+ elif self.dlc_camera_combo.count() > 0:
+ self.dlc_camera_combo.setCurrentIndex(0)
+ self._inference_camera_id = self.dlc_camera_combo.currentData()
+ else:
+ if self.dlc_camera_combo.count() > 0:
+ self.dlc_camera_combo.setCurrentIndex(0)
+ self._inference_camera_id = self.dlc_camera_combo.currentData()
+
+ self.dlc_camera_combo.blockSignals(False)
+
+ def _on_dlc_camera_changed(self, _index: int) -> None:
+ """Track user selection of the inference camera."""
+ self._inference_camera_id = self.dlc_camera_combo.currentData()
+ # Force redraw so bbox/pose overlays switch to the new tile immediately
+ if self._current_frame is not None:
+ self._display_frame(self._current_frame, force=True)
+
+ def _on_multi_frame_ready(self, frame_data: MultiFrameData) -> None:
+ """Handle frames from multiple cameras.
+
+ Priority order for performance:
+ 1. DLC processing (highest priority - enqueue immediately, only for DLC camera)
+ 2. Recording (queued writes, non-blocking)
+ 3. Display (lowest priority - tiled and updated on separate timer)
+ """
+ self._multi_camera_frames = frame_data.frames
+ src_id = frame_data.source_camera_id
+ if src_id:
+ self._fps_tracker.note_frame(src_id) # Track FPS
+
+ new_running = set(frame_data.frames.keys())
+ if new_running != self._running_cams_ids:
+ self._running_cams_ids = new_running
+ self._refresh_dlc_camera_list_running()
+
+ # Determine DLC camera (first active camera)
+ selected_id = self._inference_camera_id
+ available_ids = sorted(frame_data.frames.keys())
+ if selected_id in frame_data.frames:
+ dlc_cam_id = selected_id
+ else:
+ dlc_cam_id = available_ids[0] if available_ids else ""
+ if dlc_cam_id is not None:
+ self._inference_camera_id = dlc_cam_id
+ self._set_dlc_combo_to_id(dlc_cam_id)
+ self.statusBar().showMessage(
+ f"DLC inference camera changed to {self._label_for_cam_id(dlc_cam_id)}", 3000
+ )
+ else: # No more cameras available
+ if self._dlc_active:
+ self._stop_inference(show_message=True)
+ self._display_dirty = True
+ return
+
+ # Check if this frame is from the DLC camera
+ is_dlc_camera_frame = frame_data.source_camera_id == dlc_cam_id
+
+ # Update tile info and raw frame only when DLC camera frame arrives
+ if is_dlc_camera_frame and dlc_cam_id in frame_data.frames:
+ frame = frame_data.frames[dlc_cam_id]
+ self._raw_frame = frame
+ self._dlc_tile_offset, self._dlc_tile_size = compute_tile_info(dlc_cam_id, frame, frame_data.frames)
+
+ # PRIORITY 1: DLC processing - only enqueue when DLC camera frame arrives!
+ if self._dlc_active and is_dlc_camera_frame and dlc_cam_id in frame_data.frames:
+ frame = frame_data.frames[dlc_cam_id]
+ timestamp = frame_data.timestamps.get(dlc_cam_id, time.time())
+ self._dlc.enqueue_frame(frame, timestamp)
+
+ # PRIORITY 2: Recording (queued, non-blocking)
+ if self._rec_manager.is_active and src_id in frame_data.frames:
+ frame = frame_data.frames[src_id]
+ ts = frame_data.timestamps.get(src_id, time.time())
+ self._rec_manager.write_frame(src_id, frame, ts)
+
+ # PRIORITY 3: Mark display dirty (tiling done in display timer)
+ self._display_dirty = True
+
+ def _on_multi_camera_started(self) -> None:
+ """Handle all cameras started event."""
+ self.preview_button.setEnabled(False)
+ self.stop_preview_button.setEnabled(True)
+ active_count = self.multi_camera_controller.get_active_count()
+ self.statusBar().showMessage(f"Multi-camera preview started: {active_count} camera(s)", 5000)
+ self._update_inference_buttons()
+ self._update_camera_controls_enabled()
+
+ def _on_multi_camera_stopped(self) -> None:
+ """Handle all cameras stopped event."""
+ # Stop all multi-camera recorders
+ self._stop_multi_camera_recording()
+
+ self.preview_button.setEnabled(True)
+ self.stop_preview_button.setEnabled(False)
+ self._current_frame = None
+ self._multi_camera_frames.clear()
+ self.video_label.setPixmap(QPixmap())
+ self.video_label.setText("Camera preview not started")
+ self.statusBar().showMessage("Multi-camera preview stopped", 3000)
+ self._update_inference_buttons()
+ self._update_camera_controls_enabled()
+
+ def _on_multi_camera_error(self, camera_id: str, message: str) -> None:
+ """Handle error from a camera in multi-camera mode."""
+ self._show_warning(f"Camera {camera_id} error: {message}\nRecording stopped.")
+ self._refresh_dlc_camera_list_running()
+ if self.dlc_camera_combo.count() <= 1:
+ self._stop_inference() # We now gracefully switch DLC camera if needed, but if none left, stop inference
+ self._stop_recording()
+
+ def _on_multi_camera_initialization_failed(self, failures: list) -> None:
+ """Handle complete failure to initialize cameras."""
+ # Build error message with details for each failed camera
+ error_lines = ["Failed to initialize camera(s):"]
+ for camera_id, error_msg in failures:
+ error_lines.append(f" • {camera_id}: {error_msg}")
+ error_lines.append("")
+ error_lines.append("Please check that the required camera backend is installed.")
+
+ error_message = "\n".join(error_lines)
+ self._show_error(error_message)
+ logger.error(error_message)
+
+ def _start_multi_camera_recording(self) -> None:
+ """Start recording from all active cameras."""
+ recording = self._recording_settings_from_ui()
+ active_cams = self._config.multi_camera.get_active_cameras()
+ self._rec_manager.start_all(recording, active_cams, self._multi_camera_frames)
+
+ if self._rec_manager.is_active:
+ self.start_record_button.setEnabled(False)
+ self.stop_record_button.setEnabled(True)
+ self.statusBar().showMessage(f"Recording {len(active_cams)} camera(s) to {recording.directory}", 5000)
+ self._update_camera_controls_enabled()
+
+ def _stop_multi_camera_recording(self) -> None:
+ if not self._rec_manager.is_active:
+ return
+ self._rec_manager.stop_all()
+ self.start_record_button.setEnabled(True)
+ self.stop_record_button.setEnabled(False)
+ self.statusBar().showMessage("Multi-camera recording stopped", 3000)
+ self._update_camera_controls_enabled()
+
+ # ------------------------------------------------------------------ camera control
+ def _show_logo_and_text(self):
+ """Show the transparent logo with text below it in the preview area when not running."""
+ from PySide6.QtCore import QRect
+ from PySide6.QtGui import QColor
+
+ size = self.video_label.size()
+
+ if size.width() <= 0 or size.height() <= 0:
+ return
+
+ # Prepare blank canvas (transparent)
+ composed = QPixmap(size)
+ composed.fill(Qt.transparent)
+
+ painter = QPainter(composed)
+ painter.setRenderHint(QPainter.SmoothPixmapTransform)
+ painter.setRenderHint(QPainter.Antialiasing)
+
+ # --- Scale logo to at most 50% height (nice proportion) ---
+ max_logo_height = int(size.height() * 0.45)
+ logo = self._preview_pixmap.scaledToHeight(max_logo_height, Qt.SmoothTransformation)
+
+ # Center the logo horizontally
+ logo_x = (size.width() - logo.width()) // 2
+ logo_y = int(size.height() * 0.15) # small top margin
+
+ painter.drawPixmap(logo_x, logo_y, logo)
+
+ # --- Draw text BELOW the logo ---
+ painter.setPen(QColor(255, 255, 255))
+ painter.setFont(QFont("Arial", 22, QFont.Bold))
+
+ text = "DeepLabCut-Live! "
+ try:
+ version = importlib.metadata.version("dlclivegui")
+ except Exception:
+ version = ""
+ if version:
+ text += f"\n(v{version})"
+
+ # Position text under the logo with a small gap
+ text_rect = QRect(
+ 0,
+ logo_y + logo.height() + 15, # 15px gap under logo
+ size.width(),
+ size.height() - (logo_y + logo.height() + 15),
+ )
+
+ painter.drawText(text_rect, Qt.AlignHCenter | Qt.AlignTop, text)
+
+ painter.end()
+ self.video_label.setPixmap(composed)
+
+ def _start_preview(self) -> None:
+ """Start camera preview - uses multi-camera controller for all configurations."""
+ active_cams = self._config.multi_camera.get_active_cameras()
+ if not active_cams:
+ self._show_error("No cameras configured. Use 'Configure Cameras...' to add cameras.")
+ return
+
+ # Determine if we're in single or multi-camera mode
+ self._multi_camera_mode = len(active_cams) > 1
+
+ self.preview_button.setEnabled(False)
+ self.stop_preview_button.setEnabled(True)
+ self._current_frame = None
+ self._raw_frame = None
+ self._last_pose = None
+ self._multi_camera_frames.clear()
+ self._fps_tracker.clear()
+ self._last_display_time = 0.0
+
+ if hasattr(self, "camera_stats_label"):
+ self.camera_stats_label.setText(f"Starting {len(active_cams)} camera(s)…")
+ self.statusBar().showMessage(f"Starting preview ({len(active_cams)} camera(s))…", 3000)
+
+ # Store active settings for single camera mode (for DLC, recording frame rate, etc.)
+ self._active_camera_settings = active_cams[0] if active_cams else None
+ for cam in active_cams:
+ cam.properties.setdefault("fast_start", True)
+
+ self.multi_camera_controller.start(active_cams)
+ self._update_inference_buttons()
+ self._update_camera_controls_enabled()
+
+ def _stop_preview(self) -> None:
+ """Stop camera preview."""
+ if not self.multi_camera_controller.is_running():
+ return
+
+ self.preview_button.setEnabled(False)
+ self.stop_preview_button.setEnabled(False)
+ self.start_inference_button.setEnabled(False)
+ self.stop_inference_button.setEnabled(False)
+ self.statusBar().showMessage("Stopping preview…", 3000)
+
+ # Stop any active recording first
+ self._stop_multi_camera_recording()
+
+ self.multi_camera_controller.stop()
+ self._stop_inference(show_message=False)
+ self._fps_tracker.clear()
+ self._last_display_time = 0.0
+ if hasattr(self, "camera_stats_label"):
+ self.camera_stats_label.setText("Camera idle")
+ # self._show_logo_and_text()
+
+ def _configure_dlc(self) -> bool:
+ try:
+ settings = self._dlc_settings_from_ui()
+ except (ValueError, json.JSONDecodeError) as exc:
+ self._show_error(f"Invalid DLCLive settings: {exc}")
+ return False
+ if not settings.model_path:
+ self._show_error("Please select a DLCLive model before starting inference.")
+ return False
+
+ # Instantiate processor if selected
+ processor = None
+ selected_key = self.processor_combo.currentData()
+ if selected_key is not None and self._scanned_processors:
+ try:
+ # For now, instantiate with no parameters
+ # TODO: Add parameter dialog for processors that need params
+ # or pass kwargs from config ?
+ processor = instantiate_from_scan(self._scanned_processors, selected_key)
+ processor_name = self._scanned_processors[selected_key]["name"]
+ self.statusBar().showMessage(f"Loaded processor: {processor_name}", 3000)
+ except Exception as e:
+ error_msg = f"Failed to instantiate processor: {e}"
+ self._show_error(error_msg)
+ logger.error(error_msg)
+ return False
+
+ self._dlc.configure(settings, processor=processor)
+ self._model_path_store.save_if_valid(settings.model_path)
+ return True
+
+ def _update_inference_buttons(self) -> None:
+ preview_running = self.multi_camera_controller.is_running()
+ self.start_inference_button.setEnabled(preview_running and not self._dlc_active)
+ self.stop_inference_button.setEnabled(preview_running and self._dlc_active)
+
+ def _update_dlc_controls_enabled(self) -> None:
+ """Enable/disable DLC settings based on inference state."""
+ allow_changes = not self._dlc_active
+ widgets = [
+ self.model_path_edit,
+ self.browse_model_button,
+ self.processor_folder_edit,
+ self.browse_processor_folder_button,
+ self.refresh_processors_button,
+ self.processor_combo,
+ # self.additional_options_edit,
+ self.dlc_camera_combo,
+ ]
+ for widget in widgets:
+ widget.setEnabled(allow_changes)
+
+ def _update_camera_controls_enabled(self) -> None:
+ multi_cam_recording = self._rec_manager.is_active
+
+ # Check if preview is running
+ preview_running = self.multi_camera_controller.is_running()
+
+ allow_changes = not preview_running and not self._dlc_active and not multi_cam_recording
+
+ # Recording settings (codec, crf) should be editable when not recording
+ recording_editable = not multi_cam_recording
+ self.codec_combo.setEnabled(recording_editable)
+ self.crf_spin.setEnabled(recording_editable)
+
+ # Config cameras button should be available when not in preview/recording
+ self.config_cameras_button.setEnabled(allow_changes)
+
+ # Disable loading configurations when preview/recording is active
+ if hasattr(self, "load_config_action"):
+ self.load_config_action.setEnabled(allow_changes)
+
+ def _display_frame(self, frame: np.ndarray, *, force: bool = False) -> None:
+ if frame is None:
+ return
+ now = time.perf_counter()
+ if not force and (now - self._last_display_time) < self._display_interval:
+ return
+ self._last_display_time = now
+ self._update_video_display(frame)
+
+ def _update_display_from_pending(self) -> None:
+ """Update display from pending frames (called by display timer)."""
+ if not self._display_dirty:
+ return
+ if not self._multi_camera_frames:
+ return
+
+ self._display_dirty = False
+
+ # Create tiled frame on demand (moved from camera thread for performance)
+ tiled = create_tiled_frame(self._multi_camera_frames)
+ if tiled is not None:
+ self._current_frame = tiled
+ self._update_video_display(tiled)
+
+ def _format_recorder_stats(self, stats: RecorderStats) -> str:
+ latency_ms = stats.last_latency * 1000.0
+ avg_ms = stats.average_latency * 1000.0
+ buffer_ms = stats.buffer_seconds * 1000.0
+ write_fps = stats.write_fps
+ enqueue = stats.frames_enqueued
+ written = stats.frames_written
+ dropped = stats.dropped_frames
+ return (
+ f"{written}/{enqueue} frames | write {write_fps:.1f} fps | "
+ f"latency {latency_ms:.1f} ms (avg {avg_ms:.1f} ms) | "
+ f"queue {stats.queue_size} (~{buffer_ms:.0f} ms) | dropped {dropped}"
+ )
+
+ def _format_dlc_stats(self, stats: ProcessorStats) -> str:
+ """Format DLC processor statistics for display."""
+ latency_ms = stats.last_latency * 1000.0
+ avg_ms = stats.average_latency * 1000.0
+ processing_fps = stats.processing_fps
+ enqueue = stats.frames_enqueued
+ processed = stats.frames_processed
+ dropped = stats.frames_dropped
+
+ # Add profiling info if available
+ profile_info = ""
+ if stats.avg_inference_time > 0:
+ inf_ms = stats.avg_inference_time * 1000.0
+ queue_ms = stats.avg_queue_wait * 1000.0
+ signal_ms = stats.avg_signal_emit_time * 1000.0
+ total_ms = stats.avg_total_process_time * 1000.0
+
+ # Add GPU vs processor breakdown if available
+ gpu_breakdown = ""
+ if stats.avg_gpu_inference_time > 0 or stats.avg_processor_overhead > 0:
+ gpu_ms = stats.avg_gpu_inference_time * 1000.0
+ proc_ms = stats.avg_processor_overhead * 1000.0
+ gpu_breakdown = f" (GPU:{gpu_ms:.1f}ms+proc:{proc_ms:.1f}ms)"
+
+ profile_info = (
+ f"\n[Profile] inf:{inf_ms:.1f}ms{gpu_breakdown} queue:{queue_ms:.1f}ms "
+ f"signal:{signal_ms:.1f}ms total:{total_ms:.1f}ms"
+ )
+
+ return (
+ f"{processed}/{enqueue} frames | inference {processing_fps:.1f} fps | "
+ f"latency {latency_ms:.1f} ms (avg {avg_ms:.1f} ms) | "
+ f"queue {stats.queue_size} | dropped {dropped}{profile_info}"
+ )
+
+ def _update_metrics(self) -> None:
+ # --- Camera stats ---
+ if hasattr(self, "camera_stats_label"):
+ running = self.multi_camera_controller.is_running()
+ if running:
+ active_count = self.multi_camera_controller.get_active_count()
+
+ # Build per-camera FPS list for active cameras only
+ active_cams = self._config.multi_camera.get_active_cameras()
+ lines = []
+ for cam in active_cams:
+ cam_id = get_camera_id(cam) # e.g., "opencv:0" or "pylon:1"
+ fps = self._fps_tracker.fps(cam_id)
+ # Make a compact label: name [backend:index] @ fps
+ label = f"{cam.name or cam_id} [{cam.backend}:{cam.index}]"
+ if fps > 0:
+ lines.append(f"{label} @ {fps:.1f} fps")
+ else:
+ lines.append(f"{label} @ Measuring…")
+
+ if active_count == 1:
+ # Single camera: show just the line
+ summary = lines[0] if lines else "Measuring…"
+ else:
+ # Multi camera: join lines with separator
+ summary = " | ".join(lines)
+
+ self.camera_stats_label.setText(summary)
+ else:
+ self.camera_stats_label.setText("Camera idle")
+
+ # --- DLC processor stats ---
+ if hasattr(self, "dlc_stats_label"):
+ if self._dlc_active and self._dlc_initialized:
+ stats = self._dlc.get_stats()
+ summary = self._format_dlc_stats(stats)
+ self.dlc_stats_label.setText(summary)
+ else:
+ self.dlc_stats_label.setText("DLC processor idle")
+
+ # Update processor status (connection and recording state)
+ if hasattr(self, "processor_status_label"):
+ self._update_processor_status()
+
+ # --- Recorder stats ---
+ if hasattr(self, "recording_stats_label"):
+ if self._rec_manager.is_active:
+ summary = self._rec_manager.get_stats_summary()
+ self._last_recorder_summary = summary
+ self.recording_stats_label.setText(summary)
+ else:
+ self.recording_stats_label.setText(self._last_recorder_summary)
+
+ def _update_processor_status(self) -> None:
+ """Update processor connection and recording status, handle auto-recording."""
+ if not self._dlc_active or not self._dlc_initialized:
+ self.processor_status_label.setText("Processor: Not active")
+ return
+
+ # Get processor instance from _dlc
+ processor = self._dlc._processor
+
+ if processor is None:
+ self.processor_status_label.setText("Processor: None loaded")
+ return
+
+ # Check if processor has the required attributes (socket-based processors)
+ if not hasattr(processor, "conns") or not hasattr(processor, "_recording"):
+ self.processor_status_label.setText("Processor: No status info")
+ return
+
+ # Get connection count and recording state
+ num_clients = len(processor.conns)
+ is_recording = processor.recording if hasattr(processor, "recording") else False
+
+ # Format status message
+ client_str = f"{num_clients} client{'s' if num_clients != 1 else ''}"
+ recording_str = "Yes" if is_recording else "No"
+ self.processor_status_label.setText(f"Clients: {client_str} | Recording: {recording_str}")
+
+ # Handle auto-recording based on processor's video recording flag
+ if hasattr(processor, "_vid_recording") and self.auto_record_checkbox.isChecked():
+ current_vid_recording = processor.video_recording
+
+ # Check if video recording state changed
+ if current_vid_recording != self._last_processor_vid_recording:
+ if current_vid_recording:
+ # Start video recording
+ if not self._rec_manager.is_active:
+ # Get session name from processor
+ session_name = getattr(processor, "session_name", "auto_session")
+ self._auto_record_session_name = session_name
+
+ # Update filename with session name
+ self.filename_edit.text()
+ self.filename_edit.setText(f"{session_name}.mp4")
+
+ self._start_recording()
+ self.statusBar().showMessage(f"Auto-started recording: {session_name}", 3000)
+ logger.info(f"Auto-recording started for session: {session_name}")
+ else:
+ # Stop video recording
+ if self._rec_manager.is_active:
+ self._stop_recording()
+ self.statusBar().showMessage("Auto-stopped recording", 3000)
+ logger.info("Auto-recording stopped")
+
+ self._last_processor_vid_recording = current_vid_recording
+
+ def _start_inference(self) -> None:
+ if self._dlc_active:
+ self.statusBar().showMessage("Pose inference already running", 3000)
+ return
+ if not self.multi_camera_controller.is_running():
+ self._show_error("Start the camera preview before running pose inference.")
+ return
+ if not self._configure_dlc():
+ self._update_inference_buttons()
+ return
+ self._dlc.reset()
+ self._last_pose = None
+ self._dlc_active = True
+ self._dlc_initialized = False
+
+ # Update button to show initializing state
+ self.start_inference_button.setText("Initializing DLCLive!")
+ self.start_inference_button.setStyleSheet("background-color: #4A90E2; color: white;")
+ self.start_inference_button.setEnabled(False)
+ self.stop_inference_button.setEnabled(True)
+
+ self.statusBar().showMessage("Initializing DLCLive…", 3000)
+ self._update_camera_controls_enabled()
+ self._update_dlc_controls_enabled()
+
+ def _stop_inference(self, show_message: bool = True) -> None:
+ was_active = self._dlc_active
+ self._dlc_active = False
+ self._dlc_initialized = False
+ self._dlc.reset()
+ self._last_pose = None
+ self._last_processor_vid_recording = False
+ self._auto_record_session_name = None
+
+ # Reset button appearance
+ self.start_inference_button.setText("Start pose inference")
+ self.start_inference_button.setStyleSheet("")
+
+ if self._current_frame is not None:
+ self._display_frame(self._current_frame, force=True)
+ if was_active and show_message:
+ self.statusBar().showMessage("Pose inference stopped", 3000)
+ self._update_inference_buttons()
+ self._update_camera_controls_enabled()
+ self._update_dlc_controls_enabled()
+
+ # ------------------------------------------------------------------ recording
+ def _start_recording(self) -> None:
+ """Start recording from all active cameras."""
+ # Auto-start preview if not running
+ if not self.multi_camera_controller.is_running():
+ self._start_preview()
+ # Wait a moment for cameras to initialize before recording
+ # The recording will start after preview is confirmed running
+ self.statusBar().showMessage("Starting preview before recording...", 3000)
+ # Use a single-shot timer to start recording after preview starts
+ QTimer.singleShot(500, self._start_multi_camera_recording)
+ return
+
+ # Preview already running, start recording immediately
+ self._start_multi_camera_recording()
+
+ def _stop_recording(self) -> None:
+ """Stop recording from all cameras."""
+ self._stop_multi_camera_recording()
+
+ def _on_pose_ready(self, result: PoseResult) -> None:
+ if not self._dlc_active:
+ return
+ self._last_pose = result
+ # logger.debug(f"Pose result: {result.pose}, Timestamp: {result.timestamp}")
+ if self._current_frame is not None:
+ self._display_frame(self._current_frame, force=True)
+
+ def _on_dlc_error(self, message: str) -> None:
+ self._stop_inference(show_message=False)
+ self._show_error(message)
+
+ def _update_video_display(self, frame: np.ndarray) -> None:
+ display_frame = frame
+
+ if self.show_predictions_checkbox.isChecked() and self._last_pose and self._last_pose.pose is not None:
+ display_frame = draw_pose(
+ frame,
+ self._last_pose.pose,
+ p_cutoff=self._p_cutoff,
+ colormap=self._colormap,
+ offset=self._dlc_tile_offset,
+ scale=self._dlc_tile_scale,
+ )
+
+ if self._bbox_enabled:
+ display_frame = draw_bbox(
+ display_frame,
+ (self._bbox_x0, self._bbox_y0, self._bbox_x1, self._bbox_y1),
+ color_bgr=self._bbox_color,
+ offset=self._dlc_tile_offset,
+ scale=self._dlc_tile_scale,
+ )
+
+ rgb = cv2.cvtColor(display_frame, cv2.COLOR_BGR2RGB)
+
+ h, w, ch = rgb.shape
+ bytes_per_line = ch * w
+ image = QImage(rgb.data, w, h, bytes_per_line, QImage.Format.Format_RGB888)
+ pixmap = QPixmap.fromImage(image)
+
+ # Scale pixmap to fit label while preserving aspect ratio
+ scaled_pixmap = pixmap.scaled(
+ self.video_label.size(), Qt.AspectRatioMode.KeepAspectRatio, Qt.TransformationMode.SmoothTransformation
+ )
+ self.video_label.setPixmap(scaled_pixmap)
+
+ def _on_show_predictions_changed(self, _state: int) -> None:
+ if self._current_frame is not None:
+ self._display_frame(self._current_frame, force=True)
+
+ def _on_bbox_changed(self, _value: int = 0) -> None:
+ """Handle bounding box parameter changes."""
+ self._bbox_enabled = self.bbox_enabled_checkbox.isChecked()
+ self._bbox_x0 = self.bbox_x0_spin.value()
+ self._bbox_y0 = self.bbox_y0_spin.value()
+ self._bbox_x1 = self.bbox_x1_spin.value()
+ self._bbox_y1 = self.bbox_y1_spin.value()
+
+ # Force redraw if preview is running
+ if self._current_frame is not None:
+ self._display_frame(self._current_frame, force=True)
+
+ def _on_dlc_initialised(self, success: bool) -> None:
+ if success:
+ self._dlc_initialized = True
+ # Update button to show running state
+ self.start_inference_button.setText("DLCLive running!")
+ self.start_inference_button.setStyleSheet("background-color: #4CAF50; color: white;")
+ self.statusBar().showMessage("DLCLive initialized successfully", 3000)
+ else:
+ self._dlc_initialized = False
+ # Reset button on failure
+ self.start_inference_button.setText("Start pose inference")
+ self.start_inference_button.setStyleSheet("")
+ self.statusBar().showMessage("DLCLive initialization failed", 5000)
+ # Stop inference since initialization failed
+ self._stop_inference(show_message=False)
+
+ # ------------------------------------------------------------------ helpers
+ def _show_error(self, message: str) -> None:
+ self.statusBar().showMessage(message, 5000)
+ QMessageBox.critical(self, "Error", message)
+
+ def _show_warning(self, message: str) -> None:
+ """Display a warning message dialog."""
+ self.statusBar().showMessage(f"⚠ {message}", 5000)
+ QMessageBox.warning(self, "Warning", message)
+
+ def _show_info(self, message: str) -> None:
+ """Display an informational message dialog."""
+ self.statusBar().showMessage(message, 5000)
+ QMessageBox.information(self, "Information", message)
+
+ # ------------------------------------------------------------------ Qt overrides
+ def closeEvent(self, event: QCloseEvent) -> None: # pragma: no cover - GUI behaviour
+ if self.multi_camera_controller.is_running():
+ self.multi_camera_controller.stop(wait=True)
+
+ # Stop all multi-camera recorders
+ self._rec_manager.stop_all()
+
+ # Close the camera dialog if open (ensures its worker thread is canceled)
+ if getattr(self, "_cam_dialog", None) is not None and self._cam_dialog.isVisible():
+ try:
+ self._cam_dialog.close()
+ except Exception:
+ pass
+ self._cam_dialog = None
+
+ self._dlc.shutdown()
+ if hasattr(self, "_metrics_timer"):
+ self._metrics_timer.stop()
+
+ # Remember model path on exit
+ self._model_path_store.save_if_valid(self.model_path_edit.text().strip())
+
+ # Close the window
+ super().closeEvent(event)
diff --git a/dlclivegui/gui/recording_manager.py b/dlclivegui/gui/recording_manager.py
new file mode 100644
index 0000000..0ee7b39
--- /dev/null
+++ b/dlclivegui/gui/recording_manager.py
@@ -0,0 +1,122 @@
+# dlclivegui/services/recording_manager.py
+from __future__ import annotations
+
+import logging
+import time
+
+import numpy as np
+
+from dlclivegui.services.multi_camera_controller import get_camera_id
+from dlclivegui.services.video_recorder import RecorderStats, VideoRecorder
+
+# from dlclivegui.config import CameraSettings, RecordingSettings
+from dlclivegui.utils.config_models import CameraSettingsModel, RecordingSettingsModel
+
+log = logging.getLogger(__name__)
+
+
+class RecordingManager:
+ """Handle multi-camera recording lifecycle and filenames."""
+
+ def __init__(self):
+ self._recorders: dict[str, VideoRecorder] = {}
+
+ @property
+ def is_active(self) -> bool:
+ return bool(self._recorders)
+
+ @property
+ def recorders(self) -> dict[str, VideoRecorder]:
+ return self._recorders
+
+ def pop(self, cam_id: str, default=None) -> VideoRecorder | None:
+ return self._recorders.pop(cam_id, default)
+
+ def start_all(
+ self,
+ recording: RecordingSettingsModel,
+ active_cams: list[CameraSettingsModel],
+ current_frames: dict[str, np.ndarray],
+ ) -> None:
+ if self._recorders:
+ return
+ base_path = recording.output_path()
+ base_stem = base_path.stem
+
+ for cam in active_cams:
+ cam_id = get_camera_id(cam)
+ cam_filename = f"{base_stem}_{cam.backend}_cam{cam.index}{base_path.suffix}"
+ cam_path = base_path.parent / cam_filename
+ frame = current_frames.get(cam_id)
+ frame_size = (frame.shape[0], frame.shape[1]) if frame is not None else None
+ recorder = VideoRecorder(
+ cam_path,
+ frame_size=frame_size,
+ frame_rate=float(cam.fps),
+ codec=recording.codec,
+ crf=recording.crf,
+ )
+ try:
+ recorder.start()
+ self._recorders[cam_id] = recorder
+ log.info("Started recording %s -> %s", cam_id, cam_path)
+ except Exception as exc:
+ log.error("Failed to start recording for %s: %s", cam_id, exc)
+
+ def stop_all(self) -> None:
+ for cam_id, rec in self._recorders.items():
+ try:
+ rec.stop()
+ log.info("Stopped recording %s", cam_id)
+ except Exception as exc:
+ log.warning("Error stopping recorder for %s: %s", cam_id, exc)
+ self._recorders.clear()
+
+ def write_frame(self, cam_id: str, frame: np.ndarray, timestamp: float | None = None) -> None:
+ rec = self._recorders.get(cam_id)
+ if not rec or not rec.is_running:
+ return
+ try:
+ rec.write(frame, timestamp=timestamp or time.time())
+ except Exception as exc:
+ log.warning("Failed to write frame for %s: %s", cam_id, exc)
+ try:
+ rec.stop()
+ except Exception:
+ log.exception("Failed to stop recorder for %s after write error.")
+ self._recorders.pop(cam_id, None)
+
+ def get_stats_summary(self) -> str:
+ # Aggregate stats across recorders
+ totals = {
+ "written": 0,
+ "dropped": 0,
+ "queue": 0,
+ "max_latency": 0.0,
+ "avg_latencies": [],
+ }
+ for rec in self._recorders.values():
+ stats: RecorderStats | None = rec.get_stats()
+ if not stats:
+ continue
+ totals["written"] += stats.frames_written
+ totals["dropped"] += stats.dropped_frames
+ totals["queue"] += stats.queue_size
+ totals["max_latency"] = max(totals["max_latency"], stats.last_latency)
+ totals["avg_latencies"].append(stats.average_latency)
+
+ if len(self._recorders) == 1:
+ rec = next(iter(self._recorders.values()))
+ stats = rec.get_stats()
+ if stats:
+ from dlclivegui.utils.stats import format_recorder_stats
+
+ return format_recorder_stats(stats)
+ return "Recording..."
+ else:
+ avg = sum(totals["avg_latencies"]) / len(totals["avg_latencies"]) if totals["avg_latencies"] else 0.0
+ return (
+ f"{len(self._recorders)} cams | {totals['written']} frames | "
+ f"latency {totals['max_latency'] * 1000:.1f}ms (avg {avg * 1000:.1f}ms) | "
+ f"queue {totals['queue']} | dropped {totals['dropped']}"
+ )
diff --git a/dlclivegui/gui/theme.py b/dlclivegui/gui/theme.py
new file mode 100644
index 0000000..949a105
--- /dev/null
+++ b/dlclivegui/gui/theme.py
@@ -0,0 +1,31 @@
+# dlclivegui/utils/theme.py
+from __future__ import annotations
+
+import enum
+from pathlib import Path
+
+import qdarkstyle
+from PySide6.QtGui import QAction
+from PySide6.QtWidgets import QApplication
+
+ASSETS = Path(__file__).parent.parent / "assets"
+LOGO = str(ASSETS / "logo.png")
+LOGO_ALPHA = str(ASSETS / "logo_transparent.png")
+SPLASH_SCREEN = str(ASSETS / "welcome.png")
+
+
+class AppStyle(enum.Enum):
+ SYS_DEFAULT = "system"
+ DARK = "dark"
+
+
+def apply_theme(mode: AppStyle, action_dark: QAction, action_light: QAction) -> None:
+ app = QApplication.instance()
+ if mode == AppStyle.DARK:
+ app.setStyleSheet(qdarkstyle.load_stylesheet_pyside6())
+ action_dark.setChecked(True)
+ action_light.setChecked(False)
+ else:
+ app.setStyleSheet("")
+ action_dark.setChecked(False)
+ action_light.setChecked(True)
diff --git a/dlclivegui/main.py b/dlclivegui/main.py
new file mode 100644
index 0000000..23802d8
--- /dev/null
+++ b/dlclivegui/main.py
@@ -0,0 +1,54 @@
+import signal
+import sys
+
+from PySide6.QtCore import Qt, QTimer
+from PySide6.QtGui import QIcon, QPixmap
+from PySide6.QtWidgets import QApplication, QSplashScreen
+
+from dlclivegui.gui.main_window import DLCLiveMainWindow
+from dlclivegui.gui.theme import LOGO, SPLASH_SCREEN
+
+
+def main() -> None:
+ signal.signal(signal.SIGINT, signal.SIG_DFL)
+
+ # Enable HiDPI pixmaps (optional but recommended)
+ QApplication.setAttribute(Qt.AA_UseHighDpiPixmaps, True)
+
+ app = QApplication(sys.argv)
+ app.setWindowIcon(QIcon(LOGO))
+
+ # Load and scale splash pixmap
+ raw_pixmap = QPixmap(SPLASH_SCREEN)
+ splash_width = 600
+
+ if not raw_pixmap.isNull():
+ aspect_ratio = raw_pixmap.width() / raw_pixmap.height()
+ splash_height = int(splash_width / aspect_ratio)
+ scaled_pixmap = raw_pixmap.scaled(
+ splash_width,
+ splash_height,
+ Qt.KeepAspectRatio,
+ Qt.SmoothTransformation,
+ )
+ else:
+ # Fallback: empty pixmap
+ splash_height = 400
+ scaled_pixmap = QPixmap(splash_width, splash_height)
+ scaled_pixmap.fill(Qt.black)
+
+ splash = QSplashScreen(scaled_pixmap)
+ splash.show()
+
+ def show_main():
+ splash.close()
+ window = DLCLiveMainWindow()
+ window.show()
+
+ QTimer.singleShot(1000, show_main)
+
+ sys.exit(app.exec())
+
+
+if __name__ == "__main__": # pragma: no cover - manual start
+ main()
diff --git a/dlclivegui/pose_process.py b/dlclivegui/pose_process.py
deleted file mode 100644
index 7ae4809..0000000
--- a/dlclivegui/pose_process.py
+++ /dev/null
@@ -1,273 +0,0 @@
-"""
-DeepLabCut Toolbox (deeplabcut.org)
-© A. & M. Mathis Labs
-
-Licensed under GNU Lesser General Public License v3.0
-"""
-
-
-import multiprocess as mp
-import threading
-import time
-import pandas as pd
-import numpy as np
-
-from dlclivegui import CameraProcess
-from dlclivegui.queue import ClearableQueue, ClearableMPQueue
-
-
-class DLCLiveProcessError(Exception):
- """
- Exception for incorrect use of DLC-live-GUI Process Manager
- """
-
- pass
-
-
-class CameraPoseProcess(CameraProcess):
- """ Camera Process Manager class. Controls image capture, pose estimation and writing images to a video file in a background process.
-
- Parameters
- ----------
- device : :class:`cameracontrol.Camera`
- a camera object
- ctx : :class:`multiprocess.Context`
- multiprocessing context
- """
-
- def __init__(self, device, ctx=mp.get_context("spawn")):
- """ Constructor method
- """
-
- super().__init__(device, ctx)
- self.display_pose = None
- self.display_pose_queue = ClearableMPQueue(2, ctx=self.ctx)
- self.pose_process = None
-
- def start_pose_process(self, dlc_params, timeout=300):
-
- self.pose_process = self.ctx.Process(
- target=self._run_pose,
- args=(self.frame_shared, self.frame_time_shared, dlc_params),
- daemon=True,
- )
- self.pose_process.start()
-
- stime = time.time()
- while time.time() - stime < timeout:
- cmd = self.q_from_process.read()
- if cmd is not None:
- if (cmd[0] == "pose") and (cmd[1] == "start"):
- return cmd[2]
- else:
- self.q_to_process.write(cmd)
-
- def _run_pose(self, frame_shared, frame_time, dlc_params):
-
- res = self.device.im_size
- self.frame = np.frombuffer(frame_shared.get_obj(), dtype="uint8").reshape(
- res[1], res[0], 3
- )
- self.frame_time = np.frombuffer(frame_time.get_obj(), dtype="d")
-
- ret = self._open_dlc_live(dlc_params)
- self.q_from_process.write(("pose", "start", ret))
-
- self._pose_loop()
- self.q_from_process.write(("pose", "end"))
-
- def _open_dlc_live(self, dlc_params):
-
- from dlclive import DLCLive
-
- ret = False
-
- self.opt_rate = True if dlc_params.pop("mode") == "Optimize Rate" else False
-
- proc_params = dlc_params.pop("processor")
- if proc_params is not None:
- proc_obj = proc_params.pop("object", None)
- if proc_obj is not None:
- dlc_params["processor"] = proc_obj(**proc_params)
-
- self.dlc = DLCLive(**dlc_params)
- if self.frame is not None:
- self.dlc.init_inference(
- self.frame, frame_time=self.frame_time[0], record=False
- )
- self.poses = []
- self.pose_times = []
- self.pose_frame_times = []
- ret = True
-
- return ret
-
- def _pose_loop(self):
- """ Conduct pose estimation using deeplabcut-live in loop
- """
-
- run = True
- write = False
- frame_time = 0
- pose_time = 0
- end_time = time.time()
-
- while run:
-
- ref_time = frame_time if self.opt_rate else end_time
-
- if self.frame_time[0] > ref_time:
-
- frame = self.frame
- frame_time = self.frame_time[0]
- pose = self.dlc.get_pose(frame, frame_time=frame_time, record=write)
- pose_time = time.time()
-
- self.display_pose_queue.write(pose, clear=True)
-
- if write:
- self.poses.append(pose)
- self.pose_times.append(pose_time)
- self.pose_frame_times.append(frame_time)
-
- cmd = self.q_to_process.read()
- if cmd is not None:
- if cmd[0] == "pose":
- if cmd[1] == "write":
- write = cmd[2]
- self.q_from_process.write(cmd)
- elif cmd[1] == "save":
- ret = self._save_pose(cmd[2])
- self.q_from_process.write(cmd + (ret,))
- elif cmd[1] == "end":
- run = False
- else:
- self.q_to_process.write(cmd)
-
- def start_record(self, timeout=5):
-
- ret = super().start_record(timeout=timeout)
-
- if (self.pose_process is not None) and (self.writer_process is not None):
- if (self.pose_process.is_alive()) and (self.writer_process.is_alive()):
- self.q_to_process.write(("pose", "write", True))
-
- stime = time.time()
- while time.time() - stime < timeout:
- cmd = self.q_from_process.read()
- if cmd is not None:
- if (cmd[0] == "pose") and (cmd[1] == "write"):
- ret = cmd[2]
- break
- else:
- self.q_from_process.write(cmd)
-
- return ret
-
- def stop_record(self, timeout=5):
-
- ret = super().stop_record(timeout=timeout)
-
- if (self.pose_process is not None) and (self.writer_process is not None):
- if (self.pose_process.is_alive()) and (self.writer_process.is_alive()):
- self.q_to_process.write(("pose", "write", False))
-
- stime = time.time()
- while time.time() - stime < timeout:
- cmd = self.q_from_process.read()
- if cmd is not None:
- if (cmd[0] == "pose") and (cmd[1] == "write"):
- ret = not cmd[2]
- break
- else:
- self.q_from_process.write(cmd)
-
- return ret
-
- def stop_pose_process(self):
-
- ret = True
- if self.pose_process is not None:
- if self.pose_process.is_alive():
- self.q_to_process.write(("pose", "end"))
-
- while True:
- cmd = self.q_from_process.read()
- if cmd is not None:
- if cmd[0] == "pose":
- if cmd[1] == "end":
- break
- else:
- self.q_from_process.write(cmd)
-
- self.pose_process.join(5)
- if self.pose_process.is_alive():
- self.pose_process.terminate()
-
- return True
-
- def save_pose(self, filename, timeout=60):
-
- ret = False
- if self.pose_process is not None:
- if self.pose_process.is_alive():
- self.q_to_process.write(("pose", "save", filename))
-
- stime = time.time()
- while time.time() - stime < timeout:
- cmd = self.q_from_process.read()
- if cmd is not None:
- if (cmd[0] == "pose") and (cmd[1] == "save"):
- ret = cmd[3]
- break
- else:
- self.q_from_process.write(cmd)
- return ret
-
- def _save_pose(self, filename):
- """ Saves a pandas data frame with pose data collected while recording video
-
- Returns
- -------
- bool
- a logical flag indicating whether save was successful
- """
-
- ret = False
-
- if len(self.pose_times) > 0:
-
- dlc_file = f"{filename}_DLC.hdf5"
- proc_file = f"{filename}_PROC"
-
- bodyparts = self.dlc.cfg["all_joints_names"]
- poses = np.array(self.poses)
- poses = poses.reshape((poses.shape[0], poses.shape[1] * poses.shape[2]))
- pdindex = pd.MultiIndex.from_product(
- [bodyparts, ["x", "y", "likelihood"]], names=["bodyparts", "coords"]
- )
- pose_df = pd.DataFrame(poses, columns=pdindex)
- pose_df["frame_time"] = self.pose_frame_times
- pose_df["pose_time"] = self.pose_times
-
- pose_df.to_hdf(dlc_file, key="df_with_missing", mode="w")
- if self.dlc.processor is not None:
- self.dlc.processor.save(proc_file)
-
- self.poses = []
- self.pose_times = []
- self.pose_frame_times = []
-
- ret = True
-
- return ret
-
- def get_display_pose(self):
-
- pose = self.display_pose_queue.read(clear=True)
- if pose is not None:
- self.display_pose = pose
- if self.device.display_resize != 1:
- self.display_pose[:, :2] *= self.device.display_resize
-
- return self.display_pose
diff --git a/dlclivegui/processor/__init__.py b/dlclivegui/processor/__init__.py
deleted file mode 100644
index b97a9cc..0000000
--- a/dlclivegui/processor/__init__.py
+++ /dev/null
@@ -1 +0,0 @@
-from .teensy_laser.teensy_laser import TeensyLaser
diff --git a/dlclivegui/processor/processor.py b/dlclivegui/processor/processor.py
deleted file mode 100644
index 05eb7a8..0000000
--- a/dlclivegui/processor/processor.py
+++ /dev/null
@@ -1,23 +0,0 @@
-"""
-DeepLabCut Toolbox (deeplabcut.org)
-© A. & M. Mathis Labs
-
-Licensed under GNU Lesser General Public License v3.0
-"""
-
-"""
-Default processor class. Processors must contain two methods:
-i) process: takes in a pose, performs operations, and returns a pose
-ii) save: saves any internal data generated by the processor (such as timestamps for commands to external hardware)
-"""
-
-
-class Processor(object):
- def __init__(self):
- pass
-
- def process(self, pose):
- return pose
-
- def save(self, file=""):
- return 0
diff --git a/dlclivegui/processor/teensy_laser/__init__.py b/dlclivegui/processor/teensy_laser/__init__.py
deleted file mode 100644
index d2f10ca..0000000
--- a/dlclivegui/processor/teensy_laser/__init__.py
+++ /dev/null
@@ -1 +0,0 @@
-from .teensy_laser import *
diff --git a/dlclivegui/processor/teensy_laser/teensy_laser.ino b/dlclivegui/processor/teensy_laser/teensy_laser.ino
deleted file mode 100644
index 76a470b..0000000
--- a/dlclivegui/processor/teensy_laser/teensy_laser.ino
+++ /dev/null
@@ -1,77 +0,0 @@
-/*
- * Commands:
- * O = opto on; command = O, frequency, width, duration
- * X = opto off
- * R = reboot
- */
-
-
-const int opto_pin = 0;
-unsigned int opto_start = 0,
- opto_duty_cycle = 0,
- opto_freq = 0,
- opto_width = 0,
- opto_dur = 0;
-
-unsigned int read_int16() {
- union u_tag {
- byte b[2];
- unsigned int val;
- } par;
- for (int i=0; i<2; i++){
- if ((Serial.available() > 0))
- par.b[i] = Serial.read();
- else
- par.b[i] = 0;
- }
- return par.val;
-}
-
-void setup() {
- Serial.begin(115200);
- pinMode(opto_pin, OUTPUT);
-}
-
-void loop() {
-
- unsigned int curr_time = millis();
-
- while (Serial.available() > 0) {
-
- unsigned int cmd = Serial.read();
-
- if(cmd == 'O') {
-
- opto_start = curr_time;
- opto_freq = read_int16();
- opto_width = read_int16();
- opto_dur = read_int16();
- if (opto_dur == 0)
- opto_dur = 65355;
- opto_duty_cycle = opto_width * opto_freq * 4096 / 1000;
- analogWriteFrequency(opto_pin, opto_freq);
- analogWrite(opto_pin, opto_duty_cycle);
-
- Serial.print(opto_freq);
- Serial.print(',');
- Serial.print(opto_width);
- Serial.print(',');
- Serial.print(opto_dur);
- Serial.print('\n');
- Serial.flush();
-
- } else if(cmd == 'X') {
-
- analogWrite(opto_pin, 0);
-
- } else if(cmd == 'R') {
-
- _reboot_Teensyduino_();
-
- }
- }
-
- if (curr_time > opto_start + opto_dur)
- analogWrite(opto_pin, 0);
-
-}
diff --git a/dlclivegui/processor/teensy_laser/teensy_laser.py b/dlclivegui/processor/teensy_laser/teensy_laser.py
deleted file mode 100644
index 4535d55..0000000
--- a/dlclivegui/processor/teensy_laser/teensy_laser.py
+++ /dev/null
@@ -1,77 +0,0 @@
-from ..processor import Processor
-import serial
-import struct
-import time
-
-
-class TeensyLaser(Processor):
- def __init__(
- self, com, baudrate=115200, pulse_freq=50, pulse_width=5, max_stim_dur=0
- ):
-
- super().__init__()
- self.ser = serial.Serial(com, baudrate)
- self.pulse_freq = pulse_freq
- self.pulse_width = pulse_width
- self.max_stim_dur = (
- max_stim_dur if (max_stim_dur >= 0) and (max_stim_dur < 65356) else 0
- )
- self.stim_on = False
- self.stim_on_time = []
- self.stim_off_time = []
-
- def close_serial(self):
-
- self.ser.close()
-
- def stimulate_on(self):
-
- # command to activate PWM signal to laser is the letter 'O' followed by three 16 bit integers -- pulse frequency, pulse width, and max stim duration
- if not self.stim_on:
- self.ser.write(
- b"O"
- + struct.pack(
- "HHH", self.pulse_freq, self.pulse_width, self.max_stim_dur
- )
- )
- self.stim_on = True
- self.stim_on_time.append(time.time())
-
- def stim_off(self):
-
- # command to turn off PWM signal to laser is the letter 'X'
- if self.stim_on:
- self.ser.write(b"X")
- self.stim_on = False
- self.stim_off_time.append(time.time())
-
- def process(self, pose):
-
- # define criteria to stimulate (e.g. if first point is in a corner of the video)
- box = [[0, 100], [0, 100]]
- if (
- (pose[0][0] > box[0][0])
- and (pose[0][0] < box[0][1])
- and (pose[0][1] > box[1][0])
- and (pose[0][1] < box[1][1])
- ):
- self.stimulate_on()
- else:
- self.stim_off()
-
- return pose
-
- def save(self, file=None):
-
- ### save stim on and stim off times
- save_code = 0
- if file:
- try:
- pickle.dump(
- {"stim_on": self.stim_on_time, "stim_off": self.stim_off_time},
- open(file, "wb"),
- )
- save_code = 1
- except Exception:
- save_code = -1
- return save_code
diff --git a/dlclivegui/processors/PLUGIN_SYSTEM.md b/dlclivegui/processors/PLUGIN_SYSTEM.md
new file mode 100644
index 0000000..0c0351d
--- /dev/null
+++ b/dlclivegui/processors/PLUGIN_SYSTEM.md
@@ -0,0 +1,191 @@
+# DeepLabCut Processor Plugin System
+
+This folder contains a plugin-style architecture for DLC processors that allows GUI tools to discover and instantiate processors dynamically.
+
+## Architecture
+
+### 1. Processor Registry
+
+Each processor file should define a `PROCESSOR_REGISTRY` dictionary and helper functions:
+
+```python
+# Registry for GUI discovery
+PROCESSOR_REGISTRY = {}
+
+# At end of file, register your processors
+PROCESSOR_REGISTRY["MyProcessor_socket"] = MyProcessor_socket
+```
+
+### 2. Processor Metadata
+
+Each processor class should define metadata attributes for GUI discovery:
+
+```python
+class MyProcessor_socket(BaseProcessor_socket):
+ # Metadata for GUI discovery
+ PROCESSOR_NAME = "Mouse Pose Processor" # Human-readable name
+ PROCESSOR_DESCRIPTION = "Calculates mouse center, heading, and head angle"
+ PROCESSOR_PARAMS = {
+ "bind": {
+ "type": "tuple",
+ "default": ("0.0.0.0", 6000),
+ "description": "Server address (host, port)"
+ },
+ "use_filter": {
+ "type": "bool",
+ "default": False,
+ "description": "Apply One-Euro filter"
+ },
+ # ... more parameters
+ }
+```
+
+### 3. Discovery Functions
+
+Two helper functions enable GUI discovery:
+
+```python
+def get_available_processors():
+ """Returns dict of available processors with metadata."""
+
+def instantiate_processor(class_name, **kwargs):
+ """Instantiates a processor by name with given parameters."""
+```
+
+## GUI Integration
+
+### Simple Usage
+
+```python
+from dlc_processor_socket import get_available_processors, instantiate_processor
+
+# 1. Get available processors
+processors = get_available_processors()
+
+# 2. Display to user (e.g., in dropdown)
+for class_name, info in processors.items():
+ print(f"{info['name']} - {info['description']}")
+
+# 3. User selects "MyProcessor_socket"
+selected_class = "MyProcessor_socket"
+
+# 4. Show parameter form based on info['params']
+processor_info = processors[selected_class]
+for param_name, param_info in processor_info['params'].items():
+ # Create input widget for param_type and default value
+ pass
+
+# 5. Instantiate with user's values
+processor = instantiate_processor(
+ selected_class,
+ bind=("127.0.0.1", 7000),
+ use_filter=True
+)
+```
+
+### Scanning Multiple Files
+
+To scan a folder for processor files:
+
+```python
+import importlib.util
+from pathlib import Path
+
+def load_processors_from_file(file_path):
+ """Load processors from a single file."""
+ spec = importlib.util.spec_from_file_location("processors", file_path)
+ module = importlib.util.module_from_spec(spec)
+ spec.loader.exec_module(module)
+
+ if hasattr(module, 'get_available_processors'):
+ return module.get_available_processors()
+ return {}
+
+# Scan folder
+for py_file in Path("dlc_processors").glob("*.py"):
+ processors = load_processors_from_file(py_file)
+ # Display processors to user
+```
+
+## Examples
+
+### 1. Command-line Example
+
+```bash
+python example_gui_usage.py
+```
+
+This demonstrates:
+- Loading processors
+- Displaying metadata
+- Instantiating with default/custom parameters
+- Simulated GUI workflow
+
+### 2. tkinter GUI
+
+```bash
+python processor_gui_simple.py
+```
+
+This provides a full GUI with:
+- Dropdown to select processor
+- Auto-generated parameter form
+- Create/Stop buttons
+- Status display
+
+## Adding New Processors
+
+To make a new processor discoverable:
+
+1. **Define metadata attributes:**
+```python
+class MyNewProcessor(BaseProcessor_socket):
+ PROCESSOR_NAME = "My New Processor"
+ PROCESSOR_DESCRIPTION = "Does something cool"
+ PROCESSOR_PARAMS = {
+ "my_param": {
+ "type": "bool",
+ "default": True,
+ "description": "Enable cool feature"
+ }
+ }
+```
+
+2. **Register in PROCESSOR_REGISTRY:**
+```python
+PROCESSOR_REGISTRY["MyNewProcessor"] = MyNewProcessor
+```
+
+3. **Done!** GUI will automatically discover it.
+
+## Parameter Types
+
+Supported parameter types in `PROCESSOR_PARAMS`:
+
+- `"bool"` - Boolean checkbox
+- `"int"` - Integer input
+- `"float"` - Float input
+- `"str"` - String input
+- `"bytes"` - String that gets encoded to bytes
+- `"tuple"` - Tuple (e.g., `(host, port)`)
+- `"dict"` - Dictionary (e.g., filter parameters)
+- `"list"` - List
+
+## Benefits
+
+1. **No hardcoding** - GUI doesn't need to know about specific processors
+2. **Easy extension** - Add new processors without modifying GUI code
+3. **Self-documenting** - Parameters include descriptions
+4. **Type-safe** - Parameter metadata includes type information
+5. **Modular** - Each processor file can be independent
+
+## File Structure
+
+```
+dlc_processors/
+├── dlc_processor_socket.py # Base + MyProcessor with registry
+├── my_custom_processor.py # Your custom processor (with registry)
+├── example_gui_usage.py # Command-line example
+├── processor_gui_simple.py # tkinter GUI example
+└── PLUGIN_SYSTEM.md # This file
+```
diff --git a/dlclivegui/processors/dlc_processor_socket.py b/dlclivegui/processors/dlc_processor_socket.py
new file mode 100644
index 0000000..1ec9827
--- /dev/null
+++ b/dlclivegui/processors/dlc_processor_socket.py
@@ -0,0 +1,853 @@
+import logging
+import pickle
+import socket
+import time
+from collections import deque
+from math import acos, atan2, copysign, degrees, pi, sqrt
+from multiprocessing.connection import Listener
+from pathlib import Path
+from threading import Event, Thread
+
+import numpy as np
+from dlclive import Processor # type: ignore
+
+LOG = logging.getLogger("dlc_processor_socket")
+LOG.setLevel(logging.INFO)
+_handler = logging.StreamHandler()
+_handler.setFormatter(logging.Formatter("%(asctime)s [%(levelname)s] %(message)s"))
+LOG.addHandler(_handler)
+
+
+# Registry for GUI discovery
+PROCESSOR_REGISTRY = {}
+
+
+class OneEuroFilter:
+ def __init__(self, t0, x0, dx0=None, min_cutoff=1.0, beta=0.0, d_cutoff=1.0):
+ self.min_cutoff = min_cutoff
+ self.beta = beta
+ self.d_cutoff = d_cutoff
+ self.x_prev = x0
+ if dx0 is None:
+ dx0 = np.zeros_like(x0)
+ self.dx_prev = dx0
+ self.t_prev = t0
+
+ @staticmethod
+ def smoothing_factor(t_e, cutoff):
+ r = 2 * pi * cutoff * t_e
+ return r / (r + 1)
+
+ @staticmethod
+ def exponential_smoothing(alpha, x, x_prev):
+ return alpha * x + (1 - alpha) * x_prev
+
+ def __call__(self, t, x):
+ t_e = t - self.t_prev
+ if t_e <= 0:
+ return x
+ a_d = self.smoothing_factor(t_e, self.d_cutoff)
+ dx = (x - self.x_prev) / t_e
+ dx_hat = self.exponential_smoothing(a_d, dx, self.dx_prev)
+
+ cutoff = self.min_cutoff + self.beta * abs(dx_hat)
+ a = self.smoothing_factor(t_e, cutoff)
+ x_hat = self.exponential_smoothing(a, x, self.x_prev)
+
+ self.x_prev = x_hat
+ self.dx_prev = dx_hat
+ self.t_prev = t
+
+ return x_hat
+
+
+class BaseProcessor_socket(Processor):
+ """
+ Base DLC Processor with multi-client broadcasting support.
+
+ Handles network connections, timing, and data logging.
+ Subclasses should implement custom pose processing logic.
+ """
+
+ # Metadata for GUI discovery
+ PROCESSOR_NAME = "Base Socket Processor"
+ PROCESSOR_DESCRIPTION = "Base class for socket-based processors with multi-client support"
+ PROCESSOR_PARAMS = {
+ "bind": {
+ "type": "tuple",
+ "default": ("0.0.0.0", 6000),
+ "description": "Server address (host, port)",
+ },
+ "authkey": {
+ "type": "bytes",
+ "default": b"secret password",
+ "description": "Authentication key for clients",
+ },
+ "use_perf_counter": {
+ "type": "bool",
+ "default": False,
+ "description": "Use time.perf_counter() instead of time.time()",
+ },
+ "save_original": {
+ "type": "bool",
+ "default": False,
+ "description": "Save raw pose arrays for analysis",
+ },
+ }
+
+ def __init__(
+ self,
+ bind=("0.0.0.0", 6000),
+ authkey=b"secret password",
+ use_perf_counter=False,
+ save_original=False,
+ ):
+ """
+ Initialize base processor with socket server.
+
+ Args:
+ bind: (host, port) tuple for server binding
+ authkey: Authentication key for client connections
+ use_perf_counter: If True, use time.perf_counter() instead of time.time()
+ save_original: If True, save raw pose arrays for analysis
+ """
+ super().__init__()
+
+ # Network setup
+ self.address = bind
+ self.authkey = authkey
+ self.listener = Listener(bind, authkey=authkey)
+ self._stop = Event()
+ self.conns = set()
+
+ # Start accept loop in background
+ Thread(target=self._accept_loop, name="DLCAccept", daemon=True).start()
+
+ # Timing function
+ self.timing_func = time.perf_counter if use_perf_counter else time.time
+ self.start_time = self.timing_func()
+
+ # Data storage
+ self.time_stamp = deque()
+ self.step = deque()
+ self.frame_time = deque()
+ self.pose_time = deque()
+ self.original_pose = deque()
+
+ self._session_name = "test_session"
+ self.filename = None
+ self._recording = Event() # Thread-safe recording flag
+ self._vid_recording = Event() # Thread-safe video recording flag
+
+ # State
+ self.curr_step = 0
+ self.save_original = save_original
+
+ @property
+ def recording(self):
+ """Thread-safe recording flag."""
+ return self._recording.is_set()
+
+ @property
+ def video_recording(self):
+ """Thread-safe video recording flag."""
+ return self._vid_recording.is_set()
+
+ @property
+ def session_name(self):
+ return self._session_name
+
+ @session_name.setter
+ def session_name(self, name):
+ self._session_name = name
+ self.filename = f"{name}_dlc_processor_data.pkl"
+
+ def _accept_loop(self):
+ """Background thread to accept new client connections."""
+ LOG.debug(f"DLC Processor listening on {self.address[0]}:{self.address[1]}")
+ while not self._stop.is_set():
+ try:
+ c = self.listener.accept()
+ LOG.debug(f"Client connected from {self.listener.last_accepted}")
+ self.conns.add(c)
+ # Start RX loop for this connection (in case clients send data)
+ Thread(target=self._rx_loop, args=(c,), name="DLCRX", daemon=True).start()
+ except (OSError, EOFError):
+ break
+
+ def _rx_loop(self, c):
+ """Background thread to handle receive from a client (detects disconnects)."""
+ while not self._stop.is_set():
+ try:
+ if c.poll(0.05):
+ msg = c.recv()
+ # Handle control messages from client
+ self._handle_client_message(msg)
+ except (EOFError, OSError, BrokenPipeError):
+ break
+ try:
+ c.close()
+ except Exception:
+ pass
+ self.conns.discard(c)
+ LOG.info("Client disconnected")
+
+ def _handle_client_message(self, msg):
+ """Handle control messages from clients."""
+ if not isinstance(msg, dict):
+ return
+
+ cmd = msg.get("cmd")
+ if cmd == "set_session_name":
+ session_name = msg.get("session_name", "default_session")
+ self.session_name = session_name
+ LOG.info(f"Session name set to: {session_name}")
+
+ elif cmd == "start_recording":
+ self._vid_recording.set()
+ self._recording.set()
+ # Clear all data queues
+ self._clear_data_queues()
+ self.curr_step = 0
+ LOG.info("Recording started, data queues cleared")
+
+ elif cmd == "stop_recording":
+ self._recording.clear()
+ self._vid_recording.clear()
+ LOG.info("Recording stopped")
+
+ elif cmd == "save":
+ filename = msg.get("filename", self.filename)
+ save_code = self.save(filename)
+ LOG.info(f"Save {'successful' if save_code == 1 else 'failed'}: {filename}")
+
+ elif cmd == "start_video":
+ # Placeholder for video recording start
+ self._vid_recording.set()
+ LOG.info("Start video recording command received")
+
+ elif cmd == "set_filter":
+ # Handle filter enable/disable (subclasses override if they support filtering)
+ use_filter = msg.get("use_filter", False)
+ if hasattr(self, "use_filter"):
+ self.use_filter = bool(use_filter)
+ # Reset filters to reinitialize with new setting
+ if hasattr(self, "filters"):
+ self.filters = None
+ LOG.info(f"Filtering {'enabled' if use_filter else 'disabled'}")
+ else:
+ LOG.warning("set_filter command not supported by this processor")
+
+ elif cmd == "set_filter_params":
+ # Handle filter parameter updates (subclasses override if they support filtering)
+ filter_kwargs = msg.get("filter_kwargs", {})
+ if hasattr(self, "filter_kwargs"):
+ # Update filter parameters
+ self.filter_kwargs.update(filter_kwargs)
+ # Reset filters to reinitialize with new parameters
+ if hasattr(self, "filters"):
+ self.filters = None
+ LOG.info(f"Filter parameters updated: {filter_kwargs}")
+ else:
+ LOG.warning("set_filter_params command not supported by this processor")
+
+ def _clear_data_queues(self):
+ """Clear all data storage queues. Override in subclasses to clear additional queues."""
+ self.time_stamp.clear()
+ self.step.clear()
+ self.frame_time.clear()
+ self.pose_time.clear()
+ if self.save_original:
+ self.original_pose.clear()
+
+ def broadcast(self, payload):
+ """Send payload to all connected clients."""
+ dead = []
+ for c in list(self.conns):
+ try:
+ c.send(payload)
+ except (EOFError, OSError, BrokenPipeError):
+ dead.append(c)
+ for c in dead:
+ try:
+ c.close()
+ except Exception:
+ pass
+ self.conns.discard(c)
+
+ def process(self, pose, **kwargs):
+ """
+ Process pose and broadcast to clients.
+
+ This base implementation just saves original pose and broadcasts it.
+ Subclasses should override to add custom processing.
+
+ Args:
+ pose: DLC pose array (N_keypoints x 3) with [x, y, confidence]
+ **kwargs: Additional metadata (frame_time, pose_time, etc.)
+
+ Returns:
+ pose: Unmodified pose array
+ """
+ curr_time = self.timing_func()
+
+ # Save original pose if requested
+ if self.save_original:
+ self.original_pose.append(pose.copy())
+
+ # Update step counter
+ self.curr_step = self.curr_step + 1
+
+ # Store metadata (only if recording)
+ if self.recording:
+ self.time_stamp.append(curr_time)
+ self.step.append(self.curr_step)
+ self.frame_time.append(kwargs.get("frame_time", -1))
+ if "pose_time" in kwargs:
+ self.pose_time.append(kwargs["pose_time"])
+
+ # Broadcast raw pose to all connected clients
+ payload = [curr_time, pose]
+ self.broadcast(payload)
+
+ return pose
+
+ def stop(self):
+ """Stop the processor and close all connections."""
+ LOG.info("Stopping processor...")
+
+ # Signal stop to all threads
+ self._stop.set()
+
+ # Close all client connections first
+ for c in list(self.conns):
+ try:
+ c.close()
+ except Exception:
+ pass
+ self.conns.discard(c)
+
+ # Close the listener socket
+ if hasattr(self, "listener") and self.listener:
+ try:
+ self.listener.close()
+ except Exception as e:
+ LOG.debug(f"Error closing listener: {e}")
+
+ # Give the OS time to release the socket on Windows
+ # This prevents WinError 10048 when restarting
+ time.sleep(0.1)
+
+ LOG.info("Processor stopped, all connections closed")
+
+ def save(self, file=None):
+ """Save logged data to file."""
+ save_code = 0
+ if file:
+ LOG.info(f"Saving data to {file}")
+ try:
+ save_dict = self.get_data()
+ path2save = Path(__file__).parent.parent.parent / "data" / file
+ LOG.info(f"Path should be {path2save}")
+ pickle.dump(save_dict, open(path2save, "wb"))
+ save_code = 1
+ except Exception as e:
+ LOG.error(f"Save failed: {e}")
+ save_code = -1
+ return save_code
+
+ def get_data(self):
+ """Get logged data as dictionary."""
+ save_dict = dict()
+ if self.save_original:
+ save_dict["original_pose"] = np.array(self.original_pose)
+ save_dict["start_time"] = self.start_time
+ save_dict["time_stamp"] = np.array(self.time_stamp)
+ save_dict["step"] = np.array(self.step)
+ save_dict["frame_time"] = np.array(self.frame_time)
+ save_dict["pose_time"] = np.array(self.pose_time) if self.pose_time else None
+ save_dict["use_perf_counter"] = self.timing_func == time.perf_counter
+ return save_dict
+
+
+class MyProcessor_socket(BaseProcessor_socket):
+ """
+ DLC Processor with pose calculations (center, heading, head angle) and optional filtering.
+
+ Calculates:
+ - center: Weighted average of head keypoints
+ - heading: Body orientation (degrees)
+ - head_angle: Head rotation relative to body (radians)
+
+ Broadcasts: [timestamp, center_x, center_y, heading, head_angle]
+ """
+
+ # Metadata for GUI discovery
+ PROCESSOR_NAME = "Mouse Pose Processor"
+ PROCESSOR_DESCRIPTION = (
+ "Calculates mouse center, heading, and head angle with optional One-Euro filtering"
+ )
+ PROCESSOR_PARAMS = {
+ "bind": {
+ "type": "tuple",
+ "default": ("0.0.0.0", 6000),
+ "description": "Server address (host, port)",
+ },
+ "authkey": {
+ "type": "bytes",
+ "default": b"secret password",
+ "description": "Authentication key for clients",
+ },
+ "use_perf_counter": {
+ "type": "bool",
+ "default": False,
+ "description": "Use time.perf_counter() instead of time.time()",
+ },
+ "use_filter": {
+ "type": "bool",
+ "default": False,
+ "description": "Apply One-Euro filter to calculated values",
+ },
+ "filter_kwargs": {
+ "type": "dict",
+ "default": {"min_cutoff": 1.0, "beta": 0.02, "d_cutoff": 1.0},
+ "description": "One-Euro filter parameters (min_cutoff, beta, d_cutoff)",
+ },
+ "save_original": {
+ "type": "bool",
+ "default": False,
+ "description": "Save raw pose arrays for analysis",
+ },
+ }
+
+ def __init__(
+ self,
+ bind=("0.0.0.0", 6000),
+ authkey=b"secret password",
+ use_perf_counter=False,
+ use_filter=False,
+ filter_kwargs={},
+ save_original=False,
+ ):
+ """
+ DLC Processor with multi-client broadcasting support.
+
+ Args:
+ bind: (host, port) tuple for server binding
+ authkey: Authentication key for client connections
+ use_perf_counter: If True, use time.perf_counter() instead of time.time()
+ use_filter: If True, apply One-Euro filter to pose data
+ filter_kwargs: Dict with OneEuroFilter parameters (min_cutoff, beta, d_cutoff)
+ save_original: If True, save raw pose arrays
+ """
+ super().__init__(
+ bind=bind,
+ authkey=authkey,
+ use_perf_counter=use_perf_counter,
+ save_original=save_original,
+ )
+
+ # Additional data storage for processed values
+ self.center_x = deque()
+ self.center_y = deque()
+ self.heading_direction = deque()
+ self.head_angle = deque()
+
+ # Filtering
+ self.use_filter = use_filter
+ self.filter_kwargs = filter_kwargs
+ self.filters = None # Will be initialized on first pose
+
+ def _clear_data_queues(self):
+ """Clear all data storage queues including pose-specific ones."""
+ super()._clear_data_queues()
+ self.center_x.clear()
+ self.center_y.clear()
+ self.heading_direction.clear()
+ self.head_angle.clear()
+
+ def _initialize_filters(self, vals):
+ """Initialize One-Euro filters for each output variable."""
+ t0 = self.timing_func()
+ self.filters = {
+ "center_x": OneEuroFilter(t0, vals[0], **self.filter_kwargs),
+ "center_y": OneEuroFilter(t0, vals[1], **self.filter_kwargs),
+ "heading": OneEuroFilter(t0, vals[2], **self.filter_kwargs),
+ "head_angle": OneEuroFilter(t0, vals[3], **self.filter_kwargs),
+ }
+ LOG.debug(f"Initialized One-Euro filters with parameters: {self.filter_kwargs}")
+
+ def process(self, pose, **kwargs):
+ """
+ Process pose: calculate center/heading/head_angle, optionally filter, and broadcast.
+
+ Args:
+ pose: DLC pose array (N_keypoints x 3) with [x, y, confidence]
+ **kwargs: Additional metadata (frame_time, pose_time, etc.)
+
+ Returns:
+ pose: Unmodified pose array
+ """
+ # Save original pose if requested (from base class)
+ if self.save_original:
+ self.original_pose.append(pose.copy())
+
+ # Extract keypoints and confidence
+ xy = pose[:, :2]
+ conf = pose[:, 2]
+
+ # Calculate weighted center from head keypoints
+ head_xy = xy[[0, 1, 2, 3, 4, 5, 6, 26], :]
+ head_conf = conf[[0, 1, 2, 3, 4, 5, 6, 26]]
+ center = np.average(head_xy, axis=0, weights=head_conf)
+
+ # Calculate body axis (tail_base -> neck)
+ body_axis = xy[7] - xy[13]
+ body_axis /= sqrt(np.sum(body_axis**2))
+
+ # Calculate head axis (neck -> nose)
+ head_axis = xy[0] - xy[7]
+ head_axis /= sqrt(np.sum(head_axis**2))
+
+ # Calculate head angle relative to body
+ cross = body_axis[0] * head_axis[1] - head_axis[0] * body_axis[1]
+ sign = copysign(1, cross) # Positive when looking left
+ try:
+ head_angle = acos(body_axis @ head_axis) * sign
+ except ValueError:
+ head_angle = 0
+
+ # Calculate heading (body orientation)
+ heading = atan2(body_axis[1], body_axis[0])
+ heading = degrees(heading)
+
+ # Raw values (heading unwrapped for filtering)
+ vals = [center[0], center[1], heading, head_angle]
+
+ # Apply filtering if enabled
+ curr_time = self.timing_func()
+ if self.use_filter:
+ if self.filters is None:
+ self._initialize_filters(vals)
+
+ # Filter each value (heading is filtered in unwrapped space)
+ filtered_vals = [
+ self.filters["center_x"](curr_time, vals[0]),
+ self.filters["center_y"](curr_time, vals[1]),
+ self.filters["heading"](curr_time, vals[2]),
+ self.filters["head_angle"](curr_time, vals[3]),
+ ]
+ vals = filtered_vals
+
+ # Wrap heading to [0, 360) after filtering
+ vals[2] = vals[2] % 360
+
+ # Update step counter
+ self.curr_step = self.curr_step + 1
+
+ # Store processed data (only if recording)
+ if self.recording:
+ self.center_x.append(vals[0])
+ self.center_y.append(vals[1])
+ self.heading_direction.append(vals[2])
+ self.head_angle.append(vals[3])
+ self.time_stamp.append(curr_time)
+ self.step.append(self.curr_step)
+ self.frame_time.append(kwargs.get("frame_time", -1))
+ if "pose_time" in kwargs:
+ self.pose_time.append(kwargs["pose_time"])
+
+ # Broadcast processed values to all connected clients
+ payload = [curr_time, vals[0], vals[1], vals[2], vals[3]]
+ self.broadcast(payload)
+
+ return pose
+
+ def get_data(self):
+ """Get logged data including base class data and processed values."""
+ # Get base class data
+ save_dict = super().get_data()
+
+ # Add processed values
+ save_dict["x_pos"] = np.array(self.center_x)
+ save_dict["y_pos"] = np.array(self.center_y)
+ save_dict["heading_direction"] = np.array(self.heading_direction)
+ save_dict["head_angle"] = np.array(self.head_angle)
+ save_dict["use_filter"] = self.use_filter
+ save_dict["filter_kwargs"] = self.filter_kwargs
+
+ return save_dict
+
+
+class MyProcessorTorchmodels_socket(BaseProcessor_socket):
+ """
+ DLC Processor with pose calculations (center, heading, head angle) and optional filtering.
+
+ Calculates:
+ - center: Weighted average of head keypoints
+ - heading: Body orientation (degrees)
+ - head_angle: Head rotation relative to body (radians)
+
+ Broadcasts: [timestamp, center_x, center_y, heading, head_angle]
+ """
+
+ # Metadata for GUI discovery
+ PROCESSOR_NAME = "Mouse Pose with less keypoints"
+ PROCESSOR_DESCRIPTION = (
+ "Calculates mouse center, heading, and head angle with optional One-Euro filtering"
+ )
+ PROCESSOR_PARAMS = {
+ "bind": {
+ "type": "tuple",
+ "default": ("0.0.0.0", 6000),
+ "description": "Server address (host, port)",
+ },
+ "authkey": {
+ "type": "bytes",
+ "default": b"secret password",
+ "description": "Authentication key for clients",
+ },
+ "use_perf_counter": {
+ "type": "bool",
+ "default": False,
+ "description": "Use time.perf_counter() instead of time.time()",
+ },
+ "use_filter": {
+ "type": "bool",
+ "default": False,
+ "description": "Apply One-Euro filter to calculated values",
+ },
+ "filter_kwargs": {
+ "type": "dict",
+ "default": {"min_cutoff": 1.0, "beta": 0.02, "d_cutoff": 1.0},
+ "description": "One-Euro filter parameters (min_cutoff, beta, d_cutoff)",
+ },
+ "save_original": {
+ "type": "bool",
+ "default": False,
+ "description": "Save raw pose arrays for analysis",
+ },
+ }
+
+ def __init__(
+ self,
+ bind=("0.0.0.0", 6000),
+ authkey=b"secret password",
+ use_perf_counter=False,
+ use_filter=False,
+ filter_kwargs={},
+ save_original=False,
+ p_cutoff=0.4,
+ ):
+ """
+ DLC Processor with multi-client broadcasting support.
+
+ Args:
+ bind: (host, port) tuple for server binding
+ authkey: Authentication key for client connections
+ use_perf_counter: If True, use time.perf_counter() instead of time.time()
+ use_filter: If True, apply One-Euro filter to pose data
+ filter_kwargs: Dict with OneEuroFilter parameters (min_cutoff, beta, d_cutoff)
+ save_original: If True, save raw pose arrays
+ """
+ super().__init__(
+ bind=bind,
+ authkey=authkey,
+ use_perf_counter=use_perf_counter,
+ save_original=save_original,
+ )
+
+ # Additional data storage for processed values
+ self.center_x = deque()
+ self.center_y = deque()
+ self.heading_direction = deque()
+ self.head_angle = deque()
+
+ self.p_cutoff = p_cutoff
+
+ # Filtering
+ self.use_filter = use_filter
+ self.filter_kwargs = filter_kwargs
+ self.filters = None # Will be initialized on first pose
+
+ def _clear_data_queues(self):
+ """Clear all data storage queues including pose-specific ones."""
+ super()._clear_data_queues()
+ self.center_x.clear()
+ self.center_y.clear()
+ self.heading_direction.clear()
+ self.head_angle.clear()
+
+ def _initialize_filters(self, vals):
+ """Initialize One-Euro filters for each output variable."""
+ t0 = self.timing_func()
+ self.filters = {
+ "center_x": OneEuroFilter(t0, vals[0], **self.filter_kwargs),
+ "center_y": OneEuroFilter(t0, vals[1], **self.filter_kwargs),
+ "heading": OneEuroFilter(t0, vals[2], **self.filter_kwargs),
+ "head_angle": OneEuroFilter(t0, vals[3], **self.filter_kwargs),
+ }
+ LOG.debug(f"Initialized One-Euro filters with parameters: {self.filter_kwargs}")
+
+ def process(self, pose, **kwargs):
+ """
+ Process pose: calculate center/heading/head_angle, optionally filter, and broadcast.
+
+ Args:
+ pose: DLC pose array (N_keypoints x 3) with [x, y, confidence]
+ **kwargs: Additional metadata (frame_time, pose_time, etc.)
+
+ Returns:
+ pose: Unmodified pose array
+ """
+ # Save original pose if requested (from base class)
+ if self.save_original:
+ self.original_pose.append(pose.copy())
+
+ # Extract keypoints and confidence
+ xy = pose[:, :2]
+ conf = pose[:, 2]
+
+ # Calculate weighted center from head keypoints
+ head_xy = xy[[0, 1, 2, 3, 5, 6, 7], :]
+ head_conf = conf[[0, 1, 2, 3, 5, 6, 7]]
+ # set low confidence keypoints to zero weight
+ head_conf = np.where(head_conf < self.p_cutoff, 0, head_conf)
+ try:
+ center = np.average(head_xy, axis=0, weights=head_conf)
+ except ZeroDivisionError:
+ # If all keypoints have zero weight, return without processing
+ return pose
+
+ neck = np.average(xy[[2, 3, 6, 7], :], axis=0, weights=conf[[2, 3, 6, 7]])
+
+ # Calculate body axis (tail_base -> neck)
+ body_axis = neck - xy[9]
+ body_axis /= sqrt(np.sum(body_axis**2))
+
+ # Calculate head axis (neck -> nose)
+ head_axis = xy[0] - neck
+ head_axis /= sqrt(np.sum(head_axis**2))
+
+ # Calculate head angle relative to body
+ cross = body_axis[0] * head_axis[1] - head_axis[0] * body_axis[1]
+ sign = copysign(1, cross) # Positive when looking left
+ try:
+ head_angle = acos(body_axis @ head_axis) * sign
+ except ValueError:
+ head_angle = 0
+
+ # Calculate heading (body orientation)
+ heading = atan2(body_axis[1], body_axis[0])
+ heading = degrees(heading)
+
+ # Raw values (heading unwrapped for filtering)
+ vals = [center[0], center[1], heading, head_angle]
+
+ # Apply filtering if enabled
+ curr_time = self.timing_func()
+ if self.use_filter:
+ if self.filters is None:
+ self._initialize_filters(vals)
+
+ # Filter each value (heading is filtered in unwrapped space)
+ filtered_vals = [
+ self.filters["center_x"](curr_time, vals[0]),
+ self.filters["center_y"](curr_time, vals[1]),
+ self.filters["heading"](curr_time, vals[2]),
+ self.filters["head_angle"](curr_time, vals[3]),
+ ]
+ vals = filtered_vals
+
+ # Wrap heading to [0, 360) after filtering
+ vals[2] = vals[2] % 360
+
+ # Update step counter
+ self.curr_step = self.curr_step + 1
+
+ # Store processed data (only if recording)
+ if self.recording:
+ self.center_x.append(vals[0])
+ self.center_y.append(vals[1])
+ self.heading_direction.append(vals[2])
+ self.head_angle.append(vals[3])
+ self.time_stamp.append(curr_time)
+ self.step.append(self.curr_step)
+ self.frame_time.append(kwargs.get("frame_time", -1))
+ if "pose_time" in kwargs:
+ self.pose_time.append(kwargs["pose_time"])
+
+ # Broadcast processed values to all connected clients
+ payload = [curr_time, vals[0], vals[1], vals[2], vals[3]]
+ self.broadcast(payload)
+
+ return pose
+
+ def get_data(self):
+ """Get logged data including base class data and processed values."""
+ # Get base class data
+ save_dict = super().get_data()
+
+ # Add processed values
+ save_dict["x_pos"] = np.array(self.center_x)
+ save_dict["y_pos"] = np.array(self.center_y)
+ save_dict["heading_direction"] = np.array(self.heading_direction)
+ save_dict["head_angle"] = np.array(self.head_angle)
+ save_dict["use_filter"] = self.use_filter
+ save_dict["filter_kwargs"] = self.filter_kwargs
+
+ return save_dict
+
+
+# Register processors for GUI discovery
+PROCESSOR_REGISTRY["BaseProcessor_socket"] = BaseProcessor_socket
+PROCESSOR_REGISTRY["MyProcessor_socket"] = MyProcessor_socket
+PROCESSOR_REGISTRY["MyProcessorTorchmodels_socket"] = MyProcessorTorchmodels_socket
+
+
+def get_available_processors():
+ """
+ Get list of available processor classes.
+
+ Returns:
+ dict: Dictionary mapping class names to processor info:
+ {
+ "ClassName": {
+ "class": ProcessorClass,
+ "name": "Display Name",
+ "description": "Description text",
+ "params": {...}
+ }
+ }
+ """
+ processors = {}
+ for class_name, processor_class in PROCESSOR_REGISTRY.items():
+ processors[class_name] = {
+ "class": processor_class,
+ "name": getattr(processor_class, "PROCESSOR_NAME", class_name),
+ "description": getattr(processor_class, "PROCESSOR_DESCRIPTION", ""),
+ "params": getattr(processor_class, "PROCESSOR_PARAMS", {}),
+ }
+ return processors
+
+
+def instantiate_processor(class_name, **kwargs):
+ """
+ Instantiate a processor by class name with given parameters.
+
+ Args:
+ class_name: Name of the processor class (e.g., "MyProcessor_socket")
+ **kwargs: Parameters to pass to the processor constructor
+
+ Returns:
+ Processor instance
+
+ Raises:
+ ValueError: If class_name is not in registry
+ """
+ if class_name not in PROCESSOR_REGISTRY:
+ available = ", ".join(PROCESSOR_REGISTRY.keys())
+ raise ValueError(f"Unknown processor '{class_name}'. Available: {available}")
+
+ processor_class = PROCESSOR_REGISTRY[class_name]
+ return processor_class(**kwargs)
diff --git a/dlclivegui/processors/processor_utils.py b/dlclivegui/processors/processor_utils.py
new file mode 100644
index 0000000..448fa3a
--- /dev/null
+++ b/dlclivegui/processors/processor_utils.py
@@ -0,0 +1,126 @@
+import importlib.util
+import inspect
+from pathlib import Path
+
+
+def load_processors_from_file(file_path):
+ """
+ Load all processor classes from a Python file.
+
+ Args:
+ file_path: Path to Python file containing processors
+
+ Returns:
+ dict: Dictionary of available processors
+ """
+ # Load module from file
+ spec = importlib.util.spec_from_file_location("processors", file_path)
+ module = importlib.util.module_from_spec(spec)
+ spec.loader.exec_module(module)
+
+ # Check if module has get_available_processors function
+ if hasattr(module, "get_available_processors"):
+ return module.get_available_processors()
+
+ # Fallback: scan for Processor subclasses
+ from dlclive import Processor
+
+ processors = {}
+ for name, obj in inspect.getmembers(module, inspect.isclass):
+ if issubclass(obj, Processor) and obj != Processor:
+ processors[name] = {
+ "class": obj,
+ "name": getattr(obj, "PROCESSOR_NAME", name),
+ "description": getattr(obj, "PROCESSOR_DESCRIPTION", ""),
+ "params": getattr(obj, "PROCESSOR_PARAMS", {}),
+ }
+ return processors
+
+
+def scan_processor_folder(folder_path):
+ """
+ Scan a folder for all Python files with processor definitions.
+
+ Args:
+ folder_path: Path to folder containing processor files
+
+ Returns:
+ dict: Dictionary mapping unique processor keys to processor info:
+ {
+ "file_name.py::ClassName": {
+ "class": ProcessorClass,
+ "name": "Display Name",
+ "description": "...",
+ "params": {...},
+ "file": "file_name.py",
+ "class_name": "ClassName"
+ }
+ }
+ """
+ all_processors = {}
+ folder = Path(folder_path)
+
+ for py_file in folder.glob("*.py"):
+ if py_file.name.startswith("_"):
+ continue
+
+ try:
+ processors = load_processors_from_file(py_file)
+ for class_name, processor_info in processors.items():
+ # Create unique key: file::class
+ key = f"{py_file.name}::{class_name}"
+ # Add file and class name to info
+ processor_info["file"] = py_file.name
+ processor_info["class_name"] = class_name
+ processor_info["file_path"] = str(py_file)
+ all_processors[key] = processor_info
+ except Exception as e:
+ print(f"Error loading {py_file}: {e}")
+
+ return all_processors
+
+
+def instantiate_from_scan(processors_dict, processor_key, **kwargs):
+ """
+ Instantiate a processor from scan_processor_folder results.
+
+ Args:
+ processors_dict: Dict returned by scan_processor_folder
+ processor_key: Key like "file.py::ClassName"
+ **kwargs: Parameters for processor constructor
+
+ Returns:
+ Processor instance
+
+ Example:
+ processors = scan_processor_folder("./dlc_processors")
+ processor = instantiate_from_scan(
+ processors,
+ "dlc_processor_socket.py::MyProcessor_socket",
+ use_filter=True
+ )
+ """
+ if processor_key not in processors_dict:
+ available = ", ".join(processors_dict.keys())
+ raise ValueError(f"Unknown processor '{processor_key}'. Available: {available}")
+
+ processor_info = processors_dict[processor_key]
+ processor_class = processor_info["class"]
+ return processor_class(**kwargs)
+
+
+def display_processor_info(processors):
+ """Display processor information in a user-friendly format."""
+ print("\n" + "=" * 70)
+ print("AVAILABLE PROCESSORS")
+ print("=" * 70)
+
+ for idx, (class_name, info) in enumerate(processors.items(), 1):
+ print(f"\n[{idx}] {info['name']}")
+ print(f" Class: {class_name}")
+ print(f" Description: {info['description']}")
+ print(f" Parameters:")
+ for param_name, param_info in info["params"].items():
+ print(f" - {param_name} ({param_info['type']})")
+ print(f" Default: {param_info['default']}")
+ print(f" {param_info['description']}")
diff --git a/dlclivegui/queue.py b/dlclivegui/queue.py
deleted file mode 100644
index 59bc43c..0000000
--- a/dlclivegui/queue.py
+++ /dev/null
@@ -1,208 +0,0 @@
-import multiprocess as mp
-from multiprocess import queues
-from queue import Queue, Empty, Full
-
-
-class QueuePositionError(Exception):
- """ Error in position argument of queue read """
-
- pass
-
-
-class ClearableQueue(Queue):
- """ A Queue that provides safe methods for writing to a full queue, reading to an empty queue, and a method to clear the queue """
-
- def __init__(self, maxsize=0):
-
- super().__init__(maxsize)
-
- def clear(self):
- """ Clears queue, returns all objects in a list
-
- Returns
- -------
- list
- list of objects from the queue
- """
-
- objs = []
-
- try:
- while True:
- objs.append(self.get_nowait())
- except Empty:
- pass
-
- return objs
-
- def write(self, obj, clear=False):
- """ Puts an object in the queue, with an option to clear queue before writing.
-
- Parameters
- ----------
- obj : [type]
- An object to put in the queue
- clear : bool, optional
- flag to clear queue before putting, by default False
-
- Returns
- -------
- bool
- if write was sucessful, returns True
- """
-
- if clear:
- self.clear()
-
- try:
- self.put_nowait(obj)
- success = True
- except Full:
- success = False
-
- return success
-
- def read(self, clear=False, position="last"):
- """ Gets an object in the queue, with the option to clear the queue and return the first element, last element, or all elements
-
- Parameters
- ----------
- clear : bool, optional
- flag to clear queue before putting, by default False
- position : str, optional
- If clear is True, returned object depends on position.
- If position = "last", returns last object.
- If position = "first", returns first object.
- If position = "all", returns all objects from the queue.
-
- Returns
- -------
- object
- object retrieved from the queue
- """
-
- obj = None
-
- if clear:
-
- objs = self.clear()
-
- if len(objs) > 0:
- if position == "first":
- obj = objs[0]
- elif position == "last":
- obj = objs[-1]
- elif position == "all":
- obj = objs
- else:
- raise QueuePositionError(
- "Queue read position should be one of 'first', 'last', or 'all'"
- )
- else:
-
- try:
- obj = self.get_nowait()
- except Empty:
- pass
-
- return obj
-
-
-class ClearableMPQueue(mp.queues.Queue):
- """ A multiprocess Queue that provides safe methods for writing to a full queue, reading to an empty queue, and a method to clear the queue """
-
- def __init__(self, maxsize=0, ctx=mp.get_context("spawn")):
-
- super().__init__(maxsize, ctx=ctx)
-
- def clear(self):
- """ Clears queue, returns all objects in a list
-
- Returns
- -------
- list
- list of objects from the queue
- """
-
- objs = []
-
- try:
- while True:
- objs.append(self.get_nowait())
- except Empty:
- pass
-
- return objs
-
- def write(self, obj, clear=False):
- """ Puts an object in the queue, with an option to clear queue before writing.
-
- Parameters
- ----------
- obj : [type]
- An object to put in the queue
- clear : bool, optional
- flag to clear queue before putting, by default False
-
- Returns
- -------
- bool
- if write was sucessful, returns True
- """
-
- if clear:
- self.clear()
-
- try:
- self.put_nowait(obj)
- success = True
- except Full:
- success = False
-
- return success
-
- def read(self, clear=False, position="last"):
- """ Gets an object in the queue, with the option to clear the queue and return the first element, last element, or all elements
-
- Parameters
- ----------
- clear : bool, optional
- flag to clear queue before putting, by default False
- position : str, optional
- If clear is True, returned object depends on position.
- If position = "last", returns last object.
- If position = "first", returns first object.
- If position = "all", returns all objects from the queue.
-
- Returns
- -------
- object
- object retrieved from the queue
- """
-
- obj = None
-
- if clear:
-
- objs = self.clear()
-
- if len(objs) > 0:
- if position == "first":
- obj = objs[0]
- elif position == "last":
- obj = objs[-1]
- elif position == "all":
- obj = objs
- else:
- raise QueuePositionError(
- "Queue read position should be one of 'first', 'last', or 'all'"
- )
-
- else:
-
- try:
- obj = self.get_nowait()
- except Empty:
- pass
-
- return obj
diff --git a/dlclivegui/services/dlc_processor.py b/dlclivegui/services/dlc_processor.py
new file mode 100644
index 0000000..7d5776a
--- /dev/null
+++ b/dlclivegui/services/dlc_processor.py
@@ -0,0 +1,458 @@
+"""DLCLive integration helpers."""
+
+# dlclivegui/services/dlc_processor.py
+from __future__ import annotations
+
+import logging
+import queue
+import threading
+import time
+from collections import deque
+from contextlib import contextmanager
+from dataclasses import dataclass
+from typing import Any
+
+import numpy as np
+from PySide6.QtCore import QObject, Signal
+
+# from dlclivegui.config import DLCProcessorSettings
+from dlclivegui.processors.processor_utils import instantiate_from_scan
+from dlclivegui.utils.config_models import DLCProcessorSettingsModel
+
+logger = logging.getLogger(__name__)
+
+# Enable profiling
+ENABLE_PROFILING = True
+
+try: # pragma: no cover - optional dependency
+ from dlclive import DLCLive # type: ignore
+except Exception as e: # pragma: no cover - handled gracefully
+ logger.error(f"dlclive package could not be imported: {e}")
+ DLCLive = None # type: ignore[assignment]
+
+
+@dataclass
+class PoseResult:
+ pose: np.ndarray | None
+ timestamp: float
+
+
+@dataclass
+class ProcessorStats:
+ """Statistics for DLC processor performance."""
+
+ frames_enqueued: int = 0
+ frames_processed: int = 0
+ frames_dropped: int = 0
+ queue_size: int = 0
+ processing_fps: float = 0.0
+ average_latency: float = 0.0
+ last_latency: float = 0.0
+ # Profiling metrics
+ avg_queue_wait: float = 0.0
+ avg_inference_time: float = 0.0
+ avg_signal_emit_time: float = 0.0
+ avg_total_process_time: float = 0.0
+ # Separated timing for GPU vs socket processor
+ avg_gpu_inference_time: float = 0.0 # Pure model inference
+ avg_processor_overhead: float = 0.0 # Socket processor overhead
+
+
+# _SENTINEL = object()
+
+
+class DLCLiveProcessor(QObject):
+ """Background pose estimation using DLCLive with queue-based threading."""
+
+ pose_ready = Signal(object)
+ error = Signal(str)
+ initialized = Signal(bool)
+ frame_processed = Signal()
+
+ def __init__(self) -> None:
+ super().__init__()
+ self._settings = DLCProcessorSettingsModel()
+ self._dlc: Any | None = None
+ self._processor: Any | None = None
+ self._queue: queue.Queue[Any] | None = None
+ self._worker_thread: threading.Thread | None = None
+ self._stop_event = threading.Event()
+ self._initialized = False
+
+ # Statistics tracking
+ self._frames_enqueued = 0
+ self._frames_processed = 0
+ self._frames_dropped = 0
+ self._latencies: deque[float] = deque(maxlen=60)
+ self._processing_times: deque[float] = deque(maxlen=60)
+ self._stats_lock = threading.Lock()
+
+ # Profiling metrics
+ self._queue_wait_times: deque[float] = deque(maxlen=60)
+ self._inference_times: deque[float] = deque(maxlen=60)
+ self._signal_emit_times: deque[float] = deque(maxlen=60)
+ self._total_process_times: deque[float] = deque(maxlen=60)
+ self._gpu_inference_times: deque[float] = deque(maxlen=60)
+ self._processor_overhead_times: deque[float] = deque(maxlen=60)
+
+ def configure(self, settings: DLCProcessorSettingsModel, processor: Any | None = None) -> None:
+ self._settings = settings
+ self._processor = processor
+
+ def reset(self) -> None:
+ """Stop the worker thread and drop the current DLCLive instance."""
+ self._stop_worker()
+ self._dlc = None
+ self._initialized = False
+ with self._stats_lock:
+ self._frames_enqueued = 0
+ self._frames_processed = 0
+ self._frames_dropped = 0
+ self._latencies.clear()
+ self._processing_times.clear()
+ self._queue_wait_times.clear()
+ self._inference_times.clear()
+ self._signal_emit_times.clear()
+ self._total_process_times.clear()
+ self._gpu_inference_times.clear()
+ self._processor_overhead_times.clear()
+
+ def shutdown(self) -> None:
+ self._stop_worker()
+ self._dlc = None
+ self._initialized = False
+
+ def enqueue_frame(self, frame: np.ndarray, timestamp: float) -> None:
+ # Start worker on first frame
+ if self._worker_thread is None:
+ self._start_worker(frame.copy(), timestamp)
+ return
+
+ # As long as worker and queue are ready, ALWAYS enqueue
+ if self._queue is None:
+ return
+
+ try:
+ self._queue.put_nowait((frame.copy(), timestamp, time.perf_counter()))
+ with self._stats_lock:
+ self._frames_enqueued += 1
+ except queue.Full:
+ with self._stats_lock:
+ self._frames_dropped += 1
+
+ def get_stats(self) -> ProcessorStats:
+ """Get current processing statistics."""
+ queue_size = self._queue.qsize() if self._queue is not None else 0
+
+ with self._stats_lock:
+ avg_latency = sum(self._latencies) / len(self._latencies) if self._latencies else 0.0
+ last_latency = self._latencies[-1] if self._latencies else 0.0
+
+ # Compute processing FPS from processing times
+ if len(self._processing_times) >= 2:
+ duration = self._processing_times[-1] - self._processing_times[0]
+ processing_fps = (len(self._processing_times) - 1) / duration if duration > 0 else 0.0
+ else:
+ processing_fps = 0.0
+
+ # Profiling metrics
+ avg_queue_wait = (
+ sum(self._queue_wait_times) / len(self._queue_wait_times) if self._queue_wait_times else 0.0
+ )
+ avg_inference = sum(self._inference_times) / len(self._inference_times) if self._inference_times else 0.0
+ avg_signal_emit = (
+ sum(self._signal_emit_times) / len(self._signal_emit_times) if self._signal_emit_times else 0.0
+ )
+ avg_total = (
+ sum(self._total_process_times) / len(self._total_process_times) if self._total_process_times else 0.0
+ )
+ avg_gpu = (
+ sum(self._gpu_inference_times) / len(self._gpu_inference_times) if self._gpu_inference_times else 0.0
+ )
+ avg_proc_overhead = (
+ sum(self._processor_overhead_times) / len(self._processor_overhead_times)
+ if self._processor_overhead_times
+ else 0.0
+ )
+
+ return ProcessorStats(
+ frames_enqueued=self._frames_enqueued,
+ frames_processed=self._frames_processed,
+ frames_dropped=self._frames_dropped,
+ queue_size=queue_size,
+ processing_fps=processing_fps,
+ average_latency=avg_latency,
+ last_latency=last_latency,
+ avg_queue_wait=avg_queue_wait,
+ avg_inference_time=avg_inference,
+ avg_signal_emit_time=avg_signal_emit,
+ avg_total_process_time=avg_total,
+ avg_gpu_inference_time=avg_gpu,
+ avg_processor_overhead=avg_proc_overhead,
+ )
+
+ def _start_worker(self, init_frame: np.ndarray, init_timestamp: float) -> None:
+ if self._worker_thread is not None and self._worker_thread.is_alive():
+ return
+
+ self._queue = queue.Queue(maxsize=1)
+ self._stop_event.clear()
+ self._worker_thread = threading.Thread(
+ target=self._worker_loop,
+ args=(init_frame, init_timestamp),
+ name="DLCLiveWorker",
+ daemon=True,
+ )
+ self._worker_thread.start()
+
+ def _stop_worker(self) -> None:
+ if self._worker_thread is None:
+ return
+
+ self._stop_event.set()
+
+ # Just wait for the timed get() loop to observe the flag and drain
+ self._worker_thread.join(timeout=2.0)
+ if self._worker_thread.is_alive():
+ logger.warning("DLC worker thread did not terminate cleanly")
+
+ self._worker_thread = None
+ self._queue = None
+
+ @contextmanager
+ def _timed_processor(self):
+ """
+ If a socket processor is attached, temporarily wrap its .process()
+ to measure processor overhead time independently of GPU inference.
+ Yields a one-element list [processor_overhead_seconds] or None when no processor.
+ Always restores the original .process reference.
+ """
+ if self._processor is None:
+ yield None
+ return
+
+ original = self._processor.process
+ holder = [0.0]
+
+ def timed_process(pose, _op=original, _holder=holder, **kwargs):
+ start = time.perf_counter()
+ try:
+ return _op(pose, **kwargs)
+ finally:
+ _holder[0] = time.perf_counter() - start
+
+ self._processor.process = timed_process
+ try:
+ yield holder
+ finally:
+ # Restore even if inference/errors occur
+ self._processor.process = original
+
+ def _process_frame(
+ self,
+ frame: np.ndarray,
+ timestamp: float,
+ enqueue_time: float,
+ *,
+ queue_wait_time: float = 0.0,
+ ) -> None:
+ """
+ Single source of truth for: inference -> (optional) processor timing -> signal emit -> stats.
+ Updates: frames_processed, latency, processing timeline, profiling metrics.
+ """
+ # Time GPU inference (and processor overhead when present)
+ with self._timed_processor() as proc_holder:
+ inference_start = time.perf_counter()
+ pose = self._dlc.get_pose(frame, frame_time=timestamp)
+ inference_time = time.perf_counter() - inference_start
+
+ processor_overhead = 0.0
+ gpu_inference_time = inference_time
+ if proc_holder is not None:
+ processor_overhead = proc_holder[0]
+ gpu_inference_time = max(0.0, inference_time - processor_overhead)
+
+ # Emit pose (measure signal overhead)
+ signal_start = time.perf_counter()
+ self.pose_ready.emit(PoseResult(pose=pose, timestamp=timestamp))
+ signal_time = time.perf_counter() - signal_start
+
+ end_ts = time.perf_counter()
+ latency = end_ts - enqueue_time
+ total_process_time = end_ts - (end_ts - (inference_time + signal_time)) # keep for completeness
+
+ with self._stats_lock:
+ self._frames_processed += 1
+ self._latencies.append(latency)
+ self._processing_times.append(end_ts)
+ if ENABLE_PROFILING:
+ self._queue_wait_times.append(queue_wait_time)
+ self._inference_times.append(inference_time)
+ self._signal_emit_times.append(signal_time)
+ self._total_process_times.append(total_process_time)
+ self._gpu_inference_times.append(gpu_inference_time)
+ self._processor_overhead_times.append(processor_overhead)
+
+ self.frame_processed.emit()
+
+ def _worker_loop(self, init_frame: np.ndarray, init_timestamp: float) -> None:
+ try:
+ # -------- Initialization (unchanged) --------
+ if DLCLive is None:
+ raise RuntimeError("The 'dlclive' package is required for pose estimation.")
+ if not self._settings.model_path:
+ raise RuntimeError("No DLCLive model path configured.")
+
+ init_start = time.perf_counter()
+ dyn = self._settings.dynamic
+ if not isinstance(dyn, (list, tuple)) or len(dyn) != 3:
+ try:
+ dyn = dyn.to_tuple()
+ except Exception as e:
+ raise RuntimeError("Invalid dynamic crop settings format.") from e
+ enabled, margin, max_missing = dyn
+
+ options = {
+ "model_path": self._settings.model_path,
+ "model_type": self._settings.model_type,
+ "processor": self._processor,
+ "dynamic": [enabled, margin, max_missing],
+ "resize": self._settings.resize,
+ "precision": self._settings.precision,
+ "single_animal": self._settings.single_animal,
+ }
+ if self._settings.device is not None:
+ options["device"] = self._settings.device
+
+ self._dlc = DLCLive(**options)
+
+ # First inference to initialize
+ init_inference_start = time.perf_counter()
+ self._dlc.init_inference(init_frame)
+ init_inference_time = time.perf_counter() - init_inference_start
+
+ self._initialized = True
+ self.initialized.emit(True)
+
+ total_init_time = time.perf_counter() - init_start
+ logger.info(
+ "DLCLive model initialized successfully (total: %.3fs, init_inference: %.3fs)",
+ total_init_time,
+ init_inference_time,
+ )
+
+ # Emit pose for init frame & update stats (not dequeued)
+ self._process_frame(init_frame, init_timestamp, time.perf_counter(), queue_wait_time=0.0)
+ with self._stats_lock:
+ self._frames_enqueued += 1
+
+ except Exception as exc:
+ logger.exception("Failed to initialize DLCLive", exc_info=exc)
+ self.error.emit(str(exc))
+ self.initialized.emit(False)
+ return
+
+ # -------- Main processing loop: stop-flag + timed get + drain --------
+ # NOTE: We never exit early unless _stop_event is set.
+ while True:
+ # If stop requested, only exit when queue is empty
+ if self._stop_event.is_set():
+ if self._queue is not None:
+ try:
+ frame, ts, enq = self._queue.get_nowait()
+ except queue.Empty:
+ # NOW it is safe to exit
+ break
+ else:
+ # Still work to do, process one
+ try:
+ self._process_frame(frame, ts, enq, queue_wait_time=0.0)
+ except Exception as exc:
+ logger.exception("Pose inference failed", exc_info=exc)
+ self.error.emit(str(exc))
+ finally:
+ try:
+ self._queue.task_done()
+ except ValueError:
+ pass
+ continue # check stop_event again WITHOUT breaking
+
+ # Normal operation: timed get
+ try:
+ wait_start = time.perf_counter()
+ item = self._queue.get(timeout=0.05)
+ queue_wait_time = time.perf_counter() - wait_start
+ except queue.Empty:
+ continue
+
+ try:
+ frame, ts, enq = item
+ self._process_frame(frame, ts, enq, queue_wait_time=queue_wait_time)
+ except Exception as exc:
+ logger.exception("Pose inference failed", exc_info=exc)
+ self.error.emit(str(exc))
+ finally:
+ try:
+ self._queue.task_done()
+ except ValueError:
+ pass
+
+ logger.info("DLC worker thread exiting")
+
+
+class DLCService:
+ """Wrap DLCLiveProcessor lifecycle & configuration."""
+
+ def __init__(self):
+ self._proc = DLCLiveProcessor()
+ self.active = False
+ self._last_pose: PoseResult | None = None
+ self._processor_info = None
+
+ @property
+ def processor(self):
+ return self._proc._processor
+
+ # Expose key signals (to let MainWindow connect easily)
+ @property
+ def pose_ready(self):
+ return self._proc.pose_ready
+
+ @property
+ def error(self):
+ return self._proc.error
+
+ @property
+ def initialized(self):
+ return self._proc.initialized
+
+ def enqueue(self, frame, ts):
+ self._proc.enqueue_frame(frame, ts)
+
+ def configure(self, settings: DLCProcessorSettingsModel, scanned_processors: dict, selected_key) -> bool:
+ processor = None
+ if selected_key is not None and scanned_processors:
+ try:
+ processor = instantiate_from_scan(scanned_processors, selected_key)
+ except Exception as exc:
+ logger.error("Failed to instantiate processor: %s", exc)
+ return False
+ self._proc.configure(settings, processor=processor)
+ return True
+
+ def start(self):
+ self._proc.reset()
+ self.active = True
+ self.initialized = False
+
+ def stop(self):
+ self.active = False
+ self.initialized = False
+ self._proc.reset()
+ self._last_pose = None
+
+ def stats(self) -> ProcessorStats:
+ return self._proc.get_stats()
+
+ def last_pose(self) -> PoseResult | None:
+ return self._last_pose
diff --git a/dlclivegui/services/multi_camera_controller.py b/dlclivegui/services/multi_camera_controller.py
new file mode 100644
index 0000000..e1a1d28
--- /dev/null
+++ b/dlclivegui/services/multi_camera_controller.py
@@ -0,0 +1,440 @@
+"""Multi-camera management for the DLC Live GUI."""
+
+from __future__ import annotations
+
+import logging
+import time
+from dataclasses import dataclass
+from threading import Event, Lock
+
+import cv2
+import numpy as np
+from PySide6.QtCore import QObject, QThread, Signal, Slot
+
+from dlclivegui.cameras import CameraFactory
+from dlclivegui.cameras.base import CameraBackend
+
+# from dlclivegui.config import CameraSettings
+from dlclivegui.utils.config_models import CameraSettingsModel
+
+LOGGER = logging.getLogger(__name__)
+
+
+@dataclass
+class MultiFrameData:
+ """Container for frames from multiple cameras."""
+
+ frames: dict[str, np.ndarray] # camera_id -> frame
+ timestamps: dict[str, float] # camera_id -> timestamp
+ source_camera_id: str = "" # ID of camera that triggered this emission
+ tiled_frame: np.ndarray | None = None # Combined tiled frame (deprecated, done in GUI)
+
+
+class SingleCameraWorker(QObject):
+ """Worker for a single camera in multi-camera mode."""
+
+ frame_captured = Signal(str, object, float) # camera_id, frame, timestamp
+ error_occurred = Signal(str, str) # camera_id, error_message
+ started = Signal(str) # camera_id
+ stopped = Signal(str) # camera_id
+
+ def __init__(self, camera_id: str, settings: CameraSettingsModel):
+ super().__init__()
+ self._camera_id = camera_id
+ self._settings = settings
+ self._stop_event = Event()
+ self._backend: CameraBackend | None = None
+ self._max_consecutive_errors = 5
+ self._retry_delay = 0.1
+
+ @Slot()
+ def run(self) -> None:
+ self._stop_event.clear()
+
+ try:
+ self._backend = CameraFactory.create(self._settings)
+ self._backend.open()
+ except Exception as exc:
+ LOGGER.exception(f"Failed to initialize camera {self._camera_id}", exc_info=exc)
+ self.error_occurred.emit(self._camera_id, f"Failed to initialize camera: {exc}")
+ self.stopped.emit(self._camera_id)
+ return
+
+ self.started.emit(self._camera_id)
+ consecutive_errors = 0
+
+ while not self._stop_event.is_set():
+ try:
+ frame, timestamp = self._backend.read()
+ if frame is None or frame.size == 0:
+ consecutive_errors += 1
+ if consecutive_errors >= self._max_consecutive_errors:
+ self.error_occurred.emit(
+ self._camera_id, "Too many empty frames.\nWas the device disconnected ?"
+ )
+ break
+ time.sleep(self._retry_delay)
+ continue
+
+ consecutive_errors = 0
+ self.frame_captured.emit(self._camera_id, frame, timestamp)
+
+ except Exception as exc:
+ consecutive_errors += 1
+ if self._stop_event.is_set():
+ break
+ if consecutive_errors >= self._max_consecutive_errors:
+ self.error_occurred.emit(self._camera_id, f"Camera read error: {exc}")
+ break
+ time.sleep(self._retry_delay)
+ continue
+
+ # Cleanup
+ if self._backend is not None:
+ try:
+ self._backend.close()
+ except Exception:
+ pass
+ self.stopped.emit(self._camera_id)
+
+ def stop(self) -> None:
+ self._stop_event.set()
+
+
+def get_camera_id(settings: CameraSettingsModel) -> str:
+ """Generate a unique camera ID from settings."""
+ return f"{settings.backend}:{settings.index}"
+
+
+class MultiCameraController(QObject):
+ """Controller for managing multiple cameras simultaneously."""
+
+ # Signals
+ frame_ready = Signal(object) # MultiFrameData
+ camera_started = Signal(str, object) # camera_id, settings
+ camera_stopped = Signal(str) # camera_id
+ camera_error = Signal(str, str) # camera_id, error_message
+ all_started = Signal()
+ all_stopped = Signal()
+ initialization_failed = Signal(list) # List of (camera_id, error_message) tuples
+
+ MAX_CAMERAS = 4
+
+ def __init__(self):
+ super().__init__()
+ self._workers: dict[str, SingleCameraWorker] = {}
+ self._threads: dict[str, QThread] = {}
+ self._settings: dict[str, CameraSettingsModel] = {}
+ self._frames: dict[str, np.ndarray] = {}
+ self._timestamps: dict[str, float] = {}
+ self._frame_lock = Lock()
+ self._running = False
+ self._started_cameras: set = set()
+ self._failed_cameras: dict[str, str] = {} # camera_id -> error message
+ self._expected_cameras: int = 0 # Number of cameras we're trying to start
+
+ def is_running(self) -> bool:
+ """Check if any camera is currently running."""
+ return self._running and len(self._started_cameras) > 0
+
+ def get_active_count(self) -> int:
+ """Get the number of active cameras."""
+ return len(self._started_cameras)
+
+ def start(self, camera_settings: list[CameraSettingsModel]) -> None:
+ """Start multiple cameras; accepts dataclasses, pydantic models, or dicts."""
+ if self._running:
+ LOGGER.warning("Multi-camera controller already running")
+ return
+
+ active_settings = [s for s in camera_settings if s.enabled][: self.MAX_CAMERAS]
+ if not active_settings:
+ LOGGER.warning("No active cameras to start")
+ return
+
+ self._running = True
+ self._frames.clear()
+ self._timestamps.clear()
+ self._started_cameras.clear()
+ self._failed_cameras.clear()
+ self._expected_cameras = len(active_settings)
+
+ for settings in active_settings:
+ self._start_camera(settings)
+
+ def _start_camera(self, settings: CameraSettingsModel) -> None:
+ """Start a single camera."""
+ cam_id = get_camera_id(settings)
+ if cam_id in self._workers:
+ LOGGER.warning(f"Camera {cam_id} already has a worker")
+ return
+
+ # Normalize and store the dataclass once
+ self._settings[cam_id] = settings
+ dc = self._settings[cam_id]
+ worker = SingleCameraWorker(cam_id, dc)
+ thread = QThread()
+ worker.moveToThread(thread)
+
+ # Connections unchanged
+ thread.started.connect(worker.run)
+ worker.frame_captured.connect(self._on_frame_captured)
+ worker.started.connect(self._on_camera_started)
+ worker.stopped.connect(self._on_camera_stopped)
+ worker.error_occurred.connect(self._on_camera_error)
+
+ self._workers[cam_id] = worker
+ self._threads[cam_id] = thread
+ thread.start()
+
+ def stop(self, wait: bool = True) -> None:
+ """Stop all cameras."""
+ if not self._running:
+ return
+
+ self._running = False
+
+ # Signal all workers to stop
+ for worker in self._workers.values():
+ worker.stop()
+
+ # Wait for threads to finish
+ if wait:
+ for thread in self._threads.values():
+ if thread.isRunning():
+ thread.quit()
+ thread.wait(5000)
+
+ self._workers.clear()
+ self._threads.clear()
+ self._settings.clear()
+ self._started_cameras.clear()
+ self._failed_cameras.clear()
+ self._expected_cameras = 0
+ self.all_stopped.emit()
+
+ def _on_frame_captured(self, camera_id: str, frame: np.ndarray, timestamp: float) -> None:
+ """Handle a frame from one camera."""
+ # Apply rotation if configured
+ settings = self._settings.get(camera_id)
+ if settings and settings.rotation:
+ frame = self._apply_rotation(frame, settings.rotation)
+
+ # Apply cropping if configured
+ if settings:
+ crop_region = settings.get_crop_region()
+ if crop_region:
+ frame = self._apply_crop(frame, crop_region)
+
+ with self._frame_lock:
+ self._frames[camera_id] = frame
+ self._timestamps[camera_id] = timestamp
+
+ # Emit frame data without tiling (tiling done in GUI for performance)
+ if self._frames:
+ frame_data = MultiFrameData(
+ frames=dict(self._frames),
+ timestamps=dict(self._timestamps),
+ source_camera_id=camera_id, # Track which camera triggered this
+ tiled_frame=None,
+ )
+ self.frame_ready.emit(frame_data)
+
+ def _apply_rotation(self, frame: np.ndarray, degrees: int) -> np.ndarray:
+ """Apply rotation to frame."""
+ if degrees == 90:
+ return cv2.rotate(frame, cv2.ROTATE_90_CLOCKWISE)
+ elif degrees == 180:
+ return cv2.rotate(frame, cv2.ROTATE_180)
+ elif degrees == 270:
+ return cv2.rotate(frame, cv2.ROTATE_90_COUNTERCLOCKWISE)
+ return frame
+
+ def _apply_crop(self, frame: np.ndarray, crop_region: tuple[int, int, int, int]) -> np.ndarray:
+ """Apply crop to frame."""
+ x0, y0, x1, y1 = crop_region
+ height, width = frame.shape[:2]
+
+ x0 = max(0, min(x0, width))
+ y0 = max(0, min(y0, height))
+ x1 = max(x0, min(x1, width)) if x1 > 0 else width
+ y1 = max(y0, min(y1, height)) if y1 > 0 else height
+
+ if x0 < x1 and y0 < y1:
+ return frame[y0:y1, x0:x1]
+ return frame
+
+ def _create_tiled_frame(self) -> np.ndarray:
+ """Create a tiled frame from all camera frames.
+
+ The tiled frame is scaled to fit within a maximum canvas size
+ while maintaining aspect ratio of individual camera frames.
+ """
+ if not self._frames:
+ return np.zeros((480, 640, 3), dtype=np.uint8)
+
+ frames_list = [self._frames[idx] for idx in sorted(self._frames.keys())]
+ num_frames = len(frames_list)
+
+ if num_frames == 0:
+ return np.zeros((480, 640, 3), dtype=np.uint8)
+
+ # Determine grid layout
+ if num_frames == 1:
+ rows, cols = 1, 1
+ elif num_frames == 2:
+ rows, cols = 1, 2
+ elif num_frames <= 4:
+ rows, cols = 2, 2
+ else:
+ rows, cols = 2, 2 # Limit to 4
+
+ # Maximum canvas size to fit on screen (leaving room for UI elements)
+ max_canvas_width = 1200
+ max_canvas_height = 800
+
+ # Calculate tile size based on frame aspect ratio and available space
+ first_frame = frames_list[0]
+ frame_h, frame_w = first_frame.shape[:2]
+ frame_aspect = frame_w / frame_h if frame_h > 0 else 1.0
+
+ # Calculate tile dimensions that fit within the canvas
+ tile_w = max_canvas_width // cols
+ tile_h = max_canvas_height // rows
+
+ # Maintain aspect ratio of original frames
+ tile_aspect = tile_w / tile_h if tile_h > 0 else 1.0
+
+ if frame_aspect > tile_aspect:
+ # Frame is wider than tile slot - constrain by width
+ tile_h = int(tile_w / frame_aspect)
+ else:
+ # Frame is taller than tile slot - constrain by height
+ tile_w = int(tile_h * frame_aspect)
+
+ # Ensure minimum size
+ tile_w = max(160, tile_w)
+ tile_h = max(120, tile_h)
+
+ # Create canvas
+ canvas = np.zeros((rows * tile_h, cols * tile_w, 3), dtype=np.uint8)
+
+ # Get sorted camera IDs for consistent ordering
+ cam_ids = sorted(self._frames.keys())
+ frames_list = [self._frames[cam_id] for cam_id in cam_ids]
+
+ # Place each frame in the grid
+ for idx, frame in enumerate(frames_list[: rows * cols]):
+ row = idx // cols
+ col = idx % cols
+
+ # Ensure frame is 3-channel
+ if frame.ndim == 2:
+ frame = cv2.cvtColor(frame, cv2.COLOR_GRAY2BGR)
+ elif frame.shape[2] == 4:
+ frame = cv2.cvtColor(frame, cv2.COLOR_BGRA2BGR)
+
+ # Resize to tile size
+ resized = cv2.resize(frame, (tile_w, tile_h))
+
+ # Add camera ID label
+ if idx < len(cam_ids):
+ label = cam_ids[idx]
+ cv2.putText(
+ resized,
+ label,
+ (10, 30),
+ cv2.FONT_HERSHEY_SIMPLEX,
+ 0.7,
+ (0, 255, 0),
+ 2,
+ )
+
+ # Place in canvas
+ y_start = row * tile_h
+ y_end = y_start + tile_h
+ x_start = col * tile_w
+ x_end = x_start + tile_w
+ canvas[y_start:y_end, x_start:x_end] = resized
+
+ return canvas
+
+ def _on_camera_started(self, camera_id: str) -> None:
+ """Handle camera start event."""
+ self._started_cameras.add(camera_id)
+ settings = self._settings.get(camera_id)
+ self.camera_started.emit(camera_id, settings)
+ LOGGER.info(f"Camera {camera_id} started")
+
+ # Check if all cameras have reported (started or failed)
+ total_reported = len(self._started_cameras) + len(self._failed_cameras)
+ if total_reported == self._expected_cameras:
+ if self._started_cameras:
+ # At least some cameras started successfully
+ self.all_started.emit()
+ # If no cameras started but all failed, that's handled in _on_camera_stopped
+
+ def _on_camera_stopped(self, camera_id: str) -> None:
+ """Handle camera stop event."""
+ # Check if this camera never started (initialization failure)
+ was_started = camera_id in self._started_cameras
+ self._started_cameras.discard(camera_id)
+ self.camera_stopped.emit(camera_id)
+ LOGGER.info(f"Camera {camera_id} stopped (was_started={was_started})")
+
+ # Cleanup thread
+ if camera_id in self._threads:
+ thread = self._threads[camera_id]
+ if thread.isRunning():
+ thread.quit()
+ thread.wait(1000)
+ del self._threads[camera_id]
+
+ if camera_id in self._workers:
+ del self._workers[camera_id]
+
+ # Remove frame data
+ with self._frame_lock:
+ self._frames.pop(camera_id, None)
+ self._timestamps.pop(camera_id, None)
+
+ # Check if all cameras have reported and none started
+ total_reported = len(self._started_cameras) + len(self._failed_cameras)
+ if total_reported == self._expected_cameras and not self._started_cameras:
+ # All cameras failed to start
+ if self._running and self._failed_cameras:
+ self._running = False
+ failure_list = list(self._failed_cameras.items())
+ self.initialization_failed.emit(failure_list)
+ self.all_stopped.emit()
+ return
+
+ # Check if all running cameras have stopped (normal shutdown)
+ if not self._started_cameras and self._running and not self._workers:
+ self._running = False
+ self.all_stopped.emit()
+
+ def _on_camera_error(self, camera_id: str, message: str) -> None:
+ """Handle camera error event."""
+ LOGGER.error(f"Camera {camera_id} error: {message}")
+ # Track failed cameras (only if not already started - i.e., initialization failure)
+ if camera_id not in self._started_cameras:
+ self._failed_cameras[camera_id] = message
+ self.camera_error.emit(camera_id, message)
+
+ def get_frame(self, camera_id: str) -> np.ndarray | None:
+ """Get the latest frame from a specific camera."""
+ with self._frame_lock:
+ return self._frames.get(camera_id)
+
+ def get_all_frames(self) -> dict[str, np.ndarray]:
+ """Get the latest frames from all cameras."""
+ with self._frame_lock:
+ return dict(self._frames)
+
+ def get_tiled_frame(self) -> np.ndarray | None:
+ """Get a tiled view of all camera frames."""
+ with self._frame_lock:
+ if self._frames:
+ return self._create_tiled_frame()
+ return None
diff --git a/dlclivegui/services/video_recorder.py b/dlclivegui/services/video_recorder.py
new file mode 100644
index 0000000..92e8b05
--- /dev/null
+++ b/dlclivegui/services/video_recorder.py
@@ -0,0 +1,329 @@
+"""Video recording support using the vidgear library."""
+
+from __future__ import annotations
+
+import json
+import logging
+import queue
+import threading
+import time
+from collections import deque
+from dataclasses import dataclass
+from pathlib import Path
+from typing import Any
+
+import numpy as np
+
+try:
+ from vidgear.gears import WriteGear
+except ImportError: # pragma: no cover - handled at runtime
+ WriteGear = None # type: ignore[assignment]
+
+
+logger = logging.getLogger(__name__)
+
+
+@dataclass
+class RecorderStats:
+ """Snapshot of recorder throughput metrics."""
+
+ frames_enqueued: int = 0
+ frames_written: int = 0
+ dropped_frames: int = 0
+ queue_size: int = 0
+ average_latency: float = 0.0
+ last_latency: float = 0.0
+ write_fps: float = 0.0
+ buffer_seconds: float = 0.0
+
+
+_SENTINEL = object()
+
+
+class VideoRecorder:
+ """Thin wrapper around :class:`vidgear.gears.WriteGear`."""
+
+ def __init__(
+ self,
+ output: Path | str,
+ frame_size: tuple[int, int] | None = None,
+ frame_rate: float | None = None,
+ codec: str = "libx264",
+ crf: int = 23,
+ buffer_size: int = 240,
+ ):
+ self._output = Path(output)
+ self._writer: Any | None = None
+ self._frame_size = frame_size
+ self._frame_rate = frame_rate
+ self._codec = codec
+ self._crf = int(crf)
+ self._buffer_size = max(1, int(buffer_size))
+ self._queue: queue.Queue[Any] | None = None
+ self._writer_thread: threading.Thread | None = None
+ self._stop_event = threading.Event()
+ self._stats_lock = threading.Lock()
+ self._frames_enqueued = 0
+ self._frames_written = 0
+ self._dropped_frames = 0
+ self._total_latency = 0.0
+ self._last_latency = 0.0
+ self._written_times: deque[float] = deque(maxlen=600)
+ self._encode_error: Exception | None = None
+ self._last_log_time = 0.0
+ self._frame_timestamps: list[float] = []
+
+ @property
+ def is_running(self) -> bool:
+ return self._writer_thread is not None and self._writer_thread.is_alive()
+
+ def start(self) -> None:
+ if WriteGear is None:
+ raise RuntimeError("vidgear is required for video recording. Install it with 'pip install vidgear'.")
+ if self._writer is not None:
+ return
+ fps_value = float(self._frame_rate) if self._frame_rate else 30.0
+
+ writer_kwargs: dict[str, Any] = {
+ "compression_mode": True,
+ "logging": False,
+ "-input_framerate": fps_value,
+ "-vcodec": (self._codec or "libx264").strip() or "libx264",
+ "-crf": int(self._crf),
+ }
+ # TODO deal with pixel format
+
+ self._output.parent.mkdir(parents=True, exist_ok=True)
+ self._writer = WriteGear(output=str(self._output), **writer_kwargs)
+ self._queue = queue.Queue(maxsize=self._buffer_size)
+ self._frames_enqueued = 0
+ self._frames_written = 0
+ self._dropped_frames = 0
+ self._total_latency = 0.0
+ self._last_latency = 0.0
+ self._written_times.clear()
+ self._frame_timestamps.clear()
+ self._encode_error = None
+ self._stop_event.clear()
+ self._writer_thread = threading.Thread(
+ target=self._writer_loop,
+ name="VideoRecorderWriter",
+ daemon=True,
+ )
+ self._writer_thread.start()
+
+ def configure_stream(self, frame_size: tuple[int, int], frame_rate: float | None) -> None:
+ self._frame_size = frame_size
+ self._frame_rate = frame_rate
+
+ def write(self, frame: np.ndarray, timestamp: float | None = None) -> bool:
+ if not self.is_running or self._queue is None:
+ return False
+ error = self._current_error()
+ if error is not None:
+ raise RuntimeError(f"Video encoding failed: {error}") from error
+
+ # Capture timestamp now, but only record it if frame is successfully enqueued
+ if timestamp is None:
+ timestamp = time.time()
+
+ # Convert frame to uint8 if needed
+ if frame.dtype != np.uint8:
+ frame_float = frame.astype(np.float32, copy=False)
+ max_val = float(frame_float.max()) if frame_float.size else 0.0
+ scale = 1.0
+ if max_val > 0:
+ scale = 255.0 / max_val if max_val > 255.0 else (255.0 if max_val <= 1.0 else 1.0)
+ frame = np.clip(frame_float * scale, 0.0, 255.0).astype(np.uint8)
+
+ # Convert grayscale to RGB if needed
+ if frame.ndim == 2:
+ frame = np.repeat(frame[:, :, None], 3, axis=2)
+
+ # Ensure contiguous array
+ frame = np.ascontiguousarray(frame)
+
+ # Check if frame size matches expected size
+ if self._frame_size is not None:
+ expected_h, expected_w = self._frame_size
+ actual_h, actual_w = frame.shape[:2]
+ if (actual_h, actual_w) != (expected_h, expected_w):
+ logger.warning(
+ f"Frame size mismatch: expected (h={expected_h}, w={expected_w}), "
+ f"got (h={actual_h}, w={actual_w}). "
+ "Stopping recorder to prevent encoding errors."
+ )
+ # Set error to stop recording gracefully
+ with self._stats_lock:
+ self._encode_error = ValueError(
+ f"Frame size changed from (h={expected_h}, w={expected_w}) to (h={actual_h}, w={actual_w})"
+ )
+ return False
+
+ try:
+ assert self._queue is not None
+ self._queue.put(frame, block=False)
+ except queue.Full:
+ with self._stats_lock:
+ self._dropped_frames += 1
+ queue_size = self._queue.qsize() if self._queue is not None else -1
+ logger.warning(
+ "Video recorder queue full; dropping frame. queue=%d buffer=%d",
+ queue_size,
+ self._buffer_size,
+ )
+ return False
+ with self._stats_lock:
+ self._frames_enqueued += 1
+ self._frame_timestamps.append(timestamp)
+ return True
+
+ def stop(self) -> None:
+ if self._writer is None and not self.is_running:
+ return
+ self._stop_event.set()
+ if self._queue is not None:
+ try:
+ self._queue.put_nowait(_SENTINEL)
+ except queue.Full:
+ pass
+ # self._queue.put(_SENTINEL)
+ if self._writer_thread is not None:
+ self._writer_thread.join(timeout=5.0)
+ if self._writer_thread.is_alive():
+ logger.warning("Video recorder thread did not terminate cleanly")
+ if self._writer is not None:
+ try:
+ self._writer.close()
+ except Exception:
+ logger.exception("Failed to close WriteGear cleanly")
+
+ # Save timestamps to JSON file
+ self._save_timestamps()
+
+ self._writer = None
+ self._writer_thread = None
+ self._queue = None
+
+ def get_stats(self) -> RecorderStats | None:
+ if (
+ self._writer is None
+ and not self.is_running
+ and self._queue is None
+ and self._frames_enqueued == 0
+ and self._frames_written == 0
+ and self._dropped_frames == 0
+ ):
+ return None
+ queue_size = self._queue.qsize() if self._queue is not None else 0
+ with self._stats_lock:
+ frames_enqueued = self._frames_enqueued
+ frames_written = self._frames_written
+ dropped = self._dropped_frames
+ avg_latency = self._total_latency / self._frames_written if self._frames_written else 0.0
+ last_latency = self._last_latency
+ write_fps = self._compute_write_fps_locked()
+ buffer_seconds = queue_size * avg_latency if avg_latency > 0 else 0.0
+ return RecorderStats(
+ frames_enqueued=frames_enqueued,
+ frames_written=frames_written,
+ dropped_frames=dropped,
+ queue_size=queue_size,
+ average_latency=avg_latency,
+ last_latency=last_latency,
+ write_fps=write_fps,
+ buffer_seconds=buffer_seconds,
+ )
+
+ def _writer_loop(self) -> None:
+ assert self._queue is not None
+ try:
+ while True:
+ try:
+ item = self._queue.get(timeout=0.1)
+ except queue.Empty:
+ if self._stop_event.is_set():
+ break
+ continue
+ if item is _SENTINEL:
+ self._queue.task_done()
+ break
+ frame = item
+ start = time.perf_counter()
+ try:
+ assert self._writer is not None
+ self._writer.write(frame)
+ except OSError as exc:
+ with self._stats_lock:
+ self._encode_error = exc
+ logger.exception("Video encoding failed while writing frame")
+ self._queue.task_done()
+ self._stop_event.set()
+ break
+ elapsed = time.perf_counter() - start
+ now = time.perf_counter()
+ with self._stats_lock:
+ self._frames_written += 1
+ self._total_latency += elapsed
+ self._last_latency = elapsed
+ self._written_times.append(now)
+ if now - self._last_log_time >= 1.0:
+ self._compute_write_fps_locked()
+ self._queue.qsize()
+ self._last_log_time = now
+ self._queue.task_done()
+ finally:
+ self._finalize_writer()
+
+ def _finalize_writer(self) -> None:
+ writer = self._writer
+ self._writer = None
+ if writer is not None:
+ try:
+ writer.close()
+ time.sleep(0.2) # give some time to finalize
+ except Exception:
+ logger.exception("Failed to close WriteGear during finalisation")
+
+ def _compute_write_fps_locked(self) -> float:
+ if len(self._written_times) < 2:
+ return 0.0
+ duration = self._written_times[-1] - self._written_times[0]
+ if duration <= 0:
+ return 0.0
+ return (len(self._written_times) - 1) / duration
+
+ def _current_error(self) -> Exception | None:
+ with self._stats_lock:
+ return self._encode_error
+
+ def _save_timestamps(self) -> None:
+ """Save frame timestamps to a JSON file alongside the video."""
+ if not self._frame_timestamps:
+ logger.info("No timestamps to save")
+ return
+
+ # Create timestamps file path
+ timestamp_file = self._output.with_suffix("").with_suffix(self._output.suffix + "_timestamps.json")
+
+ try:
+ with self._stats_lock:
+ timestamps = self._frame_timestamps.copy()
+
+ # Prepare metadata
+ data = {
+ "video_file": str(self._output.name),
+ "num_frames": len(timestamps),
+ "timestamps": timestamps,
+ "start_time": timestamps[0] if timestamps else None,
+ "end_time": timestamps[-1] if timestamps else None,
+ "duration_seconds": timestamps[-1] - timestamps[0] if len(timestamps) > 1 else 0.0,
+ }
+
+ # Write to JSON
+ with open(timestamp_file, "w") as f:
+ json.dump(data, f, indent=2)
+
+ logger.info(f"Saved {len(timestamps)} frame timestamps to {timestamp_file}")
+ except Exception as exc:
+ logger.exception(f"Failed to save timestamps to {timestamp_file}: {exc}")
diff --git a/dlclivegui/tkutil.py b/dlclivegui/tkutil.py
deleted file mode 100644
index 3a5837e..0000000
--- a/dlclivegui/tkutil.py
+++ /dev/null
@@ -1,195 +0,0 @@
-"""
-DeepLabCut Toolbox (deeplabcut.org)
-© A. & M. Mathis Labs
-
-Licensed under GNU Lesser General Public License v3.0
-"""
-
-
-import tkinter as tk
-from tkinter import ttk
-from distutils.util import strtobool
-
-
-class SettingsWindow(tk.Toplevel):
- def __init__(
- self,
- title="Edit Settings",
- settings={},
- names=None,
- vals=None,
- dtypes=None,
- restrictions=None,
- parent=None,
- ):
- """ Create a tkinter settings window
-
- Parameters
- ----------
- title : str, optional
- title for window
- settings : dict, optional
- dictionary of settings with keys = setting names.
- The value for each setting should be a dictionary with three keys:
- value (a default value),
- dtype (the data type for the setting),
- restriction (a list of possible values the parameter can take on)
- names : list, optional
- list of setting names, by default None
- vals : list, optional
- list of default values, by default None
- dtypes : list, optional
- list of setting data types, by default None
- restrictions : dict, optional
- dictionary of setting value restrictions, with keys = setting name and value = list of restrictions, by default {}
- parent : :class:`tkinter.Tk`, optional
- parent window, by default None
-
- Raises
- ------
- ValueError
- throws error if neither settings dictionary nor setting names are provided
- """
-
- super().__init__(parent)
- self.title(title)
-
- if settings:
- self.settings = settings
- elif not names:
- raise ValueError(
- "No argument names or settings dictionary. One must be provided to create a SettingsWindow."
- )
- else:
- self.settings = self.create_settings_dict(names, vals, dtypes, restrictions)
-
- self.cur_row = 0
- self.combobox_width = 15
-
- self.create_window()
-
- def create_settings_dict(self, names, vals=None, dtypes=None, restrictions=None):
- """Create dictionary of settings from names, vals, dtypes, and restrictions
-
- Parameters
- ----------
- names : list
- list of setting names
- vals : list
- list of default setting values
- dtypes : list
- list of setting dtype
- restrictions : dict
- dictionary of settting restrictions
-
- Returns
- -------
- dict
- settings dictionary with keys = names and value = dictionary with value, dtype, restrictions
- """
-
- set_dict = {}
- for i in range(len(names)):
-
- dt = dtypes[i] if dtypes is not None else None
-
- if vals is not None:
- val = dt(val) if type(dt) is type else [dt[0](v) for v in val]
- else:
- val = None
-
- restrict = restrictions[names[i]] if restrictions is not None else None
-
- set_dict[names[i]] = {"value": val, "dtype": dt, "restriction": restrict}
-
- return set_dict
-
- def create_window(self):
- """ Create settings GUI widgets
- """
-
- self.entry_vars = []
- names = tuple(self.settings.keys())
- for i in range(len(names)):
-
- this_setting = self.settings[names[i]]
-
- tk.Label(self, text=names[i] + ": ").grid(row=self.cur_row, column=0)
-
- v = this_setting["value"]
- if type(this_setting["dtype"]) is list:
- v = [str(x) if x is not None else "" for x in v]
- v = ", ".join(v)
- else:
- v = str(v) if v is not None else ""
- self.entry_vars.append(tk.StringVar(self, value=v))
-
- use_restriction = False
- if "restriction" in this_setting:
- if this_setting["restriction"] is not None:
- use_restriction = True
-
- if use_restriction:
- ttk.Combobox(
- self,
- textvariable=self.entry_vars[-1],
- values=this_setting["restriction"],
- state="readonly",
- width=self.combobox_width,
- ).grid(sticky="nsew", row=self.cur_row, column=1)
- else:
- tk.Entry(self, textvariable=self.entry_vars[-1]).grid(
- sticky="nsew", row=self.cur_row, column=1
- )
-
- self.cur_row += 1
-
- self.cur_row += 1
- tk.Button(self, text="Update", command=self.update_vals).grid(
- sticky="nsew", row=self.cur_row, column=1
- )
- self.cur_row += 1
- tk.Button(self, text="Cancel", command=self.destroy).grid(
- sticky="nsew", row=self.cur_row, column=1
- )
-
- _, row_count = self.grid_size()
- for r in range(row_count):
- self.grid_rowconfigure(r, minsize=20)
-
- def update_vals(self):
-
- names = tuple(self.settings.keys())
-
- for i in range(len(self.entry_vars)):
-
- name = names[i]
- val = self.entry_vars[i].get()
- dt = (
- self.settings[name]["dtype"] if "dtype" in self.settings[name] else None
- )
-
- val = [v.strip() for v in val.split(",")]
- use_dt = dt if type(dt) is type else dt[0]
- use_dt = strtobool if use_dt is bool else use_dt
-
- try:
- val = [use_dt(v) if v else None for v in val]
- except TypeError:
- pass
-
- val = val if type(dt) is list else val[0]
-
- self.settings[name]["value"] = val
-
- self.quit()
- self.destroy()
-
- def get_values(self):
-
- val_dict = {}
- names = tuple(self.settings.keys())
- for i in range(len(self.settings)):
- val_dict[names[i]] = self.settings[names[i]]["value"]
-
- return val_dict
diff --git a/dlclivegui/utils/config_models.py b/dlclivegui/utils/config_models.py
new file mode 100644
index 0000000..98c6306
--- /dev/null
+++ b/dlclivegui/utils/config_models.py
@@ -0,0 +1,283 @@
+# config_models.py
+from __future__ import annotations
+
+import json
+from pathlib import Path
+from typing import Any, Literal
+
+from pydantic import BaseModel, Field, field_validator, model_validator
+
+Rotation = Literal[0, 90, 180, 270]
+TileLayout = Literal["auto", "2x2", "1x4", "4x1"]
+Precision = Literal["FP32", "FP16"]
+
+
+class CameraSettingsModel(BaseModel):
+ name: str = "Camera 0"
+ index: int = 0
+ fps: float = 25.0
+ backend: str = "opencv"
+ exposure: int = 500 # 0=auto else µs
+ gain: float = 10.0 # 0.0=auto else value
+ crop_x0: int = 0
+ crop_y0: int = 0
+ crop_x1: int = 0
+ crop_y1: int = 0
+ max_devices: int = 3
+ rotation: Rotation = 0
+ enabled: bool = True
+ properties: dict[str, Any] = Field(default_factory=dict)
+
+ @field_validator("fps")
+ @classmethod
+ def _fps_positive(cls, v):
+ return float(v) if v and v > 0 else 30.0
+
+ @field_validator("exposure")
+ @classmethod
+ def _coerce_exposure(cls, v): # allow None->0 and int
+ return int(v) if v is not None else 0
+
+ @field_validator("gain")
+ @classmethod
+ def _coerce_gain(cls, v):
+ return float(v) if v is not None else 0.0
+
+ @model_validator(mode="after")
+ def _validate_crop(self):
+ for f in ("crop_x0", "crop_y0", "crop_x1", "crop_y1"):
+ setattr(self, f, max(0, int(getattr(self, f))))
+ # Optional: if any crop is set, enforce x1>x0 and y1>y0
+ if any([self.crop_x0, self.crop_y0, self.crop_x1, self.crop_y1]):
+ if not (self.crop_x1 > self.crop_x0 and self.crop_y1 > self.crop_y0):
+ raise ValueError("Invalid crop rectangle: require x1>x0 and y1>y0 when cropping is enabled.")
+ return self
+
+ def get_crop_region(self) -> tuple[int, int, int, int] | None:
+ if self.crop_x0 == self.crop_y0 == self.crop_x1 == self.crop_y1 == 0:
+ return None
+ return (self.crop_x0, self.crop_y0, self.crop_x1, self.crop_y1)
+
+ @classmethod
+ def from_dict(cls, data: dict[str, Any]) -> CameraSettingsModel:
+ return cls(**data)
+
+ @classmethod
+ def from_defaults(cls) -> CameraSettingsModel:
+ return cls()
+
+ def apply_defaults(self) -> CameraSettingsModel:
+ default = self.from_defaults()
+ for field in CameraSettingsModel.model_fields:
+ if getattr(self, field) in (None, 0, 0.0):
+ setattr(self, field, getattr(default, field))
+ return self
+
+
+class MultiCameraSettingsModel(BaseModel):
+ cameras: list[CameraSettingsModel] = Field(default_factory=list)
+ max_cameras: int = 4
+ tile_layout: TileLayout = "auto"
+
+ def get_active_cameras(self) -> list[CameraSettingsModel]:
+ return [c for c in self.cameras if c.enabled]
+
+ @model_validator(mode="after")
+ def _enforce_max_active(self):
+ if len(self.get_active_cameras()) > self.max_cameras:
+ raise ValueError("Number of enabled cameras exceeds max_cameras.")
+ return self
+
+ def add_camera(self, camera: CameraSettingsModel) -> bool:
+ """Add a new camera if under max_cameras limit."""
+ if len(self.cameras) >= self.max_cameras:
+ return False
+ self.cameras.append(camera)
+ return True
+
+ def remove_camera(self, index: int) -> bool:
+ """Remove camera at given index."""
+ if 0 <= index < len(self.cameras):
+ del self.cameras[index]
+ return True
+ return False
+
+ @classmethod
+ def from_dict(cls, data: dict[str, Any]) -> MultiCameraSettingsModel:
+ cameras_data = data.get("cameras", [])
+ cameras = [CameraSettingsModel(**cam) for cam in cameras_data]
+ max_cameras = data.get("max_cameras", 4)
+ tile_layout = data.get("tile_layout", "auto")
+ return cls(cameras=cameras, max_cameras=max_cameras, tile_layout=tile_layout)
+
+ def to_dict(self) -> dict[str, Any]:
+ return {
+ "cameras": [cam.model_dump() for cam in self.cameras],
+ "max_cameras": self.max_cameras,
+ "tile_layout": self.tile_layout,
+ }
+
+
+class DynamicCropModel(BaseModel):
+ enabled: bool = False
+ margin: float = Field(default=0.5, ge=0.0, le=1.0)
+ max_missing_frames: int = Field(default=10, ge=0)
+
+ @classmethod
+ def from_tupleish(cls, v):
+ # Accept (enabled, margin, max_missing_frames)
+ if isinstance(v, (list, tuple)) and len(v) == 3:
+ return cls(enabled=bool(v[0]), margin=float(v[1]), max_missing_frames=int(v[2]))
+ if isinstance(v, dict):
+ return cls(**v)
+ if isinstance(v, cls):
+ return v
+ return cls()
+
+ def to_tuple(self) -> tuple[bool, float, int]:
+ return (self.enabled, self.margin, self.max_missing_frames)
+
+
+class DLCProcessorSettingsModel(BaseModel):
+ model_path: str = ""
+ model_directory: str = "."
+ device: str | None = "auto" # "cuda:0", "cpu", or None
+ dynamic: DynamicCropModel = Field(default_factory=DynamicCropModel)
+ resize: float = Field(default=1.0, gt=0)
+ precision: Precision = "FP32"
+ additional_options: dict[str, Any] = Field(default_factory=dict)
+ model_type: Literal["pytorch"] = "pytorch"
+ single_animal: bool = True
+
+ @field_validator("dynamic", mode="before")
+ @classmethod
+ def _coerce_dynamic(cls, v):
+ return DynamicCropModel.from_tupleish(v)
+
+
+class BoundingBoxSettingsModel(BaseModel):
+ enabled: bool = False
+ x0: int = 0
+ y0: int = 0
+ x1: int = 200
+ y1: int = 100
+
+ @model_validator(mode="after")
+ def _bbox_logic(self):
+ if self.enabled and not (self.x1 > self.x0 and self.y1 > self.y0):
+ raise ValueError("Bounding box enabled but coordinates are invalid (x1>x0 and y1>y0 required).")
+ return self
+
+
+class VisualizationSettingsModel(BaseModel):
+ p_cutoff: float = Field(default=0.6, ge=0.0, le=1.0)
+ colormap: str = "hot"
+ bbox_color: tuple[int, int, int] = (0, 0, 255)
+
+ def get_bbox_color_bgr(self) -> tuple[int, int, int]:
+ """Get bounding box color in BGR format"""
+ if isinstance(self.bbox_color, (list, tuple)) and len(self.bbox_color) == 3:
+ return tuple(int(c) for c in self.bbox_color)
+ return (0, 0, 255) # default red
+
+
+class RecordingSettingsModel(BaseModel):
+ enabled: bool = False
+ directory: str = Field(default_factory=lambda: str(Path.home() / "Videos" / "deeplabcut-live"))
+ filename: str = "session.mp4"
+ container: Literal["mp4", "avi", "mov"] = "mp4"
+ codec: str = "libx264"
+ crf: int = Field(default=23, ge=0, le=51)
+
+ def output_path(self) -> Path:
+ """Return the absolute output path for recordings."""
+
+ directory = Path(self.directory).expanduser().resolve()
+ directory.mkdir(parents=True, exist_ok=True)
+ name = Path(self.filename)
+ if name.suffix:
+ filename = name
+ else:
+ filename = name.with_suffix(f".{self.container}")
+ return directory / filename
+
+ def writegear_options(self, fps: float) -> dict[str, Any]:
+ """Return compression parameters for WriteGear."""
+
+ fps_value = float(fps) if fps else 30.0
+ codec_value = (self.codec or "libx264").strip() or "libx264"
+ crf_value = int(self.crf) if self.crf is not None else 23
+ return {
+ "-input_framerate": f"{fps_value:.6f}",
+ "-vcodec": codec_value,
+ "-crf": str(crf_value),
+ }
+
+
+class ApplicationSettingsModel(BaseModel):
+ # optional: add a semantic version for migrations
+ version: int = 1
+ camera: CameraSettingsModel = Field(default_factory=CameraSettingsModel) # kept for backward compat
+ multi_camera: MultiCameraSettingsModel = Field(default_factory=MultiCameraSettingsModel)
+ dlc: DLCProcessorSettingsModel = Field(default_factory=DLCProcessorSettingsModel)
+ recording: RecordingSettingsModel = Field(default_factory=RecordingSettingsModel)
+ bbox: BoundingBoxSettingsModel = Field(default_factory=BoundingBoxSettingsModel)
+ visualization: VisualizationSettingsModel = Field(default_factory=VisualizationSettingsModel)
+
+ @classmethod
+ def from_dict(cls, data: dict[str, Any]) -> ApplicationSettingsModel:
+ camera_data = data.get("camera", {})
+ multi_camera_data = data.get("multi_camera", {})
+ dlc_data = data.get("dlc", {})
+ recording_data = data.get("recording", {})
+ bbox_data = data.get("bbox", {})
+ visualization_data = data.get("visualization", {})
+
+ camera = CameraSettingsModel(**camera_data)
+ multi_camera = MultiCameraSettingsModel.from_dict(multi_camera_data)
+ dlc = DLCProcessorSettingsModel(**dlc_data)
+ recording = RecordingSettingsModel(**recording_data)
+ bbox = BoundingBoxSettingsModel(**bbox_data)
+ visualization = VisualizationSettingsModel(**visualization_data)
+
+ return cls(
+ camera=camera,
+ multi_camera=multi_camera,
+ dlc=dlc,
+ recording=recording,
+ bbox=bbox,
+ visualization=visualization,
+ )
+
+ def to_dict(self) -> dict[str, Any]:
+ return {
+ "version": self.version,
+ "camera": self.camera.model_dump(),
+ "multi_camera": self.multi_camera.to_dict(),
+ "dlc": self.dlc.model_dump(),
+ "recording": self.recording.model_dump(),
+ "bbox": self.bbox.model_dump(),
+ "visualization": self.visualization.model_dump(),
+ }
+
+ @classmethod
+ def load(cls, path: Path | str) -> ApplicationSettingsModel:
+ """Load configuration from ``path``."""
+
+ file_path = Path(path).expanduser()
+ if not file_path.exists():
+ raise FileNotFoundError(f"Configuration file not found: {file_path}")
+ with file_path.open("r", encoding="utf-8") as handle:
+ data = json.load(handle)
+ return cls.from_dict(data)
+
+ def save(self, path: Path | str) -> None:
+ """Persist configuration to ``path``."""
+
+ file_path = Path(path).expanduser()
+ file_path.parent.mkdir(parents=True, exist_ok=True)
+ with file_path.open("w", encoding="utf-8") as handle:
+ json.dump(self.to_dict(), handle, indent=2)
+
+
+DEFAULT_CONFIG = ApplicationSettingsModel()
diff --git a/dlclivegui/utils/display.py b/dlclivegui/utils/display.py
new file mode 100644
index 0000000..8b93403
--- /dev/null
+++ b/dlclivegui/utils/display.py
@@ -0,0 +1,217 @@
+# dlclivegui/utils/display.py
+from __future__ import annotations
+
+import cv2
+import matplotlib.pyplot as plt
+import numpy as np
+
+
+def create_tiled_frame(frames: dict[str, np.ndarray], max_canvas: tuple[int, int] = (1200, 800)) -> np.ndarray:
+ """Create a tiled canvas (1x1, 1x2, or 2x2) with camera-id labels."""
+ if not frames:
+ return np.zeros((480, 640, 3), dtype=np.uint8)
+
+ cam_ids = sorted(frames.keys())
+ frames_list = [frames[cid] for cid in cam_ids]
+ num_frames = len(frames_list)
+
+ if num_frames == 1:
+ rows, cols = 1, 1
+ elif num_frames == 2:
+ rows, cols = 1, 2
+ else:
+ rows, cols = 2, 2
+
+ max_w, max_h = max_canvas
+ h0, w0 = frames_list[0].shape[:2]
+ frame_aspect = w0 / h0 if h0 > 0 else 1.0
+
+ tile_w = max_w // cols
+ tile_h = max_h // rows
+ tile_aspect = tile_w / tile_h if tile_h > 0 else 1.0
+
+ if frame_aspect > tile_aspect:
+ tile_h = int(tile_w / frame_aspect)
+ else:
+ tile_w = int(tile_h * frame_aspect)
+
+ tile_w = max(160, tile_w)
+ tile_h = max(120, tile_h)
+
+ canvas = np.zeros((rows * tile_h, cols * tile_w, 3), dtype=np.uint8)
+
+ for idx, frame in enumerate(frames_list[: rows * cols]):
+ row = idx // cols
+ col = idx % cols
+
+ if frame.ndim == 2:
+ frame = cv2.cvtColor(frame, cv2.COLOR_GRAY2BGR)
+ elif frame.shape[2] == 4:
+ frame = cv2.cvtColor(frame, cv2.COLOR_BGRA2BGR)
+
+ resized = cv2.resize(frame, (tile_w, tile_h))
+ if idx < len(cam_ids):
+ cv2.putText(
+ resized,
+ cam_ids[idx],
+ (10, 30),
+ cv2.FONT_HERSHEY_SIMPLEX,
+ 0.7,
+ (0, 255, 0),
+ 2,
+ )
+
+ y0 = row * tile_h
+ x0 = col * tile_w
+ canvas[y0 : y0 + tile_h, x0 : x0 + tile_w] = resized
+
+ return canvas
+
+
+def compute_tile_info(
+ dlc_cam_id: str,
+ original_frame: np.ndarray,
+ frames: dict[str, np.ndarray],
+ max_canvas: tuple[int, int] = (1200, 800),
+) -> tuple[tuple[int, int], tuple[float, float]]:
+ """Return ((offset_x, offset_y), (scale_x, scale_y)) for overlaying on the tiled view."""
+ num_cameras = len(frames)
+ if num_cameras == 0:
+ return (0, 0), (1.0, 1.0)
+
+ orig_h, orig_w = original_frame.shape[:2]
+ if num_cameras == 1:
+ rows, cols = 1, 1
+ elif num_cameras == 2:
+ rows, cols = 1, 2
+ else:
+ rows, cols = 2, 2
+
+ max_w, max_h = max_canvas
+ frame_aspect = orig_w / orig_h if orig_h > 0 else 1.0
+
+ tile_w = max_w // cols
+ tile_h = max_h // rows
+ tile_aspect = tile_w / tile_h if tile_h > 0 else 1.0
+
+ if frame_aspect > tile_aspect:
+ tile_h = int(tile_w / frame_aspect)
+ else:
+ tile_w = int(tile_h * frame_aspect)
+
+ tile_w = max(160, tile_w)
+ tile_h = max(120, tile_h)
+
+ sorted_cam_ids = sorted(frames.keys())
+ try:
+ dlc_cam_idx = sorted_cam_ids.index(dlc_cam_id)
+ except ValueError:
+ dlc_cam_idx = 0
+
+ row = dlc_cam_idx // cols
+ col = dlc_cam_idx % cols
+ offset_x = col * tile_w
+ offset_y = row * tile_h
+
+ scale_x = tile_w / orig_w if orig_w > 0 else 1.0
+ scale_y = tile_h / orig_h if orig_h > 0 else 1.0
+
+ return (offset_x, offset_y), (scale_x, scale_y)
+
+
+def draw_bbox(
+ frame: np.ndarray,
+ bbox_xyxy: tuple[int, int, int, int],
+ color_bgr: tuple[int, int, int],
+ offset: tuple[int, int] = (0, 0),
+ scale: tuple[float, float] = (1.0, 1.0),
+) -> np.ndarray:
+ """Draw a bbox on the frame, transformed by offset/scale for tiled views."""
+ x0, y0, x1, y1 = bbox_xyxy
+ if x0 >= x1 or y0 >= y1:
+ return frame
+
+ ox, oy = offset
+ sx, sy = scale
+ x0s = int(x0 * sx + ox)
+ y0s = int(y0 * sy + oy)
+ x1s = int(x1 * sx + ox)
+ y1s = int(y1 * sy + oy)
+
+ h, w = frame.shape[:2]
+ x0s = max(0, min(x0s, w - 1))
+ y0s = max(0, min(y0s, h - 1))
+ x1s = max(x0s + 1, min(x1s, w))
+ y1s = max(y0s + 1, min(y1s, h))
+
+ out = frame.copy()
+ cv2.rectangle(out, (x0s, y0s), (x1s, y1s), color_bgr, 2)
+ return out
+
+
+def draw_keypoints(overlay, p_cutoff, sx, ox, sy, oy, radius, cmap, keypoints: np.ndarray, marker: int | None) -> None:
+ num_kpts = len(keypoints)
+ for idx, kpt in enumerate(keypoints):
+ if len(kpt) < 2:
+ continue
+ x, y = kpt[:2]
+ conf = kpt[2] if len(kpt) > 2 else 1.0
+ if np.isnan(x) or np.isnan(y) or conf < p_cutoff:
+ continue
+
+ xs = int(x * sx + ox)
+ ys = int(y * sy + oy)
+
+ t = idx / max(num_kpts - 1, 1)
+ rgba = cmap(t)
+ bgr = (int(rgba[2] * 255), int(rgba[1] * 255), int(rgba[0] * 255))
+ if marker is None:
+ cv2.circle(overlay, (xs, ys), radius, bgr, -1)
+ else:
+ cv2.drawMarker(overlay, (xs, ys), bgr, marker, radius * 2, 2)
+
+
+def draw_pose(
+ frame: np.ndarray,
+ pose: np.ndarray,
+ p_cutoff: float,
+ colormap: str,
+ offset: tuple[int, int],
+ scale: tuple[float, float],
+ base_radius: int = 4,
+) -> np.ndarray:
+ """Draw single- or multi-animal pose (N x 3 or A x N x 3) on the frame."""
+ overlay = frame.copy()
+ pose_arr = np.asarray(pose)
+ ox, oy = offset
+ sx, sy = scale
+ radius = max(2, int(base_radius * min(sx, sy)))
+ cmap = plt.get_cmap(colormap)
+
+ if pose_arr.ndim == 3:
+ markers = [
+ cv2.MARKER_CROSS,
+ cv2.MARKER_TILTED_CROSS,
+ cv2.MARKER_STAR,
+ cv2.MARKER_DIAMOND,
+ cv2.MARKER_SQUARE,
+ cv2.MARKER_TRIANGLE_UP,
+ cv2.MARKER_TRIANGLE_DOWN,
+ ]
+ for i, animal_pose in enumerate(pose_arr):
+ draw_keypoints(
+ overlay,
+ p_cutoff,
+ sx,
+ ox,
+ sy,
+ oy,
+ radius,
+ cmap,
+ animal_pose,
+ markers[i % len(markers)],
+ )
+ else:
+ draw_keypoints(overlay, p_cutoff, sx, ox, sy, oy, radius, cmap, pose_arr, marker=None)
+
+ return overlay
diff --git a/dlclivegui/utils/settings_store.py b/dlclivegui/utils/settings_store.py
new file mode 100644
index 0000000..6696376
--- /dev/null
+++ b/dlclivegui/utils/settings_store.py
@@ -0,0 +1,38 @@
+# settings_store.py
+
+from PySide6.QtCore import QSettings
+
+from .config_models import ApplicationSettingsModel
+
+
+class QtSettingsStore:
+ def __init__(self, qsettings: QSettings | None = None):
+ self._s = qsettings or QSettings("DeepLabCut", "DLCLiveGUI")
+
+ # --- lightweight prefs ---
+ def get_last_model_path(self) -> str | None:
+ v = self._s.value("dlc/last_model_path", "")
+ return str(v) if v else None
+
+ def set_last_model_path(self, path: str) -> None:
+ self._s.setValue("dlc/last_model_path", path or "")
+
+ def get_last_config_path(self) -> str | None:
+ v = self._s.value("app/last_config_path", "")
+ return str(v) if v else None
+
+ def set_last_config_path(self, path: str) -> None:
+ self._s.setValue("app/last_config_path", path or "")
+
+ # --- optional: snapshot full config as JSON in QSettings ---
+ def save_full_config_snapshot(self, cfg: ApplicationSettingsModel) -> None:
+ self._s.setValue("app/config_json", cfg.model_dump_json())
+
+ def load_full_config_snapshot(self) -> ApplicationSettingsModel | None:
+ raw = self._s.value("app/config_json", "")
+ if not raw:
+ return None
+ try:
+ return ApplicationSettingsModel.model_validate_json(str(raw))
+ except Exception:
+ return None
diff --git a/dlclivegui/utils/stats.py b/dlclivegui/utils/stats.py
new file mode 100644
index 0000000..23e9d57
--- /dev/null
+++ b/dlclivegui/utils/stats.py
@@ -0,0 +1,45 @@
+# dlclivegui/utils/stats.py
+from __future__ import annotations
+
+from dlclivegui.services.dlc_processor import ProcessorStats
+from dlclivegui.services.video_recorder import RecorderStats
+
+
+def format_recorder_stats(stats: RecorderStats) -> str:
+ latency_ms = stats.last_latency * 1000.0
+ avg_ms = stats.average_latency * 1000.0
+ buffer_ms = stats.buffer_seconds * 1000.0
+ return (
+ f"{stats.frames_written}/{stats.frames_enqueued} frames | "
+ f"write {stats.write_fps:.1f} fps | "
+ f"latency {latency_ms:.1f} ms (avg {avg_ms:.1f} ms) | "
+ f"queue {stats.queue_size} (~{buffer_ms:.0f} ms) | "
+ f"dropped {stats.dropped_frames}"
+ )
+
+
+def format_dlc_stats(stats: ProcessorStats) -> str:
+ latency_ms = stats.last_latency * 1000.0
+ avg_ms = stats.average_latency * 1000.0
+ profile = ""
+ if stats.avg_inference_time > 0:
+ inf_ms = stats.avg_inference_time * 1000.0
+ queue_ms = stats.avg_queue_wait * 1000.0
+ signal_ms = stats.avg_signal_emit_time * 1000.0
+ total_ms = stats.avg_total_process_time * 1000.0
+ gpu_breakdown = ""
+ if stats.avg_gpu_inference_time > 0 or stats.avg_processor_overhead > 0:
+ gpu_ms = stats.avg_gpu_inference_time * 1000.0
+ proc_ms = stats.avg_processor_overhead * 1000.0
+ gpu_breakdown = f" (GPU:{gpu_ms:.1f}ms+proc:{proc_ms:.1f}ms)"
+ profile = (
+ f"\n[Profile] inf:{inf_ms:.1f}ms{gpu_breakdown} "
+ f"queue:{queue_ms:.1f}ms signal:{signal_ms:.1f}ms total:{total_ms:.1f}ms"
+ )
+
+ return (
+ f"{stats.frames_processed}/{stats.frames_enqueued} frames | "
+ f"inference {stats.processing_fps:.1f} fps | "
+ f"latency {latency_ms:.1f} ms (avg {avg_ms:.1f} ms) | "
+ f"queue {stats.queue_size} | dropped {stats.frames_dropped}{profile}"
+ )
diff --git a/dlclivegui/utils/utils.py b/dlclivegui/utils/utils.py
new file mode 100644
index 0000000..38a8504
--- /dev/null
+++ b/dlclivegui/utils/utils.py
@@ -0,0 +1,46 @@
+from __future__ import annotations
+
+import time
+from collections import deque
+from pathlib import Path
+
+SUPPORTED_MODELS = [".pt", ".pth", ".pb"]
+
+
+def is_model_file(file_path: Path | str) -> bool:
+ if not isinstance(file_path, Path):
+ file_path = Path(file_path)
+ if not file_path.is_file():
+ return False
+ return file_path.suffix.lower() in SUPPORTED_MODELS
+
+
+class FPSTracker:
+ """Track per-camera FPS within a sliding time window."""
+
+ def __init__(self, window_seconds: float = 5.0, maxlen: int = 240):
+ self.window_seconds = window_seconds
+ self._times: dict[str, deque[float]] = {}
+ self._maxlen = maxlen
+
+ def clear(self) -> None:
+ self._times.clear()
+
+ def note_frame(self, camera_id: str) -> None:
+ now = time.perf_counter()
+ dq = self._times.get(camera_id)
+ if dq is None:
+ dq = deque(maxlen=self._maxlen)
+ self._times[camera_id] = dq
+ dq.append(now)
+ while dq and (now - dq[0]) > self.window_seconds:
+ dq.popleft()
+
+ def fps(self, camera_id: str) -> float:
+ dq = self._times.get(camera_id)
+ if not dq or len(dq) < 2:
+ return 0.0
+ duration = dq[-1] - dq[0]
+ if duration <= 0:
+ return 0.0
+ return (len(dq) - 1) / duration
diff --git a/dlclivegui/video.py b/dlclivegui/video.py
deleted file mode 100644
index d05f5b8..0000000
--- a/dlclivegui/video.py
+++ /dev/null
@@ -1,274 +0,0 @@
-"""
-DeepLabCut Toolbox (deeplabcut.org)
-© A. & M. Mathis Labs
-
-Licensed under GNU Lesser General Public License v3.0
-"""
-
-
-import os
-import numpy as np
-import pandas as pd
-import cv2
-import colorcet as cc
-from PIL import ImageColor
-from tqdm import tqdm
-
-
-def create_labeled_video(
- data_dir,
- out_dir=None,
- dlc_online=True,
- save_images=False,
- cut=(0, np.Inf),
- crop=None,
- cmap="bmy",
- radius=3,
- lik_thresh=0.5,
- write_ts=False,
- write_scale=2,
- write_pos="bottom-left",
- write_ts_offset=0,
- display=False,
- progress=True,
- label=True,
-):
- """ Create a labeled video from DeepLabCut-live-GUI recording
-
- Parameters
- ----------
- data_dir : str
- path to data directory
- dlc_online : bool, optional
- flag indicating dlc keypoints from online tracking, using DeepLabCut-live-GUI, or offline tracking, using :func:`dlclive.benchmark_videos`
- out_file : str, optional
- path for output file. If None, output file will be "'video_file'_LABELED.avi". by default None. If NOn
- save_images : bool, optional
- boolean flag to save still images in a folder
- cut : tuple, optional
- time of video to use. Will only save labeled video for time after cut[0] and before cut[1], by default (0, np.Inf)
- cmap : str, optional
- a :package:`colorcet` colormap, by default 'bmy'
- radius : int, optional
- radius for keypoints, by default 3
- lik_thresh : float, optional
- likelihood threshold to plot keypoints, by default 0.5
- display : bool, optional
- boolean flag to display images as video is written, by default False
- progress : bool, optional
- boolean flag to display progress bar
-
- Raises
- ------
- Exception
- if frames cannot be read from the video file
- """
-
- base_dir = os.path.basename(data_dir)
- video_file = os.path.normpath(f"{data_dir}/{base_dir}_VIDEO.avi")
- ts_file = os.path.normpath(f"{data_dir}/{base_dir}_TS.npy")
- dlc_file = (
- os.path.normpath(f"{data_dir}/{base_dir}_DLC.hdf5")
- if dlc_online
- else os.path.normpath(f"{data_dir}/{base_dir}_VIDEO_DLCLIVE_POSES.h5")
- )
-
- cap = cv2.VideoCapture(video_file)
- cam_frame_times = np.load(ts_file)
- n_frames = cam_frame_times.size
-
- lab = "LABELED" if label else "UNLABELED"
- if out_dir:
- out_file = (
- f"{out_dir}/{os.path.splitext(os.path.basename(video_file))[0]}_{lab}.avi"
- )
- out_times_file = (
- f"{out_dir}/{os.path.splitext(os.path.basename(ts_file))[0]}_{lab}.npy"
- )
- else:
- out_file = f"{os.path.splitext(video_file)[0]}_{lab}.avi"
- out_times_file = f"{os.path.splitext(ts_file)[0]}_{lab}.npy"
-
- os.makedirs(os.path.normpath(os.path.dirname(out_file)), exist_ok=True)
-
- if save_images:
- im_dir = os.path.splitext(out_file)[0]
- os.makedirs(im_dir, exist_ok=True)
-
- im_size = (
- int(cap.get(cv2.CAP_PROP_FRAME_WIDTH)),
- int(cap.get(cv2.CAP_PROP_FRAME_HEIGHT)),
- )
- if crop is not None:
- crop[0] = crop[0] if crop[0] > 0 else 0
- crop[1] = crop[1] if crop[1] > 0 else im_size[1]
- crop[2] = crop[2] if crop[2] > 0 else 0
- crop[3] = crop[3] if crop[3] > 0 else im_size[0]
- im_size = (crop[3] - crop[2], crop[1] - crop[0])
-
- fourcc = cv2.VideoWriter_fourcc(*"DIVX")
- fps = cap.get(cv2.CAP_PROP_FPS)
- vwriter = cv2.VideoWriter(out_file, fourcc, fps, im_size)
- label_times = []
-
- if write_ts:
- ts_font = cv2.FONT_HERSHEY_PLAIN
-
- if "left" in write_pos:
- ts_w = 0
- else:
- ts_w = (
- im_size[0] if crop is None else (crop[3] - crop[2]) - (55 * write_scale)
- )
-
- if "bottom" in write_pos:
- ts_h = im_size[1] if crop is None else (crop[1] - crop[0])
- else:
- ts_h = 0 if crop is None else crop[0] + (12 * write_scale)
-
- ts_coord = (ts_w, ts_h)
- ts_color = (255, 255, 255)
- ts_size = 2
-
- poses = pd.read_hdf(dlc_file)
- if dlc_online:
- pose_times = poses["pose_time"]
- else:
- poses["frame_time"] = cam_frame_times
- poses["pose_time"] = cam_frame_times
- poses = poses.melt(id_vars=["frame_time", "pose_time"])
- bodyparts = poses["bodyparts"].unique()
-
- all_colors = getattr(cc, cmap)
- colors = [
- ImageColor.getcolor(c, "RGB")[::-1]
- for c in all_colors[:: int(len(all_colors) / bodyparts.size)]
- ]
-
- ind = 0
- vid_time = 0
- while vid_time < cut[0]:
-
- cur_time = cam_frame_times[ind]
- vid_time = cur_time - cam_frame_times[0]
- ret, frame = cap.read()
- ind += 1
-
- if not ret:
- raise Exception(
- f"Could not read frame = {ind+1} at time = {cur_time-cam_frame_times[0]}."
- )
-
- frame_times_sub = cam_frame_times[
- (cam_frame_times - cam_frame_times[0] > cut[0])
- & (cam_frame_times - cam_frame_times[0] < cut[1])
- ]
- iterator = (
- tqdm(range(ind, ind + frame_times_sub.size))
- if progress
- else range(ind, ind + frame_times_sub.size)
- )
- this_pose = np.zeros((bodyparts.size, 3))
-
- for i in iterator:
-
- cur_time = cam_frame_times[i]
- vid_time = cur_time - cam_frame_times[0]
- ret, frame = cap.read()
-
- if not ret:
- raise Exception(
- f"Could not read frame = {i+1} at time = {cur_time-cam_frame_times[0]}."
- )
-
- if dlc_online:
- poses_before_index = np.where(pose_times < cur_time)[0]
- if poses_before_index.size > 0:
- cur_pose_time = pose_times[poses_before_index[-1]]
- this_pose = poses[poses["pose_time"] == cur_pose_time]
- else:
- this_pose = poses[poses["frame_time"] == cur_time]
-
- if label:
- for j in range(bodyparts.size):
- this_bp = this_pose[this_pose["bodyparts"] == bodyparts[j]][
- "value"
- ].values
- if this_bp[2] > lik_thresh:
- x = int(this_bp[0])
- y = int(this_bp[1])
- frame = cv2.circle(frame, (x, y), radius, colors[j], thickness=-1)
-
- if crop is not None:
- frame = frame[crop[0] : crop[1], crop[2] : crop[3]]
-
- if write_ts:
- frame = cv2.putText(
- frame,
- f"{(vid_time-write_ts_offset):0.3f}",
- ts_coord,
- ts_font,
- write_scale,
- ts_color,
- ts_size,
- )
-
- if display:
- cv2.imshow("DLC Live Labeled Video", frame)
- cv2.waitKey(1)
-
- vwriter.write(frame)
- label_times.append(cur_time)
- if save_images:
- new_file = f"{im_dir}/frame_{i}.png"
- cv2.imwrite(new_file, frame)
-
- if display:
- cv2.destroyAllWindows()
-
- vwriter.release()
- np.save(out_times_file, label_times)
-
-
-def main():
-
- import argparse
- import os
-
- parser = argparse.ArgumentParser()
- parser.add_argument("dir", type=str)
- parser.add_argument("-o", "--out-dir", type=str, default=None)
- parser.add_argument("--dlc-offline", action="store_true")
- parser.add_argument("-s", "--save-images", action="store_true")
- parser.add_argument("-u", "--cut", nargs="+", type=float, default=[0, np.Inf])
- parser.add_argument("-c", "--crop", nargs="+", type=int, default=None)
- parser.add_argument("-m", "--cmap", type=str, default="bmy")
- parser.add_argument("-r", "--radius", type=int, default=3)
- parser.add_argument("-l", "--lik-thresh", type=float, default=0.5)
- parser.add_argument("-w", "--write-ts", action="store_true")
- parser.add_argument("--write-scale", type=int, default=2)
- parser.add_argument("--write-pos", type=str, default="bottom-left")
- parser.add_argument("--write-ts-offset", type=float, default=0.0)
- parser.add_argument("-d", "--display", action="store_true")
- parser.add_argument("--no-progress", action="store_false")
- parser.add_argument("--no-label", action="store_false")
- args = parser.parse_args()
-
- create_labeled_video(
- args.dir,
- out_dir=args.out_dir,
- dlc_online=(not args.dlc_offline),
- save_images=args.save_images,
- cut=tuple(args.cut),
- crop=args.crop,
- cmap=args.cmap,
- radius=args.radius,
- lik_thresh=args.lik_thresh,
- write_ts=args.write_ts,
- write_scale=args.write_scale,
- write_pos=args.write_pos,
- write_ts_offset=args.write_ts_offset,
- display=args.display,
- progress=args.no_progress,
- label=args.no_label,
- )
diff --git a/docs/README.md b/docs/README.md
new file mode 100644
index 0000000..3cc524f
--- /dev/null
+++ b/docs/README.md
@@ -0,0 +1,262 @@
+# DeepLabCut-live-GUI Documentation Index
+
+Welcome to the DeepLabCut-live-GUI documentation! This index will help you find the information you need.
+
+## Getting Started
+
+### New Users
+1. **[README](../README.md)** - Project overview, installation, and quick start
+2. **[User Guide](user_guide.md)** - Step-by-step walkthrough of all features
+3. **[Installation Guide](install.md)** - Detailed installation instructions
+
+### Quick References
+- **[ARAVIS_QUICK_REF](../ARAVIS_QUICK_REF.md)** - Aravis backend quick reference
+- **[Features Overview](features.md)** - Complete feature documentation
+
+## Core Documentation
+
+### Camera Setup
+- **[Camera Support](camera_support.md)** - Overview of all camera backends
+- **[Aravis Backend](aravis_backend.md)** - Linux/macOS GenICam camera setup
+- Platform-specific guides for industrial cameras
+
+### Application Features
+- **[Features Documentation](features.md)** - Detailed feature descriptions:
+ - Camera control and backends
+ - Real-time pose estimation
+ - Video recording
+ - Configuration management
+ - Processor system
+ - User interface
+ - Performance monitoring
+ - Advanced features
+
+### User Guide
+- **[User Guide](user_guide.md)** - Complete usage walkthrough:
+ - Getting started
+ - Camera setup
+ - DLCLive configuration
+ - Recording videos
+ - Configuration management
+ - Common workflows
+ - Tips and best practices
+ - Troubleshooting
+
+## Advanced Topics
+
+### Processor System
+- **[Processor Plugins](PLUGIN_SYSTEM.md)** - Custom pose processing
+- **[Processor Auto-Recording](processor_auto_recording.md)** - Event-triggered recording
+- Socket processor documentation
+
+### Technical Details
+- **[Timestamp Format](timestamp_format.md)** - Synchronization and timing
+- **[ARAVIS_BACKEND_SUMMARY](../ARAVIS_BACKEND_SUMMARY.md)** - Implementation details
+
+## By Use Case
+
+### I want to...
+
+#### Set up a camera
+→ [Camera Support](camera_support.md) → Select backend → Follow setup guide
+
+**By Platform**:
+- **Windows**: [README](../README.md#windows-gentl-for-industrial-cameras) → GenTL setup
+- **Linux**: [Aravis Backend](aravis_backend.md) → Installation for Ubuntu/Debian
+- **macOS**: [Aravis Backend](aravis_backend.md) → Installation via Homebrew
+
+**By Camera Type**:
+- **Webcam**: [User Guide](user_guide.md#camera-setup) → OpenCV backend
+- **Industrial Camera**: [Camera Support](camera_support.md) → GenTL/Aravis
+- **Basler Camera**: [Camera Support](camera_support.md#basler-cameras) → pypylon setup
+- **The Imaging Source**: [Aravis Backend](aravis_backend.md) or GenTL
+
+#### Run pose estimation
+→ [User Guide](user_guide.md#dlclive-configuration) → Load model → Start inference
+
+#### Record high-speed video
+→ [Features](features.md#video-recording) → Hardware encoding → GPU setup
+→ [User Guide](user_guide.md#high-speed-recording-60-fps) → Optimization tips
+
+#### Create custom processor
+→ [Processor Plugins](PLUGIN_SYSTEM.md) → Plugin architecture → Examples
+
+#### Trigger recording remotely
+→ [Features](features.md#auto-recording-feature) → Auto-recording setup
+→ Socket processor documentation
+
+#### Optimize performance
+→ [Features](features.md#performance-optimization) → Metrics → Adjustments
+→ [User Guide](user_guide.md#tips-and-best-practices) → Best practices
+
+## By Topic
+
+### Camera Backends
+| Backend | Documentation | Platform |
+|---------|---------------|----------|
+| OpenCV | [User Guide](user_guide.md#step-1-select-camera-backend) | All |
+| GenTL | [Camera Support](camera_support.md) | Windows, Linux |
+| Aravis | [Aravis Backend](aravis_backend.md) | Linux, macOS |
+| Basler | [Camera Support](camera_support.md#basler-cameras) | All |
+
+### Configuration
+- **Basics**: [README](../README.md#configuration)
+- **Management**: [User Guide](user_guide.md#working-with-configurations)
+- **Templates**: [User Guide](user_guide.md#configuration-templates)
+- **Details**: [Features](features.md#configuration-management)
+
+### Recording
+- **Quick Start**: [User Guide](user_guide.md#recording-videos)
+- **Features**: [Features](features.md#video-recording)
+- **Optimization**: [README](../README.md#performance-optimization)
+- **Auto-Recording**: [Features](features.md#auto-recording-feature)
+
+### DLCLive
+- **Setup**: [User Guide](user_guide.md#dlclive-configuration)
+- **Models**: [Features](features.md#model-support)
+- **Performance**: [Features](features.md#performance-metrics)
+- **Visualization**: [Features](features.md#pose-visualization)
+
+## Troubleshooting
+
+### Quick Fixes
+1. **Camera not detected** → [User Guide](user_guide.md#troubleshooting-guide)
+2. **Slow inference** → [Features](features.md#performance-optimization)
+3. **Dropped frames** → [README](../README.md#troubleshooting)
+4. **Recording issues** → [User Guide](user_guide.md#troubleshooting-guide)
+
+### Detailed Troubleshooting
+- [User Guide - Troubleshooting Section](user_guide.md#troubleshooting-guide)
+- [README - Troubleshooting](../README.md#troubleshooting)
+- [Aravis Backend - Troubleshooting](aravis_backend.md#troubleshooting)
+
+## Development
+
+### Architecture
+- **Project Structure**: [README](../README.md#development)
+- **Backend Development**: [Camera Support](camera_support.md#contributing-new-camera-types)
+- **Processor Development**: [Processor Plugins](PLUGIN_SYSTEM.md)
+
+### Implementation Details
+- **Aravis Backend**: [ARAVIS_BACKEND_SUMMARY](../ARAVIS_BACKEND_SUMMARY.md)
+- **Thread Safety**: [Features](features.md#thread-safety)
+- **Resource Management**: [Features](features.md#resource-management)
+
+## Reference
+
+### Configuration Schema
+```json
+{
+ "camera": {
+ "name": "string",
+ "index": "number",
+ "fps": "number",
+ "backend": "opencv|gentl|aravis|basler",
+ "exposure": "number (μs, 0=auto)",
+ "gain": "number (0.0=auto)",
+ "crop_x0/y0/x1/y1": "number",
+ "max_devices": "number",
+ "properties": "object"
+ },
+ "dlc": {
+ "model_path": "string",
+ "model_type": "base|pytorch",
+ "additional_options": "object"
+ },
+ "recording": {
+ "enabled": "boolean",
+ "directory": "string",
+ "filename": "string",
+ "container": "mp4|avi|mov",
+ "codec": "h264_nvenc|libx264|hevc_nvenc",
+ "crf": "number (0-51)"
+ },
+ "bbox": {
+ "enabled": "boolean",
+ "x0/y0/x1/y1": "number"
+ }
+}
+```
+
+### Performance Metrics
+- **Camera FPS**: [Features](features.md#camera-metrics)
+- **DLC Metrics**: [Features](features.md#dlc-metrics)
+- **Recording Metrics**: [Features](features.md#recording-metrics)
+
+### Keyboard Shortcuts
+| Action | Shortcut |
+|--------|----------|
+| Load configuration | Ctrl+O |
+| Save configuration | Ctrl+S |
+| Save as | Ctrl+Shift+S |
+| Quit | Ctrl+Q |
+
+## External Resources
+
+### DeepLabCut
+- [DeepLabCut](http://www.mackenziemathislab.org/deeplabcut)
+- [DeepLabCut-live](https://github.com/DeepLabCut/DeepLabCut-live)
+- [DeepLabCut Documentation](http://deeplabcut.github.io/DeepLabCut/docs/intro.html)
+
+### Camera Libraries
+- [Aravis Project](https://github.com/AravisProject/aravis)
+- [Harvesters (GenTL)](https://github.com/genicam/harvesters)
+- [pypylon (Basler)](https://github.com/basler/pypylon)
+- [OpenCV](https://opencv.org/)
+
+### Video Encoding
+- [FFmpeg](https://ffmpeg.org/)
+- [NVENC (NVIDIA)](https://developer.nvidia.com/nvidia-video-codec-sdk)
+
+## Getting Help
+
+### Support Channels
+1. Check relevant documentation (use this index!)
+2. Search GitHub issues
+3. Review example configurations
+4. Contact maintainers
+
+### Reporting Issues
+When reporting bugs, include:
+- GUI version
+- Platform (OS, Python version)
+- Camera backend and model
+- Configuration file (if applicable)
+- Error messages
+- Steps to reproduce
+
+## Contributing
+
+Interested in contributing?
+- See [README - Contributing](../README.md#contributing)
+- Review [Development Section](../README.md#development)
+- Check open GitHub issues
+- Read coding guidelines
+
+---
+
+## Document Version History
+
+- **v1.0** - Initial comprehensive documentation
+ - Complete README overhaul
+ - User guide creation
+ - Features documentation
+ - Camera backend guides
+ - Aravis backend implementation
+
+## Quick Navigation
+
+**Popular Pages**:
+- [User Guide](user_guide.md) - Most comprehensive walkthrough
+- [Features](features.md) - All capabilities detailed
+- [Aravis Setup](aravis_backend.md) - Linux industrial cameras
+- [Camera Support](camera_support.md) - All camera backends
+
+**By Experience Level**:
+- **Beginner**: [README](../README.md) → [User Guide](user_guide.md)
+- **Intermediate**: [Features](features.md) → [Camera Support](camera_support.md)
+- **Advanced**: [Processor Plugins](PLUGIN_SYSTEM.md) → Implementation details
+
+---
+
+*Last updated: 2025-10-24*
diff --git a/docs/aravis_backend.md b/docs/aravis_backend.md
new file mode 100644
index 0000000..67024ba
--- /dev/null
+++ b/docs/aravis_backend.md
@@ -0,0 +1,202 @@
+# Aravis Backend
+
+The Aravis backend provides support for GenICam-compatible cameras using the [Aravis](https://github.com/AravisProject/aravis) library.
+
+## Features
+
+- Support for GenICam/GigE Vision cameras
+- Automatic device detection with `get_device_count()`
+- Configurable exposure time and gain
+- Support for various pixel formats (Mono8, Mono12, Mono16, RGB8, BGR8)
+- Efficient streaming with configurable buffer count
+- Timeout handling for robust operation
+
+## Installation
+
+### Linux (Ubuntu/Debian)
+```bash
+sudo apt-get install gir1.2-aravis-0.8 python3-gi
+```
+
+### Linux (Fedora)
+```bash
+sudo dnf install aravis python3-gobject
+```
+
+### Windows
+Aravis support on Windows requires building from source or using WSL. For native Windows support, consider using the GenTL backend instead.
+
+### macOS
+```bash
+brew install aravis
+pip install pygobject
+```
+
+## Configuration
+
+### Basic Configuration
+
+Select "aravis" as the backend in the GUI or in your configuration file:
+
+```json
+{
+ "camera": {
+ "backend": "aravis",
+ "index": 0,
+ "fps": 30.0,
+ "exposure": 10000,
+ "gain": 5.0
+ }
+}
+```
+
+### Advanced Properties
+
+You can configure additional Aravis-specific properties via the `properties` dictionary:
+
+```json
+{
+ "camera": {
+ "backend": "aravis",
+ "index": 0,
+ "fps": 30.0,
+ "exposure": 10000,
+ "gain": 5.0,
+ "properties": {
+ "camera_id": "MyCamera-12345",
+ "pixel_format": "Mono8",
+ "timeout": 2000000,
+ "n_buffers": 10
+ }
+ }
+}
+```
+
+#### Available Properties
+
+| Property | Type | Default | Description |
+|----------|------|---------|-------------|
+| `camera_id` | string | None | Specific camera ID to open (overrides index) |
+| `pixel_format` | string | "Mono8" | Pixel format: Mono8, Mono12, Mono16, RGB8, BGR8 |
+| `timeout` | int | 2000000 | Frame timeout in microseconds (2 seconds) |
+| `n_buffers` | int | 10 | Number of buffers in the acquisition stream |
+
+### Exposure and Gain
+
+The Aravis backend supports exposure time (in microseconds) and gain control:
+
+- **Exposure**: Set via the GUI exposure field or `settings.exposure` (0 = auto, >0 = manual in μs)
+- **Gain**: Set via the GUI gain field or `settings.gain` (0.0 = auto, >0.0 = manual value)
+
+When exposure or gain are set to non-zero values, the backend automatically disables auto-exposure and auto-gain.
+
+## Camera Selection
+
+### By Index
+The default method is to select cameras by index (0, 1, 2, etc.):
+```json
+{
+ "camera": {
+ "backend": "aravis",
+ "index": 0
+ }
+}
+```
+
+### By Camera ID
+You can also select a specific camera by its ID:
+```json
+{
+ "camera": {
+ "backend": "aravis",
+ "properties": {
+ "camera_id": "TheImagingSource-12345678"
+ }
+ }
+}
+```
+
+## Supported Pixel Formats
+
+The backend automatically converts different pixel formats to BGR format for consistency:
+
+- **Mono8**: 8-bit grayscale → BGR
+- **Mono12**: 12-bit grayscale → scaled to 8-bit → BGR
+- **Mono16**: 16-bit grayscale → scaled to 8-bit → BGR
+- **RGB8**: 8-bit RGB → BGR (color conversion)
+- **BGR8**: 8-bit BGR (no conversion needed)
+
+## Performance Tuning
+
+### Buffer Count
+Increase `n_buffers` for high-speed cameras or systems with variable latency:
+```json
+{
+ "properties": {
+ "n_buffers": 20
+ }
+}
+```
+
+### Timeout
+Adjust timeout for slower cameras or network cameras:
+```json
+{
+ "properties": {
+ "timeout": 5000000
+ }
+}
+```
+(5 seconds = 5,000,000 microseconds)
+
+## Troubleshooting
+
+### No cameras detected
+1. Verify Aravis installation: `arv-tool-0.8 -l`
+2. Check camera is powered and connected
+3. Ensure proper network configuration for GigE cameras
+4. Check user permissions for USB cameras
+
+### Timeout errors
+- Increase the `timeout` property
+- Check network bandwidth for GigE cameras
+- Verify camera is properly configured and streaming
+
+### Pixel format errors
+- Check camera's supported pixel formats: `arv-tool-0.8 -n features`
+- Try alternative formats: Mono8, RGB8, etc.
+
+## Comparison with GenTL Backend
+
+| Feature | Aravis | GenTL |
+|---------|--------|-------|
+| Platform | Linux (best), macOS | Windows (best), Linux |
+| Camera Support | GenICam/GigE | GenTL producers |
+| Installation | System packages | Vendor CTI files |
+| Performance | Excellent | Excellent |
+| Auto-detection | Yes | Yes |
+
+## Example: The Imaging Source Camera
+
+```json
+{
+ "camera": {
+ "backend": "aravis",
+ "index": 0,
+ "fps": 60.0,
+ "exposure": 8000,
+ "gain": 10.0,
+ "properties": {
+ "pixel_format": "Mono8",
+ "n_buffers": 15,
+ "timeout": 3000000
+ }
+ }
+}
+```
+
+## Resources
+
+- [Aravis Project](https://github.com/AravisProject/aravis)
+- [GenICam Standard](https://www.emva.org/standards-technology/genicam/)
+- [Python GObject Documentation](https://pygobject.readthedocs.io/)
diff --git a/docs/camera_support.md b/docs/camera_support.md
index 6e36e22..ed0b1e0 100644
--- a/docs/camera_support.md
+++ b/docs/camera_support.md
@@ -1,135 +1,76 @@
## Camera Support
-### Windows
-- **The Imaging Source USB3 Cameras**: via code based on [Windows code samples](https://github.com/TheImagingSource/IC-Imaging-Control-Samples) provided by The Imaging Source. To use The Imaging Source USB3 cameras on Windows, you must first [install their drivers](https://www.theimagingsource.com/support/downloads-for-windows/device-drivers/icwdmuvccamtis/) and [C library](https://www.theimagingsource.com/support/downloads-for-windows/software-development-kits-sdks/tisgrabberdll/).
-- **OpenCV compatible cameras**: OpenCV is installed with DeepLabCut-live-GUI, so webcams or other cameras compatible with OpenCV on Windows require no additional installation.
+DeepLabCut-live-GUI supports multiple camera backends for different platforms and camera types:
-### Linux and NVIDIA Jetson Development Kits
+### Supported Backends
-- **OpenCV compatible cameras**: We provide support for many webcams and industrial cameras using OpenCV via Video4Linux drivers. This includes The Imaging Source USB3 cameras (and others, but untested). OpenCV is installed with DeepLabCut-live-GUI.
-- **Aravis Project compatible USB3Vision and GigE Cameras**: [The Aravis Project](https://github.com/AravisProject/aravis) supports a number of popular industrial cameras used in neuroscience, including The Imaging Source, Point Grey, and Basler cameras. To use Aravis Project drivers, please follow their [installation instructions](https://github.com/AravisProject/aravis#installing-aravis). The Aravis Project drivers are supported on the NVIDIA Jetson platform, but there are known bugs (e.g. [here](https://github.com/AravisProject/aravis/issues/324)).
+1. **OpenCV** - Universal webcam and USB camera support (all platforms)
+2. **GenTL** - Industrial cameras via GenTL producers (Windows, Linux)
+3. **Aravis** - GenICam/GigE Vision cameras (Linux, macOS)
+4. **Basler** - Basler cameras via pypylon (all platforms)
-### Contributing New Camera Types
+### Backend Selection
-Any camera that can be accessed through python (e.g. if the company offers a python package) can be integrated into the DeepLabCut-live-GUI. To contribute, please build off of our [base `Camera` class](../dlclivegui/camera/camera.py), and please use our [currently supported cameras](../dlclivegui/camera) as examples.
-
-New camera classes must inherit our base camera class, and provide at least two arguments:
-
-- id: an arbitrary name for a camera
-- resolution: the image size
-
-Other common options include:
-
-- exposure
-- gain
-- rotate
-- crop
-- fps
-
-If the camera does not have it's own display module, you can use our Tkinter video display built into the DeepLabCut-live-GUI by passing `use_tk_display=True` to the base camera class, and control the size of the displayed image using the `display_resize` parameter (`display_resize=1` for full image, `display_resize=0.5` to display images at half the width and height of recorded images).
-
-Here is an example of a camera that allows users to set the resolution, exposure, and crop, and uses the Tkinter display:
-
-```python
-from dlclivegui import Camera
-
-class MyNewCamera(Camera)
-
- def __init__(self, id="", resolution=[640, 480], exposure=0, crop=None, display_resize=1):
- super().__init__(id,
- resolution=resolution,
- exposure=exposure,
- crop=crop,
- use_tk_display=True,
- display_resize=display_resize)
+You can select the backend in the GUI from the "Backend" dropdown, or in your configuration file:
+```json
+{
+ "camera": {
+ "backend": "aravis",
+ "index": 0,
+ "fps": 30.0
+ }
+}
```
-All arguments of your camera's `__init__` method will be available to edit in the GUI's `Edit Camera Settings` window. To ensure that you pass arguments of the correct data type, it is helpful to provide default values for each argument of the correct data type (e.g. if `myarg` is a string, please use `myarg=""` instead of `myarg=None`). If a certain argument has only a few possible values, and you want to limit the options user's can input into the `Edit Camera Settings` window, please implement a `@static_method` called `arg_restrictions`. This method should return a dictionary where the keys are the arguments for which you want to provide value restrictions, and the values are the possible values that a specific argument can take on. Below is an example that restrictions the values for `use_tk_display` to `True` or `False`, and restricts the possible values of `resolution` to `[640, 480]` or `[320, 240]`.
-
-```python
- @static_method
- def arg_restrictions():
- return {'use_tk_display' : [True, False],
- 'resolution' : [[640, 480], [320, 240]]}
-```
-
-In addition to an `__init__` method that calls the `dlclivegui.Camera.__init__` method, you need to overwrite the `dlclivegui.Camera.set_capture_device`, `dlclive.Camera.close_capture_device`, and one of the following two methods: `dlclivegui.Camera.get_image` or `dlclivegui.Camera.get_image_on_time`.
-
-Your camera class's `set_capture_device` method should open the camera feed and confirm that the appropriate settings (such as exposure, rotation, gain, etc.) have been properly set. The `close_capture_device` method should simply close the camera stream. For example, see the [OpenCV camera](../dlclivegui/camera/opencv.py) `set_capture_device` and `close_capture_device` method.
-
-If you're camera has built in methods to ensure the correct frame rate (e.g. when grabbing images, it will block until the next image is ready), then overwrite the `get_image_on_time` method. If the camera does not block until the next image is ready, then please set the `get_image` method, and the base camera class's `get_image_on_time` method will ensure that images are only grabbed at the specified frame rate.
-
-The `get_image` method has no input arguments, but must return an image as a numpy array. We also recommend converting images to 8-bit integers (data type `uint8`).
-
-The `get_image_on_time` method has no input arguments, but must return an image as a numpy array (as in `get_image`) and the timestamp at which the image is returned (using python's `time.time()` function).
-
-### Camera Specific Tips for Installation & Use:
-
-#### Basler cameras
-
-Basler USB3 cameras are compatible with Aravis. However, integration with DeepLabCut-live-GUI can also be obtained with `pypylon`, the python module to drive Basler cameras, and supported by the company. Please note using `pypylon` requires you to install Pylon viewer, a free of cost GUI also developed and supported by Basler and available on several platforms.
+### Platform-Specific Recommendations
-* **Pylon viewer**: https://www.baslerweb.com/en/sales-support/downloads/software-downloads/#type=pylonsoftware;language=all;version=all
-* `pypylon`: https://github.com/basler/pypylon/releases
+#### Windows
+- **OpenCV compatible cameras**: Best for webcams and simple USB cameras. OpenCV is installed with DeepLabCut-live-GUI.
+- **GenTL backend**: Recommended for industrial cameras (The Imaging Source, Basler, etc.) via vendor-provided CTI files.
+- **Basler cameras**: Can use either GenTL or pypylon backend.
-If you want to use DeepLabCut-live-GUI with a Basler USB3 camera via pypylon, see the folllowing instructions. Please note this is tested on Ubuntu 20.04. It may (or may not) work similarly in other platforms (contributed by [@antortjim](https://github.com/antortjim)). This procedure should take around 10 minutes:
+#### Linux
+- **OpenCV compatible cameras**: Good for webcams via Video4Linux drivers. Installed with DeepLabCut-live-GUI.
+- **Aravis backend**: **Recommended** for GenICam/GigE Vision industrial cameras (The Imaging Source, Basler, Point Grey, etc.)
+ - Easy installation via system package manager
+ - Better Linux support than GenTL
+ - See [Aravis Backend Documentation](aravis_backend.md)
+- **GenTL backend**: Alternative for industrial cameras if vendor provides Linux CTI files.
-**Install Pylon viewer**
+#### macOS
+- **OpenCV compatible cameras**: For webcams and compatible USB cameras.
+- **Aravis backend**: For GenICam/GigE Vision cameras (requires Homebrew installation).
-1. Download .deb file
-Download the .deb file in the downloads center of Basler. Last version as of writing this was **pylon 6.2.0 Camera Software Suite Linux x86 (64 Bit) - Debian Installer Package**.
+### Quick Installation Guide
-
-2. Install .deb file
-
-```
-sudo dpkg -i pylon_6.2.0.21487-deb0_amd64.deb
+#### Aravis (Linux/Ubuntu)
+```bash
+sudo apt-get install gir1.2-aravis-0.8 python3-gi
```
-**Install swig**
-
-Required for compilation of non python code within pypylon
-
-1. Install swig dependencies
-
-You may have to install these in a fresh Ubuntu 20.04 install
-
-```
-sudo apt install gcc g++
-sudo apt install libpcre3-dev
-sudo apt install make
+#### Aravis (macOS)
+```bash
+brew install aravis
+pip install pygobject
```
-2. Download swig
+#### GenTL (Windows)
+Install vendor-provided camera drivers and SDK. CTI files are typically in:
+- `C:\Program Files\The Imaging Source Europe GmbH\IC4 GenTL Driver\bin\`
-Go to http://prdownloads.sourceforge.net/swig/swig-4.0.2.tar.gz and download the tar gz
+### Backend Comparison
-3. Install swig
-```
-tar -zxvf swig-4.0.2.tar.gz
-cd swig-4.0.2
-./configure
-make
-sudo make install
-```
+| Feature | OpenCV | GenTL | Aravis | Basler (pypylon) |
+|---------|--------|-------|--------|------------------|
+| Exposure control | No | Yes | Yes | Yes |
+| Gain control | No | Yes | Yes | Yes |
+| Windows | ✅ | ✅ | ❌ | ✅ |
+| Linux | ✅ | ✅ | ✅ | ✅ |
+| macOS | ✅ | ❌ | ✅ | ✅ |
-**Install pypylon**
-
-1. Download pypylon
-
-```
-wget https://github.com/basler/pypylon/archive/refs/tags/1.7.2.tar.gz
-```
-
-or go to https://github.com/basler/pypylon/releases and get the version you want!
-
-2. Install pypylon
-
-```
-tar -zxvf 1.7.2.tar.gz
-cd pypylon-1.7.2
-python setup.py install
-```
+### Detailed Backend Documentation
-Once you have completed these steps, you should be able to call your Basler camera from DeepLabCut-live-GUI using the BaslerCam camera type that appears after clicking "Add camera")
+- [Aravis Backend](aravis_backend.md) - GenICam/GigE cameras on Linux/macOS
+- GenTL Backend - Industrial cameras via vendor CTI files
+- OpenCV Backend - Universal webcam support
diff --git a/docs/features.md b/docs/features.md
new file mode 100644
index 0000000..91d87ad
--- /dev/null
+++ b/docs/features.md
@@ -0,0 +1,473 @@
+# DeepLabCut-live-GUI Features
+
+## Table of Contents
+
+- [Camera Control](#camera-control)
+- [Real-Time Pose Estimation](#real-time-pose-estimation)
+- [Video Recording](#video-recording)
+- [Configuration Management](#configuration-management)
+- [Processor System](#processor-system)
+- [User Interface](#user-interface)
+- [Performance Monitoring](#performance-monitoring)
+- [Advanced Features](#advanced-features)
+
+---
+
+## Camera Control
+
+### Multi-Backend Support
+
+The GUI supports four different camera backends, each optimized for different use cases:
+
+#### OpenCV Backend
+- **Platform**: Windows, Linux
+- **Best For**: Webcams, simple USB cameras
+- **Installation**: Built-in with OpenCV
+- **Limitations**: Limited exposure/gain control
+
+#### GenTL Backend (Harvesters)
+- **Platform**: Windows, Linux
+- **Best For**: Industrial cameras with GenTL producers
+- **Installation**: Requires vendor CTI files
+- **Features**: Full camera control, smart device detection
+
+#### Aravis Backend
+- **Platform**: Linux (best)
+- **Best For**: GenICam/GigE Vision cameras
+- **Installation**: System packages (`gir1.2-aravis-0.8`)
+
+#### Basler Backend (pypylon)
+- **Platform**: Windows, Linux, macOS
+- **Best For**: Basler cameras specifically
+- **Installation**: Pylon SDK + pypylon
+- **Features**: Vendor-specific optimizations
+
+### Camera Settings
+
+#### Frame Rate Control
+- Range: 1-240 FPS (hardware dependent)
+- Real-time FPS monitoring
+- Automatic camera validation
+
+#### Exposure Control
+- Auto mode (value = 0)
+- Manual mode (microseconds)
+- Range: 0-1,000,000 μs
+- Real-time adjustment (backend dependent)
+
+#### Gain Control
+- Auto mode (value = 0.0)
+- Manual mode (gain value)
+- Range: 0.0-100.0
+- Useful for low-light conditions
+
+#### Region of Interest (ROI) Cropping
+- Define crop region: (x0, y0, x1, y1)
+- Applied before recording and inference
+- Reduces processing load
+- Maintains aspect ratio
+
+#### Image Rotation
+- 0°, 90°, 180°, 270° rotation
+- Applied to all outputs
+- Useful for mounted cameras
+
+### Smart Camera Detection
+
+The GUI intelligently detects available cameras:
+
+1. **Backend-Specific**: Each backend reports available cameras
+2. **No Blind Probing**: GenTL and Aravis query actual device count
+3. **Fast Refresh**: Only check connected devices
+4. **Detailed Labels**: Shows vendor, model, serial number
+
+Example detection output:
+```
+[CameraDetection] Available cameras for backend 'gentl':
+ ['0:DMK 37BUX287 (26320523)', '1:Basler acA1920 (40123456)']
+```
+
+---
+
+## Real-Time Pose Estimation
+
+### DLCLive Integration
+
+#### Model Support
+- **PyTorch**: PyTorch-exported models
+- Model selection via dropdown
+- Automatic model validation
+
+#### Inference Pipeline
+1. **Frame Acquisition**: Camera thread → Queue
+2. **Preprocessing**: Crop, resize (optional)
+3. **Inference**: DLCLive model processing
+4. **Pose Output**: (x, y) coordinates per keypoint
+5. **Visualization**: Optional overlay on video
+
+#### Performance Metrics
+- **Inference FPS**: Actual processing rate
+- **Latency**: Time from capture to pose output
+ - Last latency (ms)
+ - Average latency (ms)
+- **Queue Status**: Frame buffer depth
+- **Dropped Frames**: Count of skipped frames
+
+### Pose Visualization
+
+#### Overlay Options
+- **Toggle**: "Display pose predictions" checkbox
+- **Keypoint Markers**: Green circles at (x, y) positions
+- **Real-Time Update**: Synchronized with video feed
+- **No Performance Impact**: Rendering optimized
+
+#### Bounding Box Visualization
+- **Purpose**: Visual ROI definition
+- **Configuration**: (x0, y0, x1, y1) coordinates
+- **Color**: Red rectangle overlay
+- **Use Cases**:
+ - Crop region preview
+ - Analysis area marking
+ - Multi-region tracking
+
+### Initialization Feedback
+
+Visual indicators during model loading:
+1. **"Initializing DLCLive!"** - Blue button during load
+2. **"DLCLive running!"** - Green button when ready
+3. Status bar updates with progress
+
+---
+
+## Video Recording
+
+### Recording Capabilities
+
+#### Hardware-Accelerated Encoding
+- **NVENC (NVIDIA)**: GPU-accelerated H.264/H.265
+ - Codecs: `h264_nvenc`, `hevc_nvenc`
+ - 10x faster than software encoding
+ - Minimal CPU usage
+- **Software Encoding**: CPU-based fallback
+ - Codecs: `libx264`, `libx265`
+ - Universal compatibility
+
+#### Container Formats
+- **MP4**: Most compatible, web-ready
+- **AVI**: Legacy support
+- **MOV**: Apple ecosystem
+
+#### Quality Control
+- **CRF (Constant Rate Factor)**: 0-51
+ - 0 = Lossless (huge files)
+ - 23 = Default (good quality)
+ - 28 = High compression
+ - 51 = Lowest quality
+- **Presets**: ultrafast, fast, medium, slow
+
+### Recording Features
+
+#### Timestamp Synchronization
+- Frame-accurate timestamps
+- Microsecond precision
+- Synchronized with pose data
+- Stored in separate files
+
+#### Performance Monitoring
+- **Write FPS**: Actual encoding rate
+- **Queue Size**: Buffer depth (~ms)
+- **Latency**: Encoding delay
+- **Frames Written/Enqueued**: Progress tracking
+- **Dropped Frames**: Quality indicator
+
+#### Buffer Management
+- Configurable queue size
+- Automatic overflow handling
+- Warning on frame drops
+- Backpressure indication
+
+### Auto-Recording Feature
+
+Processor-triggered recording:
+
+1. **Enable**: Check "Auto-record video on processor command"
+2. **Processor Control**: Custom processor sets recording flag
+3. **Automatic Start**: GUI starts recording when flag set
+4. **Session Naming**: Uses processor-defined session name
+5. **Automatic Stop**: GUI stops when flag cleared
+
+**Use Cases**:
+- Event-triggered recording
+- Trial-based experiments
+- Conditional data capture
+- Remote control via socket
+
+---
+
+## Configuration Management
+
+### Configuration File Structure
+
+Single JSON file contains all settings:
+
+```json
+{
+ "camera": { ... },
+ "dlc": { ... },
+ "recording": { ... },
+ "bbox": { ... }
+}
+```
+
+### Features
+
+#### Save/Load Operations
+- **Load**: File → Load configuration (Ctrl+O)
+- **Save**: File → Save configuration (Ctrl+S)
+- **Save As**: File → Save configuration as (Ctrl+Shift+S)
+- **Auto-sync**: GUI fields update from file
+
+#### Multiple Configurations
+- Switch between experiments quickly
+- Per-animal configurations
+- Environment-specific settings
+- Backup and version control
+
+#### Validation
+- Type checking on load
+- Default values for missing fields
+- Error messages for invalid entries
+- Safe fallback to defaults
+---
+
+## Processor System
+
+### Plugin Architecture
+
+Custom pose processors for real-time analysis and control.
+
+#### Processor Interface
+
+```python
+class MyProcessor:
+ """Custom processor example."""
+
+ def process(self, pose, timestamp):
+ """Process pose data in real-time.
+
+ Args:
+ pose: numpy array (n_keypoints, 3) - x, y, likelihood
+ timestamp: float - frame timestamp
+ """
+ # Extract keypoint positions
+ nose_x, nose_y = pose[0, :2]
+
+ # Custom logic
+ if nose_x > 320:
+ self.trigger_event()
+
+ # Return results (optional)
+ return {"position": (nose_x, nose_y)}
+```
+
+#### Loading Processors
+
+1. Place processor file in `dlclivegui/processors/`
+2. Click "Refresh" in processor dropdown
+3. Select processor from list
+4. Start inference to activate
+
+#### Built-in Processors
+
+**Socket Processor** (`dlc_processor_socket.py`):
+- TCP socket server for remote control
+- Commands: `START_RECORDING`, `STOP_RECORDING`
+- Session management
+- Multi-client support
+
+### Auto-Recording Integration
+
+Processors can control recording:
+
+```python
+class RecordingProcessor:
+ def __init__(self):
+ self._vid_recording = False
+ self.session_name = "default"
+
+ @property
+ def video_recording(self):
+ return self._vid_recording
+
+ def start_recording(self, session):
+ self.session_name = session
+ self._vid_recording = True
+
+ def stop_recording(self):
+ self._vid_recording = False
+```
+
+The GUI monitors `video_recording` property and automatically starts/stops recording.
+
+---
+
+## User Interface
+
+### Layout
+
+#### Control Panel (Left)
+- **Camera Settings**: Backend, index, FPS, exposure, gain, crop
+- **DLC Settings**: Model path, type, processor, options
+- **Recording Settings**: Path, filename, codec, quality
+- **Bounding Box**: Visualization controls
+
+#### Video Display (Right)
+- Live camera feed
+- Pose overlay (optional)
+- Bounding box overlay (optional)
+- Auto-scaling to window size
+
+#### Status Bar (Bottom)
+- Current operation status
+- Error messages
+- Success confirmations
+
+### Control Groups
+
+#### Camera Controls
+- Backend selection dropdown
+- Camera index/refresh
+- FPS, exposure, gain spinboxes
+- Crop coordinates
+- Rotation selector
+- **Start/Stop Preview** buttons
+
+#### DLC Controls
+- Model path browser
+- Model type selector
+- Processor folder/selection
+- Additional options (JSON)
+- **Start/Stop Inference** buttons
+- "Display pose predictions" checkbox
+- "Auto-record" checkbox
+- Processor status display
+
+#### Recording Controls
+- Output directory browser
+- Filename input
+- Container/codec selectors
+- CRF quality slider
+- **Start/Stop Recording** buttons
+
+### Visual Feedback
+
+#### Button States
+- **Disabled**: Gray, not clickable
+- **Enabled**: Default color, clickable
+- **Active**:
+ - Preview running: Stop button enabled
+ - Inference initializing: Blue "Initializing DLCLive!"
+ - Inference ready: Green "DLCLive running!"
+
+#### Status Indicators
+- Camera FPS (last 5 seconds)
+- DLC performance metrics
+- Recording statistics
+- Processor connection status
+
+---
+
+## Performance Monitoring
+
+### Real-Time Metrics
+
+#### Camera Metrics
+- **Throughput**: FPS over last 5 seconds
+- **Formula**: `(frame_count - 1) / time_elapsed`
+- **Display**: "45.2 fps (last 5 s)"
+
+#### DLC Metrics
+- **Inference FPS**: Poses processed per second
+- **Latency**:
+ - Last frame latency (ms)
+ - Average latency over session (ms)
+- **Queue**: Number of frames waiting
+- **Dropped**: Frames skipped due to queue full
+- **Format**: "150/152 frames | inference 42.1 fps | latency 23.5 ms (avg 24.1 ms) | queue 2 | dropped 2"
+
+#### Recording Metrics
+- **Write FPS**: Encoding rate
+- **Frames**: Written/Enqueued ratio
+- **Latency**: Encoding delay (ms)
+- **Buffer**: Queue size (~milliseconds)
+- **Dropped**: Encoding failures
+- **Format**: "1500/1502 frames | write 59.8 fps | latency 12.3 ms (avg 12.5 ms) | queue 5 (~83 ms) | dropped 2"
+
+---
+
+## Advanced Features
+
+### Frame Synchronization
+
+All components share frame timestamps:
+- Camera controller generates timestamps
+- DLC processor preserves timestamps
+- Video recorder stores timestamps
+- Enables post-hoc alignment
+
+### Error Recovery
+
+#### Camera Connection Loss
+- Automatic detection via frame grab failure
+- User notification
+- Clean resource cleanup
+- Restart capability
+
+#### Recording Errors
+- Frame size mismatch detection
+- Automatic recovery with new settings
+- Warning display
+- No data loss
+
+### Thread Safety
+
+Multi-threaded architecture:
+- **Main Thread**: GUI event loop
+- **Camera Thread**: Frame acquisition
+- **DLC Thread**: Pose inference
+- **Recording Thread**: Video encoding
+
+Qt signals/slots ensure thread-safe communication.
+
+### Resource Management
+
+#### Automatic Cleanup
+- Camera release on stop/error
+- DLC model unload on stop
+- Recording finalization
+- Thread termination
+
+#### Memory Management
+- Bounded queues prevent memory leaks
+- Frame copy-on-write
+- Efficient numpy array handling
+
+### Extensibility
+
+### Debugging Features
+
+#### Logging
+- Console output for errors
+- Frame acquisition logging
+- Performance warnings
+- Connection status
+---
+---
+
+## Keyboard Shortcuts
+
+- **Ctrl+O**: Load configuration
+- **Ctrl+S**: Save configuration
+- **Ctrl+Shift+S**: Save configuration as
+- **Ctrl+Q**: Quit application
+---
diff --git a/docs/install.md b/docs/install.md
deleted file mode 100644
index fa69ab4..0000000
--- a/docs/install.md
+++ /dev/null
@@ -1,25 +0,0 @@
-## Installation Instructions
-
-### Windows or Linux Desktop
-
-We recommend that you install DeepLabCut-live in a conda environment. First, please install Anaconda:
-- [Windows](https://docs.anaconda.com/anaconda/install/windows/)
-- [Linux](https://docs.anaconda.com/anaconda/install/linux/)
-
-Create a conda environment with python 3.7 and tensorflow:
-```
-conda create -n dlc-live python=3.7 tensorflow-gpu==1.13.1 # if using GPU
-conda create -n dlc-live python=3.7 tensorflow==1.13.1 # if not using GPU
-```
-
-Activate the conda environment and install the DeepLabCut-live package:
-```
-conda activate dlc-live
-pip install deeplabcut-live-gui
-```
-
-### NVIDIA Jetson Development Kit
-
-First, please refer to our complete instructions for [installing DeepLabCut-Live! on a NVIDIA Jetson Development Kit](https://github.com/DeepLabCut/DeepLabCut-live/blob/master/docs/install_jetson.md).
-
-Next, install the DeepLabCut-live-GUI: `pip install deeplabcut-live-gui`.
\ No newline at end of file
diff --git a/docs/timestamp_format.md b/docs/timestamp_format.md
new file mode 100644
index 0000000..ac5e5a7
--- /dev/null
+++ b/docs/timestamp_format.md
@@ -0,0 +1,79 @@
+# Video Frame Timestamp Format
+
+When recording video, the application automatically saves frame timestamps to a JSON file alongside the video file.
+
+## File Naming
+
+For a video file named `recording_2025-10-23_143052.mp4`, the timestamp file will be:
+```
+recording_2025-10-23_143052.mp4_timestamps.json
+```
+
+## JSON Structure
+
+```json
+{
+ "video_file": "recording_2025-10-23_143052.mp4",
+ "num_frames": 1500,
+ "timestamps": [
+ 1729693852.123456,
+ 1729693852.156789,
+ 1729693852.190123,
+ ...
+ ],
+ "start_time": 1729693852.123456,
+ "end_time": 1729693902.123456,
+ "duration_seconds": 50.0
+}
+```
+
+## Fields
+
+- **video_file**: Name of the associated video file
+- **num_frames**: Total number of frames recorded
+- **timestamps**: Array of Unix timestamps (seconds since epoch with microsecond precision) for each frame
+- **start_time**: Timestamp of the first frame
+- **end_time**: Timestamp of the last frame
+- **duration_seconds**: Total recording duration in seconds
+
+## Usage
+
+The timestamps correspond to the exact time each frame was captured by the camera (from `FrameData.timestamp`). This allows precise synchronization with:
+
+- DLC pose estimation results
+- External sensors or triggers
+- Other data streams recorded during the same session
+
+## Example: Loading Timestamps in Python
+
+```python
+import json
+from datetime import datetime
+
+# Load timestamps
+with open('recording_2025-10-23_143052.mp4_timestamps.json', 'r') as f:
+ data = json.load(f)
+
+print(f"Video: {data['video_file']}")
+print(f"Total frames: {data['num_frames']}")
+print(f"Duration: {data['duration_seconds']:.2f} seconds")
+
+# Convert first timestamp to human-readable format
+start_dt = datetime.fromtimestamp(data['start_time'])
+print(f"Recording started: {start_dt.isoformat()}")
+
+# Calculate average frame rate
+avg_fps = data['num_frames'] / data['duration_seconds']
+print(f"Average FPS: {avg_fps:.2f}")
+
+# Access individual frame timestamps
+for frame_idx, timestamp in enumerate(data['timestamps']):
+ print(f"Frame {frame_idx}: {timestamp}")
+```
+
+## Notes
+
+- Timestamps use `time.time()` which returns Unix epoch time with high precision
+- Frame timestamps are captured when frames arrive from the camera, before any processing
+- If frames are dropped due to queue overflow, those frames will not have timestamps in the array
+- The timestamp array length should match the number of frames in the video file
diff --git a/docs/user_guide.md b/docs/user_guide.md
new file mode 100644
index 0000000..b6f1905
--- /dev/null
+++ b/docs/user_guide.md
@@ -0,0 +1,282 @@
+# DeepLabCut-live-GUI User Guide
+
+Complete walkthrough for using the DeepLabCut-live-GUI application.
+
+## Table of Contents
+
+1. [Getting Started](#getting-started)
+2. [Camera Setup](#camera-setup)
+3. [DLCLive Configuration](#dlclive-configuration)
+4. [Recording Videos](#recording-videos)
+---
+
+## Getting Started
+
+### First Launch
+
+1. Open a terminal/command prompt
+2. Run the application:
+ ```bash
+ dlclivegui
+ ```
+3. The main window will appear with three control panels and a video display area
+
+### Interface Overview
+
+```
+┌─────────────────────────────────────────────────────┐
+│ File Help │
+├─────────────┬───────────────────────────────────────┤
+│ Camera │ │
+│ Settings │ │
+│ │ │
+│ ─────────── │ Video Display │
+│ DLCLive │ │
+│ Settings │ │
+│ │ │
+│ ─────────── │ │
+│ Recording │ │
+│ Settings │ │
+│ │ │
+│ ─────────── │ │
+│ Bounding │ │
+│ Box │ │
+│ │ │
+│ ─────────── │ │
+│ [Preview] │ │
+│ [Stop] │ │
+└─────────────┴───────────────────────────────────────┘
+│ Status: Ready │
+└─────────────────────────────────────────────────────┘
+```
+
+---
+
+## Camera Setup
+
+### Step 1: Select Camera Backend
+
+The **Backend** dropdown shows available camera drivers:
+
+| Backend | When to Use |
+|---------|-------------|
+| **opencv** | Webcams, USB cameras (universal) |
+| **gentl** | Industrial cameras (Windows/Linux) |
+| **aravis** | GenICam/GigE cameras (Linux/macOS) |
+| **basler** | Basler cameras specifically |
+
+**Note**: Unavailable backends appear grayed out. Install required drivers to enable them.
+
+### Step 2: Select Camera
+
+1. Click **Refresh** next to the camera dropdown
+2. Wait for camera detection (1-3 seconds)
+3. Select your camera from the dropdown
+
+The list shows camera details:
+```
+0:DMK 37BUX287 (26320523)
+│ │ └─ Serial Number
+│ └─ Model Name
+└─ Index
+```
+
+### Step 3: Configure Camera Parameters
+
+#### Frame Rate
+- **Range**: 1-240 FPS (hardware dependent)
+- **Recommendation**: Start with 30 FPS, increase as needed
+- **Note**: Higher FPS = more processing load
+
+#### Exposure Time
+- **Auto**: Set to 0 (default)
+- **Manual**: Microseconds (e.g., 10000 = 10ms)
+- **Tips**:
+ - Shorter exposure = less motion blur
+ - Longer exposure = better low-light performance
+ - Typical range: 5,000-30,000 μs
+
+#### Gain
+- **Auto**: Set to 0.0 (default)
+- **Manual**: 0.0-100.0
+- **Tips**:
+ - Higher gain = brighter image but more noise
+ - Start low (5-10) and increase if needed
+ - Auto mode works well for most cases
+
+#### Cropping (Optional)
+Reduce frame size for faster processing:
+
+1. Set crop region: (x0, y0, x1, y1)
+ - x0, y0: Top-left corner
+ - x1, y1: Bottom-right corner
+2. Use Bounding Box visualization to preview
+3. Set all to 0 to disable cropping
+
+**Example**: Crop to center 640x480 region of 1280x720 camera:
+```
+x0: 320
+y0: 120
+x1: 960
+y1: 600
+```
+
+#### Rotation
+Select if camera is mounted at an angle:
+- 0° (default)
+- 90° (rotated right)
+- 180° (upside down)
+- 270° (rotated left)
+
+### Step 4: Start Camera Preview
+
+1. Click **Start Preview**
+2. Video feed should appear in the display area
+3. Check the **Throughput** metric below camera settings
+4. Verify frame rate matches expected value
+
+**Troubleshooting**:
+- **No preview**: Check camera connection and permissions
+- **Low FPS**: Reduce resolution or increase exposure time
+- **Black screen**: Check exposure settings
+- **Distorted image**: Verify backend compatibility
+
+---
+
+## DLCLive Configuration
+
+### Prerequisites
+
+1. Exported DLCLive model (see [DLC documentation](https://github.com/DeepLabCut/DeepLabCut/blob/main/docs/HelperFunctions.md#model-export-function))
+2. DeepLabCut-live installed (`pip install deeplabcut-live`)
+3. Camera preview running
+
+### Step 1: Select Model
+
+1. Click **Browse** next to "Model directory"
+2. Navigate to your exported DLCLive model folder
+3. Select the folder containing:
+ - `pose_cfg.yaml`
+ - Model weights (`.pb`, `.pth`, etc.)
+
+### Step 2: Choose Model Type
+We only support newer, pytorch based models.
+- **PyTorch**: PyTorch-based models (requires PyTorch)
+
+
+**Common options**:
+- `processor`: "cpu" or "gpu"
+- `resize`: Scale factor (0.5 = half size)
+- `pcutoff`: Likelihood threshold
+- `cropping`: Crop before inference
+
+### Step 4: Select Processor (Optional)
+
+If using custom pose processors:
+
+1. Click **Browse** next to "Processor folder" (or use default)
+2. Click **Refresh** to scan for processors
+3. Select processor from dropdown
+4. Processor will activate when inference starts
+
+### Step 5: Start Inference
+
+1. Ensure camera preview is running
+2. Click **Start pose inference**
+3. Button changes to "Initializing DLCLive!" (blue)
+4. Wait for model loading (5-30 seconds)
+5. Button changes to "DLCLive running!" (green)
+6. Check **Performance** metrics
+
+**Performance Metrics**:
+```
+150/152 frames | inference 42.1 fps | latency 23.5 ms (avg 24.1 ms) | queue 2 | dropped 2
+```
+- **150/152**: Processed/Total frames
+- **inference 42.1 fps**: Processing rate
+- **latency 23.5 ms**: Current processing delay
+- **queue 2**: Frames waiting
+- **dropped 2**: Skipped frames (due to full queue)
+
+### Step 6: Enable Visualization (Optional)
+
+Check **"Display pose predictions"** to overlay keypoints on video.
+
+- Keypoints appear as green circles
+- Updates in real-time with video
+- Can be toggled during inference
+
+---
+
+## Recording Videos
+
+### Basic Recording
+
+1. **Configure output path**:
+ - Click **Browse** next to "Output directory"
+ - Select or create destination folder
+
+2. **Set filename**:
+ - Enter base filename (e.g., "session_001")
+ - Extension added automatically based on container
+
+3. **Select format**:
+ - **Container**: mp4 (recommended), avi, mov
+ - **Codec**:
+ - `h264_nvenc` (NVIDIA GPU - fastest)
+ - `libx264` (CPU - universal)
+ - `hevc_nvenc` (NVIDIA H.265)
+
+4. **Set quality** (CRF slider):
+ - 0-17: Very high quality, large files
+ - 18-23: High quality (recommended)
+ - 24-28: Medium quality, smaller files
+ - 29-51: Lower quality, smallest files
+
+5. **Start recording**:
+ - Ensure camera preview is running
+ - Click **Start recording**
+ - **Stop recording** button becomes enabled
+
+6. **Monitor performance**:
+ - Check "Performance" metrics
+ - Watch for dropped frames
+ - Verify write FPS matches camera FPS
+
+### Advanced Recording Options
+
+#### High-Speed Recording (60+ FPS)
+
+**Settings**:
+- Codec: `h264_nvenc` (requires NVIDIA GPU)
+- CRF: 28 (higher compression)
+- Crop region: Reduce frame size
+- Close other applications
+
+#### High-Quality Recording
+
+**Settings**:
+- Codec: `libx264` or `h264_nvenc`
+- CRF: 18-20
+- Full resolution
+- Sufficient disk space
+
+
+### Auto-Recording
+
+Enable automatic recording triggered by processor events:
+
+1. **Select a processor** that supports auto-recording
+2. **Enable**: Check "Auto-record video on processor command"
+3. **Start inference**: Processor will control recording
+4. **Session management**: Files named by processor
+
+---
+## Next Steps
+
+- Explore [Features Documentation](features.md) for detailed capabilities
+- Review [Camera Backend Guide](camera_support.md) for advanced setup
+- Check [Processor System](PLUGIN_SYSTEM.md) for custom processing
+- See [Aravis Backend](aravis_backend.md) for Linux industrial cameras
+
+---
diff --git a/pyproject.toml b/pyproject.toml
new file mode 100644
index 0000000..46eb777
--- /dev/null
+++ b/pyproject.toml
@@ -0,0 +1,128 @@
+[build-system]
+requires = ["setuptools>=61.0", "wheel"]
+build-backend = "setuptools.build_meta"
+
+[project]
+name = "deeplabcut-live-gui"
+version = "2.0"
+description = "PySide6-based GUI to run real time DeepLabCut experiments"
+readme = "README.md"
+requires-python = ">=3.10"
+license = {text = "GNU Lesser General Public License v3 (LGPLv3)"}
+authors = [
+ {name = "A. & M. Mathis Labs", email = "adim@deeplabcut.org"}
+]
+keywords = ["deeplabcut", "pose estimation", "real-time", "gui", "deep learning"]
+classifiers = [
+ "Programming Language :: Python :: 3",
+ "Programming Language :: Python :: 3.10",
+ "Programming Language :: Python :: 3.11",
+ "Programming Language :: Python :: 3.12",
+ "License :: OSI Approved :: GNU Lesser General Public License v3 (LGPLv3)",
+ "Operating System :: OS Independent",
+ "Development Status :: 4 - Beta",
+ "Intended Audience :: Science/Research",
+ "Topic :: Scientific/Engineering :: Artificial Intelligence",
+]
+
+dependencies = [
+ "deeplabcut-live", # might be missing timm and scipy
+ "PySide6",
+ "qdarkstyle",
+ "numpy",
+ "opencv-python",
+ "pydantic>=2.0",
+ "vidgear[core]",
+ "matplotlib",
+]
+
+[project.optional-dependencies]
+basler = ["pypylon"]
+gentl = ["harvesters"]
+all = ["pypylon", "harvesters"]
+pytorch = [
+ "deeplabcut-live[pytorch]",
+]
+tf = [
+ "deeplabcut-live[tf]",
+]
+dev = [
+ "pytest>=7.0",
+ "pytest-cov>=4.0",
+ "pytest-mock>=3.10",
+ "pytest-qt>=4.2",
+ "pre-commit",
+]
+test = [
+ "pytest>=7.0",
+ "pytest-cov>=4.0",
+ "pytest-mock>=3.10",
+ "pytest-qt>=4.2",
+]
+
+[project.urls]
+Homepage = "https://github.com/DeepLabCut/DeepLabCut-live-GUI"
+Repository = "https://github.com/DeepLabCut/DeepLabCut-live-GUI"
+Documentation = "https://github.com/DeepLabCut/DeepLabCut-live-GUI"
+"Bug Tracker" = "https://github.com/DeepLabCut/DeepLabCut-live-GUI/issues"
+
+[project.scripts]
+dlclivegui = "dlclivegui:main"
+
+[tool.setuptools]
+include-package-data = true
+
+[tool.setuptools.packages.find]
+where = ["."]
+include = ["dlclivegui*"]
+exclude = ["tests*", "docs*"]
+
+[tool.pytest.ini_options]
+testpaths = ["tests"]
+python_files = ["test_*.py"]
+python_classes = ["Test*"]
+python_functions = ["test_*"]
+addopts = [
+ "--strict-markers",
+ "--strict-config",
+ "--disable-warnings",
+ # "--maxfail=1",
+ "-ra",
+ "-q",
+]
+markers = [
+ "unit: Unit tests for individual components",
+ "integration: Integration tests for component interaction",
+ "functional: Functional tests for end-to-end workflows",
+ "slow: Tests that take a long time to run",
+ "gui: Tests that require GUI interaction",
+]
+
+[tool.coverage.run]
+source = ["dlclivegui"]
+omit = [
+ "*/tests/*",
+ "*/__pycache__/*",
+ "*/site-packages/*",
+]
+
+[tool.coverage.report]
+exclude_lines = [
+ "pragma: no cover",
+ "def __repr__",
+ "raise AssertionError",
+ "raise NotImplementedError",
+ "if __name__ == .__main__.:",
+ "if TYPE_CHECKING:",
+ "@abstract",
+]
+
+[tool.ruff]
+lint.select = ["E", "F", "B", "I", "UP"]
+lint.ignore = ["E741"]
+target-version = "py310"
+fix = true
+line-length = 120
+
+[tool.ruff.lint.pydocstyle]
+convention = "google"
diff --git a/setup.py b/setup.py
index 28eb6cc..1c6d5fa 100644
--- a/setup.py
+++ b/setup.py
@@ -1,36 +1,33 @@
-"""
-DeepLabCut Toolbox (deeplabcut.org)
-© A. & M. Mathis Labs
-
-Licensed under GNU Lesser General Public License v3.0
-"""
+"""Setup configuration for the DeepLabCut Live GUI."""
+from __future__ import annotations
import setuptools
-with open("README.md", "r") as fh:
+with open("README.md", "r", encoding="utf-8") as fh:
long_description = fh.read()
setuptools.setup(
name="deeplabcut-live-gui",
- version="1.0",
+ version="2.0",
author="A. & M. Mathis Labs",
author_email="adim@deeplabcut.org",
- description="GUI to run real time deeplabcut experiments",
+ description="PySide6-based GUI to run real time DeepLabCut experiments",
long_description=long_description,
long_description_content_type="text/markdown",
url="https://github.com/DeepLabCut/DeepLabCut-live-GUI",
- python_requires=">=3.5, <3.11",
+ python_requires=">=3.10",
install_requires=[
"deeplabcut-live",
- "pyserial",
- "pandas",
- "tables",
- "multiprocess",
- "imutils",
- "pillow",
- "tqdm",
+ "PySide6",
+ "numpy",
+ "opencv-python",
+ "vidgear[core]",
],
+ extras_require={
+ "basler": ["pypylon"],
+ "gentl": ["harvesters"],
+ },
packages=setuptools.find_packages(),
include_package_data=True,
classifiers=(
@@ -40,8 +37,7 @@
),
entry_points={
"console_scripts": [
- "dlclivegui=dlclivegui.dlclivegui:main",
- "dlclivegui-video=dlclivegui.video:main",
+ "dlclivegui=dlclivegui.gui:main",
]
},
)
diff --git a/tests/cameras/test_adapters.py b/tests/cameras/test_adapters.py
new file mode 100644
index 0000000..e69de29
diff --git a/tests/cameras/test_backend_discovery.py b/tests/cameras/test_backend_discovery.py
new file mode 100644
index 0000000..cb17a0a
--- /dev/null
+++ b/tests/cameras/test_backend_discovery.py
@@ -0,0 +1,133 @@
+# tests/cameras/test_backend_discovery.py
+from __future__ import annotations
+
+import sys
+import textwrap
+from pathlib import Path
+
+import pytest
+
+from dlclivegui.cameras import factory as cam_factory
+from dlclivegui.cameras.base import _BACKEND_REGISTRY, reset_backends
+from dlclivegui.utils.config_models import CameraSettingsModel
+
+
+def _write_temp_backend_package(tmp_path: Path, pkg_name: str = "test_backends_pkg") -> str:
+ """
+ Create a temporary backend package with a single backend module that registers
+ itself using the @register_backend decorator.
+
+ Returns the *package name* to be used in CameraFactory's discovery list.
+ """
+ pkg_root = tmp_path / pkg_name
+ pkg_root.mkdir(parents=True, exist_ok=True)
+ (pkg_root / "__init__.py").write_text("# test backends package\n", encoding="utf-8")
+
+ # A backend module which registers itself as "lazyfake"
+ backend_code = textwrap.dedent(
+ """
+ from dlclivegui.cameras.base import register_backend, CameraBackend
+ from dlclivegui.utils.config_models import CameraSettingsModel
+ import numpy as np
+ import time
+
+ @register_backend("lazyfake")
+ class LazyFakeBackend(CameraBackend):
+ @classmethod
+ def is_available(cls) -> bool:
+ return True
+
+ def open(self) -> None:
+ # No-op open for testing
+ self._opened = True
+
+ def read(self):
+ # Small deterministic frame + timestamp
+ frame = np.zeros((2, 3, 3), dtype=np.uint8)
+ return frame, time.time()
+
+ def close(self) -> None:
+ self._opened = False
+
+ # Optional: friendly name for detect_cameras label
+ def device_name(self) -> str:
+ return self.settings.name or f"LazyFake #{self.settings.index}"
+ """
+ )
+ (pkg_root / "fake_backend.py").write_text(backend_code, encoding="utf-8")
+ return pkg_name
+
+
+@pytest.fixture
+def temp_backends_pkg(tmp_path, monkeypatch):
+ """
+ Fixture that creates a temporary backend package and configures CameraFactory
+ to import from it during lazy discovery. Resets the global registry/import flags.
+ """
+ # 1) Create on-disk package with a single backend
+ pkg_name = _write_temp_backend_package(tmp_path)
+
+ # 2) Ensure Python can import it
+ sys.path.insert(0, str(tmp_path))
+ try:
+ # 3) Reset registry & lazy-import flags
+ reset_backends()
+ monkeypatch.setattr(cam_factory, "_BACKENDS_IMPORTED", False, raising=False)
+ monkeypatch.setattr(cam_factory, "_BUILTIN_BACKEND_PACKAGES", (pkg_name,), raising=False)
+
+ sys.modules.pop(pkg_name, None)
+ sys.modules.pop(f"{pkg_name}.fake_backend", None)
+
+ yield pkg_name
+ finally:
+ # Cleanup sys.path
+ try:
+ sys.path.remove(str(tmp_path))
+ except ValueError:
+ pass
+ reset_backends()
+
+
+def test_backend_lazy_discovery_from_package(temp_backends_pkg):
+ """
+ Verify that calling CameraFactory.backend_names() triggers lazy import and
+ registers the backend found in the temporary package.
+ """
+ # Initially empty
+ assert len(_BACKEND_REGISTRY) == 0
+
+ names = set(cam_factory.CameraFactory.backend_names())
+ assert "lazyfake" in names, f"Expected 'lazyfake' in discovered backends, got {names}"
+ # Registry should now contain our backend
+ assert "lazyfake" in _BACKEND_REGISTRY
+
+
+def test_detect_and_create_with_discovered_backend(temp_backends_pkg):
+ """
+ Verify CameraFactory.detect_cameras() and CameraFactory.create() work
+ with the lazily-discovered backend.
+ """
+ # Trigger discovery
+ names = set(cam_factory.CameraFactory.backend_names())
+ assert "lazyfake" in names
+
+ # detect_cameras should instantiate/open/close without error and yield a label
+ detected = cam_factory.CameraFactory.detect_cameras("lazyfake", max_devices=1)
+ assert isinstance(detected, list)
+ assert len(detected) >= 1
+ # Our backend returns device_name() -> "Probe 0" (from factory) or our override in device_name
+ assert detected[0].index == 0
+ assert isinstance(detected[0].label, str)
+ assert len(detected[0].label) > 0
+
+ # create() should return an instance of our registered backend using a model-only settings
+ s = CameraSettingsModel(name="UnitCam", backend="lazyfake", index=0, fps=30.0)
+ backend = cam_factory.CameraFactory.create(s)
+ # A minimal behavior check: open/read/close work
+ backend.open()
+ frame, ts = backend.read()
+ backend.close()
+
+ assert frame is not None and getattr(frame, "shape", None) is not None
+ assert frame.shape == (2, 3, 3)
+ assert isinstance(ts, float)
diff --git a/tests/cameras/test_factory.py b/tests/cameras/test_factory.py
new file mode 100644
index 0000000..663e9a5
--- /dev/null
+++ b/tests/cameras/test_factory.py
@@ -0,0 +1,79 @@
+# tests/cameras/test_factory_basic.py
+import sys
+import types
+
+import pytest
+
+from dlclivegui.cameras import CameraFactory, DetectedCamera, base
+
+# from dlclivegui.config import CameraSettings
+from dlclivegui.utils.config_models import CameraSettingsModel
+
+
+@pytest.mark.unit
+def test_check_camera_available_quick_ping():
+ mod = types.ModuleType("mock_mod")
+
+ class MockBackend(base.CameraBackend):
+ @classmethod
+ def is_available(cls):
+ return True
+
+ @staticmethod
+ def quick_ping(i):
+ return i == 0
+
+ def open(self):
+ pass
+
+ def read(self):
+ return None, 0.0
+
+ def close(self):
+ pass
+
+ mod.MockBackend = MockBackend
+ sys.modules["mock_mod"] = mod
+ base.register_backend_direct("mock", MockBackend)
+
+ ok, msg = CameraFactory.check_camera_available(CameraSettingsModel(backend="mock", index=0))
+ assert ok is True
+
+ ok, msg = CameraFactory.check_camera_available(CameraSettingsModel(backend="mock", index=3))
+ assert ok is False
+
+
+@pytest.mark.unit
+def test_detect_cameras():
+ mod = types.ModuleType("detect_mod")
+
+ class DetectBackend(base.CameraBackend):
+ @classmethod
+ def is_available(cls):
+ return True
+
+ @staticmethod
+ def quick_ping(i):
+ return i in (0, 2) # pretend devices 0 and 2 exist
+
+ def open(self):
+ if self.settings.index not in (0, 2):
+ raise RuntimeError("no device")
+
+ def read(self):
+ return None, 0
+
+ def close(self):
+ pass
+
+ def stop(self):
+ pass
+
+ mod.DetectBackend = DetectBackend
+ sys.modules["detect_mod"] = mod
+ base.register_backend_direct("detect", DetectBackend)
+
+ detected = CameraFactory.detect_cameras("detect", max_devices=4)
+ assert isinstance(detected, list)
+ assert [c.index for c in detected] == [0, 2]
+ assert all(isinstance(c, DetectedCamera) for c in detected)
diff --git a/tests/cameras/test_fake_backend.py b/tests/cameras/test_fake_backend.py
new file mode 100644
index 0000000..ade97e0
--- /dev/null
+++ b/tests/cameras/test_fake_backend.py
@@ -0,0 +1,49 @@
+# tests/cameras/test_fake_backend.py
+import sys
+import types
+
+import numpy as np
+import pytest
+
+from dlclivegui.cameras import CameraFactory, base
+
+# from dlclivegui.config import CameraSettings
+from dlclivegui.utils.config_models import CameraSettingsModel
+
+
+@pytest.mark.functional
+def test_fake_backend_e2e():
+ mod = types.ModuleType("fake_mod")
+
+ class FakeBackend(base.CameraBackend):
+ @classmethod
+ def is_available(cls):
+ return True
+
+ def open(self):
+ self._opened = True
+
+ def read(self):
+ assert self._opened
+ img = np.zeros((10, 20, 3), dtype=np.uint8)
+ return img, 123.456
+
+ def close(self):
+ self._opened = False
+
+ def stop(self):
+ pass
+
+ mod.FakeBackend = FakeBackend
+ sys.modules["fake_mod"] = mod
+ base.register_backend_direct("fake2", FakeBackend)
+
+ s = CameraSettingsModel(backend="fake2", name="X")
+ cam = CameraFactory.create(s)
+ cam.open()
+ frame, ts = cam.read()
+
+ assert frame.shape == (10, 20, 3)
+ assert ts == 123.456
+
+ cam.close()
diff --git a/tests/conftest.py b/tests/conftest.py
new file mode 100644
index 0000000..e698dfe
--- /dev/null
+++ b/tests/conftest.py
@@ -0,0 +1,95 @@
+# tests/dlc/conftest.py
+
+from __future__ import annotations
+
+import time
+
+import numpy as np
+import pytest
+
+from dlclivegui.cameras import CameraFactory
+from dlclivegui.cameras.base import CameraBackend
+
+# from dlclivegui.config import DLCProcessorSettings
+from dlclivegui.utils.config_models import DLCProcessorSettingsModel
+
+# ---------------------------------------------------------------------
+# Test doubles
+# ---------------------------------------------------------------------
+
+
+class FakeDLCLive:
+ """A minimal fake DLCLive object for testing."""
+
+ def __init__(self, **opts):
+ self.opts = opts
+ self.init_called = False
+ self.pose_calls = 0
+
+ def init_inference(self, frame):
+ self.init_called = True
+
+ def get_pose(self, frame, frame_time=None):
+ self.pose_calls += 1
+ # Deterministic small pose array
+ return np.ones((2, 2), dtype=float)
+
+
+class FakeBackend(CameraBackend):
+ def __init__(self, settings):
+ super().__init__(settings)
+ self._opened = False
+ self._counter = 0
+
+ @classmethod
+ def is_available(cls) -> bool:
+ return True
+
+ def open(self) -> None:
+ self._opened = True
+
+ def read(self):
+ # Produce a deterministic small frame
+ if not self._opened:
+ raise RuntimeError("not opened")
+ self._counter += 1
+ frame = np.zeros((48, 64, 3), dtype=np.uint8)
+ ts = time.time()
+ return frame, ts
+
+ def close(self) -> None:
+ self._opened = False
+
+ def stop(self) -> None:
+ pass
+
+
+# ---------------------------------------------------------------------
+# Fixtures
+# ---------------------------------------------------------------------
+@pytest.fixture
+def patch_factory(monkeypatch):
+ def _create(settings):
+ return FakeBackend(settings)
+
+ monkeypatch.setattr(CameraFactory, "create", staticmethod(_create))
+ return _create
+
+
+@pytest.fixture
+def monkeypatch_dlclive(monkeypatch):
+ """
+ Replace the dlclive.DLCLive import with FakeDLCLive *within* the dlc_processor module.
+
+ Scope is function-level by default, which keeps tests isolated.
+ """
+ from dlclivegui.services import dlc_processor
+
+ monkeypatch.setattr(dlc_processor, "DLCLive", FakeDLCLive)
+ return FakeDLCLive
+
+
+@pytest.fixture
+def settings_model():
+ """A standard Pydantic DLCProcessorSettingsModel for tests."""
+ return DLCProcessorSettingsModel(model_path="dummy.pt")
diff --git a/tests/gui/camera_config/test_cam_dialog_e2e.py b/tests/gui/camera_config/test_cam_dialog_e2e.py
new file mode 100644
index 0000000..efe1797
--- /dev/null
+++ b/tests/gui/camera_config/test_cam_dialog_e2e.py
@@ -0,0 +1,103 @@
+# tests/gui/camera_config/test_cam_dialog_e2e.py
+from __future__ import annotations
+
+import numpy as np
+import pytest
+from PySide6.QtCore import Qt
+
+from dlclivegui.cameras import CameraFactory
+from dlclivegui.cameras.base import CameraBackend
+from dlclivegui.cameras.factory import DetectedCamera
+from dlclivegui.gui.camera_config_dialog import CameraConfigDialog
+from dlclivegui.utils.config_models import CameraSettingsModel, MultiCameraSettingsModel
+
+# ---------------- Fake backend ----------------
+
+
+class FakeBackend(CameraBackend):
+ def __init__(self, settings):
+ super().__init__(settings)
+ self._opened = False
+
+ def open(self):
+ self._opened = True
+
+ def close(self):
+ self._opened = False
+
+ def read(self):
+ return np.zeros((30, 40, 3), dtype=np.uint8), 0.1
+
+
+# ---------------- Fixtures ----------------
+
+
+@pytest.fixture
+def patch_factory(monkeypatch):
+ monkeypatch.setattr(CameraFactory, "create", lambda s: FakeBackend(s))
+ monkeypatch.setattr(
+ CameraFactory,
+ "detect_cameras",
+ lambda backend, max_devices=10, **kw: [
+ DetectedCamera(index=0, label=f"{backend}-X"),
+ DetectedCamera(index=1, label=f"{backend}-Y"),
+ ],
+ )
+
+
+@pytest.fixture
+def dialog(qtbot, patch_factory):
+ s = MultiCameraSettingsModel(
+ cameras=[
+ CameraSettingsModel(name="A", backend="opencv", index=0, enabled=True),
+ ]
+ )
+ d = CameraConfigDialog(None, s)
+ qtbot.addWidget(d)
+ return d
+
+
+# ---------------- End‑to‑End tests ----------------
+
+
+def test_e2e_async_camera_scan(dialog, qtbot):
+ qtbot.mouseClick(dialog.refresh_btn, Qt.LeftButton)
+
+ with qtbot.waitSignal(dialog.scan_finished, timeout=2000):
+ pass
+
+ assert dialog.available_cameras_list.count() == 2
+
+
+def test_e2e_preview_start_stop(dialog, qtbot):
+ dialog.active_cameras_list.setCurrentRow(0)
+
+ qtbot.mouseClick(dialog.preview_btn, Qt.LeftButton)
+
+ # loader thread finishes → preview becomes active
+ qtbot.waitUntil(lambda: dialog._loader is None and dialog._preview_active, timeout=2000)
+
+ assert dialog._preview_active
+
+ # preview running → pixmap must update
+ qtbot.waitUntil(lambda: dialog.preview_label.pixmap() is not None, timeout=2000)
+
+ # stop preview
+ qtbot.mouseClick(dialog.preview_btn, Qt.LeftButton)
+
+ assert dialog._preview_active is False
+ assert dialog._preview_backend is None
+
+
+def test_e2e_apply_settings_reopens_preview(dialog, qtbot):
+ dialog.active_cameras_list.setCurrentRow(0)
+ qtbot.mouseClick(dialog.preview_btn, Qt.LeftButton)
+
+ # Wait for preview start
+ qtbot.waitUntil(lambda: dialog._loader is None and dialog._preview_active, timeout=2000)
+
+ dialog.cam_fps.setValue(99.0)
+ qtbot.mouseClick(dialog.apply_settings_btn, Qt.LeftButton)
+
+ # Should still be active → restarted backend
+ qtbot.waitUntil(lambda: dialog._preview_active and dialog._preview_backend is not None, timeout=2000)
diff --git a/tests/gui/camera_config/test_cam_dialog_unit.py b/tests/gui/camera_config/test_cam_dialog_unit.py
new file mode 100644
index 0000000..83163e6
--- /dev/null
+++ b/tests/gui/camera_config/test_cam_dialog_unit.py
@@ -0,0 +1,74 @@
+# tests/gui/camera_config/test_cam_dialog_unit.py
+from __future__ import annotations
+
+import pytest
+from PySide6.QtCore import Qt
+
+from dlclivegui.cameras.factory import DetectedCamera
+from dlclivegui.gui.camera_config_dialog import CameraConfigDialog
+from dlclivegui.utils.config_models import CameraSettingsModel, MultiCameraSettingsModel
+
+
+@pytest.fixture
+def dialog(qtbot, monkeypatch):
+ # Patch detect_cameras to avoid hardware access
+ monkeypatch.setattr(
+ "dlclivegui.cameras.CameraFactory.detect_cameras",
+ lambda backend, max_devices=10, **kw: [
+ DetectedCamera(index=0, label=f"{backend}-X"),
+ DetectedCamera(index=1, label=f"{backend}-Y"),
+ ],
+ )
+
+ s = MultiCameraSettingsModel(
+ cameras=[
+ CameraSettingsModel(name="CamA", backend="opencv", index=0, enabled=True),
+ CameraSettingsModel(name="CamB", backend="opencv", index=1, enabled=False),
+ ]
+ )
+ d = CameraConfigDialog(None, s)
+ qtbot.addWidget(d)
+ return d
+
+
+# ---------------------- UNIT TESTS ----------------------
+def test_add_camera_populates_working_settings(dialog, qtbot):
+ dialog._on_scan_result([DetectedCamera(index=2, label="ExtraCam2")])
+ dialog.available_cameras_list.setCurrentRow(0)
+
+ qtbot.mouseClick(dialog.add_camera_btn, Qt.LeftButton)
+
+ added = dialog._working_settings.cameras[-1]
+ assert added.index == 2
+ assert added.name == "ExtraCam2"
+
+
+def test_remove_camera(dialog, qtbot):
+ dialog.active_cameras_list.setCurrentRow(0)
+ qtbot.mouseClick(dialog.remove_camera_btn, Qt.LeftButton)
+
+ assert len(dialog._working_settings.cameras) == 1
+ assert dialog._working_settings.cameras[0].name == "CamB"
+
+
+def test_apply_settings_updates_model(dialog, qtbot):
+ dialog.active_cameras_list.setCurrentRow(0)
+
+ dialog.cam_fps.setValue(55.0)
+ dialog.cam_gain.setValue(12.0)
+
+ qtbot.mouseClick(dialog.apply_settings_btn, Qt.LeftButton)
+
+ updated = dialog._working_settings.cameras[0]
+ assert updated.fps == 55.0
+ assert updated.gain == 12.0
+
+
+def test_backend_control_disables_exposure_gain_for_opencv(dialog):
+ dialog._update_controls_for_backend("opencv")
+ assert not dialog.cam_exposure.isEnabled()
+ assert not dialog.cam_gain.isEnabled()
+
+ dialog._update_controls_for_backend("basler")
+ assert dialog.cam_exposure.isEnabled()
+ assert dialog.cam_gain.isEnabled()
diff --git a/tests/gui/conftest.py b/tests/gui/conftest.py
new file mode 100644
index 0000000..4807c3a
--- /dev/null
+++ b/tests/gui/conftest.py
@@ -0,0 +1,130 @@
+# tests/services/gui/conftest.py
+from __future__ import annotations
+
+import pytest
+from PySide6.QtCore import Qt
+
+from dlclivegui.cameras import CameraFactory
+from dlclivegui.gui.main_window import DLCLiveMainWindow
+
+# from dlclivegui.config import (
+# DEFAULT_CONFIG,
+# ApplicationSettings,
+# CameraSettings,
+# MultiCameraSettings,
+# )
+from dlclivegui.utils.config_models import (
+ DEFAULT_CONFIG,
+ ApplicationSettingsModel,
+ CameraSettingsModel,
+ MultiCameraSettingsModel,
+)
+from tests.conftest import FakeBackend, FakeDLCLive # noqa: F401
+
+# ---------- Test helpers: application configuration with two fake cameras ----------
+
+
+@pytest.fixture
+def app_config_two_cams(tmp_path) -> ApplicationSettingsModel:
+ """An app config with two enabled cameras (fake backend) and writable recording dir."""
+ cfg = ApplicationSettingsModel.from_dict(DEFAULT_CONFIG.to_dict())
+
+ cam_a = CameraSettingsModel(name="CamA", backend="fake", index=0, enabled=True, fps=30.0)
+ cam_b = CameraSettingsModel(name="CamB", backend="fake", index=1, enabled=True, fps=30.0)
+
+ cfg.multi_camera = MultiCameraSettingsModel(cameras=[cam_a, cam_b], max_cameras=4, tile_layout="auto")
+ cfg.camera = cam_a # kept for backward-compat single-camera access in UI
+
+ cfg.recording.directory = str(tmp_path / "videos")
+ cfg.recording.enabled = True
+ return cfg
+
+
+# ---------- Autouse patches to keep GUI tests fast and side-effect-free ----------
+
+
+@pytest.fixture(autouse=True)
+def _patch_camera_factory(monkeypatch):
+ """
+ Replace hardware backends with FakeBackend globally for GUI tests.
+ We patch at the central creation point used by the controller.
+ """
+
+ def _create_stub(settings: CameraSettingsModel):
+ # FakeBackend ignores 'backend' and produces deterministic frames
+ return FakeBackend(settings)
+
+ monkeypatch.setattr(CameraFactory, "create", staticmethod(_create_stub))
+
+
+@pytest.fixture(autouse=True)
+def _patch_camera_validation(monkeypatch):
+ """
+ Accept all cameras regardless of backend and silence warning/error dialogs in the window.
+ """
+ # 1) Pretend all cameras are available
+ monkeypatch.setattr(
+ CameraFactory,
+ "check_camera_available",
+ staticmethod(lambda cam: (True, "")),
+ )
+
+ # 2) Silence GUI dialogs during tests
+ monkeypatch.setattr(DLCLiveMainWindow, "_show_warning", lambda self, msg: None)
+ monkeypatch.setattr(DLCLiveMainWindow, "_show_error", lambda self, msg: None)
+
+
+@pytest.fixture(autouse=True)
+def _patch_dlclive_to_fake(monkeypatch):
+ """
+ Ensure dlclive is replaced by the test double in the DLCLiveProcessor module.
+ (The window will instantiate DLCLiveProcessor internally, which imports DLCLive.)
+ """
+ from dlclivegui.services import dlc_processor as dlcp_mod
+
+ monkeypatch.setattr(dlcp_mod, "DLCLive", FakeDLCLive)
+
+
+# ---------- The main window fixture (focus) ----------
+
+
+@pytest.fixture
+def window(qtbot, app_config_two_cams) -> DLCLiveMainWindow:
+ """
+ Construct the real DLCLiveMainWindow with a valid two-camera config,
+ make it headless, show it, and yield it. Threads and timers are managed by close().
+ """
+ w = DLCLiveMainWindow(config=app_config_two_cams)
+ qtbot.addWidget(w)
+ # Don't pop windows in CI:
+ w.setAttribute(Qt.WA_DontShowOnScreen, True)
+ w.show()
+
+ try:
+ yield w
+ finally:
+ # The window's closeEvent stops controllers, recorders, timers, etc.
+ # Use .close() to trigger the standard shutdown path.
+ try:
+ w.close()
+ except Exception:
+ pass
+
+
+# ---------- Convenience fixtures that expose controller/processor from the window ----------
+
+
+@pytest.fixture
+def multi_camera_controller(window):
+ """
+ Return the *controller used by the window* so tests can wait on all_started/all_stopped.
+ """
+ return window.multi_camera_controller
+
+
+@pytest.fixture
+def dlc_processor(window):
+ """
+ Return the *processor used by the window* so tests can connect to pose/initialized.
+ """
+ return window._dlc
diff --git a/tests/gui/test_main.py b/tests/gui/test_main.py
new file mode 100644
index 0000000..83d4bcc
--- /dev/null
+++ b/tests/gui/test_main.py
@@ -0,0 +1,99 @@
+import pytest
+from PySide6.QtCore import Qt
+from PySide6.QtGui import QImage
+
+
+def pixmap_bytes(label) -> bytes:
+ pm = label.pixmap()
+ assert pm is not None and not pm.isNull()
+ img = pm.toImage().convertToFormat(QImage.Format.Format_RGB888)
+ ptr = img.bits()
+ ptr.setsize(img.sizeInBytes())
+ return bytes(ptr)
+
+
+@pytest.mark.gui
+@pytest.mark.functional
+def test_preview_renders_frames(qtbot, window, multi_camera_controller):
+ """
+ Validate that:
+ - Preview starts (`preview_button` clicked)
+ - Camera controller emits all_started
+ - GUI receives and renders frames to video_label.pixmap()
+ - Preview stops cleanly
+ """
+
+ w = window
+ ctrl = multi_camera_controller
+
+ with qtbot.waitSignal(ctrl.all_started, timeout=4000):
+ qtbot.mouseClick(w.preview_button, Qt.LeftButton)
+
+ qtbot.waitUntil(
+ lambda: w.video_label.pixmap() is not None and not w.video_label.pixmap().isNull(),
+ timeout=6000,
+ )
+
+ with qtbot.waitSignal(ctrl.all_stopped, timeout=4000):
+ qtbot.mouseClick(w.stop_preview_button, Qt.LeftButton)
+
+ assert not ctrl.is_running()
+
+
+@pytest.mark.gui
+@pytest.mark.functional
+def test_start_inference_emits_pose(qtbot, window, multi_camera_controller, dlc_processor):
+ """
+ Validate that:
+ - Preview is running
+ - GUI sets a valid model path
+ - Start Inference triggers DLCLiveProcessor initialization
+ - initialized(True) fires
+ - pose_ready fires at least once
+ - Preview can be stopped cleanly
+ """
+
+ w = window
+ ctrl = multi_camera_controller
+ dlc = dlc_processor
+
+ # Start preview first
+ with qtbot.waitSignal(ctrl.all_started, timeout=4000):
+ qtbot.mouseClick(w.preview_button, Qt.LeftButton)
+
+ # Ensure preview is producing actual GUI frames
+ qtbot.waitUntil(
+ lambda: w.video_label.pixmap() is not None and not w.video_label.pixmap().isNull(),
+ timeout=6000,
+ )
+
+ w.model_path_edit.setText("dummy_model.pt")
+ pose_count = [0]
+
+ def _on_pose(result):
+ pose_count[0] += 1
+
+ dlc.pose_ready.connect(_on_pose)
+
+ try:
+ # Click "Start Inference" and wait for DLCLiveProcessor.initialized(True)
+ with qtbot.waitSignal(dlc.initialized, timeout=7000) as init_blocker:
+ qtbot.mouseClick(w.start_inference_button, Qt.LeftButton)
+
+ # Validate initialized==True
+ assert init_blocker.args[0] is True
+
+ # Wait until at least one pose is emitted
+ qtbot.waitUntil(lambda: pose_count[0] >= 1, timeout=7000)
+
+ finally:
+ # Avoid leaking connections across tests
+ try:
+ dlc.pose_ready.disconnect(_on_pose)
+ except Exception:
+ pass
+
+ with qtbot.waitSignal(ctrl.all_stopped, timeout=4000):
+ qtbot.mouseClick(w.stop_preview_button, Qt.LeftButton)
+
+ assert not ctrl.is_running()
diff --git a/tests/services/test_dlc_processor.py b/tests/services/test_dlc_processor.py
new file mode 100644
index 0000000..210e820
--- /dev/null
+++ b/tests/services/test_dlc_processor.py
@@ -0,0 +1,261 @@
+import numpy as np
+import pytest
+
+from dlclivegui.services.dlc_processor import (
+ DLCLiveProcessor,
+ ProcessorStats,
+)
+
+# from dlclivegui.config import DLCProcessorSettings
+from dlclivegui.utils.config_models import DLCProcessorSettingsModel
+
+# ---------------------------------------------------------------------
+# Tests
+# ---------------------------------------------------------------------
+
+
+@pytest.mark.unit
+def test_configure_accepts_pydantic(settings_model, monkeypatch_dlclive):
+ proc = DLCLiveProcessor()
+ proc.configure(settings_model)
+
+ assert isinstance(proc._settings, DLCProcessorSettingsModel)
+ assert proc._settings.model_path == "dummy.pt"
+
+
+@pytest.mark.unit
+def test_worker_initializes_on_first_frame(qtbot, monkeypatch_dlclive, settings_model):
+ proc = DLCLiveProcessor()
+ proc.configure(settings_model)
+
+ try:
+ # First enqueued frame triggers worker start + initialization.
+ with qtbot.waitSignal(proc.initialized, timeout=1500) as init_blocker:
+ proc.enqueue_frame(np.zeros((100, 100, 3), dtype=np.uint8), timestamp=1.0)
+
+ assert init_blocker.args == [True]
+ assert proc._initialized
+ assert getattr(proc._dlc, "init_called", False)
+
+ # Optional: also ensure the init pose was delivered
+ qtbot.waitUntil(lambda: proc.get_stats().frames_processed >= 1, timeout=1500)
+
+ finally:
+ proc.reset() # Ensure thread cleanup
+
+
+@pytest.mark.unit
+def test_worker_processes_frames(qtbot, monkeypatch_dlclive, settings_model):
+ proc = DLCLiveProcessor()
+ proc.configure(settings_model)
+
+ try:
+ frame = np.zeros((64, 64, 3), dtype=np.uint8)
+
+ # The first frame should initialize DLCLive (initialized -> True) and produce the first pose.
+ with qtbot.waitSignal(proc.initialized, timeout=1500):
+ proc.enqueue_frame(frame, timestamp=1.0)
+
+ # Wait for init pose
+ qtbot.waitUntil(lambda: proc.get_stats().frames_processed >= 1, timeout=1500)
+
+ # Enqueue more frames; wait for at least one more pose
+ for i in range(10):
+ proc.enqueue_frame(frame, timestamp=2.0 + i)
+
+ qtbot.waitUntil(lambda: proc.get_stats().frames_processed >= 3, timeout=1500)
+
+ finally:
+ proc.reset()
+
+
+@pytest.mark.unit
+def test_queue_full_drops_frames(qtbot, monkeypatch_dlclive, settings_model):
+ proc = DLCLiveProcessor()
+ proc.configure(settings_model)
+
+ try:
+ frame = np.zeros((32, 32, 3), dtype=np.uint8)
+
+ # Start the worker with the first frame
+ with qtbot.waitSignal(proc.initialized, timeout=1500):
+ proc.enqueue_frame(frame, 1.0)
+
+ # Flood the 1-slot queue to force drops
+ for _ in range(50):
+ proc.enqueue_frame(frame, 2.0)
+
+ # Wait until we observe dropped frames
+ qtbot.waitUntil(lambda: proc._frames_dropped > 0, timeout=1500)
+ assert proc._frames_dropped > 0
+
+ finally:
+ proc.reset()
+
+
+@pytest.mark.unit
+def test_error_signal_on_initialization_failure(qtbot, monkeypatch):
+ """Simulate DLCLive raising on init."""
+
+ class FailingDLCLive:
+ def __init__(self, **opts):
+ raise RuntimeError("bad model")
+
+ from dlclivegui.services import dlc_processor
+
+ monkeypatch.setattr(dlc_processor, "DLCLive", FailingDLCLive)
+
+ proc = DLCLiveProcessor()
+ proc.configure(DLCProcessorSettingsModel(model_path="fail.pt"))
+
+ try:
+ frame = np.zeros((10, 10, 3), dtype=np.uint8)
+
+ error_args = []
+ init_args = []
+
+ proc.error.connect(lambda msg: error_args.append(msg))
+ proc.initialized.connect(lambda ok: init_args.append(ok))
+
+ with qtbot.waitSignals([proc.error, proc.initialized], timeout=1500):
+ proc.enqueue_frame(frame, 1.0)
+
+ assert len(error_args) == 1
+ assert "bad model" in error_args[0]
+
+ assert len(init_args) == 1
+ assert init_args[0] is False
+
+ finally:
+ proc.reset()
+
+
+@pytest.mark.unit
+def test_stats_computation(qtbot, monkeypatch_dlclive, settings_model):
+ proc = DLCLiveProcessor()
+ proc.configure(settings_model)
+
+ try:
+ frame = np.zeros((64, 64, 3), dtype=np.uint8)
+
+ # Start and wait for init
+ with qtbot.waitSignal(proc.initialized, timeout=1500):
+ proc.enqueue_frame(frame, 1.0)
+
+ # Wait for init pose
+ qtbot.waitUntil(lambda: proc.get_stats().frames_processed >= 1, timeout=1500)
+
+ # Enqueue a second frame and wait for its pose
+ proc.enqueue_frame(frame, 2.0)
+ qtbot.waitUntil(lambda: proc.get_stats().frames_processed >= 2, timeout=1500)
+
+ stats = proc.get_stats()
+ assert isinstance(stats, ProcessorStats)
+ assert stats.frames_processed >= 1
+ assert stats.processing_fps >= 0
+
+ finally:
+ proc.reset()
+
+
+@pytest.mark.unit
+def test_worker_processes_second_frame_and_updates_stats(qtbot, monkeypatch_dlclive, settings_model):
+ """
+ Explicitly verify that after initialization, a queued frame is processed:
+ - frame_processed is emitted for the second frame
+ - frames_processed >= 2 (init + 1 queued)
+ """
+ proc = DLCLiveProcessor()
+ proc.configure(settings_model)
+
+ try:
+ frame = np.zeros((64, 64, 3), dtype=np.uint8)
+
+ # First frame triggers initialization + init pose
+ with qtbot.waitSignal(proc.initialized, timeout=1500):
+ proc.enqueue_frame(frame, 1.0)
+ qtbot.waitUntil(lambda: proc.get_stats().frames_processed >= 1, timeout=1500) # init pose
+
+ # Enqueue one more frame and wait for its pose
+ proc.enqueue_frame(frame, 2.0)
+ qtbot.waitUntil(lambda: proc.get_stats().frames_processed >= 2, timeout=1500)
+ stats = proc.get_stats()
+ # >= 2: init + the second frame
+ assert stats.frames_processed >= 2
+ # queue drained
+ assert stats.queue_size == 0
+
+ finally:
+ proc.reset()
+
+
+@pytest.mark.unit
+def test_worker_survives_empty_timeouts_then_processes_next(qtbot, monkeypatch_dlclive, settings_model):
+ """
+ Verify the worker doesn't exit after queue.Empty timeouts and still processes
+ a subsequent enqueued frame (this asserts the loop continues running).
+ """
+ proc = DLCLiveProcessor()
+ proc.configure(settings_model)
+
+ try:
+ frame = np.zeros((64, 64, 3), dtype=np.uint8)
+
+ # Initialize with first frame
+ with qtbot.waitSignal(proc.initialized, timeout=1500):
+ proc.enqueue_frame(frame, 1.0)
+ qtbot.waitUntil(lambda: proc.get_stats().frames_processed >= 1, timeout=1500) # init pose
+
+ # Let the worker spin with an empty queue (several 0.1s timeouts)
+ qtbot.wait(350) # ~3-4 timeouts
+
+ # The worker thread should still be alive
+ assert proc._worker_thread is not None and proc._worker_thread.is_alive()
+
+ # Enqueue another frame and ensure it is processed
+ proc.enqueue_frame(frame, 2.0)
+ qtbot.waitUntil(lambda: proc.get_stats().frames_processed >= 2, timeout=1500)
+
+ stats = proc.get_stats()
+ assert stats.frames_processed >= 2
+
+ finally:
+ proc.reset()
+
+
+@pytest.mark.unit
+def test_queue_accounting_clears_after_processed_frame(qtbot, monkeypatch_dlclive, settings_model):
+ """
+ After a queued frame is processed:
+ - queue size returns to zero
+ - unfinished task count (if accessible) is zero
+
+ This implicitly validates correct task_done() usage for processed items.
+ Note: the init frame is not queued, so we only check queued work accounting.
+ """
+ proc = DLCLiveProcessor()
+ proc.configure(settings_model)
+
+ try:
+ frame = np.zeros((32, 32, 3), dtype=np.uint8)
+
+ # Initialize (no queue involvement for the init frame)
+ with qtbot.waitSignal(proc.initialized, timeout=1500):
+ proc.enqueue_frame(frame, 1.0)
+ qtbot.waitUntil(lambda: proc.get_stats().frames_processed >= 1, timeout=1500) # init pose
+
+ # Enqueue one queued frame
+ proc.enqueue_frame(frame, 2.0)
+ qtbot.waitUntil(lambda: proc.get_stats().frames_processed >= 2, timeout=1500)
+
+ # Queue should be drained
+ q = proc._queue
+ # It's allowed to be None if the worker shut down, but in normal run it should exist
+ if q is not None:
+ assert q.qsize() == 0
+ # CPython exposes 'unfinished_tasks'; if present, it should be zero
+ unfinished = getattr(q, "unfinished_tasks", 0)
+ assert unfinished == 0
+
+ finally:
+ proc.reset()
diff --git a/tests/services/test_multicam_controller.py b/tests/services/test_multicam_controller.py
new file mode 100644
index 0000000..5c26a86
--- /dev/null
+++ b/tests/services/test_multicam_controller.py
@@ -0,0 +1,97 @@
+# tests/services/test_multicam_controller.py
+import pytest
+
+from dlclivegui.cameras.factory import CameraFactory
+from dlclivegui.services.multi_camera_controller import MultiCameraController, get_camera_id
+
+# from dlclivegui.config import CameraSettings
+from dlclivegui.utils.config_models import CameraSettingsModel
+
+
+@pytest.mark.unit
+def test_start_and_frames(qtbot, patch_factory):
+ mc = MultiCameraController()
+
+ # One dataclass + one dict (simulate mixed inputs)
+ cam1 = CameraSettingsModel(name="C1", backend="opencv", index=0, fps=25.0).apply_defaults()
+ cam2 = {"name": "C2", "backend": "opencv", "index": 1, "fps": 30.0, "enabled": True}
+ cam2 = CameraSettingsModel.from_dict(cam2).apply_defaults()
+
+ frames_seen = []
+
+ def on_ready(mfd):
+ frames_seen.append((mfd.source_camera_id, {k: v.shape for k, v in mfd.frames.items()}))
+
+ mc.frame_ready.connect(on_ready)
+
+ try:
+ with qtbot.waitSignal(mc.all_started, timeout=1500):
+ mc.start([cam1, cam2])
+
+ # Wait for at least one composite emission
+ qtbot.waitUntil(lambda: len(frames_seen) >= 1, timeout=2000)
+
+ assert mc.is_running()
+ # We should have at least one entry with 1 or 2 frames (depending on timing)
+ assert any(len(shape_map) >= 1 for _, shape_map in frames_seen)
+
+ finally:
+ with qtbot.waitSignal(mc.all_stopped, timeout=2000):
+ mc.stop(wait=True)
+
+
+@pytest.mark.unit
+def test_rotation_and_crop(qtbot, patch_factory):
+ mc = MultiCameraController()
+
+ # 64x48 frame; rotate 90 => 48x64 then crop to 32x32 box
+ cam = CameraSettingsModel(
+ name="C",
+ backend="opencv",
+ index=0,
+ enabled=True,
+ rotation=90,
+ crop_x0=0,
+ crop_y0=0,
+ crop_x1=32,
+ crop_y1=32,
+ ).apply_defaults()
+
+ last_shape = {"shape": None}
+
+ def on_ready(mfd):
+ f = mfd.frames.get(get_camera_id(cam))
+ if f is not None:
+ last_shape["shape"] = f.shape
+
+ mc.frame_ready.connect(on_ready)
+
+ try:
+ with qtbot.waitSignal(mc.all_started, timeout=1500):
+ mc.start([cam])
+
+ # Wait until a rotated+cropped frame arrives
+ qtbot.waitUntil(lambda: last_shape["shape"] is not None, timeout=2000)
+
+ # Expect height=32, width=32, 3 channels
+ assert last_shape["shape"] == (32, 32, 3)
+
+ finally:
+ with qtbot.waitSignal(mc.all_stopped, timeout=2000):
+ mc.stop(wait=True)
+
+
+@pytest.mark.unit
+def test_initialization_failure(qtbot, monkeypatch):
+ # Make factory.create raise
+ def _create(_settings):
+ raise RuntimeError("no device")
+
+ monkeypatch.setattr(CameraFactory, "create", staticmethod(_create))
+
+ mc = MultiCameraController()
+ cam = CameraSettingsModel(name="C", backend="opencv", index=0, enabled=True).apply_defaults()
+
+ # Expect initialization_failed with the camera id
+ with qtbot.waitSignals([mc.initialization_failed, mc.all_stopped], timeout=2000) as _:
+ mc.start([cam])