Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
21 changes: 10 additions & 11 deletions claude-code/smartem-decisions/REPO-GUIDELINES.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,8 @@

## Development Environment
- **Python Version**: 3.12+ (strict requirement - utilize modern typing features)
- **venv**: Use `source .venv/bin/activate` for tools such as pip, ruff or tox
- **Package Management**: Use `pip install -e .[all]` for full development setup
- **Package Manager**: uv (https://docs.astral.sh/uv/)
- **Package Management**: Use `uv sync --extra all` for full development setup
- **Code Style**: Ruff (120 char line length) with pyright type checking

## Code Standards
Expand Down Expand Up @@ -41,17 +41,17 @@ This applies to ALL files: Python code, Markdown documentation, configuration fi
## Common Commands
```bash
# Development setup
pip install -e .[all]
uv sync --extra all

# Testing
pytest
uv run pytest

# Type checking
pyright src tests
# Type checking
uv run pyright src tests

# Linting and formatting
ruff check
ruff format
uv run ruff check
uv run ruff format

# Pre-commit checks
pre-commit run --all-files
Expand All @@ -60,8 +60,8 @@ pre-commit run --all-files
cd ../smartem-devtools/webui && npm run dev

# Database migrations
python -m alembic upgrade head
python -m alembic revision --autogenerate -m "Description"
uv run alembic upgrade head
uv run alembic revision --autogenerate -m "Description"
```

## Project Architecture
Expand All @@ -88,7 +88,6 @@ python -m alembic revision --autogenerate -m "Description"
## Testing & Quality
- **Coverage**: Tests run with coverage reporting
- **Doctests**: Documentation examples are executable
- **Multiple environments**: tox for cross-environment testing
- **CI/CD**: GitHub Actions with pre-commit hooks

## Documentation
Expand Down
38 changes: 19 additions & 19 deletions docs/development/tools.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,16 +10,16 @@ Transform single-line XML and .dm files into human-readable format with proper i

```bash
# Reformat all .xml and .dm files in a directory recursively
python tools/format_xml.py <directory_path> -r
uv run python tools/format_xml.py <directory_path> -r

# Process multiple directories
python tools/format_xml.py -r \
uv run python tools/format_xml.py -r \
../smartem-decisions-test-datasets/metadata_Supervisor_20250114_220855_23_epuBSAd20_GrOxDDM \
../smartem-decisions-test-datasets/metadata_Supervisor_20241220_140307_72_et2_gangshun \
../smartem-decisions-test-datasets/metadata_Supervisor_20250108_101446_62_cm40593-1_EPU

# Display all available options
python tools/format_xml.py --help
uv run python tools/format_xml.py --help
```

## Data Analysis and Debugging Tools
Expand All @@ -30,10 +30,10 @@ Identify duplicate foil hole manifests within directory structures to detect dat

```bash
# Display help and usage information
tools/find_foilhole_duplicates.py --help
uv run tools/find_foilhole_duplicates.py --help

# Example: Search for duplicates in test data
tools/find_foilhole_duplicates.py ./tests/testdata/bi37708-28
uv run tools/find_foilhole_duplicates.py ./tests/testdata/bi37708-28
```

### File Size Analysis
Expand Down Expand Up @@ -94,21 +94,21 @@ Comprehensive CLI tool for simulating external data processing messages that wou

```bash
# List all available message types
python tools/external_message_simulator.py list-messages
uv run python tools/external_message_simulator.py list-messages

# Individual message simulation examples
python tools/external_message_simulator.py motion-correction --foilhole-id "FH_001_001_001" --quality-score 0.85
python tools/external_message_simulator.py ctf-complete --foilhole-id "FH_001_001_001" --resolution 3.2
python tools/external_message_simulator.py gridsquare-prediction --gridsquare-id "GS_001_001" --prediction-score 0.85
python tools/external_message_simulator.py model-update --model-name "ResNet-50"
uv run python tools/external_message_simulator.py motion-correction --foilhole-id "FH_001_001_001" --quality-score 0.85
uv run python tools/external_message_simulator.py ctf-complete --foilhole-id "FH_001_001_001" --resolution 3.2
uv run python tools/external_message_simulator.py gridsquare-prediction --gridsquare-id "GS_001_001" --prediction-score 0.85
uv run python tools/external_message_simulator.py model-update --model-name "ResNet-50"

# Complete workflow simulation for a single grid square
python tools/external_message_simulator.py workflow-simulation --gridsquare-id "DEV_001"
uv run python tools/external_message_simulator.py workflow-simulation --gridsquare-id "DEV_001"

# Batch simulation with different quality scenarios
python tools/external_message_simulator.py batch-simulation --gridsquare-count 5 --scenario mixed
python tools/external_message_simulator.py batch-simulation --gridsquare-count 3 --scenario good
python tools/external_message_simulator.py batch-simulation --gridsquare-count 2 --scenario poor
uv run python tools/external_message_simulator.py batch-simulation --gridsquare-count 5 --scenario mixed
uv run python tools/external_message_simulator.py batch-simulation --gridsquare-count 3 --scenario good
uv run python tools/external_message_simulator.py batch-simulation --gridsquare-count 2 --scenario poor
```

**Available Message Types:**
Expand All @@ -126,7 +126,7 @@ Example client for testing agent-backend communication via Server-Sent Events:

```bash
# Start the SSE client to receive instructions from backend
python tools/sse_client_example.py
uv run python tools/sse_client_example.py

# The client will:
# 1. Auto-create a new session with the backend
Expand Down Expand Up @@ -161,12 +161,12 @@ pre-commit run --all-files

```bash
# Run comprehensive test suite
pytest
uv run pytest

# Type checking with pyright
pyright src tests
uv run pyright src tests

# Code formatting and linting
ruff check
ruff format
uv run ruff check
uv run ruff format
```
70 changes: 46 additions & 24 deletions docs/getting-started/for-developers.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,53 +10,75 @@ python3 --version

**Note:** The project specifically requires Python 3.12+ to utilise advanced typing features and maintain compatibility with the latest scientific computing libraries.

## Create a Virtual Environment
## Install uv (Recommended)

It is strongly recommended that you install SmartEM Decisions within a virtual environment to prevent conflicts with existing Python installations and maintain a clean development environment:
We recommend using [uv](https://docs.astral.sh/uv/) for Python package management. uv is a fast Python package manager that handles virtual environments and dependencies efficiently.

### Install uv

```bash
python3 -m venv /path/to/venv
source /path/to/venv/bin/activate
# On macOS and Linux
curl -LsSf https://astral.sh/uv/install.sh | sh

# Or with pip
pip install uv

# Or with Homebrew (macOS)
brew install uv
```

For development work, consider creating the virtual environment within the project directory:
## Installing the Library

### Development Installation (Recommended)

For development work, clone the repository and install using uv:

```bash
python3 -m venv .venv
source .venv/bin/activate
# Clone the repository
git clone https://github.com/DiamondLightSource/smartem-decisions.git
cd smartem-decisions

# Install all dependencies (creates virtual environment automatically)
uv sync --extra all
```

## Installing the Library
This will create a `.venv` directory and install all dependencies including development tools.

### Standard Installation
### Running Commands

You can install the library and its core dependencies using `pip`:
With uv, prefix commands with `uv run` to execute them in the virtual environment:

```bash
python3 -m pip install smartem-decisions
# Run tests
uv run pytest

# Type checking
uv run pyright src tests

# Linting
uv run ruff check
```

### Development Installation
### Standard Installation (pip)

For development work or if you require the latest features, install directly from the source repository:
You can also install the library using pip:

```bash
# Clone and install in development mode
git clone https://github.com/DiamondLightSource/smartem-decisions.git
cd smartem-decisions
pip install -e .[all]
python3 -m pip install smartem-decisions
```

### Installation with Optional Dependencies

Install with specific feature sets:

```bash
# Install with MCP (Model Context Protocol) support
pip install -e .[mcp]
# Using uv (recommended)
uv sync --extra mcp # Install with MCP support
uv sync --extra all # Install with all dependencies

# Install with all optional dependencies (recommended for development)
pip install -e .[all]
# Using pip
pip install -e .[mcp] # Install with MCP support
pip install -e .[all] # Install with all dependencies
```

## Verify Installation
Expand All @@ -65,17 +87,17 @@ Verify that all components are correctly installed by running:

```bash
# Test core functionality
python -c "import smartem_backend, smartem_agent, smartem_common; print('Core components imported successfully')"
uv run python -c "import smartem_backend, smartem_agent, smartem_common; print('Core components imported successfully')"

# Test MCP functionality (if installed with MCP support)
python -c "import smartem_mcp; print('MCP components available')"
uv run python -c "import smartem_mcp; print('MCP components available')"
```

You can also verify the CLI tools are available:

```bash
# Check agent CLI is accessible
python -m smartem_agent --help
uv run python -m smartem_agent --help
```

## Next Steps
Expand Down