Phase 1: Fix PyRIT 0.10.0 breaking changes and add FoundryScenario strategy mapping

[Copilot]

PyRIT 0.10.0 removed DuckDB support in favor of SQLite-only memory backend, breaking initialization. This PR implements Phase 1 of FoundryScenario integration: fixing the breaking change and establishing the strategy mapping infrastructure.

Changes

Fixed SQLite Initialization

• Updated import: DUCK_DB → SQLITE in _red_team.py
• Added memory_db_path parameter to initialize_pyrit() using output_dir/pyrit_memory.db
• Added debug logging for database location

Before:

from pyrit.common import initialize_pyrit, DUCK_DB
initialize_pyrit(memory_db_type=DUCK_DB)

After:

from pyrit.common import initialize_pyrit, SQLITE
db_path = os.path.join(self.output_dir, "pyrit_memory.db")
initialize_pyrit(memory_db_type=SQLITE, memory_db_path=db_path)
self.logger.debug(f"Initialized PyRIT with SQLite at {db_path}")

Created Strategy Mapping Layer

New _utils/strategy_mapping.py module maps Azure SDK AttackStrategy enum to PyRIT FoundryStrategy:

• convert_attack_strategy_to_foundry(): Converts strategy with validation
• is_foundry_supported(): Checks FoundryScenario compatibility
• Supports: Baseline (None), ROT13, Base64, MultiTurn, Crescendo, Jailbreak
• Excludes: IndirectJailbreak (not yet supported in PyRIT FoundryScenario)
• Import fallback handling for PyRIT version compatibility

Added Progress Tracking

Created progress.md documenting:

• Breaking changes resolved
• Component implementation status
• Design decisions and known limitations
• Future phase roadmap

Added Unit Tests

New test_strategy_mapping.py with coverage for:

• Strategy conversion correctness
• Baseline special case (returns None)
• Unsupported strategy error handling
• is_foundry_supported() validation

Impact

• Modified: 1 file (8 lines changed)
• Added: 3 files (strategy mapping, progress tracking, tests)
• Total: 334 insertions(+), 3 deletions(-)

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• scanning-api.github.com
  • Triggering command: /home/REDACTED/work/_temp/ghcca-node/node/bin/node /home/REDACTED/work/_temp/ghcca-node/node/bin/node --enable-source-maps /home/REDACTED/work/_temp/copilot-developer-action-main/dist/index.js (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

Original prompt

Phase 1: Implement PyRIT FoundryScenario Integration

Implement the FoundryScenario integration as specified in spec.md, targeting PyRIT 0.10.0+.

Changes Required

1. Fix Breaking Change: SQLite Initialization

File: sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/red_team/_red_team.py

Current (line ~234):

# Initialize PyRIT
initialize_pyrit(memory_db_type=DUCK_DB)

Updated:

# Initialize PyRIT with SQLite (only option in 0.10.0+)
db_path = os.path.join(self.output_dir, "pyrit_memory.db")
initialize_pyrit(memory_db_type=SQLITE, memory_db_path=db_path)
self.logger.debug(f"Initialized PyRIT with SQLite at {db_path}")

Import changes:

from pyrit.common import initialize_pyrit, SQLITE  # Changed from DUCK_DB

2. Create Strategy Mapping Layer

New File: sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/red_team/_utils/strategy_mapping.py

# ---------------------------------------------------------
# Copyright (c) Microsoft Corporation. All rights reserved.
# ---------------------------------------------------------
"""Strategy mapping utilities for PyRIT FoundryScenario integration."""

from typing import Dict
from pyrit.scenario.scenarios.foundry_scenario import FoundryStrategy
from .._attack_strategy import AttackStrategy


# Mapping table: azure-sdk AttackStrategy → PyRIT FoundryStrategy
ATTACK_STRATEGY_TO_FOUNDRY_STRATEGY: Dict[AttackStrategy, FoundryStrategy] = {
    # Baseline (no modifications)
    AttackStrategy.Baseline: None,  # Special case - handled separately
    
    # Single-turn converter strategies (easy)
    AttackStrategy.ROT13: FoundryStrategy.ROT13,
    AttackStrategy.Base64: FoundryStrategy.Base64,
    
    # Multi-turn attack strategies (difficult)
    AttackStrategy.MultiTurn: FoundryStrategy.MultiTurn,
    AttackStrategy.Crescendo: FoundryStrategy.Crescendo,
    AttackStrategy.PAIR: FoundryStrategy.Pair,
    
    # Jailbreak strategy
    AttackStrategy.Jailbreak: FoundryStrategy.Jailbreak,
}


def convert_attack_strategy_to_foundry(strategy: AttackStrategy) -> FoundryStrategy:
    """
    Convert azure-sdk-for-python AttackStrategy to PyRIT FoundryStrategy.
    
    Args:
        strategy: The attack strategy to convert
        
    Returns:
        Corresponding FoundryStrategy, or None for baseline
        
    Raises:
        ValueError: If strategy is not supported in FoundryScenario
    """
    if strategy == AttackStrategy.Baseline:
        return None  # Baseline handled separately
        
    if strategy not in ATTACK_STRATEGY_TO_FOUNDRY_STRATEGY:
        raise ValueError(
            f"Attack strategy {strategy} is not supported with FoundryScenario. "
            f"Supported strategies: {list(ATTACK_STRATEGY_TO_FOUNDRY_STRATEGY.keys())}"
        )
    
    return ATTACK_STRATEGY_TO_FOUNDRY_STRATEGY[strategy]


def is_foundry_supported(strategy: AttackStrategy) -> bool:
    """
    Check if an attack strategy is supported by FoundryScenario.
    
    Args:
        strategy: The attack strategy to check
        
    Returns:
        True if supported, False otherwise
    """
    return strategy in ATTACK_STRATEGY_TO_FOUNDRY_STRATEGY

3. Update Progress Tracking

File: sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/red_team/progress.md

Update with implementation progress:

# PyRIT FoundryScenario Integration - Progress Tracker

**Last Updated:** 2025-12-18  
**Current Phase:** Phase 1 - Implementation In Progress  
**Next Milestone:** Complete Core Components

## Completed Steps

### ✅ Phase 0: Planning (Completed 2025-12-17)
- [x] Conducted comprehensive FoundryScenario inventory
- [x] Created detailed technical specification (spec.md v3.0)
- [x] Identified breaking changes in PyRIT 0.10.0
- [x] Designed strategy mapping layer
- [x] Planned 4-phase migration strategy

### 🔄 Phase 1: Implementation (In Progress - Started 2025-12-18)

**Completed:**
- [x] Fixed breaking change: `DUCK_DB` → `SQLITE` in `_red_team.py` (line 234)
- [x] Created `_utils/strategy_mapping.py` with FoundryStrategy mapping
- [x] Updated imports to use `SQLITE` instead of `DUCK_DB`

**In Progress:**
- [ ] Create `_scenario_manager.py` for FoundryScenario lifecycle management
- [ ] Create `_result_converter.py` for MessagePiece → red_team_info conversion
- [ ] Add feature flag `USE_FOUNDRY_SCENARIOS` to `_red_team.py`
- [ ] Implement dual-path logic in `scan()` method

**Blocked:**
- IndirectJailbreak (XPIA) strategy - not yet supported in PyRIT FoundryScenario
  - Workaround: Keep custom XPIA logic outside FoundryScenario temporarily

## Current Implementation Status

### Breaking Changes Fixed

| Change | Status | Notes |
|--------|--------|-------|
| `DUCK_DB` → `SQLITE` | ✅ Complete | Line 234 in `_red_team.py` |
| Import path updated | ✅ Complete | `from pyrit.common import SQLITE` |
| Database path configuration | ✅ Complete | Uses `output_dir/pyrit_memory.db` |

#...

</details>



<!-- START COPILOT CODING AGENT SUFFIX -->

*This pull request was created from Copilot chat.*
>

<!-- START COPILOT CODING AGENT TIPS -->
---

💬 We'd love your input! Share your thoughts on Copilot coding agent in our [2 minute survey](https://gh.io/copilot-coding-agent-survey).