Add PyRIT 0.10.0 technical specification and progress tracking for red team module

[Copilot]

Documents the migration path for PyRIT 0.10.0 breaking changes in the red team module, including DuckDB removal, API renames, and SQLite-only backend.

Changes

• spec.md: Technical specification v3.0 covering:

  • Breaking changes: PromptRequestPiece → MessagePiece, get_prompt_request_pieces() → get_message_pieces(), DuckDB → SQLite
  • Core components: strategy mapping, scenario manager, result converter
  • SQLite-only memory backend implementation
  • Context preservation via memory labels
  • 4-phase migration strategy with exact line references (line 222: initialize_pyrit(memory_db_type=DUCK_DB) → SQLITE)

• progress.md: Aligned progress tracking with:

  • Breaking change blocker documenting line 222 fix requirement
  • Design decisions table reflecting PyRIT 0.10.0 constraints
  • Phase breakdowns, risk register, metrics

Example Breaking Change

Current code at _red_team.py:222:

initialize_pyrit(memory_db_type=DUCK_DB)  # ❌ Removed in 0.10.0

Required fix:

db_path = os.path.join(self.output_dir, "pyrit_memory.db")
initialize_pyrit(memory_db_type=SQLITE, memory_db_path=db_path)  # ✅ Only option

All code examples use PyRIT 0.10.0 APIs consistently.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

• scanning-api.github.com
  • Triggering command: /home/REDACTED/work/_temp/ghcca-node/node/bin/node /home/REDACTED/work/_temp/ghcca-node/node/bin/node --enable-source-maps /home/REDACTED/work/_temp/copilot-developer-action-main/dist/index.js (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

• Configure Actions setup steps to set up my environment, which run before the firewall is enabled
• Add the appropriate URLs or hosts to the custom allowlist in this repository's Copilot coding agent settings (admins only)

Original prompt

Update Red Team Module Documentation: spec.md and progress.md

Update the red team module's technical specification and progress tracking documents to reflect PyRIT 0.10.0 changes and align with the comprehensive v3.0 specification.

Files to Update

1. sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/red_team/spec.md
2. sdk/evaluation/azure-ai-evaluation/azure/ai/evaluation/red_team/progress.md

Specification Updates (spec.md)

Replace the entire contents of spec.md with the comprehensive v3.0 specification that includes:

1. Version Header and Breaking Changes Alert

# PyRIT FoundryScenario Integration - Technical Specification v3.0

**Last Updated:** 2025-12-18  
**Status:** Planning Complete  
**Owner:** Azure AI Evaluation Team  
**Target PyRIT Version:** 0.10.0

> **Breaking Changes in PyRIT 0.10.0:**
> - DuckDB support removed (SQLite only)
> - `PromptRequestPiece` renamed to `MessagePiece`
> - `get_prompt_request_pieces()` renamed to `get_message_pieces()`

2. Executive Summary

Update high-level approach to include:

• Message-based data model using MessagePiece
• SQLite memory (only option in PyRIT 0.10.0+)

3. Core Components

Strategy Mapping Layer (_utils/strategy_mapping.py):

from pyrit.scenario.scenarios.foundry_scenario import FoundryStrategy

ATTACK_STRATEGY_TO_FOUNDRY_STRATEGY: Dict[AttackStrategy, FoundryStrategy] = {
    AttackStrategy.Direct: FoundryStrategy.Jailbreak,
    AttackStrategy.PAIR: FoundryStrategy.Pair,
    AttackStrategy.ROT13: FoundryStrategy.ROT13,
    AttackStrategy.Base64: FoundryStrategy.Base64,
}

Scenario Manager (_scenario_manager.py):

• Initialize PyRIT with SQLite (line ~234 breaking change fix)
• Use RAI service simulation endpoint for adversarial chat
• Create FoundryScenario instances per risk category

Result Converter (_result_converter.py):

• Use get_message_pieces() instead of get_prompt_request_pieces()
• Access MessagePiece properties (not PromptRequestPiece)

4. Concise SQLite Section

Replace long comparison with:

## PyRIT Memory: SQLite (v0.10.0+)

**PyRIT 0.10.0 removed DuckDB support.** SQLite is now the only supported memory backend.

### Implementation

```python
from pyrit.common import initialize_pyrit, SQLITE

# In ScenarioManager.__init__()
db_path = os.path.join(self.output_dir, "pyrit_memory.db")
initialize_pyrit(memory_db_type=SQLITE, memory_db_path=db_path)

Memory Retrieval

from pyrit.memory import CentralMemory

memory = CentralMemory.get_memory_instance()
message_pieces = memory.get_message_pieces(
    labels={"risk_category": risk_category.value}
)

#### 5. Context Preservation with Memory Labels

Update to use `MessagePiece`:

```python
# Attach labels when creating scenario
scenario._memory_labels = {
    "risk_category": risk_category.value,
    "scan_session_id": scan_session_id,
    "objective": objective,
    "context": context_data,
    "risk_subtype": risk_subtype,
}

# Retrieve during result processing
memory = CentralMemory.get_memory_instance()
message_pieces = memory.get_message_pieces(labels={"risk_category": "violence"})

for piece in message_pieces:
    context = piece.labels.get("context", {})
    risk_subtype = piece.labels.get("risk_subtype", "")

6. Migration Strategy with Breaking Change Alert

Add to Phase 1:

**⚠️ Breaking Change Alert:** Current `_red_team.py` (line ~234) uses:
```python
initialize_pyrit(memory_db_type=DUCK_DB)  # ❌ Removed in PyRIT 0.10.0

Must update to:

initialize_pyrit(memory_db_type=SQLITE, memory_db_path=db_path)  # ✅ Only option

### Progress Updates (progress.md)

Update `progress.md` to match the specification:

#### 1. Add Breaking Change to Blockers

```markdown
4. **Breaking Change in PyRIT 0.10.0:**
   - **Issue:** Current code uses `initialize_pyrit(memory_db_type=DUCK_DB)` which no longer exists
   - **Location:** `_red_team.py` line ~234
   - **Fix Required:** Change to `initialize_pyrit(memory_db_type=SQLITE, memory_db_path=db_path)`
   - **Impact:** High - must be addressed before Phase 1 implementation

2. Update Design Decisions Table

| Decision | Rationale | Date |
|----------|-----------|------|
| Target PyRIT 0.10.0+ | Latest stable version with SQLite-only backend | 2025-12-18 |
| Use SQLite memory | Only option in PyRIT 0.10.0+ (DuckDB removed) | 2025-12-18 |
| Use MessagePiece data model | PyRIT 0.10.0 renamed PromptRequestPiece | 2025-12-18 |

Success Criteria

1. ✅ spec.md reflects PyRIT 0.10.0 APIs (MessagePiece, get_message_pieces, SQLite)
2. ✅ Breaking change documented with exact line number and fix
3. ✅ SQLite section is concise (~20 lines vs 50+)
4. ✅ All code examples use correct PyRIT 0.10.0 syntax
5. ✅ progress.md aligned with spec.md design decisions
6. ✅ Variable naming uses piece / message_pieces (not prompt)

Implementation Note...

This pull request was created from Copilot chat.

---

✨ Let Copilot coding agent set things up for you — coding agent works faster and does higher quality work when set up for your repo.