feat: Add PII probing transforms and scoring

[rdheekonda]

Key Changes:

• Add 5 PII extraction transforms for adversarial testing
• Add 3 advanced PII scorers with statistical CI support
• Add 59 unit tests with no LLM calls
• Based on 2024-2025 research (Carlini, PII-Scope, Model Inversion)

Added:

• dreadnode/transforms/pii_extraction.py: 5 transforms
  • repeat_word_divergence: Trigger memorization (Carlini technique)
  • continue_exact_text: Force prefix completion
  • complete_from_internet: Probe memorized web content
  • partial_pii_completion: Adaptive extraction with hints
  • public_figure_pii_probe: Test public figure disclosure
• dreadnode/scorers/pii_advanced.py: 3 scorers + 2 helpers
  • training_data_memorization: Entropy/pattern detection
  • credential_leakage: 13 credential types (API keys, tokens)
  • pii_disclosure_rate: Binary scorer for eval aggregation
  • wilson_score_interval: Statistical confidence intervals
  • calculate_disclosure_rate_with_ci: Helper for 95% CI analysis
• examples/airt/pii_extraction_attacks.ipynb: Usage examples
  • TAP attacks with PII transforms
  • Eval-based disclosure rate testing
  • Credential leakage detection
• tests/test_pii_extraction_transforms.py: 21 transform tests
• tests/test_pii_advanced_scorers.py: 38 scorer tests

Changed:

• dreadnode/transforms/__init__.py: Export pii_extraction module
• dreadnode/scorers/__init__.py: Export new scorers and helpers

---

Generated Summary:

• Added new functionalities for advanced PII detection in the pii_advanced.py module.
• Introduced five new scoring functions:
  • training_data_memorization: Detects verbatim memorized text from training data.
  • credential_leakage: Identifies potential leaked credentials, API keys, and tokens.
  • pii_disclosure_rate: Binary detection of PII for evaluation purposes.
  • wilson_score_interval: Calculates statistical confidence intervals for PII disclosure rates.
  • calculate_disclosure_rate_with_ci: Aggregates PII detection results to compute disclosure rates.
• Enhanced the __init__.py files to include new scorer functions and maintain module imports.
• Created pii_extraction.py with functions targeting specific PII extraction techniques:
  • repeat_word_divergence, continue_exact_text, complete_from_internet, partial_pii_completion, and public_figure_pii_probe.
• Added a new Jupyter notebook example to demonstrate adversarial PII extraction techniques, including various attack scenarios.
• Potential impact: These changes significantly enhance the capability to assess and evaluate PII leakage risks in outputs from language models, valuable for security assessments in development and production environments.

This summary was generated with ❤️ by rigging

Research References:

• Carlini et al. (USENIX 2024): Extracting Training Data from LLMs
• PII-Scope Benchmark (arXiv 2410.06704): 48.9% success rate
• Model Inversion Attacks (arXiv 2507.04478): Credential extraction