Skip to content

Conversation

@networmix
Copy link
Owner

@networmix networmix commented Dec 20, 2025

  • Restructured Monte Carlo results: baseline is returned separately, and results now contains deduplicated failure patterns with occurrence_count.
  • Removed baseline parameter from Monte Carlo APIs; baseline now runs implicitly.
  • Added FlowIterationResult.occurrence_count to track how many iterations produced each failure pattern.

Note

Monte Carlo workflows now run a separate baseline automatically, dedupe failure iterations with occurrence_count, add optional failure_trace, remove the baseline parameter, and update steps, docs, scenarios, and tests accordingly (v0.14.0).

  • Core/Execution:
    • FailureManager: always runs a separate baseline, deduplicates failure patterns, tracks occurrence_count, optional failure_trace; parallel/serial runners simplified.
  • Workflow Steps:
    • MaxFlow and TrafficMatrixPlacement: removed baseline param; accept seed; write data.baseline plus deduped data.flow_results; logging/metadata tweaks.
  • Results Schema:
    • FlowIterationResult: add occurrence_count and failure_trace; to_dict/export updated.
    • Result shape: baseline separate; flow_results = unique patterns; metadata includes unique_patterns.
  • Failure Policies:
    • apply_failures(...) accepts failure_trace and records mode/rule selections and expansions.
  • Docs/DSL:
    • References/examples updated to new schema (baseline implicit, no baseline: field in YAML); added params (seeds, include/exclude lists); examples count increased.
  • Scenarios/Tests:
    • All scenarios and tests updated to new result shape and removed baseline parameter; new tests for tracing, dedup, weighting.
  • Version:
    • Bump to 0.14.0; changelog added with breaking changes.

Written by Cursor Bugbot for commit c399e92. This will update automatically on new commits. Configure here.

- Restructured Monte Carlo results: `baseline` is returned separately, and `results` now contains deduplicated failure patterns with `occurrence_count`.
- Removed `baseline` parameter from Monte Carlo APIs; baseline now runs implicitly.
- Added `FlowIterationResult.occurrence_count` to track how many iterations produced each failure pattern.
Copilot AI review requested due to automatic review settings December 20, 2025 05:29
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces version 0.14.0 with breaking changes to the Monte Carlo analysis API. The main purpose is to restructure how Monte Carlo results are returned: baseline results are now separated from failure iterations, and failure results are deduplicated with an occurrence_count field to track pattern frequency.

Key Changes

  • Removed the baseline parameter from Monte Carlo APIs; baseline is now always run implicitly and returned separately
  • Restructured results to separate baseline from results array, with deduplicated failure patterns tracked by occurrence_count
  • Added FlowIterationResult.failure_trace to capture detailed failure policy execution trace when store_failure_patterns=True

Reviewed changes

Copilot reviewed 43 out of 43 changed files in this pull request and generated 1 comment.

Show a summary per file
File Description
ngraph/exec/failure/manager.py Core refactoring: baseline always runs first, results deduplicated with occurrence_count tracking, failure_trace support
ngraph/results/flow.py Added failure_trace and occurrence_count fields to FlowIterationResult with validation
ngraph/model/failure/policy.py Extended apply_failures() to populate failure_trace dict when requested
ngraph/workflow/max_flow_step.py Removed baseline parameter, updated to handle separate baseline result
ngraph/workflow/traffic_matrix_placement_step.py Removed baseline parameter, updated to handle separate baseline result
tests/exec/failure/test_manager.py Comprehensive test updates for new API, added tests for occurrence_count weighting
tests/exec/failure/test_manager_integration.py Updated tests for deduplicated results and failure_trace
tests/model/failure/test_failure_trace.py New test file for failure_trace functionality (378 lines)
tests/workflow/*.py Updated all workflow tests to remove baseline parameter and handle new result structure
scenarios/*.yaml Removed baseline parameter from all scenario YAML files
docs/* Updated documentation to reflect API changes
CHANGELOG.md Documented breaking changes for v0.14.0
pyproject.toml, ngraph/_version.py Version bump to 0.14.0
Comments suppressed due to low confidence (1)

ngraph/exec/failure/manager.py:590

  • The metadata field 'unique_patterns' is computed from 'num_unique_tasks', which represents unique failure patterns. However, when iterations=0 (no failures), this will be 0. The metadata should accurately reflect that unique_patterns refers to failure patterns, not including the baseline. Consider explicitly documenting this behavior or verifying that num_unique_tasks is correctly set when there are no failure iterations.
            "metadata": {
                "iterations": iterations,
                "parallelism": parallelism,
                "analysis_function": func_name,
                "policy_name": self.policy_name,
                "execution_time": elapsed_time,
                "unique_patterns": num_unique_tasks,

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment on lines +319 to +322
"FlowIterationResult.occurrence_count must be a positive int: %r",
self.occurrence_count,
)
raise ValueError("occurrence_count must be a positive int")
Copy link

Copilot AI Dec 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The validation error message should clarify that occurrence_count must be a positive integer (greater than or equal to 1), not just any positive value. The current message "occurrence_count must be a positive int" is accurate but could be more explicit about the minimum value of 1.

Suggested change
"FlowIterationResult.occurrence_count must be a positive int: %r",
self.occurrence_count,
)
raise ValueError("occurrence_count must be a positive int")
"FlowIterationResult.occurrence_count must be a positive integer (>= 1): %r",
self.occurrence_count,
)
raise ValueError(
"occurrence_count must be a positive integer (>= 1)"
)

Copilot uses AI. Check for mistakes.
@networmix networmix merged commit 6464476 into main Dec 20, 2025
13 checks passed
@networmix networmix deleted the report_cleanup branch December 20, 2025 05:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants