Skip to content

Add circuit breaker pattern to vMCP health monitoring#3528

Merged
yrobla merged 4 commits intomainfrom
issue-3036
Feb 5, 2026
Merged

Add circuit breaker pattern to vMCP health monitoring#3528
yrobla merged 4 commits intomainfrom
issue-3036

Conversation

@yrobla
Copy link
Contributor

@yrobla yrobla commented Jan 30, 2026

Implement circuit breaker state machine for backend health monitoring in Virtual MCP Server. The circuit breaker prevents cascading failures by tracking backend failures and transitioning through three states: Closed (normal operation), Open (failing), and HalfOpen (recovery testing).

Key changes:

  • Add CircuitBreaker with thread-safe state machine
  • Integrate circuit breaker with status tracker
  • Integrate circuit breaker with health monitor
  • Add comprehensive test coverage

The circuit breaker is disabled by default for backwards compatibility. Configuration wiring from YAML/CRD to health monitor is intentionally left for a follow-up PR to keep this change focused on the core implementation.

Large PR Justification

This is an atomic PR that exposes the circuit breaker capability. It includes comprehensive tests to validate it. Cannot be split as it's an isolated functionality.

Related-to: #3036

@github-actions github-actions bot added the size/XL Extra large PR: 1000+ lines changed label Jan 30, 2026
Copy link
Contributor

@github-actions github-actions bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Large PR Detected

This PR exceeds 1000 lines of changes and requires justification before it can be reviewed.

How to unblock this PR:

Add a section to your PR description with the following format:

## Large PR Justification

[Explain why this PR must be large, such as:]
- Generated code that cannot be split
- Large refactoring that must be atomic
- Multiple related changes that would break if separated
- Migration or data transformation

Alternative:

Consider splitting this PR into smaller, focused changes (< 1000 lines each) for easier review and reduced risk.

See our Contributing Guidelines for more details.


This review will be automatically dismissed once you add the justification section.

@yrobla yrobla requested review from Copilot and removed request for JAORMX, amirejaz, jerm-dro and jhrozek January 30, 2026 06:05
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR implements a circuit breaker pattern for backend health monitoring in the Virtual MCP Server to prevent cascading failures. The circuit breaker tracks backend failures and transitions through three states (Closed, Open, HalfOpen) to protect the system when backends are experiencing issues.

Changes:

  • Adds CircuitBreaker implementation with thread-safe state machine for tracking failures and managing state transitions
  • Integrates circuit breaker with StatusTracker and Monitor for health check gating
  • Includes comprehensive test coverage for circuit breaker logic and integration scenarios

Reviewed changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated 3 comments.

Show a summary per file
File Description
pkg/vmcp/health/circuit_breaker.go New implementation of CircuitBreaker with state machine logic and thread-safe operations
pkg/vmcp/health/circuit_breaker_test.go Comprehensive unit tests for circuit breaker state transitions, concurrency, and edge cases
pkg/vmcp/health/status.go Integration of circuit breaker into status tracker with initialization, state querying, and health check gating
pkg/vmcp/health/status_test.go Integration tests for circuit breaker with status tracker including disabled mode scenarios
pkg/vmcp/health/monitor.go Circuit breaker configuration, initialization, and integration with health check loop
pkg/vmcp/health/monitor_test.go End-to-end tests for circuit breaker behavior including disabled, enabled, recovery, and status reporting scenarios

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@github-actions github-actions bot added size/XL Extra large PR: 1000+ lines changed and removed size/XL Extra large PR: 1000+ lines changed labels Jan 30, 2026
@github-actions github-actions bot dismissed their stale review January 30, 2026 06:31

Large PR justification has been provided. Thank you!

@github-actions
Copy link
Contributor

✅ Large PR justification has been provided. The size review has been dismissed and this PR can now proceed with normal review.

@github-actions github-actions bot added size/XL Extra large PR: 1000+ lines changed and removed size/XL Extra large PR: 1000+ lines changed labels Jan 30, 2026
@codecov
Copy link

codecov bot commented Jan 30, 2026

Codecov Report

❌ Patch coverage is 86.44860% with 29 lines in your changes missing coverage. Please review.
✅ Project coverage is 65.68%. Comparing base (d45e969) to head (12d4d77).
⚠️ Report is 21 commits behind head on main.

Files with missing lines Patch % Lines
pkg/vmcp/health/monitor.go 70.27% 11 Missing ⚠️
pkg/vmcp/health/status.go 88.37% 7 Missing and 3 partials ⚠️
pkg/vmcp/health/circuit_breaker.go 91.20% 8 Missing ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##             main    #3528      +/-   ##
==========================================
+ Coverage   65.47%   65.68%   +0.21%     
==========================================
  Files         404      407       +3     
  Lines       39781    40195     +414     
==========================================
+ Hits        26046    26403     +357     
- Misses      11721    11754      +33     
- Partials     2014     2038      +24     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 6 out of 6 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@yrobla yrobla requested review from Copilot and jerm-dro February 4, 2026 07:49
@github-actions github-actions bot added size/XL Extra large PR: 1000+ lines changed and removed size/XL Extra large PR: 1000+ lines changed labels Feb 4, 2026
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 2 comments.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@github-actions github-actions bot added size/XL Extra large PR: 1000+ lines changed and removed size/XL Extra large PR: 1000+ lines changed labels Feb 4, 2026
amirejaz
amirejaz previously approved these changes Feb 4, 2026
@yrobla yrobla requested review from amirejaz and Copilot February 4, 2026 15:14
@github-actions github-actions bot added size/XL Extra large PR: 1000+ lines changed and removed size/XL Extra large PR: 1000+ lines changed labels Feb 4, 2026
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 1 comment.


💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@github-actions github-actions bot added size/XL Extra large PR: 1000+ lines changed and removed size/XL Extra large PR: 1000+ lines changed labels Feb 4, 2026
@yrobla yrobla merged commit 8e25a44 into main Feb 5, 2026
36 checks passed
@yrobla yrobla deleted the issue-3036 branch February 5, 2026 08:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/XL Extra large PR: 1000+ lines changed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants