Skip to content

Moves API heartbeat to it's own thread.#7331

Open
dkliban wants to merge 1 commit intopulp:mainfrom
dkliban:7315
Open

Moves API heartbeat to it's own thread.#7331
dkliban wants to merge 1 commit intopulp:mainfrom
dkliban:7315

Conversation

@dkliban
Copy link
Member

@dkliban dkliban commented Feb 18, 2026

fixes: #7315

📜 Checklist

  • Commits are cleanly separated with meaningful messages (simple features and bug fixes should be squashed to one commit)
  • A changelog entry or entries has been added for any significant changes
  • Follows the Pulp policy on AI Usage
  • (For new features) - User documentation and test coverage has been added

See: Pull Request Walkthrough

@dkliban dkliban force-pushed the 7315 branch 3 times, most recently from 1356302 to c313a9a Compare February 18, 2026 16:20
self.heartbeat()

def _heartbeat_loop(self):
while self.alive:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is this for?

super().notify()
self.heartbeat()

def _heartbeat_loop(self):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What happens if the thread dies?

super().__init__(*args, **kwargs)
self.heartbeat_thread = None

def notify(self):
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

First i thought, this is dead code. Now i think we have the opportunity to check in on the thread here.

# Store heartbeat interval for the heartbeat thread
self.heartbeat_interval = settings.API_APP_TTL // 3

# Start heartbeat thread before entering the main loop
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now why don't we then put all of that stuff right before entering the main loop?

name_template = name_template_var.get()
if name_template:
settings.set("WORKER_NAME_TEMPLATE", name_template)
if settings.API_APP_TTL < 2 * self.timeout:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you never get to see this warning?

logger.error(f"An API app with name {self.name} already exists in the database.")
exit(Arbiter.WORKER_BOOT_ERROR)

# Store heartbeat interval for the heartbeat thread
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
# Store heartbeat interval for the heartbeat thread
# Store the value of one third of the desired API application's time to live to the heartbeat interval on the current instance to be accessible by the heartbeat thread.

:mild sarcasm:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Long uploads cause API workers to miss hearbeats

2 participants

Comments