Skip to content

Automatic Storage Cleanup to Prevent Disk Exhaustion #1186

@bogdanfazakas

Description

@bogdanfazakas

Nodes are experiencing Volume creation failed errors when the /data volume fills up. Currently, ocean-node creates Docker containers and volumes for C2D jobs but does not automatically clean them up, leading to inevitable disk exhaustion regardless of initial storage capacity.

Error Message:

Compute error: Error: Volume creation failed

Current Impact: C2D jobs fail when disk is full

  • Requires manual intervention (cleanup + container restart)
  • No automatic recovery mechanism
  • Production-blocking issue

Implement automatic cleanup system with configurable retention policies: Periodic Cleanup Job & Proactive Disk Monitoring

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions