Skip to content

Conversation

@zaneb
Copy link
Member

@zaneb zaneb commented Dec 1, 2025

The default MASTER_MEMORY was changed in some agent scenarios in order
to work around a problem in OKD where FCOS would use only 20% of RAM for
the ephemeral storage instead of 50%. Other scenarios were later changed
to match when tests started failing in 4.19 due to OCPBUGS-62790.

Return the defaults to the minimum values validated by assisted-service
in ABI, so that any future regressions like OCPBUGS-62790 are caught
immediately.

This depends on openshift/installer#10133, including backports to 4.20 and 4.19.

@openshift-ci
Copy link

openshift-ci bot commented Dec 1, 2025

Skipping CI for Draft Pull Request.
If you want CI signal for your change, please convert it to an actual PR.
You can still manually trigger a test run with /test all

@openshift-ci openshift-ci bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Dec 1, 2025
@openshift-ci
Copy link

openshift-ci bot commented Dec 1, 2025

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by:
Once this PR has been reviewed and has the lgtm label, please assign dtantsur for approval. For more information see the Code Review Process.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@zaneb zaneb marked this pull request as ready for review December 16, 2025 19:40
@openshift-ci openshift-ci bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Dec 16, 2025
@openshift-ci openshift-ci bot requested review from bfournie and sadasu December 16, 2025 19:40
@zaneb
Copy link
Member Author

zaneb commented Dec 16, 2025

openshift/installer#10133 (4.21), openshift/installer#10140 (4.20), and openshift/installer#10153 (4.19) have all merged.

Previously whenever an agent scenario was specified, it overrode the
vCPU count, disk size, and memory size from the config file.

Change this so that the agent scenario values are used as a default, but
the configuration can override with the usual environment variables like
MASTER_MEMORY.
The default MASTER_MEMORY was changed in some agent scenarios in order
to work around a problem in OKD where FCOS would use only 20% of RAM for
the ephemeral storage instead of 50%. Other scenarios were later changed
to match when tests started failing in 4.19 due to OCPBUGS-62790.

Return the defaults to the minimum values validated by assisted-service
in ABI, so that any future regressions like OCPBUGS-62790 are caught
immediately.
@openshift-ci
Copy link

openshift-ci bot commented Dec 18, 2025

@zaneb: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/e2e-metal-ovn-arbiter 3678ebd link false /test e2e-metal-ovn-arbiter
ci/prow/e2e-agent-compact-ipv4 3678ebd link true /test e2e-agent-compact-ipv4
ci/prow/e2e-agent-4control-ipv4 3678ebd link false /test e2e-agent-4control-ipv4
ci/prow/e2e-agent-5control-ipv4 3678ebd link false /test e2e-agent-5control-ipv4
ci/prow/e2e-agent-ha-dualstack 3678ebd link false /test e2e-agent-ha-dualstack
ci/prow/e2e-agent-bad-dns 3678ebd link false /test e2e-agent-bad-dns
ci/prow/e2e-agent-compact-ipv4-iso-no-registry 3678ebd link false /test e2e-agent-compact-ipv4-iso-no-registry

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

@zaneb
Copy link
Member Author

zaneb commented Dec 18, 2025

Tests appear to be still failing with "no space left on disk" when pulling images, even though the fix has worked as intended and created a separate tmpfs for the ostree (and even though there is still 815MiB free):

Filesystem      Size  Used Avail Use% Mounted on
tmpfs           3.2G  122M  3.1G   4% /run
tmpfs           7.9G  6.8G  1.1G  87% /run/ephemeral_base
/dev/loop0      7.8G  7.0G  815M  90% /run/ephemeral
tmpfs           4.0G  2.4G  1.7G  60% /usr
tmpfs           7.9G   84M  7.8G   2% /tmp

This implies that even without the coreos layering change, we would be exceeding the limit for 16GiB servers now. In my local testing 16GiB servers worked OK, so I am not sure what is different here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant