Random CI builds fail with "Java heap space" on 2025-02-03

[jukzi]

All my yesterdays PR failed to build
for example https://ci.eclipse.org/platform/job/eclipse.platform/job/PR-1722/1/console
17:17:06.638 [ERROR] Failed to execute goal org.apache.maven.plugins:maven-war-plugin:3.4.0:war (default-war) on project infocenter-app: Error assembling WAR: Problem creating war: Execution exception: Java heap space -> [Help 1]

[jukzi]

There is a warning

17:17:04.624 [INFO] Building war: /home/jenkins/agent/workspace/eclipse.platform_PR-1722/ua/infocenter-web/infocenter-app/target/help.war
[1746.617s][warning][gc,alloc] pool-526-thread-1: Retried waiting for GCLocker too often allocating 5000002 words

may relate to https://bugs.openjdk.org/browse/JDK-8192647

[merks]

This is not a new problem. It's been happening for months if not years. It's kind of random, and I personally get the sense, because of the random nature, that it's an infrastructure problem that depends on which node does the build and what other nodes of the same ci instance are doing at the time of the build E.g., my oomph master build that kicks off a midnight was failing every night consistently with killed-for-using-too-much-memory for about 2 weeks but builds kicked of at other times were fine. Now it's starting to work fine again. Go figure. 😱

[laeubi]

E.g., my oomph master build that kicks off a midnight was failing every night consistently with killed-for-using-too-much-memory for about 2 weeks but builds kicked of at other times were fine. Now it's starting to work fine again. Go figure. 😱

But here the job is not killed, it get OOM exception.

[merks]

Yes, I know it's kind of different. It's just difficult to account for the random nature of the failure. While the suggested JDK issue could in theory be related, it's from 2017 so any regression related to that happened long ago...

[jukzi]

Today it works without OOME.

[jukzi]

Closing as not planned to work on. Thanks for the information it happened elsewhere too. Its good to have it documented.

[merks]

FYI, I opened this issue quite some time ago:

https://gitlab.eclipse.org/eclipsefdn/helpdesk/-/issues/4759

It's really a very frustrating problem...

[jukzi]

It's really a very frustrating problem...

More RAM might not help against a JDK issue.

[merks]

@jukzi

I recall that @laeubi added some options to Tycho for monitor the heap usage. Given how regularly (in my experience) the war building fails, would it maybe be good to add the following options to the build?

-Dtycho.printMemory=true
-Dtycho.printMemory.gc=true