Environment file naming convention

[kvrigor]

For context (all emphasis mine):

1. Marco's original post

The ".2025" in default.2025.env kind of breaks machine independency, or at least—depending on the number's meaning—it'll often not be consistent with the "stage" of other machines.

Is it the STAGE, the thing that is specific for JSC? I am trying to bring this into Marvin (using the gompi version, which is at the moment at 2024a). Any objections against this?

I think it should not be part of the name. If no other ideas from anyone right now, I'll think of something (and propose as a commit).

2. My reply

2025 is originally associated with the JSC software stage, and here I thought to expand its meaning to the default stable software stack for some machine X this year. This relates to an important issue: how do we decide on the update schedule of the TSMP2 software stack? This is another big discussion but I would postpone it for now.

3. Marco's response

So 2025 means the same as stage? Even as we now have STAGE==2024a for Marvin and 24.04 for Ubuntu?

Adding it to the env filename is no good, because, given the dot separator, awk'ing the Ubuntu case breaks positions (vielleicht important for the WFE).

I'll regard it as a sort of release then… but we have actual releases for that. I'd rather like to see the default sourcing of default.env (no number) and update the number in jsc.2025.gnu.openmpi, uni-bonn.2024a.gnu.openmpi (potential commit upcoming) and ubuntu.24.04.gnu.openmpi… oh no! the Stage is only known after sourcing the script (and the awk issue)! This is one of those problems I alluded to! (But to be fair, I did not solve that for the Marvin case either.)

[kvrigor]

To reiterate: 2025 is not necessarily the software stage. Rather, it influences which software stage to use on a given machine. It has to be interepreted in the context of the environment file name, e.g.

• default.2025.env: Load the appropriate software stack for the TSMP2 release in 2025.
  • jsc.2025.gnu.openmpi: Stages/2025 is the stable software stack on JSC in 2025.
  • uni-bonn.2024a.gnu.openmpi: 2024a is the stable software stack on Marvin in 2025.
  • ubuntu.24LTS.gnu.openmpi: 24.04 LTS is the most recent stable Ubuntu release in 2025.

How about this folder and naming structure? What's nice about this is it becomes crystal clear which machines does TSMP2 support on a given release year. The structure naturally filters out stale machines/environments.

env
├── 2025
│   ├── default.2025.env
│   ├── jsc.2025.gnu.openmpi
│   ├── ubuntu.24LTS.gnu.openmpi
│   └── uni-bonn.2024a.gnu.openmpi
├── 2026
│   ├── default.2026.env
│   ├── jsc.2026.gnu.openmpi
│   ├── ubuntu.26LTS.gnu.openmpi
│   └── uni-bonn.2026a.gnu.openmpi
.
.
.
│
└── YYYY
    ├── default.YYYY.env
    ├── machine.stage.compiler.mpilib
    └── ...

[mvhulten]

Yes, that is more clear. I like it.

A caveat is that (when you expand its meaning to the default stable software stack for some machine X this year), there is an artificial transition between those software stacks (on every 1 January 00:00). I suppose this is a general problem of year(-only) versioning and that if generally there is at most one release (supposing I can see the env/???? directories as releases) per year. I suppose that it is okay if, e.g., release 2026 is only in February 2026 for JSC and in April 2026 for Uni Bonn.

[mvhulten]

Earlier I suggested this renaming:

Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
        renamed:    jsc.2024.intel.psmpi -> jsc.2024.iimpi
        renamed:    jsc.2025.gnu.openmpi -> jsc.2025.gompi
        renamed:    jsc.2025.intel.psmpi -> jsc.2025.iimpi
        renamed:    uni-bonn.gnu.openmpi -> uni-bonn.2023a.gompi

I did not push it fwd, because—even though it looks cleaner—it is not per se an improvement.

[AGonzalezNicolas]

@mvhulten what does the "a" on uni-bonn.2023a.gompi stand for?

[mvhulten]

The "a" is part of the version number of the iimpi and gompi compiler toolchain Lmod packages (that include the compiler and accompanying MPI implementation). On Marvin current versions are iimpi/2023a and gompi/2024a.

[kvrigor]

A good naming would be unambiguously differentiate among these combinations:

• Compilers
  • GCC (gcc, gfortran)
  • Intel Classic (icc, ifort)
  • Intel LLVM ( ifx, icx)
  • LLVM (clang, flang)
• MPI libraries
  • OpenMPI
  • ParaStationMPI
  • MPICH
  • MVAPICH
  • IntelMPI

And there are a lot more combinations out there. While I also like the concise syntax, I'm afraid it's not expressive enough to easily accommodate new combinations.

[mvhulten]

In the current scheme, there is no perfect disambiguation either (if you stick with e.g. .intel). This is fine, because the disambiguation is already in the files. We just have to make it unlikely to have collisions in filenames. If this would occur, names can be adjusted then (without breaking the general scheme ORG.RELEASE.COMPILER.MPI). If you have a collision, maybe we made the choice in env files to large. Maybe you should replace the classic Intel compiler with ifx instead of renaming jsc.2025.intel.psmpi. The commit and the fact that it is saved into the install and will be saved into the build directory is enough for reproducibility.

Or you introduce jsc.2025.intelllvm.psmpi and optionally rename the original. You can see on the way.

In this Intel compiler example I'd not change the filename or introduce a new file because Intel/2024.2.0 contains both ifort and ifx and we don't have time to validate if all components including libraries and utilities stick with one or the other.* Who cares anyway? If you do care, you can find out by deep inspection in the build directory.

I think the current scheme is a fine compromise: simple looking filenames, but not too likely to get to a naming conflict. It think it's expressive enough.

Forget the toolchain Lmod meta package names; they could be loaded on some machines, but are just confusing for the user.

I think also this generalisation of stage is fine.

Sorry, too many ramblings. Bottom line is that in my opinion it's okay to close this issue.

---

* edit: Maybe I'm wrong about not being able to follow which compiler is used in all the details. I suppose CMake tells the rest what to use. Then they should use it.

[kvrigor]

Maybe you should replace the classic Intel compiler with ifx instead of renaming jsc.2025.intel.psmpi. The commit and the fact that it is saved into the install and will be saved into the build directory is enough for reproducibility.

Or you introduce jsc.2025.intelllvm.psmpi and optionally rename the original.

jsc.2025.intelllvm.psmpi is good. We can't use ifx since we're using a mix of C/C++/Fortran compilers.

In this Intel compiler example I'd not change the filename or introduce a new file because Intel/2024.2.0 contains both ifort and ifx and we don't have time to validate if all components including libraries and utilities stick with one or the other.* Who cares anyway? If you do care, you can find out by deep inspection in the build directory.

IMO it's safe to be explicit by default. Opaqueness can bite back really hard (e.g. see this related eCLM issue)

I think the current scheme is a fine compromise: simple looking filenames, but not too likely to get to a naming conflict. It think it's expressive enough.

Yes, though I'd prefer to have a shorter naming scheme if possible. We can think about it later when we have actually more combinations to support.

Bottom line is that in my opinion it's okay to close this issue.

Not yet; I plan to do the env folder restructuring on another PR. Also I think the filenaming issue is not that urgent; let's wait for inspiration to strike and come back to this later 😜