Skip to content

Managing EasyBuild-installed software on LUMI

Repositories and their function

There are three LUST-maintained repositories with EasyConfig files:

Users can have their own repo also. When it is cloned as UserRepo in $EBU_USER_PREFIX, it will be used by EasyBuildwhen configured with LUMI-EasyBuild-user.

There is another repository also, LUMI-EasyBuild-docs to manage building the LUMI Software Library

Creating and installing a new release of the LUMI software stack

TODO not complete

  • We work as much as possible with pull requests so that there is a better history of when packages were added or modified in the software stack and library.

    It also enables someone to check the layout of the easyconfigs and see if they more or less follow the guidelines and for packages centrally installed, it makes it easy to see when there is something new that needs to be installed.

  • The central software stack is no playground. Installations in there are tested as much as possible in a copy of the stack. Once a module is in there, it cannot be changed as it could disrupt work of users who are using that module at that time. This is one of the reasons why we prefer to have software as much as possible installed by the users in their account: They only need to talk to their project members to agree when a package can be changed.

    The stack exists in 5 copies on LUMI

    • The master copy is in /pfs/lustref1/appl/lumi and is available on /appl/lumi only on uan06. This implies that currently all software installation in the central stack should be done on uan06 as we want to make sure that paths that may get hard-coded in scripts and binaries use /appl/lumi and do not refer to a specific file system.

    • The copies used on the login nodes and compute nodes are in /pfs/lustrepX/appl/lumi, with X 1, 2, 3 or 4.

    The */appl/lumi directories can be written by everybody in the appl_lumi(462000009) project. Everybody in that project should use uan06 as little as possible as one may very well accidentally damage the software stack. Some applications want to upgrade themselves - we keep them out of the stack but may make errors there - or when using Python, it may decide to install additional packages in a directory that happens to be in the Python search path. The number of people in that group should be as small as possible, basically the maintainers of the software stacks installed in /appl/lumi and their backups and noone else.

    Deleting a package from the central stack is done in multiple steps:

    1. Often first a warning via the Lmod admin.lst file
    2. Delete the module
    3. And a few weeks later, delete the software from the central stack

    Every synchronisation of the central stack can take more than one hour, and the order in which various directories are synchronised is well thought out so that a module will never appear on the system if the software is not first made available on all 4 copies.

  • Every release is currently prepared on a laptop. It has a directory containing the following repositories:

  • Before creating a new release, add the release name (YYYYMMDD) to the docs/whats_new.md with a description of the changes/additions. This page will also appear in the LUMI Software Library.

Also TODO: Rework the script used to create a release and include in LUMI-SoftwareStack rather then a totally separate private repository that contains other scripts also that have little to do with LUMI.

Documenting the software stack on LUMI

LUMI Software Library

The LUMI Software Library is built automatically from a number of files in markdown syntax. The script is very naive when it tries to renumber the title levels. It goes wrong when a code block is included that starts on the first column and contains lines starting with #. The workaround is to indent such a code block with one space.

Files stored with the EasyConfigs of a package:

  • USER.md: Optional, extra information for regular users.

  • LICENSE.md: Highly preferred. Information on the license of the package.

  • README.md: Technical information:

    • Where can we find the package?

    • Is there already support in EasyBuild or Spack?

    • Overview of our past EasyConfigs and how they were built.

      Never delete information from this, or you'd have to rewrite the first entry in the new list to contain all previous elements that are necessary to understand the EasyConfig.

Some interesting examples to look at:

  • QuantumESPRESSO: uses the full range of tricks. Check the files in the GitHub: the pictures are in the 'files' subdirectory

  • ELPA used to be in the central stack but was moved out because we needed multiple configurations in a single toolchain and also had too many issues to get it to work correctly.

    It has a 'README.md' file in both LUMI-SoftwareStack and LUMI-EasyBuild-contrib but as we are phasing the 22.08 toolchain out, that example may have become irrelevant with a dead link by the time you read this.

  • buildtools: Made sure that users can actually find the included packages through the search box by mentioning them on the page.

LUMI-SoftwareStack repository

The repository contains a docs subdirectory with markdown-based documentation.

The docs/config subdirectory contains the mkdocs.yml file for the documentation, a Makefile to test the documentation and the requirements.txt file for the mkdocs setup in Python.

The documentation is updated automatically in github.io when a push to the main branch is made in the repository and can be accessed on lumi-supercomputer.github.io/LUMI-SoftwareStack/.

Documenting the actual installation process on LUMI uan06

Each installation done on LUMI uan06 is carefully documented on the "Change log for the software stack in /appl/lumi" page in the LUST eduuni wiki.

This is done before actually doing the installation and has been tested as much as possible on a copy of the software stack (currently a personal copy, could be a LUST one in 462000008).

Managing versions of dependencies

When building a new software stack, we often want to change the version of packages to a newer version. This implies that all EasyConfigs where the package appears as a dependency, should also change. The dependencies parameter is way too complex to tackle with try-amend so we needed another approach.

On LUMI, we declare the version of packages as much as possible through variables, e.g., local_zlib_version. We maintain files with all version variables in LUMI-SoftwareStack (look vor the versions-yy.MM.txt files) and LUMI-EasyBuild-contrib (look for the versions-contrib-YY.MM.txt files). For Contrib this only makes sense for packages that are likely to be used as a dependency and are typically available in only one version (though for, e.g., PLUMED., we simply define separate variables for PLUMED 8.X and 9.X).

There are two scripts to update toolchains and dependency versions:

  • upgrade-tc.py is a script developed at CSCS to update the toolchain version in an EasyConfig and also adapt its name accordingly.

  • upgrade-locals.lua is a script developed by LUST to use the variables in the version-YY.MM.txt files to update the corresponding variables in the EasyConfigs. The only downside is that it cannot adapt the name of the EasyConfig. That still has to be done manually.

It is easiest to use those scripts through aliases. E.g., Kurt Lust uses the following aliases to adapt EasyConfig files for LUMI/24.03:

alias tc2403="$installroot/LUMI-SoftwareStack/tools/upgrade-tc.py --metadata $installroot/LUMI-SoftwareStack/easybuild/config/external_modules_metadata-CPE-24.03.cfg --version 24.03 --filenames"
alias locals2403="$installroot/LUMI-SoftwareStack/tools/upgrade-locals.lua $installroot/LUMI-SoftwareStack/easybuild/easyconfigs/versions-24.03.txt"
alias contrib2403="$installroot/LUMI-SoftwareStack/tools/upgrade-locals.lua $installroot/LUMI-EasyBuild-contrib/easybuild/easyconfigs/versions-contrib-24.03.txt"

(with installroot an environment variable pointing to the local installation he uses; you could use /appl/lumi for that but would probably like to refer to your own copy of LUMI-EasyBuild-contrib).

[Next: EasyBuild tips & tricks]