Local module is a beneficial approach to efficiently organize your self-installed libraries. Additionally, EasyBuild serves as a framework for building and installing software, particularly those with dependencies. The following sections provide general guidelines on how to use local modules and EasyBuild to install and run software on LANTA.
Updated: May 2024
To utilize local modules, we suggest creating a bash command such as localmod
that contains essential setup within your $HOME/.bashrc
. The provided set of commands below will create a local module directory based on LOCALMOD_DIR
and then define the localmod
command in your $HOME/.bashrc
.
This localmod
command is needed to be executed to activate local modules, before installing software in Step 1 and before using local modules in your job script (see below).
LOCALMOD_DIR=$HOME/localmod # *** EDITABLE, set to an empty path *** mkdir -p ${LOCALMOD_DIR} # Create the directory if not already exist cat << Eof >> $HOME/.bashrc # Define the below "localmod" function in your .bashrc function localmod(){ export EASYBUILD_PREFIX=${LOCALMOD_DIR} module use ${EASYBUILD_PREFIX}/modules/all export EASYBUILD_OPTARCH=x86-milan echo "*** Import modules at ${EASYBUILD_PREFIX} on LANTA ***" } Eof |
|
By default, only you, the person who create the directory, have permission to install modules in it. However, your project members can access and use your modules by executing |
|
To install a new software, it is strongly recommended to clean your environment by re-login or open a new bash session, then execute module purge
.
To install a new software, it is strongly recommended to clean your environment by re-login or open a new bash session, then execute module purge
.
Enable local module by executing the command defined in Step 0 → localmod
Load EasyBuild → module load EasyBuild
Check your current EasyBuild configuration → eb --show-config
Note: the installpath
should be set to a local directory that you have write permission.
localmod module load EasyBuild eb --show-config |
Visit EasyConfig Standard Repo or other repositories.
Download the latest .eb file of the software that you would like to install, then rename it to<software-name>-<software-version>-<cpe-toolchain-name>-<cpe-toolchain-version>.eb
where CPE toolchain name is cpeGNU
, cpeIntel
, cpeCray
or others, corresponding to the compiler that you would like to use. For their available version, use module spider cpeXXX
or see Current CPE toolchains.
For example, download giflib-5.2.1-GCCcore-13.2.0.eb
and rename it to giflib-5.2.1-cpeCray-23.03.eb
|
Open the .eb file using a text editor such as vi
, then edit the following
toolchain
Correct the toolchain information, e.g., toolchain = {'name': 'cpeCray', 'version': '23.03'}
. This must be consistent with the CPE toolchain and file name chosen in Step 2.
builddependencies and/or dependencies
Check if the software dependencies are available in ThaiSC modules by using module spider <dep-name>
.
For example, module spider binutils
. If available, you could use them by editing each dependency information ('<dep-name>', '<dep-version>', '<dep-suffix>', <dep-toolchain>)
where
<dep-name>
is the dependency name, e.g., binutils
.
<dep-version>
is the dependency version, e.g., 2.40
.
<dep-suffix>
and <dep-toolchain>
are optional. The default will use the same toolchain as in Step 3.1 without any suffix. This is recommended. For more details, visit writing .eb.
|
As a demonstration, the .eb files for installing GNU nano (text editor) and feh (image viewer) are available at /project/common/EasyBuild
.
As a demonstration, the .eb files for installing GNU nano (text editor) and feh (image viewer) are available at /project/common/EasyBuild
.
Execute eb -D --robot-paths=$(pwd) <the-eb-file>.eb
to do a short dry run and check for dependencies.
If the command run successfully, it will display build status of the software’s dependencies.
(You could ignore warning like WARNING: Failed to determine toolchain hierarchy for ...
)
On the other hand, if you get ERROR: Missing dependencies: xxx/yyy-zzz
, this means that the EasyConfig file of the library/software name xxx
with version yyy
using toolchain zzz
is missing. You need to get the EasyConfig file for the dependency xxx/yyy-zzz
by doing Step 2 and 3 again. That is, you need to repeatedly do Step 2 and 3 for all unavailable dependencies required by the target software, and put them in the same directory.
To install the software, execute
eb -r --robot-paths=$(pwd) --parallel=1 --trace --tmpdir=$(pwd) <the-eb-file>.eb |
|
Some software may not support parallel build; therefore, |
Use module --ignore-cache avail
to check if the module is available.
Use module --ignore-cache avail
to check if the module is available.
Don’t forget to enable local module by executing the |
#!/bin/bash #SBATCH -p compute # Partition #SBATCH -N 1 # Number of nodes #SBATCH --ntasks-per-node=128 # Number of MPI processes per node #SBATCH --cpus-per-task=1 # Number of OpenMP threads per MPI process #SBATCH -t 5-00:00:00 # Job runtime limit #SBATCH -J MyJob # Job name #SBATCH -A ltxxxxxx # Account *** {USER EDIT} *** source $HOME/.bashrc localmod module purge module load <your-local-module> export OMP_NUM_THREADS=${SLURM_CPUS_PER_TASK} srun -c${SLURM_CPUS_PER_TASK} <software-command> |
Reference
https://mdc-easybuild.readthedocs.io/en/eb462/version-specific/easyconfig_parameters.html
https://ulhpc-tutorials.readthedocs.io/en/latest/tools/easybuild/
EasyConfig repositories
https://github.com/easybuilders/easybuild-easyconfigs/tree/develop/easybuild/easyconfigs
https://github.com/Lumi-supercomputer/LUMI-EasyBuild-contrib/tree/main/easybuild/easyconfigs (Software may be in a different name.)
https://github.com/eth-cscs/production/tree/master/easybuild/easyconfigs (Software may be in a different name.)