Environment Modules
On computational clusters it is very common to find users with code that depends on a particular version of a compiler or library. For example, one user may have code that refuses to compile with any GCC compiler more recent than version 4.4, whereas a new compiler optimization / feature in version 4.9 maybe very desirable for another user. The effort required to convert the code to use a different version can be significant. To overcome this problem we install multiple versions of each compiler, library or application, and let each user choose the most suitable version. Environment Modules offer a simple means of customising your environment to access the required versions of installed software.
Discover what software are installed
[samuel@prlhpcms01 ~]$ module avail ------------------------------------- /shared/modulefiles/compilers ------------------------------------- gnu/gcc4.4.7 gnu/gcc4.9.2 intel/15.2 pgi/15.3 ------------------------------------- /shared/modulefiles/libraries ------------------------------------- gnu/BLAS gnu/netcdf3.6.3 intel/jpeg9 pgi/jpeg9 gnu/gdb-7.9 gnu/netcdf4.0 intel/libpng1.5.12 pgi/libpng1.5.12 gnu/hdf51.8.14 gnu/netcdf4.3 intel/mkl pgi/mpich3.1.3 gnu/jpeg9 gnu/openmpi-gcc4.4.7 intel/mpich3.1.4 pgi/netcdf3.6.3 gnu/lapack-3.5.0 gnu/openmpi-gcc4.9.2 intel/netcdf3.6.3 pgi/netcdf4.0 gnu/libpng1.5.12 gnu/valgrind-3.10.1 intel/netcdf4.0 pgi/netcdf4.3 gnu/mpich3.1.4 gnu/zlib1.2.8 intel/netcdf4.3 pgi/openmpi-1.8.4 gnu/mvapich-psm-x86_64 intel/hdf51.8.14 intel/zlib1.2.8 pgi/zlib1.2.8 gnu/mvapich2-psm-x86_64 intel/impi5.0 pgi/hdf51.8.14 ------------------------------------- /shared/modulefiles/toolkits -------------------------------------- cuda-6.5
Add a particular version of software to your environment
[samuel@prlhpcms01 ~]$ module add intel-compiler impi5.0 gcc4.9.2
Remove a particular version of software from your environment
[samuel@prlhpcms01 ~]$ module rm gcc4.9.2
Check software that you are currently using
[samuel@prlhpcms01 ~]$ module list Currently Loaded Modulefiles: 1) intel-compiler 2) impi5.0
Remove all software from your environment
[samuel@prlhpcms01 ~]$ module purge
Switch between particular versions of software
[samuel@prlhpcms01 ~]$ module switch gcc4.4.7 gcc4.9.2
What does it mean if a version of software is marked as default?
This means that if the version number is omitted from the module add command then the default version will be selected. For example, the following command will load the default environment module for gcc, i.e. 4.9.2.
[samuel@prlhpcms01 ~]$ module add gcc
Choosing the default software version can be useful when you are not restricted to a particular version and you always want to use the most recent version. This is possible because the default version will be updated whenever new versions of the software are installed.
Find out what paths and other environment variables a module uses
You can use the module show command to list the changes that the module makes to the environment variables. For example
[samuel@prlhpcms01 ~]$ module show gcc4.4.7 ------------------------------------------------------------------- /shared/modulefiles/gcc4.4.7: setenv CC /shared/GCC/bin/gcc setenv GCC /shared/GCC/bin/gcc setenv FC /shared/GCC/bin/gfortran setenv F77 /shared/GCC/bin/gfortran setenv F90 /shared/GCC/bin/gfortran prepend-path PATH /shared/GCC/include prepend-path PATH /shared/GCC/bin prepend-path MANPATH /shared/GCC/man prepend-path LD_LIBRARY_PATH /shared/GCC/lib -------------------------------------------------------------------
[samuel@prlhpcms01 ~]$ module show intel-compiler ------------------------------------------------------------------- /etc/modulefiles/intel-compiler: prepend-path PATH /shared/intel/composer_xe_2015.2.164/bin/intel64 prepend-path LD_LIBRARY_PATH /shared/intel/composer_xe_2015.2.164/lib/intel64 prepend-path LD_LIBRARY_PATH /shared/intel/composer_xe_2015.2.164/compiler/lib/intel64 prepend-path LIBRARY_PATH /shared/intel/composer_xe_2015.2.164/compiler/lib/intel64 prepend-path MANPATH /shared/intel/composer_xe_2015.2.164/man/en_US append-path MANPATH /usr/local/share/man append-path MANPATH /usr/share/man/overrides append-path MANPATH /usr/share/man prepend-path NLSPATH /shared/intel/composer_xe_2015.2.164/compiler/lib/intel64/locale/%l_%t/%N append-path INCLUDE /shared/intel/ipp/include append-path IPPROOT /shared/intel/ipp append-path LD_LIBRARY_PATH /shared/intel/ipp/lib/intel64 append-path NLSPATH /shared/intel/ipp/lib/intel64/locale/%l_%t/%N append-path LIBRARY_PATH /shared/intel/ipp/lib/intel64 append-path TBBROOT /shared/intel/tbb append-path CPATH /shared/intel/tbb/include append-path LD_LIBRARY_PATH /shared/intel/tbb/lib/intel64 append-path LIBRARY_PATH /shared/intel/tbb/lib/intel64 append-path INCLUDE /shared/intel/mkl/include append-path NLSPATH /shared/intel/composer_xe_2015.2.164/mkl/lib/intel64/locale/%l_%t/%N append-path MKLROOT /shared/intel/composer_xe_2015.2.164/mkl prepend-path CPATH /shared/intel/composer_xe_2015.2.164/mkl/include append-path LD_LIBRARY_PATH /shared/intel/composer_xe_2015.2.164/mkl/lib/intel64 append-path FPATH /shared/intel/composer_xe_2015.2.164/mkl/include append-path LIBRARY_PATH /shared/intel/composer_xe_2015.2.164/mkl/lib/intel64 append-path NLSPATH /shared/intel/composer_xe_2015.2.164/debugger/gdb/intel64/share/locale/%l_%t setenv INTEL_LICENSE_FILE /shared/intel/flexlm/server.lic -------------------------------------------------------------------
Why does the cluster forget my choice of modules?
Environment modules only affect the current session in which they are loaded. Loading a module in one SSH session will not affect any other SSH session or even any jobs submitted from that session. Modules must be loaded in every session where they will be used.
Invoke your module commands automatically
Every time you log into the cluster or submit a job the commands in your $HOME/.bashrc file will be executed. It can be very convenient to save your environment module commands in this file. Open $HOME/.bashrc in a text editor and add your environment module commands to the section corresponding to the cluster you are using. For example:
[samuel@prlhpcms01 ~]$ cat .bashrc # .bashrc # User specific aliases and functions alias rm='rm -i' alias cp='cp -i' alias mv='mv -i' # Source global definitions if [ -f /etc/bashrc ]; then . /etc/bashrc fi module load intel-compiler module load impi5.0 module load gcc4.4.7
Module commands can also be included in job scripts. This approach is useful for preserving the required environment for each job without having to change your $HOME/.bashrc. For example:
#!/bin/bash #BSUB -J jobname #BSUB -oo outfile.%J #BSUB -eo errorfile.%J module purge module add gcc4.9.2 ./myprog