Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.nyc-ai.app/llms.txt

Use this file to discover all available pages before exploring further.

CSI HPCC ships dozens of compilers, MPI libraries, math libraries, language runtimes, and applications. Rather than installing software system-wide in conflicting versions, the center uses environment modules. Each package is described by a module file that sets PATH, LD_LIBRARY_PATH, and related variables when loaded, and cleanly reverses those changes when unloaded.
HPCC uses Lmod (Lua-based, supports dependency hierarchies) to manage application environments. The commands below target Lmod; always check the login banner and module avail on the system you’re using.

​
Core commands

CommandWhat it does
module availList all modules visible with the currently loaded prerequisites.
module spider <name>Search all modules (including those behind dependency chains) and show how to load them. LMOD only.
module whatis <name>Print a one-line description of a module.
module load <name>Load a module into the current shell.
module unload <name>Unload a previously loaded module.
module listShow everything currently loaded.
module purgeUnload everything.
module swap <old> <new>Replace one loaded module with another.

​
A typical workflow

Inspect what’s loaded by default, then change it for your job: 
module list
# Currently Loaded Modules (example):
#   1) gcc/12.3.0    2) openmpi/4.1.5

module unload openmpi/4.1.5
module load openmpi/4.1.6
Search for a package and load a specific version: 
module spider python
module load Python/3.10.4
python3 -c "import sys; print(sys.version)"
Inside a SLURM script, load every module you need before launching your program, so compute nodes start with the right environment: 
#!/bin/bash
#SBATCH --job-name=python_job
#SBATCH --nodes=1
#SBATCH --ntasks=1
#SBATCH --time=00:30:00

module purge
module load Python/3.10.4
module load NETCDF/4.2.0      # example: NETCDF compiled with PGI

cd $SLURM_SUBMIT_DIR
python3 analysis.py
Starting a script with module purge makes it reproducible. It doesn’t matter what was loaded in the submitter’s interactive shell.

​
What’s available

The module tree covers the usual HPC stack:
  • Compilers: GNU, Intel / oneAPI, NVIDIA HPC SDK, and other system-specific toolchains.
  • MPI: OpenMPI, Intel MPI, and other system-specific MPI builds.
  • Math / scientific libraries: FFTW, NetCDF, HDF5, BLAS/LAPACK implementations.
  • Languages: Python (multiple versions), Julia, R, Perl.
  • Applications: MATLAB, and domain-specific codes as installed by HPCC staff.
The exact versions available evolve over time. Always check with module avail or module spider on the system you’re using, not a memorized list.

​
Requesting new software

If a package you need isn’t installed, email the HPC Helpline with:
  • The software name and version you’d like.
  • A link to its install instructions or source.
  • Why you need it (a one-line justification is fine).
For personal Python or Julia environments, you can typically manage packages yourself inside your home directory. Use python -m venv, conda env create, or Julia’s package manager.

​
Compiling your own code

See the Program Compilation page on the HPCC Wiki for compiler flags and linker hints that work cleanly with the module tree.

​
Next steps

Submit a job

Annotated SLURM templates that show where module load fits.

Storage & quotas

Where to install your virtualenvs and Conda environments.