This article will guide you to run the Jupyter Notebook via Miniconda Mamba on the LANTA HPC system, which requires ssh tunneling to the LANTA HPC. An overview of the content can be found in the table of contents below for immediate visualization of the interesting parts.

Table of Contents

Creating an environment to run the Jupyter Notebook

Load

...

Mamba module

Use the ml av MinicondaMamba command to see which version of Miniconda Mamba is available on the LANTA HPC system.
Use the ml Miniconda3Mamba/xx.xx.x command to load the Miniconda Mamba version that you want to use. If you don't specify a version, the default version (D) is loaded, which is Miniconda3Mamba/23.311.10-0.

Code Block

username@lanta:~> ml av MinicondaMamba
---------------------- /lustrefs/disk/modules/easybuild/modules/all -----------------------
   Miniconda3Mamba/2223.11.1-1    Miniconda3/23.3.1-0-0 (D)

Use "module spider" to find all possible modules and extensions.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".
username@lanta:~> ml Miniconda3Mamba/2223.11.10-10

Create the environment

Use the conda create -n myenv python=3.9 commands to create the conda environment with myenv name and a specific version of python.
Use the conda activate myenv to activate the myenv environment.

Code Block

username@lanta:~> conda create -n myenv python=3.9
Channels:
 - conda-forge
Platform: linux-64
Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /your directory/envs/myenv

  added / updated specs:
    - python=3.9


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    capython-certificates-20233.01.10 |9.19              h06a4308|h0755675_0_cpython        22.7 MB 120 KBconda-forge
    certifiwheel-20220.1243.70           |   py39h06a4308_0 |     pyhd8ed1ab_1          15057 KB  conda-forge
    ------------------------------------------------------------
                                           Total:         270 KB22.8 MB

The following NEW packages will be INSTALLED:

  _libgcc_mutex      pkgs/mainconda-forge/linux-64::_libgcc_mutex-0.1-mainconda_forge
  _openmp_mutex      pkgsconda-forge/main/linux-64::_openmp_mutex-4.5.1-12_gnu
  bzip2   ca-certificates           pkgs/mainconda-forge/linux-64::cabzip2-certificates-20231.010.108-h06a4308hd590300_05
  certifica-certificates            pkgs/mainconda-forge/linux-64::certifica-certificates-20222024.122.72-py39h06a4308hbcca054_0
  ld_impl_linux-64   pkgs/mainconda-forge/linux-64::ld_impl_linux-64-2.3840-h1181459h41732ed_10
  libffi             pkgs/mainconda-forge/linux-64::libffi-3.4.2-h6a678d5h7f98852_65
  libgcc-ng          pkgsconda-forge/main/linux-64::libgcc-ng-1113.2.0-h1234567h807b86a_15
  libgomp            pkgs/mainconda-forge/linux-64::libgomp-1113.2.0-h1234567_1h807b86a_5
  libnsl      libstdcxx-ng       pkgsconda-forge/main/linux-64::libstdcxx-nglibnsl-11.2.0.1-h1234567hd590300_10
  ncurseslibsqlite            pkgs/mainconda-forge/linux-64::ncurseslibsqlite-63.445.2-h6a678d5h2797004_0
  openssllibuuid            pkgs/mainconda-forge/linux-64::openssllibuuid-12.38.1.1t-h7f8727eh0b41bf4_0
  piplibxcrypt                pkgs/main/conda-forge/linux-64::piplibxcrypt-224.34.136-py39h06a4308hd590300_01
  pythonlibzlib             pkgs/mainconda-forge/linux-64::pythonlibzlib-31.92.1613-h7a1cb2ahd590300_05
  readlinencurses            pkgsconda-forge/main/linux-64::readlinencurses-86.4.220240210-h5eee18bh59595ed_0
  setuptoolsopenssl            pkgs/mainconda-forge/linux-64::setuptoolsopenssl-653.62.31-py39h06a4308_0hd590300_1
  pip   sqlite             pkgs/main/linux-64conda-forge/noarch::sqlitepip-324.40.10-h5082296pyhd8ed1ab_0
  tkpython                 pkgs/mainconda-forge/linux-64::tkpython-83.69.1219-h1ccaba5h0755675_0_cpython
  readline          tzdata  conda-forge/linux-64::readline-8.2-h8228510_1
  setuptools         pkgs/mainconda-forge/noarch::tzdata-2022g-h04d1e81_0setuptools-69.2.0-pyhd8ed1ab_0
  tk   wheel              pkgs/mainconda-forge/linux-64::wheeltk-08.386.413-py39h06a4308_0
  xznoxft_h4845f30_101
  tzdata             conda-forge/noarch::tzdata-2024a-h0c530f3_0
  wheel              pkgs/main/linux-64conda-forge/noarch::xzwheel-50.243.100-h5eee18bpyhd8ed1ab_1
  zlibxz                 pkgs/mainconda-forge/linux-64::zlibxz-15.2.136-h5eee18bh166bdaf_0

Proceed ([y]/n)? y
...
username@lanta:~> conda activate myenv
(myenv) username@lanta:~>

Install Jupyter and other packages in the myenv environment

Use the conda install jupyter command to install jupyter in the myenv environment.
If you want to install other packages such as TensorFlow-GPUPyTorch, you can use the conda pip3 install -c anaconda tensorflow-gputorch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118 command to install TensorFlow-GPU PyTorch in the myenv environment.

Code Block

(myenv) username@lanta:~> conda install jupyter
...
(myenv) username@lanta:~> condapip3 install torch -ctorchvision anacondatorchaudio tensorflow-gpu--index-url https://download.pytorch.org/whl/cu118
...

Running Jupyter Notebook via ssh tunneling

Example of Slurm script for running Jupyter Notebook

Code Block

#!/bin/bash
#SBATCH -p gpu                          # Specify partition [Compute/Memory/GPU]
#SBATCH -N 1 -c 16                      # Specify number of nodes and processors per task
#SBATCH --gpus-per-task=1               # Specify the number of GPUs
#SBATCH --ntasks-per-node=4             # Specify tasks per node
#SBATCH -t 2:00:00                      # Specify maximum time limit (hour: minute: second)
#SBATCH -A ltxxxxxx                     # Specify project name
#SBATCH -J JOBNAME                      # Specify job name

module load Miniconda3Mamba/2223.11.1-10-0             # Load the module that you want to use
conda activate myenv                    # Activate your environment

port=$(shuf -i 6000-9999 -n 1)
USER=$(whoami)
node=$(hostname -s)

# jupyter notebookng instructions to the output file
echo -e "

    Jupyter server is running on: $(hostname)
    Job starts at: $(date)

    Copy/Paste the following command into your local terminal 
    --------------------------------------------------------------------
    ssh -L $port:$node:$port $USER@lanta.nstda.or.th -i id_rsa
    --------------------------------------------------------------------

    Open a browser on your local machine with the following address
    --------------------------------------------------------------------
    http://localhost:${port}/?token=XXXXXXXX (see your token below)
    --------------------------------------------------------------------
    "

# start a cluster instance and launch jupyter server

unset XDG_RUNTIME_DIR
if [ "$SLURM_JOBTMP" != "" ]; then
    export XDG_RUNTIME_DIR=$SLURM_JOBTMP
fi

jupyter notebook --no-browser --port $port --notebook-dir=$(pwd) --ip=$node

Running Jupyter Notebook with Slurm script

There are 3 steps to run Jupyter Notebook on LANTA HPC.

...

Code Block
http://127.0.0.1:8714/?token=2923d6fab4ef109f30e63a77014e632eed3fd2a5fa561929

...

Shutting down the Jupyter Notebook

When you’re done with the Jupyter Notebook session, you can start the shutdown process by closing the browser and terminal on your local machine. Then, you must cancel your job in the Slurm system of the LANTA HPC with the scancel JOBID command.

Code Block
username@lanta:~> scancel xxxxx

...

Versions Compared

Old Version 44

New Version 45

Key

Creating an environment to run the Jupyter Notebook

Load

Mamba module

Create the environment

Install Jupyter and other packages in the myenv environment

Running Jupyter Notebook via ssh tunneling

Example of Slurm script for running Jupyter Notebook

Running Jupyter Notebook with Slurm script

Shutting down the Jupyter Notebook

Related articles

Page Comparison

Versions Compared

Old Version 44

New Version 45

Key

Creating an environment to run the Jupyter Notebook

Load

Mamba module

Create the environment

Install Jupyter and other packages in the myenv environment

Running Jupyter Notebook via ssh tunneling

Example of Slurm script for running Jupyter Notebook

Running Jupyter Notebook with Slurm script

Shutting down the Jupyter Notebook

Related articles