Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

This article will guide you to run the Jupyter Notebook via Miniconda on the LANTA HPC system, which requires ssh tunneling to the LANTA HPC. An overview of the content can be found in the table of contents below for immediate visualization of the interesting parts.

Table of Contents

Creating an environment to run the Jupyter Notebook

Load Miniconda module

  1. Use the ml av Miniconda command to see which version of Miniconda is available on the LANTA HPC system.

  2. Use the ml Miniconda3/xx.xx.x command to load the Miniconda version that you want to use. If you don't specify a version, the default version (D) is loaded, which is Miniconda3/23.3.1-0.

...

Code Block
(myenv) username@lanta:~> conda install jupyter
...
(myenv) username@lanta:~> conda install -c anaconda tensorflow-gpu
...

Running Jupyter Notebook via ssh tunneling

Example of Slurm script for running Jupyter Notebook

Code Block
#!/bin/bash
#SBATCH -p gpu                          # Specify partition [Compute/Memory/GPU]
#SBATCH -N 1 -c 16                      # Specify number of nodes and processors per task
#SBATCH --gpus-per-task=1               # Specify the number of GPUs
#SBATCH --ntasks-per-node=4             # Specify tasks per node
#SBATCH -t 2:00:00                      # Specify maximum time limit (hour: minute: second)
#SBATCH -A ltxxxxxx                     # Specify project name
#SBATCH -J JOBNAME                      # Specify job name

module purge                            # Unload all modules
module load Miniconda3/22.11.1-1        # Load the module that you want to use
conda activate myenv                    # Activate your environment

port=$(shuf -i 6000-9999 -n 1)
USER=$(whoami)
node=$(hostname -s)

# jupyter notebookng instructions to the output file
echo -e "

    Jupyter server is running on: $(hostname)
    Job starts at: $(date)

    Copy/Paste the following command into your local terminal 
    --------------------------------------------------------------------
    ssh -L $port:$node:$port $USER@lanta.nstda.or.th -i id_rsa
    --------------------------------------------------------------------

    Open a browser on your local machine with the following address
    --------------------------------------------------------------------
    http://localhost:${port}/?token=XXXXXXXX (see your token below)
    --------------------------------------------------------------------
    "

# start a cluster instance and launch jupyter server

unset XDG_RUNTIME_DIR
if [ "$SLURM_JOBTMP" != "" ]; then
    export XDG_RUNTIME_DIR=$SLURM_JOBTMP
fi

jupyter notebook --no-browser --port $port --notebook-dir=$(pwd) --ip=$node

...

Code Block
http://127.0.0.1:8714/?token=2923d6fab4ef109f30e63a77014e632eed3fd2a5fa561929

...

Shutting down the Jupyter Notebook

When you’re done with the Jupyter Notebook session, you can start the shutdown process by closing the browser and terminal on your local machine. Then, you must cancel your job in the Slurm system of the LANTA HPC with the scancel JOBID command.

Code Block
username@lanta:~> scancel xxxxx

...

Related articles

Filter by label (Content by label)
showLabelsfalse
max5
spacescom.atlassian.confluence.content.render.xhtml.model.resource.identifiers.SpaceResourceIdentifier@48ae393
showSpacefalse
sortmodified
showSpacetypefalsepage
reversetrue
typelabelspagesingularity python container
cqllabel in ( "env" , "python-script" , "jupyter-apptainer" ) and space = currentSpace ( )labelssingularity python container