This article will guide you to run the Jupyter Notebook via Mamba on the LANTA HPC system, which requires ssh tunneling to the LANTA HPC. An overview of the content can be found in the table of contents below for immediate visualization of the interesting parts.
| Table of Contents |
|---|
Creating an environment to run the Jupyter Notebook
Load Mamba module
Use the
ml av Mambacommand to see which version of Mamba is available on the LANTA HPC system.Use the
ml Mamba/xx.xx.xcommand to load the Mamba version that you want to use. If you don't specify a version, the default version (D) is loaded, which is Mamba/23.11.0-0.
...
| Code Block |
|---|
(myenv) username@lanta:~> conda install jupyter
...
(myenv) username@lanta:~> pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
... |
Running Jupyter Notebook via ssh tunneling
myenv) username@lanta:~> pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118
... |
Running Jupyter Notebook via ssh tunneling
Example of Slurm script for running Jupyter Notebook on Compute node
| Code Block |
|---|
#!/bin/bash
#SBATCH -p compute # Specify partition [Compute/Memory/GPU]
#SBATCH -N 1 -c 128 # Specify number of nodes and processors per task
#SBATCH --ntasks-per-node=1 # Specify tasks per node
#SBATCH -t 2:00:00 # Specify maximum time limit (hour: minute: second)
#SBATCH -A ltxxxxxx # Specify project name
#SBATCH -J JOBNAME # Specify job name
module load Mamba/23.11.0-0 # Load the module that you want to use
conda activate myenv # Activate your environment
port=$(shuf -i 6000-9999 -n 1)
USER=$(whoami)
node=$(hostname -s)
# jupyter notebookng instructions to the output file
echo -e "
Jupyter server is running on: $(hostname)
Job starts at: $(date)
Copy/Paste the following command into your local terminal
--------------------------------------------------------------------
ssh -L $port:$node:$port $USER@lanta.nstda.or.th -i id_rsa
--------------------------------------------------------------------
Open a browser on your local machine with the following address
--------------------------------------------------------------------
http://localhost:${port}/?token=XXXXXXXX (see your token below)
--------------------------------------------------------------------
"
# start a cluster instance and launch jupyter server
unset XDG_RUNTIME_DIR
if [ "$SLURM_JOBTMP" != "" ]; then
export XDG_RUNTIME_DIR=$SLURM_JOBTMP
fi
jupyter notebook --no-browser --port $port --notebook-dir=$(pwd) --ip=$node |
Example of Slurm script for running Jupyter Notebook on GPU node
| Code Block |
|---|
#!/bin/bash
#SBATCH -p gpu # Specify partition [Compute/Memory/GPU]
#SBATCH -N 1 -c 16 # Specify number of nodes and processors per task
#SBATCH --gpus-per-task=1 # Specify the number of GPUs
#SBATCH --ntasks-per-node=4 # Specify tasks per node
#SBATCH -t 2:00:00 # Specify maximum time limit (hour: minute: second)
#SBATCH -A ltxxxxxx # Specify project name
#SBATCH -J JOBNAME # Specify job name
module load Mamba/23.11.0-0 # Load the module that you want to use
conda activate myenv # Activate your environment
port=$(shuf -i 6000-9999 -n 1)
USER=$(whoami)
node=$(hostname -s)
# jupyter notebookng instructions to the output file
echo -e "
Jupyter server is running on: $(hostname)
Job starts at: $(date)
Copy/Paste the following command into your local terminal
--------------------------------------------------------------------
ssh -L $port:$node:$port $USER@lanta.nstda.or.th -i id_rsa
--------------------------------------------------------------------
Open a browser on your local machine with the following address
--------------------------------------------------------------------
http://localhost:${port}/?token=XXXXXXXX (see your token below)
--------------------------------------------------------------------
"
# start a cluster instance and launch jupyter server
unset XDG_RUNTIME_DIR
if [ "$SLURM_JOBTMP" != "" ]; then
export XDG_RUNTIME_DIR=$SLURM_JOBTMP
fi
jupyter notebook --no-browser --port $port --notebook-dir=$(pwd) --ip=$node |
...
2. Copy/Paste the following command into your local terminal for ssh tunneling to the LANTA HPC
| Code Block |
|---|
ssh -L 871460151:x1000c2s0b0n0x1001c7s7b0n0:871460151 username@lanta.nstda.or.th -i id_rsa |
...
| Code Block |
|---|
http://127.0.0.1:871460151/tree?token=2923d6fab4ef109f30e63a77014e632eed3fd2a5fa56192913acf46197090432de43db8f65d8651361ad319861bd04f5 |
...
Shutting down the Jupyter Notebook
When you’re done with the Jupyter Notebook session, you can start the shutdown process by closing the browser and terminal on your local machine. Then, you must cancel your job in the Slurm system of the LANTA HPC with the scancel JOBID command.
| Code Block |
|---|
username@lanta:~> scancel xxxxx |
...
Related articles
| Filter by label (Content by label) | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
...