This article will guide you to run Jupyter Notebook via Miniconda on a LANTA HPC system, which requires ssh tunneling to the LANTA HPC.
Table of Contents |
---|
Creating an environment to run the Jupyter Notebook
Load Miniconda module
Use the
ml av Miniconda
command to see which version of Miniconda is available on the LANTA HPC system.Use the
ml Miniconda3/xx.xx.x
command to load the Miniconda version that you want to use. If you don't specify a version, the default version (D) is loaded, which is Miniconda3/22.11.1-1.
...
Code Block |
---|
(myenv) username@lanta:~> conda install jupyter ... (myenv) username@lanta:~> conda install -c anaconda tensorflow-gpu ... |
Running Jupyter Notebook via ssh tunneling
Example of Slurm script for running Jupyter Notebook
Code Block |
---|
#!/bin/bash #SBATCH -p gpu # Specify partition [Compute/Memory/GPU] #SBATCH -N 1 -c 16 # Specify number of nodes and processors per task #SBATCH --gpus-per-task=1 # Specify the number of GPUs #SBATCH --ntasks-per-node=4 # Specify tasks per node #SBATCH -t 2:00:00 # Specify maximum time limit (hour: minute: second) #SBATCH -A xxxxxxltxxxxxx # Specify project name #SBATCH -J JOBNAME # Specify job name module purge # Unload all modules module load Miniconda3/22.11.1-1 # Load the module that you want to use conda activate myenv # Activate your environment port=$(shuf -i 6000-9999 -n 1) USER=$(whoami) node=$(hostname -s) # #jupyterjupyter notebookng instructions to the output file echo -e " Jupyter server is running on: $(hostname) Job starts at: $(date) Copy/Paste this inthe following command into your local terminal to ssh tunnel with remote -------------------------------------------------------------------- ssh -L $port:$node:$port $USER@lanta.nstda.or.th -i id_rsa -------------------------------------------------------------------- Open a browser on your local machine with the following address -------------------------------------------------------------------- http://localhost:${port}/?token=XXXXXXXX (see your token below) -------------------------------------------------------------------- " ### start a cluster instance and launch jupyter server unset XDG_RUNTIME_DIR if [ "$SLURM_JOBTMP" != "" ]; then export XDG_RUNTIME_DIR=$SLURM_JOBTMP fi jupyter notebook --no-browser --port $port --notebook-dir=$(pwd) --ip=$node |
...
Code Block |
---|
username@lanta:~> sbatch script.sh username@lanta:~> cat slurm-xxxxx.out Jupyter server is running on: x1000c2s0b0n0 Job starts at: Fri 24 Feb 2023 09:47:09 AM +07 Copy/Paste the following thiscommand ininto your local terminal to ssh tunnel with remote -------------------------------------------------------------------- ssh -L 8714:x1000c2s0b0n0:8714 username@lanta.nstda.or.th -i id_rsa -------------------------------------------------------------------- Open a browser on your local machine with the following address -------------------------------------------------------------------- http://localhost:8714/?token=XXXXXXXX (see your token below) -------------------------------------------------------------------- [W 09:47:12.019 NotebookApp] Loading JupyterLab as a classic notebook (v6) extension. [W 2023-02-24 09:47:12.022 LabApp] 'port' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release. [W 2023-02-24 09:47:12.022 LabApp] 'notebook_dir' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release. [W 2023-02-24 09:47:12.022 LabApp] 'ip' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release. [W 2023-02-24 09:47:12.022 LabApp] 'ip' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release. [W 2023-02-24 09:47:12.022 LabApp] 'ip' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release. [I 2023-02-24 09:47:12.025 LabApp] JupyterLab extension loaded from /lustrefs/disk/modules/easybuild/software/Miniconda3/22.11.1-1/envs/tensorflow-2.6.0/lib/python3.9/site-packages/jupyterlab [I 2023-02-24 09:47:12.025 LabApp] JupyterLab application directory is /lustrefs/disk/modules/easybuild/software/Miniconda3/22.11.1-1/envs/tensorflow-2.6.0/share/jupyter/lab [I 09:47:12.028 NotebookApp] Serving notebooks from local directory: /home/yutthana/thaisc/yutthana/Jupyter_Script [I 09:47:12.028 NotebookApp] Jupyter Notebook 6.5.2 is running at: [I 09:47:12.028 NotebookApp] http://x1000c2s0b0n0:8714/?token=2923d6fab4ef109f30e63a77014e632eed3fd2a5fa561929 [I 09:47:12.028 NotebookApp] or http://127.0.0.1:8714/?token=2923d6fab4ef109f30e63a77014e632eed3fd2a5fa561929 [I 09:47:12.029 NotebookApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation). [C 09:47:12.033 NotebookApp] To access the notebook, open this file in a browser: file:///lustrefs/disk/home/yutthana/.local/share/jupyter/runtime/nbserver-46789-open.html Or copy and paste one of these URLs: http://x1000c2s0b0n0:8714/?token=2923d6fab4ef109f30e63a77014e632eed3fd2a5fa561929 or http://127.0.0.1:8714/?token=2923d6fab4ef109f30e63a77014e632eed3fd2a5fa561929 |
...
Code Block |
---|
http://127.0.0.1:8714/?token=2923d6fab4ef109f30e63a77014e632eed3fd2a5fa561929 |
...
Related articles
Filter by label (Content by label) | ||||||||||||||||||
---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
...