This article will guide you to run the Jupyter Notebook via Mamba on the LANTA HPC system, which requires ssh tunneling to the LANTA HPC. An overview of the content can be found in the table of contents below for immediate visualization of the interesting parts.
| Table of Contents |
|---|
Creating an environment to run the Jupyter Notebook
Load Mamba module
Use the
ml av Mambacommand to see which version of Mamba is available on the LANTA HPC system.Use the
ml Mamba/xx.xx.xcommand to load the Mamba version that you want to use. If you don't specify a version, the default version (D) is loaded, which is Mamba/23.11.0-0.
...
| Code Block |
|---|
(myenv) username@lanta:~> conda install jupyter ... (myenv) username@lanta:~> pip3 install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu118 ... |
Running Jupyter Notebook via ssh tunneling
Example of Slurm script for running Jupyter Notebook
...
| Code Block |
|---|
username@lanta:~> sbatch script.sh
username@lanta:~> cat slurm-xxxxx.out
Jupyter server is running on: x1000c2s0b0n0x1001c7s7b0n0
Job starts at: FriThu 2428 FebMar 20232024 0911:4700:0953 AMPM +07
Copy/Paste the following command into your local terminal
--------------------------------------------------------------------
ssh -L 871460151:x1000c2s0b0n0x1001c7s7b0n0:871460151 username@lanta.nstda.or.th -i id_rsa
--------------------------------------------------------------------
Open a browser on your local machine with the following address
--------------------------------------------------------------------
http://localhost:871460151/?token=XXXXXXXX (see your token below)
--------------------------------------------------------------------
[W 09:47:12.019 NotebookApp] Loading JupyterLab as a classic notebook (v6) extension.
[W 2023-02-24 09:47:12.022 LabApp] 'port' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
[W 2023-02-24 09:47:12.022 LabApp] 'notebook_dir' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
[W 2023-02-24 09:47:12.022 LabApp] 'ip' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
[W 2023-02-24 09:47:12.022 LabApp] 'ip' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
[W 2023-02-24 09:47:12.022 LabApp] 'ip' has moved from NotebookApp to ServerApp. This config will be passed to ServerApp. Be sure to update your config before our next release.
[I 2023-02-24 09:47:12.025 LabApp] JupyterLab extension loaded from /lustrefs/disk/modules/easybuild/software/Miniconda3/22.11.1-1/envs/tensorflow-2.6.0/lib/python3.9/site-packages/jupyterlab
[I 2023-02-24 09:47:12.025 LabApp] JupyterLab application directory is /lustrefs/disk/modules/easybuild/software/Miniconda3/22.11.1-1/envs/tensorflow-2.6.0/share/jupyter/lab
[I 09:47:12.028 NotebookAppI 2024-03-28 23:01:07.926 ServerApp] Extension package jupyter_lsp took 0.4361s to import
[I 2024-03-28 23:01:09.498 ServerApp] jupyter_lsp | extension was successfully linked.
[I 2024-03-28 23:01:09.503 ServerApp] jupyter_server_terminals | extension was successfully linked.
[I 2024-03-28 23:01:09.508 ServerApp] jupyterlab | extension was successfully linked.
[I 2024-03-28 23:01:09.534 ServerApp] notebook | extension was successfully linked.
[W 2024-03-28 23:01:12.375 ServerApp] jupyter_nbextensions_configurator | error adding extension (enabled: True): The module 'jupyter_nbextensions_configurator' could not be found (No module named 'jupyter_nbextensions_configurator'). Are you sure the extension is installed?
Traceback (most recent call last):
File "/lustrefs/disk/modules/easybuild/software/Mamba/23.11.0-0/envs/tensorflow-2.12.1/lib/python3.10/site-packages/jupyter_server/extension/manager.py", line 322, in add_extension
extpkg = ExtensionPackage(name=extension_name, enabled=enabled)
File "/lustrefs/disk/modules/easybuild/software/Mamba/23.11.0-0/envs/tensorflow-2.12.1/lib/python3.10/site-packages/jupyter_server/extension/manager.py", line 186, in __init__
self._load_metadata()
File "/lustrefs/disk/modules/easybuild/software/Mamba/23.11.0-0/envs/tensorflow-2.12.1/lib/python3.10/site-packages/jupyter_server/extension/manager.py", line 201, in _load_metadata
raise ExtensionModuleNotFound(msg) from None
jupyter_server.extension.utils.ExtensionModuleNotFound: The module 'jupyter_nbextensions_configurator' could not be found (No module named 'jupyter_nbextensions_configurator'). Are you sure the extension is installed?
[I 2024-03-28 23:01:12.393 ServerApp] notebook_shim | extension was successfully linked.
[I 2024-03-28 23:01:12.534 ServerApp] notebook_shim | extension was successfully loaded.
[I 2024-03-28 23:01:12.536 ServerApp] jupyter_lsp | extension was successfully loaded.
[I 2024-03-28 23:01:12.537 ServerApp] jupyter_server_terminals | extension was successfully loaded.
[I 2024-03-28 23:01:12.549 LabApp] JupyterLab extension loaded from /lustrefs/disk/modules/easybuild/software/Mamba/23.11.0-0/envs/tensorflow-2.12.1/lib/python3.10/site-packages/jupyterlab
[I 2024-03-28 23:01:12.549 LabApp] JupyterLab application directory is /lustrefs/disk/modules/easybuild/software/Mamba/23.11.0-0/envs/tensorflow-2.12.1/share/jupyter/lab
[I 2024-03-28 23:01:12.550 LabApp] Extension Manager is 'pypi'.
[I 2024-03-28 23:01:12.599 ServerApp] jupyterlab | extension was successfully loaded.
[I 2024-03-28 23:01:12.605 ServerApp] notebook | extension was successfully loaded.
[I 2024-03-28 23:01:12.605 ServerApp] Serving notebooks from local directory: /home/yutthanaywongnon/thaisc/yutthana/Jupyter_ScriptGPU
[I 09:472024-03-28 23:01:12.028605 NotebookAppServerApp] Jupyter NotebookServer 62.513.20 is running at:
[I 09:472024-03-28 23:01:12.028605 NotebookAppServerApp] http://x1000c2s0b0n0x1001c7s7b0n0:871460151/tree?token=2923d6fab4ef109f30e63a77014e632eed3fd2a5fa56192913acf46197090432de43db8f65d8651361ad319861bd04f5
[I 09:47I 2024-03-28 23:01:12.028605 NotebookAppServerApp] or http://127.0.0.1:871460151/tree?token=2923d6fab4ef109f30e63a77014e632eed3fd2a5fa56192913acf46197090432de43db8f65d8651361ad319861bd04f5
[I 09:472024-03-28 23:01:12.029605 NotebookAppServerApp] Use Control-C to stop this server and shut down all kernels (twice to skip confirmation).
[C 09:472024-03-28 23:01:12.033620 NotebookAppServerApp]
To access the notebookserver, open this file in a browser:
file:///lustrefs/disk/home/yutthanaywongnon/.local/share/jupyter/runtime/nbserverjpserver-4678925384-open.html
Or copy and paste one of these URLs:
http://x1000c2s0b0n0x1001c7s7b0n0:871460151/tree?token=2923d6fab4ef109f30e63a77014e632eed3fd2a5fa56192913acf46197090432de43db8f65d8651361ad319861bd04f5
or http://127.0.0.1:871460151/tree?token=2923d6fab4ef109f30e63a77014e632eed3fd2a5fa56192913acf46197090432de43db8f65d8651361ad319861bd04f5 |
2. Copy/Paste the following command into your local terminal for ssh tunneling to the LANTA HPC
...
| Code Block |
|---|
http://127.0.0.1:8714/?token=2923d6fab4ef109f30e63a77014e632eed3fd2a5fa561929 |
...
Shutting down the Jupyter Notebook
When you’re done with the Jupyter Notebook session, you can start the shutdown process by closing the browser and terminal on your local machine. Then, you must cancel your job in the Slurm system of the LANTA HPC with the scancel JOBID command.
| Code Block |
|---|
username@lanta:~> scancel xxxxx |
...
Related articles
| Filter by label (Content by label) | ||||||||||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
|
...