TARA Getting started [EN]
- Viwan Jarerattanachat
Summary for experienced HPC user
To access TARA, ssh to
[username]@tara.nstda.or.th, where[username]is your registered TARA username.Each user is allocated with 50 GB home directory (
$HOME).The project directory(s) (
/tarafs/data/project/[projxxxx]-[short-name]/) has 200 GB of space to be shared among all users in the project; where[projxxxx]is project id and[short-name]is its short name.There is a scratch directory (
/tarafs/scratch/[projxxxx]-[short-name]/) which is a fast read & write via read-intensive SSD. For temporary file only, files will be removed after 30 days of last access.Software are managed as modules using Environment module system (Lmod).
TARA usage is subjected to fairshare policy and managed by Slurm job scheduler. Examples of Slurm batch script for specific applications can be found at การติดตั้งและการงานใช้ Application.
Cheat Sheet
Download this document as a PDF
TARA System Overview
An up-to-date overview of TARA system can be found at Resources – ThaiSC.
Workflow
This is a suggested workflow for users when using TARA
- 1 1. Logging in to TARA
- 2 2. Transferring file to TARA
- 3 3. Checking Software availability
- 4 4. Software/Application installation (if applicable)
- 5 5. Running Jobs
- 6 6. Checking job status
- 6.1 Check job status
- 6.2 Cancel job
- 7 7. Transferring files to local machine
- 8 8. Checking resource usage and remaining quota
- 8.1 Service Unit
- 8.2 Disk quota
- 9 9. Removing unnecessary files
1. Logging in to TARA
The connection through VPN is required to access TARA. The instruction how to setup and connect to the VPN can be found in your email.
Accessing TARA via ssh protocol, user can use SSH client such as PuTTy, MobaXterm, etc.. In this document, we demonstrate how to log in to TARA using MobaXterm.
More information about MobaXterm is available at MobaXterm.
For macOS or Linux, you can use ssh -Y [username]@tara.nstda.or.th command in the Terminal (YouTube : Access to TARA (Linux)).
To access TARA, you can follow the steps below (YouTube : Access to TARA (Windows)).
1. Open MobaXterm, a SSH client program
2. Click on Session button to initiate the session page
3. Click on the SSH button to use SSH protocol and set the configuration as the following details.
Remote host = tara.nstda.or.th
Specify username = [username]
Port = 22
Where [username] is your TARA username.
4. Click OK button to accept the settings. The SSH terminal session will start and prompt for a password. Enter your TARA password.
If this is your first time logging in using MobaXterm, MobaXterm will ask whether you want to save your password.
For a first time login to TARA, you will be asked to set your new password.
enter your Current password
enter your New password
re-enter your New password
5. The connection between SSH session and TARA is now established. You will see the welcome message. All commands executed in this session are run on TARA.
2. Transferring file to TARA
Each user is allocated with 50 GB home directory ($HOME) and project directory(s) (/tarafs/data/project/proj[xxxx]-[short-name]/) of 200 GB to be shared among all users in the project.
Note: [projxxxx] is project id and [short-name] is your project short name.
To transfer files to TARA with Graphic User Interface (GUI), use can use FTP client such as FileZilla, WinSCP, MobaXterm, etc..
For macOS or Linux, you can use scp command in the Terminal (YouTube : File Transfer (Linux or macOS)).
In this document, we demonstrate this process using MobaXterm (YouTube : File Transfer (Windows)).
1. Open MobaXterm and click at Session button to open Session page.
2. Click at SFTP button to use SFTP protocol and set the configuration as shown.
Remote host = tara.nstda.or.th
Username = [username]
Port = 22
3. Click OK, then SFTP session will start. Now, you can drag and drop files to transfer files between TARA and your local machine.
3. Checking Software availability
TARA has reasonable number of software installed. These software are managed by module framework using Environment module system (Lmod). (YouTube : Software : Module System)
The commands in this part are executed in SSH session. If you transferring file using SFTP session, switch to the SSH session by click at SSH tap with the key icon.
To list all available software
Command : module avail
Example :
[hpcuser@tara-frontend-1 ~]$ module avail
----------------------------------------------------------------------------------------------------- /tarafs/utils/modules/modules/all -----------------------------------------------------------------------------------------------------
ANSYS/19.3 EasyBuild/4.1.0 Mako/1.1.0-GCCcore-8.3.0 ScaLAPACK/2.0.2-gompic-2019b (D) fosscuda/2019b
Advisor/2019_update5 FFTW/3.3.8-gompi-2019b Meson/0.51.2-GCCcore-8.3.0-Python-3.7.4 Singularity/3.3.0 gcccuda/2019b
Autoconf/2.69-GCCcore-8.3.0 FFTW/3.3.8-gompic-2019b (D) NASM/2.14.02-GCCcore-8.3.0 Singularity/3.4.2 (D) gnuplot/5.2.8-GCCcore-8.3.0
Autoconf/2.69 (D) FriBidi/1.0.5-GCCcore-8.3.0 NLopt/2.6.1-GCCcore-8.3.0 Szip/2.1.1-GCCcore-8.3.0 gompi/2019b
Automake/1.15.1-GCCcore-8.3.0 GATK/4.1.4.1-GCCcore-8.3.0-Java-11 NSPR/4.21-GCCcore-8.3.0 Tcl/8.6.9-GCCcore-8.3.0 gompic/2019b
...To load software
Command : module load [software]
Example :
[hpcuser@tara-frontend-1 ~]$ module load Python/3.7.4-GCCcore-8.3.0
[hpcuser@tara-frontend-1 ~]$ python --version
Python 3.7.4To list all loaded software
Command : module list
Example :
[hpcuser@tara-frontend-1 ~]$ module list
Currently Loaded Modules:
1) GCCcore/8.3.0 7) Tcl/8.6.9-GCCcore-8.3.0
2) zlib/1.2.11-GCCcore-8.3.0 (H) 8) SQLite/3.29.0-GCCcore-8.3.0
3) binutils/2.32-GCCcore-8.3.0 9) XZ/5.2.4-GCCcore-8.3.0
4) bzip2/1.0.8-GCCcore-8.3.0 10) GMP/6.1.2-GCCcore-8.3.0
5) ncurses/6.1-GCCcore-8.3.0 (H) 11) libffi/3.2.1-GCCcore-8.3.0
(H)
6) libreadline/8.0-GCCcore-8.3.0 (H) 12) Python/3.7.4-GCCcore-8.3.0
Where:
H: Hidden ModuleTo unload all loaded software
Command : module purge
Example :
For more information about LMOD system please see การใช้งาน module ใน TARA Cluster
4. Software/Application installation (if applicable)
If TARA does not have the software(s) you need installed, you can install them by yourself in either your home directory or your project directory. Please make sure that you have the software license and the license term allows to run the software on TARA.
We support EasyBuild (https://easybuild.io/), see การใช้งาน local module ใน TARA Cluster for the step by step instruction to install software as a local module. You can also check การติดตั้งและการงานใช้ Application for some specific software installation/setup instruction.
5. Running Jobs
Check TARA HPC status
Before submitting your job to run on TARA, you can check available computational resources or status of partitions by using sinfo command.
Example :
where
Item | Description |
|---|---|
PARTITION | partition name |
AVAIL | partition status (up : working, down : not working) |
TIMELIMIT | maximum time that a job can run on this partition |
NODES | number of nodes in the partition |
STATE | partition availability (idle : free, alloc : busy, mix : some core(s) in node is free) |
NODELIST | list of nodes in the partition |
Job submission
TARA usage is subjected to Fair-share policy and managed by Slurm job scheduler. To submit a job follows the following steps.
1. Create bash script using vi sbatch_script.sh (or gedit sbatch_script.sh , nano sbatch_script.sh)
2. Specify the details in the bash script according to the template provided below.
3. Use sbatch command followed by your bash script name to submit your job to SLURM
6. Checking job status
Check job status
To check your job status, using myqueue command.
Example :
Item | Description |
|---|---|
JOBID | ID of the specific job |
PARTITION | partition which job is running/queuing on |
NAME | name of the specific job |
USER | user who owns/run the job |
ST | Job status (PD : pending, R : running) |
TIME | elapsed time of the running job |
NODE | the number of node used in the run |
NODELIST(REASON) | list of node used to run (reason why the job is pending) see สถานะ Pending (PD) for more information |
Cancel job
To cancel the job in the queue, using scancel [JOBID] command
Example : scancel 140215
7. Transferring files to local machine
To transfer files from TARA to your local machine, you can use FTP client such as FileZilla, WinSCP, MobaXterm, etc.. In this document, we demonstrate this process using MobaXterm.
1. Open SFTP session in the MobaXterm (see session 2 Transferring files to TARA for instruction)
2. Drag file/directory from TARA and drop at your local machine
8. Checking resource usage and remaining quota
Service Unit
You can see the usage quantity and the remaining amount of the service unit (SU) by using sbalance. You can see SU usage by users by using -d flag (i.e. sbalance -d)
Disk quota
You can see the amount of space used and remaining using myquota
Example :
9. Removing unnecessary files
You can free up your disk space by removing unnecessary files. You can remove file by using rm [file] command and remove directory by using rm -r [directory] command.
Example :
ls -l is the command to list files or directories in the current location