TARA Getting started [EN]

Owned by Viwan Jarerattanachat, created
Last updated: Jul 19, 2022
8 min read

Summary for experienced HPC user

  1. To access TARA, ssh to [username]@tara.nstda.or.th, where [username] is your registered TARA username.

  2. Each user is allocated with 50 GB home directory ($HOME).

  3. The project directory(s) (/tarafs/data/project/[projxxxx]-[short-name]/) has 200 GB of space to be shared among all users in the project; where [projxxxx] is project id and [short-name] is its short name.

  4. There is a scratch directory (/tarafs/scratch/[projxxxx]-[short-name]/) which is a fast read & write via read-intensive SSD. For temporary file only, files will  be removed after 30 days of last access.

  5. Software are managed as modules using Environment module system (Lmod).

  6. TARA usage is subjected to fairshare policy and managed by Slurm job scheduler. Examples of Slurm batch script for specific applications can be found at การติดตั้งและการงานใช้ Application.

Cheat Sheet

Download this document as a PDF

TARA System Overview

An up-to-date overview of TARA system can be found at Resources – ThaiSC.

Workflow

This is a suggested workflow for users when using TARA

1. Logging in to TARA

The connection through VPN is required to access TARA. The instruction how to setup and connect to the VPN can be found in your email.

Accessing TARA via ssh protocol, user can use SSH client such as PuTTy, MobaXterm, etc.. In this document, we demonstrate how to log in to TARA using MobaXterm. 

More information about MobaXterm is available at https://mobaxterm.mobatek.net/.

For macOS or Linux, you can use ssh -Y [username]@tara.nstda.or.th command in the Terminal (YouTube : https://youtu.be/kr0WR46UGuc).

To access TARA, you can follow the steps below (YouTube : https://youtu.be/T6earis8Hqo).

1. Open MobaXterm, a SSH client program

2. Click on Session button to initiate the session page

MobaXterm user interface

3. Click on the SSH button to use SSH protocol and set the configuration as the following details.

Remote host = tara.nstda.or.th  

Specify username = [username]  

Port = 22

Where [username] is your TARA username.

The SSH setting

4. Click OK button to accept the settings. The SSH terminal session will start and prompt for a password. Enter your TARA password.

If this is your first time logging in using MobaXterm, MobaXterm will ask whether you want to save your password.

For a first time login to TARA, you will be asked to set your new password.

  1. enter your Current password

  2. enter your New password

  3. re-enter your New password

5. The connection between SSH session and TARA is now established. You will see the welcome message. All commands executed in this session are run on TARA.

2. Transferring file to TARA

Each user is allocated with 50 GB home directory ($HOME) and project directory(s) (/tarafs/data/project/proj[xxxx]-[short-name]/) of 200 GB to be shared among all users in the project.

Note: [projxxxx] is project id and [short-name] is your project short name.

To transfer files to TARA with Graphic User Interface (GUI), use can use FTP client such as FileZilla, WinSCP, MobaXterm, etc..

For macOS or Linux, you can use scp command in the Terminal (YouTube : https://youtu.be/ovQj9dzwE-w).

In this document, we demonstrate this process using MobaXterm (YouTube : https://youtu.be/aSwjYtY1KrA).

1. Open MobaXterm and click at Session button to open Session page.

2. Click at SFTP button to use SFTP protocol and set the configuration as shown.

Remote host = tara.nstda.or.th

Username = [username]

Port = 22

3. Click OK, then SFTP session will start. Now, you can drag and drop files to transfer files between TARA and your local machine.

3. Checking Software availability

TARA has reasonable number of software installed. These software are managed by module framework using Environment module system (Lmod). (YouTube : https://youtu.be/NAkJv2uVQZA)

The commands in this part are executed in SSH session. If you transferring file using SFTP session, switch to the SSH session by click at SSH tap with the key icon.

To list all available software

Command : module avail

Example :

[hpcuser@tara-frontend-1 ~]$ module avail ----------------------------------------------------------------------------------------------------- /tarafs/utils/modules/modules/all ----------------------------------------------------------------------------------------------------- ANSYS/19.3 EasyBuild/4.1.0 Mako/1.1.0-GCCcore-8.3.0 ScaLAPACK/2.0.2-gompic-2019b (D) fosscuda/2019b Advisor/2019_update5 FFTW/3.3.8-gompi-2019b Meson/0.51.2-GCCcore-8.3.0-Python-3.7.4 Singularity/3.3.0 gcccuda/2019b Autoconf/2.69-GCCcore-8.3.0 FFTW/3.3.8-gompic-2019b (D) NASM/2.14.02-GCCcore-8.3.0 Singularity/3.4.2 (D) gnuplot/5.2.8-GCCcore-8.3.0 Autoconf/2.69 (D) FriBidi/1.0.5-GCCcore-8.3.0 NLopt/2.6.1-GCCcore-8.3.0 Szip/2.1.1-GCCcore-8.3.0 gompi/2019b Automake/1.15.1-GCCcore-8.3.0 GATK/4.1.4.1-GCCcore-8.3.0-Java-11 NSPR/4.21-GCCcore-8.3.0 Tcl/8.6.9-GCCcore-8.3.0 gompic/2019b ...

To load software

Command : module load [software]

Example :

[hpcuser@tara-frontend-1 ~]$ module load Python/3.7.4-GCCcore-8.3.0 [hpcuser@tara-frontend-1 ~]$ python --version Python 3.7.4

To list all loaded software

Command : module list

Example :

[hpcuser@tara-frontend-1 ~]$ module list Currently Loaded Modules: 1) GCCcore/8.3.0 7) Tcl/8.6.9-GCCcore-8.3.0 2) zlib/1.2.11-GCCcore-8.3.0 (H) 8) SQLite/3.29.0-GCCcore-8.3.0 3) binutils/2.32-GCCcore-8.3.0 9) XZ/5.2.4-GCCcore-8.3.0 4) bzip2/1.0.8-GCCcore-8.3.0 10) GMP/6.1.2-GCCcore-8.3.0 5) ncurses/6.1-GCCcore-8.3.0 (H) 11) libffi/3.2.1-GCCcore-8.3.0 (H) 6) libreadline/8.0-GCCcore-8.3.0 (H) 12) Python/3.7.4-GCCcore-8.3.0 Where: H: Hidden Module

To unload all loaded software

Command : module purge

Example :

For more information about LMOD system please see การใช้งาน module ใน TARA Cluster

4. Software/Application installation (if applicable)

If TARA does not have the software(s) you need installed, you can install them by yourself in either your home directory or your project directory. Please make sure that you have the software license and the license term allows to run the software on TARA.

We support EasyBuild (https://easybuild.io/), see การใช้งาน local module ใน TARA Cluster for the step by step instruction to install software as a local module. You can also check การติดตั้งและการงานใช้ Application for some specific software installation/setup instruction.

5. Running Jobs

Check TARA HPC status

Before submitting your job to run on TARA, you can check available computational resources or status of partitions by using sinfo command.

Example :

where

Item

Description

Item

Description

PARTITION

partition name

AVAIL

partition status (up : working, down : not working)

TIMELIMIT

maximum time that a job can run on this partition

NODES

number of nodes in the partition

STATE

partition availability (idle : free, alloc : busy, mix : some core(s) in node is free)

NODELIST

list of nodes in the partition

Job submission

TARA usage is subjected to Fair-share policy and managed by Slurm job scheduler. To submit a job follows the following steps.

1. Create bash script using vi sbatch_script.sh (or gedit sbatch_script.sh , nano sbatch_script.sh)

2. Specify the details in the bash script according to the template provided below.

3. Use sbatch command followed by your bash script name to submit your job to SLURM

6. Checking job status

Check job status

To check your job status, using myqueue command.

Example :

Item

Description

Item

Description

JOBID

ID of the specific job

PARTITION

partition which job is running/queuing on

NAME

name of the specific job

USER

user who owns/run the job

ST

Job status (PD : pending, R : running)

TIME

elapsed time of the running job

NODE

the number of node used in the run

NODELIST(REASON)

list of node used to run (reason why the job is pending) see สถานะ Pending (PD) for more information

Cancel job

To cancel the job in the queue, using scancel [JOBID] command

Example : scancel 140215

7. Transferring files to local machine

To transfer files from TARA to your local machine, you can use FTP client such as FileZilla, WinSCP, MobaXterm, etc.. In this document, we demonstrate this process using MobaXterm.

1. Open SFTP session in the MobaXterm (see session 2 Transferring files to TARA for instruction)

2. Drag file/directory from TARA and drop at your local machine

8. Checking resource usage and remaining quota

Service Unit

You can see the usage quantity and the remaining amount of the service unit (SU) by using sbalance. You can see SU usage by users by using -d flag (i.e. sbalance -d)

Disk quota

You can see the amount of space used and remaining using myquota

Example :

9. Removing unnecessary files

You can free up your disk space by removing unnecessary files. You can remove file by using rm [file] command and remove directory by using rm -r [directory] command.

Example :

ls -l is the command to list files or directories in the current location

Related articles