This article will guide you to create the environment using Miniconda on a LANTA HPC system. An overview of the content can be found in the table of contents below for immediate visualization of the interesting parts.
Using Miniconda on LANTA HPC
Load Miniconda module
Use the
ml av Minicondacommand to see which version of Miniconda is available on the LANTA HPC system.Use the
ml Miniconda3/xx.xx.xcommand to load the Miniconda version that you want to use. If you don't specify a version, the default version (D) is loaded, which is Miniconda3/23.3.1-0.
username@lanta:~> ml av Miniconda ---------------------- /lustrefs/disk/modules/easybuild/modules/all ----------------------- Miniconda3/22.11.1-1 Miniconda3/23.3.1-0 (D) Use "module spider" to find all possible modules and extensions. Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys". username@lanta:~> ml Miniconda3/22.11.1-1
Unload Miniconda module
Use the module unload Miniconda3 command to unload the Miniconda.
username@lanta:~> module unload Miniconda3
Use the module purge command to unload all currently loaded modules.
username@lanta:~> module purge
Before you use the module unload Miniconda3 or module purge command, you must deactivate your environment with the conda deactivate command.
Activate your environment
Use the
conda env listcommand to view a list of your environments.If you want to activate your environment such as TensorFlow-2.6.0, you can use the
conda activate tensorflow-2.6.0command.
username@lanta:~> conda env list # conda environments: # base /lustrefs/disk/modules/easybuild/software/Miniconda3/22.11.1-1 netcdf-py39 /lustrefs/disk/modules/easybuild/software/Miniconda3/22.11.1-1/envs/netcdf-py39 pytorch-1.11.0 /lustrefs/disk/modules/easybuild/software/Miniconda3/22.11.1-1/envs/pytorch-1.11.0 tensorflow-2.6.0 /lustrefs/disk/modules/easybuild/software/Miniconda3/22.11.1-1/envs/tensorflow-2.6.0 username@lanta:~> conda activate tensorflow-2.6.0 (tensorflow-2.6.0) username@lanta:~>
Deactivate your environment
Use the conda deactivate command to deactivate your environments.
(tensorflow-2.6.0) username@lanta:~> conda deactivate username@lanta:~>
Creating the environment in the user’s home
Create the environment
Use the conda create -n myenv commands to create the environment with myenv name.
username@lanta:~> conda create -n myenv Collecting package metadata (current_repodata.json): done Solving environment: done ## Package Plan ## environment location: /your directory/envs/myenv Proceed ([y]/n)? y ...
Create the environment with a specific version of the packages
Use the conda create -n myenv python=3.9 commands to create the myenv environment with a specific version of python.
username@lanta:~> conda create -n myenv python=3.9
Collecting package metadata (current_repodata.json): done
Solving environment: done
## Package Plan ##
environment location: /your directory/envs/myenv
added / updated specs:
- python=3.9
The following packages will be downloaded:
package | build
---------------------------|-----------------
ca-certificates-2023.01.10 | h06a4308_0 120 KB
certifi-2022.12.7 | py39h06a4308_0 150 KB
------------------------------------------------------------
Total: 270 KB
The following NEW packages will be INSTALLED:
_libgcc_mutex pkgs/main/linux-64::_libgcc_mutex-0.1-main
_openmp_mutex pkgs/main/linux-64::_openmp_mutex-5.1-1_gnu
ca-certificates pkgs/main/linux-64::ca-certificates-2023.01.10-h06a4308_0
certifi pkgs/main/linux-64::certifi-2022.12.7-py39h06a4308_0
ld_impl_linux-64 pkgs/main/linux-64::ld_impl_linux-64-2.38-h1181459_1
libffi pkgs/main/linux-64::libffi-3.4.2-h6a678d5_6
libgcc-ng pkgs/main/linux-64::libgcc-ng-11.2.0-h1234567_1
libgomp pkgs/main/linux-64::libgomp-11.2.0-h1234567_1
libstdcxx-ng pkgs/main/linux-64::libstdcxx-ng-11.2.0-h1234567_1
ncurses pkgs/main/linux-64::ncurses-6.4-h6a678d5_0
openssl pkgs/main/linux-64::openssl-1.1.1t-h7f8727e_0
pip pkgs/main/linux-64::pip-22.3.1-py39h06a4308_0
python pkgs/main/linux-64::python-3.9.16-h7a1cb2a_0
readline pkgs/main/linux-64::readline-8.2-h5eee18b_0
setuptools pkgs/main/linux-64::setuptools-65.6.3-py39h06a4308_0
sqlite pkgs/main/linux-64::sqlite-3.40.1-h5082296_0
tk pkgs/main/linux-64::tk-8.6.12-h1ccaba5_0
tzdata pkgs/main/noarch::tzdata-2022g-h04d1e81_0
wheel pkgs/main/linux-64::wheel-0.38.4-py39h06a4308_0
xz pkgs/main/linux-64::xz-5.2.10-h5eee18b_1
zlib pkgs/main/linux-64::zlib-1.2.13-h5eee18b_0
Proceed ([y]/n)? y
...
Use the conda create -n myenv python=3.9 numpy=1.23.5 commands to create the myenv environment with a specific version of python and numpy.
username@lanta:~> conda create -n myenv python=3.9 numpy=1.23.5
Collecting package metadata (current_repodata.json): done
Solving environment: done
## Package Plan ##
environment location: /your directory/envs/myenv
added / updated specs:
- numpy=1.23.5
- python=3.9
The following packages will be downloaded:
package | build
---------------------------|-----------------
blas-1.0 | mkl 6 KB
ca-certificates-2023.01.10 | h06a4308_0 120 KB
certifi-2022.12.7 | py39h06a4308_0 150 KB
intel-openmp-2021.4.0 | h06a4308_3561 4.2 MB
mkl-2021.4.0 | h06a4308_640 142.6 MB
mkl-service-2.4.0 | py39h7f8727e_0 59 KB
mkl_fft-1.3.1 | py39hd3c417c_0 182 KB
mkl_random-1.2.2 | py39h51133e4_0 309 KB
numpy-1.23.5 | py39h14f4228_0 10 KB
numpy-base-1.23.5 | py39h31eccc5_0 6.7 MB
------------------------------------------------------------
Total: 154.3 MB
The following NEW packages will be INSTALLED:
_libgcc_mutex pkgs/main/linux-64::_libgcc_mutex-0.1-main
_openmp_mutex pkgs/main/linux-64::_openmp_mutex-5.1-1_gnu
blas pkgs/main/linux-64::blas-1.0-mkl
ca-certificates pkgs/main/linux-64::ca-certificates-2023.01.10-h06a4308_0
certifi pkgs/main/linux-64::certifi-2022.12.7-py39h06a4308_0
intel-openmp pkgs/main/linux-64::intel-openmp-2021.4.0-h06a4308_3561
ld_impl_linux-64 pkgs/main/linux-64::ld_impl_linux-64-2.38-h1181459_1
libffi pkgs/main/linux-64::libffi-3.4.2-h6a678d5_6
libgcc-ng pkgs/main/linux-64::libgcc-ng-11.2.0-h1234567_1
libgomp pkgs/main/linux-64::libgomp-11.2.0-h1234567_1
libstdcxx-ng pkgs/main/linux-64::libstdcxx-ng-11.2.0-h1234567_1
mkl pkgs/main/linux-64::mkl-2021.4.0-h06a4308_640
mkl-service pkgs/main/linux-64::mkl-service-2.4.0-py39h7f8727e_0
mkl_fft pkgs/main/linux-64::mkl_fft-1.3.1-py39hd3c417c_0
mkl_random pkgs/main/linux-64::mkl_random-1.2.2-py39h51133e4_0
ncurses pkgs/main/linux-64::ncurses-6.4-h6a678d5_0
numpy pkgs/main/linux-64::numpy-1.23.5-py39h14f4228_0
numpy-base pkgs/main/linux-64::numpy-base-1.23.5-py39h31eccc5_0
openssl pkgs/main/linux-64::openssl-1.1.1t-h7f8727e_0
pip pkgs/main/linux-64::pip-22.3.1-py39h06a4308_0
python pkgs/main/linux-64::python-3.9.16-h7a1cb2a_0
readline pkgs/main/linux-64::readline-8.2-h5eee18b_0
setuptools pkgs/main/linux-64::setuptools-65.6.3-py39h06a4308_0
six pkgs/main/noarch::six-1.16.0-pyhd3eb1b0_1
sqlite pkgs/main/linux-64::sqlite-3.40.1-h5082296_0
tk pkgs/main/linux-64::tk-8.6.12-h1ccaba5_0
tzdata pkgs/main/noarch::tzdata-2022g-h04d1e81_0
wheel pkgs/main/linux-64::wheel-0.38.4-py39h06a4308_0
xz pkgs/main/linux-64::xz-5.2.10-h5eee18b_1
zlib pkgs/main/linux-64::zlib-1.2.13-h5eee18b_0
Proceed ([y]/n)? y
...
Creating the environment in the project’s home
Specify a location for the environment
Use the conda create --prefix /your project directory/envs commands to create the environment on the specific location.
username@lanta:~> conda create --prefix /your project directory/envs Collecting package metadata (current_repodata.json): done Solving environment: done ## Package Plan ## environment location: /your project directory/envs Proceed ([y]/n)? y ...
Specify a location for the environment with a specific version of a package
Use the conda create --prefix /your project directory/envs python=3.9 commands to create the environment with a specific version of a package on a specific location.
username@lanta:~> conda create --prefix /your project directory/envs python=3.9
Collecting package metadata (current_repodata.json): done
Solving environment: done
## Package Plan ##
environment location: /your project directory/envs
added / updated specs:
- python=3.9
The following packages will be downloaded:
package | build
---------------------------|-----------------
ca-certificates-2023.01.10 | h06a4308_0 120 KB
certifi-2022.12.7 | py39h06a4308_0 150 KB
------------------------------------------------------------
Total: 270 KB
The following NEW packages will be INSTALLED:
_libgcc_mutex pkgs/main/linux-64::_libgcc_mutex-0.1-main
_openmp_mutex pkgs/main/linux-64::_openmp_mutex-5.1-1_gnu
ca-certificates pkgs/main/linux-64::ca-certificates-2023.01.10-h06a4308_0
certifi pkgs/main/linux-64::certifi-2022.12.7-py39h06a4308_0
ld_impl_linux-64 pkgs/main/linux-64::ld_impl_linux-64-2.38-h1181459_1
libffi pkgs/main/linux-64::libffi-3.4.2-h6a678d5_6
libgcc-ng pkgs/main/linux-64::libgcc-ng-11.2.0-h1234567_1
libgomp pkgs/main/linux-64::libgomp-11.2.0-h1234567_1
libstdcxx-ng pkgs/main/linux-64::libstdcxx-ng-11.2.0-h1234567_1
ncurses pkgs/main/linux-64::ncurses-6.4-h6a678d5_0
openssl pkgs/main/linux-64::openssl-1.1.1t-h7f8727e_0
pip pkgs/main/linux-64::pip-22.3.1-py39h06a4308_0
python pkgs/main/linux-64::python-3.9.16-h7a1cb2a_0
readline pkgs/main/linux-64::readline-8.2-h5eee18b_0
setuptools pkgs/main/linux-64::setuptools-65.6.3-py39h06a4308_0
sqlite pkgs/main/linux-64::sqlite-3.40.1-h5082296_0
tk pkgs/main/linux-64::tk-8.6.12-h1ccaba5_0
tzdata pkgs/main/noarch::tzdata-2022g-h04d1e81_0
wheel pkgs/main/linux-64::wheel-0.38.4-py39h06a4308_0
xz pkgs/main/linux-64::xz-5.2.10-h5eee18b_1
zlib pkgs/main/linux-64::zlib-1.2.13-h5eee18b_0
Proceed ([y]/n)? y
...
Activate your environment in the project’s home
If you want to activate your environment in the project’s home, you can use the conda activate /your project directory/envs command.
username@lanta:~> conda activate /your project directory/envs (/your project directory/envs) username@lanta:~>
Creating the environment from an requirements.yml file
A simple requirements.yml file
In the requirements.yml file, you need to specify the environment name and packages that you want to use.
name: envs dependencies: - python=3.9 - numpy=1.23.5 - pandas
Create the environment from the requirements.yml file in the user’s home
Use the conda env create -f requirements.yml commands to create the environment from the requirements.yml file in the user’s home.
username@lanta:~> conda env create -f requirements.yml Collecting package metadata (repodata.json): done Solving environment: done Downloading and Extracting Packages Preparing transaction: done Verifying transaction: done Executing transaction: done
Create the environment from the requirements.yml file in the project’s home
Use the conda env create -f requirements.yml --prefix /your project directory/envs commands to create the environment from the requirements.yml file on the specific location.
username@lanta:~> conda env create -f requirements.yml --prefix /your project directory/envs Collecting package metadata (repodata.json): done Solving environment: done Downloading and Extracting Packages Preparing transaction: done Verifying transaction: done Executing transaction: done
Removing the environment
Remove the environment in the user’s home
If you want to remove the myenv environment, you can use the conda remove --name myenv --all command.
username@lanta:~> conda remove --name myenv --all
Remove the environment in the project’s home
If you want to remove the environment in /your project directory/envs, you can use the rm -rf /your project directory/envs command.
username@lanta:~> rm -rf /your project directory/envs