Versions Compared

Version Old Version 12 New Version Current
Changes made by

Yutthana Wongnongwa

Yutthana Wongnongwa

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

This article will guide you to create an the environment using Miniconda on a LANTA HPC system.

Table of Contents

Using Miniconda on LANTA HPC

Load Miniconda module

...

An overview of the content can be found in the table of contents below for immediate visualization of the interesting parts.

Table of Contents

Using Mamba on LANTA HPC

Load Mamba module

  1. Use the ml av Mamba command to see which version of Miniconda Mamba is available on the LANTA HPC system.

  2. Use the ml Miniconda3Mamba/xx.xx.x command to load the Miniconda Mamba version that you want to use. If you don't specify a version, the default version (D) is loaded, which is Miniconda3Mamba/2223.11.10-10.

Code Block
username@lanta:~> ml av MinicondaMamba
---------------------- /lustrefs/disk/modules/easybuild/modules/all -----------------------
   Miniconda3Mamba/2223.11.1-10-0 (D)

Use "module spider" to find all possible modules and extensions.
Use "module keyword key1 key2 ..." to search for all possible modules matching any of the "keys".
username@lanta:~> ml Miniconda3Mamba/2223.11.10-1

...

0

Unload Mamba module

Use the

...

module unload Mamba command to unload the Mamba.

Code Block
username@lanta:~> module unload Mamba

Use the module purge command to unload all currently loaded modules.

Code Block
username@lanta:~> conda env list
# conda environments:
#
basemodule purge
Note

Before you use the module unload Mamba or module purge command, you must deactivate your environment with the conda deactivate command.

Activate your environment

  1. Use the conda env list command to view a list of your environments.

  2. If you want to activate your environment such as TensorFlow-2.6.0, you can use the conda activate tensorflow-2.12.1 command.

Code Block
username@lanta:~> conda env list
# conda environments:
#
base                     /lustrefs/disk/modules/easybuild/software/Mamba/23.11.0-0
netcdf-py39              /lustrefs/disk/modules/easybuild/software/Mamba/23.11.0-0/envs/netcdf-py39
pytorch-2.2.2            /lustrefs/disk/modules/easybuild/software/Mamba/23.11.0-0/envs/pytorch-2.2.2
tensorflow-2.12.1        /lustrefs/disk/modules/easybuild/software/Mamba/23.11.0-0/envs/tensorflow-2.12.1
username@lanta:~> conda activate tensorflow-2.12.1
(tensorflow-2.12.1) username@lanta:~>

Deactivate your environment

Use the conda deactivate command to deactivate your environments.

Code Block
(tensorflow-2.12.1) username@lanta:~> conda deactivate
username@lanta:~>

Creating the environment in the user’s home

Create the environment

Use the conda create -n myenv commands to create the environment with myenv name.

Code Block
username@lanta:~> conda create -n myenv
Channels:
 - conda-forge
Platform: linux-64
Collecting package metadata (repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /your directory/envs/myenv

Proceed ([y]/n)? y
...

Create the environment with a specific version of the packages

Use the conda create -n myenv python=3.9 commands to create the myenv environment with a specific version of python.

Code Block
username@lanta:~> conda create -n myenv python=3.9
Channels:
 - conda-forge
Platform: linux-64
Collecting package metadata (repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /your directory/envs/myenv

  added / updated specs:
    - python=3.9


The following packages will be downloaded:

    package                    |            build
    ---------------------------|-----------------
    python-3.9.19              |h0755675_0_cpython        22.7 MB  conda-forge
    wheel-0.43.0               |     pyhd8ed1ab_1          57 KB  conda-forge
    ------------------------------------------------------------
                                           Total:        22.8 MB

The following NEW packages will be INSTALLED:

  _libgcc_mutex      conda-forge/linux-64::_libgcc_mutex-0.1-conda_forge
  _openmp_mutex      conda-forge/linux-64::_openmp_mutex-4.5-2_gnu
  bzip2              conda-forge/linux-64::bzip2-1.0.8-hd590300_5
  ca-certificates    conda-forge/linux-64::ca-certificates-2024.2.2-hbcca054_0
  ld_impl_linux-64   conda-forge/linux-64::ld_impl_linux-64-2.40-h41732ed_0
  libffi             conda-forge/linux-64::libffi-3.4.2-h7f98852_5
  libgcc-ng          conda-forge/linux-64::libgcc-ng-13.2.0-h807b86a_5
  libgomp            conda-forge/linux-64::libgomp-13.2.0-h807b86a_5
  libnsl             conda-forge/linux-64::libnsl-2.0.1-hd590300_0
  libsqlite          conda-forge/linux-64::libsqlite-3.45.2-h2797004_0
  libuuid            conda-forge/linux-64::libuuid-2.38.1-h0b41bf4_0
  libxcrypt          conda-forge/linux-64::libxcrypt-4.4.36-hd590300_1
  libzlib            conda-forge/linux-64::libzlib-1.2.13-hd590300_5
  ncurses            conda-forge/linux-64::ncurses-6.4.20240210-h59595ed_0
  openssl            conda-forge/linux-64::openssl-3.2.1-hd590300_1
  pip                conda-forge/noarch::pip-24.0-pyhd8ed1ab_0
  python             conda-forge/linux-64::python-3.9.19-h0755675_0_cpython
  readline           conda-forge/linux-64::readline-8.2-h8228510_1
  setuptools         conda-forge/noarch::setuptools-69.2.0-pyhd8ed1ab_0
  tk                 conda-forge/linux-64::tk-8.6.13-noxft_h4845f30_101
  tzdata             conda-forge/noarch::tzdata-2024a-h0c530f3_0
  wheel              conda-forge/noarch::wheel-0.43.0-pyhd8ed1ab_1
  xz                 conda-forge/linux-64::xz-5.2.6-h166bdaf_0

Proceed ([y]/n)? y
...

Use the conda create -n myenv python=3.9 numpy=1.23.5 commands to create the myenv environment with a specific version of python and numpy.

Code Block
username@lanta:~> conda create -n myenv python=3.9 numpy=1.23.5
Channels:
 - conda-forge
Platform: linux-64
Collecting package metadata (repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /your directory/envs/myenv

  added / updated specs:
    - numpy=1.23.5
    - python=3.9


The following packages will be downloaded:

    package                     /lustrefs/disk/modules/easybuild/software/Miniconda3/22.11.1-1
netcdf-py39|            build
    /lustrefs/disk/modules/easybuild/software/Miniconda3/22.11.1-1/envs/netcdf-py39
pytorch-1.11---------------------------|-----------------
    libblas-3.9.0           /lustrefs/disk/modules/easybuild/software/Miniconda3/22.11.1-1/envs/pytorch-1.11.0
tensorflow-2.6.0   |21_linux64_openblas          14 KB  conda-forge
    libcblas-3.9.0             /lustrefs/disk/modules/easybuild/software/Miniconda3/22.11.1-1/envs/tensorflow-2.6.0
username@lanta:~> conda activate tensorflow-2.6.0
(tensorflow-2.6.0) username@lanta:~>

Creating an environment in the user’s home

Create an environment

Use the conda create -n myenv commands to create the conda environment with myenv name.

Code Block
username@lanta:~> conda create -n myenv
Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /your directory/envs/myenv

Proceed ([y]/n)? y
...
username@lanta:~>

Create an environment with a specific version of the packages

Use the conda create -n myenv python=3.9 commands to create the myenv environment with a specific version of python.

Code Block
username@lanta:~> conda create -n myenv python=3.9
Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /your directory/envs/myenv

  added / updated specs:
    - python=3.9

The following packages will be downloaded:

    package                    ||21_linux64_openblas          14 KB  conda-forge
    libgfortran-ng-13.2.0      |       h69a702a_5          23 KB  conda-forge
    liblapack-3.9.0            |21_linux64_openblas          14 KB  conda-forge
    numpy-1.23.5               |   py39h3d75532_0         5.6 MB  conda-forge
    python-3.9.19              |h0755675_0_cpython    build     ---------------------------|-----------------22.7 MB  conda-forge
    ca-certificates-2023.01.10python_abi-3.9             |       h06a4308_0    4_cp39           1206 KB  conda-forge
   certifi wheel-20220.12.743.0               |     py39h06a4308_0pyhd8ed1ab_1          57 KB 150 KBconda-forge
    ------------------------------------------------------------
                                           Total:        28.4 270MB
KB

The following NEW packages will be INSTALLED:

  _libgcc_mutex      pkgs/mainconda-forge/linux-64::_libgcc_mutex-0.1-mainconda_forge
  _openmp_mutex      pkgs/mainconda-forge/linux-64::_openmp_mutex-4.5.1-12_gnu
   ca-certificatesbzip2              pkgs/mainconda-forge/linux-64::cabzip2-certificates-20231.01.10-h06a4308_0
  certifi0.8-hd590300_5
  ca-certificates         pkgs/mainconda-forge/linux-64::certifica-certificates-20222024.122.72-py39h06a4308hbcca054_0
  ld_impl_linux-64   pkgs/mainconda-forge/linux-64::ld_impl_linux-64-2.38-h1181459_1
  libffi2.40-h41732ed_0
  libblas            conda-forge/linux-64::libblas-3.9.0-21_linux64_openblas
  libcblas             pkgs/mainconda-forge/linux-64::libffilibcblas-3.49.20-h6a678d5_621_linux64_openblas
  libffi   libgcc-ng          pkgsconda-forge/main/linux-64::libgcc-ng-11libffi-3.4.2.0-h1234567h7f98852_15
  libgomplibgcc-ng            pkgs/main/conda-forge/linux-64::libgomplibgcc-ng-1113.2.0-h1234567h807b86a_15
  libstdcxxlibgfortran-ng       pkgs/main/conda-forge/linux-64::libstdcxxlibgfortran-ng-1113.2.0-h1234567h69a702a_15
  ncurseslibgfortran5            pkgs/main/conda-forge/linux-64::ncurseslibgfortran5-613.2.40-h6a678d5ha4646dd_05
  openssllibgomp            pkgsconda-forge/main/linux-64::openssllibgomp-113.12.1t-h7f8727e_0
  pip0-h807b86a_5
  liblapack             pkgs/main/conda-forge/linux-64::pipliblapack-223.39.10-py39h06a430821_linux64_0openblas
  pythonlibnsl             pkgsconda-forge/main/linux-64::pythonlibnsl-32.90.161-h7a1cb2ahd590300_0
  readlinelibopenblas           pkgs/mainconda-forge/linux-64::readlinelibopenblas-80.3.226-h5eee18bpthreads_h413a1c8_0
  setuptoolslibsqlite          pkgsconda-forge/main/linux-64::setuptoolslibsqlite-653.645.32-py39h06a4308h2797004_0
  sqlitelibstdcxx-ng             pkgs/mainconda-forge/linux-64::sqlitelibstdcxx-ng-313.402.10-h5082296h7e041cc_05
  tklibuuid                 pkgs/mainconda-forge/linux-64::tklibuuid-82.638.121-h1ccaba5h0b41bf4_0
  tzdatalibxcrypt             pkgs/main/noarch::tzdata-2022g-h04d1e81_0
  wheelconda-forge/linux-64::libxcrypt-4.4.36-hd590300_1
  libzlib              pkgs/main/conda-forge/linux-64::wheellibzlib-01.382.413-py39h06a4308hd590300_05
  xzncurses                 pkgs/mainconda-forge/linux-64::xzncurses-56.24.1020240210-h5eee18bh59595ed_10
  zlibnumpy               pkgs/main/conda-forge/linux-64::zlibnumpy-1.223.135-h5eee18bpy39h3d75532_0
 Proceed ([y]/n)? y
...
username@lanta:~>

Use the conda create -n myenv python=3.9 numpy=1.23.5 commands to create the myenv environment with a specific version of python and scipy.

Code Block
username@lanta:~> conda create -n myenv python=3.9 numpy=1.23.5
Collecting package metadata (current_repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /your directory/envs/myenv

  added / updated specs:
    - numpy=1.23.5
    - python=3.9

The following packages will be downloaded:

    package  openssl            conda-forge/linux-64::openssl-3.2.1-hd590300_1
  pip                conda-forge/noarch::pip-24.0-pyhd8ed1ab_0
  python             conda-forge/linux-64::python-3.9.19-h0755675_0_cpython
  python_abi         conda-forge/linux-64::python_abi-3.9-4_cp39
  readline           conda-forge/linux-64::readline-8.2-h8228510_1
  setuptools     |    conda-forge/noarch::setuptools-69.2.0-pyhd8ed1ab_0
  tk     build     ---------------------------|-----------------
    blas-1.0       conda-forge/linux-64::tk-8.6.13-noxft_h4845f30_101
  tzdata             conda-forge/noarch::tzdata-2024a-h0c530f3_0
   wheel    |          conda-forge/noarch::wheel-0.43.0-pyhd8ed1ab_1
  xz mkl           6 KB     ca-certificates-2023.01.10 |       h06a4308conda-forge/linux-64::xz-5.2.6-h166bdaf_0

Proceed ([y]/n)?      120 KB
    certifi-2022.12.7          |   py39h06a4308_0         150 KB
    intel-openmp-2021.4.0      |    h06a4308_3561         4.2 MB
    mkl-2021.4.0               |     h06a4308_640       142.6 MB
    mkl-service-2.4.0          |   py39h7f8727e_0          59 KB
    mkl_fft-1.3.1              |   py39hd3c417c_0y
...

Creating the environment in the project’s home

Specify a location for the environment

Use the conda create --prefix /your project directory/envs commands to create the environment on the specific location.

Code Block
username@lanta:~> conda create --prefix /your project directory/envs
Channels:
 - conda-forge
Platform: linux-64
Collecting package metadata (repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /your project directory/envs

Proceed ([y]/n)? y
...

Specify a location for the environment with a specific version of a package

Use the conda create --prefix /your project directory/envs python=3.9 commands to create the environment with a specific version of a package on a specific location.

Code Block
username@lanta:~> conda create --prefix /your project directory/envs python=3.9
Channels:
 - conda-forge
Platform: linux-64
Collecting package metadata (repodata.json): done
Solving environment: done

## Package Plan ##

  environment location: /your project directory/envs

  added / updated specs:
    - python=3.9

The following packages will be downloaded:

    package              182 KB     mkl_random-1.2.2|           | build
 py39h51133e4_0   ---------------------------|-----------------
     309 KB
    numpy-1.23.5 python-3.9.19              |   py39h14f4228h0755675_0_cpython        22.7 MB 10 KBconda-forge
    numpywheel-base-10.23.543.0               |   py39h31eccc5_0  pyhd8ed1ab_1          57 6.7 MBKB  conda-forge
    ------------------------------------------------------------
   
                                       Total:       154.3 MB

The following NEW packages will be INSTALLED:

  _libgcc_mutex      pkgs/main/linux-64::_libgcc_mutex-0.1-main
  _openmp_mutex           pkgs/main/linux-64::_openmp_mutex-5.1-1_gnu
  blas     Total:        22.8 MB

The following NEW packages will be INSTALLED:

  _libgcc_mutex      pkgsconda-forge/main/linux-64::blas-1.0-mkl
  ca-certificates_libgcc_mutex-0.1-conda_forge
  _openmp_mutex      pkgsconda-forge/main/linux-64::ca-certificates-2023.01.10-h06a4308_0
  certifi_openmp_mutex-4.5-2_gnu
  bzip2              pkgsconda-forge/main/linux-64::certifibzip2-20221.120.78-py39h06a4308hd590300_05
  intelca-openmpcertificates       pkgs/main/conda-forge/linux-64::intelca-openmpcertificates-20212024.42.02-h06a4308hbcca054_35610
  ld_impl_linux-64   pkgsconda-forge/main/linux-64::ld_impl_linux-64-2.3840-h1181459h41732ed_10
  libffi             pkgsconda-forge/main/linux-64::libffi-3.4.2-h6a678d5h7f98852_65
  libgcc-ng          pkgs/mainconda-forge/linux-64::libgcc-ng-11.2.0-h1234567_1
  libgomp            pkgs/main/linux-64::libgomp-1113.2.0-h1234567_1h807b86a_5
  libgomp     libstdcxx-ng       pkgsconda-forge/main/linux-64::libstdcxxlibgomp-ng-1113.2.0-h1234567h807b86a_15
  mkllibnsl                pkgs/main/conda-forge/linux-64::mkllibnsl-20212.40.01-h06a4308hd590300_6400
  libsqlite  mkl-service        pkgs/mainconda-forge/linux-64::mkllibsqlite-service-23.445.02-py39h7f8727eh2797004_0
  mkl_fftlibuuid            pkgsconda-forge/main/linux-64::mkl_fftlibuuid-12.338.1-py39hd3c417ch0b41bf4_0
  mkl_randomlibxcrypt          pkgsconda-forge/main/linux-64::mkl_randomlibxcrypt-14.24.236-py39h51133e4hd590300_01
  ncurseslibzlib            pkgs/mainconda-forge/linux-64::ncurseslibzlib-61.2.413-h6a678d5hd590300_05
  numpy  ncurses            pkgs/mainconda-forge/linux-64::numpyncurses-16.234.520240210-py39h14f4228h59595ed_0
  openssl  numpy-base          pkgs/mainconda-forge/linux-64::numpyopenssl-base-13.232.51-py39h31eccc5_0hd590300_1
  pip    openssl            pkgs/main/linux-64conda-forge/noarch::opensslpip-124.1.1t0-h7f8727e_0
  pippyhd8ed1ab_0
  python             pkgsconda-forge/main/linux-64::pippython-223.39.119-py39h06a4308h0755675_0_cpython
  pythonreadline             pkgs/main/conda-forge/linux-64::pythonreadline-38.9.162-h7a1cb2ah8228510_01
  readlinesetuptools           pkgs/main/linux-64conda-forge/noarch::readlinesetuptools-869.2-h5eee18b_0.0-pyhd8ed1ab_0
  tk        setuptools         pkgsconda-forge/main/linux-64::setuptoolstk-658.6.313-py39h06a4308noxft_h4845f30_0101
  sixtzdata                pkgs/main/conda-forge/noarch::six-1.16.0-pyhd3eb1b0_1tzdata-2024a-h0c530f3_0
  sqlitewheel              pkgs/main/linux-64conda-forge/noarch::sqlitewheel-30.4043.10-h5082296pyhd8ed1ab_01
  tkxz                 pkgs/mainconda-forge/linux-64::tkxz-85.2.6.12-h1ccaba5h166bdaf_0

 tzdata             pkgs/main/noarch::tzdata-2022g-h04d1e81_0
  wheel              pkgs/main/linux-64::wheel-0.38.4-py39h06a4308_0
  xz                 pkgs/main/linux-64::xz-5.2.10-h5eee18b_1
  zlib               pkgs/main/linux-64::zlib-1.2.13-h5eee18b_0

Proceed ([y]/n)? y
...
username@lanta:~>

Creating an environment in the project’s home

Specify a location for an environment

Code Block
username@lanta:~> conda create --prefix /your project directory/envs

...

Proceed ([y]/n)? y
...

Activate your environment in the project’s home

If you want to activate your environment in the project’s home, you can use the conda activate /your project directory/envs command.

Code Block
username@lanta:~> conda activate /your project directory/envs
(/your project directory/envs) username@lanta:~>

Creating the environment from an requirements.yml file

A simple requirements.yml file

In the requirements.yml file, you need to specify the environment name and packages that you want to use.

Code Block
name: envs
dependencies:
  - python=3.9
  - numpy=1.23.5
  - pandas

Create the environment from the requirements.yml file in the user’s home

Use the conda env create -f requirements.yml commands to create the environment from the requirements.yml file in the user’s home.

Code Block
username@lanta:~> conda env create -f requirements.yml
Channels:
 - conda-forge
Platform: linux-64
Collecting package metadata (repodata.json): done
Solving environment: done

Downloading and Extracting Packages

Preparing transaction: done
Verifying transaction: done
Executing transaction: done

Create the environment from the requirements.yml file in the project’s home

Use the conda env create -f requirements.yml --prefix /your project directory/envs commands to create the environment from the requirements.yml file on the specific location.

Code Block
username@lanta:~> conda env create -f requirements.yml --prefix /your project directory/envs python=3.9

Activate your environment in the project’s home

Code Block
username@lanta:~> conda activate /your project directory/envs

Creating an environment from an environment.yml file

A simple environment.yml file

Code Block
name: test
dependencies:
  - python=3.9
  - numpy=1.23.5
  - pandas

Create the environment from the environment.yml file in the user’s home


Channels:
 - conda-forge
Platform: linux-64
Collecting package metadata (repodata.json): done
Solving environment: done

Downloading and Extracting Packages

Preparing transaction: done
Verifying transaction: done
Executing transaction: done

Removing the environment

Remove the environment in the user’s home

If you want to remove the myenv environment, you can use the conda remove --name myenv --all command.

Code Block
username@lanta:~> conda env create -f environment.yml

...

remove --name myenv --all

Remove the environment in the project’s home

If you want to remove the environment in /your project directory/envs, you can use the rm -rf /your project directory/envs command.

Code Block
username@lanta:~> condarm env create -f environment.yml --prefix-rf /your project directory/envs

...

Related articles

Filter by label (Content by label)
showLabelsfalse
max5
spacescom.atlassian.confluence.content.render.xhtml.model.resource.identifiers.SpaceResourceIdentifier@48ae393
showSpacefalse
sortmodified
showSpacetypefalsepage
reversetrue
typelabelspagesingularity python container
cqllabel in ( "vir-env" , "python-script" , "jupyter" ) and space = currentSpace ( )labelssingularity python container