Install your own WRF-Chem

Updated: 18 October 2023

1 1. Load required modules and set environment variables
2 2. Set install location directory (DIR)
3 3. Download WRF source code
4 4. Enable GRIB2 IO (Optional)
5 5. Configure
6 6. Compile WRF-Chem
7 7. Example: Job submission script
8 A. Compile convert_emiss.exe (Optional)

1. Load required modules and set environment variables

Paste the below lines into your terminal. (Putting these in ~/.bashrc is not recommended.)

module purge
module load cpeIntel/23.09
module load cray-hdf5-parallel/1.12.2.7
module load cray-netcdf-hdf5parallel/4.9.0.7
module load cray-parallel-netcdf/1.12.3.7
module load libpng/1.6.39-cpeIntel-23.09
module load JasPer/1.900.1-cpeIntel-23.09
module load flex/2.6.4
module load ADIOS2/2.9.1-cpeIntel-23.09

export WRF_EM_CORE=1    # Explicitly select ARW core developed by NCAR
export WRF_NMM_CORE=0   # Not NMM core
export WRF_DA_CORE=0    # Not WRFDA
export WRF_CHEM=1       # Select WRF-Chem

export NETCDF=${CRAY_NETCDF_HDF5PARALLEL_PREFIX}
export NETCDFPAR=${CRAY_NETCDF_HDF5PARALLEL_PREFIX}    
export PNETCDF=${CRAY_PARALLEL_NETCDF_PREFIX}           
export HDF5=${CRAY_HDF5_PARALLEL_PREFIX}
export JASPERINC=${EBROOTJASPER}/include              # (Optional)
export JASPERLIB=${EBROOTJASPER}/lib                  # (Optional)
export ADIOS2=${EBROOTADIOS2}

export WRFIO_NCD_LARGE_FILE_SUPPORT=1                 # (Optional)
export PNETCDF_QUILT=1                                # (Optional)

export WRF_KPP=1
export FLEX=flex
export FLEX_LIB_DIR=${EBROOTFLEX}/lib
export YACC='/usr/bin/yacc -d'

### Common mistake :: insert spaces before or after =

2. Set install location directory (DIR)

Let set an install path, DIR.

# While being in your desired install location directory, execute
export DIR=$(pwd)

3. Download WRF source code

An example for WRF 4.5.1 is as shown below.

cd $DIR
wget https://github.com/wrf-model/WRF/releases/download/v4.5.1/v4.5.1.tar.gz
tar xzf v4.5.1.tar.gz
mv WRFV4.5.1 WRF

Some chemistry options may be incompatible with the most recent WRF release. Users may need to compile WRF-Chem using older versions.

For other version, check the WRF stable branches on the GitHub repository and its official website. For WRF 3.X, users have to add ./WRFV3/chem folder with these.

4. Enable GRIB2 IO (Optional)

Must set JASPERINC and JASPERLIB in Step 1

GRIB2 IO is now deprecated in WRF 4.6.X.

Open ${DIR}/WRF/arch/Config.pl (or Config_new.pl) using a text editor such as vi
- Change the parameter $I_really_want_to_output_grib2_from_WRF from "FALSE" to "TRUE"

vi ${DIR}/WRF/arch/Config.pl           # or Config_new.pl
# Change $I_really_want_to_output_grib2_from_WRF = "FALSE" ; 
#   to   $I_really_want_to_output_grib2_from_WRF = "TRUE" ;

5. Configure

Execute module swap intel intel-oneapi to use the Intel oneAPI compilers
Remove rm -f configure.wrf in ${DIR}/WRF/configure by, for example,
executing sed -i 's/rm -rf configure.wrf//' ${DIR}/WRF/configure
Run ./configure and select 78 for the new DM + INTEL (ifx/icx) : oneAPI LLVM choice and then select 1 for basic nesting.
Edit ./configure.wrf to
SFC = ftn SCC = cc CCOMP = cc DM_FC = ftn DM_CC = cc
also add -fp-speculation=safe to FCBASEOPTS_NO_G
Proceed to “Step 6: Compile WRF-Chem” in the main guide below

Note: It is alright to do Step 2, because the cause was from using the wrong compiler wrappers that are fixed in Step 5.

5.1 Run ./configure

cd ${DIR}/WRF
./configure
# Type 50, then press Enter <-- (dmpar) INTEL (ftn/icc): Cray XC
# Type 1, then press Enter  <-- Basic nesting

5.2 Edit configure.wrf

Open ${DIR}/WRF/configure.wrf using a text editor such as vi
- Append -Wno-implicit-function-declaration -Wno-implicit-int to cc.
- Prepend -fp-model precise -fp-speculation=safe to FCBASEOPTS_NO_G.

vi configure.wrf
# Change  DM_CC = cc
#   to    DM_CC = cc -Wno-implicit-function-declaration -Wno-implicit-int
# Change  FCBASEOPTS_NO_G = -w -ftz -fno-alias -align all $(FORMAT_FREE) $(BYTESWAPIO)
#   to    FCBASEOPTS_NO_G = -fp-model precise -fp-speculation=safe -w -ftz -fno-alias -align all $(FORMAT_FREE) $(BYTESWAPIO)

5.3 Replace ./share/landread.c with ./share/landread.c.dist

(Only required for some versions, such as 3.6.1, 3.8.1 and 3.9.1.1)

cp ${DIR}/WRF/share/landread.c.dist ${DIR}/WRF/share/landread.c

5.4 Edit ./chem/KPP/kpp/kpp-2.1/src/Makefile

(Only required for some versions, i.e., 3.6.1 and 3.8.1)

Remove all -ll in -lfl -ll -o kpp

5.5 Replace KPP folder + Use cpeGNU with two additional flags

(Only required for some versions, i.e., 3.6.1)

Replace ./chem/KPP folder with a newer KPP, e.g., of 3.8.1.
Use cpeGNU → Type 34 → Type 1
Change compilers from gfortran, gcc to cc and ftn
append -fallow-argument-mismatch -fallow-invalid-boz to FCBASEOPTS_NO_G

6. Compile WRF-Chem

./compile em_real 2>&1 | tee compile.wrfchem.log # This step takes around 1 hours on LANTA.

7. Example: Job submission script

#!/bin/bash
#SBATCH -p compute              # Partition
#SBATCH -N 1                    # Number of nodes
#SBATCH --ntasks-per-node=128   # Number of MPI processes per node
#SBATCH -t 5-00:00:00           # Job runtime limit
#SBATCH -J WRFchem              # Job name
#SBATCH -A ltxxxxxx             # Account *** {USER EDIT} ***

module purge
module load cpeIntel/23.09
module load cray-hdf5-parallel/1.12.2.7
module load cray-netcdf-hdf5parallel/4.9.0.7
module load cray-parallel-netcdf/1.12.3.7
module load libpng/1.6.39-cpeIntel-23.09
module load JasPer/1.900.1-cpeIntel-23.09
module load ADIOS2/2.9.1-cpeIntel-23.09

export OMP_NUM_THREADS=1

ulimit -s unlimited

#srun -n16 ./geogrid.exe
#srun -N1 -n1 ./ungrib.exe
#srun -n16 ./metgrid.exe

srun -n16 ./real.exe
srun ./wrf.exe

A. Compile convert_emiss.exe (Optional)

./compile emi_conv 2>&1 | tee compile.convert_emiss.log

CONVERT_EMISS of some WRF releases could cause an unexpected segmentation fault at runtime (see this and this). CONVERT_EMISS of WRF 3.6.1 and 4.5.X are recommended.

Contact Us
ThaiSC support service : thaisc-support@nstda.or.th