Installation & Execution Guide

Installation & Execution Guide

This guide details the procedures for compiling and executing the solver using either the CPU-based Cantera backend or the GPU-native PelePhysics/AMReX backend.

1. System Requirements & Environment

The solver compiles as a hybrid Fortran/C++ codebase. Ensure the following compilers and libraries are loaded:

  • Fortran Compiler: gfortran (version 9+) or nvfortran (for GPU offloading).
  • C++ Compiler: g++ (C++17 standard support required) or nvc++.
  • MPI Implementation: OpenMPI (version 4.0+) or MPICH.
  • Python Environment: Python 3.9+ with pyyaml, numpy, and matplotlib (for mesh tools and diagnostics).

Loading Cluster Modules

On an HPC cluster, load the required modules to align compiler wrappers:

# Load GCC compiler suite, OpenMPI wrappers, and Miniconda
module load openmpi miniforge3

# Activate the dedicated reacting flow Python environment
module load react_env

2. Standard CPU Backend: Cantera Setup

To compile and run the solver using the standard Cantera CPU backend:

  1. Install Cantera C++ Headers & Libraries: On Ubuntu/Debian: bash sudo apt-get install libcantera-dev Or load it from the cluster's custom module environment.

  2. Locate Cantera Installation: Ensure cantera is in your compiler's default search path, or export its location: bash export CANTERA_DIR=/usr/local/include/cantera

3. High-Performance/GPU Backend: PelePhysics & AMReX

PelePhysics integrates AMReX (portable CPU/GPU grid managers) and SUNDIALS CVODE (stiff implicit ODE integration) into a unified, GPU-native thermodynamics and reaction network.

Clone the Repository

Clone the repository recursively to fetch the AMReX submodule:

git clone --recursive https://github.com/AMReX-Combustion/PelePhysics.git
cd PelePhysics

Build and Install via CMake

  1. Create a dedicated build folder: bash mkdir build && cd build
  2. Configure the build using OpenMPI compiler wrappers and enable SUNDIALS: bash cmake -DCMAKE_INSTALL_PREFIX=../install \ -DCMAKE_BUILD_TYPE=Release \ -DPELE_PHYSICS_ENABLE_SUNDIALS=ON \ -DCMAKE_CXX_COMPILER=mpicxx \ -DCMAKE_C_COMPILER=mpicc \ -DCMAKE_Fortran_COMPILER=mpifort \ ..
  3. Compile and install: bash make -j4 make install

4. Compiling the Solver

The solver compilation is controlled by environment variables toggling the active chemistry backend.

Option A: Standard Cantera CPU Build

Builds the solver using the Cantera C++ bridge and transport caching:

# Clean previous objects
make clean

# Compile for Cantera
make CHEMISTRY_BACKEND=cantera

Option B: PelePhysics C++ Build (CPU/GPU-ready)

Builds the solver using the PelePhysics bridge, linking against AMReX and CVODE:

# Setup PelePhysics home directory
export PELEPHYSICS_HOME=/home/kmukut/Radiation/PelePhysics

# Clean previous objects
make clean

# Compile specifying chemistry mechanism
make CHEMISTRY_BACKEND=pelephysics CHEMISTRY_MODEL=drm19

5. Launching Simulations

Running Locally (1 Rank / Single GPU)

./lowmach_react_hex cases/coflow_laminar_diffusion_2D_optimized/case_reacting.nml

Running on a Cluster (Multi-Rank / Multi-GPU)

Launch a parallel job using mpirun or mpiexec:

mpirun -np 4 ./lowmach_react_hex cases/coflow_laminar_diffusion_2D_optimized/case_reacting.nml
  • GPU Execution: If compiled with AMReX CUDA enabled, the ranks will automatically bind to the available GPUs on the node (e.g. Rank 0 -> GPU 0, Rank 1 -> GPU 1, etc.).