Executor¶

Overview¶

NeoFOAM uses the MPI+X approach for parallelism, where X is the execution space used for device parallelism. The Executor class uses Kokkos, provides an interface for memory management, and specifies where to execute the operations:

SerialExecutor: run on the CPU with MPI
CPUExecutor: run on the CPU with either OpenMP or C++ Threads in Combination and MPI
GPUExecutor: run on the GPU with MPI

Design¶

One of the design goals is the ability to easily switch between different execution spaces at run time, i.e. enabling the ability to switch between CPU and GPU execution without having to recompile NeoFOAM. This is achieved by passing the executor as an argument to the container types like Field as shown below.

NeoFOAM::GPUExecutor gpuExec {};
NeoFOAM::CPUExecutor cpuExec {};

NeoFOAM::Field<NeoFOAM::scalar> GPUField(gpuExec, 10);
NeoFOAM::Field<NeoFOAM::scalar> CPUField(cpuExec, 10);

The Executor is a std::variant

using executor = std::variant<CPUExecutor, GPUExecutor, SerialExecutor>;

and allows to switch between the different strategies for memory allocation and execution at runtime. We use std::visit to switch between the different strategies:

NeoFOAM::SerialExecutor exec{};
std::visit([&](const auto& exec)
           { Functor(exec); },
           exec);

that are provided by a functor

struct Functor
{
    void operator()(const SerialExecutor& exec)
    {
        std::cout << "SerialExecutor" << std::endl;
    }

    void operator()(const CPUExecutor& exec)
    {
        std::cout << "CPUExecutor" << std::endl;
    }

    void operator()(const GPUExecutor& exec)
    {
        std::cout << "GPUExecutor" << std::endl;
    }
};

The visit pattern with the above functor would print different messages depending on the executor type. To extend the library with the additional features the above functor design should be used for the different implementations.

One can check that two operators are ‘of the same type’, i.e. execute in the same execution space using the equality operators == and !=.