Draft: CMake super-builds

Notes:

• The bulk of this feature already works in the master branch and will be part of %DUNE 2.10.0!
• This is a "meta"-MR to gather all sub-tasks needed to achieve this and to explain its benefits.

Summary

The idea is to be able to enable CMake super builds following up on !848 (closed). That is, one top-level CMakeLists.txt that contains all other DUNE projects:

cmake_minimum_required(VERSION 3.24 FATAL_ERROR)
project(dune-core CXX)

# let 'find_package' know where configured projects can be found
list(APPEND CMAKE_PREFIX_PATH ${PROJECT_BINARY_DIR})
# allow dune to see imported targets across modules
set(CMAKE_FIND_PACKAGE_TARGETS_GLOBAL ON)

# include DUNE sub-directories in configuration order
add_subdirectory(dune-common)
add_subdirectory(dune-istl)
add_subdirectory(dune-geometry)
add_subdirectory(dune-grid)
add_subdirectory(dune-localfunctions)

with !1419 this file may even be generated automatically.

Faster Builds!

As an example, I configure and build all the modules for dune-functions, that is:

• dune-common
• dune-istl
• dune-geometry
• dune-uggrid
• dune-grid
• dune-localfunctions
• dune-typetree
• dune-functions

We get the faster times on both build and configuration (Tested on a Apple M3 Max 10+4 cores):

Time	dunecontrol (build+conf per module)	SuperBuild
Configuration	8.8+3.8+4.0+3.1+4.1+3.9+2.9+6.8 = 37.4s	21.4s
Build	4+1+1+5+4+7 = 22.0s	13.5s
Total	59.4s	34.9s

• Configuration is faster because it's 1 big configuration instead of 8 small configuration stages.
  • Note that this approach on global imported targets avoids one of the most consuming part of our build: find packages over and over again!
• Build is faster because all targets from different modules are together and can be built in parallel.

Simpler to use

Since building cmake projects is ubiquitous, it is much easier to teach how to use dune:

# obtain modules (with super build)
git clone --recursive <my-super-build-module>
# prepare a build directory
mkdir build && cd build
# configure the projects
CMAKE_BUILD_TYPE="Release" cmake -G Ninja ..
# build the project
cmake --build .

No need to teach about dune-isms like dunecontrol or the dune options file! For the most part, it just works like any other C++ project.

IDE Integration

Since one multi module project set up is so simple, its set up on Integrated Development Environments is also easy. Compare automatic setup in VSCode when working with super builds vs a 3 stages of manual setup when working the traditional way.

Implementation

The following are the required steps to achieve this:

• Remove pkg-config support. See !1394 (merged)
• Skip including of targets file on the cmake config file. See !1395 (merged)
• Use global targets to handle external libraries (Using CMAKE_FIND_PACKAGE_TARGETS_GLOBAL)
• (Optional) Clean up python targets/variables to allow super builds with python bindings !1412 (merged)
• (Optional) remove superfluous warning messages !1411 (merged)
• (Optional) Helper script to generate super builds on the fly !1419
• (Optional) Fix python installation problems:
  • Use correct scopes on python paths !1417
  • Use correct scopes on python wheelhouse #378
• (Optional) Add CI test to make sure this does not break in the future

Limitations

• Some projects may have collisions on target names (e.g. dune-fem/dune-fem!683 (merged) and extensions/dune-alugrid!182 (closed))
• Installation of python bindings does not work

cmake/modules/AddMPIFlags.cmake

@@ -21,14 +21,11 @@ set_package_properties("MPI" PROPERTIES
   DESCRIPTION "Message Passing Interface library"
   PURPOSE "Parallel programming on multiple processors")
 
-if(MPI_C_FOUND)
+if(TARGET Dune::Imported::MPI_C)
   set(HAVE_MPI ${MPI_C_FOUND})
-
   dune_register_package_flags(
-    COMPILE_DEFINITIONS
-      "HAVE_MPI=1;MPICH_SKIP_MPICXX=1;OMPI_SKIP_MPICXX=1;MPI_NO_CPPBIND=1;MPIPP_H;_MPICC_H"
     LIBRARIES
-      MPI::MPI_C)
+      Dune::Imported::MPI_C)
 endif()
 
 # adds MPI flags to the targets
@@ -36,11 +33,9 @@ function(add_dune_mpi_flags)
   cmake_parse_arguments(ADD_MPI "SOURCE_ONLY;OBJECT" "" "" ${ARGN}) # ignored
   set(targets ${ADD_MPI_UNPARSED_ARGUMENTS})
 
-  if(MPI_C_FOUND)
+  if(TARGET Dune::Imported::MPI_C)
     foreach(target ${targets})
-      target_link_libraries(${target} PUBLIC MPI::MPI_C)
-      target_compile_definitions(${target} PUBLIC
-        HAVE_MPI=1 MPICH_SKIP_MPICXX=1 OMPI_SKIP_MPICXX=1 MPI_NO_CPPBIND=1 MPIPP_H _MPICC_H)
+      target_link_libraries(${target} PUBLIC Dune::Imported::MPI_C)
     endforeach(target)
   endif()
 endfunction(add_dune_mpi_flags)