dune-common issueshttps://gitlab.dune-project.org/core/dune-common/-/issues2024-02-25T22:23:21Zhttps://gitlab.dune-project.org/core/dune-common/-/issues/361Installation of pkg-config / cmake files depend on libdune<module>2024-02-25T22:23:21ZChristoph GrüningerInstallation of pkg-config / cmake files depend on libdune<module>If a Dune module provides a library (like common, geometry, grid), pkg-config and CMake files are installed below the library path (e.g. `lib64`). If a module provides no library (like localfunctions, typetree), the same files go below a...If a Dune module provides a library (like common, geometry, grid), pkg-config and CMake files are installed below the library path (e.g. `lib64`). If a module provides no library (like localfunctions, typetree), the same files go below a different lirary path (e.g. `lib`).
Caused by this part of our code:
https://gitlab.dune-project.org/core/dune-common/-/blob/master/cmake/modules/DuneProject.cmake#L246-L255
Is this really our intention? Why not always using one of the two variables `CMAKE_INSTALL_LIBDIR` or `DUNE_INSTALL_NONOBJECTLIBDIR`?DUNE 2.10.0https://gitlab.dune-project.org/core/dune-common/-/issues/352DUNE ENV 2.02024-02-22T15:19:29ZRobert KDUNE ENV 2.0The desire is to build DUNE without automatic virtual env creation and pip install.
Related to
- !222.
- !960
- !1104The desire is to build DUNE without automatic virtual env creation and pip install.
Related to
- !222.
- !960
- !1104https://gitlab.dune-project.org/core/dune-common/-/issues/345Document how to make interdependent targets on different namespaces2023-10-05T11:35:07ZSantiago Ospina De Los Ríossospinar@gmail.comDocument how to make interdependent targets on different namespacesThe following discussion from !1247 should be addressed:
- [ ] @simon.praetorius started a [discussion](https://gitlab.dune-project.org/core/dune-common/-/merge_requests/1247#note_129022): (+13 comments)
> Unfortunately, this does...The following discussion from !1247 should be addressed:
- [ ] @simon.praetorius started a [discussion](https://gitlab.dune-project.org/core/dune-common/-/merge_requests/1247#note_129022): (+13 comments)
> Unfortunately, this does not work properly. If one target depends on the other target (e.g. via `target_link_libraries`), the dependency must be included before. Otherwise I get the fatal error "The following imported targets are referenced, but are missing..." automatically generated in the target file. Not sure how to fix this properly. I have to create the libraries in the right order, I think, and then it works. But this is not the way to want to resolve dependencies.https://gitlab.dune-project.org/core/dune-common/-/issues/332How to add include directories to dune targets2024-02-23T16:12:30ZSantiago Ospina De Los Ríossospinar@gmail.comHow to add include directories to dune targets### Issue
In order to move to a target based build system, our targets need to be equipped with the necessary include directories so that we can stop relying on project-wise include commands (i.e. `target_include_directories` vs `includ...### Issue
In order to move to a target based build system, our targets need to be equipped with the necessary include directories so that we can stop relying on project-wise include commands (i.e. `target_include_directories` vs `include_directories`). Currently, we are using `include_directories` in `dune_project()` so that all the targets in the project see the header files of all dependencies and the project itself. This needs to be removed in favor of the target based approach (i.e. `target_include_directories`). The question here is: what is the best alternative to enforce `target_include_directories` in all dune modules?
### Alternatives
_@simon.praetorius made a great summary in https://gitlab.dune-project.org/core/dune-common/-/merge_requests/1249#note_127842_
There are some ideas how to move the include_directories property from a global property into targets:
1. Manually setting the include dir to a target. Pro: it is explicit and works right now. Con: Every module needs to create a library. It is quiet a complicated cmake statement to get right. It should actually be something like
```cmake
target_include_directories(<target> [INTERFACE]
$<BUILD_INTERFACE:${PROJECT_BINARY_DIR}>
$<BUILD_INTERFACE:${PROJECT_SOURCE_DIR}>
$<INSTALL_INTERFACE:${CMAKE_INSTALL_INCLUDEDIR}>)
```
2. Extending the function `dune_project(<target> [INTERFACE]...)` by additional arguments to directly create a module library and set the include directories on this. Pro: expects only little change in the code, can be backwards compatible. Con: now `dune_project` has two functions, configuring the module and creating a library. See !944
3. Adding a helper function just for the purpose of semiautomatically configuring a module library, e.g., `dune_add_module_library(<target> [INTERFACE]...)`. Pro: clear separation of functionality `dune_project` vs. `dune_add_library` but still some automatism. Con: Not yet clear about backwards compatible implementation.CMake ModernizationSimon PraetoriusSimon Praetoriushttps://gitlab.dune-project.org/core/dune-common/-/issues/330[discussion] drop or simplify pip & venv magic during configure2023-04-27T14:33:51ZChristian Engwer[discussion] drop or simplify pip & venv magic during configureAs we have seen in !1235, the automagic handling of python dependencies leads to a couple of strange problems, e.g. packages installed automatically by `cmake` are not always found.
I would like to start a discussion on how we can simpl...As we have seen in !1235, the automagic handling of python dependencies leads to a couple of strange problems, e.g. packages installed automatically by `cmake` are not always found.
I would like to start a discussion on how we can simply this whole cmake-venv-pip integration. My provocative statement is that it leads to more problems than it solves and that a clear error message about missing dependencies would be far more helpful.
Opinions?https://gitlab.dune-project.org/core/dune-common/-/issues/313Python virtual environment depends on dune-common python package2023-03-01T11:23:17ZSantiago Ospina De Los Ríossospinar@gmail.comPython virtual environment depends on dune-common python packageCurrently, the [`run-in-dune-env.sh`](https://gitlab.dune-project.org/core/dune-common/-/blob/5dcb1e2b96d99e758ad06894218b8ba786725e15/cmake/scripts/run-in-dune-env.sh.in) script uses the dune-common python module to check for something ...Currently, the [`run-in-dune-env.sh`](https://gitlab.dune-project.org/core/dune-common/-/blob/5dcb1e2b96d99e758ad06894218b8ba786725e15/cmake/scripts/run-in-dune-env.sh.in) script uses the dune-common python module to check for something on the build dirs (I don't know what). This is problematic because the venv may be used in downstream modules without necessarily assuming that dune-common python package is installed (e.g. `dune-testtools`, `dune-codegen`).
A simple fix is to conditionally check if the python package is available:
```bash
# check if dune-common is installed
if python -m dune --help &> /dev/null; then
# test if build directory matches installed dune python packages
python -m dune checkbuilddirs @PROJECT_NAME@ @CMAKE_BINARY_DIR@
RESULT=$?
if [ $RESULT -ne 0 ] ; then
echo "Dune python package could not check build directories"
exit $RESULT
fi
fi
# execute the command
"$@"
```
But perhaps someone with more insight of `checkbuilddirs` can give a more appropriate solution.Samuel Burbullasamuel.burbulla@mathematik.uni-stuttgart.deSamuel Burbullasamuel.burbulla@mathematik.uni-stuttgart.dehttps://gitlab.dune-project.org/core/dune-common/-/issues/281[python] install pybind11 from pypi2023-03-01T11:23:17ZAndreas Dedner[python] install pybind11 from pypiCurrently we ship the `pybind11` headers with `dune-common` which means they have to be upgraded occasionally. A possible approach would now be to add `pybin11==version` to the `Python-requires` list in `dune.module` which would install ...Currently we ship the `pybind11` headers with `dune-common` which means they have to be upgraded occasionally. A possible approach would now be to add `pybin11==version` to the `Python-requires` list in `dune.module` which would install a working version of `pybind11` into the venv. The location of the headers ends up being `pythonEnv/lib/pythonX.X/site-packages/pybind11/include/pybind11`. So this approach would require adding that to the general include path (also for `dune-py`).
After installation this path is the output of calling `python -m pybind11 --includes`.https://gitlab.dune-project.org/core/dune-common/-/issues/279Python packages should not be installed during configure and make2023-04-09T14:02:35ZTimo KochPython packages should not be installed during configure and makeThe usual procedure of installing C++ based projects is configure, make, install. With the new Python bindings enabled per default there some IMO __unexpected behaviour__:
1. During `configure` (e.g. `dunecontrol configure`) Python pack...The usual procedure of installing C++ based projects is configure, make, install. With the new Python bindings enabled per default there some IMO __unexpected behaviour__:
1. During `configure` (e.g. `dunecontrol configure`) Python packages (like `jinja2`, `numpy`) are silently installed into the virtual environment (external if a venv activated, internal in dune-common if we are not in a venv). This doesn't even produce output.
2. During `make all` (e.g. `dunecontrol make`) Python packages are installed into the virtual environment. This time it's the Dune packages depending on the Python bindings.
3. During `make install_python` packages get installed globally (system-wide) even if a virtual env is activated. EDIT: This only happens when the internal venv is used.
__Expected behaviour__:
1. `configure` only configures (and doesn't create a venv)
2. `make all` only builds (also building Python bindings (the C++ part) that are enabled per default now)
3. `make install_python` installs Python packages (system-wide may be the default but if a venv is actived it would be nice to install it in there per default)
4. `make install_python_editable` installs Python packages editable instead (would be nice to have)
__Some current issues__
1. It currently doesn't seem possible to just configure and build without installing Python modules. This requires an internet connection (ok it prints some warnings if there is no internet and skips it I think). I might want to just build my code locally without an internet connection and compile some C++ stuff. Why do I need to install Python bindings for that? Or start my own virtual env after running configure. Then the packages are installed in a now useless internal venv. Why not wait?
2. Or in a CI pipeline, I might want to separate Python and C++ testing in different build jobs where the C++ job doesn't need to know anything about the Python job.
3. I might want to install my Python modules system-wide with `make install_python`. However, per default everything always gets installed editable inside the internal venv during `make` already, although this is completely unnecessary in that case.
4. In the case of internal venv the build dependency / flow of information is inverted. Usually, downstream modules depend on upstream modules. Now, the dune-common build directory contains information from all downstream modules.
5. Dune now (at least on Ubuntu) requires to install `python-venv` otherwise the dunecontrol fails.
Building the Python C++ binding modules per default is of course ok. But why is the __installation__ of Python packages not optional anymore (like in Dune 2.8)? Also is there a reason not to delay the installation until installed Python packages are actually needed?
__Suggestion__:
1. Don't create an internal virtual env during configure or build/make. If an external venv is activated, use that as default installation destination, if not, install system-wide per default (like for C++) ((_alternatively, default-install into an internal documented location like now but then `make install_python` should create the venv and install there and not `configure` or `make all`_))
2. Configure only configures (includes configuration of Python packages which depend on CMake information), and one can pass an installation path for the Python packages (if you don't want to install system-wide or in an external activated venv)
3. `make all` only builds libraries (no C++ tests, and no Python installation)
4. `make install_python_editable` installs Python packages into configured destination in editable mode using `pip`
5. `make install_python` installs Python packages into configured destination in normal mode using `pip`
6. `make install`? I have no idea what this currently does in combination with Python packages.
I think this would make things both simpler to implement, more transparent, consistent and expected.
__Question__:
Is the following workflow already possible (by setting some variables) but I just don't know it?:
1. disable creation of internal venv and provide path to external venv via CMake (that is not activated yet)
2. run `dunecontrol all` which _only_ configures and builds _without_ installing like in 2.8 with Python bindings enabled
3. run some install command that installs Python modules and depedencies in provided path (possibly in editable mode)
I think this is at least one reasonable workflow that should be supported.
__User perspective: suggestion vs status quo__:
I think in the minimal case the following would be possible: (Assuming I cloned all modules into a directory)
* Currently (master), to run a python script with dune is as simple as:
- `./dune-common/bin/dunecontrol --opts=my.opts all`
- `source ./dune-common/build-cmake/dune-env/bin/activate`
- `python my_python_test.py`
* With what is suggested here, I believe, the same could be achieved with:
- `./dune-common/bin/dunecontrol --opts=my.opts all`
- `./dune-common/bin/dunecontrol make install_python_editable`
- `python my_python_test.py`
__Related__:
* Even with DUNE_ENABLE_PYTHONBINDINGS=0 a virtual env is installed. Would be nice to have one variable to turn off the Python stuff. (also see #293)https://gitlab.dune-project.org/core/dune-common/-/issues/264Excessive Warnings/Messages for optional module dependencies2023-03-13T13:25:57ZTimo KochExcessive Warnings/Messages for optional module dependenciesThe current ~~CMake system~~ `dunemodules.lib` script prints a "scary" warning if an __optional__ dependency is not found. As an example, installing `dumux` with only the Dune core modules (which is a perfectly functioning setup) gives r...The current ~~CMake system~~ `dunemodules.lib` script prints a "scary" warning if an __optional__ dependency is not found. As an example, installing `dumux` with only the Dune core modules (which is a perfectly functioning setup) gives right at the beginning of `dunecontrol`:
<details>
<summary>See warnings</summary>
```shell
WARNING: could not find module 'dune-uggrid',
module is also unknown to pkg-config.
Maybe you need to adjust PKG_CONFIG_PATH!
'dune-uggrid' is suggested by dune-grid
Skipping 'dune-uggrid'!
WARNING: could not find module 'dune-alugrid',
module is also unknown to pkg-config.
Maybe you need to adjust PKG_CONFIG_PATH!
'dune-alugrid' is suggested by dumux
Skipping 'dune-alugrid'!
WARNING: could not find module 'dune-foamgrid',
module is also unknown to pkg-config.
Maybe you need to adjust PKG_CONFIG_PATH!
'dune-foamgrid' is suggested by dumux
Skipping 'dune-foamgrid'!
WARNING: could not find module 'dune-uggrid',
module is also unknown to pkg-config.
Maybe you need to adjust PKG_CONFIG_PATH!
'dune-uggrid' is suggested by dumux
Skipping 'dune-uggrid'!
WARNING: could not find module 'dune-functions',
module is also unknown to pkg-config.
Maybe you need to adjust PKG_CONFIG_PATH!
'dune-functions' is suggested by dumux
Skipping 'dune-functions'!
WARNING: could not find module 'opm-common',
module is also unknown to pkg-config.
Maybe you need to adjust PKG_CONFIG_PATH!
'opm-common' is suggested by dumux
Skipping 'opm-common'!
WARNING: could not find module 'opm-grid',
module is also unknown to pkg-config.
Maybe you need to adjust PKG_CONFIG_PATH!
'opm-grid' is suggested by dumux
Skipping 'opm-grid'!
WARNING: could not find module 'dune-subgrid',
module is also unknown to pkg-config.
Maybe you need to adjust PKG_CONFIG_PATH!
'dune-subgrid' is suggested by dumux
Skipping 'dune-subgrid'!
WARNING: could not find module 'dune-spgrid',
module is also unknown to pkg-config.
Maybe you need to adjust PKG_CONFIG_PATH!
'dune-spgrid' is suggested by dumux
Skipping 'dune-spgrid'!
WARNING: could not find module 'dune-mmesh',
module is also unknown to pkg-config.
Maybe you need to adjust PKG_CONFIG_PATH!
'dune-mmesh' is suggested by dumux
Skipping 'dune-mmesh'!
```
</details>
^ Edit: The above messages have been fixed in !1018. The following issues remain:
**The following happened many times already: A new user installs a user module (for example `dumux`). They see these warnings. They think they did something wrong. They start installing _all_ optional dependencies. These optional dependencies often have other optional dependencies, so they end up installing 20 modules. Some of these are even outdated and they end up with a broken installation.**
I personally agree with these users that this all-caps warning is definitely over the top and optional is not even mentioned there. I think from a user perspective something like
```
-- Optional dependency 'dune-uggrid' not found. Installing 'dune-uggrid' is optional.
Installing 'dune-uggrid' will enable the use of unstructured (parallel) grids with multiple geometry types.
For more information, visit https://dune-project.org/modules/dune-uggrid/.
```
The detailed message would of course require that such a help message and a documentation website is registered for a given module. As a first step
```
-- Optional dependency 'dune-uggrid' not found. Installing 'dune-uggrid' is optional.
```
would already be much nicer and may not create the same panic state of having done something wrong as the current message.
**The same goes for CMake feature tests.** If no module-specific feature tests have been found there is loads of scary messages that report on not having found some cmake script in that module and all the places where it searched. Not having module-specific feature tests is completely harmless and very common for user modules, so I don't think there should be any message about this.
**Dune dependencies** Below is an example of *excessive* output in a user module configure step. This recursively lists all the required and suggested Dune module dependencies. This output seems to be useful in case there is some version mismatch. In particular, also notice the scary output for opm-common although this package is completely optional (and not even used/supported by the user module, just a suggestion from an upstream package).
<details>
```shell
Dependencies for dumux-glymphatics: dumux (>= 3.5);dune-common (>= 2.8);dune-foamgrid (>= 2.8);dune-alugrid (>= 2.8);dune-subgrid (>= 2.8)
-- Dependencies for dumux: dune-common (>=2.8);dune-grid (>=2.8);dune-localfunctions (>=2.8);dune-istl (>=2.8)
-- Suggestions for dumux: dune-alugrid (>=2.8);dune-foamgrid (>=2.8);dune-uggrid (>=2.8);dune-functions (>=2.8);opm-common;opm-grid;dune-subgrid (>=2.8);dune-spgrid (>=2.8);dune-mmesh (>=1.2)
-- Dependencies for dune-foamgrid: dune-common (>= 2.8);dune-geometry (>= 2.8);dune-grid (>= 2.8)
-- Dependencies for dune-alugrid: dune-grid (>= 2.9)
-- Dependencies for dune-subgrid: dune-common;dune-geometry;dune-grid
-- Suggestions for dune-subgrid: dune-alugrid;dune-uggrid
-- Dependencies for dune-geometry: dune-common (>= 2.9)
-- Dependencies for dune-grid: dune-geometry (>= 2.9)
-- Suggestions for dune-grid: dune-uggrid (>=2.9)
-- Dependencies for dune-grid: dune-geometry (>= 2.9)
-- Suggestions for dune-grid: dune-uggrid (>=2.9)
-- Dependencies for dune-geometry: dune-common (>= 2.9)
-- Dependencies for dune-grid: dune-geometry (>= 2.9)
-- Suggestions for dune-grid: dune-uggrid (>=2.9)
-- Dependencies for dune-grid: dune-geometry (>= 2.9)
-- Suggestions for dune-grid: dune-uggrid (>=2.9)
-- Dependencies for dune-localfunctions: dune-geometry (>= 2.9)
-- Dependencies for dune-istl: dune-common (>= 2.9)
-- Dependencies for dune-alugrid: dune-grid (>= 2.9)
-- Dependencies for dune-uggrid: dune-common (>= 2.9)
-- Dependencies for dune-alugrid: dune-grid (>= 2.9)
-- Dependencies for dune-foamgrid: dune-common (>= 2.8);dune-geometry (>= 2.8);dune-grid (>= 2.8)
-- Dependencies for dune-uggrid: dune-common (>= 2.9)
-- Dependencies for dune-functions: dune-localfunctions (>= 2.9);dune-grid (>= 2.9);dune-istl (>= 2.9);dune-typetree (>= 2.9)
-- Could NOT find opm-common (missing: opm-common_DIR)
-- No full CMake package configuration support available. Falling back to pkg-config.
-- Found PkgConfig: /usr/local/bin/pkg-config (found version "0.29.2")
-- Checking for module 'opm-common '
-- No package 'opm-common' found
-- Could NOT find opm-grid (missing: opm-grid_DIR)
-- No full CMake package configuration support available. Falling back to pkg-config.
-- Checking for module 'opm-grid '
-- No package 'opm-grid' found
-- Dependencies for dune-subgrid: dune-common;dune-geometry;dune-grid
-- Suggestions for dune-subgrid: dune-alugrid;dune-uggrid
-- Dependencies for dune-spgrid: dune-grid (>= 2.9)
-- Could NOT find dune-mmesh (missing: dune-mmesh_DIR)
-- No full CMake package configuration support available. Falling back to pkg-config.
-- Checking for module 'dune-mmesh>=1.2'
-- No package 'dune-mmesh' found
-- Dependencies for dune-geometry: dune-common (>= 2.9)
-- Dependencies for dune-grid: dune-geometry (>= 2.9)
-- Suggestions for dune-grid: dune-uggrid (>=2.9)
-- Dependencies for dune-alugrid: dune-grid (>= 2.9)
-- Dependencies for dune-uggrid: dune-common (>= 2.9)
-- Dependencies for dune-geometry: dune-common (>= 2.9)
-- Dependencies for dune-grid: dune-geometry (>= 2.9)
-- Suggestions for dune-grid: dune-uggrid (>=2.9)
-- Dependencies for dune-localfunctions: dune-geometry (>= 2.9)
-- Dependencies for dune-istl: dune-common (>= 2.9)
-- Dependencies for dune-typetree: dune-common (>= 2.9)
-- Dependencies for dune-geometry: dune-common (>= 2.9)
-- Dependencies for dune-grid: dune-geometry (>= 2.9)
-- Suggestions for dune-grid: dune-uggrid (>=2.9)
-- Dependencies for dune-uggrid: dune-common (>= 2.9)
-- Dependencies for dune-uggrid: dune-common (>= 2.9)
-- Dependencies for dune-geometry: dune-common (>= 2.9)
-- Dependencies for dune-grid: dune-geometry (>= 2.9)
-- Suggestions for dune-grid: dune-uggrid (>=2.9)
-- Dependencies for dune-uggrid: dune-common (>= 2.9)
-- Dependencies for dune-geometry: dune-common (>= 2.9)
```
</details>
**Alberta example** Lots of output from an optional package. Again, this a transitive suggestion from an upstream module (why do I need to be informed in the downstream configure step?).
<details>
```shell
-- Checking for module 'alberta-grid_1d>=3.0'
-- No package 'alberta-grid_1d' found
-- Checking for module 'alberta-grid_2d>=3.0'
-- No package 'alberta-grid_2d' found
-- Checking for module 'alberta-grid_3d>=3.0'
-- No package 'alberta-grid_3d' found
-- Could NOT find Alberta (set PKG_CONFIG_PATH to include the location of the alberta-grid_[n]d.pc files) (missing: ALBERTA_GRID_PREFIX) (Required is at least version "3.0")
```
</details>
**-- Not enabling torture-tests**
I'm thankful. But yes, alugrid is special...
**Packages not found**
```
Could NOT find ParMETIS (missing: PARMETIS_LIBRARY PARMETIS_INCLUDE_DIR)
```
appears four times in the mentioned user module above. That makes me want to install it. Yet it's not used in the user module at all. Isn't it enough to provide this message in the summary of optional packages not found?https://gitlab.dune-project.org/core/dune-common/-/issues/259document imported moduls from new CMake versions and make them usable wiht on...2023-03-13T13:29:39ZMarkus Blattdocument imported moduls from new CMake versions and make them usable wiht only the information from dune-module-config.cmakeApplies to at least FindPkgConfig and FindPython3.
As it is now users might be up for quite some surprises and maintenance for people other than the authors is quite hard.Applies to at least FindPkgConfig and FindPython3.
As it is now users might be up for quite some surprises and maintenance for people other than the authors is quite hard.DUNE 2.9.0https://gitlab.dune-project.org/core/dune-common/-/issues/256Installed packages causing cmake trouble for source build.2024-03-02T23:44:47ZRobert KInstalled packages causing cmake trouble for source build.On an Ubuntu 20.04 system when the DUNE core packages are installed (e.g. apt install libdune-grid-dev) then the cmake for a master source tool chain fails with the following error:
CMake Warning at /usr/share/dune/cmake/modules/DunePy...On an Ubuntu 20.04 system when the DUNE core packages are installed (e.g. apt install libdune-grid-dev) then the cmake for a master source tool chain fails with the following error:
CMake Warning at /usr/share/dune/cmake/modules/DunePythonTestCommand.cmake:46 (message):
Unparsed arguments in dune_python_add_test: This often indicates typos!
Call Stack (most recent call first):
dune/python/test/CMakeLists.txt:1 (dune_python_add_test)
CMake Error at /usr/share/dune/cmake/modules/DunePythonTestCommand.cmake:54 (message):
dune_python_add_test: no COMMAND to execute specified!
Call Stack (most recent call first):
dune/python/test/CMakeLists.txt:1 (dune_python_add_test)
This means that the cmake files from the master source tree and the system get mixed up. This seem to only occur when Python bindings are enabled (-DDUNE_ENABLE_PYTHONBINDINGS=ON). However, in general it raises the question whether the mix up always occurred and it was simply not noticed.https://gitlab.dune-project.org/core/dune-common/-/issues/254Generated default CMakeLists.txt2023-12-21T23:51:28ZSimon PraetoriusGenerated default CMakeLists.txt1. In the `CMakeLists.txt` file generated by `duneproject` there are some lines that are hard to understand and not documented:
```cmake
if(NOT (dune-common_DIR OR dune-common_ROOT OR
"${CMAKE_PREFIX_PATH}" MATCHES ".*dune-common.*...1. In the `CMakeLists.txt` file generated by `duneproject` there are some lines that are hard to understand and not documented:
```cmake
if(NOT (dune-common_DIR OR dune-common_ROOT OR
"${CMAKE_PREFIX_PATH}" MATCHES ".*dune-common.*"))
string(REPLACE ${CMAKE_PROJECT_NAME} dune-common dune-common_DIR
${PROJECT_BINARY_DIR})
endif()
```
The meaning is as follows:
- If there is no variable `dune-common_DIR` (or `dune-common_ROOT`) is set, and the `CMAKE_PREFIX_PATH` list does not contain
any directory containing the string `dune-common`
- Then guess the build-dir of dune-common by replacing the module name in the module build-dir by the string "dune-common".
- This constructed build-dir is used as directory to find the dune-common package.
This sounds like a wild construction to me and I always wondered why this is actually necessary.
- If we are using `dunecontrol` then the `dune-common_DIR` is explicitly passed to all built modules.
- If we are not using `dunecontrol` we should pass the directories where to find the modules explicitly instead of wild guessing. (in my opinion)
2. The second strange part of the generate `CMakeLists.txt` file is:
```cmake
list(APPEND CMAKE_MODULE_PATH "${PROJECT_SOURCE_DIR}/cmake/modules"
${dune-common_MODULE_PATH})
```
Here we add the cmake directory of the current dune module and the corresponding directory of dune-common to the search path `CMAKE_MODULE_PATH` variable.
- The module path of `dune-common` should be added automatically inside the `find_package` to `CMAKE_MODULE_PATH`, because this is what the package exports.
- The module specific cmake directory is currently necessary, since automatically the cmake file `Dune<Module>.cmake` is included recursively by all dune modules. I'm hoping to remove this automatic inclusion in the future in favor of an extended find-package description.DUNE 2.10.0https://gitlab.dune-project.org/core/dune-common/-/issues/246Use CMake components when installing2023-11-13T12:12:16ZSantiago Ospina De Los Ríossospinar@gmail.comUse CMake components when installing### Description
While modernising my CMake, I realised that dune-common dumps all the installation files into the same bucket. While this is practical because one does not have to think what to install and what not, it is much cleaner t...### Description
While modernising my CMake, I realised that dune-common dumps all the installation files into the same bucket. While this is practical because one does not have to think what to install and what not, it is much cleaner to split installation in different components. This way, the user has much more control over the objects that get installed on its system or package. Doing this is simple: Adding a `COMPONENT` argument to every `install` CMake call.
### Proposal
My proposal would be to have 4 components:
| Component | Description | Example
| ------ | ------ | ------ |
| `Runtime` | Final scripts or binaries product of the module | Executables from application modules |
| `Development` | Header files and build system mandatory to continue downstream development | Header files, cmake macros |
| `Library` | Objects needed for both `Development` and `Runtime` usage in downstream modules | libraries for UGGrid or `libdunecommon` |
| `Documentation` | Documentation files | man pages, Doxygen |
In CMake, objects not assigned to any component are set to `Unspecified` (current behaviour). With the proposed setting, nothing should be in this bucket.
### Usage
Assuming components are set, installation of files in a given component is as simple as:
```bash
# User doesn't want to develop, just to link a final application
cmake --install . --component Library
# User wants to develop a downstream module
cmake --install . --component Library --component Development
# User wants to use final binaries
cmake --install . --component Library --component Runtime
# User wants to use everything (current behaviour)
cmake --install .
```
### Is this backwards compatible?
**Yes!** If no component is given on the command line (current usage), all components are installed. This means that the change is backwards compatible with any installation script out there.
### What next?
Discussion... If there seems to be agreement/interest, I can push a MR.CMake Modernizationhttps://gitlab.dune-project.org/core/dune-common/-/issues/228Proposal: Single build directory build2023-11-13T12:23:50ZChristoph GrüningerProposal: Single build directory buildWe are still lacking a proper idea how to get rid of the repeated CMake configure runs, which is performed for each module and all its dependencies.
Thus, I want to discuss the idea of a single build directory: the single build director...We are still lacking a proper idea how to get rid of the repeated CMake configure runs, which is performed for each module and all its dependencies.
Thus, I want to discuss the idea of a single build directory: the single build directory build.
1. A single build directory.
2. Within the directory, all core modules are built.
3. CMake configure is run for each module, but only once. All CMake variables, targets, Find modules are available for all modules (without re-run or tricky import).
4. The dependency resolution is done before the built (make) starts. It would be enough to check the presence of a modules `dune.module` file.
5. All build artifacts are co-located, e.g., `lib/` contains libdunecommon, libdunegeometry, libdunegrid and so on.
6. Quasi-circular dependency like dune-geometry and dune-localfunctions are not a problem, because whatever module is built first knows from the dependency resolution whether the other module is present or not and can conditionally built features and tests.
7. As the configuration is done together, we can ensure that all modules have the same flags, the same libraries, the same C++ version and so on.
8. It would be possible to have multiple build flavors (different compilers, flags, external dependencies), one per build directory.
9. Distribution as RPM or Deb would just include the source code, similar to the tarball. The user have to set up a build directory and build libdunecommon, libdunegeometry and so on.CMake Modernizationhttps://gitlab.dune-project.org/core/dune-common/-/issues/212Headercheck??2021-03-22T16:27:33ZSimon PraetoriusHeadercheck??What is the "headercheck" feature about? In the comments it just says: "reimplementation of the old autotools feaure". It seems to be disabled by default.
- Does anyone use this feature?
- How does it work and what does it do? It is n...What is the "headercheck" feature about? In the comments it just says: "reimplementation of the old autotools feaure". It seems to be disabled by default.
- Does anyone use this feature?
- How does it work and what does it do? It is not really documented!
- There are some functions called to remove headers from headercheck. Why is this necessary?
- Is it a necessary "feature" or just a historic inheritance from the autotools build-system?
- Is there a modern alternative?
Guessing wildly, by interpreting the name "headercheck", I would say this feature somehow checks that header files are (syntactically) valid. Since we have a lot of header-only code, that would not be tested otherwise, this could be useful. If this is the intend of that feature, we could replace it with precompiled headers. This would additionally give a spead-up in the compilation time of about 20%. CMake provides simple functionality for this purpose. See https://gitlab.dune-project.org/simon.praetorius/dune-cmake/issues/3 for a discussion of this feature.New CMake Build-Systemhttps://gitlab.dune-project.org/core/dune-common/-/issues/198Status of TBB2023-09-20T07:17:05ZNils-Arne DreierStatus of TBBI'm planing to use TBB for thread parallelism in a ISTL based project. I'm willing to prepare a MR in dune-istl when I introduced all the TBB calls. But I'm wondering what is the actual status of TBB in dune?
The build infrastructure is ...I'm planing to use TBB for thread parallelism in a ISTL based project. I'm willing to prepare a MR in dune-istl when I introduced all the TBB calls. But I'm wondering what is the actual status of TBB in dune?
The build infrastructure is there but it is not used in the core modules.
On the 2017 developer meeting was a discussion about that: https://dune-project.org/community/meetings/2017-03-devmeeting/threading-infrastructure/
That is what was agreed on:
> We could introduce an abstraction layer. But that only makes sense if we have at least a rough idea regarding the possible variability of the interface. We do not know about any other possible interface. And wrapping will in some cases be quite expensive (support wise)
and
>We standardise on TBB without abstraction/wrappers. TBB remains optional: if not available there will be not automatic threading in the core modules. Infrastructure that only makes sense in connection with TBB may be missing then. - We do not allow OpenMP in core modules. Users may use OpenMP in their own modules. - We generally disallow threads not managed by the thread pool. In special cases you may start threads outside (e.g. for communicating in the background), but you need to ensure that there are no adverse effects, e.g. performace loss due to oversubscription. - When there is equivalent functionality in TBB and the C++ standard, prefer the standard functionality.
Does that mean, that whenever I want to use TBB in the core modules if have to provide a fallback like
```c++
#if HAVE_TBB
tbb::parallel_for(..., [](...){
...
}):
#else
for(...){
...
}
#endif
```
I think that would lead to very ugly code. Should we provide at least convenient functions for the most used cases like `parallel_for`?https://gitlab.dune-project.org/core/dune-common/-/issues/188Fix broken pkg-config files2021-03-10T12:55:38ZCarsten Gräsergraeser@math.fau.deFix broken pkg-config filesCurrently using dune by installing it has several issues:
* 'pkg-config' support is broken: While 'module.pc' files are installed, using 'pkg-config' to determine the flags needed to build against dune are incomplete.
* Documentation i...Currently using dune by installing it has several issues:
* 'pkg-config' support is broken: While 'module.pc' files are installed, using 'pkg-config' to determine the flags needed to build against dune are incomplete.
* Documentation in 'dunecontrol': 'dunecontrol --help' does not tell you anything about how to install and how to specify an installation prefix.
* Documentation on https://dune-project.org/doc/installation
* While '-DCMAKE_INSTALL_PREFIX=...' in contained in a code example it's not explained what it does.
* In contrast to this documentation you have to specify the option file when calling 'dunecontrol make install'
* The documentation still mentions 'autogen'
All of this makes it hard to build code based on dune if you are neither using the dune build system nor cmake. Pkg-config would be a way out but is currently broken.https://gitlab.dune-project.org/core/dune-common/-/issues/173DuneMacros.cmake: find_dune_package version check differs from version check ...2021-04-08T21:59:39ZAnsgar Burchardtansgar.burchardt@tu-dresden.deDuneMacros.cmake: find_dune_package version check differs from version check in dunecontrolWhile looking at dune-localfunctions!117 we noticed that our buildsystem implements two different version checks:
- dunecontrol uses `check_version` (in `lib/dunemodules.lib`)
- cmake has a version check in `find_dune_package` (in `cmak...While looking at dune-localfunctions!117 we noticed that our buildsystem implements two different version checks:
- dunecontrol uses `check_version` (in `lib/dunemodules.lib`)
- cmake has a version check in `find_dune_package` (in `cmake/modules/DuneMacros.cmake`)
These version checks differ in several ways:
- dunecontrol allows multiple constraints (e.g. `dune-common (>= 2.7 && < 2.8)`), cmake only one
- dunecontrol implements `<`, `>`, `=`, `<=` and `>=`; cmake only the last three
In particular the comparison introduced in dune-localfunctions!117 is only parsed as `dune-common (>= 2.7)` in cmake; it also fails if additional whitespace is added (i.e. `dune-common (>=2.7&&<2.8)` works, but `dune-common (>= 2.7 && < 2.8)` results in a cmake error)Simon PraetoriusSimon Praetoriushttps://gitlab.dune-project.org/core/dune-common/-/issues/162dune_add_test(TARGET ...) unconditionally removes the target from `make all`2023-03-13T14:00:20ZJö Fahlkejorrit@jorrit.dedune_add_test(TARGET ...) unconditionally removes the target from `make all`This is a problem if my unit test just consists of running my main program with e.g. some particular `.ini` file and then checking the resultThis is a problem if my unit test just consists of running my main program with e.g. some particular `.ini` file and then checking the resulthttps://gitlab.dune-project.org/core/dune-common/-/issues/161Proper usage of add-all-flags macros2021-09-10T23:27:18ZCarsten Gräsergraeser@math.fau.deProper usage of add-all-flags macrosWhen updating some part of the dune-functions build system the following questions
regarding the various add-all-flags approaches came up and
I could not find satisfactory answers in the documentation:
* Should `target_link_dune_default...When updating some part of the dune-functions build system the following questions
regarding the various add-all-flags approaches came up and
I could not find satisfactory answers in the documentation:
* Should `target_link_dune_default_libraries()` still be used
or should it be deprecated in favour of `dune_target_enable_all_packages()`?
* Should the `src/` example created by `duneproject` still call
`target_link_dune_default_libraries()` altough
`dune_enable_all_packages()` is called by default?
* Should the call to `add_dune_all_flags()` in `dune_add_test()`
be replaced by `dune_target_enable_all_packages()` such that
tests are build similarly to normal executables?
* Should a library module (like, e.g. dune-functions) use
`dune_enable_all_packages()`? Or is this discouraged,
because it has to be used in all derived modules?
* Do we allow building anything without using one of the
add-all-flags approaches at all?
In view of possible binary incompatibility it looks like
"_All tests use it_" implies "_All libraries have to use it_"
which implies "_All applications have to use it_"