dune-common issueshttps://gitlab.dune-project.org/core/dune-common/-/issues2021-03-10T11:47:52Zhttps://gitlab.dune-project.org/core/dune-common/-/issues/199[CMake] Improve module / dependency system2021-03-10T11:47:52ZChristoph Grüninger[CMake] Improve module / dependency systemThe current build system is implemented in CMake and was designed to replace the former system based on Autotools. Especially the modules, variable passing, and dependency handling is bad, as every module has to re-run the tests from all...The current build system is implemented in CMake and was designed to replace the former system based on Autotools. Especially the modules, variable passing, and dependency handling is bad, as every module has to re-run the tests from all direct and indirect dependencysNew CMake Build-Systemhttps://gitlab.dune-project.org/core/dune-common/-/issues/198Status of TBB2023-09-20T07:17:05ZNils-Arne DreierStatus of TBBI'm planing to use TBB for thread parallelism in a ISTL based project. I'm willing to prepare a MR in dune-istl when I introduced all the TBB calls. But I'm wondering what is the actual status of TBB in dune?
The build infrastructure is ...I'm planing to use TBB for thread parallelism in a ISTL based project. I'm willing to prepare a MR in dune-istl when I introduced all the TBB calls. But I'm wondering what is the actual status of TBB in dune?
The build infrastructure is there but it is not used in the core modules.
On the 2017 developer meeting was a discussion about that: https://dune-project.org/community/meetings/2017-03-devmeeting/threading-infrastructure/
That is what was agreed on:
> We could introduce an abstraction layer. But that only makes sense if we have at least a rough idea regarding the possible variability of the interface. We do not know about any other possible interface. And wrapping will in some cases be quite expensive (support wise)
and
>We standardise on TBB without abstraction/wrappers. TBB remains optional: if not available there will be not automatic threading in the core modules. Infrastructure that only makes sense in connection with TBB may be missing then. - We do not allow OpenMP in core modules. Users may use OpenMP in their own modules. - We generally disallow threads not managed by the thread pool. In special cases you may start threads outside (e.g. for communicating in the background), but you need to ensure that there are no adverse effects, e.g. performace loss due to oversubscription. - When there is equivalent functionality in TBB and the C++ standard, prefer the standard functionality.
Does that mean, that whenever I want to use TBB in the core modules if have to provide a fallback like
```c++
#if HAVE_TBB
tbb::parallel_for(..., [](...){
...
}):
#else
for(...){
...
}
#endif
```
I think that would lead to very ugly code. Should we provide at least convenient functions for the most used cases like `parallel_for`?https://gitlab.dune-project.org/core/dune-common/-/issues/197GCC 10 warnings in SIMD code2020-04-26T08:51:26ZChristoph GrüningerGCC 10 warnings in SIMD codeWith a pre-release version of GCC 10 I get a bunch of warnings like the following two. Not sure whether they are false positives or not, I get them with `-Wall`. Is there a fix possible or not needed?
```
[ 27%] Building CXX object dune...With a pre-release version of GCC 10 I get a bunch of warnings like the following two. Not sure whether they are false positives or not, I get them with `-Wall`. Is there a fix possible or not needed?
```
[ 27%] Building CXX object dune/common/test/CMakeFiles/debugalignsimdtest.dir/debugalignsimd_BinaryOpsScalarVector_double.cc.o
In file included from /home/gruenich/dune/complete/dune-common/build-cmake/dune/common/test/debugalignsimd_BinaryOpsScalarVector_double.cc:6:
/home/gruenich/dune/complete/dune-common/dune/common/simd/test.hh: In lambda function:
/home/gruenich/dune/complete/dune-common/dune/common/simd/test.hh:2070:36: warning: ‘void Dune::Simd::UnitTest::checkVector()’ is deprecated: "Call check() instead, and explicitly instantiate " "checkType() and friends instead" [-Wdeprecated-declarations]
2070 | this->template checkVector<W, Rebinds, Prune>();
| ^
/home/gruenich/dune/complete/dune-common/dune/common/simd/test.hh:2057:10: note: declared here
2057 | void UnitTest::checkVector()
| ^~~~~~~~
```
and
> /home/gruenich/dune/complete/dune-common/dune/common/simd/loop.hh:38:20: warning: requested alignment ‘0’ is not a positive power of 2 [-Wattributes]
> /home/gruenich/dune/complete/dune-common/dune/common/simd/loop.hh: In instantiation of ‘class Dune::LoopSIMD<Dune::LoopSIMD<bool, 2>, 5>’:
> /home/gruenich/dune/complete/dune-common/dune/common/simd/test.hh:1585:38: required from ‘void Dune::Simd::UnitTest::checkCond() [with V = Dune::LoopSIMD<Dune::LoopSIMD<char, 2>, 5>]’
> /home/gruenich/dune/complete/dune-common/dune/common/simd/test.hh:1931:19: required from ‘void Dune::Simd::UnitTest::checkNonOps() [with V = Dune::LoopSIMD<Dune::LoopSIMD<char, 2>, 5>]’
> /home/gruenich/dune/complete/dune-common/dune/common/simd/test.hh:1907:21: required from ‘void Dune::Simd::UnitTest::checkType() [with V = Dune::LoopSIMD<Dune::LoopSIMD<char, 2>, 5>]’
> /home/gruenich/dune/complete/dune-common/build-cmake/dune/common/simd/test/looptest_vector_Type_char.cc:12:72: required from herehttps://gitlab.dune-project.org/core/dune-common/-/issues/196Enable Python Bindings with Anaconda2021-01-16T21:10:58ZLasse Hinrichsen-Bischofflh1887@mi.fu-berlin.deEnable Python Bindings with AnacondaWhen building with `-DDUNE_ENABLE_PYTHONBINDINGS=ON`, I ran intro trouble about not finding the Python libraries ("-- Could NOT find PythonLibs (missing: PYTHON_LIBRARIES PYTHON_INCLUDE_DIRS) " [...] "... get python-dev..."). I strongly...When building with `-DDUNE_ENABLE_PYTHONBINDINGS=ON`, I ran intro trouble about not finding the Python libraries ("-- Could NOT find PythonLibs (missing: PYTHON_LIBRARIES PYTHON_INCLUDE_DIRS) " [...] "... get python-dev..."). I strongly suppose this is because I'm using the Anaconda python distribution and Cmake not finding all of it.
I was able to work around this using [this hack](https://stackoverflow.com/a/48241814) but maybe this should be done in a more elegant way.Christoph GrüningerChristoph Grüningerhttps://gitlab.dune-project.org/core/dune-common/-/issues/195CI issue with parallel make and python tests2020-04-02T16:18:18ZAndreas DednerCI issue with parallel make and python testsAdding multiple Python tests which use dune-py for jit compilation leads to errors in the CI. This is probably caused by running them in parallel and some setup of the CI system. We made the following observations:
- it's not reproducib...Adding multiple Python tests which use dune-py for jit compilation leads to errors in the CI. This is probably caused by running them in parallel and some setup of the CI system. We made the following observations:
- it's not reproducible locally or using gitlab-runner locally
- ~~it works fine in some `dune-fem` modules which use the same CI images but runners hosted in Stuttgart~~ It turns out that although DUNE_MAX_TEST_CORES=4 was set the tests in the CI were not actually done in parallel. Needed to add DUNECI_PARALLEL=4 as well. Not sure why since it is not needed for dune-grid for example. So it fails with the Stuttgart runners as well
- combining all tests into one works fine
- the error is not always identical but it seems to be some issue with file permissions - sometimes it simply says that `CMakeCache.txt` is not readable for example. The most typical error seems to be
```
RuntimeError: CMake Error at /usr/share/cmake-3.13/Modules/FindMPI.cmake:1187 (try_compile):
Cannot copy output executable
''
to destination specified by COPY_FILE:
'/duneci/modules/dune-py/dune-py/CMakeFiles/FindMPI/test_mpi_C.bin'
```
which also looks like a permission issue
(see e.g. https://gitlab.dune-project.org/core/dune-common/-/jobs/146364)
Apparently the first process creates the Cmake file correctly (and also passes) while one of the later one tries to execute `Cmake` in `dune-py` but does not have the permissions required. It could be a locking issue but that part of the code seems to work fine locally (also using `gitlab-runner`) and on the Stuttgart runner.https://gitlab.dune-project.org/core/dune-common/-/issues/194dune-py module version2023-04-09T13:52:42ZAndreas Dednerdune-py module versionIn `python/dune/common/module.py` the version number for the `dune-py` module
is hard coded which can lead to mistakes (see discussion in !790 open by @tkoch
A solution would be to use the version number of the `dune-common` module and ...In `python/dune/common/module.py` the version number for the `dune-py` module
is hard coded which can lead to mistakes (see discussion in !790 open by @tkoch
A solution would be to use the version number of the `dune-common` module and add an additional token to allow for changes in structure of the `dune-py` module
The following discussion from !790 should be addressed:
- [x] @tkoch started a [discussion](https://gitlab.dune-project.org/core/dune-common/-/merge_requests/790#note_64090): (+2 comments)
> can this be automatically obtained from somewhere like `dune.module`? Seems like something easily forgot for a release.https://gitlab.dune-project.org/core/dune-common/-/issues/193variablesizecommunicator still uses fixedsize.2021-10-14T15:02:49ZRobert Kvariablesizecommunicator still uses fixedsize.https://gitlab.dune-project.org/core/dune-common/-/issues/192Eigenvalue/Eigenvector computation produces warning.2020-03-17T16:23:17ZRobert KEigenvalue/Eigenvector computation produces warning.During the CI build these warnings come up. I think it's ok to remove the constexpr before the if.
/duneci/modules/dune-common/dune/common/fmatrixev.hh:296:12: warning: 'if constexpr' only available with -std=c++1z or -std=gnu++1z
...During the CI build these warnings come up. I think it's ok to remove the constexpr before the if.
/duneci/modules/dune-common/dune/common/fmatrixev.hh:296:12: warning: 'if constexpr' only available with -std=c++1z or -std=gnu++1z
if constexpr(Tag==EigenvaluesEigenvectors) {https://gitlab.dune-project.org/core/dune-common/-/issues/191Installs an empty directory /usr/local/share/doc/dune-common/comm2020-09-08T07:52:30ZYuri VicInstalls an empty directory /usr/local/share/doc/dune-common/commFreeBSD 12FreeBSD 12https://gitlab.dune-project.org/core/dune-common/-/issues/190Installs bin/dune-ctest which is only for testing2020-03-14T20:43:33ZYuri VicInstalls bin/dune-ctest which is only for testingctest-related executables should not be installed.
FreeBSD 12ctest-related executables should not be installed.
FreeBSD 12https://gitlab.dune-project.org/core/dune-common/-/issues/189move python bindings2020-12-08T08:27:46ZAndreas Dednermove python bindingsIt was decided to move the Python bindings from dune-python into the core modules.
A few things that need to be decided on:
- I would like to keep the folder structure of `dune-foo/python/dune/foo` for the python parts of the bindings a...It was decided to move the Python bindings from dune-python into the core modules.
A few things that need to be decided on:
- I would like to keep the folder structure of `dune-foo/python/dune/foo` for the python parts of the bindings as is otherwise there would have to be a lot of change
- At the moment the C++ parts of the bindings are located in `dune-python/dune/python/common|grdi|...` which made sense for the `dune-python` module. For the core this would mean having `dune-common/dune/python/common`. Is that acceptable? We could decide to switch to `dune-common/dune/common/python` (changing all the included will be a bit painful...)
- As we started `pybind11` was undergoing major changes on a regular basis so we decided to include a copy of `pybind11` in `dune-python/dune/python/pybind11` and updating it when interesting new feature got added to `pybind11`. It's header only about 600K - here is the licence https://github.com/pybind/pybind11/blob/master/LICENSE It simplifies maintenance and users do not have to first install pybind11 (I don't it's packaged but I'm not sure). Can we keep it that way (for now), i.e. include `pybind11` in `dune-common`?
That is all I can think of for nowAndreas DednerAndreas Dednerhttps://gitlab.dune-project.org/core/dune-common/-/issues/188Fix broken pkg-config files2021-03-10T12:55:38ZCarsten Gräsergraeser@math.fau.deFix broken pkg-config filesCurrently using dune by installing it has several issues:
* 'pkg-config' support is broken: While 'module.pc' files are installed, using 'pkg-config' to determine the flags needed to build against dune are incomplete.
* Documentation i...Currently using dune by installing it has several issues:
* 'pkg-config' support is broken: While 'module.pc' files are installed, using 'pkg-config' to determine the flags needed to build against dune are incomplete.
* Documentation in 'dunecontrol': 'dunecontrol --help' does not tell you anything about how to install and how to specify an installation prefix.
* Documentation on https://dune-project.org/doc/installation
* While '-DCMAKE_INSTALL_PREFIX=...' in contained in a code example it's not explained what it does.
* In contrast to this documentation you have to specify the option file when calling 'dunecontrol make install'
* The documentation still mentions 'autogen'
All of this makes it hard to build code based on dune if you are neither using the dune build system nor cmake. Pkg-config would be a way out but is currently broken.https://gitlab.dune-project.org/core/dune-common/-/issues/187add support for std::span2020-02-26T09:58:22ZNils-Arne Dreieradd support for std::spanI'm currently thinking about a dynamic interface for the communication pattern in dune-istl. For that i would like to have a datatype like [std::span](https://en.cppreference.com/w/cpp/container/span), which is proposed for the C++20 sta...I'm currently thinking about a dynamic interface for the communication pattern in dune-istl. For that i would like to have a datatype like [std::span](https://en.cppreference.com/w/cpp/container/span), which is proposed for the C++20 standard. I found this repository on GitHub: https://github.com/tcbrindle/span which is licensed under the permissive BSL-1.0 license, that allows (as far as I know) to distribute parts of the code under GPL.
Do you think its a good idea to include that into the `Dune::Std` namespace like we did for `std::optional` and `std::variant`?
If so, I could prepare a merge request.https://gitlab.dune-project.org/core/dune-common/-/issues/186deprecated header warning not very helpful2020-02-15T14:42:52ZChristian Engwerdeprecated header warning not very helpfulCurrently deprecated headers include a `#warning` statement after the inclusion guards. This means that the warning is triggered only during the first inclusion. This might even be in one of the core modules, in case we have to support t...Currently deprecated headers include a `#warning` statement after the inclusion guards. This means that the warning is triggered only during the first inclusion. This might even be in one of the core modules, in case we have to support the deprecated interface (like in the case of function.hh).
I think it would be better to
- trigger this warning upon every inclusion f the header, this helps keeping track of all places which need to be updated
- add an `#ifndef DONT_WARN_DEPRECATED_HEADER` (or similar) to explicitly disable the warning for a particular `#include` statement and use this t avoid warning in those spots of the core modules, where we still have to use the header.
I can prepare an example patch for dune-common...https://gitlab.dune-project.org/core/dune-common/-/issues/185Add general logging library2020-02-12T17:09:28ZJö Fahlkejorrit@jorrit.deAdd general logging libraryThis came up during the [developer meeting 2020-02-12](https://dune-project.org/community/meetings/2020-02-devmeeting/).
Everyone seems to implement his own logging facility, but everything has drawbacks. Some of the previous/current a...This came up during the [developer meeting 2020-02-12](https://dune-project.org/community/meetings/2020-02-devmeeting/).
Everyone seems to implement his own logging facility, but everything has drawbacks. Some of the previous/current attemps:
- https://gitlab.dune-project.org/flyspray/FS/issues/991
- https://gitlab.dune-project.org/pdelab/dune-pdelab/merge_requests/393
- The DebugStreams in dune-common
maybe survey the existing external logging librarieshttps://gitlab.dune-project.org/core/dune-common/-/issues/184Get rid of Fortran2020-03-13T20:16:47ZChristoph GrüningerGet rid of FortranWe should get rid of Fortran. We can use the C bindings for LAPACK and BLAS. This would reduce the complexity of the current build code.
I know that two arguments are no longer valid against having Fortran
* With Ninja 1.10 Fortran is s...We should get rid of Fortran. We can use the C bindings for LAPACK and BLAS. This would reduce the complexity of the current build code.
I know that two arguments are no longer valid against having Fortran
* With Ninja 1.10 Fortran is supported.
* With LLVM 10 they got a Fortran frontend.
Any thoughts?https://gitlab.dune-project.org/core/dune-common/-/issues/183Dune test macro broken for skipped test2020-01-20T10:11:44ZTimo KochDune test macro broken for skipped testCommit a225b2da42e1007fc25fcade5a97440d190ef2f7 introduces a new handling for setting the test command, which apparently fixes something on Windows. However this commit breaks the following, in my opinion valid usage:
```
dune_add_test(...Commit a225b2da42e1007fc25fcade5a97440d190ef2f7 introduces a new handling for setting the test command, which apparently fixes something on Windows. However this commit breaks the following, in my opinion valid usage:
```
dune_add_test(NAME bla
SOURCES bla.cc
COMMAND ./bla
CMAKE_GUARD HAVE_BLA)
```
If this test is skipped the command will be `./bla` but it is set as `TARGET_FILE` which yields the following error:
```
-- Configuring done
CMake Error at DuneTestMacros.cmake:398 (_add_test):
Error evaluating generator expression:
$<TARGET_FILE:./bla>
```
Even worse, this will also happen if the test is executed via a script -- a very common usage:
```
-- Configuring done
CMake Error at DuneTestMacros.cmake:398 (_add_test):
Error evaluating generator expression:
$<TARGET_FILE:/home/bla/mytestscript.py>
```
For example the Dumux build suite currently fails with dune master on configure because of these cases:
https://git.iws.uni-stuttgart.de/buildbot/#/builders/2/builds/38https://gitlab.dune-project.org/core/dune-common/-/issues/182[release] "make install" needs a run of make doc first2020-01-10T20:05:12ZMarkus Blatt[release] "make install" needs a run of make doc firstI was a bit surprised that a simple make install fails since the pdf files are not built without running "make doc" before:
```
CMake Error at doc/comm/cmake_install.cmake:36 (file):
file INSTALL cannot find
"/home/mblatt/src/dune/du...I was a bit surprised that a simple make install fails since the pdf files are not built without running "make doc" before:
```
CMake Error at doc/comm/cmake_install.cmake:36 (file):
file INSTALL cannot find
"/home/mblatt/src/dune/dune-2.7/dune-common/build-dist/doc/comm/communication.pdf".
Call Stack (most recent call first):
doc/cmake_install.cmake:43 (include)
cmake_install.cmake:68 (include)
```
Is that expected? If it is we should probably mention it somewhere (I have neither found a comment in INSTALL nor on the website).DUNE 2.7.0https://gitlab.dune-project.org/core/dune-common/-/issues/181How to handle building examples?2022-04-25T08:30:54ZChristoph GrüningerHow to handle building examples?In dune-common, dune-grid-howto, and dune-functions we have example applications, that are examples. They are neither the library nor tests. How should we handle building examples?
1. We can build them anyway. Thus wasting CPU cycles an...In dune-common, dune-grid-howto, and dune-functions we have example applications, that are examples. They are neither the library nor tests. How should we handle building examples?
1. We can build them anyway. Thus wasting CPU cycles and user's time to just build useless examples over and over again. (status quo for dune-common)
2. Don't automatically built the examples. This is prone to bit rotting examples, thus not optimal.
3. We can create a target `examples` which builds all examples.
b) We could make an `dune_example` macro which would create an executable, which would be built for target `example` and which would be added as a compile-only test. This would prevent bit-rotting
What do you think?https://gitlab.dune-project.org/core/dune-common/-/issues/180Requirements of eigenvalue computation2021-03-24T16:24:58ZChristian EngwerRequirements of eigenvalue computationThe documentation states that the specialized eigenvalue computation of the 3x3 matrices should work for any symmetric matrix.
@oliver.sander is this algorithm supposed to work for singular matrices, i.e. eigenvalue 0?
A quick test rev...The documentation states that the specialized eigenvalue computation of the 3x3 matrices should work for any symmetric matrix.
@oliver.sander is this algorithm supposed to work for singular matrices, i.e. eigenvalue 0?
A quick test reveals that it does nt work, but perhaps this is intended. In this case the documentation should be updated accordingly.Oliver Sanderoliver.sander@tu-dresden.deOliver Sanderoliver.sander@tu-dresden.de