Core Modules issueshttps://gitlab.dune-project.org/groups/core/-/issues2022-01-26T15:37:35Zhttps://gitlab.dune-project.org/core/dune-grid/-/issues/141python bindings of fresh master build not working2022-01-26T15:37:35ZChristian Engwerpython bindings of fresh master build not working## steps to reproduce
I recreated my stack with the following opts file:
```
# where to search for modules
DUNE_CONTROL_PATH="/home/christi/Uni/Dune/"
# where to put the build directories
BUILDDIR="/home/christi/Uni/Dune/build.default"...## steps to reproduce
I recreated my stack with the following opts file:
```
# where to search for modules
DUNE_CONTROL_PATH="/home/christi/Uni/Dune/"
# where to put the build directories
BUILDDIR="/home/christi/Uni/Dune/build.default"
CMAKE_FLAGS="$CMAKE_FLAGS -DTPMC_PREFIX=~/.local/lib/python2.7/site-packages/tpmc/"
CMAKE_FLAGS="$CMAKE_FLAGS -DDUNE_ENABLE_PYTHONBINDINGS=ON -DBUILD_SHARED_LIBS=TRUE"
# CMAKE_FLAGS="$CMAKE_FLAGS -DCMAKE_BUILD_TYPE=Debug"
CMAKE_FLAGS="$CMAKE_FLAGS -DDUNE_GRID_GRIDTYPE_SELECTOR=ON"
CMAKE_FLAGS="$CMAKE_FLAGS -DCMAKE_CXX_COMPILER=g++ -DDUNE_MAX_TEST_CORES=8"
CMAKE_FLAGS="$CMAKE_FLAGS -DCMAKE_CXX_FLAGS='-fPIC -g -O2'"
```
during build I get informed that a dune python environment is setup in in `/home/christi/Uni/Dune/build.default/dune-python-env/`. Running a simple dune python script:
`test-ug.py`:
```
import math
import dune.common
import dune.grid
import sys
base = "../../../doc/grids/gmsh/"
reader = (dune.grid.reader.gmsh, base+"circle2ndorder.msh")
grid = dune.grid.ugGrid(reader, dimgrid=2)
assert(grid)
print(grid)
sys.exit(0)
```
results in
```
Traceback (most recent call last):
File "/home/christi/Uni/Dune/grid/dune/grid/test/test-ug.py", line 9, in <module>
assert(grid)
AssertionError
```
Apparently no code is generated and I also can't find a `dune-py` module, which I thought would be setup to build the dynamically generated components.
## expected behaviour
* the python venv should include a `dune-py`,
* running the script should generate and compile some additional components,
* ugGrid should be loaded from the gmsh file, and
* the script should exit with code 0https://gitlab.dune-project.org/core/dune-grid/-/issues/140Why is Geometry not copy-assignable?2023-09-28T15:37:11ZSimon PraetoriusWhy is Geometry not copy-assignable?The `Geometry` class has a deleted copy-assignment operator. Why is this operator deleted? I tried to follow the git history but it end in a commit from 9y ago: https://gitlab.dune-project.org/core/dune-grid/-/commit/432cba1ba059ea0beaac...The `Geometry` class has a deleted copy-assignment operator. Why is this operator deleted? I tried to follow the git history but it end in a commit from 9y ago: https://gitlab.dune-project.org/core/dune-grid/-/commit/432cba1ba059ea0beaac615e715a97ea69bc98c7 that was imported from SVN. As far as I can tell, all real implementations from dune-geometry are copy-assignable.
Making a class not copy-assignable results in complicated code when we want to store a geometry in a class, e.g. in the implementation of grid-functions in dune-functions. There, we need to use a `std::optional` or pointer types to construct a new copy (copy construction seems to be still allowed). This has further consequences: The `std::optional<Geometry>` then is also not copyable.
I just want to understand the reasoning behind the deleted copy operator. Maybe it cannot be implemented efficiently (for some reason) in a specific grid manager?https://gitlab.dune-project.org/core/dune-common/-/issues/287dune_python_add_test should allow for MPI runs.2023-02-28T10:17:49ZRobert Kdune_python_add_test should allow for MPI runs.Right now there seems to be no possibility to added MPI_RANKS to a python tests which is needed to ensure stability of the python codes. How difficult is it to add this?Right now there seems to be no possibility to added MPI_RANKS to a python tests which is needed to ensure stability of the python codes. How difficult is it to add this?https://gitlab.dune-project.org/core/dune-common/-/issues/286parallel build of autogenerated python modules getting stuck.2021-12-05T20:28:11ZRobert Kparallel build of autogenerated python modules getting stuck.When running the latest master on a parallel cluster then the build process of autogenerated python modules for DUNE classes gets stuck after the reference elements have been built.
The following code triggers the problem:
```
from du...When running the latest master on a parallel cluster then the build process of autogenerated python modules for DUNE classes gets stuck after the reference elements have been built.
The following code triggers the problem:
```
from dune.grid import cartesianDomain
from dune.grid import yaspGrid as leafGridView
domain = cartesianDomain([0, 0], [1, 1], [20, 20])
gridView = leafGridView(domain, dimgrid=2 )
```
It was still working before patch 04e19bb51d544292c66614f5613e175ca4fa7e55 was added to dune-common.https://gitlab.dune-project.org/core/dune-common/-/issues/285Cannot build dune-common without python3-venv2021-12-07T12:20:55ZOliver Sanderoliver.sander@tu-dresden.deCannot build dune-common without python3-venvDuring a new installation of dune-common I get
-- Found Python3: /usr/bin/python3 (found version "3.9.9") found components: Interpreter Development Development.Module Development.Embed
-- Found pip_/usr/bin/python3: TRUE
...During a new installation of dune-common I get
-- Found Python3: /usr/bin/python3 (found version "3.9.9") found components: Interpreter Development Development.Module Development.Embed
-- Found pip_/usr/bin/python3: TRUE
-- Failed to find the python package virtualenv with interpreter /usr/bin/python3. (missing: DUNE_PYTHON_virtualenv_FOUND)
-- Found venv_/usr/bin/python3: TRUE
-- Building a virtualenv in /home/sander/dune/dune-common/build-cmake/dune-env
-- WARNING: Failed to build a virtual env with pip installed, trying again without pip
-- If you are using Debian or Ubuntu, consider installing python3-venv and / or python-virtualenv
-- Failed to find the python package pip with interpreter /home/sander/dune/dune-common/build-cmake/dune-env/bin/python. (missing: pippresent)
CMake Error at cmake/modules/DunePythonVirtualenv.cmake:241 (message):
dune-common set up a virtualenv, but needs pip to be installed into it.
You can either install it yourself manually activating the virtualenv with
the activate script in your build directory /home/sander/dune/dune-common/build-cmake or you set
the CMake variable DUNE_PYTHON_ALLOW_GET_PIP to allow Dune to use get-pip.py
from https://bootstrap.pypa.io/get-pip.py
Call Stack (most recent call first):
cmake/modules/DunePythonCommonMacros.cmake:118 (include)
cmake/modules/DuneCommonMacros.cmake:51 (include)
cmake/modules/DuneModuleDependencies.cmake:109 (include)
cmake/modules/DuneProject.cmake:120 (dune_process_dependency_macros)
CMakeLists.txt:11 (dune_project)
-- Configuring incomplete, errors occurred!
Installing the Debian package `python3-venv` does indeed seem to solve the issue, but it is still inconvenient: As I don't plan on using Python features in the nearer future I don't expect to be bothered by Python problems.https://gitlab.dune-project.org/core/dune-grid/-/issues/139Is GridView::conforming really needed to be a constexpr?2022-03-10T17:45:37ZRobert KIs GridView::conforming really needed to be a constexpr?Is GridView::conforming really needed as a compile time constant?
In my opinion this feature is enough to be available dynamically, as a method on GridView. For UGGrid this can anyway only be implemented dynamically in my opinion, sinc...Is GridView::conforming really needed as a compile time constant?
In my opinion this feature is enough to be available dynamically, as a method on GridView. For UGGrid this can anyway only be implemented dynamically in my opinion, since it depends on the type of refinement used which is also a dynamic variable.
For ALUGrid it prevents the merge of Simplex and Conforming as types. These only differ by this one bool flag (conforming).https://gitlab.dune-project.org/core/dune-common/-/issues/284[dunecontrol] comma in suggestions list breaks tracking of implicit dependenc...2022-05-12T21:31:54ZChristian Engwer[dunecontrol] comma in suggestions list breaks tracking of implicit dependencies and locations are not properly appended to cmake flagsThe situation was as follows:
- pdelab suggests alugrid
- duneuro uses pdelab
- pdelab adds alugrid to the list of libs to link
- duneuro fails to build, as it didn't find alugrid, but tries to link to it
The problems was an erroneous ...The situation was as follows:
- pdelab suggests alugrid
- duneuro uses pdelab
- pdelab adds alugrid to the list of libs to link
- duneuro fails to build, as it didn't find alugrid, but tries to link to it
The problems was an erroneous suggestion entry in dune-pdelab (see pdelab/dune-pdelab!575). This syntax error was not detected but lead to an inconsistent consideration of dune-alugrid in different modules.
**Solution**: properly check the dune.module syntax in `dunecontrol`.Christian EngwerChristian Engwerhttps://gitlab.dune-project.org/core/dune-common/-/issues/283Missing python-bindings requirements should not lead to a FATAL_ERROR in CMake2021-12-07T12:20:54ZSimon PraetoriusMissing python-bindings requirements should not lead to a FATAL_ERROR in CMake### Summary
If Python is not found, the python bindings are not activated. If, however, during the setup of the python bindings some python requirement cannot be fulfilled (and also not automatically installed), CMake raises a hard error...### Summary
If Python is not found, the python bindings are not activated. If, however, during the setup of the python bindings some python requirement cannot be fulfilled (and also not automatically installed), CMake raises a hard error. This happened, e.g., in a Linux Mint 19 system (based on Ubuntu 18.04) with an unusable pip. Warning are shown and the hard error message indicates how to solve the problem, but for those who do not want to use any python bindings this is annoying, because it is absolutely not needed for the c++ library.
Proposed solution: transform `FATA_ERROR`s in CMake into `WARNING` and deactivate the python bindings instead. Maybe collect all these problems in a variable `DUNE_PYTHON_BINDINGS_REQUIREMENTS_FOUND` that is set to `FALSE` in any of the current `FATAL_ERROR` cases.
Related to #275.
# Edit
The issue is not a hard error during the installation of the requirements for the python bindings. A problem there leads to a warning only. The issue is a hard error when setting up the internal venv and pip is not available.https://gitlab.dune-project.org/core/dune-common/-/issues/280[python] requirements.txt, install_requires, and Python-requires2023-04-15T10:44:53ZTimo Koch[python] requirements.txt, install_requires, and Python-requiresIn `dune_python_install_package` a `requirements.txt` is created for each module. Usually, this file contains the development requirements of a package. How does this relate to what is stated in `Python-requires`? Is this the way to add ...In `dune_python_install_package` a `requirements.txt` is created for each module. Usually, this file contains the development requirements of a package. How does this relate to what is stated in `Python-requires`? Is this the way to add development requirements? And packaging requirements are then stated `pyproject.toml`?
Where would be the right place to add `pylint` and `flake8` in the current setup which are only needed during development?
__UPDATE:__ A `requirements.txt` file is no longer created (see !1062). This resolves the confusion. This means currently only "minimal" required dependencies can be stated (through dune.module's `Python-requires` argument which is propagated to `setup.py`'s `install_requires`, see https://packaging.python.org/discussions/install-requires-vs-requirements/ for what `install_requires` is good for). However, the current default mode when running `dunecontrol all` creates a developer environment with local internal virtual env and editable package installs. Hence, it would be nice to have a way to specify _development_ requirements (e.g. linters or testing frameworks). This is a job that is often accomplished by providing such packages in a `requirement.txt` file (see e.g. https://packaging.python.org/discussions/install-requires-vs-requirements/, https://github.com/yngvem/python-project-structure, https://gitlab.dune-project.org/samuel.burbulla/dune-mmesh/-/blob/master/requirements.txt).
__Idea/suggestion:__ if a `requirement.txt` is found in the configured Python dune module (in the build directory) next to `setup.py`, the `requirements.txt` is used. There are several possibilities what "used" means, the simplest is that `requirements.txt` is considered as `pip install -e -r requirements.txt` when doing an editable installation.https://gitlab.dune-project.org/core/dune-geometry/-/issues/28dune-geometry creates pngs when configuring with CMake2021-12-11T19:02:18ZTimo Kochdune-geometry creates pngs when configuring with CMakeWhen inkscape is found dune-geometry creates pngs at configure time. This is highly unexpected.
I would expect this to occur when I type `make doc`. Probably need a new CMake function for that in dune-common that just adds a custom comma...When inkscape is found dune-geometry creates pngs at configure time. This is highly unexpected.
I would expect this to occur when I type `make doc`. Probably need a new CMake function for that in dune-common that just adds a custom command instead of executing.https://gitlab.dune-project.org/core/dune-common/-/issues/278add function to add a Python package during configure time2023-04-09T14:00:31ZAndreas Dedneradd function to add a Python package during configure timeProblem
-------
See https://gitlab.dune-project.org/quality/dune-testtools/-/merge_requests/142:
the module `dune-testtool` contains a Python package that needs to be installed during configure time so that tests within the module are ru...Problem
-------
See https://gitlab.dune-project.org/quality/dune-testtools/-/merge_requests/142:
the module `dune-testtool` contains a Python package that needs to be installed during configure time so that tests within the module are run correctly.
Solution
--------
A possible solution is to add a cmake function that allows installation of a Python package during configure time (after the internal venv is setup).
Alternative: add a flag to the existing `install_python` function to move installation of the package from the end of the build to the configure phase.https://gitlab.dune-project.org/core/dune-common/-/issues/277test packaging in CI2021-11-25T09:55:52ZAndreas Dednertest packaging in CIIt would be good to add a call to `dunepackaging.py --onlysdist` and then a `pip install` from the generated `tar.gz` either directly to the CI or to the `dune-nightly-test`.It would be good to add a call to `dunepackaging.py --onlysdist` and then a `pip install` from the generated `tar.gz` either directly to the CI or to the `dune-nightly-test`.https://gitlab.dune-project.org/core/dune-common/-/issues/276Add function to python package to check state of cmake variables2021-11-25T09:57:01ZAndreas DednerAdd function to python package to check state of cmake variablesProvide function `dune.packagemetadata.cmakeSetting(flag)` to return the value of a cmake flag during the configuration of a package (assuming it was added to the metadata). With this we could figure out what `HAVE_MPI` was during build ...Provide function `dune.packagemetadata.cmakeSetting(flag)` to return the value of a cmake flag during the configuration of a package (assuming it was added to the metadata). With this we could figure out what `HAVE_MPI` was during build to see if we should require `mpi4py` or, for example, if `albertaGrid` or `petsc` can be used. That was in the past done rather crudely with looking at `CMakeCahce.txt` and `config.h` in `dune-py` but the new metadata mechanism would make this a lot more elegant.https://gitlab.dune-project.org/core/ci-config/-/issues/2Broken CI config2021-11-05T10:16:14ZTimo KochBroken CI configSee https://gitlab.dune-project.org/core/dune-common/-/blob/master/.gitlab-ci.ymlSee https://gitlab.dune-project.org/core/dune-common/-/blob/master/.gitlab-ci.ymlSimon PraetoriusSimon Praetoriushttps://gitlab.dune-project.org/core/dune-common/-/issues/274[Python] Only looking for package in namespace/folder dune2021-11-09T20:57:45ZTimo Koch[Python] Only looking for package in namespace/folder duneThe new Python bindings seem to be looking for modules in the folder or namespace `dune`.
The `dumux` Python bindings are not part of the `dune` namespace packages but implement their own namespace package `dumux`.
Is there a possibility...The new Python bindings seem to be looking for modules in the folder or namespace `dune`.
The `dumux` Python bindings are not part of the `dune` namespace packages but implement their own namespace package `dumux`.
Is there a possibility to allow for this in the new style?https://gitlab.dune-project.org/core/dune-common/-/issues/271[Python] current python package require system mpi2021-11-25T09:53:12ZAndreas Dedner[Python] current python package require system mpiProblem
-------
Currently `mpi4py` is included in the python requirements in `dune-common`. On systems with MPI there were occasional problems with running python scripts where MPI was initialized on the C++ side. Instead we have
```
tr...Problem
-------
Currently `mpi4py` is included in the python requirements in `dune-common`. On systems with MPI there were occasional problems with running python scripts where MPI was initialized on the C++ side. Instead we have
```
try:
from mpi4py import MPI
except ImportError
pass
```
in `dune.common.__init__.py` to avoid this issue. On system without MPI `pip install dune-common` fails since `mpi4py` cannot be build.
Solution
--------
No idea how one can have _optional_ dependencies in `setup.py`https://gitlab.dune-project.org/core/dune-common/-/issues/270New and old way of Python installation and compatibility2021-11-20T13:58:33ZTimo KochNew and old way of Python installation and compatibilityWhat constitutes the new way of installing the python bindings? Is it a breaking change? Is there already documentation (I guess !960).
The master branch in dumux wants to be compatible with dune 2.8 and dune master, therefore both insta...What constitutes the new way of installing the python bindings? Is it a breaking change? Is there already documentation (I guess !960).
The master branch in dumux wants to be compatible with dune 2.8 and dune master, therefore both installation methods should be supported. Is this possible?
Until now the CI did something like this (after running dunecontrol)
```
# Adds all Python modules found in other Dune modules to the PYTHONPATH
./dune-common/bin/dunecontrol bexec "echo -n :\$(pwd)/python >> $(pwd)/pythonpath.txt"
export PYTHONPATH=$PYTHONPATH$(cat pythonpath.txt)
rm pythonpath.txt
# Sets up the dune-py module for JIT compilation of Python binding code
./dune-common/bin/setup-dunepy.py --opts=cmake.opts install
```
This fails with the current dune master. (Fails to import a dune module in Python.) I guess this is expected, because the Python bindings in dumux don't implement all the changes from !960.
I needed to install `python3-venv` on Ubuntu otherwise the installation crashed, is that expected? (dune-2.8 with python works fine without installing `python3-venv`)https://gitlab.dune-project.org/core/dune-common/-/issues/269CMAKE_FLAGS not correctly forwarded to dune-py.2022-01-09T21:21:23ZRobert KCMAKE_FLAGS not correctly forwarded to dune-py.When running dunecontrol for building dune modules then it seems that the setting from config.opts are not correctly forwarded to the build process in dune-py. I tested `-DUSE_OPENMP=OFF` in config.opts. The cmake configuration in dune-p...When running dunecontrol for building dune modules then it seems that the setting from config.opts are not correctly forwarded to the build process in dune-py. I tested `-DUSE_OPENMP=OFF` in config.opts. The cmake configuration in dune-py continues to show `USE_OPENMP=ON`.
Example config.opts:
```
MAKE_FLAGS=-j4
CMAKE_FLAGS="-DCMAKE_CXX_FLAGS=\"-O3 -DNDEBUG\" \
-DUSE_OPENMP=OFF \
-DCMAKE_DISABLE_FIND_PACKAGE_Vc=TRUE \
-DCMAKE_DISABLE_FIND_PACKAGE_LATEX=TRUE"
```https://gitlab.dune-project.org/core/dune-common/-/issues/268Python overhaul problems2022-01-09T21:21:47ZAndreas DednerPython overhaul problemsThis is to collect problems that are coming up and if possible solutions.
**Problems:**
---------
- [x] Problems with cmake cache in dune-py, see #269.This is to collect problems that are coming up and if possible solutions.
**Problems:**
---------
- [x] Problems with cmake cache in dune-py, see #269.https://gitlab.dune-project.org/core/dune-common/-/issues/267mpi4py without mpi on system2021-10-27T09:24:59ZAndreas Dednermpi4py without mpi on systemA large number of warning make the CI difficult to read - most of them are of the form
```
/builds/core/dune-common/dune/common/simd/loop.hh:145: warning: ignoring '#pragma omp simd' [-Wunknown-pragmas]
/builds/core/dune-common/dune/comm...A large number of warning make the CI difficult to read - most of them are of the form
```
/builds/core/dune-common/dune/common/simd/loop.hh:145: warning: ignoring '#pragma omp simd' [-Wunknown-pragmas]
/builds/core/dune-common/dune/common/simd/loop.hh:146: warning: ignoring '#pragma omp simd' [-Wunknown-pragmas]
146 | DUNE_SIMD_LOOP_ASSIGNMENT_OP(%=);
```
see e.g. https://gitlab.dune-project.org/core/dune-common/-/jobs/287920