[added to !932] WIP: set correct cmake dir in module.py for installed dune modules

mentioned in merge request infrastructure/dune-nightly-test!1 (closed)

The assumption is that "installed modules can be found via pkg-config" (see also dunemodules.lib). Why should this be the case? pkg-config searches in default locations, but if the module is installed in a non-default location, how should pkg-config find the .pc file?

The dunecontrol script sets the environmental variable PKG_CONFIG_PATH="$PKG_CONFIG_PATH:$PREFIX_DIR/lib/pkgconfig" with PREFIX_DIR="$(canonicalpath "$0")/..", i.e., one directory above the dunecontrol script.

So, this is different from the module.py file. Neither the PREFIX_PATH nor the PKG_CONFIG_PATH are set. What should be the appropriate values for these?

I remember that we had the same problem with the pip installed module in the virtual env which of course were not found without the PKG_CONFIG_PATH. I remembered that we modified the path in setup-dunepy.py in exactly the way you describe i.e., somehow relavtive to the setup-dunepy.py location. But we removed that again and replaced it with os.path.join(get_local(),'lib','dunecontrol'] where get_local is the path to the virtual env or ~/.local.

One idea could be to add a hint for pkg-config (a path that is added to the PKG_CONFIG_PATH variable in the call to pkg-config). This hint could be filled with a path computed from the location of the dune.module file. If it is an installed module, it is located in something like <prefix>/lib/dunecontrol/<module> and the .pc file is located in <prefix>/lib/pkgconfig. A module in the source dir does not have such a directory structure. So one could distinguish between these two installation types. Or (for the ci-test) I simply set the PKG_CONFIG_PATH to the location where to find the .pc files, manually.

At the moment we seem to be running into many minor problems here and there. I would suggest manually fixing them for now and describing them in a common issue. Once we have an overview and have something working, we can see how to fix them.

Yes, I will first try to set some global directory variables like CMAKE_PREFIX_PATH in the ci tests to at least find everything. And then I'll try to understand better the differences of the module.py and dunecontrol script.

For the tutorial I am just working on using the setup-dunepy script to build the dune-py module before running the tests - there I can pass the config opts file as a parameter similar to dunecontrol and the problem doesn't come up. But it definitely would be good to get the 'create dune-py on first import of a dune package' approach to work without having to define a lot of environment variables. I was always hoping one could add a 'cmakeflags.py.in' file to the 'dune-common/python/dune/common' folder with a '@DUNECMAKEFLAGS@ entry that is replace during the configure step of dune-common - but cmake doesn't seem to export that variable. Of course we could replace individual variables there, i.e., CXX and CXXFLAGS for example (at least I hope that is possible?)

mentioned in merge request !928 (closed)

This is included in !928 (closed)

changed title from WIP: set correct cmake dir in module.py for installed dune modules to [added to !932] WIP: set correct cmake dir in module.py for installed dune modules

mentioned in merge request !932 (merged)

added in !932 (merged)

closed