[feature][Python] Make use of venv optional. Default is ON.

added 1 commit

2d828ce1 - [feature][Python] Make use of venv optional. Default is ON.

added 3 commits

2d828ce1...44024ca3 - 2 commits from branch master
b20b1032 - [feature][Python] Make use of venv optional. Default is ON.

@andreas.dedner: This seems to work for me. I'm testing with the full test collection. Do you remember what were the things that did not work in the previous implementation?

@tkoch: Please test this. You were the loudest advocate for this feature at the developer meeting, so I expect some input here. Please also read !1104 (closed).

added 1 commit

c5c9f630 - [feature][Python] Make use of venv optional. Default is ON.

Compare with previous version

@alexander.mueller: You can take a look at this feature. I don't think it will change anything, but it's a good test.

@robert.kloefkorn : Yes thanks, that's true. As mentioned in #328, I thinks this is interesting for docker containers, where venvs are not always needed.

added 1 commit

95e234af - [feature][Python] Make use of venv optional. Default is ON.

Compare with previous version

Great thanks! I gave it a try. The first part goes as expected: I don't get a venv anymore and the Python bindings are still compiled, great!

For the tests, I ran into two problems

The script set-dune-pythonpath only sets the path for the module itself. (Actually, this is probably the correct thing, because we don't know which other modules we might want to combine this with.) Therefore when using it in run-in-dune-env it only sets the PYTHONPATH for that module's Python package which means the tests fail. In set-dune-pythonpath there is a loop over MODULE_PATHS which I don't understand. MODULE_PATHS seems to be always only one path the way it is set. Am I missing something?
Per default they seem to be run with python instead of python3 which on my ancient system happens to point to Python 2.7. Is there a reason we can't use python3? Are there some venvs in which python3 doesn't exist?

One thing which I can now do, which I find great is: configuring and compiling a dune module stack and then afterwards I can run

python3 -m venv venv
source venv/bin/activate
./dune-common/bin/dunecontrol bexec python3 -m pip install python

which means I have separated concerns with respect to configuring and building C++ code and installing Python packages. This is also useful in CI setups or in Docker containers where the two steps can easily be split now. (With this method the tests also work. I assume in that case set-dune-pythonpath is executed which would set one module path but the other modules are still found because they are installed. Of course in this scenario I wouldn't really want the PYTHONPATH to be polluted. Maybe, we could modify the else-if-check in run-in-dune-env to first see if some VENV is active by checking $VIRTUAL_ENV)