improve pickling of discrete function and add paraview file reader

Description

There are two things this MR attempts (together with the accompanying branch in dune-grid and dune-common with the same name):

• make the pickling of discrete function easier. In the tutorial we have a checkpointer that is based around the dune-fem version where the grid is first pickled, spaces have to be setup by hand before discrete function data can be retrieved. In this version it is possibly to directly pickle df and that will pickle the space, gridviews and grids automatically as indented by the python pickle module.
• provide a paraview file reader than can directly work on these pickle dumps. This makes it possible to write loss free data files instead of going through the vtk file format. Also higher order dfs are still available and I've added a 'level' slider to the reader to sup-sample the grid within paraview.

Testing

There are a few scripts in dune-fem/vtkreader:

• test.py script for simple testing. Writs a dump file when dump is passed in as command line parameter otherwise the dump file is loaded.
• generate.py script for generating a dbf file that can be read into paraview (that's dune binary format perhaps dpf (dune pickle file) would be better).
• reader.py read in the dbf file from generate.py for testing that the pickling works.
• transform.py can be used in paraview to manipulate the visualization

Reader

To test the reader, run generate.py which should produce the file dump.dbf. For paraview to autoload the reader (located in dune-fem/python/dune/fem/reader) one needs to set the environment variable PV_PLUGIN_PATH. To get the right command run

python -m dune.fem reader

or directly (in .bashrc for example)

export PV_PLUGIN_PATH=`python -m dune.fem readerpath`

Open paraview in the folder containing the generated dbf files which should turn up in the File->Open dialog. Select one of the dbf files. This should hopefully show the grid and some discrete functions.

Changing the value of Level slider should show the function in more detail. This should even work with no existing dune-py folder - the same dune modules need to be available obviously.

In addition a script file (as example transform to use the transform.py script) can be entered into the Transform field which then provides a drop down list transfct which enable a number of different transformations (taking the derivatives for example) build directly on the discrete function.

Small Issues

• A problem is a bug in either pvpython or ufl: pvpython uses it's own stdout/stderr streams which are missing an isatty attribute which the ufl.log module expects. A small change needs to be made:adding a try/except block:

  solution: this is now fixed by replacing the stdout/stderr objects when loading the dunereader (would be nice to
  really fix this). This can be either fixed in ufl, in paraview, or we could make dune-fem work without ufl by testing
  for ufl more thoroughly in the different parts of dune-fem.

• paraview uses it's own python version and the dune modules have to use the same (probably)

  solution: can't fix this

• the pickled objects also read in the dune.generated modules needed but doesn't include a mechanism for jit
  compiling them so they have to be available in the setup where paraview is run. So copying the dumps from a remote
  machine to a local one can cause problems if the required spaces/grids were not compiled on the local machine.

  solution: the source files for the generate cc module are added to the pickle dump and recompiled during load as necessary.

• Something seems to be wrong with the GridListener:

  Warning: Restrict and Prolongation disabled - possibly no HierarchicGrid registered for this GridView Would be nice to be able to use dune.fem's prolongation/restriction functionality from within paraview.

  solution: seems to work now

• There seems to be no way to control in which order modules are imported during pickling. So for example a load from a discrete function might import the generate space module before the grid module.

Implemented Solution: the pickletools module (build in) can be used to extract all modules needed to unpickle a stream (this is not done during pickling). Then an iteration process can be used to find a workable order to load all dune.generated modules (EAFP style) during unpickling.

Additional features

If the reader is of interest we could add additional features like

• add the reader to the dune.fem, i.e., dune-fem/python/dune/reader and provide a flag to dune.__main__ to set PV_PLUGIN_PATH to point to this folder - this could be done in dune.common. We could then have multiple dune readers but then we have to make the file extension module dependent.

• really nice would be a Python filter that allows to enter a ufl expression that is based on the source dfs input from the reader and returns uflFunctions, i.e., to get gradient or curl or something not based on the vtk datasets but on the dune dfs. The issue is to access the dune objects from the reader so far I only know how to access the generated vtk object (UnstructedGrid) but can't access the gridview and dfs in the filter.

  solution: one can provide the name of a python script that provides some transformation function. What is missing in this approach is a way to trigger a modification event when the transform script is changed, e.g., a button that the user can press or a special key press event that triggers the reader to regenerate the data.

• projecting the data to coarser grid levels to make it visualize faster for setup.

• adaptive subsampling either by prescribing a region or by using some error indicator This could be achieved by a combined tesselate\pointData function with a filter vector or callback indicator used in the element for loop to fix the refinement level.

• reading in dumps from parallel runs

• add a reader for dgf files - would be nice to be able to inspect dgf files directly in paraview. We can probably use ALUGrid to read most files but would have to inspect the dgf file to decide which dimgrid/world to use and if to use simplex/cube.

• add vector data (note that due to the use of the subsampling (even for level 0) the cells are always disconnected so handling cell data extra should not be required?

  solution: not needed since paraview always assumes data with dimR>1 is vector valued and if dimRange=3 then allows the use of glyphs - can be done by a transformation as shown in the example.

• pickle of UFLFunctions.

• pickle of FilteredGV

Code part with UFL Problem

The dunereader includes a hacky solution to the problem of the missing isatty attribute, It is also possible to fix this in ufl/log.py as shown here:

# Colors if the terminal supports it (disabled e.g. when piped to
# file)
try: # added
    if sys.stdout.isatty() and sys.stderr.isatty():
        RED = "\033[1;37;31m%s\033[0m"
        BLUE = "\033[1;37;34m%s\033[0m"
        GREEN = "\033[1;37;32m%s\033[0m"
    else:
        RED = "%s"
        BLUE = "%s"
        GREEN = "%s"
except AttributeError: # added
    RED = "\033[1;37;31m%s\033[0m"
    BLUE = "\033[1;37;34m%s\033[0m"
    GREEN = "\033[1;37;32m%s\033[0m"