Skip to content

Draft: Optionally pass source / target ranks to communication DataHandles

Linus Seelinger requested to merge feature/communicators-with-ranks into master

This change passes the source / target rank of a message down to the DataHandle. That information is already available where the DataHandles are called, so it's merely a matter of passing it through. For backwards compatibility I added SFINAE's to still accept DataHandles without the extra argument.

That's what such a DataHandle looks like when used in PDELab's GenEO with algebraic overlap:

    template<typename V>
    struct MultiGatherScatter
    {
      static typename V::value_type gather(const V& a, int i)
      {
        return (*a.localVector_)[i];
      }
      static void scatter (V& a, typename V::value_type v, int i, int source_rank)
      {
        (*a.getVectorForRank(source_rank))[i]=v;
      }
    };

(the change is the optional int source_rank)

The motivation in that case is that we need to gather basis functions from all neighbors. The old interface does not allow storing them independently, since it's never known where data in scatter came from.

There is an alternative proposal by @christi avoiding SFINAE by allowing to pass in custom MessageGather and MessageScatter implementations, which then could provide the above interface to DataHandles, see discussion in MR.

I personally prefer my proposal since the source/target rank info as already there and only needs to be passed through. That should make it cleaner on the user side, and in my view is worth the SFINAE.

@christi 's approach would avoid SFINAE and allow more flexibility if other modifications would have to be made to the DataHandle interface, but requires implementing custom MessageGather and MessageScatter from the user in order to get rank information.

I'm fine with implementing @christi 's approach as well if that's where consensus lies, so I'd like to hear if there are any opinions on this.

Merge request reports