Communication Patterns.
Basis for discussion at the dev meeting
Motivation
- keep data distribution independent from algorithms
- different sources for data distribution:
- Grid + Discretization(MCMGMapper)
- ParMetis
- Global IndexSets
- some distributed fancy matrix format
- ...
Owner - Overlap - Copy semantic
- Owner: disjunct decomposition of DoFs
- Owner ∪ Overlap: DoFs of the local Problem (Domain decomposition)
- Copy: Auxiliary DoFs
Requirements
- Scalar product: sum over owner DoFs
- Operator:
- Overlapping: copies owner DoFs to corresponding remote DoFs
- Nonoverlapping: sums all Dofs up
- ... (?)
- Preconditioner:
- Domain Decomposition (Schwarz like): project to local problem - set all Copy DoFs to 0.
- AMG: ?
Stored Information
- Every local index has a tag
t\in T=(\text{Owner, Overlap, Copy})
and a list of tuplesR\subset \mathbb{N}\times T
containing remote ranks that store this DoF as well and their tags. - If
R
is empty,t
must be Owner -> Don't store anything
Interface
- ScalarProduct:
- Currently in ISTL: mask with 1 on Owners and 0 else
- Better: List of non-Owner indices: iterate in between
- Data Consistency: (Copy|Sum) FROM localTags TO remoteTags
Problems
- Hierarchical indices?!
- Do we want to allow other tags that owner,overlap,copy
Edited by Nils-Arne Dreier