Better documentation for copyCopyToAll::copyCopyToAll
The meaning of copyCopyToAll::copyCopyToAll could benefit from some improvement. Witness Christian's mail:
Dear all,
can someone enlighten me?
What is the meaning of copyCopyToAll::copyCopyToAll(...)?
It seems strange to me. We start with a partition of indices according to owner and then we might have copies of indices owned by an other process.
Now, copyCopyToAll is supposed to communicate those indices tagged as copy to all other processes having this index. But what is this supposed to do? a) Assume that the copies don't contain consistent values. Now every process is sending around his own value. Depending on which message arrives last the receiving process gets an arbitrary value. This seems to be wrong and we consequently we should not be allowed to call copyCopyToAll, if the copies are not consistent. b) assume we have consistent copies. Why should be communicate at all? The copies are sent around, but we retrieve the value that we previously sent to an other process.
It seems I misunderstood something dramatically.
Best Christian
PS: the two functions OwnerOverlapCopyCommunication::addOwnerOverlapToAll and OwnerOverlapCopyCommunication::addOwnerCopyToAll are not used in ISTL at all. Is anybody using them outside of the core modules?
And Markus' answer:
Hi,
On Wed, Dec 20, 2017 at 09:35:51PM +0100, Christian Engwer wrote:
can someone enlighten me?
What is the meaning of copyCopyToAll::copyCopyToAll(...)?
exactly what the name says it sends the values marked as copy to all others.
It seems strange to me. We start with a partition of indices according to owner and then we might have copies of indices owned by an other process.
Now, copyCopyToAll is supposed to communicate those indices tagged as copy to all other processes having this index. But what is this supposed to do? a) Assume that the copies don't contain consistent values. Now every process is sending around his own value. Depending on which message arrives last the receiving process gets an arbitrary value. This seems to be wrong and we consequently we should not be allowed to call copyCopyToAll, if the copies are not consistent.
Well, you are missing some patterns. Just assume the partitioning is not static, but you repartition from N to n processors (n>N). For an entity that was previously marked as copy on two adjacent processors but will only be on one process after the repartitioning you might need this. Maybe the entity had less visible neighbours on each process before the repartitioning than before. If you e.g. repartition a matrix, then the some row entries might even need to up added up.
Concerning "not allowed". This is C++ and you are allowed to shoot yourself in the food.
b) assume we have consistent copies. Why should be communicate at all? The copies are sent around, but we retrieve the value that we previously sent to an other process.
Again: Due to repartitioning to fewer processors.
Markus
It seems I misunderstood something dramatically.
Best Christian
PS: the two functions OwnerOverlapCopyCommunication::addOwnerOverlapToAll and OwnerOverlapCopyCommunication::addOwnerCopyToAll are not used in ISTL at all. Is anybody using them outside of the core modules?
The were used in downstream modules in the past and if somebody decides to do additive domain decomposition solvers the might be again.