More precisely, the processor which owns the requesting domain invokes a method that calls the Fortran 90 remesh library (active method). The other processors invoke a method whose role is to synchronize with the active method and to get back a new domain if the Domain Manager decides to split the requesting domain. As the RMI API guarantees that methods invoked from different physical machines execute on different threads, this synchronization uses standard Java thread primitives.
Currently, the RMIs return a Java Object which encapsulates all the arrays required to describe the modified domain and the new one if the requesting domain has been split or merged. It has been shown that moving Objects through RMI is much less efficient than moving primitive arrays, because of the cost of marshaling/unmarshaling Objects instead of primitive arrays [2]. Designing a domain linearization layer to optimize this transfer could be realized if necessary.
Eventually, the thread associated with the Solver code reaches the point where it needs the new mesh. Then, the finalizer of the RPC is called, as another HPF_LOCAL routine. This routine joins with the RMI thread, hence blocks if it happens that the RMI has not completed yet. Then the mesh data are updated with the return parameters. This is the only non-overlapped copy-back overhead, which is unavoidable because all the data related to the old mesh (matrices and solution vectors) are in use up to this point in time.