1. Stay with what we have now.
2. Get rid of MPI_WINDOW_IN/MPI_WINDOW_OUT (and, perhaps, FENCE), and specify
that each call to MPI_BARRIER, MPI_GET_COUNTER, or MPI_RMW (MPI locks) counts
as an implicit call to WINDOW_IN and/or WINDOW_OUT. There are several
implementation problems to figure out:
(a) How do I associate a synchronization call with "cache coherence" operations
on a particular window? It might be too expensive, on systems with partial
memory coherence, to flush all the cache, or keep track of the locations that
were remotely updated since the last sync. In the current design, a call to
WINDOW_IN/OUT synchronizes the window it has as argument. This works OK with
MPI_BARRIER and for GET_COUNTER, which also have a window (comm) argument; but
may require changes in the current proposal for locks, which are not associated
with windows. This will not work OK if we get rid of windows altogether, since
the user provides no information on which part of memory need to be made
coherent.
(b) How do we know that the current synchronization is "IN" or "OUT"? we will
need to keep track of dirty bits, at least one bit per window that is set when
a PUT to the window occur, and is reset when the window is synchronized. This
may not be too bad. A call to GET_COUNTER generates an implicit WINDOW_IN; a
call to BARRIER generates an implicit WINDOW_IN if there has been a remote put
since last synchornization; it generates an implicit WINDOW_OUT, otherwise.
Getting rid of windows altogehter has the problem, as I explained before, or
making cache coherence operations more expensive.