MPI_Type_send/recv

Dick Treumann (treumann@kgn.ibm.com)
Thu, 11 Apr 1996 15:04:17 -0400

There has been discussion of restricting the datatypes allowed
as the target_datatype in MPI_Put etc to "portable types" ie. Those
constructed from a single basic type with offsets expressed as
multiples of the size of that type. There has also been pressure
to find a way to support arbitrary structures as target_datatype
and the proposal for MPI_Type_send and MPI_Type_recv look
reasonable for this purpose, especially because I do not think
they are nearly as complex as page 21 makes them sound. (I refer to
the representation requirements on lines 20-24)
(I consider the problem identified on line 29 for MPI_Send/Recv
and new type MPI_TYPE to be closer to a fatal flaw than to a
added complexity.)

Consider the example of tasks A and B and the intent to do an
MPI_Put from A to B where the tasks are apt to be on machines
of different architecture. The most pleasant way to code task
A is to use MPI_Type_... calls locally to construct a datatype and
then use that locally grown and understood type as the
target_datatype. As long as this is a "portable type" we know
how to locally construct what I will call a "shipable form" that uses
offsets expressed in terms of multiples of basic type size. We
already agree that an MPI_Put will need to convey this "shipable
form" to the target and that somewhere along the way the data
content of the message and the offsets will need to be converted.
My presumption is that an implimentation will use the
MPI_Type_commit as the opportunity to compile this "shipable
form" so it does not need to be built at each Put.

Now suppose we wish the MPI_Put on task A to use a target_datatype
which depends on the byte offsets appropriate for task B. If we
are willing to ask that task B build the "shipable form" and send
it to A via MPI_Type_send/recv then this "shipable form" can
use byte offsets which will never require conversion. It can
include all structure types. This gives us two "shipable forms",
one is portable and the other locale_specific. As long as every
MPI_Put etc. tags each "shipable form" it transmits, the data
conversion/distribution at the target, B, can interpret the datatype
correctly. (This presumes the locale_specific type is the one
built at B and not something built at C and sent to A.)

This gives us two ways to arrive at an MPI_Datatype for use at task A.

1) Local use of MPI_Type_... calls will build a normal local datatype
which will have a either a portable or locale_specific(local)
"shipable form" associated with it after commit, depending on whether
it uses byte offsets.
2) Use of MPI_Type_... calls at B together with MPI_Type_send/recv
to transfer it to A will result in a datatype at A which has a
locale_specific(foreign) "shipable form" and a "foreign" flag which
says it is an error to use it in any context except target_datatype.
(An MPI_Send with such a type is meaningless as is an
MPI_Type_extent etc. Such use is an easily detected error.)

The remaining bits of the puzzle are that an MPI_Type_send of any
datatype will transmit a locale_specific "shipable form". If the
type being used in the MPI_Type_send at B happens to be portable it
is converted to locale_specific before it is sent. This is reasonable
because the only use A will make of it is to route it back to B
as a target_datatype for interpretation on B. The MPI_Type_recv
will mark the datatype it generates as foreign and locale-specific.
It will be a detectable error to use any except a portable or a
locale_specific(foreign) MPI_Datatype as target_datatype on the
MPI_Put etc. In particular, a locale_specific(local) cannot be used.

Comments ????

-- 
Dick Treumann                               POWER Parallel Systems
(Internet) treumann@kgn.ibm.com             IBM  -- Poughkeepsie, NY
(VNET)     TREUMANN at KGNVMC               Tel: (914) 433-7846
(internal) treumann@windsurf.kgn.ibm.com    Fax: (914) 433-8363