[mpi-21] Proposal: Dynamically-Sized Messages in MPI-3

[Doug Gregor <dgregor@xxxxxxxxxx>]

As MPI matures, we expect to see more and more libraries building on  
top of MPI to provide additional functionality and bindings for  
different languages, particularly higher-level, object-oriented  
languages like Java, C++, C#, and Python. In these object-oriented  
languages, users expect to be able to transmit objects via MPI.  
Existing libraries already implement this behavior, serializing and de- 
serializing objects as needed.

Unfortunately, MPI makes the receipt of the serialized representations  
of objects unnecessarily hard. The problem is that, in general, the  
receiver cannot know the length of the serialized data before it posts  
the receive. Obvious approaches to solving this problemâe.g., probing  
for the message before receiving itâwork well within a single-thread  
environment but fail within a multi-threaded
environment.

We propose to extend MPI to support receiving messages of arbitrary  
length. In this scheme, the MPI implementation will allocate a buïer  
of the appropriate size and return that buïer to the user containing  
the received data. With this extension, it becomes signiïcantly  
easier for users and library developers to transmit data of arbitrary  
size in a thread-safe manner. Additionally, since the thread-safe  
workarounds for this problem involve the transmission of multiple  
messages (one with a synchronous send), these exten-
sions may improve application performance in languages that require  
object serialization, such as C#, Java, and Python.

The full proposal, intended for MPI 3.0, is attached. Comments always  
welcome!

	- Doug

Attachment: any_size.pdf
Description: Adobe PDF document