In a distributed memory system (with no hw support for put)
when a send is posted, the source has to move at least the message header to the destination.
This header will be sent, whether a matching receive is posted or not at the destination. Since very
few systems can tolerate that data sits on the wire forever, the data will be accepted, at some level
of the communication protocol stack, at the receiving node. If communication uses IP, then an
interrupt occurs because the LAN adapter signals incoming data. If communication uses some user-space
driver, then this driver has to poll periodically the wire. In either cases it is not too expensive
to invoke the next layer of the communication protocol when data has arrived. I doubt very much that
any hardware vendor
will accept a design where a badly written program can clog the network. Some form of flow control,
and therefore some form of "offline polling" or of interrupt driven receive will exist in the
communication subsystem. I suspect, in fact, that such "offline" polling does occur in the system
that Greg implemented, even if incoming data is not processed, at the next level of the communication
stack, until an MPI call occurs at the receiving node. The bottom layer of IP either buffers the data
at the receive node or instruct the sender to resend latter.