MPI handlers need preemption

Dennis Cottel (dennis@rats-b.nosc.mil)
Mon, 6 May 96 14:49:24 PDT

At the April Forum meeting while discussing MPI handlers, I argued that
handlers should act like "interrupt routines" because that is not only
what users need, but what they expect based on the behavior of past and
existing implementations.

At that point, Joel Clark of Intel spoke up to say that in the current
Paragon implementation, NX hrecv handlers don't preempt the running
(computing) process. Instead, he said, the thread dealing with handler
routines is simply scheduled to run like any other thread and at the
same priority as the computing thread.

Since then, we've done some experiments, which Joel has confirmed, that
show that Paragon NX handler code starts running within a few hundred
microseconds. The handler does not wait for the computing process to
voluntarily give up the CPU, nor does the handler wait for the computing
process's time slice to be exhausted.

In fact, it appears from Joel's measurements that in the newest version
of the Paragon OS (R1.4), a separate pthread which has done a blocking
receive also runs "immediately". That is, a user could implement
interrupt handler functionality using a pthread and normal MPI receive
calls.

This preemptive behavior is a requirement if users are to be able
to build layered message passing functionality above MPI. User's
can't get the same functionality any other way.

Since my only experience with message passing handlers has been with NX
on the iPSC/860 and Paragon, could someone describe the handler
capabilities of message passing libraries on other systems and whether
they provide interrupt-like behavior?

--Dennis

Dennis Cottel, dennis@nosc.mil, (619) 553-1645
NCCOSC, RDT&E Division (NRaD), San Diego, CA