I saw in the standard that only preemptive threads are=20
considered for use in MPI. I assumed that MPI requires=20
target platforms to provide thread-safe system calls and=20
C runtime libraries (and other libs if used in applications);=20
so, all library calls one makes in applications do not=20
interfere with MPI calls no matter which threads call them.
So, in the perfect world (Windows NT and Solaris approach=20
this by providing thread-safe libc - malloc is perfectly=20
thread-safe in WinNT - I tested myself)=20
there are only two cases: MPI_THREADS_NOT_SUPPORTED,=20
The pathological cases such as use of pthread_barrier=20
can be quite easily avoided by better design decisions.
I worked on making our MPI for WinNT thread-safe,=20
and I did not need any barriers.=20
But, as Mark pointed out, not all vendors provide=20
thread-safe libc. For instance, in AIX, malloc is=20
not thread-safe; I am afraid other vendors also=20
have their problems with libc thread safety.=20
Hence, it might be wise to conduct a study of common
problems with vendor libraries thread-safety and=20
come up with a consistent view on the resulting=20
It is also important, I think, to allow a thread-compliant=20
MPI application to work only with limited number of these=20
cases and return NOT_SUPPORTED for others.=20
Otherwise, one have to provide many versions of=20
MPI calls and also check every time an MPI call made=20
whether the user keeps his word concerning thread usage.
This definitely won't contribute to better performance.
From: Rolf Rabenseifner
Sent: 18 =F4=E5=E2=F0=E0=EB=FF 1997 =E3. 3:33=20
To: Boris V Protopopov
Subject: Re: mpi on a single thread
Hi Boris, Marc and everybody,
> I disagree with this subdivision of MPI implementations, =20
> as well as MPI applications.
Yes, I also do not like more subdivision than necessary,
but I do not want that any MPI implementation must name itsself
less thread-"compliant" than it is, i.e. I do not want that any
MPI implementation must name itsself wit "MPI_THREADS_NOT_SUPPORTED"
if it can support the usage of threads by the application in some
First to your final statement:=20
> In my opinion, there are only two choices:
> thread-safe MPI implemetation (MPI_MLTIPLE_THREADS) and=20
> non-thread-safe MPI implementaiton (MPI_THREADS_NOT_SUPPORTED)
> because if one uses MPI calls only in one thread, or if MPI calls
> occure in different threads but sequentially can be handled without
> problems by a non-thread-safe MPI implementation.
I want to remember on Eric Salo's Mail:
| Date: Wed, 5 Feb 1997 15:50:33 -0800
| > A related question: are there any impelementations where
| > multithreading is unsafe even when only one thread is making
| > MPI calls?
| Our 2.0 release suffered from this problem, but that has been fixed
| in 3.0 so it's a moot point.
For example a thread-based MPI-implementation that uses internally
pthread_barrier and an application that calls MPI only from the
main thread, but uses other threads and also pthread_barrier
may cause serious trouble because pthread_barrier does not
differentiate between different threads, it uses only a counter.=20
>From this I have learned that there is a real difference between
my case 1 and my cases 2&3:
: 1. MPI implementations that do not allow that the application uses
: 2. MPI implementations that allow that the application uses threads,
: but it expects that MPI calls are issued always from the main;
: 3. MPI implementations that allow that the application uses threads,
: but it expects that MPI calls are never called in a way that they
: run in parallel;
> Rolf, what are the implications on an MPI implementation that make the
> cases 2 & 3 different?
> I think, in both cases MPI calls are called sequentially. The fact
> that the calls are made from different threads (3) rather than from
> one thread (2) does not effect neither MPI implementations nor=20
Here I want to remember on Marc's Mail; he seems to see more problems
with 3 than with 2:
| 2. MPI implementations that allow that the application uses threads,
| but it expects that MPI calls are issued always from the main;
| MPI_THREAD_QUERY will return MPI_SINGLE_THREAD
| 3. MPI implementations that allow that the application uses threads,
| but it expects that MPI calls are never called in a way that they
| run in parallel;
| Hard to define in an accurate manner (since MPI activites can=20
| happen while the user makes
| no MPI calls). Also, of doubtful usage: if the user synchronizes to
| ensure mutual exclusion of MPI calls, it might as well have these
| calls executed by one thread: communication buffers are likely to be
| on the heap.
But I have problems with Marc's answer:
- To the answer to my case 2:
The actual proposal (10th Feb.) says that=20
MPI_SINGLE_THREAD =3D=3D MPI library is thread compliant and ... =
thread-compliant: two concurrently running threads may make MPI calls
(p. 22 lines 5-7).=20
Therefore MPI implementations that support my cases 1-2 or 1-3 are
not allowed to name themself "MPI_SINGLE_THREAD".
Therefore I believe that the text must be changed.
- To the answer "doubtful usage" to my case 3:
The scenario I have in my mind makes
Therefore it does not belong to my case 2.
It does not call all MPI routines from the same thread, because
it generates and deletes the MPI-calling threads sometimes.
But I cannot imagine, that there is any MPI implementation that
fulfils case 2 but not case 3.
But there may be some chance for optimization if the application
only calls MPI routines from the main thread?
> Again, what's the difference between the cases (a), (b), and (c) =
> a. the application says that it will use only the main thread;
> b. the application says that MPI calls are done always from=20
> the main thread;
> c. the application says that MPI calls are done never in parallel;
> d. the application says that it wants to issue MPI calls from all
> threads in parallel=20
They are the same categories than 1.-4., but for the purpose that
the application may say that it will use less thread-functionality=20
than possible with a given MPI implementation.
If there is no need to differentiate between 2/b and 3/c than we
can remove 2/b -- but not 3/c as currently done by Marc's text.
> Who cares which particular thread called an MPI call if all MPI calls=20
> are done sequentially?
Is there any vendor that must care about this?=20
Rolf Rabenseifner (Computer Center )
Rechenzentrum Universitaet Stuttgart (University of Stuttgart)
Allmandring 30 Phone: ++49 711 6855530
D-70550 Stuttgart 80 FAX: ++49 711 6787626