I saw in the standard that only preemptive threads are=20
considered for use in MPI. I assumed that MPI requires=20
target platforms to provide thread-safe system calls and=20
C runtime libraries (and other libs if used in applications);=20
so, all library calls one makes in applications do not=20
interfere with MPI calls no matter which threads call them.
So, in the perfect world (Windows NT and Solaris approach=20
this by providing thread-safe libc - malloc is perfectly=20
thread-safe in WinNT - I tested myself)=20
there are only two cases: MPI_THREADS_NOT_SUPPORTED,=20
and MPI_MULTI_THREADED.
The pathological cases such as use of pthread_barrier=20
can be quite easily avoided by better design decisions.
I worked on making our MPI for WinNT thread-safe,=20
and I did not need any barriers.=20
But, as Mark pointed out, not all vendors provide=20
thread-safe libc. For instance, in AIX, malloc is=20
not thread-safe; I am afraid other vendors also=20
have their problems with libc thread safety.=20
Hence, it might be wise to conduct a study of common
problems with vendor libraries thread-safety and=20
come up with a consistent view on the resulting=20
cases.=20
It is also important, I think, to allow a thread-compliant=20
MPI application to work only with limited number of these=20
cases and return NOT_SUPPORTED for others.=20
Otherwise, one have to provide many versions of=20
MPI calls and also check every time an MPI call made=20
whether the user keeps his word concerning thread usage.
This definitely won't contribute to better performance.
Thanks, Boris.
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=
=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D=3D
From: Rolf Rabenseifner
Sent: 18 =F4=E5=E2=F0=E0=EB=FF 1997 =E3. 3:33=20
To: Boris V Protopopov
Cc: mpi-external@mcs.anl.gov
Subject: Re: mpi on a single thread
Hi Boris, Marc and everybody,
=20
> I disagree with this subdivision of MPI implementations, =20
> as well as MPI applications.
Yes, I also do not like more subdivision than necessary,
but I do not want that any MPI implementation must name itsself
less thread-"compliant" than it is, i.e. I do not want that any
MPI implementation must name itsself wit "MPI_THREADS_NOT_SUPPORTED"
if it can support the usage of threads by the application in some
manner.
First to your final statement:=20
> In my opinion, there are only two choices:
> thread-safe MPI implemetation (MPI_MLTIPLE_THREADS) and=20
> non-thread-safe MPI implementaiton (MPI_THREADS_NOT_SUPPORTED)
> because if one uses MPI calls only in one thread, or if MPI calls
> occure in different threads but sequentially can be handled without
> problems by a non-thread-safe MPI implementation.
I want to remember on Eric Salo's Mail:
| Date: Wed, 5 Feb 1997 15:50:33 -0800
| > A related question: are there any impelementations where
| > multithreading is unsafe even when only one thread is making
| > MPI calls?
|=20
| Our 2.0 release suffered from this problem, but that has been fixed
| in 3.0 so it's a moot point.
For example a thread-based MPI-implementation that uses internally
pthread_barrier and an application that calls MPI only from the
main thread, but uses other threads and also pthread_barrier
may cause serious trouble because pthread_barrier does not
differentiate between different threads, it uses only a counter.=20
>From this I have learned that there is a real difference between
my case 1 and my cases 2&3:
: 1. MPI implementations that do not allow that the application uses
: threads;
: 2. MPI implementations that allow that the application uses threads,
: but it expects that MPI calls are issued always from the main;
: 3. MPI implementations that allow that the application uses threads,
: but it expects that MPI calls are never called in a way that they
: run in parallel;
> Rolf, what are the implications on an MPI implementation that make the
> cases 2 & 3 different?
> I think, in both cases MPI calls are called sequentially. The fact
> that the calls are made from different threads (3) rather than from
> one thread (2) does not effect neither MPI implementations nor=20
> applications.
Here I want to remember on Marc's Mail; he seems to see more problems
with 3 than with 2:
| 2. MPI implementations that allow that the application uses threads,
| but it expects that MPI calls are issued always from the main;
| ****
| MPI_THREAD_QUERY will return MPI_SINGLE_THREAD
| *****
|=20
| 3. MPI implementations that allow that the application uses threads,
| but it expects that MPI calls are never called in a way that they
| run in parallel;
| ****
| Hard to define in an accurate manner (since MPI activites can=20
| happen while the user makes
| no MPI calls). Also, of doubtful usage: if the user synchronizes to
| ensure mutual exclusion of MPI calls, it might as well have these
| calls executed by one thread: communication buffers are likely to be
| on the heap.
| ****
But I have problems with Marc's answer:
=20
- To the answer to my case 2:
The actual proposal (10th Feb.) says that=20
MPI_SINGLE_THREAD =3D=3D MPI library is thread compliant and ... =
(p.24:5)
and
thread-compliant: two concurrently running threads may make MPI calls
(p. 22 lines 5-7).=20
Therefore MPI implementations that support my cases 1-2 or 1-3 are
not allowed to name themself "MPI_SINGLE_THREAD".
Therefore I believe that the text must be changed.
- To the answer "doubtful usage" to my case 3:
The scenario I have in my mind makes
pthread_create(MPI_routine,...)=20
...
pthread_join(...)=20
Therefore it does not belong to my case 2.
It does not call all MPI routines from the same thread, because
it generates and deletes the MPI-calling threads sometimes.
But I cannot imagine, that there is any MPI implementation that
fulfils case 2 but not case 3.
=20
But there may be some chance for optimization if the application
only calls MPI routines from the main thread?
> Again, what's the difference between the cases (a), (b), and (c) =
below?
>=20
> a. the application says that it will use only the main thread;
> b. the application says that MPI calls are done always from=20
> the main thread;
> c. the application says that MPI calls are done never in parallel;
>=20
> d. the application says that it wants to issue MPI calls from all
> threads in parallel=20
They are the same categories than 1.-4., but for the purpose that
the application may say that it will use less thread-functionality=20
than possible with a given MPI implementation.
If there is no need to differentiate between 2/b and 3/c than we
can remove 2/b -- but not 3/c as currently done by Marc's text.
> Who cares which particular thread called an MPI call if all MPI calls=20
> are done sequentially?
Is there any vendor that must care about this?=20
Best regards
Rolf=20
=20
Rolf Rabenseifner (Computer Center )
Rechenzentrum Universitaet Stuttgart (University of Stuttgart)
Allmandring 30 Phone: ++49 711 6855530
D-70550 Stuttgart 80 FAX: ++49 711 6787626
Germany rabenseifner@rus.uni-stuttgart.de