An obvious question to me at least is: "Who is the user?"
The person who must write an MPI application and get it right it seems to be the "user" Erez is thinking of.
The organization that buys and operates a parallel machine to solve problems is also a "user".
The implementor of an MPI offering serves both kinds of user. Preserving specific freedoms for the implementor is not about making the implementor's job easy.
Let's pretend an MPI implementator who can assume the user will obey the standard could make MPI_Isend 5 times as fast as one that must protect some user who ignores the standard. In this case, the second class of "user" would probably say. "Let the MPI programmer learn the rules. I want the machine performance."
Now let's pretend everyone agrees the performance advantage of being able to assume the user will always honor the rule will never exceed 0.00001% on any machine, now or in the future. In this case both types of user and the implementor will all agree that the rule should be removed and the application writer relieved of the need to remember an "unnatural" rule.
I said *pretend* in both cases because I know the real situation is far less clear than the absurd performance numbers I used.
My point is that any proposed simplification of the standard will not simply be a trade off between
1) making life as easy as possible for people who write MPI libraries
or
2) or making it as easy as possible for people who write MPI applications.
I think it is likely the forum will conclude that the send buffer access rules should be relaxed but there should be more to the discussion than "Do you care about users or don't you?"
Dick
Dick Treumann - MPI Team/TCEM
IBM Systems & Technology Group
Dept 0lva / MS P963 -- 2455 South Road -- Poughkeepsie, NY 12601
Tele (845) 433-7846 Fax (845) 433-8363
owner-mpi-21@xxxxxxxxxxxxx wrote on 01/09/2008 02:58:21 PM:
> This issue is related to the "send buffer access". I say "wait,
> let's make it right for the users by restricting the implementers".
>
> The trick you described is legal with the current standard, but is
> so surprisingly wrong for many of the users.
>
>
> -----Original Message-----
> From: owner-mpi-21@xxxxxxxxxxxxx [mailto:owner-mpi-21@xxxxxxxxxxxxx]
> On Behalf Of Supalov, Alexander
> Sent: Wednesday, January 09, 2008 11:43 AM
> To: mpi-21@xxxxxxxxxxxxx
> Subject: RE: [mpi-21] Proposal EH2: add const keyword to the C bindings
>
> Wait, we're talking about standard here. The program you showed is plain
> incorrect. The trick I described is legal. All according to the
> standard. Let's keep our bearings straight: we don't want to make broken
> programs work by reducing the freedom of implementors.
>
> --
> Dr Alexander Supalov
> Intel GmbH
> Hermuelheimer Strasse 8a
> 50321 Bruehl, Germany
> Phone: +49 2232 209034
> Mobile: +49 173 511 8735
> Fax: +49 2232 209029
>
>
> -----Original Message-----
> From: owner-mpi-21@xxxxxxxxxxxxx [mailto:owner-mpi-21@xxxxxxxxxxxxx] On
> Behalf Of Erez Haba
> Sent: Wednesday, January 09, 2008 8:12 PM
> To: mpi-21@xxxxxxxxxxxxx
> Subject: RE: [mpi-21] Proposal EH2: add const keyword to the C bindings
>
> Alexander, you really don't want to implement the sender side tricks as
> described below. This immediately breaks any program that async sends
> the same buffer to two or more ranks, e.g.,
>
> MPI_Isend(buf.. rank=3.);
> MPI_Isend(buf.. rank=4.);
>
> Although this is not compliant with the standard, you can be sure that
> many applications and commercial applications are using this pattern.
>
> Thanks,
> .Erez
>
> -----Original Message-----
> From: owner-mpi-21@xxxxxxxxxxxxx [mailto:owner-mpi-21@xxxxxxxxxxxxx] On
> Behalf Of Supalov, Alexander
> Sent: Wednesday, January 09, 2008 4:14 AM
> To: mpi-21@xxxxxxxxxxxxx
> Subject: RE: [mpi-21] Proposal EH2: add const keyword to the C bindings
>
> Hi,
>
> Let's not forget that gcc is not the only C compiler to consider.
> There's Intel, Microsoft, PGI, and some other popular compilers.
>
> My primary problem with the const keyword is that it (seems to) prohibit
> sender side tricks like the in-place little- to big-endian conversion
> before sending and back afterwards.
>
> And let's not forget that most Fortran bindings are built on top of C
> ones. What will happen to the copy-in/out done by some Fortran 90+
> compilers if the underlying C binding uses const declaration for the
> inner C call, I don't know.
>
> Best regards.
>
> Alexander
>
> --
> Dr Alexander Supalov
> Intel GmbH
> Hermuelheimer Strasse 8a
> 50321 Bruehl, Germany
> Phone: +49 2232 209034
> Mobile: +49 173 511 8735
> Fax: +49 2232 209029
>
>
> -----Original Message-----
> From: owner-mpi-21@xxxxxxxxxxxxx [mailto:owner-mpi-21@xxxxxxxxxxxxx] On
> Behalf Of Dries Kimpe
> Sent: Wednesday, January 09, 2008 11:28 AM
> To: mpi-21@xxxxxxxxxxxxx
> Subject: Re: [mpi-21] Proposal EH2: add const keyword to the C bindings
>
> * Richard Treumann <treumann@xxxxxxxxxx> [2008-01-08 15:40:20]:
>
> > The MPI rules presumably would allow the block of code to access
> buf[0] and
> > buf[2] before and after the call to MPI_Send and the MPI_Send will not
> > change buf[0] or buf[2]. The practical effect of labeling buf[] const
> will
> > be correct as long as the MPI rules are followed and there is no code
> in
> > the vicinity that references buf[1].
>
> I'm a little bit worried about the fact that const buf[] (in C) spans
> the
> whole array, while the MPI_Datatype only includes elements 0 and 2; (And
> even if char[] gets degraded to char * and only affects the first array
> element, the problem remains when switching D1 and D2 in the example
> below).
>
> This difference could(or could not) lead to problems, even if the user
> follows the MPI rules. (Assuming Richard's interpretation of the const
> keyword is 100% accurate)
>
> Given 2 mpi datatypes:
> D1: byte, 1byte-hole byte
> D2: 1byte-hole, byte
>
> Using these definitions, we only pass buf (and not buf[1]) to the MPI
> send
> and receive functions
>
> so:
>
> char buf[3];
>
> 1: some_nonconst_irecv_func (buf, D1);
> 2: read buf[1]
> 3: some_const_send_func (buf, D2);
> 4: modify buf[1]
> 5: some_const_send_func (buf, D2);
> 6: some_complete_func ();
> 7: read buf[0], buf[2]
>
> Even though the user doesn't access buf[0] or buf[2] between the receive
> start and the receive complete, the compiler might.
>
> Assume an architecture that doesn't do non-aligned memory accesses,
> or one where byte access (write/read) is much slower than native-word
> accesses.
>
> In that case, the read on line 2 might actually read buf[0] and buf[1]
> into a register. In the same fashion, the modify on line 4 might write
> buf[0] and buf[1] (maybe even using the buf[0] value stored in a
> register)
>
> Even more so, since buf was only passed to const functions between line
> 2
> and 7, the compiler might assume on line 7 that it still has the correct
> value of buf[0] in a register.
>
> In that case, even though the user program conforms to the MPI rules,
> the resulting binary would be flawed.
>
> That being said, I've done some tests and couldn't trick gcc into
> this behaviour. Maybe because gcc thinks it isn't slower to do byte
> access on the 32 bit x86 machine (which doesn't have a problem with
> unaligned memory access) i'm using right now,
> --or--
> maybe because it has a rule that prohibits this behaviour.
>
> If it is the first, then we might have a problem om some architectures;
> If it is the second, than the const shouldn't pose a problem.
>
> As for me, I'm thinking that this is what the restrict/and or volatile
> keyword is for (and that the const definition given in the previous mail
> is too severe). Unless I'm wrong, restrict indicates that the data isn't
> aliased somewhere else and that the compiler can safely assume the data
> won't be modified by a function call that doesn't have some (non-const)
> reference to the data as a parameter. Except when using restrict, using
> a
> const parameter doesn't buy you anything at all except some ease of mind
> that a well-behaved compiler will inform you when you're trying to
> modify
> the const parameter within the function body.
>
> Besides, the fact that a function call is made using a const or nonconst
> parameter is a non-issue all together if one allows
>
> ``The value of buf[1] can change asynchronously any time between
> MPI_Irecv and MPI_Wait without any need for a function call.''
>
> Greetings,
> Dries
>
> PS. I've tested using the following program:
>
> int nonconstfunc (void * b);
> int constfunc (const void * b);
>
> char b[4];
>
> int main (int argc, char ** args)
> {
> b[0]=1; b[1]=2; b[2]=3; b[3]=4;
> nonconstfunc(b);
> b[0]+=1; /* cause read of B0 */
> constfunc(b);
> b[0]=b[0]+b[2]; /* cause read of B2 */
> b[0]*=2;
> constfunc(b);
> }
>
> which compiles into (gcc4, -O3, -march=prescott):
> movl $b, (%esp)
> call nonconstfunc
> addb $1, b
> movl $b, (%esp)
> call constfunc
> movzbl b+2, %eax
> addb b, %al
> movsbl %al,%eax
> addl %eax, %eax
> movb %al, b
> movl $b, (%esp)
> call constfunc
>
> ---------------------------------------------------------------------
> Intel GmbH
> Dornacher Strasse 1
> 85622 Feldkirchen/Muenchen Germany
> Sitz der Gesellschaft: Feldkirchen bei Muenchen
> Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer
> Registergericht: Muenchen HRB 47456 Ust.-IdNr.
> VAT Registration No.: DE129385895
> Citibank Frankfurt (BLZ 502 109 00) 600119052
>
> This e-mail and any attachments may contain confidential material for
> the sole use of the intended recipient(s). Any review or distribution
> by others is strictly prohibited. If you are not the intended
> recipient, please contact the sender and delete all copies.
>
> ---------------------------------------------------------------------
> Intel GmbH
> Dornacher Strasse 1
> 85622 Feldkirchen/Muenchen Germany
> Sitz der Gesellschaft: Feldkirchen bei Muenchen
> Geschaeftsfuehrer: Douglas Lusk, Peter Gleissner, Hannes Schwaderer
> Registergericht: Muenchen HRB 47456 Ust.-IdNr.
> VAT Registration No.: DE129385895
> Citibank Frankfurt (BLZ 502 109 00) 600119052
>
> This e-mail and any attachments may contain confidential material for
> the sole use of the intended recipient(s). Any review or distribution
> by others is strictly prohibited. If you are not the intended
> recipient, please contact the sender and delete all copies.
>