RE: [mpi-21] MPI Forum suggestions 11/14

[Erez Haba <erezh@xxxxxxxxxxxxx>]

Can you elaborate or send a pointer related to that architecture?

The const keyword is a contract between the caller and the caller and the callee. It means in our example that the MPI_Send function will not change the user supplied buffer. Is this the case with the SX systems?

-----Original Message-----
From: owner-mpi-21@xxxxxxxxxxxxx [mailto:owner-mpi-21@xxxxxxxxxxxxx] On Behalf Of Hubert Ritzdorf
Sent: Sunday, November 18, 2007 11:27 AM
To: mpi-21@xxxxxxxxxxxxx
Subject: Re: [mpi-21] MPI Forum suggestions 11/14

This is correct. The SX systems are not cache coherent. The send buffer
is read directly from
memory and may cause a cache flush. Thus, the const  keyword  would
not  be  correct.

Hubert Ritzdorf
NEC Laboratories Europe
IT Research Division
NEC Europe Ltd.


Snir, Marc wrote:
> If I remember correctly, NEC or some other Japanese vendor had a system with non-choherent shared memory were this added restriction was important. I guess that a read could cause a flush to be canceled.
>
> On 11/17/07 1:45 AM, "Steven Ericsson-Zenith" <steven@xxxxxxxxxxxxx> wrote:
>
>
>
> I think there is an intersect here with Erez's interest in const.
>
> In a parallel language it should be possible to send a constant value
> and read it at the same time (formally that would be fine). Indeed, it
> is possible in Occam and Ease/Carnap. I don't know the semantics of
> const in C functions, I assume it requires either a constant
> parameter, a pass by value copy operation or some other guarantee that
> the const is not written to (even by concurrent processes).
>
> The issue in MPI has to do with the termination properties, as I
> recall; i.e., the validity of the send buffer is not deterministic
> since the read is not synchronized with the send (so you can't know in
> an interleaving that a subsequent alteration - including deallocation
> - to the buffer value has not occurred). Futzing with the send buffer
> during a send operation seems to me to be an unforgivable rational.
>
> It really is equivalent to the usage rules one finds in a process
> oriented language (like CSP/Occam or Ease/Carnap), an asynchronous
> operation is, in effect, a distinct process and the buffer (if it is a
> variable) ceases to be in scope in all other processes during the
> period of concurrency. Indeed, such variables can only come back into
> scope if there is a cooperative barrier, they cannot come back into
> scope from  subordinate processes. Occam overcomes this by not
> allowing subordinates, Ease/Carnap overcomes this by only allowing
> constants (i.e., pass by value) to be passed to subordinates.
>
> With respect,
> Steven
> --
> Dr. Steven Ericsson-Zenith
> Institute for Advanced Science & Engineering
> http://iase.info
> http://senses.info
>
>
> On Nov 16, 2007, at 3:25 PM, Erez Haba wrote:
>
>
>> Thanks for the pointer!!!
>>
>> Can someone elaborate on the rational to prohibit read access to a
>> buffer being sent?
>>
>> I think that it's more than a little loss of functionally. Consider
>> a multi threaded application that wishes to send some data to other
>> ranks and at the same time process this data locally?
>> Also consider the scenario when rank x sends the same data to rank y
>> and z concurrently? does the text below prohibit that?
>>
>>
>> Quote
>> --------------
>>
>> In a multi-threaded implementation of MPI, the system may de-
>> schedule a thread that is blocked on a send or receive operation,
>> and schedule another thread for execution in the same address space.
>> In such a case it is the user's responsibility not to access or
>> modify a communication buffer until the communication completes.
>> Otherwise, the outcome of the computation is undefined.
>>
>>
>> [] Rationale.
>>
>> We prohibit read accesses to a send buffer while it is being used,
>> even though the send operation is not supposed to alter the content
>> of this buffer. This may seem more stringent than necessary, but the
>> additional restriction causes little loss of functionality and
>> allows better performance on some systems --- consider the case
>> where data transfer is done by a DMA engine that is not cache-
>> coherent with the main processor. ( End of rationale.)
>>
>> ------------
>>
>>
>> -----Original Message-----
>> From: Lisandro Dalcin [mailto:dalcinl@xxxxxxxxx]
>> Sent: Friday, November 16, 2007 3:03 PM
>> To: mpi-21@xxxxxxxxxxxxx
>> Cc: Erez Haba
>> Subject: Re: [mpi-21] MPI Forum suggestions 11/14
>>
>> On 11/16/07, Erez Haba <erezh@xxxxxxxxxxxxx> wrote:
>>
>>> Such implementations would be really bad for muti-threaded
>>> applications, or those using async send. Is there an MPI contract
>>> that the application should not access the send buffer for read
>>> while the send is not complete?
>>>
>> AFAIK, there is. Please, see:
>>
>> http://www.mpi-forum.org/docs/mpi-11-html/node40.html
>>
>>
>>> -----Original Message-----
>>> From: owner-mpi-21@xxxxxxxxxxxxx [mailto:owner-mpi-21@mpi-
>>> forum.org] On Behalf Of Darius Buntinas
>>> Sent: Friday, November 16, 2007 1:11 PM
>>> To: mpi-21@xxxxxxxxxxxxx
>>> Subject: Re: [mpi-21] MPI Forum suggestions 11/14
>>>
>>>
>>> I believe the standard allows an implementation to modify the
>>> buffer in
>>> a send call, as long as it's changed back before it completes.
>>>
>>> I heard (don't remember from who), that there was an implementation
>>> that
>>> would do endian conversion in place on the source buffer before
>>> sending,
>>> then change it back.
>>>
>>> Changing the standard to require the buffer pointer to be const could
>>> theoretically disallow that implementation.  So adding consts
>>> wouldn't
>>> be a minor change.
>>>
>>> -d
>>>
>>> On 11/16/2007 02:06 PM, David Gingold wrote:
>>>
>>>> On Nov 16, 2007, at 2:42 PM, Jeff Squyres wrote:
>>>>
>>>>
>>>>> 1. const: Sure, this seems like a good idea.  But it *is* an API
>>>>> change, and I am in agreement with Bill Gropp that there should
>>>>> be no
>>>>> API changes for MPI 2.1 (except for bug fixes).  Even if the API
>>>>> changes supposedly won't matter (because someone will find a case
>>>>> where it does matter).  More specifically: this is a slippery slope
>>>>> for 2.1.  Making one "minor" change will lead to more minor changes
>>>>> which will lead to ...
>>>>>
>>>> If an implementation declares in its mpi.h:
>>>>
>>>>    int MPI_Send(const void *, int, MPI_Datatype, int, int,
>>>> MPI_Comm);
>>>>
>>>> does this violate the existing standard?
>>>>
>>>> I don't see how this would break code that calls MPI_Send().  C
>>>> libraries made a similar transition with the advent of ANSI C; I
>>>> believe
>>>> the const semantics are defined in a way that lets one do just this.
>>>>
>>>> The const declaration allows an important optimization when
>>>> compiling
>>>> the calling functions.  But if an implementation is allowed to do
>>>> this,
>>>> then perhaps there is no need to change to the standard.
>>>>
>>>> -dg
>>>>
>>>>
>>>
>> --
>> Lisandro Dalcín
>> ---------------
>> Centro Internacional de Métodos Computacionales en Ingeniería (CIMEC)
>> Instituto de Desarrollo Tecnológico para la Industria Química (INTEC)
>> Consejo Nacional de Investigaciones Científicas y Técnicas (CONICET)
>> PTLC - Güemes 3450, (3000) Santa Fe, Argentina
>> Tel/Fax: +54-(0)342-451.1594
>>
>>
>
>
>
>
>
>
>
>
> Marc Snir
> Director, Illinois Informatics Initiative
> U Illinois at Urbana-Champaign
> LIS Bldg., Room 123
> 501 E Daniel St
> Champaign, IL 61820
> (217) 244 6568
>
>
>