here is a write up of the notes I took at the last meeting. I have to
prepare a report for Esprit anyway to justify their funding half my
expenses (and that report gets into the public domain anyway), so I
thought other people might also like a copy.
These are in no way official minutes, but I hope they have the votes
and action points correct. Feel free to berate me if I got something
wrong, or to use these in any way you feel like.
Enjoy
-- Jim
James Cownie
Dolphin Interconnect Solutions
Phone : +44 117 9071438
E-Mail: jcownie@dolphinics.com
MPI 2 Chicago O'Hare Sept 3-6
=============================
James Cownie 11 Sept
This is a report on the MPI meeting held at the Ramada hotel Chicago
O'Hare on September 3 - 6. This report only covers the meeting from
lunch on September 4, since I was unable to attend the Tuesday meeting
having booked flights prior to the decision to meet on Tuesday.
Summary
=======
One sided has now passed a first formal vote (for the second time !)
Dynamic has now passed two formal votes, and (subject to correction of
typographical and binding errors) is frozen.
External Interfaces
-------------------
Starting at 1p.m. Steve Huss-Ledermann presented the chapter for a
formal vote. On a point of order James Cownie was allowed a formal
vote as he personally had attended 2 of the last three meetings,
though Dolphin Interconnect Solutions had not. There were therefore 19
organisations present with official votes.
Voting on section 7.2: Generalised requests
7.2.1 Introduction 16: 0: 3
7.2.2 Functionality 18: 1: 3
7.2.3 Deferred pending handler discussions
7.2.4 example => no vote needed
Voting on 7.3: Communicator ID
Deferred.
Voting on 7.4: Accessing MPI datatypes
7.4.1: Amendment to return exactly the info given to create the
datatype 17: 0: 6 <=== Amendment passes
7.4.1 as amended 20: 0: 3
7.4.2: Deleted
7.4.3: Deferred for further discussion, over potential problems
with char constants > 128, Fortran, ...
Voting on 7.5: Packing of datatypes
Deferred.
Section 7.6: True extent of datatypes deleted, to be replaced by a
better solution in the MISC chapter.
Voting on 7.7: caching on MPI handles.
Deferred, pending resolution of a variety of issues
When a handle with attributes is deleted are the calls to the
attribute delete functions ordered ? (Particularly when the
handle is MPI_COMM_WORLD).
There is continuing indecision on whether it makes sense to
cache on MPI_GROUP objects.
Voting on 7.8: Identifying requests 18: 0: 6
Voting on 7.9: Naming Objects deferred
Voting on 7.10: Allowing User Errors. Deferred because of concerns
over the meaning when the error handling mode is ERRORS_RETURN.
It appears that there probably isn't a real problem, but this
needs to be exhaustively checked.
Voting on 7.11: Allowing user functions at MPI_FINALIZE. Deferred
because of unclarified issues with order of
* communicator deletion
* attribute delete function invocation
* precise definition of Finalize.
Marc Snir introduced section 7.12: MPI and threads no votes were taken.
7.12.1: There was discussion on the need for functions
MPI_Thread_initialize to be called before a thread can
invoke MPI. (MPI_Init also performs MPI_Thread_init
for the calling thread, so single threaded code
doesn't change).
MPI_Thread_finalize for symmetry with Thread_initialize
MPI_Thread_initialized to allow a thread to know if it
needs to call Thread_initialize
7.12.2: Much discussion on the precise requirements for error
handler invocation. Should they be invoked
1) on any thread
2) on the thread which set up the operation
3) on the thread which tests/waits for completion
No decision was reached.
7.12.4: Global thread functions. The whole section has been deleted.
Broke for coffee 3.p.m.
Extended Collective Operations
------------------------------
Andrew Lumsdaine stood in for Tony Skjellum who had got sick and was
in hospital. (He's now recovered).
All votes here were straw votes.
Andrew presented the suggested new operations:
Intercommunicator collective operations
Non-blocking collective operations
Persistent collective operations
Miscellaneous new blocking collective operations
There was some discussion on the meaning/implementability of cancel on
collective operations. The current draft has a logical extension of
the pt2pt cancel, however the net effect is that cancel is useless on
collective operations. Some people want a more local cancel, however
it is extremely unclear if this would be implementable.
Straw votes were taken
Remove most non-blocking collective operations 17: 6: 6
Remove all ditto 5:15: 9
(The most popular non-blocking collective seems to be the non-blocking
barrier.)
Persistent non-blocking collectives only match
other persistent non-blocking collectives 24: 0: 3
Do we want persistient blocking collective calls 14: 4:11
Section 6.4: Intercommunicator constructors
Do we want Comm_create, Comm_split on Intercommunicator 15: 3: 8
Do we want Intercommunicator partition 0:13:14
Section 6.5: Intercommunicator collective ops 19: 2: 5
Do we want in-place operations
Simple cases 19: 1: 8
Hard cases 6:13: 8
Section 6.6: Extended functions
Do we want persistent blocking functions as detailed here 1: 4:21
(Issues here are to do with when you bind blocking property to a
persistent handle : at handle create time, or later).
Delete ICALL, ICALL_SPLIT, COMM_DUP_LOCAL, COMM_SPLIT_LOCAL
15: 0:10
(This may have suffered from Tony's absence, since no one could
understand what these functions were supposed to do !)
Section 6.7: Channels
Deferred pending thinking on the possibilty of constructing equivalent
functionality by pairing persistent requests.
5-35p.m. adjourned.
Miscellaneous chapter
---------------------
Thursday 8-30 a.m. Rusty Lusk started formal voting on the
Miscellaneous chapter. 21 voting organisations were present.
Discussion references a new version of the chapter prepared by Rusty
late on Wednesday night, and handed out at the meeting.
Section 3.1: Version number
An amendment was proposed to remove the Get_version function 1:13:9
(The argument being that all it allows you to do is check that you
have linked against the correct library.)
Section 3.1 as not amended 18: 0: 2
Section 3.2.1: STATUS_IGNORE 19: 1: 3
Section 3.2.2: Non-destructive test of status 23: 0: 1
Section 3.3: MPI_ERR_KEYVAL 21: 0: 3
Section 3.4: MPI_True_extent
Section 3.5: MPI_Set_extent
Discussion at length about the implications of changing a
property of a datatype, rather than constructing a new
one. Implementors disliked modifying an existing datatype
since many implementations reference count datatypes,
therefore achieving the required semantics could require
significant changes to code.
Amendment to replace set_extent with a new datatype
constructor which would change the extent 15: 0: 6
Section 3.4 and 3.5 as amended 22: 0: 0
Section 3.6:
Section 3.6.1:Clarification of finalize.
Much discussion on whether finalize must return
a) at all
b) somewhere
c) everywhere
Problems arise becuase this is really a binding issue to do
with how "an MPI process" is bound to an implementation
(e.g. it could be a Posix process, or a thread, or
...). Different choices for finalize semantics depend on this
choice of binding.
Allow finalize not to return 11: 6: 8
Disallow all MPI calls after Finalize 15: 1: 4
Section deferred, pending further clarification, and possible
addition of an MPI_Finalized call.
Section 3.6.2: Clarification of Intercomm_create 22: 1: 1
Section 3.6.3: Clarification of MPI_TYPE_SIZE 19: 0: 3
Section 3.6.4: Clarification of MPI_REDUCE
Discussion on whether the proposal overly restricts
implementations.
Amendment to delete lines 34-38 (which specify exactly which
functions will be called) and replace with advice to someone
which says that any combination of user functions may be
called (from one to all). 22: 0: 1
Section 3.6.4 as amended 22: 0: 1
Section 3.7: New datatype constructors
Section 3.7.1: Simple_struct the formal definition as given is wrong,
the section has been postponed pending a correct definition.
3.7.2: Contiguous struct 6: 7:10 (i.e. rejected)
3.7.3: Indexed_block 10: 7: 7
Process Creation and Management
-------------------------------
At 10-45a.m. Bill Saphir introduced this section for second formal
vote. (So this should not change again...)
Sections 4.3.1, 4.3.2: MPI_SPAWN 22: 0: 3
4.3.3: MPI_SPAWN_MULTIPLE 20: 0: 3
4.3.4: SPAWN_INDEPENDENT
Much discussion on this and the associated MONITOR and SIGNAL
functions in 4.5.2 and 4.5.3.
Vote on deleting 4.3.4 SPAWN_INDEPENDENT
4.3.5 SPAWN_MULTIPLE_INDEPENDENT
4.3.6 non-blocking independent functions
4.5.2 Signalling
4.5.3 Process monitoring
14: 7: 4 <== DELETED
Vote to move all of the stuff just deleted to the JOD,
as a recommendation to "do it this way if you do it" 19: 1: 5
Miscellanea
-----------
Starting after lunch at 1p.m. Steve Huss-Ledermann discussed issues
with the MPI_STATUS object (many more functions need to access it, and
some user functions need to set values in it). The proposal is to
provide functions for setting the fields in an MPI_STATUS.
Rusty Lusk then introduced the Misc chapter. Formal votes 23-25
votes.
Section 10.1: Portable mpi process startup
The command is to be called mpiexec (rather than mpirun) thus
avoiding any compatibility issues with existing mpirun
commands.
The proopsal states that if unspecifed the number of processes
requested should be one. An amendment was proposed to leave
the default unspecified. 14: 4: 7
Section 10.1 as amended 22: 1: 2
10.3: Language inter-operability Marc Snir
10.3.1, 10.3.2: Intro and assumptions 21: 0: 1
10.3.3: Initialization 23: 0: 1
10.3.4: Handle transfer 22: 0: 0
10.3.5, 10.3.6:Opaque objects and datatypes
23: 0: 2
10.3.7: Addresses deferred
10.3.8, 10.3.9: Groups and communicators
23: 0: 1
10.3.10: Attributes deferred
10.3.11:,12,13: Requests, Error handlers, Reduce operators
23: 0: 0
10.3.14: Status deferred
10.3.15:,16: Constants, inter-language communiction
21: 0: 0
Dynamic
-------
At 2-25 p.m. Bill Saphir continued the Dynamic chapter.
Section 4.3.6: Non-blocking spawn (with independent functions removed,
see above) 12: 1:10
4.4.7: Reserved key values 15: 1: 6
4.4.1: Names, addresses etc 15: 0: 8
4.4.2: Server routines 13: 0:10
4.4.3: Client routines 11: 0:11
4.4.4: Name publishing 11: 0:12
The function names are likely to change to
Service_publish, Service_retract or Service_unpublish
4.4.5: Iaccept 12: 1:11
Iconnect 9: 3:12
4.4.6: Reserved keys
Amendement to add "ip_address" as an additional
reserved key 12: 0:11
4.4.6 as amended 13: 0:10
C++ Bindings
------------
At 3-40 p.m. Andrew Lumsdaine attempted to present the C++
bindings. There was a great deal of discussion as to whether MPI needs
C++ bindings, or whether C++ programmers should simply use the
existing C bindings. Discussion continued at such length that there
was no time for any formal votes on the C++ bindings. An extremely
informal straw poll was held on whether to keep the C++ bindings. This
was slightly in favour of keeping them, but without a majority. (When
I asked for this vote I explicitly stated that I would not publish the
results, so you'll have to ask me in person if you want the numbers !)
Fortran 90 Bindings
-------------------
At 4-30p.m. Bill Spahir presented the current state of the F90
bindings. There are significant outstanding issues here, in particular
because
* F90 is a strongly type-checked language without any escape
routes,
* F90 explicitly does not say anything about the layout of data in
store, while MPI has an implicit model of store layout.
There are a number of possible approaches, but none achieves full MPI
and full F90. Further work is clearly required.
One Sided Communications
------------------------
At 8-30a.m. on Friday Marc Snir introduced the newly revised version
of the One sided chapter for a first formal vote. (The version of
one-sided which had previously received a first vote has been
abandoned). 24 voting organisations present.
Instead of a communicator the one-sided operations now take a window
object.
Section 1.2: Initialisation
There was discussion on whether an info object was the correct
way to pass optional/additional information into the RMA_init
call. No amendment was proposed, so an info argument remains.
There was discussion on the use (and sematics) of the
displacement argument.
Amendment to delete the displacement argument 3:14: 7
Amendment to replace the displacement argument
with a datatype whos extent would be used 3:16: 5
1.2 as not amended 21: 2: 1
Section 1.3: Communication calls
SGI proposed an amendment to re-order the arguments to the
calls so that the buffer came after the datatype 3:14: 6
Amendment that Fortran character data need not be supported in
these calls 12: 4: 7
1.3 as amended 21: 0: 3
Section 1.4: Synchronization calls
Much discussion in 1.4.3 on the fact that this section (which
allows third party transfers) requires an active agent at each
node.
Amendment Locks are restricted to windows which are in
MEM_ALLOCed windows 16: 7: 2
(This allows a simpler implementation on machines which can
share some regions of the address space, but not all of it).
1.4 as amended 21: 1: 2
Section 1.5: Progress
There was a discussion of the progress rules. As written these
are stronger than some people's view of the MPI-1 rules, while
conforming to other people's view. (This really means that the
MPI-1 progress rules are open to interpretation).
Amendment The progress rules are the same as in MPI-1 19: 1: 4
Whole one-sided chapter 20: 2: 3
Meeting dates
-------------
At 10-30a.m. Rusty Lusk presented the possible meeting dates
October 8-11 (Tuesday-Friday) confirmed
January 21-23 (Tuesday-Thursday) tentative
March 5- 7 (Wednesday-Friday) "
April 23-25 (Wednesday-Friday) "
There was some question of moving to California for these meetings to
avoid getting snowed in at O'Hare.
California 11
Chicago 13
So the meetings will remain at the Ramada O'Hare.
Miscellany
----------
At 11-05 Marc Snir presented the difficult inter-operability problems
in a 64 bit environment (section 10.3.7).
Much discussion ensued, Eric Salo will draft an alternative that
provides 5 Fortran functions with non-portable bindings. (I.e. the
user may have to change her source code between different conforming
MPI implementations if she uses these functions in F77).
A straw vote on the idea of these (without seeing them) 7: 5: 10
Section 10.3.10: Attributes. No conclusion was reached a possible
alternative would be to have new functions.
At 11-45a.m. Rusty introduced 10.7 on continuable errors.
At 11-50a.m. Bill Saphir discussed the MPI_APPNUM attribute which
would allow an application to work out how many similar processes
there are to it when started in a multi-executable job.
Meeting closed at noon.