If MPI-2 provides a way to specify how many processes to start, then it
should also define the relationship between the "soft" key values in
MPI_Spawn_multiple when one or more binaries has a non-"hard"
specification. This is necessary in order to control the allocation of
nodes beyond the minimum requirements. For instance, for best load
balancing, a user may not want all remaining nodes to be allocated to a
single binary.
In the simplest (default?) case, you might just want to deal the extra
nodes out to the binaries one after the other until you run out.
However, it has worked well and been very useful for us to be able to
specify a weighting factor for each different binary. Here's a proposal
that allows the user to control allocation of the extra nodes. [Note
that I have no particular attraction for one syntax over another.]
Allow an optional floating point value as the final element in the
list of triplets; this value specifies a weighting factor for
allocating nodes to binaries after the minimum requirements have been
satisfied.
Here's an example:
Load three binaries with the following key values for soft:
1:100,0.5
20
5,2.5
Result: At least 26 nodes are required for success: load the first
binary on 1 node, the second on 20 nodes, and the third on 5 nodes.
Any remaining nodes are allocated with a 0.5-to-2.5 weighting to the
first and third binaries until the first binary reaches its specified
maximum of 100 nodes, then all the remaining nodes are allocated to
the third binary.
This has worked well in practice for a system we have developed. Of
course, the MPI-2 standard document would need a few more paragraphs to
pin down the edge effects, defaults, allocation order, interaction with
stepped triples, etc.
Dennis Cottel, dennis@nosc.mil, (619) 553-1645
NCCOSC, RDT&E Division (NRaD), San Diego, CA