Right now MPI_Init involves all the processes. I presume at MPI_Init
(or through some previous majic) decides on ranks, etc. for the
processes participating. If arbitrary numbers of processes can call
MPI_Init, how does it know the number involved? For example, if you
started your job on 10 nodes, how can the MPI implementation tell if
it should wait until all 10 nodes call MPI_Init or initialize MPI
after one node calls MPI_Init. If one calls MPI_Init and then another
calls MPI_Init an hour later does it join the first node (i.e.,
MPI_COMM_WORLD becomes size 2) or does it start its own MPI world?
I don't see how to tell these apart without adding info to MPI_Init.
Am I missing something here?