I have been testing code using Intel MPI (version 4.1.3 build 20140226) and the Intel compiler (version 15.0.1 build 20141023) with 1024 or more total processes. When we attempt to run on 1024 or more processes we receive the following error:
MPI startup(): ofa fabric is not available and fallback fabric is not enabled
Anything less than 1024 processes does not produce this error, and I also do not receive this error with 1024 processes using OpenMPI and GCC.
I am using the High Performance Conjugate Gradient benchmark as my test code, although we have received the same errors with other test codes.