Quantcast
Channel: Clusters and HPC Technology
Viewing all articles
Browse latest Browse all 927

openmp application performance dropped with I_MPI_ASYNC_PROGRESS=enable

$
0
0

Hi,

I tried MPI/openmp process pining, it seems that When I use non-blocking api(Iallreduce) and specific I_MPI_ASYNC_PROGRESS like the following command, it I set I_MPI_ASYNC_PROGRESS=enable, then application will spent much more time on libiomp.so(kmp_hyper_barrier_release), and vmlinux also got a little hotter, compare with (I_MPI_ASYNC_PROGRESS=disable), is there any issue with my configuration? I use vtune and it shows that all the cores are pin in the right cores. the only difference is core 67 is used by MPI communication thread. 

========command=================

mpirun    -n 2 -ppn 1    -genv OMP_PROC_BIND=true -genv  I_MPI_ASYNC_PROGRESS= -genv I_MPI_ASYNC_PROGRESS_PIN=67 -genv I_MPI_PIN_PROCS=0-66 -genv OMP_NUM_THREADS=67  -genv I_MPI_PIN_DOMAIN=sock -genv I_MPI_FABRICS=ofi -f ./hostfile   python train_imagenet_cpu.py  --arch alex --batchsize 256 --loaderjob 68  --epoch 100 --train_root /home/jiangzho/imagenet/ILSVRC2012_img_train --val_root /home/jiangzho/imagenet/ILSVRC2012_img_val --communicator naive /home/jiangzho/train.txt /home/jiangzho/val.txt

Zone: 

Thread Topic: 

Question

Viewing all articles
Browse latest Browse all 927

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>