[Opm] OPM Flow multi-node simulations stuck at domain decomposition step

Wed Mar 11 14:57:26 UTC 2020

Dr. Blatt,

That's a typo. Each node has 36 processors.

Thank you,
Yogi

-----Original Message-----
From: Markus Blatt [mailto:markus at dr-blatt.de] 
Sent: Wednesday, March 11, 2020 5:09 AM
To: opm at opm-project.org
Subject: Re: [Opm] OPM Flow multi-node simulations stuck at domain decomposition step

Hi Yogi,

On Tue, Mar 10, 2020 at 02:15:19PM -0700, Yogi Pandey wrote:
> Simulation is being run on 4 nodes with 32 processors each using following command:
> 
> mpirun --display-map -mca btl self -x UCX_TLS=rc,self,sm -x HCOLL_ENABLE_MCAST_ALL=0 -mca coll_hcoll_enable 0 -x UCX_IB_TRAFFIC_CLASS=105 -x UCX_IB_GID_INDEX=3 --cpu-set 0,1,2,3,4,5,6,7,8,9,10,11,12,13,14,15,16,17,18,19,20,21,22,23,24,25,26,27,28,29,30,31,32,33,34,35 -np 144 --hostfile /etc/opt/rdma/hostfile /mnt/nfs-share/etc/opm-flow/opm-simulators/build/bin/flow --parameter-file=/mnt/nfs-share/data/norne/params
> 

Out of curiosity. Is there a special reason why --cpu-set runs until 35 with 32 cpus per node? Might you be oversubscribing a node?

Markus

-- 
Dr. Markus Blatt
OPM-OP AS
_______________________________________________
Opm mailing list
Opm at opm-project.org
https://urldefense.com/v3/__https://opm-project.org/cgi-bin/mailman/listinfo/opm__;!!GqivPVa7Brio!PzEFn5GeHUI5x99SaYH--Hknvm5xqFnzoGH3dAajzd68arR8HmCN_kZfOQeehcacfg$ 