[Opm] OPM Flow RedHat 7 Binary Package Build with MPI Support

Sun May 6 23:56:37 UTC 2018

Dear Arne and Alf,

Thank you for the clarifications.

I am using the below yum repo found on the OPM website when doing the
installation. When installing the opm-simulators-openmpi-bin package to get
the mpi binaries it fails to install with the below errors related to
dependencies of trilinos. Is there another missing yum repo that I need to
add to resolve these broken dependencies?

Many thanks!

sudo yum-config-manager --add-repo
http://www.opm-project.org/package/opm.repo

sudo yum install opm-simulators-bin                         <-----This
installs fine

sudo yum install opm-simulators-openmpi-bin          <-----This fails

--> Finished Dependency Resolution
Error: Package: trilinos-11.2.5-1.12.x86_64 (opm)
           Requires: libscotcherr.so.0()(64bit)
Error: Package: trilinos-11.2.5-1.12.x86_64 (opm)
           Requires: libscotch.so.0()(64bit)
Error: Package: trilinos-11.2.5-1.12.x86_64 (opm)
           Requires: libdmumps-5.1.2.so()(64bit)
Error: Package: libopm-simulators1-openmpi-2018.04-0.x86_64 (opm)
           Requires: libptscotcherr.so.0()(64bit)
Error: Package: libopm-simulators1-openmpi-2018.04-0.x86_64 (opm)
           Requires: libptscotch.so.0()(64bit)
Error: Package: libopm-simulators1-openmpi-2018.04-0.x86_64 (opm)
           Requires: libscotch.so.0()(64bit)
Error: Package: trilinos-11.2.5-1.12.x86_64 (opm)
           Requires: libglpk.so.36()(64bit)
Error: Package: opm-simulators-openmpi-bin-2018.04-0.x86_64 (opm)
           Requires: libptscotcherr.so.0()(64bit)
Error: Package: opm-simulators-openmpi-bin-2018.04-0.x86_64 (opm)
           Requires: libscotch.so.0()(64bit)
Error: Package: opm-simulators-openmpi-bin-2018.04-0.x86_64 (opm)
           Requires: libptscotch.so.0()(64bit)
Error: Package: trilinos-11.2.5-1.12.x86_64 (opm)
           Requires: libpord-5.1.2.so()(64bit)
Error: Package: trilinos-11.2.5-1.12.x86_64 (opm)
           Requires: libhdf5.so.8()(64bit)

On Fri, May 4, 2018 at 6:18 AM, Alf Birger Rustad <abir at statoil.com> wrote:

> Hi Mohamad,
>
>
>
> As Arne Morten says, you will need to tell Red Hat what MPI version to
> use. The way this is done is by first making the module command available.
> You achieve this by the following command:
>
> . /usr/share/Modules/init/bash
>
>
>
> Now you can query for installed modules with:
>
> module avail
>
>
>
> Lastly, you can tell it to use openmpi with:
>
> module add mpi/openmpi-x86_64
>
>
>
> On ubuntu, you do not need to do anything. If openmpi is installed, mpirun
> just works.
>
>
>
> Cheers,
>
> Alf
>
>
>
> *From:* Opm [mailto:opm-bounces at opm-project.org] *On Behalf Of *Arne
> Morten Kvarving
> *Sent:* 4. mai 2018 11:22
> *To:* sindimo <sindimo at gmail.com>; opm at opm-project.org
> *Subject:* Re: [Opm] OPM Flow RedHat 7 Binary Package Build with MPI
> Support
>
>
>
> hi there,
>
>
>
> on redhat things work a little different. in particular there are separate
> packages for mpi.
>
>
>
> you have to install opm-simulators-openmpi-bin
>
>
>
> and load the mpi/openmpi-x86_64 module in the shell where you want to
> execute flow.
>
>
>
> arnem
> ------------------------------
>
> *Fra:* saudiaramco at gmail.com <saudiaramco at gmail.com> på vegne av sindimo <
> sindimo at gmail.com>
> *Sendt:* 4. mai 2018 11:15:36
> *Til:* Arne Morten Kvarving; opm at opm-project.org
> *Emne:* Re: [Opm] OPM Flow RedHat 7 Binary Package Build with MPI Support
>
>
>
> Hi Arne,
>
>
>
> Thank you for your clarification, I really appreciate it.
>
>
>
> I have updated to the latest OPM release of Flow 2018.04 and tested again
> on RedHat 7. I still don't see any performance improvement with the new
> RedHat 7 binaries as if MPI was still not working. On Ubuntu it works fine.
>
>
>
> For example I ran the Norne model once on 1 processor and once on 2
> processors on the RedHat 7 machine and the Ubuntu machine (identical
> machines on Amazon AWS m4.2xlarge instances) and these are the timings I
> get:
>
>
>
>
>
> Norne on RedHat:
>
> 1 processor:  769.437 seconds
>
> 2 processors: 839.285 seconds
>
>
>
> Norne on Ubuntu:
>
> 1 processor:  889.158 seconds
>
> 2 processors: 585 seconds
>
>
>
>
>
> If I do "top" command on RedHat while job is launched on 2 processors, I
> can see 2 processors running. However it doesn't seem that they are running
> in parallel, it seems as if each process is running a separate serial copy
> of flow, hence the run on 2 processors is even slower than the serial run.
>
>
>
> Also a quick check using "ldd" on the flow binaries on both Ubuntu and
> RedHat shows that the Ubuntu executable is indeed linked to the MPI
> library, while the RedHat binary still doesn't show any linked MPI
> libraries:
>
>
>
>
>
> On Ubuntu:
>
>
>
> ubuntu at ip-172-31-15-200:~$ ldd /usr/bin/flow | grep -i mpi
>
> libmpi_cxx.so.1 => /usr/lib/libmpi_cxx.so.1 (0x00007f350423c000)
>
> libmpi.so.12 => /usr/lib/libmpi.so.12 (0x00007f3503f66000)
>
>
>
>
>
> On RedHat:
>
>
>
> [ec2-user at ip-172-31-15-201 ~]$ ldd /usr/bin/flow | grep mpi
>
> [ec2-user at ip-172-31-15-201 ~]$
>
>
>
> [ec2-user at ip-172-31-15-201 ~]$ rpm -qa | grep opm
>
> libopm-common1-2018.04-0.x86_64
>
> opm-upscaling-devel-2018.04-0.x86_64
>
> opm-simulators-bin-2018.04-0.x86_64
>
> libopm-grid1-2018.04-0.x86_64
>
> libopm-upscaling1-2018.04-0.x86_64
>
> libopm-simulators1-2018.04-0.x86_64
>
> opm-upscaling-2018.04-0.x86_64
>
>
>
>
>
> I would appreciate any help with this please to get the parallel version
> of Flow working on RedHat.
>
>
>
> Thank you for all of your help.
>
>
>
> Sincerely,
>
>
>
> Mohamad
>
>
>
>
>
> On Thu, Apr 26, 2018 at 2:26 AM, Arne Morten Kvarving <
> Arne.Morten.Kvarving at sintef.no> wrote:
>
> hi,
>
>
>
> currently the rpm's are not mpi enabled. this will change in the upcoming
> 2018.04 release.
>
>
>
> it's a bit involved building on rhel, as you need some packages not in
> base or epel. in particular, you need to trilinos (or rather, just zoltan
> which is part of trilinos) to get efficient mpi support, as well as dune
> and such. if it's not extremely pressing i would suggest waiting for the
> release.
>
>
>
> arnem
> ------------------------------
>
> *Fra:* Opm <opm-bounces at opm-project.org> på vegne av M. S. <
> sindimo at gmail.com>
> *Sendt:* 25. april 2018 22:46:33
> *Til:* opm at opm-project.org
> *Emne:* [Opm] OPM Flow RedHat 7 Binary Package Build with MPI Support
>
>
>
> Dear All,
>
>
>
> I am interested in running OPM Flow with MPI support on RedHat 7.
>
>
>
> I've installed the binary packages on 2 different machines, one with
> RedHat 7 and the other with Ubuntu 16.04 using these instructions:
>
>
>
> https://opm-project.org/?page_id=245
>
>
>
> The Ubuntu packages ran fine with MPI and I can see performance
> improvement when running some of the SPE models, however the RedHat 7
> version doesn't seem to have been built with MPI support.
>
>
>
> RedHat 7 currently has native mpi package that can be easily installed by
> yum (based on mpich-3):
>
>
>
> sudo yum -y install mpich-devel
>
>
>
> Would someone be able please to build the Flow binaries for RedHat 7 with
> MPI support, or perhaps if they already exist can you please point me to
> where to get them?
>
>
>
> I attempted building from source code but the Flow third party library
> prerequisite installation seems overwhelming.
>
>
>
> If anyone already has their RedHat 7 build environment setup and would be
> able to help I would truly appreciate it. From the instructions on the
> website, it says to enable MPI support you just need to pass this option to
> cmake during the build “-DUSE_MPI=1” .
>
>
>
> Thank you for all of your help, I really appreciate it.
>
>
>
> Sincerely,
>
>
>
> Mohamad Sindi
>
> MIT
>
>
>
>
> -------------------------------------------------------------------
> The information contained in this message may be CONFIDENTIAL and is
> intended for the addressee only. Any unauthorized use, dissemination of the
> information or copying of this message is prohibited. If you are not the
> addressee, please notify the sender immediately by return e-mail and delete
> this message.
> Thank you
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <//opm-project.org/pipermail/opm/attachments/20180506/ddeb5fa1/attachment.html>