[Opm] New opm-repo

[Alf Birger Rustad]

Thanks for the feed-back everyone! Actually, even on a serial run I have output writing accounting for 11% of the computational time on Norne. Splitting it in a separate thread with asynchronous writing should bring that down to practically zero with no extra dependencies required, and without memory duplication. Making the writing fully parallel seems like a larger undertaking which can be done for the mpi version.

Cheers,
Alf
________________________________________
From: Opm [opm-bounces at opm-project.org] on behalf of Markus Blatt [markus at dr-blatt.de]
Sent: Tuesday, March 15, 2016 11:20 AM
To: opm at opm-project.org
Subject: Re: [Opm] New opm-repo

Hi,

On Tue, Mar 15, 2016 at 08:47:52AM +0000, Alf Birger Rustad wrote:
> Thanks for the announcement! There is one issue that I hope
> everybody with informed opinions can contribute to. Today, writing
> results is a performance bottleneck flow. Hence, we should find a
> way for flow to write results asynchronously, i.e., the simulator
> should not need to wait computing the next timestep while the
> results are written. It has been suggested to split out the writing
> in a separate process to accomplish this, but I am not sure how this
> could be done in a good way. Implementing the writer as a server
> seems like over-engineering, and we probably do not want to mandate
> MPI to run the simulator. The simplest option I can think of is
> splitting the writing to a new thread with openmp.

As I am the one who brought this up, I have to relativate this a bit.
I experienced this bottleneck for a parallel run with 8 cores. The
time spent for output was 31% of the total time. For a sequential run
it is probably below 10% but I have not measured it.

Minimizing the time for the ouput of a sequential run as proposed
comes at cost, too. We need to duplicate the memory.

What I am concerned about is that the output is not parallel at all
which will bite us more when using more cores. Having a dedicated
output core is just a first step. Ultimately, we need several cores
doing output in parallel in a parallel file format. But this can wait
some time, I guess.

Cheers,

Markus

--
Dr. Markus Blatt - HPC-Simulation-Software & Services http://www.dr-blatt.de
Hans-Bunte-Str. 8-10, 69123 Heidelberg, Germany,  USt-Id: DE279960836
Tel.: +49 (0) 160 97590858


-------------------------------------------------------------------
The information contained in this message may be CONFIDENTIAL and is
intended for the addressee only. Any unauthorised use, dissemination of the
information or copying of this message is prohibited. If you are not the
addressee, please notify the sender immediately by return e-mail and delete
this message.
Thank you