Decoupled MapReduce using MPI one-sided communication

Date and Time: 
Tuesday 2018 Apr 3rd
CG Auditorium
Sergio Rivas-Gomez

In this talk, we present an on-going work that proposes the integration of MPI one-sided communication in MapReduce frameworks. By using a decoupled strategy, we aim to overlap computations with communications during the Map and Reduce phases. Hence, we effectively increase the performance in situations where the workload is unexpectedly unbalanced. In addition, we present a novel mechanism for decentralized job-stealing, which enables MPI processes to obtain ownership of input datasets. Preliminary results will also be illustrated in comparison with a reference implementation that uses traditional message-passing communication.

Speaker Description: 

Sergio Rivas-Gomez is a Ph.D. student in High-Performance Computing at KTH Royal Institute of Technology (Sweden) under the supervision of Erwin Laure and Stefano Markidis. His research focuses on the integration of unified programming interfaces that enable new mechanisms for handling high-performance I/O on Exascale. Sergio is also part of the SAGE Project, one of the leading Exascale efforts of the European Union.

PDF icon MapReduce1S_seaconf18.pdf5.89 MB

Event Category: