Abstract - The trend of “big data growth” presents enormous challenges, but it also presents incredible scientific and business opportunities. Together with the data explosion, we are also witnessing a dramatic increase in data processing capabilities, thanks to new powerful parallel computer architectures and more sophisticated algorithms. In this talk we describe the algorithmic design and the optimization techniques that led to the unprecedented processing rate of 15.3 trillion edges per second on 64 thousand BlueGene/Q nodes, that allowed the in-memory exploration of a petabyte-scale graph in just a few seconds. We believe that these techniques can be successfully applied to a broader class of graph algorithms.
Short bio - Fabrizio Petrini is the manager of the High Performance Analytics Department of the IBM TJ Watson Research Laboratory. His research interests include various aspects of multi-core processors and supercomputers, including high-performance interconnection networks, network interfaces, fault tolerance, and data-intensive computing algorithms for mining large data sets. He is the recipient of numerous awards for DOE, IEEE and ACM, including best paper awards from the international conference on supercomputing (SC 2003), the international supercomputing conference (ISC 2009), and the international parallel and distributed processing symposium (IPDPS 2003 and 2014).