Single Machine Graph Analytics on Massive Datasets Using Intel Optane DC Persistent Memory
share
Intel Optane DC Persistent Memory is a new kind of byte-addressable memory with higher density and lower cost than DRAM. This enables affordable systems that support up to 6TB of memory. In this paper, we use such a system for massive graphs analytics. We discuss how such a system should be deployed for such applications, and evaluate three existing shared-memory graph frameworks, Galois, GAP, and GraphIt, on large real-world web-crawls. We recommend algorithms and runtime optimizations for getting the best performance on such systems using the results of our study. We also show that for these applications, the Galois framework on Optane DC PMM is on average 2x faster than D-Galois, the state-of-the-art distributed graph framework, on a production cluster with similar compute power. Thus, Optane DC PMM yields benefits in productivity, performance, and cost for massive graph analytics.
READ FULL TEXT