Abstract

Data movement is a key bottleneck to scalability of modern computer systems. The race to provide higher usable bandwidth to the compute system (processors) with less energy is garnering increasing interest. As systems scale out, the problem of providing usable bandwidth at a reasonable power level only multiplies. Despite the realization that data movement is a key bottleneck, programmers still waste a sizeable portion of bandwidth. On average, across several benchmarks, L1-D cache utilization ranges from approximately 45% through a high of 91%. Improving the utilization can greatly improve energy efficiency through: reduced data movement, reduced processor effort during execution (effort expended managing memory), and through improved cache line reuse (when lines are fully utilized).

Gather-scatter broadly refers to any technique that takes a non-contiguous section of memory and makes it contiguous so that it can be fetched together from memory (also called packing). Some of these techniques can be utilized to pack the data from sparse representations so that they fully utilize bursts from memory and fully utilize cache lines. The end result is that fewer data moves between levels in the memory hierarchy and cache. This talk will cover some reasons for dark bandwidth, two different use cases for gather/scatter near memory, and potential outcomes resulting from turning data access patterns with poor utilization into either compacted high reuse data or into streaming data depending on use case.

Talk outline

1. Introduce the data movement problem and “Dark Bandwidth” in general
2. Describe what is really happening at the architecture and micro-architecture level (use HPCG SpMV as a running example)
3. Re-introduce the problem at another level now that the basics are commonly understood
4. Describe some current solutions, past solutions, and what can be done for the future (again, using SpMV as an example)
5. Show results for our current implementation of SPiDRE
6. Discuss the trade-offs in interfacing
1. Turning sparse, high reuse distance access to dense streaming
2. Compacting data for high reuse and less data movement
7. Impact on computing/programming models/runtime and future work

References:

1. Beard, J.C., & Randall, J. (2017). Eliminating Dark Bandwidth: a data-centric view of scalable efficient performance, post-Moore. Proc. High Performance Computing Post-Moore (HCPM’17)
2. Beard, J. C. (2017, October). The sparse data reduction engine: chopping sparse data one byte at a time. In Proceedings of the International Symposium on Memory Systems (pp. 34-48). ACM.