Adaptive In-Cache Streaming for Efficient Data Management in General Purpose Processors Nuno Neves1,2, Pedro Tomás1,2, Nuno Roma1,2 Email: {nuno.neves,pedro.tomas,nuno.roma}@inesc-id.pt 1INESC-ID Investigação e Desenvolvimento, Rua Alves Redol 9, 1000-029 Lisboa, Portugal 2Instituto Superior Técnico, Universidade de Lisboa, Av. Rovisco Pais, 1049-001 Lisboa, Portugal 1. Introduction Conventional address-based models, supported on cache structures to mitigate the memory wall problem, often struggle when dealing with memory-bound applications or arbitrarily complex data-patterns that can be hardly captured by prefetching mechanisms. Stream-based techniques have proven to efficiently tackle such limitations, although not well-suited to handle all types of applications. To mitigate the limitations of both communication paradigms, an efficient unification is proposed, by means of a novel in-cache stream paradigm, capable of seamlessly adapting the communication between address-based and stream-based models. A new data-pattern dynamic descriptor graph specification, capable of handling regular arbitrarily complex data-patterns, was designed to improve the main memory bandwidth utilization through data reutilization and reorganization techniques. 2. In-Cache Stream Communication Morphable In-Cache Stream controllers, supporting memory-addressed and packed-stream data accesses; Memory-aware stream management controller (SMC), deploying efficient memory access optimization techniques (bandwidth optimization, data reorganization and reutilization, and in-time stream manipulation). 3. Set-Associative Cache Hybridization A n-way set-associative memory is simultaneously managed by two independent modules: a hybrid cache controller and a stream controller. Typical memory-addressed accesses are assured by the cache controller, by using any arbitrarily replacement and write policies. The stream controller adapts and reuses the resources of the n-way set-associative cache memory as dedicated stream buffers. 4. Dynamic Descriptor Graph Specification Independently of their application domain, deterministic algorithms are characterized by complex memory access patterns that can be represented by the n-dimensional affine function: By combining several functions, any deterministic data-pattern can be described independently of its complexity. A new Dynamic Descriptor Graph specification encodes any number of n-dimensional affine functions and chains them together in a graph-like structure to describe arbitrarily complex memory access patterns. 5. Memory-Aware Stream Generation The generation of data streams is performed by a memory-aware SMC, which relies on the descriptor specification through a dedicated PDC module. A burst controller and a reorder buffer optimize memory bandwidth and exploit data reorganization and reutilization through in-time stream manipulation. 6. Data-Pattern Generation Efficiency 7. Memory Bandwidth Optimization 8. Performance and Energy Efficiency The combination of the efficient memory access generation and the memory-aware burst and buffering optimizations allow increasing a typical DDR memory throughput close to its theoretical maximum. The proposed specification allows a steady one address-per-cycle memory access generation, requiring up to 8100x less description memory space when compared to state-of-the-art solutions. Memory throughput and access latency optimization for a DDR3 module accessed via a 100 MHz AXI bus. The observed performance increases, averaging 65x, and the measured up to 91% energy savings, result in overall processing energy efficiency (EDP) improvements as high as 245x. Acknowledgments: This work was partially supported by national funds through Fundação para a Ciência e a Tecnologia (FCT) under Project UID/CEC/50021/2013 and Grant SFRH/BD/100697/2014 .