Temporal and Spatial Locality: A Time and a Place for Everything Rick Bunt University of Saskatchewan Carey Williamson University of Calgary

December 6, 2003 Temporal and Spatial Locality: A Time and a Place for Everything Slide 2 of 29 Roll the credits …  Roberta Bodnarchuk  Milind Deshpande  Kevin Froese  Neeraj Gulati  Sung Hung Ling  Shikharesh Majumdar  Shane McDonald  Jennifer Murphy  Adeniyi Oke  Judy Peachey

December 6, 2003 Temporal and Spatial Locality: A Time and a Place for Everything Slide 3 of 29 Outline 1. What is locality 2. Elements of locality 3. Types of locality 4. Applications of locality 5. Changing locality 6. Concluding remarks

December 6, 2003 Temporal and Spatial Locality: A Time and a Place for Everything Slide 4 of 29 What is Locality  Parachor Curve  “during any interval of execution, a program favors a subset of its pages, and this set of favored pages changes slowly” [Denning 1970] An empirically observed phenomenon that has substantial intuitive appeal and numerous practical implications

December 6, 2003 Temporal and Spatial Locality: A Time and a Place for Everything Slide 5 of 29 Impact of Locality  Acceptable page fault rates can be achieved even when the memory allocated to a program is much less than that required to store all of its pages  Internet routers can make high speed routing decisions with very modest forwarding caches  Mobile users can work with remotely stored files even though they are located far from the file server

December 6, 2003 Temporal and Spatial Locality: A Time and a Place for Everything Slide 6 of 29 Known Aliases  The law of scattering  The principle of least effort  The rule  Concentration of productivity  The law of diminishing returns

December 6, 2003 Temporal and Spatial Locality: A Time and a Place for Everything Slide 7 of 29 The Underlying Concept  There is a very large population of items, many more than we can manage  There is a small core of relevant items on which we can productively focus our attention  This core will continue to be relevant long enough to justify our attention

December 6, 2003 Temporal and Spatial Locality: A Time and a Place for Everything Slide 8 of 29 Locality Through the Ages  Bradford’s Law of Scattering [1934]  Zipf’s Principle of Least Effort [1949]  Many applications before we discovered it  population distribution, distribution of wealth, distribution of biological species, article distribution in journals, and word usage in natural language.  has been used to plan the location of libraries and other facilities, to model the popularity of television programs, and to order search keys in hashing tables

December 6, 2003 Temporal and Spatial Locality: A Time and a Place for Everything Slide 9 of 29 Elements of Locality  Concentration:  Persistence: [Bunt and Murphy 1984] [Arlitt and Williamson 1997]

December 6, 2003 Temporal and Spatial Locality: A Time and a Place for Everything Slide 10 of 29 Types of Locality  Temporal Locality:  Clustering in time: items referenced in the immediate past have a high probability of being re-referenced in the immediate future  Spatial Locality:  Clustering in space: items located physically near an item referenced in the immediate past have a high probability of being re-referenced in the immediate future

December 6, 2003 Temporal and Spatial Locality: A Time and a Place for Everything Slide 11 of 29 Visualizing Locality  The memory map [Hatfield and Gerald 1971]

December 6, 2003 Temporal and Spatial Locality: A Time and a Place for Everything Slide 12 of 29 Visualizing Locality  Locality curves: An intrinsic measure [Bunt and Murphy 1984]

December 6, 2003 Temporal and Spatial Locality: A Time and a Place for Everything Slide 13 of 29 Visualizing Locality  The Zipf Distribution [Oke and Bunt 2002]

December 6, 2003 Temporal and Spatial Locality: A Time and a Place for Everything Slide 14 of 29 Applications of Locality  Memory management  File systems  Networks  Web applications

December 6, 2003 Temporal and Spatial Locality: A Time and a Place for Everything Slide 15 of 29 Applications in Memory Management  Various approaches to memory management address the principles of locality in different ways:  Recency-based replacement policies such as LRU appeal to the presence of temporal locality in a memory reference pattern.  Fetch policies that bring in adjacent items along with the item requested appeal to spatial locality.  Hit ratios are high when good choices are made.

December 6, 2003 Temporal and Spatial Locality: A Time and a Place for Everything Slide 16 of 29 Program Behaviour  We know how our programs are structured – program behaviour tells us how they execute  Considerable interest throughout the 1960’s, 70’s and 80’s  in measuring the referencing behaviour of executing programs  in developing models of this behaviour  in informing the design of approaches to manage resources

December 6, 2003 Temporal and Spatial Locality: A Time and a Place for Everything Slide 17 of 29 Applications in File Systems  Some examples:  Rodriguez-Rosell [1976] – an early empirical study of locality properties in database references  Smith [1981] – characterized long-term file reference behaviour with a focus on file migration  Williamson and Bunt [1986] – characterized short-term file reference behaviour with a focus on file system management  Majumdar and Bunt [1986 ] – evidence of phase behaviour in file reference traces  Smith [1978], Kearns and DeFazio [1983, 1989] and Verkamo [1985] – locality properties in database references

December 6, 2003 Temporal and Spatial Locality: A Time and a Place for Everything Slide 18 of 29 Applications to File Caching  Locality properties have been exploited successfully in the development of the caching techniques that are common in today’s distributed file systems.  Applications to mobile computing  Froese and Bunt [1999] explored the application of temporal locality in the design of optimistic caching techniques to support the file system activities of mobile users  Lindsey et al. [2003] reported evidence of spatial locality in the usage patterns on a campus wireless network

December 6, 2003 Temporal and Spatial Locality: A Time and a Place for Everything Slide 19 of 29 Applications in Networks  Even on early networks, measurements showed that packet traffic patterns are far from random  structural properties in the network packet behaviour reflect the particular applications, protocols, and network technologies being used temporal correlations in the timing structure of the packet arrival process spatial correlations in the destinations seen in consecutive (or nearly consecutive) packets on the network  These structural characteristics arise from large application-layer messages (such as those that result from file transfers), that require multiple network-layer packets for transport on the network, and from the bidirectional exchanges of data packets and acknowledgement packets typical in reliable data transfer protocols such as TCP

December 6, 2003 Temporal and Spatial Locality: A Time and a Place for Everything Slide 20 of 29 Train Spotting  Jain and Routhier [1986] proposed the packet train model for network traffic characterization based on observations of correlated arrivals  provided both a descriptive mechanism for understanding network traffic patterns and a generative mechanism for network traffic modelling  A packet train is a sequence of related network packets that traverse the network like train cars behind a steaming locomotive  characterized by short (often deterministic) inter-arrival times between the consecutive packets of a train, with slightly longer (and more random) inter-arrival times between trains

December 6, 2003 Temporal and Spatial Locality: A Time and a Place for Everything Slide 21 of 29 Locality in Web Traffic [Arlitt and Williamson 1997]

December 6, 2003 Temporal and Spatial Locality: A Time and a Place for Everything Slide 22 of 29 Application to Document Caching  Effective caching of Web documents can significantly lower the number of requests processed by a Web server, reduce the packet traffic volume on the Internet backbone, and improve the user- perceived Web response time.  Locality is the key to success  Temporal locality in Web accesses implies that Web objects accessed in the recent past are likely to be accessed again (perhaps by some other client) in the near future.  Spatial locality in Web accesses reflects Web page structure (i.e., multiple related Web objects) or geographic correlations in the surfing behaviour of Web users.  Caching can be applied at the servers, at the clients, or at proxies

December 6, 2003 Temporal and Spatial Locality: A Time and a Place for Everything Slide 23 of 29 Exploiting Locality in Web Requests  Temporal locality  To the extent that temporal locality is present, recency-based policies such as LRU can be effective for a Web cache.  Spatial locality  Pre-fetching can be successful if spatial locality is present. The persistent connection feature of HTTP exploits the fact that multiple Web objects are often requested consecutively from the same Web server.  Spatial locality can also reflect geographic correlations in the surfing behaviours of Web users, and can make geographic caching and content distribution networks effective.

December 6, 2003 Temporal and Spatial Locality: A Time and a Place for Everything Slide 24 of 29 Changing Locality Characteristics  Intentionally: Program restructuring  Inadvertently: Filtering locality

December 6, 2003 Temporal and Spatial Locality: A Time and a Place for Everything Slide 25 of 29 Program Restructuring  The basic premise  improving the locality of a program will improve its performance  A variety of methods were proposed to do this  (Replacement) strategy-based methods  Strategy-independent methods  The basic idea  divide the program’s address space into small blocks  cluster the blocks onto pages according to the dynamic reference characteristics (via restructuring graph).

December 6, 2003 Temporal and Spatial Locality: A Time and a Place for Everything Slide 26 of 29 Effects of Restructuring Restructuring improves locality: Restructuring improves performance: [Bunt and Murphy 1984] [Peachey, Bunt and Colbourn 1985]

December 6, 2003 Temporal and Spatial Locality: A Time and a Place for Everything Slide 27 of 29 Damaging Locality  The locality characteristics of an arriving request stream can be damaged in two ways:  The interleaving of independent request streams (the aggregration effect)  The removal of “hits” by upstream caches (the filtering effect)  This presents problems for traditional approaches to cache management

December 6, 2003 Temporal and Spatial Locality: A Time and a Place for Everything Slide 28 of 29 The Cache Filter Effect [Froese and Bunt 1996]

December 6, 2003 Temporal and Spatial Locality: A Time and a Place for Everything Slide 29 of 29 Concluding Remarks  Locality has returned great value  helps us to understand why certain approaches to resource management are effective (or ineffective)  inspires new approaches to managing resources  Productive application across a wide range of domains reflects that locality speaks to a basic behavioural property (“concentration of productivity”)  Should be, and will continue to be, an important research topic