1. John D. Kubiatowicz UC Berkeley
Locality Aware Mechanisms for Large-scale Networks: The Tapestry Infrastructure
John D. Kubiatowicz
UC Berkeley
Confine load on infrastructure
Prospective on differences
Heuristics vs. primitives
more outline, flow control through talk
more details on content, pictures
diffusing impact, ripples are load placed on the network

2. Global Scale Applications
Clear demand for global scale applications
Exploiting collective resources
File sharing, data dissemination, shared computation
Wide-area issues
Scalability, fault-handling, adaptability, manageability
Decentralized Object Location and Routing (DOLR)
Provides scalable message routing to objects
Tahoe Retreat 06/02
John Kubiatowicz

3. Utility-based Storage: OceanStore
Pac
Bell
Sprint
IBM
AT&T
Canadian
OceanStore
Tahoe Retreat 06/02
John Kubiatowicz

4. Locality, Locality, Locality
“The ability to exploit local resources over remote ones whenever possible”
“-Centric” approach
Client-centric, server-centric, data source-centric
Requirements:
Find data quickly, wherever it might reside
Locate nearby object without global communication
Permit rapid object migration
Verifiable: can’t be sidetracked
Data name cryptographically related to data
Infrastructure level
How to optimize load on infrastructure
Not application specific
Tahoe Retreat 06/02
John Kubiatowicz

5. Basic Tapestry Mesh 4 2 3 1 NodeID 0xEF34 NodeID 0xEF34 0xEF31 0xEFBA
0xFF37
0xE555
0xE530
0xEF44
0x0999
0x099F
0xE399
0xEF40
Tahoe Retreat 06/02
John Kubiatowicz

6. Randomization and Locality
Tahoe Retreat 06/02
John Kubiatowicz

7. Parallel Insert Algorithms (SPAA ’02)
Massive parallel insert is important
We now have algorithms that handle “arbitrary simultaneous inserts”
Construction of nearest-neighbor mesh links
Log2 n message complexityfully operational routing mesh
Objects kept available during this process
Incremental movement of pointers
Interesting Issue: Introduction service
How does a new node find a gateway into the Tapestry?
Tahoe Retreat 06/02
John Kubiatowicz

8. Highly Dynamic Systems
Instability is the common case….!
Small half-life for P2P apps (1 hour????)
Congestion, flash crowds, misconfiguration, faults
BGP convergence 3-30 mins!
Mobile clients in semi-connected mode
Must Use Overlay under instability!
Must be somehow:
Insensitive to faults and denial of service attacks
Route around bad servers and ignore bad data
Repairable infrastructure
Easy to reconstruct routing and location information
Without care, worst case: sub-optimal paths, network partitions, broken invariants  loss of availability
Anthony D. Joseph:
Loss from routing reconfiguration delays of minutes or more: BGP convergence time of 3 to 15 minutes
Stress model is now semi-connected versus semi-disconnected (ie, bigger islands of connectivity
Tahoe Retreat 06/02
John Kubiatowicz

9. Continuous Self-Repair
All state is Soft State
Continuous probing, selection of neighbors
Periodic restoration of state
Stability through statistics:
Redundancy at many levels:
Neighbor links, Object Roots, etc.
Dynamic Stabilization:
Integrate/remove themselves automatically
Pointer state routed around faulty node
Markov Models:
What is a misbehaving router? Communication link?
What level of redundancy necessary?
Are we under attack?
Tahoe Retreat 06/02
John Kubiatowicz

10. State Maintenance Maintenance of link conditions, routing state
Example: soft-state beacons measure link conditions between neighbors (Tapestry, Scribe)
Locality awareness
Naïve: messages sent across overlay hops at same rate regardless of actual network distance
Traffic can scale with length of overlay hop, possibly causing congestion
Alternatives:
Scale soft-state frequency w/ length of overlay hop
External fault-detection / measurement platform
Soft-stae is the mechanism, and possibly not a good one
Tahoe Retreat 06/02
John Kubiatowicz

11. Locality-Based Heartbeats
Tahoe Retreat 06/02
John Kubiatowicz

12. Redundant Routing Keep multiple routes for every link
Pre-compute alternatives
Quick adaptation (using up pre-computing)
Destination-rooted spanning tree
For each node N, reachable by all other nodes
All paths union to form spanning tree rooted at N
Convergence
# of candidates for the next hop decreases by b every hop, where b = base of Tapestry ID
Nodes routing to same ID converge as a function of the network density between them
Tahoe Retreat 06/02
John Kubiatowicz

13. Convergence
Tahoe Retreat 06/02
John Kubiatowicz

14. First Reachable Link Selection (FRLS)
Tahoe Retreat 06/02
John Kubiatowicz

15. FRLS Reachability
Tahoe Retreat 06/02
John Kubiatowicz

16. Proactive Multicast
Tahoe Retreat 06/02
John Kubiatowicz

17. Self-Organizing Soft-State Replication
Simple algorithms for placing replicas on nodes in the interior
Intuition: locality properties of Tapestry help select positions for replicas
Tapestry helps associate parents and children to build multicast tree
Preliminary results show that this is effective
Tahoe Retreat 06/02
John Kubiatowicz

18. The Living Network Model
Gaia: a living network
James Lovelock [1979]
Large scale self-management
Locally constrained interactions  scalability, performance
Layered control structure
Upward propagation of aggregate data
Survival via active redundancy & self-repair
Catastrophic failures handled by top level control (human interaction)
Tahoe Retreat 06/02
John Kubiatowicz

19. Conclusion Decentralized Object Location and Routing (DOLR)
Important to be able to route to objects
… With Locality
Use of local resources whenever possible!
Continuous adaptation, repair
System never quite fully stable
Continuous convergence
Keep objects as available as possible
Tahoe Retreat 06/02
John Kubiatowicz