1. Research Opportunities in IP Wide Area Storage
George Porter
Li Yin
Department of EECS
U.C. Berkeley
12/26/2018
SAHARA Retreat

2. Outline Trends Challenges for wide-area storage
Programmability inside networks
Common techniques to hide latency
Functionality that will benefit applications
Network Support for that functionality
Reconsidering the programmability model and application space
Feedback

3. Outline Trends Challenges for wide-area storage
Programmability inside networks
Common techniques to hide latency
Functionality that will benefit applications
Network Support for that functionality
Reconsidering the programmability model and application space
Feedback

4. Storage Wide-Area Networking
Metro Area
SAN technology works well in relatively small area (metro-wide)
There is a desire to implement storage applications in the wide-area
Comparable performance as small area storage applications

5. Outline Trends Challenges for wide-area storage
Programmability inside networks
Common techniques to hide latency
Functionality that will benefit applications
Network Support for that functionality
Reconsidering the programmability model and application space
Feedback

6. Challenges in Wide-Area Storage
Speed of light is constant
Long distance implies propagation delay
Network dynamics
Variation of cross traffic load
Routes changes
Increasing storage capacity
Transmit huge amount of data across wide-area

7. Challenges in Wide-Area Storage
Simple operation: Host writes data to the remote target disk
Send data to the remote target
Write disk
Where is the bottleneck?
Disk?
Network Link?
Distance?
Performance Degradation
Local Disk Write Operation Time
Extra Delay Caused by Network Local Operation Time
Extra Delay Caused By the Network
Time

8. Challenges in Wide-Area Storage
Three Cases:
Case 1: Limited link bandwidth
Case 2: Small data set with high bandwidth
Case 3: Large data set with high bandwidth

9. Challenges in Wide-Area Storage
Case 1: Limited link bandwidth
Transmission Time
Local Disk Write Operation Time
More data to be transmitted
Time
Time
Extra Delay Caused By the Network

10. Challenges in Wide-Area Storage
Case 1: Limited link bandwidth
As more data to be transmitted:
The performance degradation caused by the transmission delay gets larger
Propagation delay does not matter
As the disk getting faster, more bandwidth is required to shift the bottleneck away from the network

11. Challenges in Wide-Area Storage
Case 2: Small data set with high link bandwidth
In this case, the throughput is very sensitive to the distance, especially when it becomes of the order of the disk latency
Local Disk Write Operation Time
Larger Distance
Time
Time

12. Challenges in Wide-Area Storage
Case 3: Large data set with high link bandwidth
Local Disk Write Operation Time
More Data to be Transmitted
Time
Time
Time
Extra Delay Caused By the Network

13. Challenges in Wide-Area Storage
Case 3: Large data set with high link bandwidth
In this case, disk is the bottleneck, the network only introduces the propagation delay which can be ignored as more data to be transmitted
As the disk getting faster, more bandwidth is required to shift the bottleneck away from the network

14. Challenges in Wide-Area Storage
Where is the bottleneck?
Link Bandwidth
Size of data to be transmitted
Disk Speed
Key issue in the wide-area storage is how to reduce the latency
Latency introduced by the network
Latency introduced by the storage

15. Outline Trends Challenges for wide-area storage
Programmability inside networks
Common techniques to hide latency
Functionality that will benefit applications
Network Support for that functionality
Reconsidering the programmability model and application space
Feedback

16. Common Techniques to Hide Latency
Caching
Parallelism
Pipelining
Prefetching …
Where and how to implement these techniques for wide-area storage applications?

17. Code at edge –vs- in the fabric
Location of data separated from use of data
Idea is to put processing near the data it acts on
Better visibility into network conditions, dynamics
Big performance gains if we can act on streams of data in the datapath
Network processors are more powerful today
A good match?

18. Outline Trends Challenges for wide-area storage
Programmability inside networks
Common techniques to hide latency
Functionality that will benefit applications
Network Support for that functionality
Reconsidering the programmability model and application space
Feedback

19. Gather Digital animation editing Large dataset visualization
Synchronous
Asynchronous
N-to-1 disk copies (KaZaa)
Recreate dataset from multiple sources/disks (scientific experiment)
Restore backup
Digital animation editing
Large dataset visualization

20. Gather Techniques Network Primitives Digital animation editing
Caching
Parallelism
Prefetching
Network Primitives
FS semantic information
Store block location state in router
View into network routes/conditions
Table lookup in router
Modify disk requests to point to correct locations
Join data streams to deliver coherent data to app
Orthogonal path selection
Synchronous
Digital animation editing
Large dataset visualization

21. Gather Digital animation editing Large dataset visualization
Synchronous
Asynchronous
N-to-1 disk copies (KaZaa)
Recreate dataset from multiple sources/disks (scientific experiment)
Restore backup
Digital animation editing
Large dataset visualization

22. Gather Techniques Network Primitives
Join data streams to deliver coherent data to app
Orthogonal path selection
Volume state in routers
Replicate SCSI requests
Reorder SCSI responses
Techniques
Pipelining
Avoid congestion/optimize for bandwidth
Network Primitives
FS semantic information
Store block location state in router
View into network routes/conditions
Table lookup in router
Modify disk requests to point to correct locations
Asynchronous
N-to-1 disk copies (KaZaa)
Recreate dataset from multiple sources/disks (scientific experiment)
Restore backup

23. Scatter State dissemination Updating mapping tables
Synchronous
Asynchronous
Disaster-recovery application
Experimental data unloading
State dissemination
CDN/web server updating?
Gaming?
Updating mapping tables

24. Scatter Techniques Network Primitives State dissemination
Delay-sensitive path selection
Congestion avoidance
Synchronization
Network Primitives
Network monitoring
FS semantic information
Store block location state in router
View into network routes/conditions
Table lookup in router …
Synchronous
State dissemination
CDN/web server updating?
Gaming?
Updating mapping tables

25. Scatter State dissemination Updating mapping tables
Synchronous
Asynchronous
Disaster-recovery application
Experimental data unloading
State dissemination
CDN/web server updating?
Gaming?
Updating mapping tables

26. Scatter Techniques Network Primitives Disk location/selection
Load balancing
Physical distance knowledge
Network Primitives
Network monitoring
FS semantic information
Store block location state in router
View into network routes/conditions
Table lookup in router …
Asynchronous
Disaster-recovery application
Experimental data unloading

27. Outline Trends Challenges for wide-area storage
Programmability inside networks
Common techniques to hide latency
Functionality that will benefit applications
Network Support for that functionality
Reconsidering the programmability model and application space
Feedback

28. Useful Network Primitives What is reasonable and possible?
FS semantic information
Store block location state in router
View into network routes/conditions
Table lookup in router
Modify disk requests to point to correct locations
Join data streams to deliver coherent data to app
Orthogonal path selection
Volume state in routers
Replicate SCSI requests
Reorder SCSI responses
Others?

29. Outline Trends Challenges for wide-area storage
Programmability inside networks
Common techniques to hide latency
Functionality that will benefit applications
Network Support for that functionality
Reconsidering the programmability model and application space
Feedback

30. Your Feedback?
12/26/2018
SAHARA Retreat