1. EnsemBlue: Integrating Distributed Storage and Consumer Electronics Daniel Peek and Jason Flinn University of Michigan

2. 2 Bringing Distributed Systems Home Distributed systems ignore Consumer Electronic Devices (CEDs) Much user interaction is through CEDs Integrate CEDs into distributed systems

3. University of Michigan3 CED Integration Difficult Application-specific resources Heterogeneous interfaces Closed platforms –Cannot execute our code

4. University of Michigan4 Today

5. University of Michigan5 Hypothesis: Distributed file systems can help manage CEDs and multimedia EnsemBlue My Data My Devices

6. University of Michigan6 EnsemBlue Based on BlueFS [Nightingale 04] –Client-server design –Single namespace –Supports mobile clients –Designed for small group of users File Server

7. University of Michigan7 Outline Motivation Device-Specific Namespaces Device-Specific Functionality Disconnected Collections of Devices Evaluation Conclusion

8. University of Michigan8 Device-Specific Protocol Connecting CEDs CEDs cannot perform DFS protocol DFS Protocol? Distributed File System Device-Specific Protocol Attach: computer speaks for CEDWorks with disconnected clients DFS Protocol

9. University of Michigan9 Device-Specific Namespaces User prefers one organization CED requires another organization Classical Jazz F01 F02 F03 F04

10. University of Michigan10 Where to store the mapping? Device-Specific Namespaces Cannot store in DFS due to partial caching Make CEDs self-describing

11. University of Michigan11 Receipts What should mapping look like? Object 1.123.46F2 ~dpeek/classical/song.mp3/iPod/f27/ABCD.mp3 Path lookups may fail with partial caching Receipts bypass path resolution

12. University of Michigan12 Device-Specific Execution Updating indexes Transcoding Type-specific caching Organizers

13. University of Michigan13 Device-Specific Execution Cannot execute on CED Leverage general computers –Take action when files change Problem: Need notification of file changes

14. University of Michigan14 Notification Via Persistent Queries Don’t need new mechanism Leverage cache consistency mechanism –Structure notifications as file system object –Robust to crash –Handles disconnected operation updatestruncate readprocess *.mp3

15. University of Michigan15 MP3 M4A Persistent Queries Example: M4A to MP3 transcoder File Server M4A to MP3 Transcoder *.m4a M4A MP3 *.m4a MP3

16. University of Michigan16 Disconnected Devices Server gives safety, availability, consistency Server mediates communication File Server

17. University of Michigan17 Disconnected Devices Disconnected devices cannot interact! File Server

18. University of Michigan18 Disconnected Devices Peer-to-Peer better supports mobility CEDs store and forward updates File Server

19. University of Michigan19 Ensembles Ensembles can avoid these problems Select a pseudo file server –Consistent view of all files –Improved data access –Update propagation Pseudo File Server ? ?

20. University of Michigan20 Outline Motivation Device-Specific Namespaces Device-Specific Functionality Disconnected Collections of Devices Evaluation Conclusion

21. University of Michigan21 Microbenchmarks Persistent query execution is fast –<200 ms for 1,000s of matches, GBs of data Persistent queries have low overhead –Negligible even for 10,000 queries Ensemble formation as fast as 20 ms –Proportional to data transfer size

22. University of Michigan22 Organizing My Photos Photo Organizer iCal *.jpg File Server

23. University of Michigan23 Managing My Music iPod DB Updater MP3 DB MP3DB MP3DB *.mp3 File Server MP3

24. University of Michigan24 Conclusion Added CEDs to distributed systems Device-specific data organization Device-specific functionality Supports TiVo, iPod, digital cameras, media players, cell phones, PDAs

25. University of Michigan25 Questions ?