1. ©2001 Pål HalvorsenINFOCOM 2001, Anchorage, April 2001 Integrated Error Management in MoD Services Pål Halvorsen, Thomas Plagemann, and Vera Goebel University of Oslo, UniK- Center for Technology at Kjeller Norway

2. ©2001 Pål HalvorsenINFOCOM 2001, Anchorage, April 2001 Overview Application Scenario Integrated Error ManagementIntegrated Error Management –Basic idea –Code requirements –Possible solutions –Evaluation Conclusions

3. ©2001 Pål HalvorsenINFOCOM 2001, Anchorage, April 2001 Media-on-Demand server: Applicable in applications like News- or Video-on-Demand provided by city-wide cable or pay-per-view companies Application Scenario Network Multimedia Storage Server Project goals: Optimize performance within a single server: Reduce resource requirements Maximize number of clients Retrieval is the bottleneck: Some important factors: Memory management Communication protocol processing Error management

4. ©2001 Pål HalvorsenINFOCOM 2001, Anchorage, April 2001 Traditional Error Management FEC Encoder FEC Decoder

5. ©2001 Pål HalvorsenINFOCOM 2001, Anchorage, April 2001 Integrated Error Management FEC Encoder FEC Decoder

6. ©2001 Pål HalvorsenINFOCOM 2001, Anchorage, April 2001 Correcting Code Requirements Erasure (known errors) and error (unknown errors) correcting Decoding throughput (recovery performance) Amount of redundant data Applicable within a single file Cost of decoding function dependent on the corruption rate

7. ©2001 Pål HalvorsenINFOCOM 2001, Anchorage, April 2001 Possible Schemes Error and erasure correcting codes  Bad performance (< 1.7 Mbps)  Not dependent of corruption rate (< 3.5 Mbps without any corruption) Erasure correcting codes  Adequate performance (> 6 Mbps)  Corrects only erasures Combined schemes –Erasure correcting / Error correcting  Capable of recovering from most errors  Even more redundant data  Performance still a problem (< 1.7 Mbps) –Erasure correcting / Error detection  Adequate performance (> 6 Mbps)  Capable of recovering from most errors  (Almost) no additional redundant data

8. ©2001 Pål HalvorsenINFOCOM 2001, Anchorage, April 2001 Prototype Design Combination of the traditional UDP checksum and an erasure correcting code (Cauchy-based Reed-Solomon Erasure) Disk array with 8 disks, i.e., 7 information disks and 1 parity disk: Write operations a la RAID level 4/5 Read operations a la RAID level 0 One codeword, contained in one or more stripes, is read as one read operation Each striping unit consists multiple symbols each transmitted as a UDP packet

9. ©2001 Pål HalvorsenINFOCOM 2001, Anchorage, April 2001 Codeword Size and Amount of Redundancy Requirements: Codeword_size  n  stripe_size, n  Z Recover from one disk failure and (most) network errors  Our current approach: –Using a codeword of 256 symbols with 32 redundant symbols  corrects one out of 8 disks  corrects most errors in the network according to our error model, i.e., any 32 out of 256 packets

10. ©2001 Pål HalvorsenINFOCOM 2001, Anchorage, April 2001 Packet (Symbol) Sizes - I Correcting code throughput suggests a large symbol size Low throughput

11. ©2001 Pål HalvorsenINFOCOM 2001, Anchorage, April 2001 Correcting code (startup) delay suggests a small symbol size Packet (Symbol) Sizes - II Low throughput High startup delay

12. ©2001 Pål HalvorsenINFOCOM 2001, Anchorage, April 2001 Results and Observations – I Experimental Setup Assuming coding performance is only bottleneck Transmitted a 225 MB file (corresponding to a 5 minutes 6 Mbps playout) Used a worst-case loss scenario Used several different machines

13. ©2001 Pål HalvorsenINFOCOM 2001, Anchorage, April 2001 Results and Observations – II Server Side No extra disk space needed No additional time needed to retrieve parity data No overhead managing retransmissions Increased buffer space and bandwidth requirement of 12.5% storing and transmitting redundant data. Encoding throughput limitation of ~3 Mbps (Intel, 166 MHz) to ~25 Mbps (Intel, 933 MHz) is eliminated by retrieving parity data from disk  The server can support more concurrent users

14. ©2001 Pål HalvorsenINFOCOM 2001, Anchorage, April 2001 Results and Observations – III Client Side Increased buffer requirement for decoding the received data (2 MB, 3 MB, 5 MB, and 9 MB using 1 KB, 2 KB, 4 KB, and 8 KB packets, respectively) Additional CPU requirements recovering lost/corrupted data Additional (worst case) startup delay between ~0.1 s (Intel, 933 MHz) and ~4.5 s (Intel, 166 MHz) decoding the first block can be experienced In-time decoding  hiccup free presentation

15. ©2001 Pål HalvorsenINFOCOM 2001, Anchorage, April 2001 Conclusion The INSTANCE project aims at optimizing the data retrieval in an MoD server Integrated Error Management Integrated Error Management Future work More information: www.unik.no/~paalh/instance