1. CS 414 - Spring 2011 CS 414 – Multimedia Systems Design Lecture 27 – Media Server (Part 3) Klara Nahrstedt Spring 2011

2. Administrative MP3 – posted today CS 414 - Spring 2011

3. Some Interesting Facts DBMS2.com Source (May 2009)  Facebook had 400 terabytes of disks managed by Hadoop/Hive with an approx. 6:1 compression ratio  Facebook’s Hadoop/Hive system ingests 15 terabytes of new data per day  Facebook had 610 Hadoop nodes (in May 2009) running in a single cluster and was heading for 1000 Yahoo had 2000 nodes (in May 2009) and was heading for 4000 CS 414 - Spring 2011

4. Some Interesting Facts Source: www.slideshare.net (March 2011)www.slideshare.net Current data sets:  NYSE: 8PB; Google > 12PB; Data Volumes:  NYSE: 1.5 TB daily;  Facebook: 350 M users; 3.5B shared items/week  Facebook adds > 100K users, 55M ‘status’ updates, 80M photos daily CS 414 - Spring 2011

5. Outline Disk Scheduling SCAN-EDF Group Sweeping Mixed Scheduling Admission Control File System Metadata/Indexing Block Size Issues CS 414 - Spring 2011

6. Disk Scheduling Policies Goal of Scheduling in Traditional Disk Management  Reduce cost of seek time  Achieve high throughput  Provide fair disk access Goal of Scheduling in Multimedia Disk Management  Meet deadline of all time-critical tasks  Keep necessary buffer requirements low  Serve many streams concurrently  Find balance between time constraints and efficiency CS 414 - Spring 2011

7. EDF (Earliest Deadline First) Disk Scheduling Each disk block request is tagged with deadline Policy:  Schedule disk block request with earliest deadline  Excessive seek time – high overhead  Pure EDF must be adapted or combined with file system strategies CS 414 - Spring 2011

8. EDF Example CS 414 - Spring 2011 Note: Consider that block number Implicitly encapsulates the disk track number

9. SCAN-EDF Scheduling Algorithm Combination of SCAN and EDF algorithms Each disk block request tagged with augmented deadline  Add to each deadline perturbation Policy:  SCAN-EDF chooses the earliest deadline  If requests with same deadline, then choose request according to scan direction CS 414 - Spring 2011

10. Implementation of SCAN-EDF Notation:  D i be deadline of disk block request ‘i’  N i be track (block) position on disk  N max be maximum number of disk tracks Deadline Modification:  D i + f(N i )  f(N i ) converts track number of ‘i’ into a small perturbation of deadline  Perturbation small enough so that D i + f(N i ) ≤ D j + f(N j ) for D i ≤ D j Possible f(N i ) = N i /N max CS 414 - Spring 2011

11. SCAN EDF Example (N max = 100) CS 414 - Spring 2011

12. Enhanced SCAN-EDF (1) Use more accurate perturbation of deadline Consider  Actual track position of disk head ‘N’  N max – max number of disk tracks  N i – next track to be considered CS 414 - Spring 2011 Head Moves Upwards

13. Enhanced SCAN-EDF (2) Algorithm:  If head moves upwards (towards N max ), then  (a)  (b) CS 414 - Spring 2011

14. Enhanced SCAN-EDF (3) If head moves downwards (towards 1), then (a) (b) CS 414 - Spring 2011

15. Group Sweeping Algorithms Policy:  Each Request consists of (Deadline, Block Number )  Disk Block Requests served in cycles  In one cycle, requests divided into groups according to similar deadlines  Within group use SCAN  As we retrieve blocks, we may need smoothing buffers to ensure continuity CS 414 - Spring 2011

16. Group Sweeping Example CS 414 - Spring 2011

17. Mixed Scheduling (uses SSTF – Shortest Seek Time First) CS 414 - Spring 2011 Example of SSTF

18. Mixed Scheduling CS 414 - Spring 2011 SSTF (Shortest Seek Time First) + Balanced Strategy

19. Admission Control CS 414 - Spring 2011 Client 1 retrieves K1 blocks in one round Client 2 retrieves K2 blocks Client 3 retrieves K3 blocks Client 4 retrieves K4 blocks Server

20. Admission Control Disk block requests are timed  Media server must determine admit a stream serve (schedule) a stream without having negative effect on other streams already serviced. Deterministic Guarantees  Admission control considers worst case scenario when admitting new stream  Constrained Disk Placement Example: M - size of blocks, G – size of gabs, r dt – data transfer of disk CS 414 - Spring 2011

21. Admission Control CS 414 - Spring 2011 α – overhead switching from one round (‘j-1’) to another round (j), and then transmitting the first block of the ‘j’ round β – transmission time of (K i -1) blocks in ‘j’ round, i=1,..4 K i – number of blocks retrieved by client ‘i’ η i – Block granularity retrieved for client ‘i’ (e.g., in Bytes) R i – playback rates of client ‘i’ (e.g., in Bytes per second) Minimal Intra- K i blocks delayCost to switch and move K i blocks

22. Admission Control Statistical Guarantees  Deadlines are guaranteed with certain probability  Admission control considers statistical behavior of the disk system while admitting new stream (average performance) Best effort Service  No guarantees CS 414 - Spring 2011

23. Conclusion The data placement, scheduling, are very important for any media server design and implementation. Still need to consider multimedia file system and caching – next lecture CS 414 - Spring 2011