OMFS An Object-Oriented Multimedia File System for Cluster Streaming Server CHENG Bin, JIN Hai Cluster & Grid Computing Lab Huazhong University of Science.

Slides:



Advertisements
Similar presentations
QoS Aware Scheduling in a Cluster-Based Web Server Jiani Guo Architecture Lab Department of Computer Science and Engineering University of California,
Advertisements

Efficient Event-based Resource Discovery Wei Yan*, Songlin Hu*, Vinod Muthusamy +, Hans-Arno Jacobsen +, Li Zha* * Chinese Academy of Sciences, Beijing.
CS Spring 2009 CS 414 – Multimedia Systems Design Lecture 28 – Media Server (Part 3) Klara Nahrstedt Spring 2009.
CS Spring 2012 CS 414 – Multimedia Systems Design Lecture 35 – Media Server (Part 4) Klara Nahrstedt Spring 2012.
CHAINING COSC Content Motivation Introduction Multicasting Chaining Performance Study Conclusions.
Serverless Network File Systems. Network File Systems Allow sharing among independent file systems in a transparent manner Mounting a remote directory.
Study of Hurricane and Tornado Operating Systems By Shubhanan Bakre.
1 Multimedia Chapter Introduction to multimedia 7.2 Multimedia files 7.3 Video compression 7.4 Multimedia process scheduling 7.5 Multimedia file.
CS Spring 2011 CS 414 – Multimedia Systems Design Lecture 27 – Media Server (Part 3) Klara Nahrstedt Spring 2011.
1 Symmetrical Pair Scheme: a Load Balancing Strategy to Solve Intra- movie Skewness for Parallel Video Servers Song Wu and Hai Jin Huazhong University.
DataGrid is a project funded by the European Union 22 September 2003 – n° 1 EDG WP4 Fabric Management: Fabric Monitoring and Fault Tolerance
Mohamed Hefeeda 1 School of Computing Science Simon Fraser University, Canada End-to-End Secure Delivery of Scalable Video Streams Mohamed Hefeeda (Joint.
Beneficial Caching in Mobile Ad Hoc Networks Bin Tang, Samir Das, Himanshu Gupta Computer Science Department Stony Brook University.
1 A Framework for Lazy Replication in P2P VoD Bin Cheng 1, Lex Stein 2, Hai Jin 1, Zheng Zhang 2 1 Huazhong University of Science & Technology (HUST) 2.
Lecture 17 I/O Optimization. Disk Organization Tracks: concentric rings around disk surface Sectors: arc of track, minimum unit of transfer Cylinder:
CS 333 Introduction to Operating Systems Class 18 - File System Performance Jonathan Walpole Computer Science Portland State University.
1 Introduction to Load Balancing: l Definition of Distributed systems. Collection of independent loosely coupled computing resources. l Load Balancing.
RDMA ENABLED WEB SERVER Rajat Sharma. Objective  To implement a Web Server serving HTTP client requests through RDMA replacing the traditional TCP/IP.
The new The new MONARC Simulation Framework Iosif Legrand  California Institute of Technology.
Database Systems: Design, Implementation, and Management Eighth Edition Chapter 11 Database Performance Tuning and Query Optimization.
CS Spring 2012 CS 414 – Multimedia Systems Design Lecture 34 – Media Server (Part 3) Klara Nahrstedt Spring 2012.
A Web Services Based Streaming Gateway for Heterogeneous A/V Collaboration Hasan Bulut Computer Science Department Indiana University.
RAID-x: A New Distributed Disk Array for I/O-Centric Cluster Computing Kai Hwang, Hai Jin, and Roy Ho.
Ch 4. The Evolution of Analytic Scalability
Object-based Storage Long Liu Outline Why do we need object based storage? What is object based storage? How to take advantage of it? What's.
Performance Tradeoffs for Static Allocation of Zero-Copy Buffers Pål Halvorsen, Espen Jorde, Karl-André Skevik, Vera Goebel, and Thomas Plagemann Institute.
Database Systems: Design, Implementation, and Management Eighth Edition Chapter 10 Database Performance Tuning and Query Optimization.
File System Benchmarking
Flash An efficient and portable Web server. Today’s paper, FLASH Quite old (1999) Reading old papers gives us lessons We can see which solution among.
A Novel Adaptive Distributed Load Balancing Strategy for Cluster CHENG Bin and JIN Hai Cluster.
A Metadata Based Approach For Supporting Subsetting Queries Over Parallel HDF5 Datasets Vignesh Santhanagopalan Graduate Student Department Of CSE.
1 Towards Cinematic Internet Video-on-Demand Bin Cheng, Lex Stein, Hai Jin and Zheng Zhang HUST and MSRA Huazhong University of Science & Technology Microsoft.
HPDC 2014 Supporting Correlation Analysis on Scientific Datasets in Parallel and Distributed Settings Yu Su*, Gagan Agrawal*, Jonathan Woodring # Ayan.
Scalable Web Server on Heterogeneous Cluster CHEN Ge.
CS Spring 2011 CS 414 – Multimedia Systems Design Lecture 30 – Media Server (Part 6) Klara Nahrstedt Spring 2011.
Building a Parallel File System Simulator E Molina-Estolano, C Maltzahn, etc. UCSC Lab, UC Santa Cruz. Published in Journal of Physics, 2009.
An I/O Simulator for Windows Systems Jalil Boukhobza, Claude Timsit 27/10/2004 Versailles Saint Quentin University laboratory.
ICPP 2012 Indexing and Parallel Query Processing Support for Visualizing Climate Datasets Yu Su*, Gagan Agrawal*, Jonathan Woodring † *The Ohio State University.
A Measurement Based Memory Performance Evaluation of High Throughput Servers Garba Isa Yau Department of Computer Engineering King Fahd University of Petroleum.
Large Scale Parallel File System and Cluster Management ICT, CAS.
HPDC 2013 Taming Massive Distributed Datasets: Data Sampling Using Bitmap Indices Yu Su*, Gagan Agrawal*, Jonathan Woodring # Kary Myers #, Joanne Wendelberger.
2007/03/26OPLAB, NTUIM1 A Proactive Tree Recovery Mechanism for Resilient Overlay Network Networking, IEEE/ACM Transactions on Volume 15, Issue 1, Feb.
CCGrid, 2012 Supporting User Defined Subsetting and Aggregation over Parallel NetCDF Datasets Yu Su and Gagan Agrawal Department of Computer Science and.
CS Spring 2009 CS 414 – Multimedia Systems Design Lecture 30 – Media Server (Part 5) Klara Nahrstedt Spring 2009.
Compiler and Runtime Support for Enabling Generalized Reduction Computations on Heterogeneous Parallel Configurations Vignesh Ravi, Wenjing Ma, David Chiu.
Improving Disk Throughput in Data-Intensive Servers Enrique V. Carrera and Ricardo Bianchini Department of Computer Science Rutgers University.
MROrder: Flexible Job Ordering Optimization for Online MapReduce Workloads School of Computer Engineering Nanyang Technological University 30 th Aug 2013.
Sep. 17, 2002BESIII Review Meeting BESIII DAQ System BESIII Review Meeting IHEP · Beijing · China Sep , 2002.
Efficiency of small size tasks calculation in grid clusters using parallel processing.. Olgerts Belmanis Jānis Kūliņš RTU ETF Riga Technical University.
Storing and Serving Multimedia. What is a Media Server? A scalable storage manager Allocates multimedia data optimally among disk resources Performs memory.
A Measurement Based Memory Performance Evaluation of Streaming Media Servers Garba Isa Yau and Abdul Waheed Department of Computer Engineering King Fahd.
6.894: Distributed Operating System Engineering Lecturers: Frans Kaashoek Robert Morris
The IEEE International Conference on Cluster Computing 2010
Operating System concerns for Multimedia Multimedia File Systems -Jaydeep Punde.
Ohio State University Department of Computer Science and Engineering Servicing Range Queries on Multidimensional Datasets with Partial Replicas Li Weng,
CS Spring 2009 CS 414 – Multimedia Systems Design Lecture 27 – Media Server (Part 2) Klara Nahrstedt Spring 2009.
1 Hierarchical Parallelization of an H.264/AVC Video Encoder A. Rodriguez, A. Gonzalez, and M.P. Malumbres IEEE PARELEC 2006.
Optimizing Distributed Actor Systems for Dynamic Interactive Services
Jonathan Walpole Computer Science Portland State University
CS 414 – Multimedia Systems Design Lecture 31 – Media Server (Part 5)
Chapter 11: File System Implementation
Diskpool and cloud storage benchmarks used in IT-DSS
Oracle 11g Real Application Clusters Advanced Administration
Local secondary storage (local disks)
Accelerating MapReduce on a Coupled CPU-GPU Architecture
Chapter 7 Multimedia 7.1 Introduction to multimedia
Chapter 7 Multimedia 7.1 Introduction to multimedia
Chapter 7 Multimedia 7.1 Introduction to multimedia
Chapter 7 Multimedia 7.1 Introduction to multimedia
Chapter 7 Multimedia 7.1 Introduction to multimedia
Presentation transcript:

OMFS An Object-Oriented Multimedia File System for Cluster Streaming Server CHENG Bin, JIN Hai Cluster & Grid Computing Lab Huazhong University of Science & Technology HPCAsia, BeiJing, China, Dec. 2005

2 Outline Background Our Object-Oriented Method Improvement Schemes Performance Evaluation Conclusion

3 Background From Centralized Streaming Server to Cluster Streaming Server.

4 Background Existing cluster streaming servers I/O Bottleneck Poor Scalability Cluster File System Block-based File Operations Have to care about the media file format

5 Background Existing multimedia file systems Tiger Shark, PVFS  A general cluster file system HERMES, Symphony  focus on a QoS-aware disk scheduling algorithm, data placement, and cache policies. EXT3NS ( MM’05)  a local multimedia file system, not special for cluster streaming server

6 Background There are the following problems existing.  Most of them are used as a general cluster file system, such as PVFS  Not be specially designed for cluster streaming servers.  No uniform Interface

7 Background The features of media files  Various of file formats, such mp4, mov, avi  Different files have different objection organizations  Be consist of logical object units The features of cluster streaming server  Only perform reading operations  Seek objections according to timestamp.  Retrieve objects, such as meta object, data objects

8 Our goals An object-oriented multimedia file system specially designed for cluster streaming server  To provide an object-oriented method to fetch the media data  To present a single system image  To mask differences of multimedia file format  To separate the processing of multimedia files from cluster streaming servers  To enhance the performance of cluster streaming server.

9 Object Oriented Method Traditional processing method avi mov rm Cluster Streaming Server Parser_avi Parser_mov Parser_rm Block Parsing Object RTP Server RTSP Server consume a lot of CPU and memory resource, not efficient

10 Object Oriented Method avi mov rm Cluster Streaming Server Parser_avi Parser_mov Parser_rm RTP Server RTSP Server DB Meta Objects RTP Ojbects Uniform Interface Pre-processing Reading Apply the rule: Make the common case faster

11 Object-Oriented Framework

12 Optimization Schemes Bypassing kernel buffer Data Server NICDISK Data Server NICDISK 12 DataSend Command Kernel Module Bypassing

13 Optimization Schemes Creating time-based block indexing map WriteRead Data Object Key Frame Map

14 Performance Evaluation Simulation Setup  Our cluster streaming server has one Web Server, one Control Server, eight Data Servers, and one OMFS admin node. ---Two 1.4 GB AMD Opteron CPU ---2 GB Memory Mb/s Network Card ---avi, mp4, wmv Format  A multithreads simulator to generate requests, which accord with Poisson Distribution.

15 Performance Evaluation Metrics  Max Concurrent Stream Number  Max Throughput  CPU Overhead  Average VCR Waiting Time

16 Simulation Results Max Concurrent Stream Number about 12% improvement

17 Simulation Results Max Throughput

18 Simulation Results CPU Overhead

19 Simulation Results Delay of VCR Operation Streaming Server Type TotalTime (Sec.) AverageTime (Sec.) Darwin Streaming Server Our Cluster streaming Server with Time-based Indexing Map

20 Conclusion OMFS has the following features:  Simplify the design of cluster streaming server  Improve its performance  optimization schemes have great positive effect to the performance improvement of cluster streaming server.

21 That is all, Thank you!

22 Overview of OMFS

23 Inherited Tree of Objects

24 Uniform Interface OMFS_Write(int object_type, void * object_content) OMFS_Read(int object_type, void * object_content)