1. ETRI Site Introduction
Han Namgoong,

2. ETRI
Government sponsored Research Institute
 3,000 staffs, 500M USD (year 2009)
 focus on technologies of broadcasting, software and
contents, IT convergence, and convergence components
and materials

3. ETRI Cluster Topology (1/2)

4. ETRI Cluster Topology (2/2)
Key Masters
Server Pool
+Agent
+10,000Nodes
Monitoring
+Provisioning Proxy
+DHCP
+Agents
+256 nodes
+Provisioning Server
+LVS
+DB
+40 Group Masters
Cluster
Master
Database
File System
Group
+Global Service Dispatcher
+Disaster Recovery
+100 Data Centers
Distributed
Procesing

5. Video based Internet Application Services
ETRI Cluster Software Stack and Services ( )
Video based Internet Application Services
UGC Search Service
IPTV Service
e-Learning Service
Internet
Services
Common
Components
Video Management Components
Security
Cluster
Management
Production
Tagging
Store
Retrieval
Delivery
Large Scale
Parallel Processing
Large Scale Data Mgmt.
User/
service
authen.
Cluster
Orchestration
Service Data
Management
Job Partition
and Merge
Distributed Data Store
Device/
kernel
authen.
Distributed Job
Scheduling
Provisioning
Data Access and
Recovery
Global File System
Service
Mgmt.
File Metadata
Management
File Store
And Replication
File Remote
Backup/Archiving
Resources
Monitoring
Platform OS and HW
Low Power OS
Node Manager
Low Power HW

6. Monitoring Tool for Large Cluster System
Research Topics ( )
Monitoring Tool for Large Cluster System
- current monitoring SW  heavy overhead cpu/memory
 small/light monitoring tool
2. Management of Big Video Feature Data
- Google YouTube(2006)
* Upload : 70,000 per day, Viewing : 100 Million play per day
- Keyword based Retrieval (vague, imprecise,..)
- Content based Retrieval (not simple interface/slow result)
 Integrated Query(Keyword + Content based)
3. Elimination of Duplicated Video Data
- Lots of same video files occupied storage spaces.
 File (NOT data) deduplication is strongly required.

7. Cloud stack (OSS) for evaluation
Schedule ( )
Phase 1 : ~
Cloud stack (OSS) for evaluation
- System management/Monitoring tool
- Middleware(Web/AP/DB server)
- Linux(CentOS,..)
- Virtualization(Xen, KVM)
- Distributed file-system/DB (Hadoop, Hbase)
- Authentication(OpenLDAP)
Evaluation point
- Error recovery procedure, configuration, structure
- Add resource(planned, unexpected)
- Remove resource by degrade of load, and Migration
- Overhead of virtualization, distributed file-system, distributed DB
- Authentication between systems
Source : Tomomi Suzuki, Status report of Cloud Computing activity,
Japan OSS Promotion Forum,

8. Schedule ( )
2. Phase 2 : ~
Selection of Requirements
Develop , Test and Deployment
- Monitoring Tool for Large Cluster System
- Management of Big Video Feature Data
- Elimination of Duplicated Video Data
Distributed Processing
- Fail-over of task execution node and job manage node
- Distributed task processing based on data location
- Configurable job scheduling : 9 policies
……..
Distributed Data Management based on Hadoop/Hbase
- Multi dimensional map model
- Support a composite row key
- Column group based storage model
- Distribute partitions splited by a composite row key
- Data access control by user and privilege management
………

9. Parallel Processing Model
Plans, Expectations (1/3)
Category
Hadoop/MapReduce
What
Expectations
Parallel Processing Model
- Map/Reduce Programming Model
- I/O Source : HDFS, LFS, Hbase
- Map/Reduce Programming Model
I/O Source : + new-FS, new-DB
Enlargement of parallel processing target
Cluster Size
- Thousand nodes
- Manually configure
Thousand nodes
Automatically configure
Easy to manage parallel processing cluster
Job Control
- None
- Execution control based on user
Access control to parallel processing cluster
Job Scheduling
- Direct Priority, FIFO
- Priority management by job
- 9 configurable scheduling policies
- Priority management by job, Group and user
Support of various jobs
Task Distribution
- Consideration of data location and node position
- Consideration of data location, node position and node resource
Increase of node utilization
High Availability
- Fail-over of task execution node
Fail-over of task execution node
Fail-over of job manage node
- Increase availability
Reduction of Job execution time

10. Plans, Expectations (2/3)
Category
Hbase
What
Expectations
Data Model
- Multi dimensional map
- Row key : single field
Multi dimensional map
Row key : composite field
Easy to construct key
Video Manage
- None
High dimensional index manage
k-NN search
Provide large scale video content based retrieval
Data Storage Model
Column oriented
Per column
Column oriented
Per column group
Performance enhancement
Data Distribution
Distribute partitions splited by row key
- Distribute partitions splited by row key
- Distribute clusters by high dimensional index
Performance enhancement of key-based/content based retrieval
Access Control
- None
User management
Privilege management of table/column
Provide data security
High Availability
Fail-over of partition management node
 serial processing log file and parallel recovery
Fail-over of partition management node
 parallel processing log file and parallel recovery
Fail-over of master node
- Increase availability
- Reduction of down time
Query Language
- Use in shell
- Use in application
Easy to develop application

11. Plans, Expectations (3/3)
Category
Function
What
OSCAR
Cluster
Orchestration
Structure
Hierarchical
Flat
Scalability
Automatic Reconfiguration
Pxe+DHCP
Availability
Independent HA Tool
Activ-active(2 head node)
Management Interface
Web
X-GUI, Command(C3)
Communication
XML
XDR, XML
IP Management
Server Configuration
DHCP auto/static
Maximum Nodes
10,000 per data center /
Max. 1,000,000
Oscar 440
Load Balancing
Front-end LVS, Back-end new-DP
Front-end PBS, TORQUE, MAUI
Service
Management
Node Reconfiguration
By Load Balancing
Yes
None
Master
Master Node Configuration
Hierarchy (Key Master, Cluster Master, Group Master)
Head node
Resources
Monitoring
Monitoring Tool
Proprietary
Ganglia
Provisioning
Provisioning (image)
OS imaging
Provisioning (streaming)
SW streaming
SW tar/rpm

12. Thank you