OSD Metadata Management Xianbo Zhang 10/14/02

Region Concept . . . Client Metadata Server Client IP Networks Client osd . . . Client IP Networks osd Client osd osd osd Client osd

Region Concept . . . Client Metadata Server1 Client IP Networks osd . . . Client IP Networks Region 2 osd Region 3 Metadata Server2 Metadata Server3 Client osd osd osd Client osd

Region Concept (Cont.) Interested networks partitioned into regions Each region has local metadata server, osd devices and clients Each metadata server has information about osd devices, objects, and client policies in this region. Also each metadata server has direct or indirect way to communicate with other servers. All metadata servers form a decentralized system, each server has the same(or similar) capabilities and responsibilities

Region Concept (Cont.) With the services and information provided by metadata system, object can migrate or replicate itself to the right region to make most of the data request/access happen in the same local region, minimizing network latency, reducing WAN traffic, and providing good locality, scalability, high performance, high availability, … Special algorithms/data structures need to be developed to achieve these goals.

Functionality Requirements for Region based Infrastructure Region self-organization: Manage dynamic region division, merging based on specified criteria. osd device registration: through simple configuration, the device can register with any known metadata server and will be managed by the right region. Client registration: client registers with any known server, and will be migrated to the nearest server automatically along with its capabilities

Functionality Requirements for Region based Infrastructure (Cont.) Floating among regions: Client: Client with its policy may move among regions. Object: Object may be migrated or replicated among regions according to object access pattern, object attribute requirements such as data location sensitivity. Device: Moving osd device from one region to another region is transparent to client. The transparency may be achieved through object migration/replication.

Functionality Requirements for Region based Infrastructure (Cont.) Fault-tolerance (avoiding single point of failure): Server: three levels of failover support, that is, server local backup/mirroring, server remote backup/mirroring, for most important data, erasure code can be used among servers for data disaster recovery. Object: object replication among osd devices, among different regions

Functionality Requirements for Region based Infrastructure (Cont.) Data access pattern monitoring: Servers monitor data requests within region. According to the access pattern, object may be migrated, replicated or stripped among osd devices to achieve good locality, load balancing and high scalability. Load balance among metadata servers can be achieved based on this same data access monitoring.

Functionality Requirements for Region based Infrastructure (Cont.) Common services provided: Object location : Locate the nearby copy of requested object upon authenticated client request. Message routing : overlay network built among servers, message can be routed efficiently for resource discovery and system management. Here each server acts as server, client and router.

Functionality Requirements for Region based Infrastructure (Cont.) Common services provided (Cont.): Distributed lock services for metadata and objects. Object consistency service among osd devices according to object’s unique consistency requirements Automatic backup based on object attributes, supporting remote backup Remote mirroring

Schematic Explanation of Data Access within a Region After authentication and authorization, client directly access data with capability from metadata server IP Networks Client Metadata Server osd osd Client osd

Schematic Explanation of Data Access outside a Region Distributed lookup protocol find nearby object copy via the distributed system formed by metadata servers Direct data access to that osd device with required data IP Networks Metadata Server Client osd osd Metadata Server osd osd Client osd

Enterprise vs. Globe/Internet Enterprise has one consistent administration policy. Each client of global storage system has its own. Enterprise has trusted servers. Global storage system has untrusted servers. Enterprise has limited size when comparing to Internet, object location could be different from popular peer-to-peer global systems.