Two-Tier Architecture of OSD Metadata Management Xianbo Zhang, Keqiang Wu 11/11/2002

Region Balance lookup cost & location information update cost Data Locality: –Make the metadata server close to the user –Make user data close to the user Enterprise: campuses geographically distributed, a lot of data traffic happens within the campus. Each campus may have special security, policy control, performance requirements

P2P among Regions Good Scalability Resilience: each region has relatively independent metadata, functionality, resilient to network failure such as WAN connection loss Availability: Any server can take the responsibility of the down server

Region Self-organization Configure server to recognize potential OSD devices, clients. Device/client via configuration, neighbor or broadcasting to join the region. Region splitting/merging: operation happens based on performance requirements (e.g. throughput, response time, …). Size of region has several effects.

Server Main Responsibilities Naming: mapping URN to object GUID, can be done using mechanism similar to DNS Location: mapping GUID to region where storing objects. P2P systems such as CAN, Pastry may be modified/adopted here. Object migration and replication (based on access monitoring) –Done by active object (possible) –Done by metadata servers –Concurrent accesses

Mapping ObjectGUID –ObjectGUID=RegionID+ObjectID –ObjectGUID independent from Reg Primary copy of object will be stored in object owner’s region ObjectGUID randomly generated, possible embedded with user info RegionIDs=Hash1(ObjectGUID) –Object may be highly demanded in other region, how to do migration/replication? Stored to OSD Device –How to map ObjectGUID to specific device? –Security (R/W permission), Load balance

Object Mapping Issues How to efficiently find Region based on RegionID How to locate nearby object copy based on objectGUID –How to define nearby copy # of hops? Server response latency, bandwidth? How to find a good location to migrate/replicate hot object (it’s possible actual users of the object is far away from the the mapped location) –R/W pattern –User geographic location

Fault Tolerance Server: three levels of failover support, that is, server local backup/mirroring, server remote backup/mirroring, for most important data, erasure code can be used among servers for data disaster recovery. –Regional backup server can also be used for load balance

Content based Searching Some mechanism similar to CAN Content search sent to region managers, and sent to OSD devices. Devices respond with objectGUIDs if they have the requested contents. Concept of Active Disk can be helpful.

How ObjectID Map to Region Explanation is detailed in OceanStore, Pastry, etc. Region ID