Architecting Your Data and Metadirectory Model Brendan Bellina, University of Notre Dame Base CAMP - Tempe, Arizona February 5-7, 2003 Copyright Brendan Bellina, This work is the intellectual property of the author. Permission is granted for this material to be shared for non-commercial, educational purposes, provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of the author. To disseminate otherwise or to republish requires written permission from the author.

Base CAMP - February 5-7, 2003Middleware: Directories 2 Presentation Overview - Visual

Base CAMP - February 5-7, 2003Middleware: Directories 3 …only strong characters can resist the temptation of superficial analysis. Albert Einstein

Base CAMP - February 5-7, 2003Middleware: Directories 4 Enterprise Directory Model Dr. Thomas Barton

Base CAMP - February 5-7, 2003Middleware: Directories 5 What is meant by “Metadirectory”? A technology or class of functionality required to build an enterprise directory infrastructure. Provides the infrastructure capable of maintaining consistency and data integrity between the chosen enterprise directory and the other local and system- or application-specific directories that will always be present in the organization. - “Enterprise Directory Infrastructure: Meta-directory Concepts and Functions”, Jamie Lewis, The Burton Group, July, 1998

Base CAMP - February 5-7, 2003Middleware: Directories 6 Role of the Metadirectory The glue that binds directories together The directory umbrella which covers all directories The duct tape of your directory infrastructure

Base CAMP - February 5-7, 2003Middleware: Directories 7 Metadirectory Processes - Overview The “Join” -Using identity matching to produce a registry of constituents with links (aliases or alternate keys) back to source systems. -“Intelligence” -Managing how data is inserted, modified, and deleted from the registry based upon the business rules of the institution. Consumer Provisioning Notifying/populating the directory consumers appropriately.

Base CAMP - February 5-7, 2003Middleware: Directories 8 Example – Whatsamatter U

Base CAMP - February 5-7, 2003Middleware: Directories 9 Identify Your Data Sources

Base CAMP - February 5-7, 2003Middleware: Directories 10 Directory Sources – You want sources? We got sources! Faculty Students Donors Alumni accounts Windows 2000 Windows NT etc/passwd Novell etc/aliases Oracle Trustees Vendors Athletic Fans Portal users Applicants Staff Affiliates Retirees And more!!!

Base CAMP - February 5-7, 2003Middleware: Directories 11 Source Issues - Quantity of diverse sources - Platform differences - Differences in quality of data entered - People with multiple simultaneous roles - Data ownership issues – politics - Varying availability of data sources - Sometimes too much data – 34 address types?!?

Base CAMP - February 5-7, 2003Middleware: Directories 12 Identity Matching Haven’t I seen you somewhere before? Students who are also part-time staff Staff or faculty who take classes People who arrive, and leave, and return, and…

Base CAMP - February 5-7, 2003Middleware: Directories 13 Identity Matching Generally forced to use infrequently changing attributes to attempt to determine when two records describe the same person: -U.S. Social Security Number or other government assigned unique single lifetime pseudo-meaningless short easy-to-memorize alpha-numeric identifier -Formal name (at birth or initial contact) -Date of birth -Gender (at birth or initial contact) -Permanent home address … Quality of the data really matters!

Base CAMP - February 5-7, 2003Middleware: Directories 14 Building the Registry: Choice of ETL Tools Choose an ETL (extract-transform-load) tool: - Perl scripts – most common approach at this time, fairly easy to write, can be difficult to maintain - Java applications - So whatever happened to MetaMerge? -WANTED: MACE-Dir current effort underway to document recommended feature-sets based on member experience

Base CAMP - February 5-7, 2003Middleware: Directories 15 Building the Registry: Choice of Storage -Relational database -Referential integrity controls -Support for complicated relations -Very scalable But… Need to take the time to select and model the data.

Base CAMP - February 5-7, 2003Middleware: Directories 16 Building the Registry: Choice of Storage -LDAP Directory -Not good choice for historical content -Not good for large objects -Not good for frequent updates But… Standard Object Classes already exist – inetOrgPerson, eduPerson, posixAccount, etc.

Base CAMP - February 5-7, 2003Middleware: Directories 17 Building the Registry: Choice of Storage Why not both?!? When time is limited (and when isn’t it?) it is worth considering which is best to do first.

Base CAMP - February 5-7, 2003Middleware: Directories 18 Building the Registry: Choice of Model Choose a model: “fat” or “thin” “thin”: registry will contain only the information required to provide linkages back to systems of record. Requires systems of record to be both highly available and readily accessible. “fat”: registry will contain and serve, in addition to linkage information, information about an entry to consuming applications, reducing the dependency on the systems of record. Fat registries are more common than thin registries.

Base CAMP - February 5-7, 2003Middleware: Directories 19 Building the Registry: A Notre Dame moment -Fortunate enough to have a pre-existing “registry” for most but not all people. Unfortunate enough that it is a proprietary non-relational database. -For rapid development put up an LDAP directory populated from the registry and supplemental data sources. -Vendor apps use it for authN, but internally developed applications have continued to access source systems directly.

Base CAMP - February 5-7, 2003Middleware: Directories 20 Metadirectory Processes: “Intelligence” The application of an institution’s business rules and policies within the metadirectory. This involves the creation of a unique identifier (guid), rules regarding the creation and removal of registry entries and the population of attributes, and providing for operational reporting and auditing requirements.

Base CAMP - February 5-7, 2003Middleware: Directories 21 Unique Identifiers “There can be only one!!!” One entry per person, that is. Establish a globally unique identifier (guid) for each person in the registry. - Unchanging and persistent - Non-recyclable - Unique - Meaningless - Hidden

Base CAMP - February 5-7, 2003Middleware: Directories 22 Addressing Institutional Policies - Reformatting data to meet standards (telephone) - Breaking up data into discrete parts (addresses, names) - Consolidating/summarizing data (statuses) - Population of default attributes - Population of groups - Default authorizations - Resolving partial or missing data from sources

Base CAMP - February 5-7, 2003Middleware: Directories 23 Operational Design Requirements - Data flow requirements – batch or real-time? - Recovery planning – thresholds, roll-back, grace periods, logging - Problem resolution tools for the helpdesk and administrators - Audit reporting

Base CAMP - February 5-7, 2003Middleware: Directories 24 Metadirectory Processes – Consumer Provisioning Consumers are the applications which make use of information presented in the enterprise directory infrastructure. The metadirectory provisioning process ensures that data is made available to the consumer interfaces. Often modern consumers can interface via the LDAP protocol, but often multiple LDAP directories are required to meet consumer needs.

Base CAMP - February 5-7, 2003Middleware: Directories 25 Why Being “LDAP-Enabled” Isn’t Enough There is no clear definition of what being “LDAP- enabled” really means. Vendor usage of LDAP terminology may mistakenly (?) lead to false assumptions and unrealistic expectations. Conclusion: Examine vendor offerings carefully. Remember: “LDAP” is a four-letter word!!! The jury is still out: MSAD Java Bush

Base CAMP - February 5-7, 2003Middleware: Directories 26 Why Being “LDAP-Enabled” Isn’t Enough A high-performance Enterprise Directory, available 7x24 via the LDAP protocol, is not enough. “LDAP-enabled” applications may not be compatible with your Enterprise LDAP Directory. Therefore, a multi-directory architecture will be required.

Base CAMP - February 5-7, 2003Middleware: Directories 27 Multiple Consumers Application specific or “embedded” directories will be needed for several reasons: - Performance needs, particularly for updates - Application-specific data - Special access - Security requirements - Because vendors seem to want it that way

Base CAMP - February 5-7, 2003Middleware: Directories 28 Integrating Multiple Directories Methods: - LDIF - ETL / Metadirectory products - EAI messaging tools - Log processing Unavoidable, so Plan For It

Base CAMP - February 5-7, 2003Middleware: Directories 29 Resource Provisioning Automated handling of the tasks associated with the establishment, modification, and deletion of resources and entitlements provided to people as they join or leave an organization or undergo changes in affiliation or status. Wouldn’t it be nice!

Base CAMP - February 5-7, 2003Middleware: Directories 30 Resource Provisioning What to do? -Identify existing automated processes -Identify existing manual processes -Directory-enable processes where possible How are people doing this today? - Perl - ETL / Metadirectory products

Base CAMP - February 5-7, 2003Middleware: Directories 31 Why Are There More Questions Than Answers? -Confusion over terminology, created in part by metadirectory vendors -Merging of directory and metadirectory vendors (where have all the vendors gone?) -Tools and standards are still maturing -Getting early success is fairly easy, going beyond white pages can prove difficult – for institutions that are riddled with exceptions centralized authorization and provisioning can be very complex

Base CAMP - February 5-7, 2003Middleware: Directories 32 The Education Enterprise Infrastructure Equation © EEIEq: H(C + D + R) = (F + R)  0 High Complexity + High Demand + High Return = Minimal Applied Funding + Minimal Applied Resources The EEIEq Axiom: Successful implementation despite EEIEq results in increases in components on the left side of the EEIEq equation, with no noticeable effect on components on the right side of the equation.

Base CAMP - February 5-7, 2003Middleware: Directories 33 Links Internet2 - MACE-Dir Metadirectories page Internet2 Metadirectories Practices document Author: