Automating Health Checks for Legacy NDS ® Carl Coleman Primary Support Engineer Novell, Inc. Martin Plouffe Primary Support Engineer Novell, Inc. David Stagg Primary Support Engineer Novell, Inc.

Vision…one Net A world where networks of all types—corporate and public, intranets, extranets, and the Internet—work together as one Net and securely connect employees, customers, suppliers, and partners across organizational boundaries Mission To solve complex business and technical challenges with Net business solutions that enable people, processes, and systems to work together and our customers to profit from the opportunities of a networked world

Our Goals Discuss and demonstrate existing tools for legacy Novell Directory Services ® (NDS) (pre-Novell eDirectory™) networks Explain the obituary process Provide you with options to effectively maintain healthy Directory Services Provide some disaster recovery tips Briefly introduce you to the ease of iMonitor

Presentation Agenda 1.Health Check concepts and processes Before eDirectory is installed in your tree 2.What is an Obituary? Explaining the obituary process 3.Scripting to automate the Health Check Before eDirectory is installed in your tree 4.The iMonitor Health Check intro After your first eDirectory server is installed 5.Q & A—Ask the Support Engineer

Your Benefits Tools and methods to check, maintain and prepare your existing systems for upgrade to eDirectory Comparison of existing legacy tools and methods with the greatly enhanced ease of use in NDS ® iMonitor Opportunity to discuss your environment and situations with Premium Support Engineers

Deployed Versions Novell eDirectory and NDS Product VersionBuild VersionPlatforms NetWare 5.1 SP4 (NDS 7)DS.nlm v7.57NetWare 5.1 NetWare 5.1 SP 4 (NDS 8)DS.nlm v8.79NetWare 5.1 eDirectory 8DS.nlm & DS.dlm v8.79NetWare 5.0,Win NT/2K eDirectory 8.5.xDS v85.23NetWare 5.x,Win,Solaris NetWare 6 (eDirectory 8.6)DS.nlm v NetWare 6 eDirectory 8.6.1DS v NW 5.1,NW 6,Win,Solaris,Linux NetWare 6 SP1 (eDirectory 8.6.2)DS.nlm v NetWare 6 eDirectory 8.6.2DS v103xx.xxNW 5.1,NW 6,Win,Solaris,Linux eDirectory 8.7DS v10410.xxNW 5.1,NW 6,Win,Solaris,Linux,AIX NDS eDirectory

Differences Between eDirectory and NDS NetWare 6 NetWare NDSeDirectory NOS directory focused on managing NetWare ® servers A cross-platform, scalable, standards-based directory used for managing identities that span all aspects of the network—eDirectory is the foundation for eBusiness NetWare 5

Part 1 The Health Check

Part 1—The Health Check What is a Health Check? Why do a Health Check? When to do a Health Check Health Check requirements How to do a Health Check For the purposes of this presentation the term “Health Check” applies to Directory Services

What Is a Health Check Proactive troubleshooting Preventative maintenance Monitoring processes within NDS

Why Do a Health Check  Stability  Efficiency  Reliability  Prevention  Diagnose problems  Identify potential problems

Health Check Decision Factors Health Check types  Basic  Complete Environment  Static  Dynamic

Health Check Types—Basic Basic Health Check  NDS version check  Time synchronization  Partition continuity

Health Check Types—Complete A complete Health Check is a basic Health Check, plus the checking of Background processes  External references  Obituaries  Remote server IDs  Unknown objects  Schema synchronization

Health Check—Static Tree Static tree  Minimal routine changes  Minimal adding or deleting of user objects  Create or change partitions only a couple times a year  Add or remove servers only a couple times a year

Health Check—Dynamic Tree Dynamic tree  Tree under constant change  Adding many users  Adding/upgrading servers  Adding OS versions (NT, NW, Linux, Solaris)  Company reorganization  Creating/deleting partitions  Moving objects to new contexts

When?—The Environment Rule WHEN ??? Large tree/dynamic environment  Basic check—daily  Complete check—weekly Large tree/static environment  Basic check—weekly  Complete check—monthly

When?—The Change Rule WHEN ??? Do a complete Health Check before and after ANY major change  Adding or removing a server  Upgrading a server to a newer OS  Upgrading eDirectory to a newer version on a server  Adding, removing or moving partition replicas

Health Check Requirements Assumptions (dangerous to assume but…)  It is assumed that you have knowledge of your tree structure, network resources, WAN topology, network locations and the physical layout and server placement  It is also assumed that you have supervisor access to the [Root] object  If you do not have the above information, it may be time for some additional research

Health Check Tools Legacy NDS tools NNDS Manager DDSREPAIR DDSTrace SET commands DDSTRACE.NLM DDSDIAG DDSBROWSE eDirectory tools NNDS iMonitor (shown later)

Basic Health Check Steps Basic health check  DS versions DSREPAIR.NLM NDS Manager DSDIAG.NLM  Time synchronization DSREPAIR.NLM DSDIAG.NLM  Partition continuity NDS Manager DSTRACE.NLM SET DSTRACE DSDIAG.NLM

DS Versions—DSREPAIR.NLM

DS Versions—NDS Manager

View Tree|Object|Preferences

Time Synchronization—DSREPAIR.NLM

Replica depth -1=Holds no replica 0=Holds [Root]

Partition Continuity—NDS Manager

List of all servers holding a replica

Partition Continuity—DSTRACE.NLM

Partition namePartition continuity successful

Partition Continuity—SET DSTRACE SET DSTRACE=+S SET DSTRACE=*H Partition name Partition continuity successful

Complete Health Check Steps Includes a Basic Health Check and  Checking background processes External references Obituaries Remote server IDs Unknown objects Schema synchronization Limber and other background processes

External References or XRefs Pointer to an object that doesn’t exist in a replica on this server but does exist in a replica on another server  Static XREF Group membership - Trustee assignment  Dynamic XREF Login at a server –XREF to another server Automatically deleted after a 192 hrs (settable)

External References LOAD DSRepair | Advanced Options | Check External References

Obituaries Load DSREPAIR –A | Advanced Options | Check External References

Remote Server IDs Load DSREPAIR | Advanced … | Servers … | <a server> | Repair All...

Unknown Objects Yellow circle in NDS or eDirectory IIndicates an object is missing a mandatory attribute Can appear after an upgrade If you delete a mandatory attribute If NDS or eDirectory has become corrupted OOften resolve themselves—but the number of objects often indicate the overall health of the tree Unknown Objects should RARELY be DELETED Identify the true problem first

A lack of “?” objects is not a definite indication of NO unknown objects— you are only checking the replica on one server If you have “?” objects and you delete the object and it reappears—Replica Ring Mismatch  Occurs when one replica contains an NDS object that another replica doesn’t have

Verify that all servers are running the same NDS schema Just because the replicas are in sync doesn’t mean that the schemas are in sync Schema Synchronization

Background Status Check To check current background status on a single server use the following DSTRACE setting  SET DSTRACE=*ST This parameter will report on the current status of external reference, obituary, limber and schema background processes You may wish to capture the output to a log file for review

Part 2 What is an Obituary?

The Obituary process Obituary states (or Flags) Obituary types Obituary classifications Tools Part 2—What Is an Obituary?

An attribute placed on an object to identify and control changes made to that object There are four Obituary states (Flags) There are 13 Obituary types An Obituary Is

Demo The Obituary Process

Issued—This is the initial state which is assigned to the obituary when it is issued—it is designated by Flag = 0000 Notified—This state is used on Secondary Obituaries to indicate that the NDS server specified within the Obituary has been notified of the Primary Obituary—it is designated by Flag = 0001 OK To Purge—This state is assigned to the Obituary to indicate that it has reached the second stage of processing—it is designated by Flag = 0002 Purgeable—This state is used on Secondary Obituaries to indicate that the NDS server specified within the Obituary has been notified that the Obituary is Purgeable—it is designated by Flag = 0004 Obituary States

Restored—Used to designate an object that has been restored from a backup system, this obituary attribute will be found on the newly restored object—it is designated by Type = 0000 Dead—Used to designate an object that is being deleted, this obituary is assigned to the object that is being deleted—it is designated with by Type = 0001 Moved—Used to designate an object that is being moved to another name context, this obituary is assigned to a NON PRESENT version of the object that exists in the original name context—it is designated by Type = 0002 Obituary Types

Inhibit Move—Used to designate an object that has been moved from another name context, this obituary is assigned to the relocated object in its new name context—this should be on a PRESENT object. It is designated by Type = 0003 Old RDN—Used to designate an object whose Relative Distinguished Name has been changed, this obituary is assigned to the renamed object that has the new RDN—it is designated by Type = 0004 New RDN—Used to designate an object whose Relative Distinguished Name has been changed, this obituary is assigned to the NON PRESENT object that has the original RDN—it is designated by Type = 0005

Obituary Types Backlink—Used to identify an NDS Server which holds either an external reference or a replica of the object, this obituary is assigned to the object that has been assigned the primary obituary— These should always have a primary obituary. It is designated by Type = 0006 Tree Old RDN—Used to designate a partition root object whose Relative Distinguished Name has been changed, this obituary is assigned to the renamed partition root object that has the new RDN— It is designated by Type = 0007 Tree New RDN—Used to designate a partition root object whose Relative Distinguished Name has been changed, this obituary is assigned to the NON PRESENT object that has the original RDN—It is designated by Type = 0008

Obituary Types Purge All—Used internally only—It is designated by Type = 0009 Move Sub-Tree—Used to designate a partition root object that is being moved to another name context, this obituary is assigned to the partition root object that is being moved—It is designated by Type = 000A Moved From—Used so that if an object has been moved from a different user-created partition, its previous location is known to the NDS (This was new to NetWare 5) It is designated by Type = 000B Used By—Used to notify a user-created partition that references an external reference that the external reference is going to be deleted, moved, or renamed—It is designated by Type = 000C

Obituary Classifications All Obituaries are processed according to the Obituary classification There are four classes of Obituaries Only certain types of Obituaries can fall under certain classes

Obituary Classifications Primary Obituaries  Restored  Dead  Moved  New RDN Secondary Obituaries  Used By  Backlink Informational Obituaries  Inhibit Move  Moved From  Old RDN  Tree Old RDN Special Case Obituaries  Tree New RDN  Purge All  Move Sub-Tree

Tools and Utilities The following are tools and utilities that will report or show objects with an Obituary attribute  DSREPAIR  DSBROWSE  DSDIAG  iMonitor

What’s Involved Background processes  NDS Replica Synchronization process  NDS Janitor Process  NDS Replica Purge Process Object attributes  Transitive vector\Synchronized Up To  Obituary Attribute  Replica Attribute  Backlink Attribute (NetWare 4.x or 5.x Trees)  Used By Attribute (NetWare 5.x Tree)  Purge Vector Note : In eDirectory 8.6.0, Obituary Vector replaces the Purge Vector

Background Processes All of the processing of an Obituary is handled by the NDS Replica Synchronization process and the Janitor Process At the point the Obituary is changed to a Purgeable state, the Janitor Process removes the Obituary attribute from the entry The next Purger Process cycle will remove the entry from the local database Obituaries will only be purged, if the Modification Time Stamp of the PURGABLE obituary is OLDER then the timestamp value recorded for the Purge Vector attribute, for each server replica number used in the Transitive Vector (TV) The purpose of this attribute is to prevent Obituaries being purged before all replicas have been notified about it

Part 3 Scripting the Health Check

Part 3—Scripting the Health Check Tools for Health Check Tools for server automation  Advantages and limitations of each Methods to automate the Health Check steps  Automation examples  Getting the reports Some disaster recovery tips and tricks

Tools for Health Check NDS Manager  No major updates since NetWare 4.x  Graphical interface on workstation  No automation or reporting ability DSTrace screen and log file  Can be difficult to understand, no built in help  Complex settings and information displayed

Tools for Health Check DSRepair.nlm  Server-centric  Requires keyboard entry DSDiag.nlm (Last update 6 Jan 1999)  Limited support (legacy tool)  Requires keyboard entry

Tools for Server Automation NCF  Command scripts CRON.NLM  Server scheduling STUFKEY.NLM  Keyboard entry TOOLBOX.NLM  Copy/move commands HRZIP.NLM  Server ZIP and unZIP tool

Tools Available—Links CRON.NLM h TOOLBOX.NLM h STUFKEY.NLM h h HRZIP/HRUNZIP.NLM IIn the extra files—TUT333.ZIP

Other Tools ??? On-site Admin Pro  Not a supported tool  Limited availability Batch files on workstation  Requires workstation Third-party software  Extra costs NDS iMonitor with eDirectory

Methods of Automation 1. Identify steps to complete tasks 2. Create STUFFKEY scripts and test 3. Deploy scripts to servers 4. Schedule processes in CRONTAB 5. Create TOOLBOX NCF scripts  Rename report files  Copy files to central server 6. Later you can review reports  Identify and resolve any errors

Disaster Recovery Tips and Tricks While you are automating, add some disaster recovery and routine maintenance steps Schedule daily, weekly or monthly  DSREPAIR –RC—Grab a DIB (before backups?)  TRUSTBAR—Each volume to backup trustees  TOOLBOX PURGE script to remove deleted files  CONFIG /DS—Create server configuration file  Copy Server configuration files to a central storage server  Copy server logs to a central log server

Demo Sample Automation Processes

Part 4 The NDS iMonitor Way

This section is a brief look at NDS iMonitor These slides are courtesy of the eDirectory Core Team Check out session IO 216—Introduction to NDS ® iMonitor for full details NDS iMonitor

iMonitor provides efficient, anytime, anyplace monitoring and diagnostic capability to all servers in your eDirectory tree NDS iMonitor

NDS iMonitor 1 Feature Overview Agent health summary Synchronization information Known servers Agent process status Hyperlinked NDS trace Agent configuration Agent triggers Agent synchronization Database cache Partition list Object/schema browser Agent information NDS repair Agent activity and verb statistics Error information eDirectory 8.5 (build version 85.xx or greater)

Enhanced usability\human factors Agent health checks DirXML™ monitor Search Reports Obituary Agent information Custom Enhanced NDS repair Agent configuration Replication filters Object browse Entry synchronization Schema synchronization list NDS iMonitor 1.5 Feature Overview eDirectory 8.5 (build version 85.xx or greater)

NDS iMonitor 1.5 Feature Overview eDirectory 8.6 (build version xx or greater) Agent configuration Schema synchronization Agent activity Synchronization activity Change cache browse Object browse Send entry to all replicas External references Agent information System level entries (schema root, pseudo server) Free Upgrade to iMonitor 1.5 for eDirectory 8.5 usershttp://download.novell.com “Novell eDirectory” Section

NDS iMonitor 2 Feature Overview Enhanced usability/human factors Shared HTTP stack Reports Object statistics Advertising Tree-wide health checks Connection monitor Inbound connections Outbound connections Identities Contexts Iterations Bad addresses eDirectory 8.7 (build version xx or greater)

NDS iMonitor 2 Feature Overview Event monitor Event statistics Event registration Event rights Event trace Expanded NDS trace Full system level entry browse (Pseudo server, schema root) eDirectory 8.7 (build version xx or greater)

Manage eDirectory Two iMonitor links registered under Manage eDirectory NetWare Remote Manager Integration

Anatomy of an NDS iMonitor Page Navigator frame Assistant frame Data frame Replica frame

The eDirectory Utility Knife Multiple tools in one You don’t leave the tool to go to another one You don’t have to access several different servers Agent Summary Agent Configuration NDS Trace NDS Repair DirXML ™ Tools Report Tool Search Tool Agent Health Object and Schema Browse Agent Synchronization Partition List Known Servers Agent Process Status Agent Activity, Verb and Event Statistics Plus: Inbound and Outbound Connection Monitor Error Information

Health Summary

Server Synchronization Replica filters Replica type

Replica Synchronization

Agent Health—Replica Check

Schema sync Obituaries External references Limber status Repair status Agent Process Status

DS Trace Configuration set dstrace=on set dstrace=+sync set dstrace=+misc set dstrace=+buffers set dstrace=+janit set dstrace=+init set dstrace=+obit set dstrace=+dsa set ttf=on set dstrace=*r

Links to NDS agent information

Agent Information Connection information eDirectory build number eDirectory ping Address reversal IP ping Referral information

Links to error information

Error Information Error descriptions Possible causes Troubleshooting actions Link to latest Novell documentation, TIDs, and white papers Index of errors

NDS Repair

NDS Repair as a Scheduled Event

Server versions Time synchronization Obituaries Service advertising Object statistics Agent health (agent and tree-wide) User-defined Scheduled reports More to come Reports

Server versions Time synchronization Obituaries Service advertising Object statistics Agent health (agent and tree-wide) User-defined Scheduled reports More to come Reports

By proxy NetWare 4.11 NDS 6.xx NetWare 5.1 eDirectory NT/Win2k eDirectory NetWare 5 SP4 eDirectory Solaris eDirectory NDS iMonitor NT Server eDirectory NetWare 5 NDS 7.xx Direct NDS iMonitor iMonitor Modes of Operation

Not every server in the tree must be running NDS iMonitor Only one server is required to be upgraded Single point of access for dial-in Accesses NDS iMonitor over a slower-speed link while NDS iMonitor accesses DS information over higher-speed links Makes previous NDS version information accessible Server-centric features only available where NDS iMonitor is installed NetWare 5.1 eDirectory NetWare 4.11 NDS 6.xx NetWare 5 eDirectory NetWare 5 NDS 7.xx NT/Win2k eDirectory Solaris eDirectory NDS iMonitor Proxy Mode

NetWare 4.11 NDS 6.xx NetWare 5.1 eDirectory NDS i Monitor NT/Win2k eDirectory NetWare 5 eDirectory NDS i Monitor Solaris eDirectory NDS i Monitor NT/Win2k eDirectory NetWare 5 NDS 7.xx Full server-centric feature set Reduced network bandwidth (faster access) Access by proxy still available for all versions of NDS Direct Mode

NDS iMonitor Requirements eDirectory 85.x or higher on at least one supported server in the Tree Browser support  HTML 3 browsers  Netscape 4.06 or higher  IE 4 or higher Monitorable NDS and eDirectory versions  NetWare 4.11 NDS or higher  eDirectory for NT/Win2K all versions  eDirectory for Solaris all versions  All other eDirectory platform releases

Other NDS iMonitor Sessions IO104—Using the Web-Based Management Utilities in NetWare 6 IO 216—Introduction to NDS iMonitor TUT229—Practical NDS iMonitor: Case Studies in eDirectory Diagnosis TUT331—eDirectory in Depth TUT200—Hands-On: Utilizing the New Features and Tools in NetWare 6 TUT231—Tips and Tricks for Using eDirectory Utilities

Part 5 Ask the Support Engineer