1. The Instant Replay MA for FIM
Bob Bradley, MVP, MCTS, FIM Team founding member

2. Background Bob Bradley, FIM MVP 2012, 2013
Work for UNIFY Solutions in Australia with Carol Wapshere and other colleagues in “The FIM Team”
Specialize in event-driven FIM solutions (you will see this in demo)
Began working with MIIS full time in 2004 (then ILM2007)
Have worked full time on FIM since mid 2009
Worked closely for 2 solid years with the MCS Identity Management Practice lead in Australia on the biggest FIM sites in Australia, neither of which were “OOTB”
In working with FIM I came up with a couple of unique ideas – including this one

3. Presentation Outline Inspiration and Concept Construction and Demo
Use Case Scenarios and Demo
Additional use case: Maintaining References
Advanced Implementation and Demo
Conclusion

4. Inspiration
The Replay MA was inspired by limitations encountered using the FIM MA …
The FIM MA is very different from any other type
Additional rules apply, e.g.
only one instance of the FIM MA allowed per sync service
only one FIM service connected to a single sync service
one-to-one “like with like” attribute mappings only
only direct flows configurable in the MA wizard only
no manual precedence allowed when FIM MA contributes an attribute value to the MV
Constraints such as the ones above can impose solution limitations … ones that we might find ourselves looking for ways around 
The FIM Portal and FIM Sync Engine are connected by a special management agent (the FIM MA). Significant restrictions are in place to ensure that the FIM MA is only used within certain (sometimes inhibiting) constraints, such as the following:
There can only ever be one instance of the FIM MA per sync server instance
There can only ever be one FIM Service connected to a single FIM Synchronization Service
Objects can only ever be mapped between the FIM MA CS and MV on a 1-1 basis
Attribute flows for the FIM MA between the MV and CS can only ever be direct (no rules extensions allowed), and must be configured in the MA wizard
Manual precedence rules cannot be defined to include attributes contributed to the Metaverse by the FIM MA
The above restrictions mean that there is effectively no real flexibility around how to design configurations to achieve common synchronization requirements. Specifically there are several emerging use cases (which I will come to shortly) which can NOT be achieved using the synchronization engine and can only be approximated using custom workflow activities.  As a result, there is no documented means of using the FIM Synchronization engine to achieve certain desirable outcomes.

5. Concept
The Replay MA is a very low-cost option (in terms of development as well as processing overhead) for providing the FIM Metaverse with an additional feed of the same objects already present in an existing MA (connected or not)
In the special case of the FIM MA, this provides added benefits, including restoring the advanced flow rule and manual precedence options otherwise denied for the FIM MA, seemingly leaving you no option but to implement “equal precedence”

6. Concept (continued)
This session will walk you through how to create a standard text file MA that works alongside the FIM MA, allowing you to overcome advanced flow and precedence restrictions.
You will also see how, with a more advanced configuration, you can also achieve that sought-after flexibility with reference attributes.

7. Construction Export the configuration of your target MA (FIM MA)
Run ReplayLDIF-GenerateSchema.ps1 to transform the DSML file to an LDIF file template for a new text MA
Create a new LDIF file MA from the template
Be selective in your attribute flows, flowing only those objects and properties that you want to
Configure enhanced precedence and advanced flow rules as necessary
The Instant Replay MA is a concept which leverages standard FIM Sync Engine features in a way not considered before to allow certain configuration options that are often not possible otherwise. Using what is essentially a read-only clone of any existing MA (including the FIM MA), any attribute can be contributed by the cloned MA in lieu of the original MA, thereby allowing for standard and extended options involving these attribute flows.
Here’s how …
I've created a generic mechanism to generate a DSML template file for a new DSML Text File FIM management agent from the audit drop file (also DSML but in an incompatible format). The components of the solution are as follows:
A powershell script to transform the audit drop file (DSML) into LDIF format
An XSLT stylesheet which performs the transformation
Operational software to coordinate the standard MA run profile with the replay MA run profile (Event Broker)

8. Basic Implementation
Configure audit drop files for your target MA, and use these as the source for your Replay MA
Configure the ReplayLDIF-GenerateData.ps1 script to transform the DSML drop file into LDIF format
Test and refine
Use automation to orchestrate run profile sequencing on the back of the source run profile sequence
The Instant Replay MA is a concept which leverages standard FIM Sync Engine features in a way not considered before to allow certain configuration options that are often not possible otherwise. Using what is essentially a read-only clone of any existing MA (including the FIM MA), any attribute can be contributed by the cloned MA in lieu of the original MA, thereby allowing for standard and extended options involving these attribute flows.
Here’s how …
I've created a generic mechanism to generate a DSML template file for a new DSML Text File FIM management agent from the audit drop file (also DSML but in an incompatible format). The components of the solution are as follows:
A powershell script to transform the audit drop file (DSML) into LDIF format
An XSLT stylesheet which performs the transformation
Operational software to coordinate the standard MA run profile with the replay MA run profile (Event Broker)

9. Demo
Construction

10. Use Case Scenarios Avoid using equal precedence
Derive multiple/alternative import mappings from the same FIM Portal property
Selectively import reference values
Import reference values as strings (not just FIM MA) using direct or advanced flow rules
Implement manual precedence for import flows involving the FIM Portal
The following are 3 specific scenarios where my colleague Carol has used the Replay MA idea for her clients …
Use cases for this MA include the following:
Eliminate the need to configure "equal precedence" for scenarios where there is no alternative when involving the FIM MA
There are several scenarios here (e.g. group membership for migrated groups should become authoritative in the portal post migration) which are presently not achievable without configuring equal precedence. This is always problematic and would be good to avoid by introducing a 3rd authoritative source for group membership which can trump the others.
Provide a means for FIM portal attributes to be used to derive additional columns (incl. in advanced attribute flows).
The FIM MA allows only direct 1-1 attribute flows between like object classes in the FIM Portal and the FIM Metaverse using fixed class schema. One scenario is where you wish to join on something other than the mv GUID – e.g. on the manager attribute so as to enable flow of the manager display name (redundantly) to the subordinate.
Provide a means for FIM portal attributes to be used to be treated as different attribute types (incl. in advanced attribute flows).
The FIM MA allows only direct 1-1 attribute flows between like object classes in the FIM Portal and the FIM Metaverse using fixed class schema. This prevents the use of advanced flow rules in such cases as only flowing reference attributes based on the value of another attribute of the same identity, or flowing reference types as strings to allow for advanced flow rules. * Note: there is a documented alternative (advanced) for this scenario when working with Portal sync rules.
Provide a means to define MANUAL precedence by enabling advanced attribute flows (rules extensions) from the FIM Portal
The FIM MA allows only direct 1-1 attribute flows, and as a result any attribute contributed by the FIM Portal cannot be included in a “manual precedence rule” when the FIM MA is the only means of sourcing this attribute from the FIM Portal
The following 3 slides are graphics of specific examples of the above use cases …

11. Use Case: FIM MA Not Precedent
End Date
HR Precedent for Staff
Portal Precedent for Contractors
Termination WF in Portal HR
TermDate AD
FIM MA
Skipped: Not Precedent
Metaverse
EmployeeEndDate
accountExpires
employeeEndDate
Direct Flows
HR Precedent over Replay MA
Replay MA
FIM MA precedent
Equal precedence
Manual precedence
EmployeeEndDate

12. Use Case: Advanced Import Flow
Unique ID
Generate Person ID
Must not change
Other uses
Application ID AD
FIM MA
PersonID
Metaverse
ObjectID
employeeID
personID
Replay MA
Only flow if
not present
New portal object changes personID
ObjectID
personID does not change

13. Use Case: Selective Reference Flow
Staged Group Migration
Some precedent in Portal
Some precedent in Notes
Must be identical in Notes and AD
Notes
Members AD
FIM MA
FIMAuthoratative
Metaverse
Member
member
member
Selective Join
Replay MA
Member
FIMAuthoratative
Equal precedence
FIM RTM – no scoped SRs
Replay MA precedent over Notes

14. Demo
Basic solution use case scenario

15. FIM Back-links Use Case #1 Leveraging relative-to-resource MPRs
Relative-to-resource idea saves on set/MPR proliferation, which is a known cause for FIM performance degradation
This style of MPR comes with the hidden cost of maintaining the references
Multi-value reference must be maintained in sync with each collection of administrators for a location
High processing overhead in maintaining this via workflow
Need Housekeeping to ensure integrity (topic for another time!)
Sync option is far more attractive … just need to support deltas!
Use case #1
A multi-value reference attribute locationAdministrators on PERSON must be maintained in sync with the PersonID assigned to each LOCATION object in the FIM Portal. In this scenario each user in any given location must be updated with the current set of Location Administrators when location changes. This is to allow a (delegation style) policy "Org admins can manage the users in their assigned Orgs" to be implemented.
This example involves the updating of all user objects in the same location as the person administering a location.
In my FIM Housekeeping talk I discussed the way workflows are often used to maintain such “back links”, and the inspiration for the Housekeeping idea was from just this kind of scenario. Often there is no alternative but to go down this route … but in many cases the FIM Replay MA offers a pleasing alternative …
There are several significant drawbacks to implementing intra-FIM sync style workflows, but they all amount to effectively the same thing - a lack of state management capability. This is where the Sync Engine excels. Data issues and environment failures can lead to workflow exceptions, and the only way to address this problem is to mitigate it with housekeeping type functionality (i.e. calculate what the backlinks should be and compare this to what they actually are ... and do this iterating over the entire user base on a regular maintenance cycle).
One of the biggest drains on FIM performance is the number of dynamic sets defined, and the number of policies (MPRs) which are defined on these sets. The reason for the above design ("relative to resource" style MPR) is that only one set, workflow and policy triple is required to implement such Location policy this way, as opposed to one MPR/Set/Workflow triple for each and every location in existence (and these invariably are added and removed over time, meaning that other policy would be required to generate these triples to achieve the equivalent zero maintenance of the "relative to resource" idea). However, the cost of this MPR style is entirely determined by the overhead of maintaining these multi-value reference attributes such as the ones on PERSON in the above example. So the workflow overhead is effectively the "lesser of two evils".
A synchronisation approach to the same problem has none of these problems, with ongoing integrity assessable at a glance. Although the workflow approach has no doubt been replicated many times throughout the FIM world by now, it is a (presently necessary) weakness in the design, and this will only happen more and more as new ideas emerge with how to use FIM. A sync approach is far more streamlined, simple and consistent (when references are synthesized by a tool such as Identity Broker and not hand-crafted XSLT to support deltas), and I expect that not only are the overheads much less, but the confidence in the integrity is priceless (especially when we're talking about access control here).

16. FIM Back-links Use Case #2
Set definitions on derived references to support MPRs
Maintain Person.memberOf multi-value property derived from group.member
ADUC console in AD shows a user’s group membership in the “Member Of” tab … however this is just a run-time inversion of the Group’s “Member” property, and cannot be synchronised
Could support set transition or request MPRs such as “All new users in the TEC2012 group are notified of their membership”, or simply “All users are notified of set membership changes”

17. Advanced Implementation
Configure audit drop files for your target MA
Use extended XSLT to transform the DSML file into an LDIF file
Configure additional derived “back link” MA properties
Be selective in your attribute flows, flowing only those objects and properties that you want to
Configure enhanced precedence and advanced flow rules as necessary, as well as derived “back-link” flows
Use an automation tool to orchestrate run profile sequencing
A more advanced variation on the FIM Replay MA involves the computing of “back-link” reference properties, by applying transformations on the data read from the DSML drop file.
Here we take the basic concept, but instead of loading directly into FIM via an LDIF MA, we extend the MA to incorporate additional properties. Why do I do this?
To demonstrate what can be done, I’ll apply an extended XSLT transformation (additional code) to the FULL IMPORT DSML FILE to derive additional reference properties in the generated LDIF file. We’ll then see what happens when a delta import DSML file is processed under this model, thereby demonstrating what is required to support delta imports from your Replay MA.

18. Maintaining References
What’s involved in enforcing referential integrity in FIM?
Think of all the possible use cases
Identify all the relevant sets
Construct action workflows
Construct set transition MPRs
Cross your fingers and hope nothing breaks  …
Here the FIM Replay MA can give you that peace of mind you need …
So what can you do to ensure FIM has the best chance possible of maintaining this referential integrity:
Consider all possible use cases which would result in a change to the target (reference) property (e.g. person.roles, or person.locationAdministrators)
Identify changes to sets which together cover all the above use cases, and from which workflows can be run to reinforce referential integrity
Create action workflows to reinforce referential integrity to cover each set change
Define set transition MPRs to fire workflows for each set
Apply MPRs retrospectively
... with the work required above it is easy to see how scenarios can be missed, and how you can be left with some doubt that at any given time you can be confident you have full referential integrity
=> here is where the HouseKeeping Fairy can save you …

19. Demo
Advanced back-link generation scenarios

20. Conclusion
The FIM Replay MA is a very simple, low cost option of providing the FIM Metaverse with an additional feed of the same data already present in an existing MA.
In the special case of the FIM MA, this provides added benefits, including avoiding having to go down the “equal precedence” route.

21. More Info My blog: bobbradley1967.wordpress.com
LinkedIn: au.linkedin.com/in/bradleybob
Twitter: twitter.com/unificator (#FIM2010)
The FIM Team: thefimteam.com
My Company:
FIM Forum: social.technet.microsoft.com/Forums/en-US/ilm2
Bob Bradley:

22. PostScript
On 09/18/12 9:52 PM, Jason Bell wrote (LinkedIn): I didn't realize you had it posted yet... It is funny, shortly after TEC - I was inspired by the Replay MA concept and developed a ECMA 2.0 that let you select the MA to replay from a dynamic drop down list. Does a full dynamic schema discovery. I got it all up and running for single value attributes and forgot about it... then the other day I needed it and got to thinking that I should share it with you when I get it fully functional. I have been working on various ECMA 2.0 Management Agents to easily perform tasks that have historically required out-of-band processes. The Replay MA idea fit into this category. So anyway, when I get it done - I would like to show it to you and make sure you get due credit. Hopefully we will see you again at TEC in Keep up the great Blogs!

23. Questions?