1. Middleware: Plumbing at the Next Layer
23 November 2018

2. Topics Interactive Tutorial Discussion Acks and axioms
About the policy aspect
Directories
Security
Shibboleth
Federations and InCommon
Authorization and diagnostics
Grids
Discussion
Implications to applications and network engineers
Using middleware for network engineering
11/23/2018

3. MACE (Middleware Architecture Committee for Education)
Purpose - to provide advice, create experiments, foster standards, etc. on key technical issues for core middleware within higher education
Membership - Bob Morgan (UW) Chair, Tom Barton (Chicago), Scott Cantor (Ohio State), Steven Carmody (Brown), Michael Gettes (Duke), Keith Hazelton (Wisconsin), Paul Hill (MIT), Jim Jokl (Virginia), Mark Poepping (CMU), Bruce Vincent (Stanford), David Wasley (California), Von Welch (Grid)
European members - Brian Gilmore (Edinburgh), Ton Verschuren (Netherlands), Diego Lopez (Spain)
Creates working groups in major areas, including directories, interrealm access control, PKI, video, P2P, etc.
Works via conference calls, s, occasional serendipitous in-person meetings...
I suspect that some audiences aill need varying amounts of mitoivaition with respect to the “why should I care about middleware” question.
- why should I care about middleware?
- why does it need a HE/I2 initiative?
- Related initiatives… globus/grid, …
- Relation to the NMI...
11/23/2018

4. The National Science Foundation Middleware Initiative (NMI)
NSF Program to develop and deploy common middleware infrastructures
Two major themes
Scientific computing and data environments (ala Grids)
Common campus and inter-institutional middleware infrastructure (ala Internet2/EDUCAUSE/SURA work)
“Random acts of middleware” projects to complement major systems integration work
Issues periodic NMI releases of software, services, architectures, objectclasses and best practices – R4 most current release
11/23/2018

5. Middleware Axioms Work the core areas
Focus on support for collaboration
Use federated administration as the lever; have the enterprise broker most services (authentication, authorization, resource discovery, etc.) in inter-realm interactions
Develop a consistent directory infrastructure within R&E
Provide security while not degrading privacy.
Foster interrealm trust fabrics: federations and virtual organizations
Leverage campus expertise and build rough consensus
Support for heterogeneity and open standards
Influence the marketplace; develop where necessary
11/23/2018

6. Interrealm and Federation
Given the strong collaborations within the academic community, there is an urgent need to create inter-realm tools, so
Build consistent campus middleware infrastructure deployments, with outward facing objectclasses, service points, etc. and then
Federate those enterprise deployments, using the outward facing campus infrastructure, with interrealm attribute transports, trust services, etc. and then
Leverage that federation to enable a variety of applications from network authentication to instant messaging, from video to web services, and then, going forward
Create tools and templates that support the management and collaboration of virtual organizations by building on the federated campus infrastructures.
11/23/2018

7. Core Middleware Scope
Identity and Identifiers – namespaces, identifier crosswalks, real world levels of assurance, etc.
Authentication – campus technologies and policies, interrealm interoperability via PKI, Kerberos, etc.
Directories – enterprise directory services architectures and tools, standard objectclasses, interrealm and registry services
Authorization – permissions and access controls, delegation, privacy management, etc.
Integration Activities – open management tools, use of virtual, federated and hierarchical organizations, enabling common applications with core middleware
11/23/2018

8. A Map of Campus Middleware Land
11/23/2018

9. Federated administrationVO VO O T
A CM O T
CM A
Campus 1
Campus 2 T T T
Federation
11/23/2018

10. Campus Core Middleware Architecture: (Origin perspective)
11/23/2018

11. R&E Sector Successes Building campus middleware infrastructures
Recipes, Roadmaps, Business Plans, Training
Developing effective consensus processes in directories
eduPerson and eduOrg
Creating key inter-institutional tools
KX.509, OpenSAML and Shibboleth
Leveraging infrastructure for institutional value
Cost savings and efficiencies
Impact and Outreach
11/23/2018

12. About the Policy Aspects
Middleware is a land where technology mixes a lot with policy:
Who owns my data?
Who can see my data?
Controlled vocabularies consensus
Naming consensus
Privacy
Why should I trust your authentication processes?
As geeks in paradise, you will be largely spared these issues…
11/23/2018

13. Campus tech/policy issues
Who operates the enterprise directory services?
Who within central IT operates the central component
Getting legacy apps owners to provide source data
Who can see/update what data fields in the directory
Complex political and legal issues
What data is included in the directory
To what degree can/must departmental applications use central directory and authentication services
Who does and pays for integration?
11/23/2018

14. Intercampus tech/policy
Is my assignment of values to attributes consistent with yours?
What data can be exchanged between what applications and users? (attribute release policies)
How do your identity enrollment and authentication processes compare to mine?
What do you do with attributes that I provide to you?
11/23/2018

15. The core grist
Directories
Security
Shibboleth
11/23/2018

16. Shibboleth AA Process Users Home Org Resource Owner 4
OK, I redirect your
request now to
the Handle Service
of your home org. 3 2
Please tell me
where are you from? 1
SHIRE
I don’t know you.
Not even which home
org you are from.
I redirect your request
to the WAYF
WAYF HS 5 6
I don’t know you.
Please authenticate
Using WEBLOGIN
Users Home Org
Resource Owner 7
User DB
Credentials
OK, I know you now.
I redirect your request
to the target, together
with a handle
Attributes 10
Manager
Resource
OK, based on the
attributes, I grant
access to the
resource
SHAR
Handle 8
I don’t know the
attributes of this user.
Let’s ask the Attribute
Authority
Handle 9 AA
Let’s pass over the
attributes the user
has allowed me to
release
Resource
11/23/2018

17. Shibboleth Architecture
11/23/2018

18. Shibboleth Architecture -- Managing Trust
Federation
Shib
engine
Attribute
Server
Target Web Server
Browser
Origin Site
Target Site
11/23/2018

19. Milestones Project formation - Feb 2000 Stone Soup
Process - began late summer 2000 with bi-weekly calls to develop scenario, requirements and architecture.
Linkages to SAML established Dec 2000
Architecture and protocol completion - Aug 2001
Design - Oct 2001
Coding began - Nov 2001
Alpha-1 release – April 24, 2002
OpenSAML release – July 15, 2002
v1.0 April 2003
v1.1 July 2003
(V2.0 early 2004)
11/23/2018

20. Shibboleth Status
Relatively straightforward to install, provided there is good web services understanding and middleware infrastructure (authentication, directories, webISO, etc.).
Target - works with Apache(1.3 and 2.0) and IIS targets; Java origins for a variety of Unix platforms.
Two active development mechanisms
Extensions to the core - V2.0 likely to include portal support, identity linking, etc. Original Shib crew
Tool work - Work underway on intuitive graphical interfaces for attribute release policies, resource protection, etc. – Going international
11/23/2018

21. GUI’s to manage Shibboleth
11/23/2018

22. Sysadmin ARP GUI
A tool to help administrators (librarians, central IT sysadmins, etc) set attribute release policies enterprise-wide
For access to licensed content
For linking to outsourced service providers
Has implications for end-user attribute release manager (Autograph)
GUI design now actively underway, lead by Stanford
Plumbing to follow shortly
11/23/2018

23. End-user attribute release manager (Autograph)
Intended to allow end-users to manage release policies themselves and, perhaps, understand the consequences of their decisions
Needs to be designed for everyone even though only 3% will use it beyond the defaults.
To scale, must ultimately include extrapolation on settings, exportable formats, etc.
11/23/2018

24. Privacy Management Systems
11/23/2018

25. Personal Resource Manager
11/23/2018

26. Federated administrationVO VO O T
Apps CM O T
CM Apps
Other
feds
Campus 1
Campus 2 T T T
Federation
11/23/2018

27. Unified field theory of Trust
Bridged, global hierarchies of identification-oriented, often government based trust – laws, identity tokens, etc.
Passports, drivers licenses
Future is typically PKI oriented
Federated enterprise-based; leverages one’s security domain; often role-based
Enterprise does authentication and attributes
Federations of enterprises exchange assertions (identity and attributes
Peer to peer trust; ad hoc, small locus personal trust
A large part of our non-networked lives
New technology approaches to bring this into the electronic world.
Virtual organizations could leverage any of these fabrics
11/23/2018

28. Requirements for federations
Federation operations
Includes limited rules of engagement
Federating software
Exchange assertions
Link and unlink identities
Federation data schema
Federation privacy and security requirements
11/23/2018

29. The Point of Privacy Kudos for Shibboleth!
Liberty Alliance Has Missed the Point eWeek November 24, By  Jim Rapoza 
What I'd like to see the group do is add more mechanisms to make it easy for third-party developers to create tools that give users total control over how their data is shared. A good model for this is the Internet2 group's Shibboleth ID management specification, which was designed mainly for academic institutions. In Shibboleth, users have built-in controls that give them final say over how their data is controlled.
11/23/2018

30. Federating Software Comparison
Liberty Alliance
V 1.1 of their functional specs released; 2.0 under discussion
Federation itself is out of scope
Open source implementations not emphasized
Current work is linked identities
Shibboleth
V1.1 released; 2.0 under development
Most standards-based; pure open source in widening use
Current work is attribute release focused; linking identities in 2.0.
Can Shibboleth and Liberty converge?
WS-*
Complex framework, consisting of 9 areas, which can form a whole cloth solution to the problem space, but which need to closely interact with each other to do so.
Standards process and IPR issues uncertain
Will need considerable convention and detail to resolve into a working instantiation
Can Shibboleth/InCommon be a working instantiation within WS-*? Under active discussion with Microsoft
11/23/2018

31. Shibboleth-based federations
InQueue
InCommon
Club Shib
SWITCH
NSDL
State networks
Medical networks
Financial aid networks
Life-long learning communities
11/23/2018

32. The Research and Education Federation Space
REF
Cluster
InQueue
(a starting point)
InCommon
SWITCH
The Shib
Research Club
Other national nets
Other clusters
Other
potential US
R+E feds
State of Penn
Fin Aid Assoc
NSDL
Indiana
Slippery slope
- Med Centers, etc
11/23/2018

33. InCommon federation Federation operations – Internet2 VPT
Federating software – Shibboleth 1.0 and above
Federation data schema - eduPerson or later and eduOrg or later
Federation privacy and security requirements – in discussion, could be
Privacy requirements:
Initially, destroy received attributes immediatley upon use
Security requirements:
Initially, enterprises post local I/A and basic business rules for assignment of eduPersonAffiliation values
Likely to progress towards standardized levels of authn
11/23/2018

34. InCommon Management
Designed by ad hoc national group of CIO’s and content providers
Resulting executive committee now being convened
Executive committee members include Jerry Campbell, (USC), Lev Gonick (CWRU), Clair Goldsmith (Texas System), Mark Luker (EDUCAUSE),Tracy Mitrano (Cornell), Susan Perry (Mellon), Carrie Regenstein (Wisconsin), Mike Teetz, (OCLC), David Yakimischak (JSTOR).
Prospectus at
11/23/2018

35. Early management issues
Opening date and configuration (pricing, packaging, etc.)
Rules for membership
Federation operational rules
Federation security and privacy rules (local I/A, attribute protection, etc)
Procedures for conflict resolution among members
Accommodating new business drivers – non-web services
11/23/2018

36. InQueue The “holding pond”
Is a persistent federation with “passing-through” membership…
Operational today. Can apply for membership via InQueue Federation guidelines
Requires eduPerson attributes
Operated by Internet2; open to almost anyone using Shibboleth in an R&E setting or not…
Fees and service profile to be established shortly: cost-recovery basis
11/23/2018

37. InQueue Origins 11.25.03 Rutgers University
University of Wisconsin
New York University
Georgia State University
University of Washington
University of California Shibboleth Pilot
University at Buffalo
Dartmouth College
Michigan State University
Shibboleth Development Origin
The Ohio State University
UCLA
Internet2
Carnegie Mellon University
National Research Council of Canada
Columbia University
University of Virginia
University of California, San Diego
Brown University
Penn State University
Cal Poly Pomona
London School of Economics
University of North Carolina at Chapel Hill
CU-Boulder
UT Arlington
UTHSC-Houston
University of Michigan
11/23/2018

38. Major targets
Campuses that are also origins, wanting to share campus-based content
Content providers – EBSCO, OCLC, JSTOR, Elsevier, Napster, etc
Learning Management Systems – WebCT, Blackboard, OKI, etc
Outsourced Service Providers – purchasing systems, dormitory management companies, locksmiths, etc.
11/23/2018

39. Trust pivot points in federations
In response to real business drivers and feasible technologies
increase the strengths of
Campus/enterprise identification, authentication practices
Federation operations, auditing thereof
Campus middleware infrastructure in support of Shib (including directories, attribute authorities and other Shib components) and auditing thereof
Relying party middleware infrastructure in support of Shib
Moving in general from self-certification to external certification
11/23/2018

40. Federated Applications
Personal Privacy and Resource Managers
Digital rights management
Role-based access controls
Desktop videoconferencing
Interrealm calendaring
Authenticated instant messaging
P2P
Shibbed *
11/23/2018

41. What’s next Authority systems Middleware diagnostics
Virtual organization support
11/23/2018

42. Stanford Authz Model
11/23/2018

43. Stanford AuthR Goals
Simplification of authority policy, management and interpretation. We should be able to summarize the full rights and privileges of an individual "at a glance" or let departmental administrators view and manage together all authority in their department or division.
Consistent application of authority rules via infrastructure services and synchronization of administrative authority data across systems.
Integration of authority data with enterprise reference data to provide extended services such as delegation and automatic revocation of authority based on status and affiliation changes.
Role-based authority, that is, management of privileges based on job function and assignments rather than attached to individuals.
11/23/2018

44. Authr Deliverables The deliverables consist of
A recipe, with accompanying case studies, of how to take a role-based organization and develop apprpriate groups, policies, attributes etc to operate an authority service
Templates and tools for registries and group management
a Web interface and program APIs to provide distributed management (to the departments, to external programs) of access rights and privileges, and
delivery of authority information through the infrastructure as directory data and authority events.
11/23/2018

45. Middleware Diagnostics Problem Statement
The number and complexity of distributed application initiatives and products has exploded within the last 5 years
Each must create its own framework for providing diagnostic tools and performance metrics
Distributed applications have become increasingly dependent not only on the system and network infrastructure that they are built upon, but also each other
11/23/2018

46. Goals
Create an event collection and dissemination infrastructure that uses existing system, network and application data (Unix/WIN logs, SNMP, Netflow©, etc.)
Establish a standardized event record that normalizes all system, network and application events into a common data format
Build a rich tool platform to collect, distribute, access, filter, aggregate, tag, trace, probe, anonymize, query, archive, report, notify, perform forensic and performance analysis
11/23/2018

47. Event Record Standard
Normalization of each diagnostic data feed type (SHIB, HTTP, Syslog, RMON, etc.) into a common event record
The tagging of specific events to help downstream correlation processes
GRID Application Log
Normalization
And Event
Tagging
Variable Star Catalog DB
Application
HTTP Access log
GRIDAPP:TIME:HOST:UID:…
SHIB log
SHIB:TIME:HOST:UID…
DB Access Log
DB:TIME:HOST:REQ:ASTRON
RMON Events
RMON:HOST:TIME:DSTPORT..
NETFLOW:TIME:SRC:DST:…
Cisco NetFlow Events
11/23/2018

48. Topological Architecture
Alerting apps, filtering
data to federation
and API to NMS
Federation specific
reporting, performance
and Forensic apps
Massive collection,
normalization,
filtering or tagging
Archive, querying
Reporting,
Performance, and
Forensic apps
Enterprise
Federation
Application Host
Diagnostic Host
11/23/2018

49. Data to diagnostic host
Lightweight module that exists on application host or diagnostic host
Four data operators, each optional for a given stream
Normalization operator is specific to each input stream
Operators
Filter
Anonymizer
Aggregation
Normalization
Input Stream Examples
App (Shibboleth) Log
Data to diagnostic host
TLS
Unix/WIN Log
Network (SNMP,NetFlow)
Http Log
Instructions from
Diagnostic host
Collection
Management
11/23/2018

50. Internal Housekeeping
Diagnostic Module
Dedicated to data collection, management, manipulation and installed on diagnostic host
Hosts can be pipelined to disseminate processing load and enforce privacy policies
Same data operators as Application Module, plus query and archive
Diagnostic host
communication
Event collection
from application
hosts
TLS
TLS
Data Management and
Internal Housekeeping
SHIB
Shell
Diagnostic Tools
and NMS
Archive
Query
Control and
Diagnostic
Application
Platform
HTTPS DB
11/23/2018

51. Diagnostic Applications
Rich tool platform and API that enables rapid development of diagnostic applications
Diagnostic Host
Remote or Local Applications
Control and
Diagnostic
Application
Platform
SHIB
Shell
Forensic
Tracing
Reporting
HTTPS
Historical Analysis
Alerting and Notification
Security Risk Analysis
NMS API .
11/23/2018

52. Involved Groups And Organizations
Internet2
End-to-End Performance Initiative
Specific Middleware and GRIDS working groups
SHIB, P2P, I2IM, etc.
IETF working groups
SYSLOG
RMOMMIB
Efforts in academic and research
11/23/2018

53. Issues and Vision Privacy
Rich and diverse sets of event data can be used to infer user behavior
Limiting access to this data and using data anonymization methods is essential to insure the privacy of the end user, or an enterprise as a whole
Intenet2 Diagnostic Infrastructure Initiative
Create a secure highly distributed set of services and tools to aid in end-to-end problem, security, and performance analysis
Establish an standard event record to be used as the basis for describing all network, system and application level anomalies and behavior
11/23/2018

54. Virtual Organizations
Geographically distributed, enterprise distributed community that shares real resources as an organization.
Examples include team science (NEESGrid, HEP, BIRN, NEON), digital content managers (library cataloguers, curators, etc), life-long learning consortia, etc.
On a continuum from interrealm groups (no real resource management, few defined roles) to real organizations (primary identity/authentication providers)
A required cross-stitch on the enterprise trust fabric
11/23/2018

55. Leveraging V.O.s Today VO User Federation Enterprise Target Resource
11/23/2018

56. Leveraging V.O.s Today VO User Federation Enterprise Target Resource
Authority
System
Federation
Enterprise
Target Resource
11/23/2018

57. Integration
Users want integration among their growing number of applications
Fewer logins, transparency in access
Similar user interfaces
Similar security and privacy management
Institutions want integration
Leverage common infrastructure
Similar admin interfaces
Coordinated security and privacy management
Similar access control mechanisms
A common inter-institutional trust service and attribute exchange
11/23/2018

58. Integration Technical Policy What needs to be integrated
Globally, maybe not a lot – schema, exchange protocols, trust
Locally, a lot – db, os, web server, authn/z
Who defines integration
What is a well-plumbed environment on campus and between campuses
Who does the integration
The plumbed service?
The new applications?
Policy
Who pays for integration
How consistent does open-source license approach need to be
11/23/2018

59. Intermission
11/23/2018

60. Discussion Topics
Middleware and its impact on applications and that impact on networks
Using middleware to help manage and secure networks
New security services
New security-aware services
11/23/2018

61. Middleware and Apps Killer desktop video
Killer integrated communication and presence
11/23/2018

62. Middleware and Network/Host Opportunities
Authentication to keep the network open…
User/device/state
After authentication, capabilities
Personal firewall settings
Enterprise exceptions
Leverage federations
At the network layer
At the host security layer
Preserving leading edge needs with trailing edge security
Videoconferencing and NATs
Grids and firewalls
P2P and perimeters
High performance DDOS
11/23/2018

63. Federated Security Services
Federated networks
Share a common network substrate
Share a common trust fabric
Together they could permit…
Collaborative incident analysis and response
Security-aware capabilities
Moving it into the broader Internet
11/23/2018

64. Collaborative Incident Analysis
Moving beyond the “border” to see network-wide views
I’m seeing activity X? Are others seeing it? What variants are they seeing? Real-time attack recognition
From the central observatory, let me see the full address of the attacking node at site Y in the federation
I’m seeing an attack ostensibly from source address z at enterprise Y. Let me look at logging within site Y to verify
Correlate signatures and traffic among sites A-Z to provide an early warning system of DDOS
Let external experts from site Z examine our forensic information to assist our diagnostics
Requires federated backbone (meters, log files, etc) and federated trust fabric (for scaling, role-based access control, contact info, etc.)
11/23/2018

65. Collaborative incident analysis
Scaling requires managing large data sets
Centralized – the Abilene Observatory, perhaps others
Distributed – on a per enterprise level
Which in turn requires a clear data model
Common event records, likely distilled and reformatted from native logs
Is enterprise-level security sufficient
And also pluggable modules for harvesting
And also a trust fabric that permits multiple levels of authentication and fine-grain authorization
11/23/2018

66. Federated Security-aware Capabilities
Federated user network authentication for on-the-road science
Control spam through federated verification of sending enterprises
Permit end-end videoconferencing through firewalls and NATs
Allow enterprise-specific patching paradigms to coexist
Create end-end transparency for use of Grids
Personal firewall configuration based on authorization
11/23/2018

67. Moving it into the broader Internet
Picking approaches that are deployable and build on embedded bases
Federated substrata among those on common backbones
Interfederation issues – how hard will they be
International discrepancies in privacy
International IdSP’s - legalisms
11/23/2018

68. Middleware and Personal Security
Using federations and identity service providers
Coupling privacy and security
Privacy management by the end-user working with their IdSP
Absolute privacy as the union and balance of persona
Accessing the audit logs of the IdSP
Getting the defaults right for the uninterested users
Making a marketplace
Liberty Alliance
WS-Stack
Open source Shib
11/23/2018