OAuth 2.0 Improvements Losing Half a Leg

Authentication in Ed-Fi ODS API 2.x OAuth 2.0 Authorization Code Grant Type (a.k.a. “3-Legged OAuth”) “Inspired” by OAuth 2.0 Authorization Code Grant Type Resource owner (user) attempts to access secured resource, receives redirect to log in Resource owner logs in and is then asked by authorization server to approve the client application’s (client_id) access, and receives an authorization code binding the approval to the client_id. Client application presents its client_id, secret, and the authorization code to get the access token to call the API on the resource owner’s behalf. Siriwardena P. Advanced API Security, Securing APIs with OAuth 2.0, OpenID Connect, JWS, and JWE. Apress; 2014: 135.

Authentication in Ed-Fi ODS API 2.x Authentication (as implemented) Authentication in Ed-Fi ODS API 2.x (a.k.a. “2.5-Legged OAuth”) “Inspired” by Authorization Code Grant, however, with no user involved, process was reduced to: Client presents key to server, receives authorization code (Steps 2 and 5). Client presents key, secret and authorization code to server, receives access token. (Steps 7 and 8) Issues: Authorization code effectively “binds” the client with itself, which is pointless. This is non-standard – It isn't 2-Legged OAuth or 3-Legged OAuth, but something in between.

Authentication in Ed-Fi ODS API v3 OAuth 2.0 Client Credentials Grant Type (a.k.a. “2-Legged” OAuth) Recommended OAuth flow when the client is the resource owner. Token carries a fixed expiration time (no longer has a rolling expiration) Benefits: A simpler process Implements a standard Works out-of-the-box with tooling (e.g. SwaggerUI and Postman) Siriwardena P. Advanced API Security, Securing APIs with OAuth 2.0, OpenID Connect, JWS, and JWE. Apress; 2014: 135.

Ed-Fi ODS API v3 Authentication - Postman On the “Authorization” tab… For Type, select “OAuth 2.0” Click “Get New Access Token” On “Get New Access Token” dialog… For “Grant Type” select “Client Credentials” 1 2 3

Ed-Fi ODS API v3 Authentication - Powershell Invoke-WebRequest -Uri http://localhost.fiddler:54746/oauth/token ` -Method Post ` -Headers @{ "Authorization"="Basic cG9wdWxhdGVkU2FuZGJveDpwb3B1bGF0ZWRTYW5kYm94U2VjcmV0"} ` -ContentType "application/x-www-form-urlencoded;charset=UTF-8" ` -Body "grant_type=client_credentials" StatusCode : 200 StatusDescription : OK Content : { "access_token": "c5e39f70fceb47f89b6b7837c855138d", "expires_in": 30, "token_type": "bearer" }

Ed-Fi ODS API v3 Authentication - Fiddler Request: POST http://localhost:54746/oauth/token HTTP/1.1 Authorization: Basic cG9wdWxhdGVkU2FuZGJveDpwb3B1bGF0ZWRTYW5kYm94U2VjcmV0 User-Agent: Mozilla/5.0 (Windows NT; Windows NT 10.0; en-US) WindowsPowerShell/5.1.14393.1532 Content-Type: application/x-www-form-urlencoded;charset=UTF-8 Host: localhost:54746 Content-Length: 29 Connection: Keep-Alive grant_type=client_credentials Response: { "access_token": "5d8222ab8c9f4781848cd5937d4c93cf", "expires_in": 30, "token_type": "bearer" }

Ed-Fi ODS API Change Events Moving Beyond Just Data Collection

A Little History The Ed-Fi ODS API was originally conceived by Ed Comer (the “Ed” in Ed-Fi) and Richard Charlesworth (CIO) for Tennessee Department of Education in 2013. Rather than continue the use of delayed batch data collection processes, the desire was to make it feasible for data to be transmitted to the SEA within 15 minutes. The Data Collection use case is now well-established, with API implementations now in 5 SEAs (Arizona, Nebraska, Tennessee, Wisconsin, Wyoming). (But what about Data Integration?)

Early Feedback Early Community Feedback (2015): The Ed-Fi 2.0 Standard (and thus the API surface area) is huge and difficult to approach for more specific use cases (such as rostering or assessments) 78 “Domain” API resources (with about 175 contained entities) 131 Types 50 Descriptors The API is “chatty” for data integration purposes, requiring multiple calls to gather the relevant data.

Improvements from Feedback API 2.x Improvements (2015/2016) API Profiles - Provide the ability to define a constrained surface area on the API, in terms of what resources are covered by the profile and what specifically is accessible on each of the resources. API Composites - Provides the ability for API hosts to define “composite” resources which combine data from multiple API resources server-side. Both were aimed at improving capabilities needed for Data Integration use cases.

The Need for Change While Composites provides much needed functionality, it alone is not enough. We talked with several potential Ed-Fi adopters interested in the Data Integration, and found that without a facility for tracking and reporting changes it isn’t viable with current API functionality.

Ed-Fi ODS API Change Events Overview of Planned Implementation Built on top of API Composites functionality Changes made through the Data Collection API impact Composite Resources. These impacted Composite Resources will be identified and logged as change events. Each change event will contain the change type, the Composite Resource’s full JSON body, and the associated EducationOrganizationId. Authorization will be performed using existing API authorization implementation, based on the caller’s association to the event’s EdOrgId. Utilizes SQL Server Change Tracking and batch processing to log the “events”. Will support both Ed-Fi ODS API v2.x and v3.

Ed-Fi ODS API Changes Events Envisioned Data Integration Approach for Change Events Consumers Perform Initial Synch Make a request to the new “availableChanges” resource to get the latest change event Id, saving it for the subsequent “Synch” process. {"firstChangeEventId": 1234, "lastChangeEventId": 2345} Perform a “full synch” of the needed Composite resource(s). Process Changes Make a request to the new “availableChanges” resource to get the latest change event Ids, saving it for the subsequent “Synch” process. Ensure that the ChangeEventId from the previous processing is >= “firstChangeEventId” Request the events since the last processing, as follows: /enrollment/events?q=changeEventId>2345

Caveats – Change Processing Change detection on Composites relies on the data stored in the Change Events log entries. Change events will only be retained for 30 days. After 30 days with no changes, a composite resource may be identified as potentially impacted, and will be logged and reported as a change (when in reality it has not been impacted). When processing the change events, be prepared to encounter events that don’t actually contain changes, whether due to activity in the Ed-Fi ODS API during processing or because of the retention policy.

Caveats - Privacy Change events are recorded with the full Composite Resource. Profiles (which can be used for implementing a Privacy Policy) are not applied to this version of the JSON. To address this concern, change event data can be constrained through normal API Profile facilities. Prevents the “Data” from being read directly from the event. Affected Consumers will need to request the current versions of the Composite Resources (which will be Profile-constrained). Support for a new “ids” parameter will be added to support this use case, allowing for specific resource instances to be retrieved.

Caveats - Performance Design goals is to minimize the impact to the operation of the Data Collection API. Batch processing of the SQL Server tracked changes into Ed-Fi Change Events could generate a heavy read load on the system, with great variability depending on the design of the Composite resources. Ed-Fi will be reviewing and updating the “Enrollment” and “Assessment” composite resources prior to release of the feature to try and minimize the amount of “spidering”. We may end up making recommendations for optimizations in certain deployments, utilizing SQL Server Availability Groups for read-replicas, or similar.