OPEN DATA PROTOCOL (ODATA) DEEP DIVE A common, open, RESTful Web protocol for querying and updating data that provides a uniform way to unlock data and free it from silos that exist in applications today.

ODATA = REST Resource-oriented o Entities modeled as URI-addressable Resources o Relationships modeled as links (URIs) o CRUD = POST, GET, PUT/PATCH, DELETE Hypermedia-driven o Navigate from Service Document to Sets to Members to related items to… o Links in Payload for editing, navigation, operations, etc. + Data Model URLs, operations, namespaces, derived from declarative model + Common Conventions Common query string options Representation within Formats + Common Formats ATOM, JSON

USE OF HTTP HTTP Methods GET, POST, PUT/PATCH, DELETE HTTP Headers Content Negotiation o Accept, Accept-Charset, Accept-Encoding, Accept-Language o Content-Type, Content-Length, Content-Encoding, Content-Language Concurrency o Etags, If-Match, If-None-Match Etc… HTTP Result Codes 2xx Success Codes 3xx Redirection 4xx Client Error Messages

ODATA FEATURE AREAS Versioning Service Metadata Data Requests Data Modification Relationships Content Types Extensibility

VERSIONING

PROTOCOL VERSIONING DataServiceVersion header Describes the protocol version according to which the request/response should be interpreted MaxDataServiceVersion Request Header The maximum protocol version that the client can accept MinDataServiceVersion Request Header The minimum protocol version the client can accept -Clients SHOULD specify a MaxDataService -Services return a response formatted according to the latest protocol version the service supports that is less than MaxDataServiceVersion but at least MinDataServiceVersion

SCHEMA VERSIONING When a data model or other breaking change is made to the service, the service URL is generally versioned. For example: odata.netflix.com/v1/Catalog odata.netflix.com/v2/Catalog

SERVICE METADATA

SERVICE DOCUMENT Root of service exposes a “Service Document” Describes collections of Entities GET /odata.netflix.com/v2/Catalog HTTP/1.1 <service xmlns=" xmlns:app=" xmlns:atom=" xml:base=" Default - Genres - Titles - TitleAudioFormats - TitleAwards - People - TitleScreenFormats - Languages <service xmlns=" xmlns:app=" xmlns:atom=" xml:base=" Default - Genres - Titles - TitleAudioFormats - TitleAwards - People - TitleScreenFormats - Languages

ENTITY DATA MODEL Exposed on /$metadata Describes Entity Model of Data Service Entity Types Complex Types Association Types EntityContainers GET /odata.netflix.com/v2/Catalog/$metadata HTTP/1.1 <edmx:DataServices m:DataServiceVersion="1.0" xmlns:m=" <Schema xmlns=" xmlns:d=" Namespace="Netflix.Catalog.v2"> + <edmx:DataServices m:DataServiceVersion="1.0" xmlns:m=" <Schema xmlns=" xmlns:d=" Namespace="Netflix.Catalog.v2"> +

ENTITY TYPE Named structures with keys Support Inheritance Properties String, Integer, Boolean, DateTime, Spatial datatypes Collections, Complex Types Relationship Properties May be marked as "Open" May return "dynamic" properties not described in $metadata <NavigationProperty Name="Titles" Relationship="Self.Genre_Titles" ToRole="Genre_Titles_Target" FromRole="Genre_Titles_Source"/> <NavigationProperty Name="Titles" Relationship="Self.Genre_Titles" ToRole="Genre_Titles_Target" FromRole="Genre_Titles_Source"/>

COMPLEX TYPE Named Structures w/o keys Support Inheritance Don't support relationships Because they don't have an independent ID

ASSOCIATIONS Define relationships (navigation paths) between Entities Bi-directional Specify Cardinality of each end Can also impose integrity constraints

ENTITY CONTAINER Contains EntitySets – collections of Entity Instances AssociationSets – relationships between entities FunctionImports – Actions/Functions exposed by service

DATA REQUESTS

QUERYING DATA Querying a Set GET /v2/Catalog/Genres HTTP/1.1 Requesting an Individual Entity by ID GET /v2/Catalog/Genres('Adventures') HTTP/1.1 Requesting an Individual Property GET /v2/Catalog/Genres('Adventures')/Name HTTP/1.1 Requesting an Individual Property Raw Value GET /v2/Catalog/Genres('Adventures')/Name/$value HTTP/1.1

$FILTER Basic predicates, built-in functions GET /v2/Catalog/Titles?$filter=Name eq 'The Sting' HTTP/1.1 Filter on Properties of a derived type Only returns entities of that type that match the predicate GET /v2/Catalog/Awards?$filter=Netflix.AcademyAward/Category HTTP/1.1 Query based on collection Membership using Any/All GET /v2/Catalog/Titles?$filter=Genres/Any(g:g/Name eq 'Adventure') HTTP/1.1 GET /v2/Catalog/Titles?$filter=Genres/Any(g:g/Name eq 'Adventure') HTTP/1.1

OPERATORS Logical Operators eq, ne, gt, ge, lt, le, and, or, not Mathematic Operators add, sub, mult, div, mod Grouping Operator () NULL literal null

BUILT-IN FUNCTIONS String Functions substringof, endswith, startswith, length, indexof, substring, tolower, toupper, trim, concat DateTime Functions year(), month(), day(), hour(), minute(), second() Math Functions round(), floor(), ceiling() Type Functions isof() Casting Insert type in path Geo Functions geo.length, geo.intersects, geo.distance

$ORDERBY Comma separated list of properties, expressions GET /v2/Catalog/Titles?$orderby=AverageRating,ReleaseYear HTTP/1.1 GET /v2/Catalog/Titles?$orderby=AverageRating,ReleaseYear HTTP/1.1 Can specify asc (default) or desc GET /v2/Catalog/Titles?$orderby=AverageRating desc HTTP/1.1

$TOP/$SKIP Enables Client-side paging through large result sets GET /v2/Catalog/Titles?$top=10&$orderby=AverageRating desc HTTP/1.1 GET /v2/Catalog/Titles?$top=10&$orderby=AverageRating desc HTTP/1.1 GET /v2/Catalog/Titles?$top=10&$skip=10 &$orderby=AverageRating desc HTTP/1.1 GET /v2/Catalog/Titles?$top=10&$skip=10 &$orderby=AverageRating desc HTTP/1.1 If no $orderby, server guarantees stable ordering

$EXPAND Specify comma separated list of navigation property paths to include in response GET /v2/Catalog/Titles?$expand=Cast,Awards HTTP/1.1 GET /v2/Catalog/Titles?$expand=Cast,Awards HTTP/1.1 Result are returned inline according to the appropriate format

$SELECT Narrow the set of fields returned GET /v2/Catalog/Titles?$select=Name,Synopsis,Rating HTTP/1.1 Related properties must be specified in $expand GET /v2/Catalog/Titles?$select=Name,Cast/Name&$expand=Cast HTTP/1.1 GET /v2/Catalog/Titles?$select=Name,Cast/Name&$expand=Cast HTTP/1.1 Can include all properties using * Does not include navigation properties GET /v2/Catalog/Titles?$select=* HTTP/1.1

$COUNT/$INLINECOUNT Get just the count Can include additional query operators GET /v2/Catalog/Titles/$count HTTP/1.1 Include the count with the results $inlinecount ignores $top/$skip but includes $filter GET /v2/Catalog/Titles?$inlinecount=allpages&$top=10 HTTP/1.1

$FORMAT Specify a particular format "atom" o application/atom+xml, o application/atomsvc+xml for service document "json" o application/json "xml" o application/xml Overrides content type specified in headers

SERVER DRIVEN PAGING Server controls the maximum number of records to return at a time Each "page" of results includes a "next link" for retrieving the next page of results Pages can vary in size; some may be blank i.e., return records as computed or across a federation

DATA MODIFICATION

INSERT POST entity to the EntitySet collection Returns inserted entity Use PREFER header to request no content returned Media Resources have special rules according to AtomPub: POST media resource to the collection Returns Media Link Entry Update the Media Link Entry

UPDATE PUT to the edit-link to replace Unspecified values are set to null or default PATCH to the edit-link to affect only specified values Unspecified values are left unchanged Use PREFER header to request content returned Use entity etag in if-match for concurrency control Obtained from header or payload

DELETE DELETE to the edit-link to delete Use entity etag in if-match for concurrency control Obtained from header or payload

BATCH REQUESTS "Batch" multiple statements in a single multi-part mime request Results for each statement returned in a multi-part mime response Multiple Data Modification statements may be grouped in a single "ChangeSet" ChangeSets are atomic Reference the results of other statements in the changeset o i.e., add a child to an added parent May have multiple ChangeSets per Batch

RELATIONSHIPS

REQUESTING RELATIONSHIP LINKS Use $links to request the relationship links for a particular navigation property Results are returned as an array of URIs o GET /v2/Catalog/Titles/Genres('Adventures')/$links/Titles HTTP/1.1 GET /v2/Catalog/Titles/Genres('Adventures')/$links/Titles HTTP/1.1

CREATING LINKS TO NEW ENTITIES Create a new Entity with a new related entity by POSTing the entity with the related entity as inline content ("Deep Inserts") Create a new entity related to an existing entity by POSTing to the relationship collection of the existing entity POST /v2/Catalog/Titles/Genres('Adventures')/$links/Titles HTTP/1.1 …

CREATING LINKS TO EXISTING ENTITIES Create a relationship between two existing entities by POSTing the URL of the one entity to the appropriate $links collection of the other entity o POST /v2/Catalog/Titles/Genres('Adventures')/$links/Titles HTTP/1.1

DELETING LINKS Remove a relationship by sending a DELETE request to the $links collection specifying the url to remove o DELETE /v2/Catalog/Titles/Genres('Adventures')/$links/Titles HTTP/1.1

CONTENT TYPES (FORMATS)

ATOM FORMAT Multiple entities represented as a Count optionally returned in element Next link returned as element Each entity represented as an is unique id specifies type Self, edit s element for Relationships o Inline content as child element containing feed or element element for editing media Properties nested in element Individual properties namespace qualified with data namespace For Media Resources, has a src property to the media stream o is a sibling of For non-Media Resources, is a child of Functions/Actions returned as and elements

<feed xml:base=" xmlns:d=" xmlns:m=" xmlns=" Genres T18:37:18Z Adventures T19:08:35Z <link rel=" type="application/atom+xml;type=feed" title="Titles" href="Genres('Adventures')/Titles" /> <category term="Netflix.Catalog.v2.Genre" scheme=" /> Adventures <feed xml:base=" xmlns:d=" xmlns:m=" xmlns=" Genres T18:37:18Z Adventures T19:08:35Z <link rel=" type="application/atom+xml;type=feed" title="Titles" href="Genres('Adventures')/Titles" /> <category term="Netflix.Catalog.v2.Genre" scheme=" /> Adventures ATOM PAYLOAD - EXAMPLE

JSON VERBOSE FORMAT Results wrapped in a "d" wrapper Multiple entities returned as JSON Array property named "Result" o Optionally has __Count property for number of elements in query result (across all pages) o Optionally has "__next" property for next link Each Entity has o A "__metadata" property containing: – URI – Type – Media Link, edit-media link, content type, etag for media resources o A "__deferred" property for each un-expanded relationship containing the URL for the related entity/collection o Properties a Name/Value pairs Each Complex Type has a "__metadata" property containing the Type

JSON PAYLOAD - EXAMPLE { "d" : { "results": [ { "__metadata": { "uri":" "type": "Netflix.Catalog.v2.Genre" }, "Name": "Adventures", "Titles": { "__deferred": { "uri":" } ], "__count": "326", "__next": " Genres/?$skiptoken='Adventures'" }

NEW JSON ODATA FORMAT Effort to simplify JSON More readable JSON format Reduce size Preserve OData-ness o Still Hypermedia-driven Uses Namespacing Mechanism Removes almost all metadata from payloads Calculates values from conventions or templates in metadata 80-90% more efficient than Atom Future contribution to TC

NEW JSON PAYLOAD - EXAMPLE { "odata.metadata":" "count":"830", "value": [ { "Name": "Adventures", } ], "nextLink":" " }

EXTENSIBILITY

CUSTOM ACTIONS Side-effecting operations that may or may not return a value May be exposed on instances (Hypermedia-driven) Use entity etag in if-match for concurrency control Obtained from header or payload

CUSTOM FUNCTIONS Extend query language May be exposed on instances (Hypermedia-driven) Use entity etag in if-match for concurrency control Obtained from header or payload

VOCABULARIES An OData Vocabulary defines a shareable set of Annotations that may be applied to metadata or instance data Extended Property Information o Units of measurement o Read-only, read-write Ontologies o This entity represents the common concept of a person Validation o This integer property is between 10 and 20 Display hints o Caption field, grouping, navigation Service Capabilities o Query limitations, etc. Extension o Analytics

ANNOTATIONS CONSIST OF: A Target: The construct against which the annotation is applied o May be any Model Construct (type, property, etc.) A Term: The global name of the annotation o For example, org.odata.display.Title A Value: The value for the annotation o May be a static value, path, or expression A Vocabulary is a set of terms in a common namespace

ANNOTATIONS MAY… Be defined in a common format Machine readable (i.e., for validation) Be embedded in a model To define extended metadata Be applied through an external annotation application file Externally annotate existing services Be applied to Instance Data Information that may vary per instance

TYPES OF ANNOTATIONS ValueTerm A single annotation term applied to a type, property, association, etc. Complex TypeTerm Generally used to describe an is-a relationship Relationships may be modeled as properties of individual or collections of other complex type terms Entity TypeTerm Most prescriptive; define keys, relationships

UNANNOTATED ENTITY TYPE <Schema xmlns=" Namespace="Sales"> <Schema xmlns=" Namespace="Sales">

VOCABULARY DEFINITIONS “Person” vocabulary TypeTerm: Display Vocabulary Value Terms <Schema xmlns=" Namespace="myorg.schemas.person"> <Schema xmlns=" Namespace="myorg.schemas.person"> <Schema xmlns=" Namespace="org.odata.display"> <Schema xmlns=" Namespace="org.odata.display">

EXTERNAL ANNOTATION FILE <Schema xmlns=" Namespace="Person"> <Schema xmlns=" Namespace="Person">

ANNOTATED METADATA <Schema xmlns=" Namespace="Sales"> <Schema xmlns=" Namespace="Sales">

INSTANCE ANNOTATIONS { "odata.json.metadata": " "results" : { “CustomerID" : 123, "FirstName" : "John", "LastName" : "Public", " " : : { odata.org.display.readonly : true } } { "odata.json.metadata": " "results" : { “CustomerID" : 123, "FirstName" : "John", "LastName" : "Public", " " : : { odata.org.display.readonly : true } }

ANNOTATION/METADATA REFERENCING AnnotationReference Specifies a separate document that annotates this model Constrain to individual namespaces, qualifiers through "Include" Reference Brings a separate model "into scope" for use within this model

SUMMARY OData = REST o Entities, Entity Sets as URI addressable Resources o Hypermedia Driven o CRUD = POST, GET, PUT/PATCH, DELETE + Data Model o Entity Relational Model + Common Conventions o Query Syntax, Server Driven Paging, … + Common Formats o Atom, JSON + Extensibility o Custom Functions, Actions, Shared Annotations

BACKUP

{ "d": { "results": [ { "__metadata": { "uri": " "type": "ODataDemo.Product" }, "ID": 0, "Name": "Bread", "Description": "Whole grain bread", "ReleaseDate": "\/Date( )\/", "DiscontinuedDate": null, "Rating": 4, "Price": "2.5", "Category": { "__deferred": { "uri": " } }, "Supplier": { "__deferred": { "uri": " } }, { "__metadata": { "uri": " "type": "ODataDemo.Product" }, "ID": 1, "Name": "Milk", "Description": "Low fat milk", "ReleaseDate": "\/Date( )\/", "DiscontinuedDate": null, "Rating": 3, "Price": "3.5", "Category": { "__deferred": { "uri": " } }, "Supplier": { "__deferred": { "uri": " } }], "__count": 9 } { "odata.json.metadata": " "results": [ { "ID": 0, "Name": "Bread", "Description": "Whole grain bread", "ReleaseDate": "\/Date( )\/", "DiscontinuedDate": null, "Rating": 4, "Price": "2.5" }, { "ID": 1, "Name": "Milk", "Description": "Low fat milk", "ReleaseDate": "\/Date( )\/", "DiscontinuedDate": null, "Rating": 3, "Price": "3.5" }], "__count": 9 } OData.svc/Products?$top=2&$inlinecount=allpages&$format=json

QUERY $filter Basic predicates, built-in functions $sort Properties, expressions $select Narrow the set of fields returned $top/$skip Client-side paging $expand Include related entities $count/$inlinecount Include count of entities Server Driven Paging

ENTITY DATA MODEL Entity Types Named structures with keys Support Inheritance May be Open Properties String, Integer, Boolean, DateTime, Spatial datatypes Collections, Complex Types Relationship Properties Complex Types Named Structures w/o keys Can’t have relationships to ComplexTypes Associations Expose navigation paths EntityContainer Contains EntitySets, AssociationSets, FunctionImports <NavigationProperty Name="Titles" Relationship="Self.Genre_Titles" ToRole="Genre_Titles_Target" FromRole="Genre_Titles_Source"/>