Rest and Json

Web 2.0 What is Web 2.0? Commonly associated with web applications that facilitate interactive information sharing, interoperability, user-centered design and collaboration on the WWW. Usually connected with the notions of read-write web, social web but also programmable web. Typical characteristics of Web 2.0 applications Users can produce and consume data on a Web 2.0 site Web is used as a participation platform Users can run software applications entirely through a Web browser Data and services can be easily combined to create mashups

Web 2.0 File Sharing: File Management Flickr (Images), YouTube (Videos), Wikipedia (Online Encyclopedia), Blogs, Open Source Community (Linux). File Management Tagging Social Websites and Communication: Facebook, LastFM, StudiVZ, LinkedIn, Xing. Open Systems: APIs, partly open source; allow extensions by users.

Facebook More than 400 million active users 50% of our active users log on to Facebook in any given day More than 35 million users update their status each day More than 60 million status updates posted each day More than 3 billion photos uploaded to the site each month More than 5 billion pieces of content (web links, news stories, blog posts, notes, photo albums, etc.) shared each week More than 3.5 million events created each month More than 3 million active Pages on Facebook More than 1.5 million local businesses have active Pages on Facebook 1 Taken from http://www.facebook.com/press/info.php?statistics on June 6th, 2010.

Wikipedia 3,315,577 Articles (English Wikipedia)1 12,485,100 registered users1 Clever mechanisms combined with human intelligence High quality articles Self-organized control Semi-openess 1 Taken from http://en.wikipedia.org/wiki/Special:Statistics on June 6th, 2010.

YouTube Exceeds 2 billion views a day1 24 hours of video uploaded every minute 70% of YouTube’s traffic comes from outside the U.S. Google paid 1.6 Billion dollars for YouTube in 2006. 1 Taken from http://www.viralblog.com/research/youtube-statistics on June 6th, 2010.

Web 2.0 The solution can be seen in the form of Web 2.0 services Large quantities of data are on the Web The data needs to be managed in an appropriate manner Retrieved, queried, analyzed, transformed, transferred, stored, etc. Technical solutions are needed to enable a truly Programmable Web Easy integration of data and services on the Web Desktop apps should work with Web apps Flickr uploadr, Google calendar update/sync Web apps should work with the other Web apps LinkedIn can import your Facebook friends Facebook can import your Dopplr trips Mashups should be enabled Easy service composition The solution can be seen in the form of Web 2.0 services

Mashups

Example Mashup: Housingmaps.com Housingmaps.com is a mashup created of Craigslist A centralized network of online communities, featuring free online classified advertisements – with sections devoted to jobs, housing, personals, for sale, services, community, gigs, résumés, and discussion forums. Google Maps The properties described in Craigslist are placed on a map. The true power of the applied Web 2.0 approach comes from the fact that it is "in no way affiliated with craigslist or Google” It consumes Web 2.0 services provided by Craigslist and Google

Web APIs & Services Data providers usually have an incentive to offer Web APIs Web 2.0 services enable easier access to data Google maps, Geonames, phone location… Microformats (vcard, calendar, review…) Data feeds Various functionalities are offered through Web APIs Publishing, messaging, payment… Web APIs are powering the vision of the Web as a computational platform

Not just the Internet Also the intranet Consider Within a business Government Consider Utah government website and applications Services are provided and used by other services and applications Model-view-controller architecture Model can be viewed as a resource Services are used to interact with the model Business applications Distribute the services

Representational State Transfer (REST) A style of software architecture for distributed hypermedia systems such as the World Wide Web. REST is basically client/server architectural style Requests and responses are built around the transfer of "representations" of "resources". Architectural style means Set of architectural constraints. Not a concrete architecture. An architecture may adopt REST constraints. HTTP is the main and the best example of a REST style implementation But it should not be confused with REST

REST principles Information is organized in the form of resources Sources of specific information, Referenced with a global identifier (e.g., a URI in HTTP). Components of the network (user agents and origin servers) communicate via a standardized interface (e.g., HTTP) exchange representations of these resources (the actual documents conveying the information). Any number of connectors (e.g., clients, servers, caches, tunnels, etc.) can mediate the request, but each does so without being concern about anything but its own request an application can interact with a resource by knowing two things: the identifier of the resource and the action required no need to know whether there are caches, proxies, gateways, firewalls, tunnels, or anything else between it and resource The application needs to understand the format of the information (representation) returned.

REST Architecture Client-server Separation of concerns Networking Clients are separated from servers by a uniform interface. Networking Clients are not concerned with data storage, which remains internal to each server the portability of client code is improved. Servers are not concerned with the user interface or user state servers can be simpler and more scalable. Independent evolution Servers and clients may also be replaced and developed independently, as long as the interface is not altered.

REST Architecture Stateless communication Uniform Interface Scalability, reliability No client context being stored on the server between requests. Each request from any client contains all of the information necessary to service the request. Resources are conversationally stateless Any conversational state is held in the client. Uniform Interface Simplicity (vs. efficiency) Large-grained hypermedia data transfer Example: Create, Retrieve, Update, Delete

REST Architecture Caching Code-on-demand Efficiency, scalability Well-managed caching partially or completely eliminates some client-server interactions, further improving scalability and performance. Consistency issues As on the World Wide Web, clients are able to cache responses. Responses must implicitly or explicitly, define themselves as cacheable or not, to prevent clients reusing stale or inappropriate data in response to further requests. Code-on-demand Extending client functionality Servers are able to temporarily extend or customize the functionality of a client by transferring to it logic that it can execute. Examples may include compiled components such as Java applets and client-side scripts such as JavaScript.

RESTful Web Service A RESTful Web service is: A set of Web resources. Interlinked. Data-centric, not functionality-centric. Machine-oriented.

Hotel booking service service description search results hotel info Example: hotel booking Hotel booking service service description search results hotel info confirmation my bookings payment

Hotel booking workflow Example: hotel booking Hotel booking workflow Retrieve service description Submit search criteria according to description Retrieve linked details of interesting hotels Submit payment details according to selected rate description Retrieve confirmation of booking 2b. Retrieve list of user's bookings

hypermedia -> operations Example: hotel booking hypermedia -> operations search(date, city)  list of hotels & rates getHotelDetails(hotel)  hotel details reserve(rate, creditCard)  confirmationID getConfirmationDetails(confID)  confirmation details listMyBookings()  list of confirmationIDs

REST Simple Typically, each action is a url Arguments often become part of the query string in a GET HTTP request methods indicate the desired action to be performed on the identified resource: GET Requests a representation of the specified resource. GET should not be used for operations that cause side-effects (problematic with robots and crawlers). Those operations are called safe operations. POST Submits data to be processed (e.g., from an HTML form) to the identified resource. The data is included in the body of the request. PUT Uploads a representation of the specified resource. DELETE Deletes the specified resource.

REST Issues Security Control Data Transfer Stateless Authentication? Authorization? Integrity? etc Control Data Transfer JSON, XML, etc. Security issues here too

REST - Not a Standard REST is not a standard You will not see the W3C putting out a REST specification. You will not see IBM or Microsoft or Sun selling a REST developer's toolkit. REST is just a design pattern You can't bottle up a pattern. You can only understand it and design your Web services to it. REST does prescribe the use of standards: HTTP URL XML/HTML/GIF/JPEG/etc. (Resource Representations) text/xml, text/html, image/gif, image/jpeg, etc. (Resource Types, MIME Types)

Learn by Example The REST design pattern is best explained with an example. Example: a company deploying three Web services using the REST design pattern.

Parts Depot Web Services Parts Depot, Inc has deployed some web services to enable its customers to: get a list of parts get detailed information about a particular part submit a Purchase Order (PO)

The REST way of Designing the Web Services HTTP GET request URL 1 Parts List HTTP response Response (HTML/XML doc) HTTP GET request URL 2 Part Data Web Server HTTP response Response (HTML/XML doc) PO (HTML/XML) HTTP POST URL 3 PO HTTP response URL to submitted PO

Web Service for Clients to Retrieve a List of Parts Service: Get a list of parts The web service makes available a URL to a parts list resource. A client uses this URL to get the parts list: http://www.parts-depot.com/parts Note that how the web service generates the parts list is completely transparent to the client. This is loose coupling.

REST Fundamentals Create a resource for every service. Identify each resource using a URL.

Data Returned - Parts List <?xml version="1.0"?> <Parts> <Part id="00345" href="http://www.parts-depot.com/parts/00345"/> <Part id="00346" href="http://www.parts-depot.com/parts/00346"/> <Part id="00347" href="http://www.parts-depot.com/parts/00347"/> <Part id="00348" href="http://www.parts-depot.com/parts/00348"/> </Parts> Note that the parts list has links to get detailed info about each part. This is a key feature of the REST design pattern. The client transfers from one state to the next by examining and choosing from among the alternative URLs in the response document.

Web Service for Clients to Retrieve a Particular Part Service: Get detailed information about a particular part The web service makes available a URL to each part resource. For example, here's how a client requests a specific part: http://www.parts-depot.com/parts/00345

Data Returned - Part 00345 <?xml version="1.0"?> <Part> <Part-ID>00345</Part-ID> <Name>Widget-A</Name> <Description>This part is used within the frap assembly</Description> <Specification href="http://www.parts-depot.com/parts/00345/specification"/> <UnitCost currency="USD">0.10</UnitCost> <Quantity>10</Quantity> </Part> Again observe how this data is linked to still more data - the specification for this part may be found by traversing the hyperlink. Each response document allows the client to drill down to get more detailed information.

Questions and Answers

Question What if Parts Depot has a million parts, will there be a million static pages? For example: http://www.parts-depot/parts/000000 http://www.parts-depot/parts/000001 ... http://www.parts-depot/parts/999999

Answer We need to distinguish between a logical and a physical entity. The above URLs are logical. They express what resource is desired. They do not identify a physical object. The advantage of using a logical identifier (URL) is that changes to the underlying implementation of the resource will be transparent to clients (that's loose coupling!). Quite likely, Parts Depot will store all parts data in a database. Code at the Parts Depot web site will receive each logical URL request, parse it to determine which part is being requested, query the database, and generate the part response document to return to the client.

Answer (cont.) Contrast the above logical URLs with these physical URLs: http://www.parts-depot/parts/000000.html http://www.parts-depot/parts/000001.html ... http://www.parts-depot/parts/999999.html These URLs are clearly pointing to physical (HTML) pages. If there are a million parts it will not be very attractive to have a million static pages. Furthermore, changes to how these parts data is represented will effect all clients that were using the old representation.

Question What if I have a complex query? For example: Show me all parts whose unit cost is under $0.50 and for which the quantity is less than 10 How would you do that with a simple URL?

Answer For complex queries, Parts Depot will provide a service (resource) for a client to retrieve a form that the client then fills in. When the client hits "Submit" the browser will gather up the client’s responses and generate a URL based on the responses. Thus, oftentimes the client doesn't generate the URL (think about using Amazon - you start by entering the URL to amazon.com; from then on you simply fill in forms, and the URLs are automatically created for you).

Summary of the REST Design Pattern

The REST Design Pattern Create a resource for every service. Uniquely identify each resource with a logical URL. Design your information to link to other information. That is, the information that a resource returns to a client should link to other information in a network of related information.

The REST Design Pattern (cont.) All interactions between a client and a web service are done with simple operations. Most web interactions are done using HTTP and just four operations: retrieve information (HTTP GET) create information (HTTP PUT) update information (HTTP POST) delete information (HTTP DELETE)

Data transfer encoding json Data transfer encoding

JSON JavaScript Object Notation Minimal Text-based. Subset of JavaScript Language Independent. Light-weight. Easy to parse. “JSON (JavaScript Object Notation) is a lightweight data-interchange format. It is easy for humans to read and write. It is easy for machines to parse and generate” – JSON.org JSON is a subset of JavaScript

What does it look like? { "firstName": "John", "lastName": "Smith", "address": { "streetAddress": "21 2nd Street", "city": "New York", "state": "NY", "postalCode": 10021 }, "phoneNumbers": [ "212 555-1234", "646 555-4567" ] } Name/Value Pairs Child properties String Array Number data type

Values Strings Numbers Booleans Objects Arrays null

Value

Strings Sequence of 0 or more Unicode characters No separate character type A character is represented as a string with a length of 1 Wrapped in "double quotes" Backslash escapement

String

Numbers Integer Real Scientific No octal or hex No NaN or Infinity Use null instead

Number

Booleans true false null A value that isn't anything

Object Objects are unordered containers of key/value pairs Objects are wrapped in { } , separates key/value pairs : separates keys and values Keys are strings Values are JSON values struct, record, hashtable, object

Object

Object {"name":"Jack B. Nimble","at large": true,"grade":"A","level":3, "format":{"type":"rect","width":1920, "height":1080,"interlace":false, "framerate":24}}

Object { "name": "Jack B. Nimble", "at large": true, "grade": "A", "format": { "type": "rect", "width": 1920, "height": 1080, "interlace": false, "framerate": 24 }

Array Arrays are ordered sequences of values Arrays are wrapped in [] , separates values JSON does not talk about indexing. An implementation can start array indexing at 0 or 1.

Array

Array ["Sunday", "Monday", "Tuesday", "Wednesday", "Thursday", "Friday", "Saturday"] [ [0, -1, 0], [1, 0, 0], [0, 0, 1] ]

Arrays vs Objects Use objects when the key names are arbitrary strings. Use arrays when the key names are sequential integers.

JSON in Ajax & Javascript XMLHttpRequest Obtain responseText Parse the responseText responseData = JSON.parse(responseText); jsonstring = '{ "name": "Jack B. Nimble", "at large": true, "grade": "A", "format": { "type": "rect", "width": 1920, "height": 1080, "interlace": false, "framerate": 24 } }'; //jsonobject = eval('(' + jsonstring + ')'); jsonobject = JSON.parse(jsonstring); document.write("The object<br>"); document.write("name: ", jsonobject.name, "<br>"); document.write("grade: ", jsonobject.grade, "<br>"); document.write("format: ", jsonobject.format, "<br>");

Security Is it safe to use eval with XMLHttpRequest? The JSON data comes from the same server that vended the page. eval of the data is no less secure than the original html. Best to use a parser JSON.parse(string) instead of eval. Never trust the client The client cannot and will not keep our secrets and cannot and will not protect our interests. Do not trust machines not under your absolute control. The server must validate everything the client tells it.

JSON Simple, common representation of data. Communication between servers and browser clients. Communication between peers. Language independent data interchange. Simultaneously human- and machine-readable format; Support for Unicode, allowing almost any information in any human language to be communicated; Self-documenting format that describes structure and field names as well as specific values; Strict syntax and parsing requirements that allow the necessary parsing algorithms to remain simple, efficient, and consistent; Ability to represent the most general computer science data structures: records, lists and trees.

JSON vs. XML JSON XML Data Structure No validation system XSD No namespaces Has namespaces (can use multiples) Parsing is just an eval Fast Security issues Parsing requires XML document parsing using things like XPath In JavaScript you can work with objects – runtime evaluation of types In JavaScript you can work with strings – may require additional parsing Security: Eval() means that if the source is not trusted anything could be put into it. Libraries exist to make parsing safe(r) Security: XML is text/parsing – not code execution.

JSON vs. XML which to use? Scenario 1: You have a website (say Twitter.com) and you want to expose a public API to build apps. Issue JSON XML The public will be parsing data in. You must make it secure. Run checks against the data in the object to make sure it’s secure. You are working on objects so you must also check for potential code access issues. Run checks against the data to make sure it’s secure. Data must be in a specific format. Build something that parses the objects. XML Schema

JSON vs. XML which to use? Scenario 2: You have a website (say gmail.com) and your front end needs to show entries from a mailbox, but needs to be dynamic and so you will use a lot of JavaScript. Issue JSON XML Your in house developers know objects and would like to use them. JSON is JavaScript objects. Write JavaScript to parse the XML to objects. The site is secure but you worry about people checking the page source. You page has JavaScript in it and (maybe) code which communicates with a private backend server. No major issues.

JSON vs. XML Which of them should you use? Use Both – They both have strengths and weaknesses and you need to identify when one is stronger than the other.