Unit 07 : AJAX, JSON, REST and CouchDB COMP 5323 Web Database Technologies and Applications 2014

Doctrine of Fair Use This PowerPoint is prepared for educational purpose and is strictly used in the classroom lecturing. We have adopted the "Fair Use" doctrine in this PowerPoint which allows limited copying of copyrighted works for educational and research purposes.

Learning Objectives Understand the often-used web database technologies and implementation

Outline AJAX JSON HTTP/REST CouchDB

1 AJAX

Scenario Car Classifieds website has a dropdown with the makes of all the cars. Based on the selection of the “makes” dropdown the “models” dropdown has to be populated with the correct models provided by the manufacturer.

Sample Applications Sample 1: Get Time from the server. Sample 2: Passing parameters to the server. Sample 3: Passing objects as parameters. Sample 4: Getting a class from the server. Writing your own AJAX.

History of AJAX Starts with web pages Static web pages Static html page is loaded No interaction with user Dynamic web pages Html page is generated dynamically Interaction with user Becomes slower as functionality increases Speed becomes untolerable, so AJAX has been born

Need Prevents unnecessary reloading of a page. When we submit a form, although most of the page remains the same, whole page is reloaded from the server. This causes very long waiting times and waste of bandwidth. AJAX aims at loading only the necessary information, and making only the necessary changes on the current page without reloading the whole page.

What is Ajax? Asynchronous Javascript And XML It is a client side technology. Connection between client side script and server side script. It is a technique that combines a set of known technologies in order to create faster and more user friendly web pages so that we have better user experience Not a stand-alone technology Not a technology, but a group of technologies. Basically, server roundtrips without posting back and no browser “flickering”.

Classic VS Ajax Ajax Engine emphasis

Technologies Used AJAX uses: Javascript (for altering the page) XML (for information exchange) ASP or JSP (server side)

Simple Processing AJAX is based on Javascript, and the main functionality is to access the web server inside the Javascript code. We access to the server using special objects; we send data and retrieve data. When user initiates an event, a javascript function is called which accesses server using the objects. The received information is shown to the user by means of the Javascript’s functions.

Structure of System Client/Server architecture XMLHTTP object is used to make request and get response in Client side Request can be done via “GET” or “POST” methods “GET”: parameters are attached to the url used to connect. “POST”: parameters are sent as parameters to a function Not many changes in Server side Response is a combination of xml tags

C/S Processes Most of the time client requires two files Html page: handles GUI and calls a function of JavaScript. JavaScript: handles the business logic of the system In JavaScript, a request is sent to client and response is taken from server via XMLHTTP object Response of server should be returned in xml file structure Only requested data is returned

XMLHTTP Object The object that is used to connect to the remote server is called XMLHTTP. It resembles the Java’s URL object that is used to access a specific URL and get the contents.

Creating the object For IE 5.5: new ActiveXObject(“Microsoft.XMLHTTP”) For Mozilla: new XMLHttpRequest()

Sending information After creating the object, we can send information to the web server and get the answer using this object’s functions: GET METHOD: open(“GET”, url, true) send() POST METHOD: open("POST", url, true) send(“date=11-11-2006&name=Ali") Third argument tells that data will be processes asynchronously. When server responds, the “OnReadyStateChange” event handler will be called.

Retrieving information We get the returned value with the property “responseXML”. Example http://geogle.comp.polyu.edu.hk/cwd/ajax/example01.html http://geogle.comp.polyu.edu.hk/cwd/ajax/books.xml

Example 1 We want to input data into a textbox. We want the textbox to have intellisense property; guess entries according to input. http://www.w3schools.com/ajax/ajax_example.asp Only the ‘span’ part of the html code is changed.

Data Exchange in AJAX

Example 2 Another example: http://www.w3schools.com/ajax/ajax_database.asp Therefore, by using AJAX, unnecessary exchange of data is prevented, web pages become: More interactive Faster More user friendly

Pros/Cons Advantages: Disadvantages: Independent of server technology. Apart from obtaining the XMLHTTP object, all processing is same for all browser types, because Javascript is used. Permits the development of faster and more interactive web applications. Disadvantages: The back button problem. People think that when they press back button, they will return to the last change they made, but in AJAX this doesn not hold. Possible network latency problems. People should be given feedback about the processing. Does not run on all browsers.

2 JSON

Discussion Please list all potential data formats to share your data or exchange data over the Internet?

Data Sharing: Quandl Quandl is a platform covering millions of financial and economic time-series datasets from hundreds of sources. Its long-term mission is to make all numerical data on the internet easy to find and easy to use.

Data Sharing: Example http://www.quandl.com/help/api

Data Exchange: XML XML carries a lot of baggage, and it doesn't match the data model of most programming languages. When most programmers saw XML for the first time, they were shocked at how ugly and inefficient it was. It turns out that that first reaction was the correct one. XML has big problems as a data-interchange format, but the disadvantages are compensated for by the benefits of interoperability and openness.

Data Exchange: XML There is another text notation that has all of the advantages of XML, but is much better suited to data-interchange. That notation is JavaScript Object Notation (JSON). JSON promises the same benefits of interoperability and openness, but without the disadvantages.

XML Example <menu id="file" value="File"> <popup> <menuitem value="New" onclick="CreateNewDoc()" /> <menuitem value="Open" onclick="OpenDoc()" /> <menuitem value="Close" onclick="CloseDoc()" /> </popup> </menu> http://json.org/example

XML Example < menu id="file" value="File"> <popup> <menuitem value="New" onclick="CreateNewDoc()" /> <menuitem value="Open" onclick="OpenDoc()" /> <menuitem value="Close" onclick="CloseDoc()" /> </popup> </menu > “menu”: { “id”: “file” “value”: “File” “popup”: { “menuitemvalue” :[ ] }

JSON < menu … = ….. > < .… > </menu

JSON Despite the name called “JavaScript Object Notation”, JSON is a (mostly) language-independent way of specifying objects as name-value pairs

JSON syntax, I An object is an unordered set of name/value pairs The pairs are enclosed within braces, { } There is a colon between the name and the value Pairs are separated by commas Example: { "name": "html", "years": 5 } An array is an ordered collection of values The values are enclosed within brackets, [ ] Values are separated by commas Example: [ "html", ”xml", "css" ]

JSON syntax, II A value can be: A string, a number, true, false, null, an object, or an array Values can be nested Strings are enclosed in double quotes, and can contain the usual assortment of escaped characters Numbers have the usual C/C++/Java syntax, including exponential (E) notation All numbers are decimal--no octal or hexadecimal Whitespace can be used between any pair of tokens

eval The JavaScript eval(string) method compiles and executes the given string The string can be an expression, a statement, or a sequence of statements Expressions can include variables and object properties eval returns the value of the last expression evaluated When applied to JSON, eval returns the described object

eval http://www.w3schools.com/jsref/jsref_eval.ASP

JSON and methods? In addition to instance variables, objects typically have methods There is nothing in the JSON specification about methods However, a method can be represented as a string, and (when received by the client) evaluated with eval Obviously, this breaks language-independence Also, JavaScript is rarely used on the server side

Comparison of JSON and XML Similarities: Both are human readable Both have very simple syntax Both are hierarchical Both are language independent Both can be used by Ajax Differences: Syntax is different JSON is less verbose JSON can be parsed by JavaScript’s eval method JSON includes arrays Names in JSON must not be JavaScript reserved words XML can be validated JavaScript is not typically used on the server side

JSON and eval http://www.w3schools.com/json/json_eval.asp

YAML YAML can be taken as an acronym for either Yet Another Markup Language YAML Ain’t Markup Language Like JSON, the purpose of YAML is to represent typical data types in human-readable notation YAML is (almost) a superset of JSON, with many more capabilities (lists, casting, etc.) Except: YAML doesn’t handle escaped Unicode characters Consequently, JSON can be parsed by YAML parsers When JSON isn’t enough, consider YAML

XML <user id="babooey" on="cpu1"> <firstname>Bob</firstname> <lastname>Abooey</lastname> <department>adv</department> <cell>555-1212</cell> <address password="xxxx">ahunter@example1.com </address> <address password="xxxx">babooey@example2.com </address> </user>

YAML user: id: babooey             computer : cpu1             firstname: Bob             lastname: Abooey             cell: 555-1212             addresses:                 - address: babooey@example1.com                   password: xxxx                 - address: babooey@example2.com                   password: xxxx

Reference for YAML http://www.yaml.org/ http://jessenoller.com/blog/2009/04/13/yaml-aint-markup-language-completely-different

3 REST

HTTP Model

Web Applications CLIENT Same domain MySQL HTTP DATABASE MySQL Protocol WEB SERVER HTTP

Accessing Data Using HTTP Specify SQL queries directly in the URL, for example: http://IISServer/nwind?sql=SELECT+*+FROM+Customers+FOR+XML+AUTO&root=root The FOR XML clause returns the result as an XML document instead of a standard rowset. The root parameter identifies the single top-level element. http://technet.microsoft.com/en-us/library/aa226553%28v=sql.80%29.aspx

What is REST? REST (Representational state transfer)is a design pattern. It is a certain approach to creating Web Services. To understand the REST design pattern, let's look at an example (learn by example).

Example: Airline Reservation Service Suppose that an airline wants to create a telephone reservation system for customers to call in and make flight reservations. The airline wants to ensure that its premier members get immediate service, its frequent flyer members get expedited service and all others get regular service. There are two main approaches to implementing the reservation service...

Approach 1 "Press 1 for Premier, Press 2 for…" The airline provides a single telephone number. Upon entry into the system a customer encounters an automated message, "Press 1 if you are a premier member, press 2 if you are a frequent flyer, press 3 for all others." Premier Customer Representative Premier Members F.F. Customer Representative Answering Machine Airline Reservations Frequent Flyer Members Regular Customer Representative Regular Members

Approach 2 Telephone Numbers are Cheap! Use Them! The airline provides several telephone numbers - one number for premier members, a different number for frequent flyers, and still another for regular customers. Premier Customer Representative 1-800-Premier Premier Members F.F. Customer Representative 1-800-Frequent Frequent Flyer Members Regular Customer Representative 1-800-Reservation Regular Members

Discussion In Approach 1 the answering machine introduces an extra delay, which is particularly annoying to premier members. (Doesn't everyone hate those answering systems) With Approach 2 there is no intermediate step. Premier members get instant pickup from a customer service representative. Others may have to wait for an operator.

Web-Based Reservation Service Suppose now the airline (kings-air.com) wants to provide a Web reservation service for customers to make flight reservations through the Web. Just as with the telephone service, the airline wants to ensure that its premier members get immediate service, its frequent flyer members get expedited service, all others get regular service. There are two main approaches to implementing the Web reservation service. The approaches are analogous to the telephone service...

Approach 1 One-Stop Shopping The airline provides a single URL. The Web service is responsible for examining incoming client requests to determine their priority and process them accordingly. client Premier Customer Premier Members Web Reservation Service Determine Priority F.F. Customer client Frequent Flyer Members Regular Customer client Regular Members

Approach 1 Disadvantages There is currently no industry accepted practice (rules) for expressing priorities, so rules would need to be made. The clients must learn the rule, and the Web service application must be written to understand the rule. This approach is based upon the incorrect assumption that a URL is "expensive" and that their use must be rationed. The Web service is a central point of failure. It is a bottleneck. Load balancing is a challenge. It violates Tim Berners-Lee Web Design, Axiom 0 (see next slide).

Web Design, Axiom 0 (Tim Berners-Lee, director of W3C) Axiom 0: all resources on the Web must be uniquely identified with a URI. resource1 URL1 resource2 URL2 resource3 URL3

Approach 2: URLs are Cheap! Use Them! The airline provides several URLs - one URL for premier members, a different URL for frequent flyers, and still another for regular customers. Premier Member Reservation Service http://www.kings-air/reservations/premier client Premier Members Frequent Flyer Reservation Service http://www.kings-air/reservations/frequent-flyer client Frequent Flyer Members Regular Member Reservation Service http://www.kings-air/reservations/regular client Regular Members

Approach 2 Advantages The different URLs are discoverable by search engines and UDDI registries. It's easy to understand what each service does simply by examining the URL, i.e., it exploits the Principle of Least Surprise. There is no need to introduce rules. Priorities are elevated to the level of a URL. "What you see is what you get." It's easy to implement high priority - simply assign a fast machine at the premier member URL. There is no bottleneck. There is no central point of failure. Consistent with Axiom 0.

Recap We have looked at a reservation service. We have seen a telephone-based version and a Web-based version of the reservation service. With each version we have seen two main approaches to implementing the service. Which approach is the REST design pattern and which isn't? See the following slides.

This Ain't the REST Design Pattern Premier Customer Representative Premier Members F.F. Customer Representative Answering Machine Airline Reservation Frequent Flyer Members Regular Customer Representative Regular Members

This is the REST Design Pattern Premier Customer Representative 1-800-Premier Premier Members F.F. Customer Representative 1-800-Frequent Frequent Flyer Members Regular Customer Representative 1-800-Reservation Regular Members

This ain't the REST Design Pattern client Premier Customer Premier Members Reservation Web Service Determine Priority F.F. Customer client Frequent Flyer Members Regular Customer client Regular Members

This is the REST Design Pattern Premier Member Reservation Service http://www.kings-air/reservations/premier client Premier Members Frequent Flyer Reservation Service http://www.kings-air/reservations/frequent-flyer client Frequent Flyer Members Regular Member Reservation Service http://www.kings-air/reservations/regular client Regular Members

Two Fundamental Aspects of the REST Design Pattern Resources Every distinguishable entity is a resource. A resource may be a Web site, an HTML page, an XML document, a Web service, a physical device, etc. URLs Identify Resources Every resource is uniquely identified by a URL. This is Tim Berners-Lee Web Design, Axiom 0.

The Three Fundamental Aspects of the REST Design Pattern Resources URLs Simple Operations In this tutorial we discussed how Resources and URLs are fundamental to REST. In a follow up tutorial we will discuss how Simple Operations are also fundamental to REST.

REST REST provides a lighter weight alternative. REST relies on a simple URL in many cases. In some situations you must provide additional information in special ways, but most Web services using REST rely exclusively on obtaining the needed information using the URL approach. REST can use four different HTTP 1.1 verbs (GET, POST, PUT, and DELETE) to perform tasks.

Example: Weather API http://openweathermap.org/API

REST way of Implementing the web services

Service – Get parts list The web service makes available a URL to a parts list resource. Client uses : http://www.parts-depot.com/parts Document Client receives : <?xml version="1.0"?> <p:Parts xmlns:p="http://www.parts-depot.com" xmlns:xlink="http://www.w3.org/1999/xlink"> <Part id="00345" xlink:href="http://www.parts-depot.com/parts/00345"/> <Part id="00346" xlink:href="http://www.parts-depot.com/parts/00346"/> <Part id="00347" xlink:href="http://www.parts-depot.com/parts/00347"/> <Part id="00348" xlink:href="http://www.parts-depot.com/parts/00348"/> </p:Parts>

Service – Get detailed part data The web service makes available a URL to each part resource. Client uses : http://www.parts-depot.com/parts/00345 Document Client receives : <?xml version="1.0"?> <p:Part xmlns:p="http://www.parts-depot.com" xmlns:xlink="http://www.w3.org/1999/xlink"> <Part-ID>00345</Part-ID> <Name>Widget-A</Name> <Description>This part is used within the frap assembly</Description> <Specification xlink:href="http://www.parts-depot.com/parts/00345/specification"/> <UnitCost currency="USD">0.10</UnitCost> <Quantity>10</Quantity> </p:Part>

Service – Submit purchase order (PO) The web service makes available a URL to submit a PO. 1)The client creates a PO instance document (PO.xml) 2)Submits the PO.xml(HTTP POST) 3)PO service reponds with a URL to the submitted PO.

COMMAND The HTTP verbs comprise a major portion of our “uniform interface” constraint and provide us the action counterpart to the noun-based resource. The primary or most-commonly-used HTTP verbs (or methods, as they are properly called) are POST, GET, PUT, and DELETE. These correspond to create, read, update, and delete (or CRUD) operations, respectively.

Query and JSON

4 CouchDB

Introduction Created By : Damien Katz Year : 2005 Language : Erlang License : Apache Software Foundation(2008)

Who uses CouchDB?

Introduction NoSQL Database Uses Map/Reduce queries written in javascript

Document-Oriented DBMS Data is stored in documents ......and not in relations like an RDBMS

SQL vs CouchDB SQL CouchDB Relational Non-Relational Tables Documents with types Rows and Columns Document Fields SQL Query Engine Map / Reduce Engine

CouchDB Features Data Representation - Using JSON Interaction - Futon / CouchDB API Querying - Map / Reduce Design Documents - Application code (Language : Javascript) Documents can have attachments

HTTP API Messages are self-described via HTTP Headers and HTTP Status Codes. URIs identify resources. HTTP Methods define operations on the resources.

HTTP Request Methods Method Description PUT GET POST DELETE COPY PUT requests are used to create new resources where the URI of the request is different to the resource that is to be created. GET requests are used to request data from the database. POST requests are used to update the existing data, at the same resource the URI is requested from. DELETE requests to delete databases and documents. Copies one resource to another resource.

HTTP REST SQL REST Insert into… HTTP PUT /db/id Select * from HTTP GET /db/id Update Delete HTTP DELETE /db/id DBA HTTP GET /mydb/, HTTP GET/_all_dbs, HTTP PUT /_replicate, HTTP POST /mydb/_bulk_docs Cachable

HTTP Status Codes Status Code Description 200 (OK) 201 (Created) 304 (Not Modified) 400 (Bad Request) 404 (Not Found) 405 (Method Not Allowed) 409 (Conflict) 412 (Precondition Failed) 500 (Internal Server Error) The request was successfully processed. The document was successfully created. The document has not been modified since the last update. The syntax of the request was invalid. The request was not found. The request was made using an incorrect request method. The request failed because of a database conflict. could not create a database- a database with that name already exists. The request was invalid and failed, or an error occurred within the CouchDB server.

Curl Command - Server API Command to check if CouchDB is working at all? curl http://127.0.0.1:5984/ Response : {"couchdb":"Welcome","version":"0.10.1"}

Curl Command - Database API Command to get a list of Databases : curl -X GET http://127.0.0.1:5984/_all_dbs Command to create a Database : curl -X PUT http://127.0.0.1:5984/DB_name Curl's -v option : curl -vX PUT http://127.0.0.1:5984/DB_name Command to destroy a Database : curl -X DELETE http://127.0.0.1:5984/DB_name

Curl Command - Document API Command to create a document : curl -X PUT http://127.0.0.1:5984/albums/ 6ert2gh45ji6h6tywe324743rtbhgtrg \ -d '{"title":"abc","artist":"xyz"}' Command to get a UUID : curl -X GET http://127.0.0.1:5984/_uuids Command to retrieve a Document : curl -X GET http://127.0.0.1:5984/albums/ 6ert2gh45ji6h6tywe324743rtbhgtrg

Curl Command - Document API Command for attachments : curl -vX PUT http://127.0.0.1:5984/albums/ 6ert2gh45ji6h6tywe324743rtbhgtrg/ \ artwork.jpg?rev=2-2739352689 --data-binary @artwork.jpg -H "Content-Type: image/jpg" To view the image in the browser, URL is : http://127.0.0.1:5984/albums/6ert2gh45ji6h6 tywe324743rtbhgtrg/artwork.jpg

Curl Command-Replication API Command to replicate a Database : curl -vX POST http://127.0.0.1:5984/ _replicate \ -d '{"source":"albums","target": "albums_replica"}'

Reference CouchDB - The Definitive Guide , J. Chris Anderson, Jan Lehnardt & Noah Slater