Syndication For Search Engines Presented By: Nageswari Vallabhaneni Sukumar Manduva

Google Data GData provides a simple standard Protocol for reading and writing data on web. Atom or RSS Atom Publishing Protocol HTTP GET request HTTP PUT request. Results as an Atom or RSS feed

Feed Provides frequently updated content. Syndication of a web feed allows users to subscribe to it. A web feed is also called syndicated feed.

Google Data API Authentication Protocol Basics Common Elements: "Kinds" Batch Processing Google Base API You Tube API

Authentication Client app will act on behalf of a particular user of a Google service. Access to account specific information Request Contains Headers for (X-GData-Key, Authorization). X-GData-Key  Developer Key Authorization  ClientLogin (or) AuthSub

Authentication Client app will act on behalf of a particular user of a Google service. GData uses two different authentication systems … 1. "ClientLogin" system 2. "AuthSub" system

ClientLogin Authentication ClientLogin Method is used for stand alone applications. Users username, password and the application are encoded in the Authentication request. After a successful authentication request, use the Auth value to create an Authorization header for each GData request: Authorization:GoogleLoginauth=yourAuthValue

ClientLogin Authentication

ClientLogin Authentication If a ClientLogin-authenticated GData request fails, you may receive any of a variety of HTTP error status codes in response 403 Account disabled 403 Account deleted 401 Token disabled 401 Token expired

AuthSub Authentication When an API Request is made the Developer Key Header Specifies on behalf of Which registered developer the request is coming.

AuthSub Authentication "AuthSub" system Used if your client is a multi-user web application client To acquire an AuthSub token your web app must redirect the user to the AuthSubRequest URL at Google which has “NEXT” query parameter. Gets an authentication token without ever handling the user's account login information

AuthSub Authentication After you've acquired an authentication token, you use that token to create an Authorization header for each request: Authorization: AuthSub token="yourAuthToken"

AuthSub Authentication If an AuthSub-authenticated GData request fails, you may receive any of a variety of HTTP error status codes in response 401 Token invalid 401 Token disabled 401 Token expired 401 Token revoked

AuthSub Authentication Header for 401 Token Revoked Error 401 Token revoked WWW-Authenticate: AuthSub realm="http://www.google.com/accounts/AuthSubRequest" Content-Type: text/html; charset=UTF-8 Expires: Wed, 26 Mar 2008 17:11:31 GMT Cache-Control: private, max-age=0 Content-Length: 149 Date: Wed, 26 Mar 2008 17:11:31 GMT Server: GFE/1.3

Protocol Basics Requesting a feed or other resource: Assume there's a feed called /Newsfeed, with no entries in it.... To see it, send the following request to the server: GET /Newsfeed

Protocol Basics… The server responds: 200 OK <?xml version="1.0"?> <feed xmlns="http://www.w3.org/2005/Atom"> <title>e-News</title> <updated>2008-03-26T16:25:00-08:00</updated> <id>http://www.cnn.com/Newsfeed</id> <author> <name>Jo March</name> </author> <link href="/Newsfeed" rel="self"/> </feed>

Protocol Basics… Inserting a new entry: POST /Newsfeed <?xml version="1.0"?> <entry xmlns="http://www.w3.org/2005/Atom"> <author> <name>Paul Harvey</name> <email>Pharvey@gmail.com</email> </author> <title type="text">HeadLines</title> <content type="text">VT Beat UVa</content> </entry>

Protocol Basics… The server responds: 201 CREATED <?xml version="1.0"?> <entry xmlns="http://www.w3.org/2005/Atom"> <id>1</id> <link rel="edit" href="http://cnn.com/Newsfeed/1/1/"/> <updated>2008-03-26T16:26:03-08:00</updated> <author> <name>Paul Harvey</name> <email>Pharvey@gmail.com</email> </author> <title type="text">HeadLines</title> <content type="text">VT Beat UVa</content> </entry>

Protocol Basics… Updating an entry : To update an existing entry, use PUT, with the entry's edit URI Changing entry's text from (" VT Beat UVa ") to (“VT embarrassed UVa."): PUT /myFeed/1/1/ <?xml version="1.0"?> <entry xmlns="http://www.w3.org/2005/Atom"> <id>1</id> <link rel="edit" href="http://cnn.com/Newsfeed/1/1/"/> <updated>2008-03-26T16:28:05-08:00</updated> <author> <name>Paul Harvey</name> <email>Pharvey@gmail.com</email> </author> <title type="text"> Head Lines </title> <content type="text"> VT embarrassed UVa.</content> </entry>

Protocol Basics… The server responds: 200 OK <?xml version="1.0"?> <entry xmlns="http://www.w3.org/2005/Atom"> <id>1</id> <link rel="edit" href="http://cnn.com/Newsfeed/1/2/"/> <updated>2008-03-26T16:28:05-08:00</updated> <author> <name>Paul Harvey</name> <email>Pharvey@gmail.com</email> </author> <title type="text"> Head Lines </title> <content type="text"> VT embarrassed UVa.</content> </entry> Edit URI has changed..It now ends with "/2/" instead of "/1/". The final number in the edit URI is a version number.

Protocol Basics… Searching for a string: To do a full-text search for a particular string , send a GET request with the q parameter. GET /Newsfeed?q=VT

Protocol Basics… The server responds with all the entries that match the search string “VT” 200 OK <?xml version="1.0"?> <feed xmlns="http://www.w3.org/2005/Atom"> <title>e-News</title> <updated>2008-03-26T16:26:03-08:00</updated> <id>http://www.cnn.com/Newsfeed</id> <author> <name>Jo March</name> </author> <link href="/Newsfeed" rel="self"/> <entry> <id>1</id> <link rel="edit" href="http:// www.cnn.com/Newsfeed /1/2/"/> <name>Paul Harvey</name> <email>Pharvey@gmail.com</email> <title type="text"> Head Lines </title> <content type="text"> VT embarrassed UVa </content> </entry> </feed>

Protocol Basics… Deleting an entry: To delete an existing entry, send a DELETE request, using the entry's edit URI DELETE /Newsfeed/1/2/ The server responds: 200 OK

Common Elements: "Kinds" Definiton:The collection of elements that together describe a particular item is called a kind For example, The collection of elements that together describe a contact is called the Contact "kind."

Common Elements: "Kinds" Some of the elements in the ‘kind’ are ordinary Atom or RSS elements; Others are defined by Google in a namespace called the "Google data namespace.“ And are represented ‘gd:’ Ex:<gd:phone number>

Common Elements: "Kinds" When an entry in a Google data API feed uses a given kind An Entry contains an <atom:category> element with Its scheme attribute set to "http://schemas.google.com/g/2005#kind" Its term attribute set to the name of the schema URL followed by the name of the kind Ex: a Contact entry includes the following element: <atom: category scheme="http://schemas.google.com/g/2005#kind" term="http://schemas.google.com/g/2005#contact"/>

Common Elements: "Kinds" Example : Contact Kind http://code.google.com/apis/gdata/elements.html#gdContactKind

Batch Processing Batch processing gives you the ability to execute multiple operations in one request, rather than having to submit each operation individually. For example, the following feed includes four operations: <feed> <entry> <batch:operation type="insert"/> ... what to insert ... </entry> <batch:operation type="update"/> ... what to update ... <batch:operation type="delete"/> ... what to delete ... <batch:operation type="query"/> ... what to query ... </feed> http://code.google.com/apis/gdata/batch.html#Submit_HTTP

Overview Table Feature GData Atom RSS2.0 Syndication Format Y Queries Updates Optimistic Concurrency Authentication

Google Base Data API Overview Feeds Used Authentication Querying

Overview The Google Base data API enables developers to : Query Google Base data to create applications and mashups. Input and manage Google Base items programmatically. Google Base is a place where you can submit all kinds of content for Google to host and to make searchable online

Overview… Google Base terminology : Google Base data API services Data items Attributes Item types Google Base data API services searching for data items discovering metadata inserting, updating, and deleting data items

Feeds Used Google Base uses two feeds for data Snippets Feed Items Feed

Feeds Used… Snippets Feed : The URL for the snippets feed is: http://www.google.com/base/feeds/snippets The snippets feed is read-only. For the general public and provides a slightly shortened description

Feeds Used… Items Feed : The URL for the items feed is: http://www.google.com/base/feeds/items The items feed is read-write. A private customer-specific feed for customers to insert, update, delete, and query their own data. This feed requires authentication

Google Base data API Hiding Attributes You can control whether attributes are visible by specifying the XML attribute access="private" <g:private_id type="text" access="private"> this is not visible by others </g:private_id>

Google Base data API Hiding items You can use the <app:draft> extension element , to upload an item as a draft. You can see draft items on your own items feed. In order to mark an item as a draft, include the <app:control> element <entry> ... <app:control xmlns:app='http://purl.org/atom/app#'> <app:draft>yes</app:draft> </app:control> </entry>

Authentication Authenticating users http://www.google.com/base/api/demo/html/demo.html#authenticate

Querying Querying for items A Google Base query is a specially encoded URL that contains parameter ‘bq’ and uses an HTTP GET method to communicate with the server http://www.google.com/base/api/demo/html/demo.html#running

Querying… Querying for metadata Each data item in Google Base is an instance of an item type. The item types promoted by Google come with a set of approved attributes http://www.google.com/base/api/demo/html/demo.html#metadata

YouTube Data API Feeds and Entries Uploading Searching Updating Deleting Sharing Subscribing

Video Feeds and Entries The result of a video feed retrieval or search will give us an Atom feed that has one <entry> tag for each of the video in the result set. The Atom feed that we get will also have all the details of the individual video’s we got through search. The feed will also contain the total number of results in the list, the index of the first item in the list, the number of items in the list and other metadata about the feed. Understanding video feeds and entries

Uploading Videos 1.Direct Uploading: This method first stores the There are two methods of uploading videos on to You Tube they are: 1.Direct Uploading: This method first stores the files on your server and then on You Tube through an API Request. 2.Browser-based Uploading: This method directly sends the users uploaded files to the You Tube servers

Direct Uploading Process flow diagram for Direct Uploading

Browser-based Uploading Process flow diagram for Browser-based Uploading

Searching For Videos Searching for standard video feeds: Standard feeds contain lists of videos that either reflect YouTube user behavior, such as top-rated and most viewed video feeds, or were selected by YouTube staff, such as recently featured and mobile video feeds. To retrieve a standard feed, send an HTTP GET request to the URL associated with that feed. http://gdata.youtube.com/feeds/api/standardfeeds/top_rated?time=today

Standard Feeds Name Feed Id Example Top rated top_rated Top favorites http://gdata.youtube.com/feeds/api/standardfeeds/top_rated Top favorites top_favorites http://gdata.youtube.com/feeds/api/standardfeeds/top_favorites Most viewed most_viewed http://gdata.youtube.com/feeds/api/standardfeeds/most_viewed Most recent most_recent http://gdata.youtube.com/feeds/api/standardfeeds/most_recent Most discussed most_discussed http://gdata.youtube.com/feeds/api/standardfeeds/most_discussed Most linked most_linked http://gdata.youtube.com/feeds/api/standardfeeds/most_linked Most responded most_responded http://gdata.youtube.com/feeds/api/standardfeeds/most_responded Recently featured recently_featured http://gdata.youtube.com/feeds/api/standardfeeds/recently_featured Videos for mobile phones watch_on_mobile http://gdata.youtube.com/feeds/api/standardfeeds/watch_on_mobile

Searching For Videos Searching for a Users Video’s: Retrieving a particular users videos http://gdata.youtube.com/feeds/api/users/username/uploads http://gdata.youtube.com/feeds/api/users/abimm/uploads Searching for Related Video’s: http://gdata.youtube.com/feeds/api/videos/0qz8gsVyFaA/related

Searching For Videos Parameter Definition http://gdata.youtube.com/feeds/api/videos? Search requests can include any of the following parameters. Parameter Definition vq The vq parameter specifies a search query term. orderby Parameter that will be used to sort Max-results Maximum number of results in the result set author Video’s uploaded by a specific user alt Specifies the format of the feed to be returned Start-index Specifies the index of the first matching result lr Restricts the search to videos that have a title, description or keywords in a specific language.

Searching For Videos Parameter Definition format The format parameter specifies that videos must be available in a particular video format. racy Allows a search result set to include restricted content as well as standard content. restriction Identifies the IP address that should be used to filter videos that can only be played in specific countries time Restricts the search to videos uploaded within the specified time.

Searching Searching for a video with a keyword of abimm http://gdata.youtube.com/feeds/api/videos?vq=abimm Searching for a video with a keyword of abimm and orderby published date http://gdata.youtube.com/feeds/api/videos?vq=abimm&orderby=published Searching for a video with a keyword of demo and orderby published date and by abimm author http://gdata.youtube.com/feeds/api/videos?vq=demo&orderby=published&author=abimm

Updating a Video Entry To update a video, send an HTTP PUT request to the URL identified in the video entry's <link> tag where the rel attribute value is edit. <link rel='edit' type='application/atom+xml' href='http://gdata.youtube.com/feeds/apis/users/abimm/uploads/914IMX9RSTE'> The body of the PUT request is an Atom XML entry that contains information about the video. excluding an element will delete the information that already exists for that video Updating a video entry

Deleting a Video Entry Deleting a Video Entry To delete a video, send an HTTP DELETE request to the edit URL for that video DELETE /feeds/api/users/USER_ID/uploads/VIDEO_ID HTTP/1.1 Host: gdata.youtube.com Content-Type: application/atom+xml Authorization: AuthSub token=AUTHORIZATION_TOKEN X-GData-Client: CLIENT_ID X-GData-Key: key=DEVELOPER_KEY Deleting a Video Entry

Sharing Videos To send a video http://gdata.youtube.com/feeds/users/abimm/inbox 1. API request to retrieve information about a video. 2. After watching the video, the user clicks a link to send the video to one or more contacts. 3. Your application submits a request to retrieve the contact list for the currently authenticated user. 4. Your application displays a form that allows the user to select one or more contacts to send the video to. You send an HTTP PUT request to each selected contact.

Subscriptions A subscription refers to an entity that notifies a user when new videos are uploaded. Retrieving Subscriptions: To request a feed of a particular user's subscriptions, send an HTTP GET request to the following URL. http://gdata.youtube.com/feeds/api/users/username/subscriptions http://gdata.youtube.com/feeds/api/users/abimm/subscriptions

References http://code.google.com/apis/gdata/ http://code.google.com/apis/base/ http://code.google.com/apis/youtube/overview.html www.YouTube.com