Exploratory Data Analysis & Visualization, Spring 2015: Intro to APIs and examples in Python Yu Tian, Christine Lee March 31, 2015

What we aim to go over today:  Overview to APIs. Why should we care?  How do APIs work?  Indeed.com API quick example with Python/R  Demo of Yelp API with Python/R - 1 -

- 2 - What are APIs?  API stands for Application Programming Interface – a way for applications to talk to each other. APIs are the secret plumbing of the web. They are a way to get the functionality of an application (such as Twitter, Facebook) without visiting the website for that application.  Why is it valuable? Websites are designed to cater to human's strengths. Yet, what happens when you face a very time-intensive task, like copying the contact info for a thousand customers from one site to another? You would love to delegate this work to a computer so it can be done quickly and accurately. The solution is an API. API is the tool that makes a website's data digestible for a computer. Through it, a computer can view and edit data, just like a person can by loading pages and submitting forms.

- 3 - Interfaces & servers APIs create standards for what kind of information applications expect to receive from other applications, and from user interfaces.  One interface might talk to several applications. One application might talk to several interfaces. For example, twitter.com is just one interface for twitter; facebook.com is just one interface for facebook.  (It helps to remember that the API is simply another program running on the server.)

sides in an integration of an API: client & server When two systems (websites, desktops, smartphones) link up through an API, we say they are "integrated." In an integration, you have two sides, each with a special name. One side we have already talked about: the server. This is the side that actually provides the API. The other side is the "client." This is a separate program that knows what data is available through the API and can manipulate it, typically at the request of a user. A great example is a smartphone app that syncs with a website. When you push the refresh button your app, it talks to a server via an API and fetches the newest info.

- 5 - An example: smartphone app pizza parlor API  APIs receive “calls” that tell them: –Who you are –What you want the API to do –What it needs to know in order to do that  Example: In this scenario, the pizza parlor API is the server and the smartphone app is the client. A customer uses the app to choose a pizza and then hits a button to place the order. As soon as the button is pressed, the app knows it needs to make a request to the pizza parlor API.

- 6 - Why APIs matter Before 2010 the concept of API use is not widespread…  Projected Open API Growth Rate (Source: Craig Burton and Phil Windley)

- 7 - Why APIs matter Why do companies open up API use? 1) APIs let your product grow faster than you can build it. Twitter’s case: anyone who wants to build a better product/app with our platform, go ahead -  What happened then: Many developers then built apps/businesses with the Twitter API  What this meant: 1.Twitter users have different ways to access the Twitter system through different apps; 2. The developer ecosystem enabled / help drive growth of the Twitter platform to where it is today. 2) APIs might be important for product launches. Examples: - Google+ in its first 2 years; - Bitly’s bundle product didn’t have an API and hence lost developers’ interest early on.

- 8 - Limitations with API use Some key limitations:  Not every platform or company has an API, or useful API. Examples: Google+ in its first 2 years; Bitly’s bundle product  Some APIs may have limitations or restrictions to full data access Example: LinkedIn’s API will be restricting full access to select partners  If you are developing an app/business on a platform’s API, you may be at the mercy of that company’s API policy and agendas

- 9 - APIs made it possible for data to be exchanged in ways that weren’t possible before OAuth lets an application interact with a platform on behalf of a user. Application: Favstar.fm Platform: Twitter User: You

Oauth example: Syncing AmEx card with Foursquare to unlock savings  AmEx uses Foursquare on your (user’s) behalf. Which means AmEx can access your Foursquare data, but NOT that Foursquare can access your AmEx data. Application: AmEx Platform: Foursquare User: You

Authentication We know who the client and server are, and we know they speak in specific data formats to understand each other. Knowing how to talk, though, leaves an important question: how does the server know the client is who it claims to be? Key terms:  Authentication: process of the client proving its identity to the server  Credentials: secret pieces of info used to prove the client's identity (username, password...)  Basic Auth: scheme that uses an encoded username and password for credentials  API Key Auth: scheme that uses a unique key for credentials

Authentication There are several techniques APIs use to authenticate a client. These are called authentication schemes. Let's take a look at two of these schemes now.  Basic Authentication - only requires a username and password. Why does using the same username and password to access the API and manage the account is not ideal? There may be times when the client should have different permissions than the account owner. Take for example a business owner who hires a contractor to write a program that uses an API on their behalf. Trusting the contractor with the account credentials puts the owner at risk because an unscrupulous contractor could change the password, locking the business owner out of their own account.  API Key Authentication - API Key authentication is a technique that overcomes the weakness of using shared credentials by requiring the API to be accessed with a unique key. In this scheme, the key is usually a long series of letters and numbers that is distinct from the account owner's login password. The owner gives the key to the client, very much like a hotel gives a guest a key to a single room.

Demo: Indeed.com API request  A web search on indeed.com  Indeed API’s available parameters

APIs receive “calls” that tell them: - Who you are - What you want the API to do - What it needs to know in order API call in Python – using an example from a Github repo –create a free Publisher account to receive an API key and access tools to receive job data –define required parameters

API result output from Python converted to a csv table

Now, the fun part – mapping API data onto geographic map in R  How to visualize Indeed job search results in an interesting way?  Packages used in R: Leaflet

On selecting map tiles / creating custom base maps on Mapbox  Insert text here –Insert text here

Indeed job search results mapped with Leaflet package in R

Zooming in..

References for this presentation Sources for some of the case examples an illustrations in the section on Intro to APIs: –Matt LeMay, “The What and Why of APIs” –Zapier, Introduction to APIs -