1. Introduction to gathering and analyzing data via APIs Gus Cavanaugh
Using APIs with Python
Introduction to gathering and analyzing data via APIs
Gus Cavanaugh

2. Agenda Brief overview of Data Analytics Office Hours Meetup
Introduce the Application Programming Interface (API)
Overview of different types of APIs
General process for finding and using an API
Practical example: gather tweets from Twitter
Practical example: perform sentiment analysis with IBM’s Watson
Resources for future learning

3. Data Analytics Office Hours
About me: Consultant trying to use data to solve business problems
Meetups have been super helpful for learning technical subjects. But most are in-depth presentations on a topic. As such, I often quickly get lost.
When I gave meetup talks, the best questions were often asked by individuals after the talk was over. Worse, they were usually foundational questions that meant the attendee missed out on a lot of value
I want this meetup to be the place where those types of questions are asked.
Presentations will be short or non-existent – just enough to cover some basic ideas. After that, it’s up to us.
To put it crudely, I want this to be the forum for “stupid” questions. If one person asks one question they would be afraid to raise their hand and ask at a large meetup, this will have been a success

4. My ask of you Introduce yourself Ask Questions… Through the webinar
Through the meetup: Analytics-Office-Hours/
Via On
On to the material!

5. Application Programming Interfaces (APIs) allow you to access data programmatically (with code). Generally speaking this means you get way more better data from a given website than if you used only your web browser.

6. What this looks like with Python
Hit Twitter API
Hit Watson API

7. What is an API?
Application Programming Interface (API): code talking to code
When we use a computer, we generally visit web sites using our browser. Here we see the web sites Graphical User Interface (GUI). While GUIs are great for interacting with the web site on a small scale, we may want to gather or provide more information than is convenient with via manually doing that with a web browser
To solve this problem, companies will create APIs such that users of their site can gather or provide data by writing code rather than using their browser. The data and functionality they make available is entirely up to them.
Most companies offer some combination of free and paid access. Companies will often enforce this by limiting the data one can access or limiting the frequency of access.
Yelp will only allow free users of their API to access only one random review of a restaurant
Twitter will provide the same data to free and paid users, but will restrict the number of calls free users can make in a given time period

8. REST APIs – a “good enough” intro
REST stands for Representational State Transfer
What you need to know: when an application is RESTful, you can access data at specific URLs via HTTP
Example:
When I visit my twitter timeline, I’m making an HTTP GET request with my browser. Twitter sends HTML, CSS, and JS to my browser, which my browsers renders into the Twitter website
With Twitter’s REST API, I write Python code to make an HTTP GET request to the essentially the same URL. Twitter returns JSON formatted data to me (just text). I can now parse that JSON with Python to get the data I want

9. Finding and Accessing APIs
If you find yourself heavily using a website, it may be a good candidate for using the website’s API (almost always the case if one is gathering data for analysis)
To find out if a website has an API, look for a “developers” link on the companies home page. Or just google “<website> + API”
Most companies with APIs will provide documentation at a url similar to dev.company.com, .e.g.,
You will need to register with each with each company
You may be asked to provide a url: (your local host) will work – any valid url, regardless of whether it is a real website, will work.
You will be given API credentials, usually a key & secret
DO NOT SHARE THESE WITH ANYONE EVER!!!
The keys and secrets you see in this presentation are fake

10. Using the API Read the documentation!
This is where you’ll find the data you can access and the url you will need to use
You will need to make your API call – generally a GET or PUT request – with the information specified in the company’s documentation. Think of this like a secret handshake. You need to begin the sequence
They will respond with data in a format that they have specified in the documentation
We’ll use the Python library Requests to make our HTTP Requests
If it’s not installed, you can install it with pip install requests

11. Twitter Example Libraries required: python-twitter
Make API call with python- twitter library
Pass in arguments for
username and number of
tweets
Parse response with pandas
Not required, just nice

12. Watson Example Libraries required: requests, json
Unlike with twitter, we don’t have a Python wrapper for Watson
We have to use requests to make a post request
Then we need to use the json library to parse the results
Finally we use pandas (optional) to clean the response