%GetTweet - A New SAS Macro To Fetch and Summarize Tweets Satish Garla Goutam Chakraborty Oklahoma State University

Agenda  Background  Process  %GetTweet Macro  Examples on macro utility  Tweets vs Retweets  Tweet Report  Understanding Networks  Conclusion

Background  Growth of Social Media Analytics  Rise of Twitter  Average number of users per day: 460,000*  Average number of Tweets per day: 140 Million*  1 Billion Searches per day**  Challenge: Getting the right data (Tweets)  Twitter Search API and SAS * During Feb-Mar 2011 Source: blog.twitter.com ** During Oct 2010 Source: blog.twitter.com

Process HTTP Procedure communicates with Twitter via the Search API Twitter returns the data in XML format SAS XML Engine is used to convert XML files to SAS Data sets SAS Data set Summary Reports Text Mining Sentiment Analysis GetTweet

%GetTweet Macro  USP: Collecting customized tweets  %GetTweet(WORDS=,PHRASE=,ANY=,NONE=,HASH=,FROM=, TO=,SINCE=,UNTIL=,QUESTION=,CODE=,PATH=);  CODE=  Base 64 encoded Twitter Username and Password Ex: dXhsdgfhsdefhJ….  PATH=  Folder Directory  Other requirements,  XML Mapper Code

Examples  Collecting Tweets that Mention the Word “Cancer” %GetTweet (WORDS=Cancer, CODE= &authorization, PATH=&path);  “Cancer” referred as  Disease  Sun sign / Astrology

Examples  Collecting Tweets without a Specific Set of Words %GetTweet (WORDS=Cancer, NONE=Zodiac Scorpio Pisces Astrology,..);  “Cancer” referred as  Disease  Sun sign

Examples  Collecting Tweets on a specific day %GetTweet (WORDS=Cancer, SINCE= , UNTIL= ,..);  Collecting Tweets with the words “SAS” or “Analytics” %GetTweet ( ANY=SAS Analytics, CODE= &authorization, PATH=&path);  Other Keyword parameters  FROM= TO= HASH= QUESTION=

Tweets Vs. Retweets  Retweets Tweet: Monday Keynote Update ~ Bret Michaels: Music Icon, Actor, 'Celebrity Apprentice' Winner Keynotes #DMA Retweet: Monday Keynote Update ~ Bret Michaels: ………………… Re-Retweet: Monday Keynote Update ~ Bret Michaels: …….  Strength of Networks and Influencers  Challenges in Text Mining Retweets Tweet: Rigorous exercise would help in curing breast cancer  Data sets of Tweets and Retweets are created

Tweet Report  Basic data cleaning using PERL  Generate PDF report  Report for #DMA2010  Page 1 (Tweet Report)  Page 2 (Top Retweets)

Tweet Report (#SASGF11)

Understanding Networks  Insights on Influencers in Twitter networks  SAS/GRAPH ® NV Workshop

Conclusion  Capture data from Twitter using PROC HTTP and Twitter API  Use GetTweet Macro  SAS Data set of customized Tweets and Retweets  Generate Summary Reports  Exploring Twitter Networks using SAS NV workshop  Customize the macro to solve research requirements *Full Macro Program is available in the paper