Enhancing Substantive Analytical Procedures with Third-Party Generated Information from Social Media My research focuses on … examine the usefulness of social media info in enhancing the effectiveness .Social media information however, its relevance to auditing remains underexplored. Presented by: Andrea M. Rozario Ph.D. Candidate

Kara Stein – SEC Commissioner 2015 Introduction (1) “Investors, and others, are accessing and analyzing massive amounts of information from sources, like social media, unimaginable just a few years ago. This new data may be empowering investors to make smarter investment decisions” Kara Stein – SEC Commissioner 2015 So it’s of interest to examine how auditors canuse information from social media to make smarter audit decisions, and enhance AQ

Introduction (2) PCAOB AS 2110 Internal Financial Internal Nonfinancial Less reliable Develop account expectation External Financial External Nonfinancial More reliable but not timely! Analytical Procedures Compare account expectation to actual AS 2110: Identifying and Assessing Risks of Material Misstatement 1 – introduce analytical procedures – what they are, and what steps auditors follow to perform them 2 – in this research study I focus on the first step So it is also interesting to examine how auditors are performing this audit, specifically, analytical procedures, in this highly digitized society Then and the now – for analytical procedures THIS IS HOW AUDITORS TYPICALLY develop expectations…. BUT PROBLEM - THEY HAVE LIMITATIONS, INTERNAL INFO MAY BE TIMELY, but could be manipulated by management EXTERNAL (external financial – analyst forecasts) INFO COULD BE MORE RELIABLE, BUT NOT AVAILABLE IN TIMELY MATTER SOMETHING NEEDS TO CHANGE Determine whether difference is significant PCAOB AS 2110

Introduction (3) PCAOB AS 2110 Internal Financial Internal Nonfinancial Less reliable Develop account expectation External Financial External Nonfinancial More reliable but not timely! Analytical Procedures Compare account expectation to actual External Nonfinancial Information from Social Media So it is also interesting to examine how auditors are performing this audit, specifically, analytical procedures, in this highly digitized society Then and the now – for analytical procedures THIS IS HOW AUDITORS TYPICALLY develop expectations…. BUT PROBLEM - INTERNAL INFO MAY BE TIMELY, but could be manipulated by management EXTERNAL (external financial – analyst forecasts) INFO COULD BE MORE RELIABLE, BUT NOT AVAILABLE IN TIMELY MATTER SOMETHING NEEDS TO CHANGE Determine whether difference is significant PCAOB AS 2110

Objectives Do Twitter proxies of consumer interest and consumer satisfaction enhance substantive analytical procedures for the revenue account? Investigate whether information generated by third-parties on social media can Improve the prediction performance of substantive analytical procedures Improve the error detection performance of substantive analytical procedures

Motivation Social media postings contain incremental information about firms’ stock market prices, and sales performance (e.g. Bollen, Mao, Zheng 2011; Tang 2017) Inspection findings indicate that accounting firms fail to develop precise expectations (PCAOB 2007; PCAOB 2016a) Social media consumer postings about firms’ products and brands could be used as a source of audit evidence

Findings Contribution Auditors can benefit from incorporating social media information in continuous substantive analytical models Model with lagged sales, TCI, and GDP outperforms other models Contribution Contributes to the auditing literature by investigating the relevance of social media information that is generated by third parties to auditing -outperforms other models – including other Twitter models and the benchmark models Contribution - The findings for prediction and error detection performance suggest that TCI has incremental value in the absence of contemporaneous firm information, or macroeconomic information, and that it can complement models with macroeconomic information

Research Questions RQ 1A: For the revenue account, do traditional substantive analytical models that contain Twitter-based information produce more accurate predictions than traditional substantive analytical models that do not incorporate it? RQ 1B: For the revenue account, do continuous substantive analytical models that contain Twitter-based information produce more accurate predictions than continuous substantive analytical models that do not incorporate it? RQ 2A: For the revenue account, are traditional substantive analytical models that contain Twitter-based information better at detecting errors than traditional substantive analytical models that do not incorporate it? RQ 2B: For the revenue account, are continuous substantive analytical models that contain Twitter-based information better at detecting errors than continuous substantive analytical models that do not incorporate it? I differentiate between traditional/substantive Generally, auditors use PY, but it is possible for more timely aps to produce more accurate predictions, so I want to test this expectation

Research Design (1) Sample – 24 B2C industries Likefolio, https://home.likefolio.com/, and Compustat Quarterly financial information is interpolated into monthly observations and matched with Twitter data

Research Design (2) Twitter Measures Likefolio, https://home.likefolio.com/, provided customer interest and satisfaction for products and brands Mapping of brands and products to the company Customer Interest to Buy TCI: total count of tweets related to the firm’s product or brand past/future interest to buy TCS: ratio of positive tweets to total (positive and negative) tweets I want to emphasize that this dataset is very uniquie -Mappiing of thosuands of brands/products that belong to a company. Extract tweets at the product/brand level and they aggregate it at the firm level Customer Sentiment

Research Design (3) Traditional SAPs Continuous SAPs (1) (5) (6) (2) Models Traditional SAPs Continuous SAPs (1) (5) (6) (2) (7) (8) (3) (9) (10) (4) (11) (12)

Twitter Consumer Interest Twitter Consumer Satisfaction Results (1) Prediction Performance – 24 industries Twitter Consumer Interest Twitter Consumer Satisfaction Model (1) vs. (5) (2) vs. (7) (3) vs. (9) (4) vs. (11) (1) vs. (6) (2) vs. (8) (3) vs. (10) (4) vs. (12) Traditional - SAP 16 of 24 14 of 24 15 of 24 12 of 24 Continuous - SAP 19 of 24 21 of 24 18 of 24 22 of 24 20 of 24 Go across – traditional SAP, simple model is able to produce better expectations just as the more complex models. Continuous SAPs models 7 and 11 as well as model 12. You can see that compared to traditional, continuous SAP generate superior predictions, especially continuous SAPs that have TCI information.

Twitter Consumer Interest Results (2) Error Detection Performance – 24 industries Twitter Consumer Interest False Positive False Negative Model (1) vs. (5) (2) vs. (7) (3) vs. (9) (4) vs. (11) Traditional - SAP 14 of 24 11 of 4 of 6 of 5 of Continuous - SAP 16 of 15 of 10 of 24 9 of 8 of Now we move on to error detection performance. First we examine the results for TCS for FP and FN error rates and then we do the same for TCS. Ideally the model should be able to produce reasonably low FP and FN errors For fn we see that the results are inferior to the benchmark models. Where the better performing models in this case are models 7 and 11 which are able to produce lower FN error for only 6 of the 24 industries Now for Continuous SAPs error detection performance generally improves for FPs compared to TSAP. For FN error detection performance improves as well compared to TSAP, but it is still inferior to the benchmark models as the better performing model in this case is model 7, which is able to produce lower fn error for only 10 of the 24 industries that are analyzed. Which comes at the cost of higher FN rates

Twitter Consumer Interest Results (3) Error Detection Performance Cost Ratio – 24 industries Twitter Consumer Interest   1 to 1 1 to 2 Model (1) vs. (5) (2) vs. (7) (3) vs. (9) (4) vs. (11) Traditional - SAP 9 of 24 4 of 10 of 24 11 of 10 of 24 8 of Continuous - SAP 17 of 24 20 of 24 14 of 24 15 of 24 18 of 24 of So now I analyze the ratio of the costs between the two types of errors as an additional criteria to determine the models that are more effective for error detection performance . I use a cost ratio of 1:1 and 1:2 TSAP – 7 is better as it is able to generate relatively low FP and FN errors for 11 industries So we can see that for TCI model 7 is the better model when we take the ratio of the costs of the different types of errors into account (meaning that here the model sacrifices the least amount of FN to achieve lower FPs; produces relatively low FPs and FNs at these costs)

Conclusion Used third-party generated Tweets of firms’ products and brands to provide insights into the usefulness of this information in enhancing substantive analytical procedures Continuous substantive analytical procedures with Twitter information can benefit auditors, especially models with TCI Limitations? 2 – that include lagged sales, or models with TCI that include lagged sales, and GDP

Kara Stein – SEC Commissioner 2015 Hmm… What is the next disruptive technology? How to apply it to auditing? Would it be appealing to the regulators? My professors? Introduction (1) “Investors, and others, are accessing and analyzing massive amounts of information from sources, like social media, unimaginable just a few years ago. This new data may be empowering investors to make smarter investment decisions” Kara Stein – SEC Commissioner 2015 Me and Abby giving auditor face…so we are thinking

Thank you! Would love any suggestions/feedback