1. Enhancing Substantive Analytical Procedures with Third-Party Generated Information from Social Media
My research focuses on … examine the usefulness of social media info in enhancing the effectiveness .Social media information however, its relevance to auditing remains underexplored.
Presented by:
Andrea M. Rozario
Ph.D. Candidate

2. Kara Stein – SEC Commissioner 2015
Introduction (1)
“Investors, and others, are accessing and analyzing massive amounts of information from sources, like social media, unimaginable just a few years ago. This new data may be empowering investors to make smarter investment decisions”
Kara Stein – SEC Commissioner 2015
So it’s of interest to examine how auditors canuse information from social media to make smarter audit decisions, and enhance AQ

3. Introduction (2) PCAOB AS 2110
Internal Financial
Internal Nonfinancial
Less reliable
Develop account expectation
External Financial
External Nonfinancial
More reliable but not timely!
Analytical Procedures
Compare account expectation to actual
AS 2110: Identifying and Assessing Risks of Material Misstatement
1 – introduce analytical procedures – what they are, and what steps auditors follow to perform them
2 – in this research study I focus on the first step
So it is also interesting to examine how auditors are performing this audit, specifically, analytical procedures, in this highly digitized society
Then and the now – for analytical procedures
THIS IS HOW AUDITORS TYPICALLY develop expectations….
BUT PROBLEM - THEY HAVE LIMITATIONS, INTERNAL INFO MAY BE TIMELY, but could be manipulated by management
EXTERNAL (external financial – analyst forecasts) INFO COULD BE MORE RELIABLE, BUT NOT AVAILABLE IN TIMELY MATTER
SOMETHING NEEDS TO CHANGE
Determine whether difference is significant
PCAOB AS 2110

4. Introduction (3) PCAOB AS 2110
Internal Financial
Internal Nonfinancial
Less reliable
Develop account expectation
External Financial
External Nonfinancial
More reliable but not timely!
Analytical Procedures
Compare account expectation to actual
External Nonfinancial Information from Social Media
So it is also interesting to examine how auditors are performing this audit, specifically, analytical procedures, in this highly digitized society
Then and the now – for analytical procedures
THIS IS HOW AUDITORS TYPICALLY develop expectations….
BUT PROBLEM - INTERNAL INFO MAY BE TIMELY, but could be manipulated by management
EXTERNAL (external financial – analyst forecasts) INFO COULD BE MORE RELIABLE, BUT NOT AVAILABLE IN TIMELY MATTER
SOMETHING NEEDS TO CHANGE
Determine whether difference is significant
PCAOB AS 2110

5. Objectives
Do Twitter proxies of consumer interest and consumer satisfaction enhance substantive analytical procedures for the revenue account?
Investigate whether information generated by third-parties on social media can
Improve the prediction performance of substantive analytical procedures
Improve the error detection performance of substantive analytical procedures

6. Motivation
Social media postings contain incremental information about firms’ stock market prices, and sales performance (e.g. Bollen, Mao, Zheng 2011; Tang 2017)
Inspection findings indicate that accounting firms fail to develop precise expectations (PCAOB 2007; PCAOB 2016a)
Social media consumer postings about firms’ products and brands could be used as a source of audit evidence

7. Findings Contribution
Auditors can benefit from incorporating social media information in continuous substantive analytical models
Model with lagged sales, TCI, and GDP outperforms other models
Contribution
Contributes to the auditing literature by investigating the relevance of social media information that is generated by third parties to auditing
-outperforms other models – including other Twitter models and the benchmark models
Contribution - The findings for prediction and error detection performance suggest that TCI has incremental value in the absence of contemporaneous firm information, or macroeconomic information, and that it can complement models with macroeconomic information

8. Research Questions
RQ 1A: For the revenue account, do traditional substantive analytical models that contain Twitter-based information produce more accurate predictions than traditional substantive analytical models that do not incorporate it?
RQ 1B: For the revenue account, do continuous substantive analytical models that contain Twitter-based information produce more accurate predictions than continuous substantive analytical models that do not incorporate it?
RQ 2A: For the revenue account, are traditional substantive analytical models that contain Twitter-based information better at detecting errors than traditional substantive analytical models that do not incorporate it?
RQ 2B: For the revenue account, are continuous substantive analytical models that contain Twitter-based information better at detecting errors than continuous substantive analytical models that do not incorporate it?
I differentiate between traditional/substantive
Generally, auditors use PY, but it is possible for more timely aps to produce more accurate predictions, so I want to test this expectation

9. Research Design (1) Sample – 24 B2C industries
Likefolio, and Compustat
Quarterly financial information is interpolated into monthly observations and matched with Twitter data

10. Research Design (2) Twitter Measures
Likefolio, provided customer interest and satisfaction for products and brands
Mapping of brands and products to the company
Customer Interest to Buy
TCI: total count of tweets related to the firm’s product or brand past/future interest to buy
TCS: ratio of positive tweets to total (positive and negative) tweets
I want to emphasize that this dataset is very uniquie
-Mappiing of thosuands of brands/products that belong to a company. Extract tweets at the product/brand level and they aggregate it at the firm level
Customer Sentiment

11. Research Design (3) Traditional SAPs Continuous SAPs (1) (5) (6) (2)
Models
Traditional SAPs
Continuous SAPs
(1)
(5)
(6)
(2)
(7)
(8)
(3)
(9)
(10)
(4)
(11)
(12)

12. Twitter Consumer Interest Twitter Consumer Satisfaction
Results (1)
Prediction Performance – 24 industries
Twitter Consumer Interest
Twitter Consumer Satisfaction
Model
(1) vs. (5)
(2) vs. (7)
(3) vs. (9)
(4) vs. (11)
(1) vs.
(6)
(2) vs.
(8)
(3) vs.
(10)
(4) vs.
(12)
Traditional - SAP
16 of 24
14 of 24
15 of 24
12 of 24
Continuous - SAP
19 of 24
21 of 24
18 of 24
22 of 24
20 of 24
Go across – traditional SAP, simple model is able to produce better expectations just as the more complex models. Continuous SAPs models 7 and 11 as well as model 12. You can see that compared to traditional, continuous SAP generate superior predictions, especially continuous SAPs that have TCI information.

13. Twitter Consumer Interest
Results (2)
Error Detection Performance – 24 industries
Twitter Consumer Interest
False Positive
False Negative
Model
(1) vs. (5)
(2) vs. (7)
(3) vs. (9)
(4) vs. (11)
Traditional - SAP
14 of 24
11 of
4 of
6 of
5 of
Continuous - SAP
16 of
15 of
10 of 24
9 of
8 of
Now we move on to error detection performance.
First we examine the results for TCS for FP and FN error rates and then we do the same for TCS. Ideally the model should be able to produce reasonably low FP and FN errors
For fn we see that the results are inferior to the benchmark models. Where the better performing models in this case are models 7 and 11 which are able to produce lower FN error for only 6 of the 24 industries
Now for Continuous SAPs error detection performance generally improves for FPs compared to TSAP. For FN error detection performance improves as well compared to TSAP, but it is still inferior to the benchmark models as the better performing model in this case is model 7, which is able to produce lower fn error for only 10 of the 24 industries that are analyzed.
Which comes at the cost of higher FN rates

14. Twitter Consumer Interest
Results (3)
Error Detection Performance Cost Ratio – 24 industries
Twitter Consumer Interest
1 to 1
1 to 2
Model
(1) vs. (5)
(2) vs. (7)
(3) vs. (9)
(4) vs. (11)
Traditional - SAP
9 of 24
4 of
10 of 24
11 of
10 of 24
8 of
Continuous - SAP
17 of 24
20 of 24
14 of 24
15 of 24
18 of 24 of
So now I analyze the ratio of the costs between the two types of errors as an additional criteria to determine the models that are more effective for error detection performance . I use a cost ratio of 1:1 and 1:2
TSAP – 7 is better as it is able to generate relatively low FP and FN errors for 11 industries
So we can see that for TCI model 7 is the better model when we take the ratio of the costs of the different types of errors into account (meaning that here the model sacrifices the least amount of FN to achieve lower FPs; produces relatively low FPs and FNs at these costs)

15. Conclusion
Used third-party generated Tweets of firms’ products and brands to provide insights into the usefulness of this information in enhancing substantive analytical procedures
Continuous substantive analytical procedures with Twitter information can benefit auditors, especially models with TCI
Limitations?
2 – that include lagged sales, or models with TCI that include lagged sales, and GDP

16. Kara Stein – SEC Commissioner 2015
Hmm… What is the next disruptive technology?
How to apply it to auditing?
Would it be appealing to the regulators?
My professors?
Introduction (1)
“Investors, and others, are accessing and analyzing massive amounts of information from sources, like social media, unimaginable just a few years ago. This new data may be empowering investors to make smarter investment decisions”
Kara Stein – SEC Commissioner 2015
Me and Abby giving auditor face…so we are thinking

17. Thank you!
Would love any suggestions/feedback