1. GS/PPAL 6200 3.00 Research Methods and Information Systems
February 14 & 15, 2017
Professor Brenda Spotton Visano
Office: 130 McLaughlin
Voice Mail: (416) ext

2. Agenda
Other Data Sources: where to find them, how to assess their usability
Articles to be discussed:
Bevan, S., Baumgartner, F. R., Johnson, E. W., & McCarthy, J. D. (2013). Understanding selection bias, time-lags and measurement bias in secondary data sources: Putting the Encyclopedia of Associations database in broader context. Social science research, 42(6), Available at
Oulasvirta, L. (2014). The reluctance of a developed country to choose International Public Sector Accounting Standards of the IFAC. A critical case study. Critical Perspectives on Accounting, 25(3), (Available via York University Libraries)

3. Other Sources of Data Focus: Data not collected by researcher’s survey
Includes: personal documents, mass media outputs, institutional documents, data collected by other researchers or by statistical agencies (i.e., “secondary” data)

4. Scott’s Criteria Choosing other data sources:
Are the data/documents authentic? Is the origin known/confirmed?
Are the data/documents credible? Do they represent what they say they represent?
Are the data/documents representative? If they are supposed to be emblematic, are they?
Is the meaning of the data/documents clear?

5. Catalogues of Secondary Data
Data Catalogues and Resources
York University Libraries
→ Research & Learn → Research Guides → Finding Types of Information: Data & Statistics
Finding existing data for a specific concept
→ Research Data Centre → <odesi>
Odesi: A web-based data exploration, extraction and analysis tool. It provides researchers the ability to search for survey questions (variables) across hundreds of datasets, including Statistics Canada master and public-use microdata files.
Need help? Contact Walter Giesbrecht, Data librarian at Scott Library

6. Integrity of Secondary Data and Appropriateness for Use
Understanding the Source
Who produced the data/document? What is the public reputation of the producer?
What are the primary operating objectives of the producer of the data?
Why were the data produced? What was the original purpose for which the data were collected? Who benefits and how?

7. Integrity of Secondary Data and Appropriateness for Use (cont’d)
Understanding the Data Reported
What were the operational definitions of key concepts?
What was the intended population coverage (who, when)?
What was the rule to decide inclusion/exclusion?
Is the filter for deciding inclusion/exclusion consistent over time?
Who interprets for whom and who represents whom?

8. Secondary Data Issues Secondary Data Issues
Criteria for Evaluating Research
Secondary Data Issues
Reliability: are the results stable over time and consistent over researcher?
Measurement or Construct Validity: does the indicator reflect well the concept we intended to measure?
Internal Validity: is there a direct causal relationship between dependent and independent variables?
External Validity: Can we generalize the results?
Bias from changes in official definitions and policies over time
Selection biases due to filtering inclusions/ exclusions and reporting lags
Measurement bias from interview effects, question order, respondent interpretations and coding effects

9. Resolutions Understand sources and structure of potential biases
Is there a second set of data of similar coverage with which you could compare data accuracy?
Are biases consistent over sampling? How will the biases affect results? Either correct for bias or note the bias in your write up and explain possible effect on results.

10. Questions for Reviewing Bevan et al.
Overview
Identify the main sections of the paper
What motivates the study?
What are the authors’ research questions?
What is (are) the authors’ conclusion(s)?
Content Details
What are their examples of Selection bias? Sampling bias? Measurement bias? What are the sources? What are their resolutions?

11. Questions for Reviewing Oulasvirta
Overview
Identify the main sections of the paper
What motivates the study?
What is the author’s research question?
What is the author’s research method?
What is (are) the authors’ conclusion(s)?
Content Details
Is accounting an objective factual exercise or an interpretive exercise?
What are the barriers identified to adopting the international accounting standards (in Finland)?

12. Technical Notes for Oulasvirta
Cash basis accounting = report income and expenses as payment received/made
Accrual basis accounting = consistent treatment of economic events; accounts for outstanding commitments; more accurate measure of net income
For more information: IMF Technical Notes: Implementing Accrual Accounting in the Public Sector by J. Cavanagh, S. Flynn, and D. Moretti (2016)