1. Statistics without surveys 30 th January 2009

2. Statistics without surveys There are methods of quantitative analysis that do not rely on surveys. Three that we will discuss are: –Content analysis –Analysis of historical materials –Observation studies All involve the operationalisation of concepts and coding of data, as well as decisions about sampling and so none are immune from criticisms aimed at these processes, and the subjectivity involved therein. But since all three largely involve unobtrusive methods, they tend not to involve the (artificial, potentially power-laden, and much criticised) interactions found in survey interviews. We will conclude by looking briefly at network analysis. This is actually an alternate method of statistical analysis, but one that has been developed in relative isolation to mainstream statistics, and one that has different starting assumptions and utilizes different sorts of data sets.

3. Content Analysis Method of transforming symbolic content of a document (such as words or images) from a qualitative unsystematic form into a quantitative systematic form.

4. Possible Units of Analysis for Content Analysis …but a unit of analysis may also be: a film, a scene, a TV episode, a wall (containing graffiti), a rubbish bin, a politician’s speech, a web-site, or a blog posting…

5. Sampling in Content Analysis You can use random sampling methods in content analysis just like you can in surveying people. For example: You are interested in whether politicians have become increasingly or decreasingly respectful of their opponents over the last decade. And you decide to study speeches made in Parliament. You could study every speech made over the last ten years. This would capture the entire relevant population. However if you did not have the resources to read every speech… You could use multistage cluster sampling and randomly select a month (say February) and then randomly select days in that month (say 13 th, 16 th, 23 rd and 27 th ), and then study every speech given on those days (or the next weekday after these days if they fell on a weekend) in each year of the ten years in the study.

6. Manifest vs. Latent Content Manifest Content The objective, surface, or concrete content Examples: -Number of times the word homosexual appears in the newspaper -Number of times someone drinks alcohol in a TV show -Number of pictures of women in a text-book Latent Content The underlying or implicit meanings Examples: -How approvingly or disapprovingly homosexual behavior is mentioned in a newspaper -How intoxicated someone becomes after drinking alcohol on a TV show -Whether women are performing masculine of feminine tasks when they are pictured in a text-book

7. Coding in Content Analysis Categorizing raw data (behaviors or elements) into a limited number of standardized categories, suitable for analysis You need to develop your own coding scheme that: may be based on existing coding schemes or on your orienting theories may emerge (inductively) from looking at the data The coding scheme must have categories that are: 1. exhaustive 2. mutually exclusive 3. theoretically relevant

8. Counting and Record Keeping 1.Your coding must be numerical in order to analyze it statistically (even where you are coding latent content) 2.Record keeping should distinguish units of analysis and of observation (i.e. a particular newspaper versus an editorial in that newspaper) 3.Remember to record the base number (total). n.b. this issue is resolved if every observation is recorded and coded)

9. Coding Text: Leviticus

10. Sample Tally Sheet (partial) Note: this is a tally sheet that takes the newspaper to be the unit of analysis. If each editorial was the unit of analysis there would be a line for each editorial (rather than for each newspaper).

11. Sample table describing findings

12. Strengths of Content Analysis Economy of time and money. Easy to repeat a portion of the study if necessary. Permits study of processes over time. Researcher seldom has any effect on the subject being studied. Reliability – consistent results over time (especially with manifest content).

13. Weaknesses of Content Analysis Limited to the examination of recorded communications. Problems of validity are likely – are the sources meaningful measures of what we want to measure? – for example: is what appears in the media a good representation of political currents? Are blogs a good representation of the population’s opinions? Latent measures inevitably involve subjective interpretation.

14. Comparative-Historical Research Much comparative historical research does not use statistics. However if you are looking at change over time or are comparing different countries or regions there are a large number of statistics that can be used: Macro Secondary statistics – e.g. World Bank “development indicators” i.e. mortalitity rates; televisions per 1000 population; Literacy rates. Or “OECD Main Economic Indicators” – i.e. foreign direct investment; GDP; GNP… etc. Primary statistics – these are datasets that you construct for yourself from historical and comparative research. They may document anything from the strength and political composition of particular trade unions in a particular time and place; to land- holding patterns in different regions as described by local tax- records; to speeches made by Vice-Chancellors of UK universities at public forums over the last century… To conduct quantitative analysis of primary historical research it just needs to be systematically coded.

15. Sampling Comparative- Historical Events If you are going to use comparative-historical data to create a dataset it is important to think about whether you have data from the entire population of events that you are interested in (i.e. every strike that occurred in the UK between 1990 and 2000), or whether you are focusing on a subset (thirty strikes that occurred in the UK between 1990 and 2000). If you present statistical information for a subset of events you are sampling and the same issues of occur as any other time that you sample data: your findings are only statistically generalisable if the sampling is random (or if each event has an equal probability of selection into the subset). On the other hand, there are often substantive reasons to choose specific “important” events to be part of your subset (i.e. large- scale strikes that involved media campaigns). This is legitimate and statistics gleaned from these may be interesting and informative. However they are not statistically generalisable to all events (i.e. strikes generally) and so inferential statistics are not appropriate.

16. Observation studies Observation is not just the preserve of qualitative methods. Quantitative methods can be applied where structured or systematic observation is carried out. Like qualitative observation studies (and surveys), this involves cross-sectional data (we can only observe the present). Unlike qualitative observation, structured or systematic observation is not inductive but requires the prior determination of what to observe (although this may be suggested by initial unstructured observations).

17. The observation schedule To produce quantitative data an observation schedule or coding scheme is required. This describes what is to be observed and how what is observed should be coded. For example, if I were observing in the Library Café and was interested in interactions between students and the staff working at the cash-registers I could code each student’s behaviour in the following way: 1.No conversation, no eye contact, no smile 2.Eye contact and/or smile, no conversation 3.Conversation, only as required by the transaction 4.Conversation as required by the transaction and polite thanks. 5.Conversation that goes beyond transaction and polite thanks.

18. The observations must be focused – and relevant to the research question The schedule (like closed questions in a questionnaire) should have categories that are mutually exclusive and exhaustive Recording should involve as little observer interpretation as possible – this is where reliability is diminished. The observation schedule

19. Sampling in Structured Observations It is important to be clear about the unit of analysis – are you sampling events/situations, interactions, or individuals? Sampling must consider the dimension of time in determining who, where, and when to make observations. It may sometimes be appropriate to sample at multiple time periods and in multiple sites.

20. Benefits and Drawbacks of Structured Observation Like other ‘unobtrusive measures’ structured observation may avoid researcher contamination – enabling the study of people in their natural environment. Unlike surveys it does not depend on the negotiation of meaning between interviewer and interviewee (or the interviewee’s accurate representation of her behaviour). Unlike qualitative observation studies it can produce relatively reliable data and since observation (with a schedule) can be undertaken by more than one researcher, it enables large-scale data collection. However the researcher will only ‘see’ the predetermined categories of action that the schedule specifies. These may not be the categories of action that are relevant to participants. Since structured observation precludes questioning participants about their motives or opinions, it is wholly dependent on observing behaviour and on the ability of the researcher to appropriately assess this. It is ahistorical, in that it can only assess behaviour in the moment (unlike surveys which can ask, albeit imperfectly, about people’s pasts, or other methods such as content analysis, historical or secondary data analysis).

21. Network Analysis Network Analysis is based on the assumption that people’s actions are interdependent and so it is critical to describe the networks of relationships that exist. It is characterized by a distinctive methodology encompassing techniques for collecting data, statistical analysis, visual representation, etc Critically, network analysis uses Matrices to analyse the relationships between people, organisations and institutions. It also uses graph theory.

22. Network analysis is concerned with attributes of pairs of individuals, of which binary relations are the main kind. Some examples of dyadic attributes: Kinship: brother of, father of Social Roles: boss of, teacher of, friend of Affective: likes, respects, hates Cognitive: knows, views as similar Actions: talks to, has lunch with, attacks Flows: number of cars moving between Distance: number of miles between Co-occurrence: is in the same club as, has the same colour hair as Mathematical: is two links removed from

23. The relationships needn’t be between individuals Ties could be… between corporations, or between political organizations, or between community groups, or any combination of these.

24. Divided We Stand Political books were selected from the New York Times Bestseller List as starting points for 'snowball sampling'. Two books are linked in the network if they were purchased by the same person -- "Customers who bought this book also bought:". The pattern reveals two distinct clusters with dense internal ties. (early 2004) Are these two clusters connected by non-political books? In the map there is a path of 4 steps from the most central Blue book to the most central Red book. Using current fiction titles we do not find a shorter path! Using Da Vinci Code the centers of the clusters are 7 degrees/steps apart, The Five People You Meet in Heaven and South Beach Diet result is 9 degrees apart and The Last Juror takes over 15 steps to connect the centers.

25. Flow of Gas Into Europe Flow of Gas Into Europe – helps to understand the recent conflict between Russia and Ukraine. Co-Authorship Map of Social Network ScholarsCo-Authorship Map of Social Network Scholars – sheds light on academic cooporation and idea-exchange. Other Examples from orgnet.com

26. Weaknesses in Network Analysis It is difficult to get information on complete networks (as this involves getting information from all individuals/organizations). This is required for many of the methods involved. Network analysis has been criticised for being better at analysing relationships between people (or ‘nodes’) than the structural and material aspects of power.

27. Network analysis games… Network analysis has been used to develop six degrees of Kevin Bacon (the parlour game developed from the notion that everyone is separated from everyone else by just six degrees of separation). The aim in Six Degrees of Kevin Bacon is to link any movie star to Kevin Bacon via films that they have both been in in less than six steps. Can you think of anyone who is more than three degrees of separation from Kevin Bacon? You can check your answer at: http://oracleofbacon.org (also linked via the link page on the module website). This site also allows you to link any other stars together.http://oracleofbacon.org