ANALYZING QUALITATIVE DATA Assoc. Prof. Dr. Şehnaz Şahinkarakaş

BASIC STEPS I. Prepare the data (Transcription) II. Develop codes / categories (content analysis) III. Complete and revise the system of categories based on the data IV. Report the result of data analysis

PREPARING THE DATA (TRANSCRIBING) Transcribing data is very important when your study is mostly based on interviews Transcription is time-consuming and demanding Try to transcribe your data soon after the interview, when it is still fresh in your mind.

Points to consider for transcription Begin the transcript with the name of the participants and a pseudonym you have given to him/her Transcribe all the interview carefully and completely (words are the data of the qualitative study!) It’s possible to ask someone else to transcribe the data for you, but if you prepare yourself, you will be familiar with it.

You may use some conventions such as: Brackets [ ]: your addition to transcript Question mark ?: interviewer’s words couldn’t be deciphered Three periods …: there is a pause (or missing data) Italics or bold: emphatic stress

Types of transcription Simple transcription “Jeffersonian/Jefferson” transcription Summarising protocol

Simple Transcription IR: So today is March seventh. I’m at Annals of Internal Medicine and I’ll be interviewing Dr. P.E. And for the record can you state your name and position? DrE: It’s P.E. I’m deputy editor of Annals of Internal Medicine. IR: Okay. And how long have you been working at Annals? DrE: Since 1978. It’s a long time. I was associate editor from 1978 to 1999 and I’ve been deputy editor from 1999 to 2003.

“Jeffersonian” transcript IR= So: today is March seventh, I’m at Annals of Internal Medicine and I’ll be interviewing doctor Pete Ernest. (0.4) A::nd um for the record can you state your name and position? IE= It’s Pete Ernest, I’m deputy editor of Annals of Internal Medicine. IR= Okay. And how long have you been working at Annals? (0.4) IE= Since nineteen seventy eight. It’s a lo::ng time. I was uh:: associate editor from nineteen seventy eight to nineteen ninety ni:ne, …

Summarising Protocol Contains only the important contents of the interview. Important passages can be verbatim transcripts E.g. Dr. E: I’m deputy editor of Annals of Internal Medicine. I was associate editor from 1978 to 1999, and I was deputy editor from 1999 to 2003. My sub-specialty is pulmonary disease which I practice every day at the University of Pennsylvania.

Developing Codes/Categories (Content Analysis) Content analysis is used to determine the presence of certain words, concepts, themes, phrases, characters, or sentences within texts or sets of texts and to quantify this presence in an objective manner. How can this be done? Making sense out of the text data Dividing the text into segments Labelling the segments with codes Examining codes (for overlap and redundancy) Putting them in broad themes

Getting Ready to Code Mainly two ways of categorization: 1. A Priori Coding (Relational Analysis) : Before the analysis begins, the researcher determines the categories which are based on previous knowledge, experience, literature, or theory (i.e. Develops a coding scheme) These can be identified from a range of sources: Previous research or theory Research or evaluation questions you are addressing Questions and topics from your interview schedule PS. Conduct a pre-test to the coding scheme to see if it is working.

A Sample Coding Scheme IRL Teacher starts a learning episode by asking questions relevant to the learning process IS Teacher starts a learning episode by assigning a learner to answer a question or answer the next exercise. RNS Not successful response by the learner. RS Successful response by the learner. CA Learners use L1 to answer a question or to say something in L1 irrelevant to learning. CC Teacher uses L1 to reaffirm learner’s answers. CES Teacher uses L1 to explain/simplify instructions, questions or exercises. CP Teacher uses L1 to prompt/encourage learners to participate in the lesson. FDE Teacher provides learners with descriptive feedback FDEB Teacher provides boys with descriptive feedback FDEG Teacher provides girls with descriptive feedback FEVL Teacher provides learners with evaluative feedback RAC Teacher repeats the answer to confirm FEVB Teacher provides boys with evaluative feedback FEVG Teacher provides girls with evaluative feedback FE Teacher provides learners with explicit feedback FEB Teacher provides boys with explicit feedback FEG Teacher provides girls with explicit feedback FI Teacher provides learners with implicit feedback FIB Teacher provides boys with implicit feedback FIG

A sample qualitative data (classroom observation) T: I m going to give you some words now. Words we use in present perfect simple. What kind of words do we use? Katerina K: Since, ehh already. T: thank you. Rita? R: just, ever T: just, ever. Rafaella? R: yet, often T: Sasha? S: ever T: ever. Nichola? N: before. T: not really.

T: I m going to give you some words now T: I m going to give you some words now. Words we use in present perfect simple. What kind of words do we use? Katerina K: Since, ehh already. T: thank you. Rita? R: just, ever T: just, ever. Rafaella? R: yet, often T: Sasha? S: ever T: ever. Nichola? N: before. T: not really.

A sample coding for the scheme T: I m going to give you some words now. Words we use in present perfect simple. What kind of words do we use? Katerina [IS] Teacher starts a learning episode by assigning a learner to answer a question or answer the next exercise. K: Since, ehh already. [RS] Successful response by the learner. T: thank you. Rita? [RAC] Teacher repeats the answer to confirm R: just, ever [RS] Successful response by the learner. T: just, ever. Rafaella? [RAC] Teacher repeats the answer to confirm R: yet, often [RS] Successful response by the learner. T: Sasha? S: ever [RS] Successful response by the learner. T: ever. Nichola? [RAC] Teacher repeats the answer to confirm N: before. [RNS] Not successful response by the learner. T: not really. [FEVL] Teacher provides learners with evaluative feedback

A sample coding for the scheme T: I m going to give you some words now. Words we use in present perfect simple. What kind of words do we use? Katerina [IS] K: Since, ehh already. [RS] T: thank you. Rita? [RAC] R: just, ever [RS] T: just, ever. Rafaella? [RAC] R: yet, often [RS] T: Sasha? S: ever [RS] T: ever. Nichola? [RAC] N: before. [RNS] T: not really. [FEVL]

2. Emergent Coding (Grounded Coding--Conceptual Analysis): Categories emerge as the analysis continues. Good categories are Exhaustive (all relevant content can be categorized) Mutually exclusive (data can be placed in only one category) Independent (assignment of data to one category does not affect the assignment of other data to a category)

Coding the Material (Analyzing the Data) If you are using a priori coding, analyze the data according to the codes you have already identified. If you are using grounded/emergent coding, there are some steps to follow

Steps for Emergent Coding (Creswell, 2012) 1. Get a sense of the whole: read the transcripts and take notes in the margins 2. Take one document (e.g. one interview); go through it and consider the underlying meaning. 3. Start coding the document: identify text segments (word/sentence/paragraph for a code) and assign a code word for the segment (In first coding, don’t use lots of different codes; try to assign only a few—you can separate them later)

4. Make a list of all code words 4. Make a list of all code words. Group similar codes and check redundancies (reduce if necessary) 5. Check the data with this list (grouped and reduced) and check whether new codes emerge. Circle the quotes that support the codes 6. Reduce the list of codes to get 5 - 7 themes (categories).

Sample Coding 1

Sample Coding 2

Reliability & Validity The primary reason for content analysis is to collect objective information, so reliability is very important and it means the degree to which multiple coders yield identical results. In general there are two stages in determining coding reliability. Ensure that the classification system is understood and applied to each unit separately. Compare how multiple coders classify a sample of the same material. (80% of agreement is good enough)

Main Advantages of Content Analysis 1. Quantifies largely qualitative information 2. Adds qualitative richness to otherwise quantitative data 3. Helps researchers learn more about issues of interest.

Disadvantages of Content Analysis 1. Usually limited to recorded information 2. Might have validity problems Analysts can achieve acceptable agreements (reliability) but the true meanings of categories might still be a problem. 3. Be challenged as too subjective.

Exercise for coding!

1. Decide which code sums up what is being told in each line (A Priori) List of Codes: breaking up getting pregnant insecurity lifestyle settling down not wanting to move uncertainty moving in together proposal 1. When I found out I was pregnant, I wasn’t sure if I wanted to get married and he … 2. wasn’t the settling down kind. He was old enough to bring up a child but I knew he … 3. wasn’t ready to. He was in the Navy he liked the life and preferred going off with … 4. his friends and that bothered me. At first I hoped something would happen so I … 5. didn’t have the baby and I wanted him to marry me ’cause he wanted to not … 6. because I was pregnant. Anyway when the baby was born we broke up and I have … 7. seen him a couple of times but he has phoned lots and says he will marry me. He … 8. wanted me to marry him and go and live with him, but I didn’t want to leave home. 9. He leaves the Navy in 6 months so I’m getting the flat ready for him to move in. But … 10.I still worry he’ll go off with his friends and won’t be able to give up the life.

Original coding for the first 3 lines(exercise 1) 1. When I found out I was pregnant, I wasn’t sure if I wanted to get married and he … (getting pregnant) 2. wasn’t the settling down kind. He was old enough to bring up a child but I knew he … (settling down) 3. wasn’t ready to. He was in the Navy he liked the life and preferred going off with … (lifestyle) 4. his friends and that bothered me. At first I hoped something would happen so I … 5. didn’t have the baby and I wanted him to marry me ’cause he wanted to not … 6. because I was pregnant. Anyway when the baby was born we broke up and I have … 7. seen him a couple of times but he has phoned lots and says he will marry me. He … 8. wanted me to marry him and go and live with him, but I didn’t want to leave home. 9. He leaves the Navy in 6 months so I’m getting the flat ready for him to move in. But … 10. I still worry he’ll go off with his friends and won’t be able to give up the life.

Original coding for all the lines(exercise 1) 1. When I found out I was pregnant, I wasn’t sure if I wanted to get married and he … (getting pregnant) 2. wasn’t the settling down kind. He was old enough to bring up a child but I knew he … (settling down) 3. wasn’t ready to. He was in the Navy he liked the life and preferred going off with … (lifestyle) 4. his friends and that bothered me. At first I hoped something would happen so I … (uncertaintly) 5. didn’t have the baby and I wanted him to marry me ’cause he wanted to not … (insecurity) 6. because I was pregnant. Anyway when the baby was born we broke up and I have … (breaking up) 7. seen him a couple of times but he has phoned lots and says he will marry me. He … (proposal) 8. wanted me to marry him and go and live with him, but I didn’t want to leave home. (not wanting to move) 9. He leaves the Navy in 6 months so I’m getting the flat ready for him to move in. But … (moving in together) 10. I still worry he’ll go off with his friends and won’t be able to give up the life.

2. Write a code that summarizes what happens in each line (Grounded) 1. It was challenging, after living in Italy for 6 months and then I moved home before … 2. I started university. I was used to doing things my own way when it suited me … 3. and not having to tell people where I was going. I was living with friends and they … 4. didn’t care what I did or where I went. It was really hard to go back to sort of … 5. thinking of others … ’cause Mum and Dad wanted to know where I was going and … 6. who with, which was a nightmare. My parents were strict but I had a lot of … 7. freedom growing up, as long as I didn’t overstep the boundaries. After I came … 8. back from Italy they realised I was more independent and things changed and … 9. they didn’t try and stop me doing things anymore but they would still let know if … 10. they didn’t approve.

Original Coding (exercise 2) 1. It was challenging, after living in Italy for 6 months and then I moved home before … moving away 2. I started university. I was used to doing things my own way when it suited me … independence 3. and not having to tell people where I was going. I was living with friends and they … freedom 4. didn’t care what I did or where I went. It was really hard to go back to sort of … moving back home 5. thinking of others … ’cause Mum and Dad wanted to know where I was going and … control 6. who with, which was a nightmare. My parents were strict but I had a lot of … 7. freedom growing up, as long as I didn’t overstep the boundaries. After I came … boundaries 8. back from Italy they realised I was more independent and things changed and … growing up 9. they didn’t try and stop me doing things anymore but they would still let know if … letting go 10. they didn’t approve. disapproval

What themes (main categories) can you generate? 1. It was challenging, after living in Italy for 6 months and then I moved home before … moving away 2. I started university. I was used to doing things my own way when it suited me … independence 3. and not having to tell people where I was going. I was living with friends and they … freedom 4. didn’t care what I did or where I went. It was really hard to go back to sort of … moving back home 5. thinking of others … ’cause Mum and Dad wanted to know where I was going and … control 6. who with, which was a nightmare. My parents were strict but I had a lot of … 7. freedom growing up, as long as I didn’t overstep the boundaries. After I came … boundaries 8. back from Italy they realised I was more independent and things changed and … growing up 9. they didn’t try and stop me doing things anymore but they would still let know if … letting go 10. they didn’t approve. disapproval

3. Code the following data and develop themes/categories We were playing for our title. (the injury) It was pretty bad I got a concussion and I hurt my jaw it was probably one of the worst concussions I have had so far This is my thirteenth year (of playing this sport). I’m on a high school varsity team I also play on another like semi-competitive team everyone kind of just goes for the ball it wasn’t really a foul or anything I don’t really remember it people say it was like my own team mate that kicked me.(smiling) but I don’t really blame her, she was just trying to stop the goal I was really frustrated

A Sample Coding (exercise 3) Raw data themes Higher order sub-themes (the injury) It was pretty bad I got a concussion and I hurt my jaw it was probably one of the worst concussions I have had so far experience of her injury This is my thirteenth year (of playing this sport). I’m on a high school varsity team I also play on another like semicompetitive team experience of playing football Everyone kind of just goes for the ball It wasn’t really a foul or anything I don’t really remember it people say it was like my own team mate that kicked me.(smiling) but I don’t really blame her, she was just trying to stop the goal feelings about the accident

4. Following transcripts are taken from an interview with 3 people. Read the responses of the interviewees and develop the common themes/categories that come up.

4. Interview Question: How would you assess the level of automation in your office? 21 Overall, the system is pretty good. The information we have access to is very helpful. But, updates to the data base are batch processed. If the system allowed real-time updates this would give case workers more up-to-date information. 22 The terminals hurt my eyes. They are hard to look at for long periods of time. The system is very slow retrieving information, and much of the information we get is miscoded or incorrect. Terminal time is also limited because of frequent down time for repairs. I can’t say the current system helps me much in doing my job. 23 There is a serious lack of printers. Case workers need to be able to print out information on our terminals, but this is time consuming because we all have to send output to one printer. Lots of days the printer is offline for repairs which complicates the situation