1. Managing a Research Project: Analytical Methods and Tools
Patricia Ensworth
Harborlight Management Services LLC

2. Etic and emic approaches
Researcher’s interpretation of participant’s experience
Emic
Participant’s interpretation of own experience

3. Types of research Strategic Exploratory Forensic
Making a yes/no decision
Choosing among several alternative solutions
Exploratory
Entering a new market
Creating a new product, service, or system
Developing new procedures
Forensic
Determining the cause of a problem
Improving performance

4. Recruitment methods Recommendations Volunteers Screening service
Management
Experts
Salespeople
Customer service staff
Volunteers
Screening service
Snowball sample
Data mining

5. Data collection methods
Archive review
Literature review
Vendor review
Data analytics
Questionnaire
Unstructured interview
Structured interview
Observation
Participant observation
Diary study
Survey
Brainstorming session
Focus group
Usability test

6. Data collection methods
Process
Goal
Archive review
Search organization’s internal records for metrics, issue logs, lessons learned, system diagrams.
Explore history and evolution of issues, identify SMEs, determine current knowledge.
Literature review
Search external printed and on-line books, articles, and documents for relevant material.
Same as for archive review, placing organization in context with comparable enterprises.
Vendor review
Examine RFPs, contracts, and service level agreements. Review communications between vendor and client. Inspect service logs.
Identify patterns of issues and understand vendor’s perspective.
Data analytics
Develop queries and algorithms to review large quantities of electronic data.
Identify patterns of issues and behaviors.
Questionnaire
Create a standard list of questions to be asked by an interviewer.
Ensure that all interviewees are asked the same questions.
Unstructured interview
Ask interviewee general open-ended questions and allow the conversation to develop spontaneously in directions related to the central topic.
Explore background issues, understand interviewee’s perspective, generate ideas.
Structured interview
Ask interviewee specific closed-ended questions and ensure the conversation stays on topic.
Gather data for making decisions.

7. Data collection methods (cont’d)
Process
Goal
Observation
Watch a person or group of people engaging in the behaviors under study.
Identify actual behaviors in contrast to reported behaviors.
Participant observation
Accompany a person or group of people engaging in the behaviors under study and learn to perform the behaviors as an apprentice or assistant.
Understand the lived experience of the person or group of people.
Diary study
Provide participant with a form containing questions about the attitudes and behaviors under study and ask him/her to fill it out multiple times on a regular basis.
Explore the changes in the behaviors and attitudes over time.
Survey
Send participant a form containing questions about the attitudes and behaviors under study and ask him/her to fill it out once.
Gather similar types of data from multiple participants for comparison, categorization, and statistical computation.
Brainstorming
Assemble a group of people who have an interest in a topic, ask for their ideas, and record all their responses without giving them criticism or analysis.
Explore a broad range of perspectives.
Focus group
Assemble a group of people who have an interest in a topic, lead them in a structured discussion about specific questions, and record all their responses without giving them criticism or analysis.
Obtain insights for making decisions.
Usability test
Provide participant with an item under study, ask him/her to use it in specific ways, and record all their behaviors and responses without giving them criticism or analysis.
Verify the viability and functionality of a design.

8. Data collection activities
Demo
Walkthrough
Task analysis
Card sorting
Affinity grouping
Model building
Collaging
Mind mapping
Location mapping
Movement mapping
Social network analysis
Information network analysis

9. Data collection activities
Activity
Process
Goal
Demo
Participant shows how to use an item.
Explore participant’s understanding of and relationship with the item.
Walkthrough
Participant show how to navigate a location or process.
Explore participant’s understanding of and relationship with the location or process.
Task analysis
Participant explains how to perform a sequence of actions to accomplish a goal.
Explore behavior range of universals, specialties, alter-natives, and peculiarities. Establish dependencies between tasks, define roles and responsibilities, identify workarounds.
Card sorting
Participant reviews a group of index cards that already have one idea written on each card, and sorts them into piles according to categories he/she creates. Cards can also be used for task analysis.
Explore alternative methods of organizing a group’s standard information and activities.
Affinity grouping
Participant is presented with a topic and responds with ideas he/she associates with that topic. Participant records each idea on a separate card, then sorts them into piles according to categories he/she creates.
Explore alternative methods of organizing an individual’s personal information and activities.
Model building
Participant creates a three-dimensional object that represents the topic under study.
Explore alternative visions and interpretations of the topic.
Collaging
Participant creates a collage that represents the topic under study.
Explore alternative attitudes and experiences of the topic.

10. Data collection activities (cont’d)
Activity
Process
Goal
Mind mapping
Participant is presented with a topic and responds with ideas he/she associates with that topic. Participant focuses on each first-level idea and responds with second-level ideas he/she associates with the topic. The process continues for several rounds. A map is drawn showing the initial topic at the center and the subsidiary ideas as radiating branches.
Explore alternative cognitive structures, imagined relation-ships, and specialized language usages.
Location mapping
Participant draws a map of a place.
Explore alternative views of important sites and relative distances.
Movement mapping
Participant is given a map of a location and draws lines showing habitual travel patterns. Line weight varies according to the frequency, style according to the method, and color according to the reason.
Explore environmental constraints, space layout, and resource usage.
Social network analysis
Participant draws a diagram showing people with whom he/she interacts. Line weight varies according to the frequency, style according to the method, and color according to the reason.
Explore community structure, collaboration methods, and technology trends.
Information network analysis
Participant draws a diagram showing the flow of information as inputs and outputs through the systems necessary to perform his/her work.
Explore current system usage in contrast to original design, identify workarounds, discover manual processes.

11. Research plan example Archive review Literature review Vendor review
Data analysis
Questionnaire
Unstructured interview
Observation
Structured interview
Diary study
Survey
Focus group
Usability test

12. Observation goals Activities Environments Interactions Objects Users
Tools employed
Sequence of actions
Types of interactions
Methods of organization
Workarounds
Source: Observing the User Experience

13. Behavior variation
Source: Ralph Linton, The Study of Man

14. Diary study

15. Survey – demographic info

16. Survey – single vs. multiple response

17. Survey – numerical answers

18. Survey - ranking

19. Survey - scales

20. Data analysis tools - quantitative
Pie chart
Bar graph
Pareto chart
Histogram
Distribution curve
Run chart
Comparison table
Cross-tabulation table
Scatter diagram
Spider/Radar chart

21. Data analysis tools - quantitative
Process
Goal
Comparison table
For multiple categories and numerical data elements that do not need to be counted or calculated, create a table with the elements as columns and the categories as rows.
Determine the relative quantities for the elements.
Cross-tabulation table
For multiple categories and data elements that need to be counted or calculated, create a pivot table with the elements as columns and the categories as rows.
Histogram
For one numerical data element, create a series of equal ranges, and sort the responses into the ranges. Graph the height of each range on the x-axis and the number of responses on the y-axis.
Determine the relative number of responses in each range.
Distribution curve
For one numerical data element, create a histogram. Calculate the mean and superimpose a bell curve of normal distribution on the histogram.
Determine the unique pattern of the responses.
Run chart
For one numerical data element that has been recorded on multiple occasions over time, graph the time sequence intervals on the x-axis and the measurements on the y-axis.
Determine the range of variation to establish acceptable values.
Bar graph
For one textual data element, sort the responses into categories. Count the items in each category. Create a vertical or horizontal scale showing the range of totals as bars of different heights.
Determine the relative sizes of the categories.

22. Data analysis tools – quantitative (cont’d)
Process
Goal
Pie chart
For one textual data element, sort the responses into categories. Count the items in each category. Create a circle and divide it into segments showing the range of totals as segments of different sizes.
Determine the relative sizes of the categories.
Pareto chart
For one textual data element, create a vertical bar graph with the categories arranged in order of size from left to right. Show the number of responses in a scale on the left and the percentages from zero to 100 as a scale on the right.
Determine the relative importance of the categories.
Scatter diagram
For two numerical data elements, create a graph with a range for one on the x-axis and the other on the y-axis. Plot the two elements as a point where the values intersect.
Determine the strength of correlation between the two data elements.
Spider/Radar chart
For multiple numerical data elements and multiple textual data elements, create a circular graph with zero at the center and the largest numerical quantity at the circumference. Sort the textual data elements into categories. Arrange the categories around the circle and draw a line connecting the zero midpoint to each category at the circumference. Draw a scale for the numerical data elements in the style of a target. Plot the numbers for each category on the line where it intersects the scale. Connect the data points.
Determine the similarities and differences in the numerical data between the categories.

23. Pie chart

24. Bar graph

25. Pareto chart

26. Histogram

27. Bar graph vs. Histogram

28. Data distribution curve

29. Run chart

30. Comparison table

31. Cross-tabulation table

32. Scatter diagram of correlation

33. Spider / Radar chart

34. Data analysis tools - qualitative
Comparison table
T-chart
Mind map
Activity network diagram
Force field analysis
Fishbone chart
Storyboard
Personas
Flowchart with swimlanes
Procedure analysis

35. Data analysis tools - qualitative
Process
Goal
Comparison chart
Method:
All
Activity:
Artifact:
Chart with data elements as columns and categories as rows.
Understand the relative value of the data elements.
T-chart
Table comparing two aspects of one data element.
Understand the relative value of the aspects.

36. Data analysis tools – qualitative (cont’d)
Process
Goal
Mind map
Method:
Structured interview
Activity:
Mind mapping
Artifact:
Visual representation of associations.
Understand alternative cognitive structures, imagined relationships, and specialized language usages.
Activity network diagram
Interview, observation, participant observation
Demo, walkthrough, task analysis
Flowchart representing the normal logical sequence of activities without consideration of resources available or the participation of specific individuals or groups.
Understand which tasks cannot begin until others have been completed, identify gaps and redundancies in processes, establish stakeholder dependencies.
Force field analysis
Interview, diary study, survey, brainstorming, focus group.
Discussion
T-chart summarizing a group’s pro and con attitudes about a topic.
Understand political and operational issues without reference to specific individuals or groups.

37. Data analysis tools – qualitative (cont’d)
Process
Goal
Fishbone chart
Method:
Archive review, interview, observation, participant observation, brainstorming, focus group, usability test
Activity:
Demo, walkthrough, task analysis
Artifact:
Diagram showing the causes of a problem categorized by people, process, equipment, materials, environment, management etc.
Understand the range of factors to a problem without reference to specific individuals or groups.
Storyboard
Interview, observation, participant observation
Task analysis, location mapping, movement mapping, social network analysis, information network analysis
Graphic novel-style narrative illustrating a process that involves multiple actors.
Understand the ways in which people interact with each other, their tools, and their environment.
Persona
Literature review, vendor review, data analytics, interview, survey, focus group
Affinity grouping, collaging, model building
A fictional profile of an imaginary person who represents a category of customer, user, or stakeholder.
Understand the various needs and desires of different categories, develop acceptance criteria, form marketing plan.

38. Data analysis tools – qualitative (cont’d)
Process
Goal
Flowchart with swimlanes
Method:
Data analytics, interview, observation, participant observation, diary study
Activity:
Demo, walkthrough, task analysis, social network analysis, information network analysis
Artifact:
Flowchart representing the normal logical sequence of activities in a process as performed by specific individuals or groups.
Understand the roles and responsibilities of stakeholders in a process.
Procedure analysis
Data analytics, interview, observation, participant observation, diary study, focus group
Textual narrative describing a process that involves multiple actors, comprising both normal scenarios and common exceptions, workarounds, and failure recovery methods.
Understand the behavior of stakeholders not only when conditions are ideal but also when problems arise.

39. Comparison table

40. T-chart

41. Mind map

42. Activity network diagram - simple

43. Activity network diagram - complex

44. Force field analysis

45. Fishbone diagram template

46. Fishbone diagram example

47. Storyboard

48. Persona

49. Flow chart with swim lanes

50. Procedure analysis