Rapid Analysis Farrokh Alemi, Ph.D.

Analysis takes time and reflection People must be lined up and their views sought. People must be lined up and their views sought. Ideas need to be sorted through. Ideas need to be sorted through. Data needs to be collected, stored, retrieved, examined and displayed. Data needs to be collected, stored, retrieved, examined and displayed. Reports needs to be prepared. Reports needs to be prepared. Policymakers may have to decide without the full benefit of the analysis Policymakers may have to decide without the full benefit of the analysis

Objectives How analysis could be made faster without sacrificing its quality How analysis could be made faster without sacrificing its quality An examination of what takes time in the analysis process An examination of what takes time in the analysis process Where one can speed up the work without affecting the quality of the report Where one can speed up the work without affecting the quality of the report

Typical Analysis Preparation Preparation Arrange for contracts and mandate to start Arrange for contracts and mandate to start Coordinate kick off meeting to clarify purpose & scope of the analysis Coordinate kick off meeting to clarify purpose & scope of the analysis Find relevant experts and decision makers Find relevant experts and decision makers Design study instruments and survey forms Design study instruments and survey forms Data collection Data collection Collect observations Collect observations Collect experts opinions Collect experts opinions Store data Store data Analyze data Analyze data Retrieve data Retrieve data Clean the data (classify data; check distribution and range of data, edit data) Clean the data (classify data; check distribution and range of data, edit data) Examine accuracy of data (Check for errors in logic, check for errors in transfer of data) Examine accuracy of data (Check for errors in logic, check for errors in transfer of data) Examine if experts were in consensus Examine if experts were in consensus Calculate expected values or model scores Calculate expected values or model scores Calculate the correspondence between model and experts' judgments Calculate the correspondence between model and experts' judgments Presentation Presentation Distribute draft report Distribute draft report Prepare presentation Prepare presentation Get input from audience before meeting Get input from audience before meeting Present results at meeting Present results at meeting

Speedup Analysis through More Preparation 1.Draft the final report at the start 1.Draft the final report at the start 2.Avoid group kick-off meetings 2.Avoid group kick-off meetings 3.Get access to the right experts and through them to right data 3.Get access to the right experts and through them to right data

1. Draft The Final Report at Start Write the introduction, the methods section, & the results section Write the introduction, the methods section, & the results section Include all related tables and appendices Include all related tables and appendices Saves the time spent on clarifying the procedures Saves the time spent on clarifying the procedures Clarifies what data are needed Clarifies what data are needed Clarifies procedures Clarifies procedures Generating automatic web content Generating automatic web content

2. Avoid group kick-off meetings Meet & work individually first Meet & work individually first Easier to arrange Easier to arrange Require less coordination Require less coordination Facilitate later larger face-to-face meetings Facilitate later larger face-to-face meetings

3. Get access to the right experts and through them to right data

Speedup Data Collection 4.Collect only the needed data 4.Collect only the needed data 5.Reduce the data collected by sampling 5.Reduce the data collected by sampling 6.Replace data collection with observations of others 6.Replace data collection with observations of others 7.Validate subjective indices on objective data 7.Validate subjective indices on objective data 8.Plan ahead for rapid data collection 8.Plan ahead for rapid data collection 9.Let technology collect the data 9.Let technology collect the data

4. Collect only the needed data Many collect data they do not use Many collect data they do not use Not experienced by the designer Not experienced by the designer Short surveys will collect less data, have better response rates and waste less of organization’s resources Short surveys will collect less data, have better response rates and waste less of organization’s resources Prepare final report at start Prepare final report at start Ask if responses lead to action Ask if responses lead to action

5. Reduce the data collected by sampling Larger sample, longer time to completion Larger sample, longer time to completion Start with a small group of people and expand the sample if needed Start with a small group of people and expand the sample if needed Shift from sampling the event to measuring time to the event Shift from sampling the event to measuring time to the event For example, one can radically reduce the number of patients examined by looking at time between two wrong side surgeries as opposed to number of wrong side surgeries. For example, one can radically reduce the number of patients examined by looking at time between two wrong side surgeries as opposed to number of wrong side surgeries.

6. Replace data collection with observations of others Subjective data Subjective data Objective data Objective data Both are suspect. Both are suspect. Subjective data can reduce the data collection burden Subjective data can reduce the data collection burden Subjective data it is not meant the likes and dislikes of a person but observations made by that person

7. Validate subjective indices on objective data If experts specify the parameters then there is no need to put aside data for parameter estimation If experts specify the parameters then there is no need to put aside data for parameter estimation The need for data is reduced, The need for data is reduced, For example, severity indices can be constructed from subjective opinions and tested against objective data. For example, severity indices can be constructed from subjective opinions and tested against objective data.

8. Plan ahead for rapid data collection Alerts of upcoming surveys Alerts of upcoming surveys Consent collected Consent collected For example, changes in substance abuse rates within United States collected from emergency rooms For example, changes in substance abuse rates within United States collected from emergency rooms Maintain networks of informants and consents ahead of analysis Maintain networks of informants and consents ahead of analysis

9. Let technology collect the data Computers call patients, find them, ask them, analyze the responses and fax the results Computers call patients, find them, ask them, analyze the responses and fax the results More truthful More truthful No later data entry No later data entry

Speedup Data Analysis 10. Clean the data & generate reports automatically 11. Analyze emerging patterns before all data are available 12. Use software to automatically analyze data

10. Clean the data & generate reports automatically

11. Analyze emerging patterns before all data are available Exit polling Exit polling Time to reoccurrence Time to reoccurrence Provide mock up analysis

12. Use software to analyze data Many existing software Many existing software Read reviews Read reviews

Speedup Presentation 13. Set up presentation meeting months in advance 14. Present to each decision maker privately prior to the meeting

13. Set up presentation meeting months in advance Decision makers are busy Decision makers are busy Make an appointment before the report is ready Make an appointment before the report is ready Create pressure to produce the findings on time Create pressure to produce the findings on time Critical path Critical path Helps decision makers wait for the report Helps decision makers wait for the report

14. Present to each decision maker privately prior to the meeting Get input privately, revise report Get input privately, revise report No surprise group meetings No surprise group meetings

Discussion No evidence of total time saved No evidence of total time saved Expectations are changing Expectations are changing Time and motion study can increase market Time and motion study can increase market More time planning less time executing More time planning less time executing Whose time is saved? Whose time is saved? The person who plans the analysis The person who plans the analysis The person who executes it The person who executes it The person who receives It The person who receives It