The Random Sampling/Tracking Tool: A Response to Over- Surveying Steve Graves Intel Corporation

Overview  Problem Statement  Recommended Solution  Features  User Interface  Demo  Benefits  Concerns  Next Steps  Possible tool enhancements  Beyond Intel  Summary

Problem Statement  Perception that too many surveys are deployed to end users from IT – Source: Annual Partnership Excellence Survey with top Intel Execs  Likely causes identified (based on experience as a survey consultant) –Groups tended to survey entire population of interest instead of a sample (e. g. survey of habits of frequent travelers -- 26,000 people) –When criteria for a population is only partially available a screening survey is necessary – results in more people surveyed –Simple random sampling will tend to hit a small percentage of people multiple times

Same Participant General Population Study A Study B

Problem Statement  Effective and efficient process to stop over- surveying does not exist –Previous process relied on incomplete information and “Guardians” – go between groups –No mechanism to track who has received surveys –Guardians may be overly cautious –Meetings used to determine what to do – time consuming and inefficient –Process is not followed universally

Recommended Solution  Sampling tool containing a repository of Intel employee data –Utilizes data from employee database (non- confidential) –Generates random samples of target audience –Tracks who has been sampled over a given time period – currently 5 months –Sampled individuals will not be selected again for during this period

Recommended Solution Database perspective

Recommended Solution Database perspective – previously sampled individuals removed before Sample is generated

Recommended Solution  Incorporate into an overall process where researchers are required to utilize the tool and consult with sampling advisors –Advisors can advise researchers on appropriate sampling techniques – so researchers to survey an entire population (e.g. 3,000 instead of 26,000)  Currently implemented using RAD database programming language –Tool has been in use for almost 2 years –Tool is performing reliably and customers are satisfied –Usage is only through word of mouth –Approximately 60 samples generated last year

Sampling Tool Features  Current features –Tracking and recording mechanism –Recording of samples generated by others –Random Sampling of entire Intel population –Random sample from group that uses criteria not part of employee database (for example: Employees who use PDAs) –Random sampling of Sub-Groups (For example: Employees in Folsom California) –Stratified sampling

User Interface

Demo (Double Click to Start)

Benefits  Saves researchers time –Reduces the need for screening surveys –No need to re-invent the wheel –Saves time compared to previous process

Benefits  Improvement in sampling process –Reduce or eliminate “over-sampling” –Delay surveys when target population is not available –Researchers receive consultation on using appropriate sample sizes –Distribute sampling more evenly across groups

Concerns  May cause sampling bias in some situations

Next Steps  Need to develop more sophisticated business rules around sampling process  Currently working to make this an IT wide process –Early results are promising  Looking at making it an Intel wide process  Port to SQL server and have a web based solution –Current access is through Remote Desktop Connection  Develop strategies to mitigate sampling bias

Possible Future Tool Enhancements  Complex samples through UI – currently requires custom query  Linking to Other database that contains product usage information

Beyond Intel  Similar tools could be used in similar situations in other companies or institutions

Summary  Problem: Over-surveying by our IT organization  Response: Random Sampling/Tracking Tool in conjunction with business rules to manage survey sampling process  Benefits: Saves researchers time, reduces over-surveying, improves sampling process  Working to make this an IT wide official process and eventually Intel wide

Summary  Next steps: –Link to other databases –Improve business rules –More robust software platform –Strategies to mitigate sampling bias  Similar tools/business rules could be used in other institutions