Cmpe 589 Spring 2006

Sampling Target population Cost Sample is representative of population (measure statistical average age is 37- if you get 20 for mean for age – you might have a problem)

Sampling Methods Unrestricted Random Sample – Pick a number (300) – psuedo Random number generator Systematic Sampling- Arbitrary starting point- then take every seventh or eighth person Stratified Sample- Subdivide population before sampling –Sex –Sex and Educational Level –5% Cluster Sample- No master population list- (list of cities, list of zip codes, survey in each city (sample within clusters) Quota Sampling- select until quota is filled Household Sampling Convenient Sample

Survey Design Issues Questionnaire Rating Scale Checklist –1-3 are in printed form Interview

Survey Design Issues Questionnaire- structured item with yes, no, or maybe as answers - (free response items as alternatives- but you need coding scheme) Likert Scale Completely Satisfied   Completely Dissatisfied Social Science – Neutral or unbiased wording—not the way to do it in Quality Assurance –System is easy to learn –System is hard to use –System is hard to learn

Survey Design Issues One Way Item #A B CDE Second Way Item 1ABCDE M F 00000

Errors Sampling Errors- Sampling distribution of mean x sample mean and Mx population mean Non-Response Error- Less than 60% of Response Rate –Short Questionnaire (2-3 pages) –Letter of Explanation –Guarantee Privacy –Mail second copy –Limit the number of personal questions –Telephone non-response people –Missing Demographics Data

Problems Testing Can’t guarantee perfection When do you decide to stop fixing bugs and ship product? Testing is directed normal product: use- what about extreme conditions?

Usability Expert Reviews – external person critiques design) –Heuristic evaluation (checklist) –Guidelines review (internal standards) –Consistency Inspection (family products) –Cognitive Walkthrough (Structured Walkthrough) –Formal usability inspection (court room) Usability Laboratory Approach- (One Way mirror to spy on customer while they use the product) –Think aloud – problem solving –Video tape –Field Test – alpha/beta –Can you break this –Competitive Usability Testing Survey

Software Engineering Data Collection Trick- make sure data collection is not a burden on developers Project Process Quality Management P. 118 Basili & Weis –Establish the goal of data collection –Develop a List of questions related to the goal –Establish data categories –Design test data collection forms –Collect and Validate Data –Analyze Data

Software Engineering Data Collection Software Engineering data tends to be error prone? –Self reporting of data (protocol is ignored) –Unclear questions