Evaluation of Speak Project 2b Due: March 20 th, in class

Overview Experiments to evaluate performance of your VoIP application (Project 2) Focus not only on how your software performs, but also – design, implementation of experiments – analysis – writeup

Details Two parts – In-depth user study with friend (called “group”) More forgiving users, less objective – Briefer user study outside group For both parts, evaluate by user perception – Brief, minute conversation – Record user opinions (1-10, or similar) – Pencil and paper, software, …

In Group Baseline case: 0% loss, UDP, no speech detection, 40ms Compare versus: – Loss: 0%, 1%, 5%, 10%, 20% – Latency (by increasing the sample interval): 40ms, 100ms, 250ms, 500ms, 1000ms – Connection type: TCP, UDP – Speech detection: on, off Pick (and justify) perceptual quality measure

Outside Group 1-3 people outside of your group Record some basic information – age, gender, profession, VoIP use… Loss rates of 0%, 5%, 20% Latencies of 40ms, 500ms, 1000ms Record – Quality score (scale 1-10, or similar) – Subjective comments ("sounded good", "was hard to talk", etc.). – Ask to compare with land-line phone call

Report Introduction (brief) – Motivation and Hypotheses Background on your software (as needed) Design of your experiments (brief, since given) – Details on above Analysis (detailed) Conclusions (brief) – Summarize findings Abstract – 1 paragraph – Write last, goes first

Hints See Hints for Project 1b – Graphing Refer to grading comments from Project 1b – General comments will hold for this report, too

Hand In Hardcopy! – Due at beginning of class

Grading 8% Abstract 7% Introduction Background (as needed) 20% Experiment design – In group (1/2) – Out of group (1/2) 60% Results and analysis – In group (1/2) – Out of group (1/2) 5% Conclusions Other – Writing, clarity, errors, etc. (can lose points) (Grading Rubric on Web Page)