Schepens Eye Research Institute Development of an Objective, Language-Based Measure of Perceived Video Quality Daniel R. Saunders, Peter J. Bex, and Russell L. Woods Schepens Eye Research Institute and Harvard Medical School Boston, Massachusetts

Schepens Eye Research Institute Dedicated to fighting blindness and visual impairment Treatments and assistive technology for eye disorders Example: Age-related Macular Degeneration –Affects at least 1.75 million in the U.S. –Central vision loss, leading to reduced contrast sensitivity and visual acuity –No cure The Schepens Eye Research Institute Daniel Saunders

Schepens Eye Research Institute Digital video enhancement Daniel Saunders Contrast enhancement (Peli and Peli, 1984; DigiVision CE-3000)

Schepens Eye Research Institute Digital video enhancement Daniel Saunders Digital magnification

Schepens Eye Research Institute How to evaluate enhancements? Easy to measure preference: –Dual monitor –Rating Adaptive contrast enhancement has been shown to be preferred in many cases. (Kim et al., 2004; Fullerton et al., 2008) But subjective. An objective measure would derive from how much it helps people perform a task. Reading, driving, watching TV. Daniel Saunders

Schepens Eye Research Institute Information transfer Principle: The better the perceived quality of a video clip, the more (and more accurate) information the viewer should be able to report about it. Impaired perception should result in reduced amount of accurate information. Use of multiple choice quiz questions has problems –Creation of questions is subjective –Labour intensive –Also may have low sensitivity (Peli, 2005) Daniel Saunders

Schepens Eye Research Institute Using free recall to measure information transfer Daniel Saunders “Describe this video as if to someone who hasn't seen it.” Standard set of 200 video clips, 30 seconds each, from Hollywood films and nature documentaries.

Schepens Eye Research Institute Collecting “normal” responses 4000 responses from Amazon.com Mechanical Turk workers (20 per clip) working from home 2400 responses from normally-sighted in a lab setting, stratified by age and gender (in progress) Daniel Saunders A spotlight displays the Batman symbol. Batman arrives on a rooftop and a beautiful redhead appears from behind the spotlight wearing a black negligee and robe. She flirtatiously walks over to the caped crusader and they converse. Batman swoops on. As he lands a woman comes out and starts talking to him. There is some plan afoot that Batman must thwart and she seems helpful by her look. Batman flies onto a roof top and starts talking to a blonde woman. The scene is very dark. We first see a bat spotlight, then Batman flies into the room. A woman with long hair comes out and speaks with Batman. It is a very dark room with only a "Batman" Logo shining into the night. This is how Batman is called upon. Once he arrives, it is in a darken room with a very seductress Nicole Kidman to greet him there. It starts out with a huge bright spotlight with Batman's stencil on it shining up into the night. It's a very dark scene, when Batman arrives. He is then approached by a young woman with long wavy red hair in a black robe and slinky nightgown. There is a slight breeze blowing her hair. A giant floodlight has a cutout of Batman in it. Batman appears out of nowhere and walks by the spotlight. A woman wearing a sexy dress walks into the room and talks with Batman. Examples:

Schepens Eye Research Institute Evaluate responses in terms of their discrepancy Daniel Saunders A spotlight displays the Batman symbol. Batman arrives on a rooftop and a beautiful redhead appears from behind the spotlight wearing a black negligee and robe. She flirtatiously walks over to the caped crusader and they converse. Batman swoops on. As he lands a woman comes out and starts talking to him. There is some plan afoot that Batman must thwart and she seems helpful by her look. Batman flies onto a roof top and starts talking to a blonde woman. The scene is very dark. We first see a bat spotlight, then Batman flies into the room. A woman with long hair comes out and speaks with Batman. It is a very dark room with only a "Batman" Logo shining into the night. This is how Batman is called upon. Once he arrives, it is in a darken room with a very seductress Nicole Kidman to greet him there. It starts out with a huge bright spotlight with Batman's stencil on it shining up into the night. It's a very dark scene, when Batman arrives. He is then approached by a young woman with long wavy red hair in a black robe and slinky nightgown. There is a slight breeze blowing her hair. A giant floodlight has a cutout of Batman in it. Batman appears out of nowhere and walks by the spotlight. A woman wearing a sexy dress walks into the room and talks with Batman. Score for an individual is average distance of their responses from the normally-sighted responses. A high score indicates poor vision/perceived video quality.

Schepens Eye Research Institute Text passage similarity metrics Baseline: shared word count (common word removed) –Performance (Mechanical Turk): Mean shared words within a clip = 6.8, outside a clip = 1.7. Accuracy at matching responses to clips = 95%. Latent Semantic Analysis could deal with synonyms etc. Used in essay grading and finding articles by matching abstracts. –Not our wheelhouse! Daniel Saunders

Schepens Eye Research Institute Assessing validity of the measure Perceived video quality = Perception + video quality. Measure can be used to assess video, with applications in e.g. video compression algorithms. Future studies: Normally-sighted participants watch blurred video. –Greater blur should cause larger discrepancy scores –Within-subject design, so compensates for variability Visually-impaired participants watch enhanced video. Daniel Saunders

Schepens Eye Research Institute Daniel Saunders What we have: An application (evaluating the benefit of video enhancement to the visually impaired). A dataset (free descriptions of 200 video clips). What we need: A metric of text passage discrepancy. (LSA?) Thanks!

Schepens Eye Research Institute Mechanical Turk for data collection Daniel Saunders 13 / responses in 30 days

Schepens Eye Research Institute No objective measure of perceived video quality Most are based on ratings or side-by-side comparisons Relevant to perception as well as to videos. Daniel Saunders 14 / 31

Schepens Eye Research Institute Age-related macular degeneration Affects more than 1.75 million people in the United States(Friedman et al., 2004), 3 million by Causes 50% of all cases of blindness in the United States (WHO, 2002). No cure, only aids. Which aids really help? (the same question as perceived visual quality) Daniel Saunders 15 / 31

Schepens Eye Research Institute TV is an important activity How can we help people watch TV? How can we evaluate whether we’ve helped? Daniel Saunders 16 / 31 20% of seniors who went in for a visual impairment exam said that one of their main motivations was watching TV. 69% of people with central field loss reported at least “some” difficulty with watching TV