Counting How Many Words You Read

Reading Habit Quantifying reading habit Provides insights on language skills, effective learning and critical thinking Identifying a reading log, time spent on reading at different time of day may provide these insights Electrooculography and optical eye tracking provide the opportunity to track reading

Motivational Applications Quantifying reading habit Might help to identify the children with reading disabilities People can increase their reading speed without any problem Quantifying self

Contributions Quantifying how much words a user reads working for both optical eye tracking and electrooculography Lowest error rate between 5 – 15% for word count estimation Initial hint that these method can work with commercial smart glasses

Fixation Fixation or visual fixation is the maintaining of the visual gaze on a single location https://www.youtube.com/watch?v=kiFpMbfj_08

Saccade Saccade Alternate saccades and visual fixations, quick, simultaneous movement of both eyes between two phases of fixation in the same direction can be associated with a shift in frequency of an emitted signal or a movement of a body part or device Alternate saccades and visual fixations, https://www.youtube.com/watch?v=kiFpMbfj_08

Eye Tracking Electrooculography Uses electrodes Measures change in potential when eye moves as eye is considered as a diphole between cornea and retina Cheap, but gives relative eye movements

Eye Tracking Optical Tracking Uses cameras and infrared light to track eye gaze Binocular gaze estimates at a sampling freq. 30Hz Video resolution with resolution 1280X960 pixel Provides higher accuracy but requires high power to infer eye position, motion and gaze based on iris shape Provides raw eye gaze data (Saccade and fixation data)

Proposed Approach

Proposed Approach Preprocessing Reading detection Line-break detection Word count estimation

Preprocessing Optical tracking Combines several close by fixations into larger duration fixation

Reading Detection Calculate these features in 3 sec framing window Apply a SVM classifier For EOG, blink duration and frequency is also used as a feature For optical system mean fixation duration and variance of fixation count is used

Line-break Detection Reading is dominated by two types of saccades Detection of the dynamic line breaks using the distribution of the horizontal component of saccade Reading is dominated by two types of saccades short one in reading direction Longer saccade against the reading Two steps Combine consecutive saccade against reading direction Horizontal Saccade Direction (HSD) histogram

Line-break Detection Dynamic line break detection Saccade amplitude sa Horizontal direction component sdh -1 for the against main reading and +1 for the in reading Horizontal saccade direction component HSD = sa * sdh HSD distribution is noted next

Line-break Detection Combine all saccades, calculate HSD and combine them in HSD histogram Fit a mixture of 2 Gaussians and take distance between two maximas

Line-break Detection Two types of saccades Short Long larger one is the average reading saccade direction * amplitude Second maximum average line break saccade direction

Line-break Detection Two steps Combine consecutive saccade against reading direction

Determine line break

Word Count Estimation Basic Word Count (1)Avg. word count from the document read time (2) Estimate from line breaks Uses the information – avg. words per line Words per line – easy for electronic system For paper--------image retrieval technique

Word Count Estimation Basic Word Count SVR count (1)Avg. word count from the document read time (2) Estimate from line breaks Uses the information – avg. words per line Words per line – easy for electronic system For paper- image retrieval technique SVR count Uses features total time read, sum of all saccades distance, sum of the line break saccade distances, number of line breaks and sum of the reading saccade distances Train a SV regression

Experimental Setup Mobile optical eye tracking Mobile eye tracker 2.0 Binocular gaze estimates at a sampling freq. 30Hz Video resolution with resolution 1280X960 pixel Can capture saccades of 33 ms or slower

Experimental Setup Baseline Experiment Eye tracking glass Office scenario, 9 subjects 14 documents to read 10 simple text (PET) , 4 difficult (SAT) Word length 135 to 414 (mean 245) Eye gaze and scene camera are recorded Comprehensive questions Text understanding

Experimental Setup Devices Experiment Impact of device type and document length on word count accuracy 5 documents of different length read on different devices (size)– paper, smartphone, computer screen, tablet---same font 12 Varying line length Office scenario, 10 subjects 5 documents to read Word length 135 to 414 (mean 245) Perform other activities (solving sudoku, talking, playing games)

Experimental Setup Electrooculography 8 participants Active electrodes Sampling rate 1kHz

Results – Optical Tracking Evaluation- Leave one out (of n users) Reading detection is accurate Line break detection HSD Histogram for line break detection on different devices For baseline dataset, line break method reduces error from 15% to 6% Device dataset – 62% to 8%

Results – Optical Tracking Word-count estimation Change in line length Line break Reading from the devices is challenging – high error

Results – EOG Line break and Word-count estimation EOG performs better than Mobile tracker

Commercial eyewear – JINS MEME 4 participants, 3 days https://jins-meme.com/ja/

Application