1. ScanShot : Detecting Document Capture Moments and Correcting Device Orientation Jeungmin Oh, Woohyeok Choi, Joohyun Kim, Uichin Lee (Dept. of Knowledge Service Engineering, KAIST) 1

2. 2 Document capturing using a smartphone camera becomes very common

3. Many apps support document capturing, however…

4. 4 If we take a photo of document… Capturing a document

5. 5 Occasionally, captured photos had with erroneous orientation Viewing a captured document

6. 6 Portrait Upside Down Landscape RightLandscape Left Why do such orientation errors happen?

7. 7

8. 8 We ran photo taking experiment…

9. 9 The error rate for landscape task exceeds 80%

10. 10 Solution 1. Detect a document capture moment 2. Fix an orientation error by tracking rotation 3. Display an overlay indicator for manual correction

11. Acc-Z G 1. Detecting document capture moments 2. Fixing an orientation error by tracking rotation Accelerometer is used for detecting document capturing behavior 3. Displaying an overlay indicator for manual correction

12. Gyro-Z Gyroscope is used for tracking rotation to fix an orientation error 1. Detecting document capture moments 2. Fixing an orientation error by tracking rotation 3. Displaying an overlay indicator for manual correction

13. ScanShot successfully reduced erroneous orientations in the landscape document capturing task 13 1. Detecting document capture moments 2. Fixing an orientation error by tracking rotation 3. Displaying an overlay indicator for manual correction -0.59 -0.89

14. (a) overlay with incorrect orientation (b) overlay with correct orientation ScanShot displays overlay screen if needed 1. Detecting document capture moments 2. Fixing an orientation error by tracking rotation 3. Displaying an overlay indicator for manual correction

15. Q&A 15

16. Sensor data 16 Portrait, no rotation Landscape, rotation Portrait, no rotation Landscape, rotation AccelerometerGyroscope

17. CV-based approach 17

18. Grips for document capturing task External H/W 18

19. Variation of overlays 19 (a) overlay with correct orientation (b) overlay with incorrect orientation

20. Acc-Z Gyro-Z G