1. The Case for Operating System Management of User Attention
Kyungmin Lee, Jason Flinn, and Brian Noble
University of Michigan

2. Trend in mobile app interaction
Using apps while
performing primary tasks
Apps initiate interactions
Kyungmin Lee

3. Interaction in various user contexts
User’s current primary activity ✖ ✖ ? ✔
Application is unaware of user’s context!
Kyungmin Lee

4. Existing solution: Let user decide
Set policy for each app
Disable all interactions
Too coarse grained! All or nothing
Kyungmin Lee

5. Our proposed approach Mobile OS Mobile sensors Extract user’s context
Interactions
Deliver now
Modify format
Defer
Interactions
Kyungmin Lee

6. Outline Motivation Our vision Our proposed approach Challenges
Kyungmin Lee

7. Our vision Do not interrupt! User’s current context Interrupt?
Can you pick up milk?
From: Your wife
User’s current context
Interrupt?
Do not interrupt!
Kyungmin Lee

8. Interrupt! via audio interaction
Our vision
Dangerous road conditions ahead
User’s current context
Interrupt?
Interrupt! via audio interaction
Kyungmin Lee

9. Our vision Interrupt! User’s current context Interrupt?
Can you pick up milk?
From: Your wife
User’s current context
Interrupt?
Interrupt!
Kyungmin Lee

10. Manage user attention as a resource
100%
<Priority level>
It’s a scheduling problem!
User’s activity
Attention level
100%
Visual Auditory Cognitive Haptic
Attention level
100%
<Priority level>
Visual Auditory Cognitive Haptic
Interaction
Attention demand
Visual Auditory Cognitive Haptic
Kyungmin Lee

11. Our proposed approach Priority Attention level Attention level
Very low
Low
Medium
High
Very high
100%
Attention level
User’s current context
Visual Auditory Cognitive Haptic
Kyungmin Lee

12. Our proposed approach Priority Attention demand Attention demand
Very low
Low
Medium
High
Very high
100%
Can you pick up milk?
From: Your wife
Interrupt?
Attention demand
Visual Auditory Cognitive Haptic
Kyungmin Lee

13. Our proposed approach No delivery! Attention level after delivery
Medium priority
Can you pick up milk?
From: Your wife
100%
Attention level
High priority
Visual Auditory Cognitive Haptic
No delivery!

14. Our proposed approach Attention level after delivery
Very high priority
Dangerous road conditions ahead
100%
Attention level
High priority
Visual Auditory Cognitive Haptic

15. Our proposed approach Deliver! Attention level after delivery
Change to audio modality
Very high priority
Dangerous road conditions ahead
100%
Attention level
High priority
Visual Auditory Cognitive Haptic
Deliver!

16. Our proposed approach Deliver! Attention level after delivery
Cognitive attn. load has dropped
Medium priority
Can you pick up milk?
From: Your wife
100%
Attention level
High priority
Visual Auditory Cognitive Haptic
Deliver!

17. Challenges in determining priority
Med. priority
High priority
From: A colleague
From: A colleague
High priority
Low priority
Friend’s request
Friend’s request
Kyungmin Lee

18. Learn from user’s behavior
High priority
Low priority
Kyungmin Lee

19. Interaction’s attention demand
Extend AMC (Mobisys ‘13)
Button size ✔
Button closeness
Text contrast ratio
Word count ✖
Animation
Scrolling
Kyungmin Lee

20. Interaction’s attention demand
Extend AMC (Mobisys ‘13)
Attention demand
Demand level
Visual Auditory Cognitive
Kyungmin Lee

21. Estimating user’s attention level
Very high priority
Same activity, but different priority level
100%
Highly engaged activity
Attention level
Low priority
Visual Auditory Cognitive Haptic
Lowly engaged activity
Kyungmin Lee

22. Conclusion Our vision: Right interaction at the right time
Our proposed approach
Treat user attention as a shared resource
Determine priorities of interaction and activity
Consider Attention level supply vs. demand
Kyungmin Lee

23. Questions?
Kyungmin Lee