1. “Curious Places” October, 2007 Key Centre of Design Computing and Cognition, University of Sydney A Room that Adapts using Curiosity and Supervised Learning Kathryn Merrick, Mary Lou Maher, Rob Saunders

2. Overview Adaptable, Intelligent Environments Curious Supervised Learning A Curious, Virtual, Sentient Room Limitations and Future Work

3. The computer for the 21 st century Hundreds of computers per room Computers come and go (Weiser, 1991) Adaptability is important at two levels: The middleware level The behaviour level Adaptable, Intelligent Environments

4. Adaptable Middleware Resource management and communication Adaptability has been widely considered at this level Real time interaction Presence services Ad hoc networking Intelligent Room Project Gaia BLIP Systems

5. Adaptable Behaviour Adapting behaviour to human activities Supervised Learning The “Neural Network House” Data mining Considered in fixed domains How can we achieve adaptive behaviour in response to changing hardware or software?

6. Adaptability by Curiosity and Learning Curiosity adapts focus of attention to relevant learning goals Learning adapts behaviour to fulfil goals Curious reinforcement learning Curious supervised learning MySQL Database Proje ctor Rear project ion screen PC Bluetooth blip nodes Agent Curious Information Display Curious Research Space

7. Supervised Learning “Learning from examples” A supervised learning problem P can be represented formally by: A set S of sensed states A set A of actions A set X of examples X i = (S i, A i ) A policy π : S  A

8. Complex, Dynamic Environments Contain multiple learning problems P = {P 1, P 2, P 3 …} Learning problems in P may change over time Addition of new problems Removal of obsolete problems

9. Aim to focus attention on states, actions and examples from a subset of problems Works by filtering Identify potential tasks to learn or act upon Compute curiosity values Arbitrate on what to filter High curiosity may trigger learning or action Low curiosity does not Modelling Curiosity for Supervised Learning S (t), X (t) S (t) X (t) Curiosity LearningAction Observations and events Task Selection Curiosity Value Arbitration

10. The Curious Supervised Learning Agent Past states, examples and actions are stored in an experience trajectory Y Experiences may influence curiosity A (t) S (t) S (t), X (t) Y (t-1) sensors effectors L A π (t) SL π (t-1) X (t) M = { Y (t) U π (t) } π (t) Y (t) C

11. A university meeting room in Second Life Seminars and Meetings Tutorials Skype-conferencing A Curious, Virtual, Sentient Room

12. Floor Sensors SMART Board and Chairs BLIP System Lights Virtual Sensors and Effectors

13. Meta-Sensors and Meta- Effectors BLIP System provides an up-to- date list of current sensors and effectors and acts as an intermediary for communication Agent does not communicate directly with sensors and effectors Agent has a ‘sensor of sensors’ and an ‘effector of effectors’

14. The Curious Room Agent Computational model of novelty used for curiosity Table-based supervised learning using associations Learns accurately but Unable to generalise

15. Behaviour of the Curious Place Avatar enters  Lights go on Avatar sits  SMART Board on  Lights off

16. Preliminary Evaluation ~6 repetitions by human controlled avatars required for learning Can adapt to new devices Can adapt simple behaviours to form more complex sequences

17. Limitations Current prototype is proof-of- concept only, no significant empirical results yet Issue of if/when/how to ‘forget’ behaviours Is an interface required for manual editing or override of learned behaviours?

18. Future Work Further work on curiosity models Design a suite of experiments to test attention focus in Environments of increasing complexity Dynamic environments More complex tasks