1. Life Logging Melekam Tsegaye Shaun Bangay Alfredo Terzoli Research area: Wearable, Pervasive and Ubiquitous Computing http://www.cs.ru.ac.za/research/students/g98t4414/research25/11/2004

2. Life logging  Recording/capturing everything you see and do.  Past, present and future events  Why ? for the sake of improving your memory for the sake of improving your memory Your memory defines who you are! Your memory defines who you are!  Amnesia?

3. Forms of life logs  Manual paper diaries  Electronic diaries on PDAs  Audio/Video recordings  Emergence of the Weblog - blog

4. Current Trends  Weblogs are mostly text and image based e.g. Blogger.com (blog via email) e.g. Blogger.com (blog via email)  Next natural transition is audio blogging Individuals make an audio recording, publish to a web server e.g. audioblog.com Individuals make an audio recording, publish to a web server e.g. audioblog.com  Hardware needed to start life logging Camera, microphone, local storage (Digicam/Cell phone) Camera, microphone, local storage (Digicam/Cell phone) Web server Web server

5. audioblog.com  Effortless audio logging  recording audio to a remote server from your web browser your web browser cell phone cell phone email email  audioblog.com recently introduced a video blogging extension

7. Life-log applications  Nokia lifeblog Software for organizing video clips recorded using a cell phone camera Software for organizing video clips recorded using a cell phone camera

9. Life-log applications(2)  Microsoft sense cam Passive Capture, improved participation in events Passive Capture, improved participation in events ACM workshop on Continuous archival and retrieval of personal experiences New York, USA October 2004ACM workshop on Continuous archival and retrieval of personal experiences New York, USA October 2004 Correlations and relationships from sensors used to manage captured images Correlations and relationships from sensors used to manage captured images light intensity sensorlight intensity sensor x/y tilt (tilt sensor )x/y tilt (tilt sensor ) motion sensor to avoid taking blurry pictures (accelerometer)motion sensor to avoid taking blurry pictures (accelerometer) temperaturetemperature infra red sensorinfra red sensor clockclock locationlocation MMC card storageMMC card storage 2xAA cell batteries, 12 hours2xAA cell batteries, 12 hours Size of a pagerSize of a pager Useful: quick replay of a day’s events Useful: quick replay of a day’s events

10. Our work  wPim - Wearable audio enhanced personal information management system  Record audio wherever you are Organize people and events (past, future and present) Organize people and events (past, future and present) Retrieve using time of capture and textual descriptors Retrieve using time of capture and textual descriptors

11. Current Work  Once you capture your memories how do you retrieve segments of interest for later use ? using time, location ? using time, location ?  We want to analyze data produced by multiple sensors at time of av capture and extract useful information at time of av capture and extract useful information to be used to auto tag complex streams such as video and audio to be used to auto tag complex streams such as video and audio  Assembling a large number of sensors and conducting experiments can be difficult Some sensors are not available in a convenient form Some sensors are not available in a convenient form We waste too much time dealing with hardware issues and produce poor software solutions We waste too much time dealing with hardware issues and produce poor software solutions

12. Current Work(2)  The approach we take in our attempts to solve the problems mentioned earlier is To define multiple virtual sensors that sense environmental and user states To define multiple virtual sensors that sense environmental and user states make the virtual sensors generate data (essentially, simulate the process sensing data) make the virtual sensors generate data (essentially, simulate the process sensing data) extract useful contextual information from the generated data extract useful contextual information from the generated data make use of the extracted information to build a contextual tag database make use of the extracted information to build a contextual tag database use the information in the tag database to automatically tag complex data streams such as video and audio streams use the information in the tag database to automatically tag complex data streams such as video and audio streams  Not simulation only, we want to be able use a combination of virtual and real sensors to run our experiments E.g. A wearable life logging facility that uses a combination of virtual and real sensors E.g. A wearable life logging facility that uses a combination of virtual and real sensors

13. Prototype sensors for: Video, Audio, Location other sensors and I/O devices through a multi- channel (a2d) data capture device.

14. Problems  Social issues Privacy Privacy Laws Laws Hostility and fear from society Hostility and fear from society First person experience capture vs big brother scenario First person experience capture vs big brother scenario  Hardware limitations Wireless network connectivity is not yet ubiquitous, bandwidth limitations Wireless network connectivity is not yet ubiquitous, bandwidth limitations long lasting power sources long lasting power sources comfortable wearable hardware comfortable wearable hardware

15. Questions ?