THE SECOND LIFE OF A SENSOR: INTEGRATING REAL-WORLD EXPERIENCE IN VIRTUAL WORLDS USING MOBILE PHONES Sherrin George & Reena Rajan

TOPICS Introduction Introduction System Design Challenges System Design Challenges System Prototype Implementation System Prototype Implementation

INTRODUCTION Second Life represents the latest and most successful online services for entertainment and business. Second Life represents the latest and most successful online services for entertainment and business. This paper focuses on providing a virtual representation of humans, their surroundings as sensed by the humans themselves, and interactions with the members of their social networks. This paper focuses on providing a virtual representation of humans, their surroundings as sensed by the humans themselves, and interactions with the members of their social networks. Sensors embedded in mobile phones can be used to infer real-world activities, that in turn can be reproduced in virtual settings. Sensors embedded in mobile phones can be used to infer real-world activities, that in turn can be reproduced in virtual settings.

SYSTEM DESIGN CHALLENGES Activity Recognition and Visualization Activity Recognition and Visualization Intermittent Connectivity Intermittent Connectivity Scalability Scalability Data Sharing and Group Activities Data Sharing and Group Activities External sensing Devices External sensing Devices Context Reflection Context Reflection Privacy and Social Implications Privacy and Social Implications

Activity Recognition and Visualization Activity Inference:- consists of the extraction of high-level activities from time series of raw data sampled from sensors. Activity Inference:- consists of the extraction of high-level activities from time series of raw data sampled from sensors. Mapping:- mapping physical activities in the real world into different/same virtual ones. Mapping:- mapping physical activities in the real world into different/same virtual ones. Visualization & Information Accuracy:- some users may want to discard the information if the inference is not sufficiently accurate. Visualization & Information Accuracy:- some users may want to discard the information if the inference is not sufficiently accurate.

Intermittent Connectivity Cell phones may experience intermittent connectivity due to limits in network coverage. Cell phones may experience intermittent connectivity due to limits in network coverage. Avatars should be set to a meaningful state. Avatars should be set to a meaningful state. Delay tolerant mechanisms can be adopted to temporarily store user status locally on the phone in case of disconnection. Delay tolerant mechanisms can be adopted to temporarily store user status locally on the phone in case of disconnection.

Scalability To reduce the number of status updates flowing into the Second Life Visualization, the status/activity of the virtual avatar is updated only when the inferred status/activity of the human user changes. To reduce the number of status updates flowing into the Second Life Visualization, the status/activity of the virtual avatar is updated only when the inferred status/activity of the human user changes. Running activity recognition algorithms on the mobile devices reduces communication costs and computational burden on the server. Running activity recognition algorithms on the mobile devices reduces communication costs and computational burden on the server.

Data Sharing and Group Activities The system should leverage the sharing of raw or processed data among different people in physical proximity by means of short range radio communication. The system should leverage the sharing of raw or processed data among different people in physical proximity by means of short range radio communication. Individuals will be able to update not only their status, but also those of their neighbors. Individuals will be able to update not only their status, but also those of their neighbors.

External sensing Devices A small size sensing module external to the phone can be used to allow for more accurate activity inference compared with using embedded phone sensors which may be ill positioned. A small size sensing module external to the phone can be used to allow for more accurate activity inference compared with using embedded phone sensors which may be ill positioned.

Privacy and Social Implications The users should be able to disconnect from the virtual worlds at any time or be able to interrupt the sampling of their activities.

SYSTEM PROTOTYPE IMPLEMENTATION CenceMe CenceMe Bridging CenceMe and Virtual Worlds Bridging CenceMe and Virtual Worlds Design and Preliminary Evaluation of the Prototype Design and Preliminary Evaluation of the Prototype

CenceMe CenceMe is a general personal sensing system that enables members of social networks to share their sensing presence extracted by means of mobile phones with their buddies. CenceMe is a general personal sensing system that enables members of social networks to share their sensing presence extracted by means of mobile phones with their buddies. CenceMe can be used to interconnect virtual worlds and social networking systems such as Facebook. CenceMe can be used to interconnect virtual worlds and social networking systems such as Facebook.

Bridging CenceMe and Virtual Worlds

Design and Preliminary Evaluation of the Prototype The integration is based on the exploitation of virtual CenceMe objects that the Second Life avatar can wear or carry. This object alters the avatar animation to reflect the actual activity state of a human user carrying a cell phone that is running the CenceMe sensing application. The integration is based on the exploitation of virtual CenceMe objects that the Second Life avatar can wear or carry. This object alters the avatar animation to reflect the actual activity state of a human user carrying a cell phone that is running the CenceMe sensing application. The CenceMe object is implemented as a standard second Life object that we program to communicate with the CenceMe infrastructure via XML encoded requests and responses exchanged via HTTP. The CenceMe object is implemented as a standard second Life object that we program to communicate with the CenceMe infrastructure via XML encoded requests and responses exchanged via HTTP. Second Life CenceMe object behavior is written in LSL. Second Life CenceMe object behavior is written in LSL. The avatar can disconnect from CenceMe system by taking off the object. The avatar can disconnect from CenceMe system by taking off the object.

Design and Preliminary Evaluation of the Prototype contd.. The communication between Second Life and CenceMe platforms is pull based. The communication between Second Life and CenceMe platforms is pull based. A library of potential Second Life activities is maintained from which the user chooses to map specific real-world activities. A library of potential Second Life activities is maintained from which the user chooses to map specific real-world activities. Each virtual object within Second Life retrieves data independently from the CenceMe server. Each virtual object within Second Life retrieves data independently from the CenceMe server.

Delay Measurements An average delay of 30 sec is observed in reporting the activity changes. An average delay of 30 sec is observed in reporting the activity changes. This delay is due to the activity recognition process on the phones, the transmission of the inferred activity to the back-end, its retrieval from CenceMe database, and its display in Second Life. This delay is due to the activity recognition process on the phones, the transmission of the inferred activity to the back-end, its retrieval from CenceMe database, and its display in Second Life.

Thank You