1 Prof. Soon Ju Kang, Ph.D. Director, Center of Self-Organizing Software Platform(CSOS) Kyungpook National University

2 Motivation Aging baby boomers – epidemic of Alzheimer’s Disease Deinstitutionalization of the cognitively disabled Nationwide shortage of caretaking professionals  Healthcare Crisis

3 Motivation Develop technology to  Support independent living by people with cognitive disabilities At home At work Throughout the community  Improve health care Long term monitoring of activities of daily living (ADL’s) Intervention before a health crisis  Patient-Centered or Connected Patient Healthcare

4 In this Talk,.. Introducing KNU-CSOS’s service infra-structure for personal activity monitoring and recognition  Self-Organizing Software Platform(SoSp)  Protocols for SoSp Tracking and predicting a user’s daily life activity  SoSp Watch as a Personal Activity Blackbox Advanced services using the SoSp Watch:  Matching the activities with alerting  Reminding and Asisting activities Conclusion

5 Self-Organizing Software Platform(SoSp) Infra-Structure

6 Self-Organization in Natural Systems

7 Characteristics of Self-Organizing Systems C SiSi C S C S C S Direct interaction via signals Local work in progress Indirect communication via the environment Interacting Among Individuals and with the Environment  Direct communication among neighboring systems(gossiping)  Indirect communication via the environment (stigmergy)  No centralized supervisor then no. single failure point  Primitive functions : Identifying location of itself and neighbors Clustering and collaboration Delay Tolerant Communication

8 Self-Organization in IT world viewpoints  Stationary(Anchor) node based location-awareness(Indoor LBS)  Direct peer-to-peer communication with proximity-based neighbor identification  Peer-to-Peer based fully distributed computing architecture  Opportunistic service ☞ Automatically activated when the time, the location and the users just matched.

9 Proposed Network Architecture  SoSp Service Infra Overlay network architecture atop of Internet and accommodate various wireless communication protocols supporting several mobile devices R(SoSp-Router) acts two roles :  Communication access-point of all mobile devices  Location anchor node for all mobile devices(stand for a unit space. E.g. room, floor, kitchen..) All service are provided by the router covering the unit space! -> never use any centralized server  Direct communication among neighboring systems(gossiping)  Indirect communication via the environment (stigmergy)  Interaction with (stimulation by) the neighbors and environment Identifying location of itself and neighbors Clustering and collaboration Delay Tolerant Communication  Opportunistic service (activated when the time, the location, the users just matched) 9 Where is the nearest printer? Mobile Device SOS-Router Resource Device R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R R

10 Assumption Stationary Nodes(R)(SoSp Router)  Know current location  Know neighbor stationary nodes  Can notify current location to the mobile nodes which are located in its area  Can be added or deleted dynamically  No. centralized management server(self-organizing overlay network) Mobile Nodes(SoSp Client)  ID-client : act as a Personal ID device  Resource-client: device supply a certain service  Can move freely  Low power consumption is essential

11 Assumption 5cm SoSp-Clients (Resources, ID)

12 How to Organize the Distance Graph s45 s44 s42 Node s45 information IDs45 IP GroupRTLAB LocationE Neighbor List (dist, node_id) { (2, s42), (3, s44) } Node: SoSp-Router Edge: Physical distance between two neighbor SoSp-Routers path  Each SoSp-Router has only the information of neighboring nodes connected through physical path.

13 Suggested Protocols in SoSp Infra SoSp Router ↔ Smart Mobile Device(Smart phone or pad)  SSLP(SmartDevice to SoSp-Router Service Lookup Protocol) SoSp Router ↔ SoSp Router  LSDP (Location-based Service Discovery Protocol) SoSp Router ↔ SoSp Client  LIDx&AMD (Location ID exchange and Asynchronous Message Delivery) SoSp Client(Resource) ↔ SoSp Client(Mobile ID device)  PniP (Proximity based Neighbor Identification Protocol ) LSDP SSLP LIDx & AMD PniP

14 Middleware in SoSp-Router SoSp-Router ANT+ BLE WiFi Device Proxy Resource Manager Resource Repository Lookup Engine Neighbor List SN Manager Agent Manager Service Agents Printer Service Health Service Agent n … Resource Device Mobile Device RF (LIDx & AMD) WiFi (SSLP)

15 Basic LIDx

16 Timeline of LIDx & AMD

17 PniP (Proximity-based Neighbor Identification Protocol) Mobile ID device (ex. watch) Resource device (ex. Sphygmomanometer) Broadcasting wake-up signal ID and Proximity information Match appropriate device with service profile Service data communication… Press start button PniP

18 Basic PniP (3)loop Mobile ID1Resource DeviceMobile ID 2 1 : Power on() 2 : wake up signal 3 : wake up signal 4 : wake up()5 : wake up() 6 : TRX ON()7 : TRX ON() 8 : Service advertising9 : Service advertising 10 : RSSI value()11 : RSSI value() 12 : Mobile ID & RSSI13 : Mobile ID & RSSI 14 : Select mobile() 15 : User service data 16 : Service end 17 : time out

19 Tracking and predicting a user’s behavior (Personal Activity)

20 SoSp Watch (Mobile ID Device) -Act as a Personal Activity Blackbox -SoSp Protocols implemented (LiDx/AMD, PniP, EAR,etc) - Machine-to-machine direct communication without no human intervention

21 EAR (Exercise Activity Recognition) task

22 Activity Accumulating using SoSp Watch ① come home (PniP) ② weight measurement (PniP) 9:00 [front]come 9:05 [living]weight 9:15 [room]medicine 9:30 [front]out ④ go out (PniP) [living room] [bedroom] LIDx ③ take medicine (PniP)

23 Activity Reminding Scenario Alert for taking medicine SoSp LIDx & AMD “Take your pills” messaging Invalid user !! Successful authorization  Emission pill Smart Pill reminder

24 Predefined Activity Library in SoSp Watch

25 Activity Matching and Alerting in Real-Time Predefined activity(making Tea) Matching the accumulated activity in real-time LIDx EARPNIP EAR Alerting! [Don’t turn off the oven] Time line Enter the Kitchen Get Mug Activate Microwave oven Pouring water Drinking Time line Enter the Kitchen Get MugTurn on oven Pouring water Drinking Turn-off oven

26 Remote Activity Assisting thru SoSp Watch if(room4) turn_on light OOO 25 May 2013 Activity Schedule 7:00 Light ON 7:10 TV ON 7:20 Computer ON 7:50 TV OFF …

27 Implementation and Evaluation

28 SoSp-Router H/W Implementation AC Power Ethernet ANT+ WiFi Bluetooth Size: 6cm x 6cm Installed in the unit space 800MHz ARM Cortex A8 CPU with embedded linux 4 USB 2.0 ports for wireless communication modules (WiFi, , Bluetooth, ANT, etc.)

29 SoSp-Client H/W Implementation Resource device with PniP host module Mobile ID devices with PniP client module

30 SoSp Watch (Mobile ID device) UBINOS(KNU CSOS RTOS) based WATCH platform SDK Act as a Blackbox for Personal Activity Recording and Delivery SoSp Middleware and Protocols implemented in the SDK Service Profile and Predefined Activity Library included Real-time and immediate activity alerting, assisting and reminding functions with the help of neighboring smart device(smartphone or pad) Ultra low power consumption base on the opportunistic computing

31 SoSp Watch Power Consumption Control Sleep(Just Watch) PniP with Resource Device In-door Locationing(LIDx)Out-door Exercise Activity

32 Conclusion Healthcare Crisis  Motivation of this research Suggested Solution  SoSp Service Infra  In-door location-aware protocol  LIDx  Opportunistic service  PniP  Exercise Activity  EAR SoSp Watch as Personal Activity Blackbox  Personal Activity Accumulation and Reporting(Blackbox)  Activity Matching and Alerting in real-time  Personal Activity Reminding in real-time Future Work  Submit the SoSp protocols as global standard protocols in Health or Well-being industries

33