Design la proiectul MyDiary Cursuri 7-8 Design la proiectul MyDiary

Proiectul nostru: MyDiary Un agent capabil să “înțeleagă” situațiile în care se află Masterul său, să le transpună în text și să conducă un dialog relativ la ele

Ce vrem să facă sistemul? Să “știe”, în cât mai multe momente ale zilei: ce fac unde mă aflu cu cine mă întâlnesc despre ce vorbesc cine îmi scrie mesaje (?)… de ce fac anumite lucruri Să fie capabil să formeze fraze care exprimă ce am făcut Să fie capabil să poarte un dialog despre ziua de azi, cea de ieri cea de mâine (?)

Un graf al situațiilor – ierarhie de clase sleeping awake at_home moving in_activity alive in_theater in_park shopping not_moving driving walking jogging driving_in_town driving_outside

Module 1. LOCALISATION 2. MOVEMENT 3. FACES 4. SOUNDS&VOICES 5. INFERENCES 6. BEHAVIOUR 7. TEXT GENERATION 8. QUESTION-ANSWERING 9. INTERFACES 10. CLIENT-SERVER 11. STANDARDS AND EVALUATION

MyDiary – schema generală Activities graph Behaviour sensors Text generation Diary repr. Localisation Movement Inferences Question answering Diary Faces MyDiary Sounds& voices

Interfețe și comunicații MyDiary Localisation Inferences COMM GPS activități zilnice Movement COMM vibrații Faces imagini COMM Sounds/Voices sunete COMM

Interfețe și comunicații Diary repr. Inferences Text generation SIT SIT Diary NL query QUE dialogul de seară MyDiary Question answering NL

Interfețe și comunicații Diary repr. Inferences Text generation SIT SIT NL answer SIT dialogul de seară MyDiary Question answering NL Diary

Exemplu (din cursul 5) TIME=7:30, PROXIMITY = bedroom, CLOCK = awake_time, SOUND = awake_ringing, GG:LIGHT = intensity_raise, WiFi_WAVE = movement, ACCELERATION = zero, GPS = null, GEO_POSITION = home(remember_last_position) MyDiary output: Astă dimineață am fost trezit de alarmă la ora 7:30 în dormitorul meu.

Communication standards: Localisation <COMM module=“LOC” time=“…” location=“…” proximity=“…”> <COMM module=“LOC” time=“7:30” location=“home” proximity=“bedroom”> TIME=7:30, PROXIMITY = bedroom, CLOCK = awake_time, SOUND = awake_ringing, GG:LIGHT = intensity_raise, WiFi_WAVE = movement, ACCELERATION = zero, GPS = null, GEO_POSITION = home(remember_last_position)

Communication standards: Movement <COMM module=“MOV” time=“…” movement=“…”> <COMM module=“MOV” time=“7:30” movement=“zero”> TIME=7:30, PROXIMITY = bedroom, CLOCK = awake_time, SOUND = awake_ringing, GG:LIGHT = intensity_raise, WiFi_WAVE = movement, ACCELERATION = zero, GPS = null, GEO_POSITION = home(remember_last_position)

Communication standards: Faces <COMM module=“FAC” time=“…” face=“…”> <COMM module=“FAC” time=“7:30” face=“zero”> TIME=7:30, PROXIMITY = bedroom, CLOCK = awake_time, SOUND = awake_ringing, GG:LIGHT = intensity_raise, WiFi_WAVE = movement, ACCELERATION = zero, GPS = null, GEO_POSITION = home(remember_last_position)

Communication standards: Sounds&Voices <COMM module=“S&V” time=“…” sound=“…” voice=“…”> <COMM module=“S&V” time=“7:30” sound=“awakeRinking” voice=“zero”> TIME=7:30, PROXIMITY = bedroom, CLOCK = awake_time, SOUND = awake_ringing, GG:LIGHT = intensity_raise, WiFi_WAVE = movement, ACCELERATION = zero, GPS = null, GEO_POSITION = home(remember_last_position)

Communication standards: Situation Representation <SITUATION module=“INF” time=“…” situationID=“…” instanceOf=“…” location=“…” proximity=“…”> <SITUATION module=“INF” time=“2014/11/03/23:18” situationID=“sleeping048” instanceOf=“sleeping” location=“home” proximity=“bedroom”> <SITUATION module=“INF” time=“2014/11/04/7:30” situationID=“awake049” instanceOf=“awake” location=“home” proximity=“bedroom”> MyDiary output: Astă dimineață am fost trezit de alarmă la ora 7:30 în dormitorul meu.

Exemplu TIME=7:45, PROXIMITY = bedroom, ACCELERATION = unknown INTERVAL_END=7:46, ACCELERATION = walking, TIME=7:46, ACCELERATION = unknown, PROXIMITY = kitchen MyDiary output: La 7:46 am intrat în bucătărie. MyDiary reasoning: Ce am făcut între 7:30 și 7:46?

Communication standards: Movement & Location <COMM module=“MOV” time=“…” movement=“…”> <COMM module=“MOV” time=“7:45:03” movement=“unknown”> <COMM module=“MOV” time=“7:46:15” movement=“walking”> <COMM module=“MOV” time=“7:46:25” movement=“unknown”> <COMM module=“LOC” time=“7:46:25” proximity=“kitchen”> TIME=7:45, PROXIMITY = bedroom, ACCELERATION = unknown INTERVAL_END=7:46, ACCELERATION = walking, TIME=7:46, ACCELERATION = unknown, PROXIMITY = kitchen

Communication standards: Situation Representation <SITUATION module=“INF” time=“…” situationID=“…” instanceOf=“…” location=“…” proximity=“…”> <SITUATION module=“INF” time=“2014/11/03/7:46” situationID=“atHome050” instanceOf=“atHome” location=“home” proximity=“kitchen”> MyDiary output: La 7:46 am intrat în bucătărie.

Exemplu TIME=8:16, MO:BAROMETER = change_low2high(20m); ACCELERATION = steps, PROXIMITY = lost MyDiary output: La 8:16 am ieșit din casă și am coborât scările. MyDiary reasoning: Am intenționat să merg ?undeva.

Communication standards: Movement & Location <COMM module=“MOV” time=“…” movement=“…”> <COMM module=“MOV” time=“8:16” movement=“descending”> <COMM module=“LOC” time=“8:16” location=“home” proximity=“zero”> TIME=8:16, MO:BAROMETER = change_low2high(20m); ACCELERATION = steps, PROXIMITY = lost

Communication standards: Situation Representation <SITUATION module=“INF” time=“…” situationID=“…” instanceOf=“…” location=“…” proximity=“…”> <SITUATION module=“INF” time=“8:16” situationID=“leavingHome058” instanceOf=“leavingHome” location=“home” proximity=“zero”> MyDiary output: La 8:16 am ieșit din casă și am coborât scările.

Exemplu TIME=8:17, ACCELERATION = zero, BLUETOOTH = in_car; GPS = movement(speed<5 km/h) MyDiary output: La 8:17 am scos maşina din garaj. MyDiary reasoning: Am plecat cu mașina ?undeva.

Communication standards: Movement & Location <COMM module=“MOV” time=“…” movement=“…”> <COMM module=“LOC” time=“8:17:00” location=“myCar”> <COMM module=“MOV” time=“8:17:10” movement=“inCar” speed=“<5”> TIME=8:17, ACCELERATION = zero, BLUETOOTH = in_my_car; GPS = movement(speed<5 km/h)

Communication standards: Situation Representation <SITUATION module=“INF” time=“…” situationID=“…” instanceOf=“…” location=“…” proximity=“…” speed=“…”> <SITUATION module=“INF” time=“8:16” situationID=“gettingCarOutOfGarage060” instanceOf=“gettingCarOutOfGarage” location=“myCar” proximity=“zero” speed=“<5”> MyDiary output: La 8:17 am scos maşina din garaj.

Exemplu INTERVAL=8:17-8:28, ACCELERATION = car, BLUETOOTH = in_car; GPS = movement(speed<50 km/h), GEO_PATH = {street_S1, street_S2, street_S3} TIME=8:28, ACCELERATION = walking, BLUETOOTH = zero, GPS = movement(speed<5 km/h), GEO_POSITION = location_L1 MyDiary output: Între 8:17 și 8:28 am condus maşina de acasă până la location_L1.

Communication standards: Movement & Location <COMM module=“MOV” time=“…” movement=“…”> <COMM module=“MOV” time=“8:17:00” movement=“inCar”> <COMM module=“LOC” time=“8:17:10” location=“inMyCar” speed=“<50” geo_loc=“street_S1”> <COMM module=“LOC” time=“8:22:10” location=“inMyCar” speed=“<50” geo_loc=“street_S2”> INTERVAL=8:17:00-8:17:10, ACCELERATION = car, BLUETOOTH = in_myCar; GPS = movement(speed<50 km/h), GEO_PATH = {street_S1, street_S2}

Communication standards: Movement & Location <COMM module=“MOV” time=“…” movement=“…”> <COMM module=“MOV” time=“8:28” movement=“walking” speed=“<5”> <COMM module=“LOC” time=“8:28” location=“location_L1”> TIME=8:28, ACCELERATION = walking, BLUETOOTH = zero, GPS = movement(speed<5 km/h), GEO_POSITION = location_L1

Communication standards: Situation Representation <SITUATION module=“INF” time=“…” situationID=“…” instanceOf=“…” location=“…” proximity=“…”> <SITUATION module=“INF” time=“8:17” situationID=“drivingCar061” instanceOf=“drivingCar” location=“myCar” proximity=“zero” geo_loc=“street_S1 street_S2 location_L1”> MyDiary output: Între 8:17 și 8:28 am condus maşina de acasă până la location_L1.

Despre recunoașterea activităților H. Kautz. "A formal theory of plan recognition". PhD thesis, University of Rochester, 1987. N. Lesh and O. Etzioni. "A sound and fast goal recognizer". In Proceedings of the International Joint Conference on Artificial Intelligence, 1995.