1. Design la proiectul MyDiary
Cursuri 7-8
Design la proiectul MyDiary

2. Proiectul nostru: MyDiary
Un agent capabil să “înțeleagă” situațiile în care se află Masterul său, să le transpună în text și să conducă un dialog relativ la ele

3. Ce vrem să facă sistemul?
Să “știe”, în cât mai multe momente ale zilei:
ce fac
unde mă aflu
cu cine mă întâlnesc
despre ce vorbesc
cine îmi scrie mesaje (?)…
de ce fac anumite lucruri
Să fie capabil să formeze fraze care exprimă ce am făcut
Să fie capabil să poarte un dialog
despre ziua de azi, cea de ieri
cea de mâine (?)

4. Un graf al situațiilor – ierarhie de clase
sleeping
awake
at_home
moving
in_activity
alive
in_theater
in_park
shopping
not_moving
driving
walking
jogging
driving_in_town
driving_outside

5. Module 1. LOCALISATION 2. MOVEMENT 3. FACES 4. SOUNDS&VOICES
5. INFERENCES
6. BEHAVIOUR
7. TEXT GENERATION
8. QUESTION-ANSWERING
9. INTERFACES
10. CLIENT-SERVER
11. STANDARDS AND EVALUATION

6. MyDiary – schema generală
Activities graph
Behaviour
sensors
Text generation
Diary
repr.
Localisation
Movement
Inferences
Question answering
Diary
Faces
MyDiary
Sounds&
voices

7. Interfețe și comunicații
MyDiary
Localisation
Inferences
COMM
GPS
activități zilnice
Movement
COMM
vibrații
Faces
imagini
COMM
Sounds/Voices
sunete
COMM

8. Interfețe și comunicații
Diary
repr.
Inferences
Text generation
SIT
SIT
Diary NL
query
QUE
dialogul de seară
MyDiary
Question answering NL

9. Interfețe și comunicații
Diary
repr.
Inferences
Text generation
SIT
SIT NL
answer
SIT
dialogul de seară
MyDiary
Question answering NL
Diary

10. Exemplu (din cursul 5)
TIME=7:30, PROXIMITY = bedroom, CLOCK = awake_time, SOUND = awake_ringing, GG:LIGHT = intensity_raise, WiFi_WAVE = movement, ACCELERATION = zero, GPS = null, GEO_POSITION = home(remember_last_position)
MyDiary output: Astă dimineață am fost trezit de alarmă la ora 7:30 în dormitorul meu.

11. Communication standards: Localisation
<COMM module=“LOC” time=“…” location=“…” proximity=“…”> <COMM module=“LOC” time=“7:30” location=“home” proximity=“bedroom”>
TIME=7:30, PROXIMITY = bedroom, CLOCK = awake_time, SOUND = awake_ringing, GG:LIGHT = intensity_raise, WiFi_WAVE = movement, ACCELERATION = zero, GPS = null, GEO_POSITION = home(remember_last_position)

12. Communication standards: Movement
<COMM module=“MOV” time=“…” movement=“…”> <COMM module=“MOV” time=“7:30” movement=“zero”>
TIME=7:30, PROXIMITY = bedroom, CLOCK = awake_time, SOUND = awake_ringing, GG:LIGHT = intensity_raise, WiFi_WAVE = movement, ACCELERATION = zero, GPS = null, GEO_POSITION = home(remember_last_position)

13. Communication standards: Faces
<COMM module=“FAC” time=“…” face=“…”> <COMM module=“FAC” time=“7:30” face=“zero”>
TIME=7:30, PROXIMITY = bedroom, CLOCK = awake_time, SOUND = awake_ringing, GG:LIGHT = intensity_raise, WiFi_WAVE = movement, ACCELERATION = zero, GPS = null, GEO_POSITION = home(remember_last_position)

14. Communication standards: Sounds&Voices
<COMM module=“S&V” time=“…” sound=“…” voice=“…”> <COMM module=“S&V” time=“7:30” sound=“awakeRinking” voice=“zero”>
TIME=7:30, PROXIMITY = bedroom, CLOCK = awake_time, SOUND = awake_ringing, GG:LIGHT = intensity_raise, WiFi_WAVE = movement, ACCELERATION = zero, GPS = null, GEO_POSITION = home(remember_last_position)

15. Communication standards: Situation Representation
<SITUATION module=“INF” time=“…” situationID=“…” instanceOf=“…” location=“…” proximity=“…”> <SITUATION module=“INF” time=“2014/11/03/23:18” situationID=“sleeping048” instanceOf=“sleeping” location=“home” proximity=“bedroom”> <SITUATION module=“INF” time=“2014/11/04/7:30” situationID=“awake049” instanceOf=“awake” location=“home” proximity=“bedroom”>
MyDiary output: Astă dimineață am fost trezit de alarmă la ora 7:30 în dormitorul meu.

16. Exemplu TIME=7:45, PROXIMITY = bedroom, ACCELERATION = unknown
INTERVAL_END=7:46, ACCELERATION = walking,
TIME=7:46, ACCELERATION = unknown, PROXIMITY = kitchen
MyDiary output: La 7:46 am intrat în bucătărie.
MyDiary reasoning: Ce am făcut între 7:30 și 7:46?

17. Communication standards: Movement & Location
<COMM module=“MOV” time=“…” movement=“…”> <COMM module=“MOV” time=“7:45:03” movement=“unknown”> <COMM module=“MOV” time=“7:46:15” movement=“walking”> <COMM module=“MOV” time=“7:46:25” movement=“unknown”> <COMM module=“LOC” time=“7:46:25” proximity=“kitchen”>
TIME=7:45, PROXIMITY = bedroom, ACCELERATION = unknown
INTERVAL_END=7:46, ACCELERATION = walking,
TIME=7:46, ACCELERATION = unknown, PROXIMITY = kitchen

18. Communication standards: Situation Representation
<SITUATION module=“INF” time=“…” situationID=“…” instanceOf=“…” location=“…” proximity=“…”> <SITUATION module=“INF” time=“2014/11/03/7:46” situationID=“atHome050” instanceOf=“atHome” location=“home” proximity=“kitchen”>
MyDiary output: La 7:46 am intrat în bucătărie.

19. Exemplu
TIME=8:16, MO:BAROMETER = change_low2high(20m); ACCELERATION = steps, PROXIMITY = lost
MyDiary output: La 8:16 am ieșit din casă și am coborât scările.
MyDiary reasoning: Am intenționat să merg ?undeva.

20. Communication standards: Movement & Location
<COMM module=“MOV” time=“…” movement=“…”> <COMM module=“MOV” time=“8:16” movement=“descending”> <COMM module=“LOC” time=“8:16” location=“home” proximity=“zero”>
TIME=8:16, MO:BAROMETER = change_low2high(20m); ACCELERATION = steps, PROXIMITY = lost

21. Communication standards: Situation Representation
<SITUATION module=“INF” time=“…” situationID=“…” instanceOf=“…” location=“…” proximity=“…”> <SITUATION module=“INF” time=“8:16” situationID=“leavingHome058” instanceOf=“leavingHome” location=“home” proximity=“zero”>
MyDiary output: La 8:16 am ieșit din casă și am coborât scările.

22. Exemplu
TIME=8:17, ACCELERATION = zero, BLUETOOTH = in_car; GPS = movement(speed<5 km/h)
MyDiary output: La 8:17 am scos maşina din garaj.
MyDiary reasoning: Am plecat cu mașina ?undeva.

23. Communication standards: Movement & Location
<COMM module=“MOV” time=“…” movement=“…”> <COMM module=“LOC” time=“8:17:00” location=“myCar”> <COMM module=“MOV” time=“8:17:10” movement=“inCar” speed=“<5”>
TIME=8:17, ACCELERATION = zero, BLUETOOTH = in_my_car; GPS = movement(speed<5 km/h)

24. Communication standards: Situation Representation
<SITUATION module=“INF” time=“…” situationID=“…” instanceOf=“…” location=“…” proximity=“…” speed=“…”> <SITUATION module=“INF” time=“8:16” situationID=“gettingCarOutOfGarage060” instanceOf=“gettingCarOutOfGarage” location=“myCar” proximity=“zero” speed=“<5”>
MyDiary output: La 8:17 am scos maşina din garaj.

25. Exemplu
INTERVAL=8:17-8:28, ACCELERATION = car, BLUETOOTH = in_car; GPS = movement(speed<50 km/h), GEO_PATH = {street_S1, street_S2, street_S3}
TIME=8:28, ACCELERATION = walking, BLUETOOTH = zero, GPS = movement(speed<5 km/h),
GEO_POSITION = location_L1
MyDiary output: Între 8:17 și 8:28 am condus maşina de acasă până la location_L1.

26. Communication standards: Movement & Location
<COMM module=“MOV” time=“…” movement=“…”> <COMM module=“MOV” time=“8:17:00” movement=“inCar”> <COMM module=“LOC” time=“8:17:10” location=“inMyCar” speed=“<50” geo_loc=“street_S1”> <COMM module=“LOC” time=“8:22:10” location=“inMyCar” speed=“<50” geo_loc=“street_S2”>
INTERVAL=8:17:00-8:17:10, ACCELERATION = car, BLUETOOTH = in_myCar; GPS = movement(speed<50 km/h), GEO_PATH = {street_S1, street_S2}

27. Communication standards: Movement & Location
<COMM module=“MOV” time=“…” movement=“…”> <COMM module=“MOV” time=“8:28” movement=“walking” speed=“<5”> <COMM module=“LOC” time=“8:28” location=“location_L1”>
TIME=8:28, ACCELERATION = walking, BLUETOOTH = zero, GPS = movement(speed<5 km/h),
GEO_POSITION = location_L1

28. Communication standards: Situation Representation
<SITUATION module=“INF” time=“…” situationID=“…” instanceOf=“…” location=“…” proximity=“…”> <SITUATION module=“INF” time=“8:17” situationID=“drivingCar061” instanceOf=“drivingCar” location=“myCar” proximity=“zero” geo_loc=“street_S1 street_S2 location_L1”>
MyDiary output: Între 8:17 și 8:28 am condus maşina de acasă până la location_L1.

29. Despre recunoașterea activităților
H. Kautz. "A formal theory of plan recognition". PhD thesis, University of Rochester, 1987.
N. Lesh and O. Etzioni. "A sound and fast goal recognizer". In Proceedings of the International Joint Conference on Artificial Intelligence, 1995.