1. GRASP Management Meeting March 09 -2009 Munich

2. Agenda
EU reporting
Comments on the deliverables
Plan for integration
Lunch
Plan for review
Plan for integration
Wrap up

3. EU reporting
A periodic report within 60 days of the end of each reporting period . The periodic report comprises:
a)      An overview, including a publishable summary of the progress of work towards the objectives of the project, including achievements and attainment of any milestones and deliverables identified in Annex I. This report should include the differences between work expected to be carried out in accordance with Annex I and that actually carried out,
b)      An explanation of the use of the resources, and
c)    A Financial Statement from each beneficiary and each third party, if applicable, together with a summary financial report consolidating the claimed Community contribution of all the beneficiaries (and third parties) in an aggregate form, based on the information provided in Form C by each beneficiary. 

4. Parts of PPR
Publishable summary: max 4 pages, should be publishable by the Commission: project objectives and work done; should be updated every reporting period
Project objectives for the period: short overview of objectives as indicated in Annex1
Work progress and achievements: progress overview and relation to the state of the art; work packages progress with main results and deviations
Deliverable and milestones tables

5. Parts of PPR Project management Management activities
Problems and changes
List of project meetings, venues
Deviations from the milestones
Cooperation and collaboration (also with other projects)

6. Parts of PPR Dissemination Website List of publications
List of events: organized and planned for the next period
List of exploitable results

7. What we promised?

8. What we promised?
1. Month 12: Definition and implementation of initial ontology:
We will provide a description and a model of how selective visual processing is affected by object properties, task and object knowledge (D2). This will make basis for the definition of the initial ontology (D4). First prototype of human body detection and tracking will be developed (D3). A model of the integration of the individual modules will be given (D8) and will consider the integration of the control strategies (D5), the definitions of grasping primitives (D5) and modules for visual attention (D6). We will be able to demonstrate open-loop grasping and object manipulation strategies on several platforms using stereo based representations of object attributes.
Planned prototypes and demonstrations:
A first operational prototype of models for pre-reasoning and visual attention (D6).
Demonstration on the humanoid head for monitoring of the environment for surprises (D7).
Demonstration of the initial hand/body tracking system on the humanoid head (D3).

9. Comments on the deliverables

10. Plan for integration
GRASP ’image’
Scenario(s)
Who/What/When

11. Learning and abstraction Execution and Grounding
Human data:
Learning and abstraction
WP1, WP2
LEARNING LOOP
EMULATION
WP2, WP4
Control cycle:
Execution and Grounding
World/Simulator
Planning/Reasoning
”Mental model”
WP2, WP7
generation
Action
Perception
INTROSPECTION
WP3, WP6,
WP7
WP4, WP5
SURPRISE
MISSION LOOP
Task generation

12. Symbolic representation
Task-level planning
WP4, WP5
WP2, WP7
WP3, WP6,
WP7
Off-line Perception:
Vision and Haptics
Memory system
Symbolic representation
Embodiment specific attributes
Planning
Execution and Prediction
World/Simulator
WP2, WP7, WP4, WP3?
Reasoning
Low-level or feature level
WP2, WP3?, WP5?
Human grasp: Learning and abstraction
Sensorimotor
experience
WP1, WP2

13. Some issues Scenario in GRASP Libraries in GRASP Vision in GRASP
Haptics in GRASP
Hand models (robot and human)
Objects in GRASP
Knowledge representation in GRASP: objects, actions

14. Scenario year 1
Handling some the of 8 objects on the table (not in the basket)
WP1: observe a human grasping one of these objects and provide the tracked 3D model of the hand and a classification of the type of grasp
Note: A grasp is defined by:
Grasp type
Grasp starting point
Approaching direction
Hand orientation
WP1 (Heiner): provide kinematics of the grasps, grasping points and covert and overt attention (see Daniel’s presentation this afternoon)

15. Scenario year 1
Handling some the 8 objects on the table (not in the basket)
WP2: Discrete mapping of observed human grasp activities to one/two hand robots. Extract DMPs from the observed movement, Representations for integration
WP3: Demonstrate the grasping cycle using the grasp types form WP2, object type and attributes from WP4 at the UJI platform
WP4: Background/foreground segmentation in the case of textured objects, grasp points generation, pose of the object (6D) (bounding boxes), Identify the primitive shapes (3D model fitting)
WP5: definition of the expectation model necessary for detecting surprise

16. Scenario year 1
Handling some the 8 objects on the table (not in the basket)
WP6:
Initial version of the simulator
Integration of COLLADA and PAL to OpenRAVE
Reproduction of what have been demonstrated in WP1, Mapping from WP2, location form WP4 on ARMAR in OpenRAVE
WP7:
Proposal for integration for the described scenario with focus on how to represent objects, actions (MMM, DMPs), input/output definitions, OpenRAVE/MCA
Reproduction of grasping cylindrical textured objects on ARMAR using the 6D pose from WP4

17. Objects (1) Object representations via meshes in all workpackages
Cylinder-like and box-like objects
GRASP objects (8 items)
Boxed salt (SFB 588), object ID 2
Cylindrical salt (SFB 588), object ID 3
Gauloises red
Zwieback (SFB 588), object ID 11
Cups (Dani’s cups, i4280.JPG)
Two different cups (two each to generate textured and no-textured)
Complete representation of one object
Meshes
Stereo

18. Objects (2) Original monocular images (10 views) Darius
Stereo images (5 views) for Markus, Lech
Stereo information (Markus, Lech)
Internal and external calibration
(Depth map for the 5 stereo views )
Who will provide the meshes for these objects (UniKarl)
Geometrical models of the objects are also needed

19. Input-output for scenario year 1
WP1
Input: Human experiments from WP1 (LMU, Daniel)  FORTH
Output:
Grasp type (1 of 4) and approaching vector (FORTH, Antonis, Georgios)
(Human grasping library)
WP2
Input: Human Experiments from WP1 (LMU, Daniel)
Grasp Ontology, i.e. hierarchy of human hand postures, discrete mapping to Barrett and Karlsruhe hand; approach vector relation to the objects in the database (KTH, OB, Dani, Dan and Thomas)
Representation of humans grasps using DMPs (Martin, Tamim)

20. Input-output for scenario year 1
WP3:
Input: Results form KTH (Output WP2), 6D pose of grasp objects (WP4, Chavdar), object type (WP4, Chavdar)
Output: grasping cycle on the UIJ platform (UJI, Javier) and (LUT, Janne)
WP4:
Input: Stereo-images from UJI plus calibration data (UJI, Antonio)
Output: 2.5 point cloud + mesh (TUW, Lech and Mario); 6D object pose estimation (TUM, Chavdar, TUW, Markus and Lech); remote distributed computing)
WP5:
Input: Ontology hierarchy from WP2 (Maria, Dan and Thomas), object and scene representation from WP4 (TUM, Darius)
Output: Stereo sequences of humans manipulation

21. Input-output for scenario year 1
WP6
Input:
Results from KTH (Output WP2), 6D pose of grasp objects (WP4, Chavdar), object type (WP4, Chavdar)
ARMAR controller OpenRAVE Plugin (UniKarl, Stefan, Markus)
Output:
Execution of observed movements (WP1) in the Simulation (OpenRAVE in its original version) on ARMAR
(Demonstration of collision detection for the introspection detection, i.e. collision with the other 7 objects on the table, UniKarl, Tamim)
WP7:
Input: grasp ontology (KTH), action representation (UniKarl, KTH), objects representations (TUW, Markus), IO from all other WPs (see above) ; models of prediction (WP2, WP5, ????)
Output: Demonstrate integration of OpenRAVE and MCA (UniKarl, Stefan, Tamim) and the predict-act-perceive cycle on ARMAR (UniKarl, Markus, Tamim).

22. Representations of object, action and surprise
COLLADA file
ID, category, shape, mesh, weight, material, inertia, CoM
Grasp types
Action
Vocabulary of actions
Reach (6D pose)
Pre-shape (grasp type)
Grasp (approach vector, hand orientation, grip forces)
Lift (move in Cartesian space)
Transport (move in Cartesian space)
Place (move in Cartesian space, contact force)
Release (open hand)
Representations of object, action, …. and surprise
Object-action Complexes (OAC)
Embodiment specific and embodiment invariant

23. When/Who/What

24. Review meeting I Spain, June 9-10 (Tuesday-Wednesday) Reviewers
Everybody arrives late on 7th (Sunday) or before 10 on 8th
Final preparations on 8th
Reviewers
Yiannis Demiris: cognitive architectures, mapping/representations
Peter Allen: grasping, simulation
Carlos Balaguer: control
June 9th: WP presentations
June 10th: WP presentations, demosntrations, reviewers’ meeting

25. Plan for review Presentations Demos:
Format and requirements will be sent out
Demos:

26. Example

27. Input/Output definitions

28. Learning/Mission PREDICT- ACT – PERCEIVE Learning: Mission: Predict:
Predict: simulator as the forward model
Mission:

29. Detailed plan for collaboration

30. To do list Images: DEADLINE: April, 30th May, 20th – convergence ……
Tamim/Antonio/Ville: WP6, WP7 -> Markus (System integration, achitecture, simulator)
Dani/Darius/Tamim: WP2, WP5 -> Memory
Markus, Darius, Ville: WP4, WP5, WP3 -> Surprise detection (fore learning and for mission) – 2 figures
Antonis, Heiner, Dani, Tamim: Human grasping loop (motion primitives, actions, skills, task)
DEADLINE: April, 30th
May, 20th – convergence ……
June