1. WP 7: Management Stipulated versus actual work
Heiner Löllmann and Walter Kellermann
EARS SB Meeting in Erlangen
December, 2016

2. WP 7: Quality Management
Purpose
Final Review
Comparison between Description of Work (DoW) and performed work by the reviewers can be expected
Deviations must be reasoned by the consortium (as part of the project quality management)
Aim of this presentation
Overview of performed tasks versus planned tasks (according to the DoW and the submitted periodic reports)
Explanations for deviations
Submitted Deliverables are not considered
WP 7: Quality Management

3. WP 1: Embodied Acoustic Sensing for Real-world Environments
Tasks
T1.1 Design of head-embodied anthropomorphic microphone array (M1-M27)
BGU (16 PM): Work on optimal microphone positions and prototype head 
FAU (3 PM): FAU has evaluated the optimum array configurations for speech enhancement (Master thesis Oliveria) 
T1.2 Design of adaptive robomorphic microphone array (M1-M27)
FAU (6 PM): Concept developed (IWAENC 2014 paper) 
UBER (6 PM): joint work with FAU on the robomorphic array, data synchronization for the prototype head 
T1.3 Active sensing through sensorimotor interaction (M13-M36)
UBER (12 PM): Investigations into optimal robot behaviors and their influence on perception in cooperation with BGU 
BGU (6 PM): DOA estimation using microphone arrays of humanoid robots (TASLP paper), relation between spatial information and naturaless of robot motions 
FAU (3PM)): Joint work with BGU, head and robomorphic array are combined for improved source localization 
WP 7: Quality Management

4. WP 1: Embodied Acoustic Sensing for Real-world Environments
Tasks
T1.4 Sound field representation and analysis (M1-M30)
BGU (12PM): thorough investigation of SH representation for sound fields 
T1.5 Audio-visual data alignment (M1-M27)
INRIA (6 PM): work on local relative transfer function (RTF), localization exploiting the local-RTF, work on data synchronization 
BGU (6 PM): work on source localization at the code camps, some efforts have been shifted from T1.5 to T2.1 (localization and mapping) ()
Additional efforts for T1.5
work of UBER on the ego-sphere to merge audio and visual tracking and localization estimates
MAP data object of IMPERIAL
WP 7: Quality Management

5. WP 2: Acoustic Scene Analysis
Tasks
T.2.1 Acoustic source localisation and environment mapping (M1-M30)
IMPERIAL (21 PM): extensive work on SLAM (by Christine), PIV-based localization (Alastair) 
FAU (6 PM): Generalization of RTF and its use for multiple-source localization (A. Deleforge, EUSIPCO 2015) 
ALD (5 PM): support in implementing the DOA estimation algorithms of IMPERIAL and BGU on the prototype system ()
T2.2 Focussing by adaptive robomorphic arrays (M7-M30)
FAU (12 PM): development, implementation and evaluation of the robomorphic array (ongoing work and beyond expectation) ()
T2.3 Acoustic source and environment tracking (M13-M21)
FAU (6 PM): addressed mainly by IMPERIAL, efforts of FAU shifted towards T2.6 to account for the unforeseen task of ego-noise suppression ()
IMPERIAL (6 PM): PIV-based estimation (Alastiar, should be moved to T2.1), and SLAM methods (Christine) 
WP 7: Quality Management

6. WP 2: Acoustic Scene Analysis
Tasks
T2.4 Spatial filtering (M1-M30)
FAU (9 PM): HRTF-based BF designs 
BGU (5 PM): SH-based BF designs 
ALD (5 PM): support in implementing the algorithms on the prototype system ()
T2.5 Acoustic echo cancellation (M13-M30)
FAU (6 PM): evaluation of joint AEC and Beamforming for NAO (DAGA 2015 paper), AEC implementation for prototype system 
T2.6 Multichannel noise reduction and interference suppression (M13-M36)
FAU (9 PM): ego-noise reduction by dictionary learning (Antoine & Alex) 
BGU (5 PM): SH-domain LCMV beamformer 
T2.7 Robust dereverberation for robot audition and interaction (M1-M30,
IMPERIAL (30 PM): linear prediction- and SH-based dereverberation 
FAU (6 PM): CDR-based post-filter used for CHiME-Challenge and prototype 
WP 7: Quality Management

7. WP 2: Acoustic Scene Analysis
Additional work not planned in the DoW
T2.4 Spatial filtering
INRIA: probabilistic framework for source separation !
WP 7: Quality Management

8. WP 3: Audio-visual Disambiguation
Tasks
T3.1 Extraction of 3D visual localisation and recognition cues (M7-M24)
INRIA (12 PM)): work on face detection, face-pose estimation, multiple-person tracking 
T3.2 3D Audio-visual event localisation (M13-M30)
INRIA (12 PM): audio-visual fusion based on cross-modal filtering, audio-visual localization 
ALD (5 PM): Implementing and evaluating the visual tracker of INRIA on NAO 
T3.3 Audio-visual event classification (M24-M36)
INRIA (12 PM): work on characterization of a scene by multiple audio events 
ALD (5PM): Development and integration of sound event recognition system ()
WP 7: Quality Management

9. WP 4: Robot Embodied Cognition and Interaction
Tasks
T4.1 Learning internal models for robot interaction and prediction of sensory and motor states (M1-M36)
UBER (28 PM): model for internal body representations 
T4.2 Attention system (M7-M24)
UBER (6 PM): work on the ego-sphere 
INRIA (2 PM): focus on other tasks, integration of tracking system in the ego-sphere 
T4.3 Optimal behaviours for event recognition and localisation (M1-M30)
UBER (8 PM): investigation for exploration strategies like global babbling, collaboration with BGU on investigating robot motions from HRI and acoustic perspective 
INRIA (3 PM): Visual tracking with controlled motions of the robot head ()
WP 7: Quality Management

10. WP 4: Robot Embodied Cognition and Interaction
Tasks
T4.4 Integrate knowledge from voice dialogue (M1-M12)
ALD (4 PM): work on dialogue manager, detection of desired speaker by dialogue manager (?) ()
T4.5 Interaction initiated by the robot (M7-M30)
UBER (6 PM): work on the attention system (ego-sphere) and interaction of ego-sphere with dialogue manager and tracking system ()
ALD (6 PM): work on event-triggered robot actions ()
INRIA (4 PM): person tracker that embeds the control of head movements ()
task not really fulfilled as the robot reacts only, which should be clarified in the final presentation and reports
WP 7: Quality Management

11. WP 5: System Integration and Validation
Tasks
T5.1 Design of software architecture for audio integration (M1-M6)
ALD (6 PM): accomplished but mostly done by UBER ()
T5.2 Integration of microphone arrays into Nao (M13-M30) ALD (6 PM), BGU (5 PM), FAU (1 PM)
accomplished by mainly by ALD and BGU with support by FAU (as planned) 
but additional efforts by UBER, INRIA and FAU !
T5.3 Software Integration (M13-M30), ALD (8 PM), FAU (1 PM), IMPERIAL (1 PM), BGU (1 PM), UBER (1 PM), INRIA)
accomplished but with much more efforts by all Partners than originally planned 
T5.4 Development of the applications and evaluation (M1-M36, ALD (11 PM), FAU (1 PM), IMPERIAL (2 PM), BGU (2 PM), UBER (2 PM), INRIA)
mostly accomplished (ongoing work) ()
lack of time for the evaluation due to unexpected implementation problems, which also reasons the vague numbers (in Table 7.1)
WP 7: Quality Management

12. WP 6: Dissemination and Exploitation
Tasks
T6.1 Dissemination activities (M1-M36, FAU (4 PM), IMPERIAL (2 PM), BGU (1 PM), UBER (1 PM), INRIA (1 PM), ALD (2 PM))
achieved according to DoW (-> presentation on WP 6) 
T6.2 Robot Audition Community Management (M1-M36, FAU (3 PM), IMPERIAL (1 PM), BGU (1 PM), UBER (1PM), INRIA (1 PM), ALD (1 PM))
achieved according to DoW (presentation on WP 6) 
T6.3 Exploitation (M1-M36, ALD (3 PM), FAU (3 PM))
partly achieved ()
some results incorporated in commercial NAO
outlook on exploitation provided by ALD
no patent application (since not deemed to be worth the money) and algorithms cannot be patented
WP 7: Quality Management

13. WP 7: Quality Management
Conclusions
Most of the planned tasks have been tackled according to plan!
Some efforts have been rescheduled to address task not foreseen in the DoW such as
ego-noise suppression
data synchronization
Additional work has been conducted
INRIA for WP2 (source separation)
All Partners for hardware and software integration(!)
Next steps
Revision of the slides based on the Partners’ feedback
Explanations for the deviations will be incorporated in the SB Annual Progress Report and the presentations for the Review Meeting
FAU provides a template slide for the Review Meeting
WP 7: Quality Management