September M4 meeting Delft1 VUT Brno meeting data collection for M4 Petr Jenderka, Stanislav Sumec, Igor Potůček, Petr Motlíček, Jan Černocký, Pavel Zemčík Brno University of Technology, Czech Republic

September M4 meeting Delft2 Goals Provide M4 and follow-ups with more data, complementary to ICSI and IDIAP Different speakers, different accent (heavily accented CZEnglish) Setup of a secondary M4 meeting room: low-cost, mobile, built from off-the-shelve products, easy to install and operate. Plans for meeting recordings Plans for annotations.

September M4 meeting Delft3 The rooms IDIAP room very valuable but expensive, difficult to assemble/setup and impossible to bring outside IDIAP Our meeting room: recordings of up to 4 persons 4 lapel mikes, no mike arrays one camera with hyperbolic mirror allowing for almost 360 degree view. audio/video recording on one notebook. Need for cheaper, lighter, and “buyable” room

September M4 meeting Delft4 Setup

September M4 meeting Delft5 4 people around a mirror

September M4 meeting Delft6

September M4 meeting Delft7 Original wrapped image

September M4 meeting Delft8 Hardware 4 lapel microphones Sennheiser MKE 2 P-C (same as in IDIAP) 2 mixers Behringer EURORACK UB1204-PRO providing gain control and phantom power supply (actually 90% of mixers are not used, but they’re cheaper than special mike pre-amps with phantom feeders…) two Hi-Fi PC-MCIA sound-cards VX-pocket (we know them from the SpeeCon project)

September M4 meeting Delft9 Hardware (2) digital camera SONY CLIP 345 hyperbolic mirror by NEOVISION (Prague) FireWire interface The camera provides 2 additional (global) audio channels. The cost: 4.2 kEUR w/o the notebook

September M4 meeting Delft10 Recording a home-made (S. Sumec) tool, based on Direct- show libraries, under Windows. post-processing tool (I. Potúček) for unwrapping video to panoramic format of view. All audio and video streams are stored in real time: two audio wav files with four audio channels, one DV-compressed video avi file with video and stereo audio stream. Plan for M4: >30 hours of meetings

September M4 meeting Delft11 Unwrapped image

September M4 meeting Delft12 Annotations – major meeting events annotations using the „annotation editor“ by Staňa Sumec (audio, video, hotkeys, XML). This annotation will be done internally by pre- and post-grads at UPGM. Plan: >20 hours

September M4 meeting Delft13 Annotations - speech ETCA Transcriber with ICSI MultiChannel extension. An annotator hired for speech annotations, just beginning to be operational. Plan: >10 hours of speech annotations. Both annotations: quality check protocols? (quite an issue in our DB projects SpeechDat-E and SpeeCon!)

September M4 meeting Delft14 Current work and future Recording, annotations, integration to the Media Server mmm.idiap.ch Still quite some issues to solve (position of the mirror, lighting in the room – what can be done on the resulting video?) Extensions – PowerPoint and Acrobat Reader slide/change timing grabber, T0 must be synced with the recording software … and research with the data (phoneme recognition, mouth area detection, speaker activity detection) – where does the simplified setup degrade performances?

September M4 meeting Delft15 Questions, comments ? THE DEMO Please have a look during the break :-)