1. The Big ASC RA Training Friday 17 – Saturday 18 June 2011 UWS

2. Programme for Friday 17/06/11 09:00 Welcome – Paperwork for casual payment (to be returned at Morning tea) 09:30 Introduction (Denis Burnham / Dominique Estival) Overview of the Big ASC project Responsibilities of the recording RA 10:00The Black Box (Steven Fazio / Lei Jing): Unpacking and setting up 11:00 Morning tea 11:30The data collection protocol (Dominique Estival / Lei Jing) Setting up and starting the SSCP / Video and audio calibration 13:00Lunch (UWS) 13:30Running a recording session (Dominique Estival) 14:00Practice Session 1 15:30 Afternoon tea 16:00 Interview and Story re-retelling (John Ingram): Practice Session 2 17:30 Pack up the Black Boxes 18:30 Dinner at The Mill

3. Programme for Saturday 18/06/11 09:30 Unpack and set up the Black Boxes 10:00The Map Task (Dominique / Denis / Maria): Practice Session 3 11:00 Morning tea 11:30Recruitment, keeping records, etc. (Dominique Estival / Denis Burnham) 13:00Lunch (Bankstown Golf Club) 14:00Data saving and data upload (Lei Jing / Steve Cassidy) 15:00Questions? More practice 16:00 Pack up the Black Boxes 17:00Departure

4. Introduction

5. The Big ASC: Introduction Linkage Infrastructure & Equipment Fund (LIEF) 13 partners - UWS lead institution Who are we? Denis Burnham – MARCS – Lead CI Dominique Estival – MARCS – Project officer Steven Fazio – MARCS – Technical officer Lei Jing – MARCS – Software engineer Maria Christou – Expert Assistant

6. The Big ASC University of Western Sydney Macquarie University The Australian National University The Flinders University of South Australia The University of Melbourne The University of New South Wales The University of Queensland The University of Sydney The University of Western Australia University of Canberra University of Tasmania Australian Speech Science and Technology Association Max Planck Institute for Psycholinguistics

7. RA and IT-support from partner institutions MQ: Greg Flannery UNE: Mark Conroy UNSW: Ronny Kurniawan Sydney U: Sarah de Launey UQ: Nahyun Kwon, John Ingram Melbourne U: Joshua Clothier, Cara Penry-Williams Flinders: Marissa Milne, Hywel Stoakes, Tom Anderson UWA: Damien Pontifex, Serajul Haque U Tas: Rosie Maunder UC: Sharifa Alghowinem ANU: Katharine Gosling, Geoff Collin CSU/Bathurst: Rebekah Lockart + Catherine Watson from Auckland U

8. The Big ASC: aims and goals 1.Establish the infrastructure for the collection of large quantities of AV speech data. 2.Provide standardised recording equipment and protocols for data collection. 3.Collect AV data from many locations across multiple sessions per speaker. 4.Provide standardised annotation for (at least a subset of) the data. 5.Provide access to the annotated data and the meta-data via a centralised storage facility. AusTalk: not only a large Australian speech corpus, but also the only Australian corpus of AV speech in the public domain.  designed to cater to various clients with a variety of interests (Phonetics and Linguistics; Psycholinguistics; Language Technology; Speech Engineering; Speech pathology; Forensic Speech Science)

9. The Big ASC: Infrastructure Corpus Collection: Equipment and Protocol Standard Speech Science Infrastructure Black Box (SSSIBB) Standard Speech Collection Protocol (SSCP) - Session protocol - Software Implementation Corpus Distribution: Data Storage and Server

10. The Big ASC: Data Collection STATEUNINREGIONALNOTHERNTOTAL NSWUWS TOWNSVILLE48 MQ ARMIDALE48 UNSW48 EMOTION3684 USYD48 DISORDERED1664 QLDUQ120 VICMELB120GEELONG48 168 SAFLINDERS96AS/DARWIN48AUSAB48192 WAUWA96 TASUTAS48 ACTUC36BATHURST48 84 ANU48 TOTAL 660 240 1001000

11. Responsibilities of the RA from the Position Description for the job: Manage the recruitment of participants Set up the recording equipment Conduct the recording sessions Upload the data from the local computers to a central server.

12. Concretely.... Manage the recruitment of participants: Contact the participants who have registered through the website for your area. Advertise in your local uni, church, RSL, etc... if more are needed. Set up the recording equipment: Responsible for the BB and running it. Conduct the recording sessions: Set up a recording schedule, contact the participants, welcome them, ensure the questionnaire is completed and the consent form signed. Be patient and manage successful interactions with the public. Upload the data from the local computers to a central server: Responsible for saving the data at the end of each session, checking it and transferring it to the HDD at the end of the day.

13. The Black Box

14. Infrastructure for data collection Standard Speech Science Infrastructure Black Box (SSSIBB) Hardware: 1 computer, 1 digital audio acquisition device, 3 desktop microphones, 2 head-worn microphones, 2 stereo cameras packed together in a reinforced box which folds out into a table with integrated shelving  standardised: same equipment used at all locations, in the same configuration and with the same setup and spacing.  portable: data collection can be conducted in any location and can be extended to new locations as needed.

17. NoItemMakeModel NumberCompanySerial NoVendor T01-116U MIXER RACK WORKSTATION cases.com.au T01-21ru sliding shelf cases.com.au T01-32RU Rackmount Drawer cases.com.au T01-4Phillips Head Screwdriver Bunnings T01-5Flat Head Screwdriver Bunnings T01-6Head worn microphone 1AudioTechnica AT892c AudioTechnica Turra Music T01-7 xlr adapter 1 AT8539 Phantom Power / XLR adapter AudioTechnica Turra Music T01-8Head worn microphone 2AudioTechnica AT892c AudioTechnica Turra Music T01-9 xlr adapter 2 AT8539 Phantom Power / XLR adapter AudioTechnica Turra Music T01-10boundary micsBehringer c-2 mic array Mic 1 Behringer Turra Music T01-11boundary micsBehringer c-2 mic array Mic 2 Behringer Turra Music T01-12boundary mics stereo adapterBehringer c-2 mic array stand Behringer Turra Music T01-13xlr cable 1xlr budget cable Turra Music T01-14xlr cable 2xlr budget cable Turra Music T01-15Far-Field microphoneShure MX391/O Shure Turra Music T01-16far-Field microphone xlr adaptershure Turra Music T01-17 Stereo Camera 1 BumbleBee2 Pointgrey T01-18 Stereo Camera 2 BumbleBee2 Pointgrey T01-19 firewire cable camera 1 pointgrey custom cable Pointgrey T01-20 firewire cable camera 2 pointgrey custom cable Pointgrey T01-21 dual bus firwire pci express card pointgrey pci express card Pointgrey T01-22 GPIO Sync Cable Custom made GPIO to audio cable Flinders Uni T01-23Capture Computercore i7 rackmount 4ru Impact Custom Impact T01-24KeyboardMS Impact T01-25MouseMS Impact T01-26RA monitorDell E170S 17 inch Flat Panel Monitor Dell ITP T01-27Subject monitorDell E170S 17 inch Flat Panel Monitor Dell ITP T01-28Audio DAQM-Audio FastTrack Ultra8R M-Audio Turra Music T01-29power supply audio DAQ M-Audio Turra Music T01-30USB cable audio DAQD type USD cable M-Audio Turra Music T01-31serial cable audio DAQ M-Audio Turra Music T01-32External hard driveSamsung STORY Station 2TB Samsung ITP T01-33power supply external HDD Samsung ITP T01-34USB cable external HDD Samsung ITP T01-35Light MeterLux & Fc Light Meter Jaycar T01-36RA Head PhonesKOSS UR-20 headphones KOSS Turra Music T01-37RA Head Phones3.5MM STEREO SOCKET to 6.5MM STEREO PLUG adaptor KOSS Turra Music T01-38 Monitor arm / stand subject Atdec Visidec Focus MICRO LCD Single Arm, VF-M Atdec Pacific Notebooks T01-39 Monitor arm / stand tool Allen Key tool to fix bolts to stand Atdec Pacific Notebooks T01-40 Monitor arm / stand base plate black base plate Atdec Pacific Notebooks T01-41 Monitor arm / stand bolt 1 Long allen key bolt to mount the monitor stand arm to the table Atdec Pacific Notebooks T01-42 Monitor arm / stand bolt 2 Short allen key bolt to mount custom mounting plate Atdec Pacific Notebooks T01-43 Monitor arm / stand vesa bolts bag of VESA bolts to attach monitor Atdec Pacific Notebooks T01-44 Tripod mount for stero camera 1 jaycar QC3333 + custom mounting plate Jaycar T01-45 Tripod mount for stero camera 1B Manfrotto 700RC2 tripod head Manfrotto Dirt Cheap Cameras T01-46 quick shoe plate Manfrotto Dirt Cheap Cameras T01-47 Tripod mount for stero camera 2 Manfrotto 700RC2 tripod head Manfrotto Dirt Cheap Cameras T01-48quick shoe plateManfrotto Dirt Cheap Cameras T01-49umbrella light Main SubjectUmbrella Soft Ebay Hypop T01-50 Umbrella Refector Ebay Hypop T01-51 Tripod Ebay Hypop T01-52 Dual lamp adapter Ebay Hypop T01-53 65W lamps x4 Ebay Hypop T01-54 T01-55 T01-56 T01-57umbrella light maptask onlyUmbrella Soft Ebay Hypop T01-58 Umbrella Refector Ebay Hypop T01-59 Tripod Ebay Hypop T01-60 Dual lamp adapter Ebay Hypop T01-61 65W lamps x4 Ebay Hypop T01-62 T01-63 T01-64 T01-65Background / screen Main Subjectblue screen + base plate T01-66 carry bag T01-67 mounting pole 1 T01-68 mounting pole2 T01-69 Background / maptask only blue screen + base plate T01-70 carry bag T01-71 mounting pole 1 T01-72 mounting pole2 T01-73Partition for maptask400x500 3mm black acrylic board Lingel Lingel - Milperra T01-74Main Subject Chair T01-75Maptask Only Chair List in your pack

18. The Standard Protocol

19. Standard Corpus Components Isolated Words Digits Read Sentences Interview Story: reading/retelling Map Task Yes/No (Emotions)

20. Standard Protocol Session 1Session 2Session 3 TaskTimeTaskTimeTaskTime Calibration (+ 3D face) 10Calibration3 3 Opening Yes/No3 2 2 Words10Words10 Words10 Read Narrative5 Interview 15 Map Task (First run)20 Re-told Narrative10 Switch Sp.A and Sp.B 5 Read Digits5 5 Map Task (Second run) 20 Read Sentences8 Conversation 5 Words10 Closing Yes/No2 2 2 44 45 77

21. Running a session

22. Session Set-up 1. General setup – to be done every day before recording sessions - Measure location of light sources and distance to camera(s) (manual measurement) : 80cm between light and camera - Check Bumblebee camera output is being recorded - Check microphone output is being recorded - Audio and video recording of the recording environment (ambient noise and lighting profile) without a speaker present (60s) 2. Session setup – to be done before each session - Ensure the equipment is running: computer, camera, audio. - Ensure the 4 microphones are still plugged in. - Set chair for speaker and adjust background screen behind speaker

24. Calibration Camera setup Ensure face of speaker is within the red square. If necessary, adjust seating position. Check light level. If necessary adjust brightness. 3D procedure Speaker is on chair, facing the camera. Ask the speaker to turn 90° to the right. Turn speaker to every 15° starting from -90° (left profile) to +90° ( right profile), take 5s at each position (total = 135s). It is important that the speaker turns their head such that the face points in the required direction. Facial calibration While the speaker repeats these sequences, check on the M-Audio display that the levels are within the limits. Setting audio levels Adjust the levels for each microphone so they are set within the levels shown.

25. Session 1

26. Yes/No Questions Ask questions as casually as possible to elicit natural responses. The questions can be modified as necessary. The only requirement is that they be “Yes/No” questions, and that you can categorise the answer as positive or negative. 1) So let’s see, for the record, you are [Animal ID], right? 2) And is this your first recording session for the project? 3) Is there anything unclear with the instructions so far? 4) Would you like me to explain the process again? 5) Do you have any questions about what you need to do? 6) Can you see the screen? 7) Are you comfortable with the position of the chair and the microphone? 8) Shall we start then? After the response to each question, click on “Answer Yes” or “Answer No”, depending on the response. Then click on “Start” for the next question. If the yes/no answer to a question requires clarification, give your explanations before clicking “Start” for the next question.

27. Isolated Words A set of 323 words is presented one at a time on the screen. Different orders for S1, S2, S3. Ask the speaker to read each one silently before reading them aloud. Some are not real words, but made-up to elicit particular sound combinations. For these, we have provided a guide for pronunciation on the screen under the word.

28. hun sounds like fun

29. Words If the speaker makes a mistake: click “Pause” then when you are both ready to record again click “Start”.

30. Numbers A set of 12 strings of 4 digits is presented one string at a time on the screen. 1 2 3 4 We do not want: “one thousand two hundred and thirty-four” We want: “one two three four”

31. 0837:zero eight three seven

32. 942o: nine four two oh

33. Story reading Arthur the Rat 7 successive screens Ask the speaker to read each silently before reading aloud.

34. Story re-telling: suggestions Well, what did you think of the story of ‘Arthur the rat’? If they answer ‘a bit weird’, ask: In what way did it seem weird or odd? If they ‘liked the story’, ask: What did you like about it? Stories often convey a ‘moral’ or a lesson. What was the moral of this story, Arthur the Rat? And did you agree with it? Most kids like being told stories. Why do you think that is so? What sorts of stories used to appeal to you as a kid? Stories are often used to educate the young about their heritage or their identity. How did this apply to you, or to your age group when you were growing up? What stories were you brought up with? Can you recall a particular story that made a strong impression on you when you were young? Where do you mainly get your stories from nowadays? From books, from movies, from yarns told by friends? From the TV? What story caught your attention recently?

35. Session 2

36. Sentences A set of 59 sentences is presented one at a time on the screen. Different orders for S1 and S2 (this is taken care of by the program). Ask the speaker to read each one silently through before reading them aloud. If the speaker makes so many mistakes they get frustrated, you can skip to the next component (only as a last resort)

37. Interview: Topics 1.A dangerous or exciting episode in my life: Something I’m never likely to forget… 2. A topic that I feel quite strongly about: Something that really gets on my goat… something that really ought to be changed … 3. My best holiday ever was when … 4. A most embarrassing incident … 5. Changes in the district: How this place has changed over the years; what it used to be like around here compared with the way things are now… 6. What we used to get up to as kids: Ways that we used to entertain ourselves, but are perhaps not so available to kids today … 7. One of the most interesting characters that I have ever met …

38. Interview: Guidelines No “PARSNIP” ! Politics Alcohol Religion Sex Narcotics -isms Personal

39. Session 3

40. Map Task Two people, one the instruction giver (IG) and one the instruction follower (IF). Each has a map of the same environment, but there are small differences between the maps. The IG's map has a route marked to some target location from some start point. The IG's task is to communicate this route to the IF so that he or she can mark the route on their copy of the map.

41. Map Task: Information Giver

42. Map Task: Information Follower

43. Map Task: Instructions

44. MapTask set up for Speaker B

45. Map Task: logistics Open Session 3.1: Speaker 1 at Main Table Calibration – Opening – Words Introduce Speaker 2 Sit Speaker 2 at Second 2 – Calibration for Speaker 2 at Second Table Calibration for Speaker 1 at Main Table (lower camera) Map Task 1 (20mns): Speaker 1 = IG / Speaker 2 = IF Speaker 2 goes over the route they took and holds up the map to the camera Close and finalise Session 3.1 Switch Speaker 1 and Speaker 2 (position and role) Open Session 3.2: Calibration for Speaker 1 at Second Table – Calibration for Speaker 2 at Main Table Map Task 2 (20mns) Speaker 2 = IG / Speaker 1 = IF Speaker 1 goes over the route they took and holds up the map to the camera Conversation between Speaker 1 and Speaker 2 (5mns) Escort Speaker 1 out of the room Calibration for Speaker 2 at Main Table (upper camera) Words – Closing for Speaker 2 Close and finalise Session 3.2

46. Conversation after Map Task Ask the participants to talk to each other about: whether they enjoyed the task, how difficult it was, what part of the instructions at the beginning helped them understand the task, what they liked or didn't like about the other participant's directions, what features of the landmarks helped or hindered, what they found useful.

47. Conversation after Map Task Ask the participants to talk to each other about: whether they enjoyed the task, how difficult it was, what part of the instructions at the beginning helped them understand the task, what they liked or didn't like about the other participant's directions, what features of the landmarks helped or hindered, what they found useful.

48. Data saving and upload

49. Data Handling Procedure (1) https://austalk.edu.au/trac/wiki/DataHandlingProcedure https://austalk.edu.au/trac/wiki/DataHandlingProcedure Data is recorded on the Black Box for each session and stored on the local hard drive (D:) of the machine. However, the local drive can't store a lot of data (how many sessions?) and we need to move data to backup storage and forward it to Macquarie to be uploaded to the server. This page details the procedure to be followed at the end of each day's recording. The initial working assumption is that no compression of video is to be done on the black box machine. This means that a single session is up to 120G of raw data. If we later introduce compression, these procedures may be able to be altered. During Recording All data will be stored on the D: drive in the D:\recordings\ folder, each speaker will be stored in a folder named for the session and the speaker. The folder name will be something like Spkr1_1241_Session1 where 1_1241 is the speaker numerical id (corresponding to the colour and animal) and Session1 is the first session. There will be folders within this for each component that was recorded. After Each Session After each session...

50. Data Handling Procedure (2) https://austalk.edu.au/trac/wiki/DataHandlingProcedure https://austalk.edu.au/trac/wiki/DataHandlingProcedure End of Day At the end of each day you should have all of the data from that day's recordings on the hard drive of the black box. This could amount to around 500G if you ran 4 separate recording sessions. We need to move this to backup storage and produce something that can be sent to Macquarie for uploading to the server. Write to two USB drives, one will be kept locally, one sent to MQ Verify that the data was copied correctly to both drives Label the drives with the dates, speaker and session IDs they contain Delete original data from the internal hard drive ready for the next day's recording Copying 500G to a single USB drive should take around 4hrs (est. around 40M/s transfer rate). Copying to two different disks shouldn't take twice as long as the limiting factor is the USB transfer speed not the read speed. End of Week (or every few days) Send the full hard drives to MQ for uploading to server. Drives should be properly packed (bubble wrap, plastic bag) to avoid data loss due to shocks/water damage. Local USB drives should be stored in a secure location away from possible water damage etc. If you have access to local storage (eg. your institution has a large scale data storage facility) then a copy could be uploaded to that as an additional backup.

51. Data Handling Procedure (3) https://austalk.edu.au/trac/wiki/DataHandlingProcedure https://austalk.edu.au/trac/wiki/DataHandlingProcedure End of Week (or every few days) Send the full hard drives to MQ for uploading to server. Drives should be properly packed (bubble wrap, plastic bag) to avoid data loss due to shocks/water damage. Local USB drives should be stored in a secure location away from possible water damage etc. If you have access to local storage (eg. your institution has a large scale data storage facility) then a copy could be uploaded to that as an additional backup. At Macquarie Once drives are received at MQ the data will be uploaded to the server, verified, and the original USB drives will be cleared and sent back to the collection site. All data on the server will be backed up to a secondary storage facility (tape). The uploaded data will be visible via the web server. Once the drive is returned to the site, you can confirm that the data is available via the web and clear the data from the local USB drive copy. Alternately, this could be kept as a local backup if sufficient USB drives are available.

52. Recruitment and keeping records

53. Recruitment Registered 16/06/2011Required ACT24 132 NSW423 196 NT3 96 QLD70 168 SA49 96 TAS124 48 VIC148 168 WA51 96 8921000 Expressions of interest: https://austalk.edu.au/express-interest.html https://austalk.edu.au/express-interest.html

54. Demographic spread Age/SES/GenderProf-MNProf-MProf-FNprof-F < 3010 31-4910 > 5010 Total: 120 See doc on wiki: https://austalk.edu.au/trac/attachment/wiki/CollectionSchedule/Demographics.xlsx

55. Consent forms Each institution must have their own (see model from UWS) Use your own Ethics approval number Need to be signed before Questionnaire

56. Questionnaire Website: https://austalk.edu.au/forms/participant-questionnaire.html Need to be completed before recording. Speaker ID: colour + Australian animal – e.g. “Blue Northern funnelweb spider” – Same ID for all 3 sessions

57. Payment to participants Each speaker receives $25 for each of the first and second sessions, and $40 for the third. Payment is organised by each institution. 2 options: 1) Cash 2) Bank transfer Model forms from UWS – to be adapted by each institution: 1) Cash advance for the RA2) Claim for payment Cash payment to participant ATO form

58. Bonus We really need each speaker to come back and do the 3 sessions. Incentive for YOU: $25 for each full set of 3 sessions To be paid in a travel fund by your institution  Complete the spreadsheet

59. Documentation Wiki: https://austalk.edu.au/trac/wikihttps://austalk.edu.au/trac/wiki

60. http://austalk.edu.au