8th Annual Post-Graduate Research Symposium Robert Smith School of Informatics and Engineering Institute of Technology, Blanchardstown Irish Sign Language Research Forum Human Language Technologies Research Group https://robertsmithresearch.wordpress.com/ 8th Annual Post-Graduate Research Symposium Exploring the relationship between depicting verbs, constructed actions and non-manual features in Irish Sign Language 11th Dec 2017, DIT

Contents Motivation Short overview of topic area Research questions Methodological approach Findings Knowledge contribution Research Impact 11/11/2018 Robert Smith

Motivation Fidelity of Sign Language generation avatar performance Findings of Smith, R.G., (2014). Access for Deaf Community 11/11/2018 Robert Smith

Short Overview Irish Sign Language Often referenced with acronym “ISL” Unique indigenous language used by the Irish Deaf Community (45,000 users approx.) Not English, Not Irish. Unique. Most countries have their own signed language Limited research and poorly resourced -> cannot avail of language technologies. 11/11/2018 Robert Smith

Short Overview (2) Visual gestural modality results in multichannel articulation. Manual Features Articulated by hands and arms Include handshape, position, hand orientation and movement Nonmanual Features Articulated by head, parts of the face, shoulders and torso Consider body lean, chin up/down, eyebrows up/down etc.… Basic ISL example (negation) GO | DON’T GO 11/11/2018 Robert Smith

Research questions (RQ) 1: What semantic meaning and grammatical function do NMFs add to ISL? (RQ) 2: What is the NMF meaning contribution to ISL? [SHAPE-OF-JAR] INDEX+fr [DOG] CLIMB-UP SLEEP 11/11/2018 Robert Smith

Research questions (RQ) 3: How can the SOI corpus be extended to represent NMFs and their relation to other multilevel linguistic phenomena? How does the SOI corpus compare to international norms? Are there NMFs missing? Etc.… 11/11/2018 Robert Smith

Research questions (RQ) 4: How are spatially modified verbs utilised by ISL signers in comparison to the lexical form, in the SOI corpus and what are the drivers of this? (1) Depicting verb (2) Indicating verb [DOG] CLIMB-UP SEE-SOMETHING 11/11/2018 Robert Smith

Research questions (RQ) 4: How are spatially modified verbs utilised by ISL signers in comparison to the lexical form, in the SOI corpus and what are the drivers of this? (3) Plain verb (4) Constructed Action HEAR TIRED LIE-BACK SLEEP 11/11/2018 Robert Smith

Methodology Dataset: SOI corpus (Leeson et al., 2006) Developed in order to capture a snapshot of ISL as it was used during that time. Natural ISL narratives across 3 different genre Demographically representative: 63 videos 40 signers aged between 18 and 65 from 5 locations across the Republic of Ireland It includes male and female signers 11/11/2018 Robert Smith

Step 1: Annotate a subset 11/11/2018 Robert Smith

Analysis Subset of the corpus to be annotated by January 11 videos 3,500 sign tokens approx. 22 minutes of video approx. Analyses of dataset Comparative frequency/collocation analysis – specific NMFs with specific grammatical class Market basket analysis – Which NMFs occur most commonly with specific grammatical class N-Gram analysis – What is the most common word order? 11/11/2018 Robert Smith

Some prelim indicators 11/11/2018 Robert Smith

Some prelim indicators 11/11/2018 Robert Smith

Some prelim indicators 11/11/2018 Robert Smith

Original contributions ISL technical analysis Description of grammatical classes in ISL Including an analysis on how these are used in the language. Description of NMFs, their form and function in ISL Description of depicting verbs in ISL and how they correlate with constructed actions. Digital corpus building - SOI corpus Additional annotations for future linguistic research Focus is on improving the quality of computer generated signing avatar performance. 11/11/2018 Robert Smith

Research impact Access for Deaf community The average reading age of Deaf school leavers is comparable to that of an 8-9 year old hearing child(Irish National Rehabilitation Board, 1991; Conrad, 1979). The ratio of qualified interpreters to Deaf people in Ireland equates to 250:1 (Leeson, 2003). Focus on sign Language generation and recognition technologies to improve access. This must begin at the language level – currently, not enough is known about ISL to build these technologies. Improves understanding of ISL for future research, teaching and interpreting. 11/11/2018 Robert Smith

Questions ? 11/11/2018 Robert Smith

References Conrad, R., 1979. The Deaf School Child: Language and Cognitive Function. London: Harper Collins Publishers. Irish National Rehabilitation Board, 1991. Access to colleges: A survey of third level centres. Dublin: National Rehabilitation Board, National Advisory Committee on Training and Employment. Leeson, L., 2003. Sign Language Interpreters: Agents of Social Change in Ireland. s.l., Languages ofIreland, Dublin, Ireland: Four Courts Press. Leeson, L., Saeed, J., Macduff, A. & Byrne-Dunne, D., 2006. Moving Heads and Moving Hands: Developing a Digital Corpus of Irish Sign Language. Carlow, Ireland, Information Technology and Telecommunications, pp. 25-26 Smith, R.G., (2014). The role of emotional and facial expression in synthesised sign language avatars. Thesis. Institute of Technology Blanchardstown. 11/11/2018 Robert Smith