1. Data-Driven Machine Translation for Sign Languages Sara Morrissey PhD topic NCLT/CNGL Workshop 23 rd July 2008

2. outline  background  main problems  data-driven MT for SLs  experiments and results  conclusions

3. background  communication  interpreters and technological aids  machine translation –automatic and confidential –native language of users  rule-based approaches (Veale et al., 1998, Marshall & Sáfár, 2002)  data-driven approaches –Bauer et al., 1999, Stein et al., 2006, Wu et al., 2007

4. main problems  representation no formally adopted writing system  linguistic analysis little research  appropriate data difficult to find  evaluation visual-spatial nature rules out automatic

5. data-driven MT for SLs  initial prototype system using Dutch SL  MaTrEx system  Air Traffic Information System (ATIS) Corpus  595 English sentences  multi-lingual – ISL parallel corpus creation  manual annotation with semantic glosses

6. data representation (Early morning flights between Cork and Belfast) EARLY MORNING BETWEEN be-CORK CORK FLY BELFAST BETWEEN ref-BELFAST ref-CORK

7. M A T R E X : data-driven machine translation  English  ISL bilingual database

8. translation directions  SL RecognitionSL Generation SL Annotation Spoken Language Text

9. experiments and results  machine translation experiments  2 segmentation methodologies  type 1 chunks uses Marker Hypothesis (Green, 1979)  type 2 uses dual segmentation method 1.Early morning flights between Cork and Belfast 2. early morning flights between Cork and Belfast

10. experiments and results SystemBLEUWERPER EN—ISL ISL—EN Baseline + T1 chunks + T2 chunks Baseline + T1 chunks + T2 chunks 38.85 39.11 39.05 52.18 51.31 50.32 46.02 45.90 46.02 38.48 37.39 39.56 34.33 34.20 34.21 29.67 30.63 32.32

11. animation  real human signing preferred (Naqvi, 2007) but impractical  avatar animation  criteria: realistic, consistent, functional, fluid  Poser Animation Software Version 6.0  50 randomly selected sentences, 66 hand- crafted videos  problem of fluidity

12. animation ‘or’ ‘e’ how much flight http://www.computing.dcu.ie/~smorri/ISL_AnimationDemo.html

13.  human evaluation experiments  4 native Deaf human monitors  web-based evaluation of 50 ISL translations  evaluated intelligibility and fidelity  82% animations = intelligible  72% animations = good-excellent translations  HCI analysis using Nielsen’s approach experiments and results

14. conclusion  MT methodology never before applied to SLs  multi-component system, bidirectional system  practical, technological alternative to help alleviate communication and comprehension for Deaf community  positive automatic and manual evaluation  scope for incorporating different SL representation methodologies and segmentation techniques

15. thank you questions? smorri@computing.dcu.ie