Automatic speech recognition of code-switched speech for South African languages Ewald van der Westhuizen Digital signal processing group DSP lab, Rm E355, E&E Engineering Stellenbosch University

Introduction Automatic Speech Recognition – human speech to text ASR systems suited for monolingual speech Code-switching – multilingual speakers speak more than one language in a conversation or utterance Code-switching degrades ASR performance

Example Speech in South African soap operas Example video: Setswana, Zulu and English

Example Speech is natural, conversational, fast, emotion, accented Code-switching is spontaneous Filler words and hesitations Noisy environment (door, footsteps, paper rustle) Speech overlap

Modelling code-switched language Code-switching examples are sparse Synthesise artificial examples of code-switching Word embeddings Represent words as vectors in multidimensional space Capture semantic and syntactic relationships of words Synthesised examples enhance training data used for language modelling

Querying word embedding models Trigger i- ama- Query word album advertisers sim.wrd cos.scr Result words song 0.87 promotors 0.77 movie 0.81 creatives 0.73 film employers 0.72 soundtrack sponsors series 0.74 fans 0.70 i- song, i- movie ama- promotors, ama- creatives

Code-switching Inter-sentence Intra-sentence Intra-word Example: Insertional CS Alternational CS Intra-word Example: You’ve got no idea how vinnig I’ve been slaan-ing this bymekaar.