1. Automatic speech recognition of code-switched speech for South African languages
Ewald van der Westhuizen
Digital signal processing group
DSP lab, Rm E355, E&E Engineering
Stellenbosch University

2. Introduction Automatic Speech Recognition – human speech to text
ASR systems suited for monolingual speech
Code-switching – multilingual speakers speak more than one language in a conversation or utterance
Code-switching degrades ASR performance

3. Example Speech in South African soap operas
Example video: Setswana, Zulu and English

4. Example Speech is natural, conversational, fast, emotion, accented
Code-switching is spontaneous
Filler words and hesitations
Noisy environment (door, footsteps, paper rustle)
Speech overlap

5. Modelling code-switched language
Code-switching examples are sparse
Synthesise artificial examples of code-switching
Word embeddings
Represent words as vectors in multidimensional space
Capture semantic and syntactic relationships of words
Synthesised examples enhance training data used for language modelling

6. Querying word embedding models
Trigger i-
ama-
Query word
album
advertisers
sim.wrd
cos.scr
Result words
song
0.87
promotors
0.77
movie
0.81
creatives
0.73
film
employers
0.72
soundtrack
sponsors
series
0.74
fans
0.70
i- song, i- movie
ama- promotors, ama- creatives

7. Code-switching Inter-sentence Intra-sentence Intra-word Example:
Insertional CS
Alternational CS
Intra-word
Example:
You’ve got no idea how vinnig I’ve been slaan-ing this bymekaar.