1. Advanced Artificial Intelligence Educational Data Mining
Chung-Ang University, Ho Han
Hello, my name is Ho Han, in MI Lab. My presentation will show you about Educational Data Mining.

2. Introduction
In an ambient intelligence (AmI) environment, electronic devices provide a wide variety of applications and intelligent services to users.
Advanced Artificial Intelligence / Chung-Ang University / Ho Han

3. Introduction
In this days, Many Students and teacher have increased their demand for computer assisted teaching.
Here is a Computer assisted pronunciation training, we call this CAPT ,that is a representative part of educational data mining.
Advanced Artificial Intelligence / Chung-Ang University / Ho Han

4. Introduction
Why CAPT?
Traditional pronunciation teaching system, one-to-one tutoring most effective but with high cost.
Advanced Artificial Intelligence / Chung-Ang University / Ho Han

5. Introduction CAPT enables users to self-correct their pronunciation.
So it eliminates the diffculty associated with finding tutors who are native speakers.
This system is helpful to users who feel uncomfortable in oral presentations.
Advanced Artificial Intelligence / Chung-Ang University / Ho Han

6. Introduction Close? Cloth? I wear blue close today Do you mean cloth?
Here is a simple procedure of CAPT.
At first user speech a samples.
Advanced Artificial Intelligence / Chung-Ang University / Ho Han

7. Introduction Close? Cloth? I wear blue close today Do you mean cloth?
And Computer detect unacceptable pronunciation.
Advanced Artificial Intelligence / Chung-Ang University / Ho Han

8. Introduction Close? Cloth? Do you mean cloth? I wear blue close today
At last, Computer provide feedback to user.
Advanced Artificial Intelligence / Chung-Ang University / Ho Han

9. Introduction Close? Cloth? I wear blue close today Do you mean cloth?
However, because of their intensive computational requirements, It is not capable of IOT (Internet of thing) environment.
Advanced Artificial Intelligence / Chung-Ang University / Ho Han

10. SAPT
SAPT
Fortunately, MI Lab have already proposed a smartphone-assisted pronunciation learning technique.
(we call this SAPT)
Pronunciation training application should be a hand-held devices such as smartphones to improve their availability.
Advanced Artificial Intelligence / Chung-Ang University / Ho Han

11. SAPT SAPT Word recommendation technique Recommendation function
However, conventional CAPT systems are unable to execute on such lightweight devices, because require significant computational resources.
So we propose word recommendation technique and a series of built-in functions to reduce the computational burden.
Advanced Artificial Intelligence / Chung-Ang University / Ho Han

12. SAPT SAPT Tired… cloth …clothclothcloth…
In conventional pronunciation systems, users practice the same words or sentences repeatedly until acceptable.
The system bored users to death.
To avoid this situation, SAPT to provide diverse and effective feedback.
Advanced Artificial Intelligence / Chung-Ang University / Ho Han

13. SAPT
vase
valance
When a person pronounces a word incorrectly, there is a high probability that the person will pronounce word with similar pronunciations incorrectly.
This concept improve our SAPT system.
Advanced Artificial Intelligence / Chung-Ang University / Ho Han

14. SAPT vase various valance
For example, suppose that a person mispronounces the words “vase” and “valance”.
Then there is high probability also mispronounce the word “various”.
Advanced Artificial Intelligence / Chung-Ang University / Ho Han

15. SAPT various vase valance Feedback
In this case, our system can recommend the word ``various'' for a user.
Advanced Artificial Intelligence / Chung-Ang University / Ho Han

16. SAPT Bag of phoneme model
To accomplish this, the word pronunciation is represented as a bag of phonemes.
Using this bag of phonemes, the relationship of phonemes with error words is determined.
Advanced Artificial Intelligence / Chung-Ang University / Ho Han

17. SAPT Let’s know how to process.
Step 1, Users pronounce testing words displayed on the smartphone.
In this example, test word set is consisted of the three words – “vase”, “let”, and “valance”.
Advanced Artificial Intelligence / Chung-Ang University / Ho Han

18. SAPT
Step 2, the system tests how the spoken words are actually recognized.
In this case, “vase” is recognized as “base” because of mispronunciation.
As same, “valance” is recognized as “balance”.
Advanced Artificial Intelligence / Chung-Ang University / Ho Han

19. SAPT Bag of phoneme model
Step 3, The system represents the pronunciation of a word as a bag of phonemes
and generates a dataset based on the bag of phoneme model.
Advanced Artificial Intelligence / Chung-Ang University / Ho Han

20. SAPT
Step 4, the system assesses correlation of the phonemes and mispronunciation.
This shows that two phonemes /v/ and /s/ are highly correlated,
because they include the error words “vase” and “valance”.
Advanced Artificial Intelligence / Chung-Ang University / Ho Han

21. SAPT Step 5, the system assigns a selection probability to each word.
This shows an example of the word recommendation based on correlation analysis.
A circle with a thick line indicates that the phoneme is strongly correlated with the error pronunciation.
Advanced Artificial Intelligence / Chung-Ang University / Ho Han

22. SAPT Phoneme /v/ and /s/ are included three words.
But these two phonemes are only two words.
Based on this, the system assigns a selection probability value to each word in order to recommend a word set.
Advanced Artificial Intelligence / Chung-Ang University / Ho Han

23. SAPT Step 6. The system recommends selected words.
This shows that the two words, “valance” and “various”, were selected because of their high selection probability.
Advanced Artificial Intelligence / Chung-Ang University / Ho Han

24. SAPT At last Step 7. The user practices selected words.
Some of the above steps is described in detail in the following sections.
Advanced Artificial Intelligence / Chung-Ang University / Ho Han

25. Test word set
700 words
A set of words for the pronunciation test was collected by seven experts.
A total of 700 words were selected according to the use frequency of the words used in Korean elementary schools for English education.
Advanced Artificial Intelligence / Chung-Ang University / Ho Han

26. Why Korean elementary schools ?
Test word set
Why Korean elementary schools ?
700 words
It because the purpose of this system is a pronunciation correcting.
For example If unknown words are included, then user will mispronounce the words
Advanced Artificial Intelligence / Chung-Ang University / Ho Han

27. Word recommendation method
Now, I will show you the word recommendation method.
Here is a sample words and bag of phonemes.
The value 1 for a test result indicates unacceptable pronunciation, 0 for otherwise.
Advanced Artificial Intelligence / Chung-Ang University / Ho Han

28. Word recommendation method
Let W denote a set of words that is composed of n words.
A word 𝑤 𝑖 can be represented based on the occurrence of d phonemes in its pronunciation, thereby like this.
Advanced Artificial Intelligence / Chung-Ang University / Ho Han

29. Word recommendation method
If the value of the j-th element in 𝑤 𝑖 is 1
If the value of the j-th element in 𝑤 𝑖 is 1 denoted as like this,
and then it indicates that the j-th phoneme is required for the pronunciation of 𝑤 𝑖 .
(In another case, 𝑤 𝑖,𝑗 to 0)
Advanced Artificial Intelligence / Chung-Ang University / Ho Han

30. Word recommendation method1
From the perspective of each phoneme, 𝑃 𝑗 ∈ 0,1 𝑛×1 (where 1≤ j ≤ d ) is a column vector of the j-th phoneme in the word 𝑤 𝑖 , if the value of the i-th element is 1.
Advanced Artificial Intelligence / Chung-Ang University / Ho Han

31. Word recommendation method1
Next, the test result 𝑡 𝑖 ∈ 𝑇 represents acceptable / unacceptable pronunciation for 𝑤 𝑖 ,
Where 𝑇∈ 0,1 𝑛×1 .
Advanced Artificial Intelligence / Chung-Ang University / Ho Han

32. Word recommendation method1
we employed the Pearson correlation coefficient.
which is one of the most widely used statistical measures, to calculate the correlation between two variables.
Advanced Artificial Intelligence / Chung-Ang University / Ho Han

33. Word recommendation method1
cov( 𝑃 𝑗 , T ) denotes the covariance between 𝑃 𝑗 and T.
var( 𝑃 𝑗 ) and var(T ) are the variance of 𝑃 𝑗 and T , respectively.
Advanced Artificial Intelligence / Chung-Ang University / Ho Han

34. Word recommendation method
The lower and upper bound of C( 𝑃 𝑗 , T ) is given as like this.
value is 1 in the case of a perfect correlation, -1 a perfect inverse correlation.
Advanced Artificial Intelligence / Chung-Ang University / Ho Han

35. Word recommendation method
Here is an example data set after performing the pronunciation test.
The example data set shows that this user mispronounced all of the words including /g/.
Advanced Artificial Intelligence / Chung-Ang University / Ho Han

36. Word recommendation method
Correlation C(/g/,T) is 1, and C(/k/,T) is -1.
Advanced Artificial Intelligence / Chung-Ang University / Ho Han

37. User interface
After the first round is completed, 70 words are selected to train the user.
This is a user interface in the practice phase.
Advanced Artificial Intelligence / Chung-Ang University / Ho Han

38. Thank you Q&A Thank you. Do you have any questions?
Advanced Artificial Intelligence / Chung-Ang University / Ho Han