Advanced Artificial Intelligence Educational Data Mining Chung-Ang University, Ho Han Hello, my name is Ho Han, in MI Lab. My presentation will show you about Educational Data Mining.

Introduction In an ambient intelligence (AmI) environment, electronic devices provide a wide variety of applications and intelligent services to users. Advanced Artificial Intelligence / Chung-Ang University / Ho Han

Introduction In this days, Many Students and teacher have increased their demand for computer assisted teaching. Here is a Computer assisted pronunciation training, we call this CAPT ,that is a representative part of educational data mining. Advanced Artificial Intelligence / Chung-Ang University / Ho Han

Introduction Why CAPT? Traditional pronunciation teaching system, one-to-one tutoring most effective but with high cost. Advanced Artificial Intelligence / Chung-Ang University / Ho Han

Introduction CAPT enables users to self-correct their pronunciation. So it eliminates the diffculty associated with finding tutors who are native speakers. This system is helpful to users who feel uncomfortable in oral presentations. Advanced Artificial Intelligence / Chung-Ang University / Ho Han

Introduction Close? Cloth? I wear blue close today Do you mean cloth? Here is a simple procedure of CAPT. At first user speech a samples. Advanced Artificial Intelligence / Chung-Ang University / Ho Han

Introduction Close? Cloth? I wear blue close today Do you mean cloth? And Computer detect unacceptable pronunciation. Advanced Artificial Intelligence / Chung-Ang University / Ho Han

Introduction Close? Cloth? Do you mean cloth? I wear blue close today At last, Computer provide feedback to user. Advanced Artificial Intelligence / Chung-Ang University / Ho Han

Introduction Close? Cloth? I wear blue close today Do you mean cloth? However, because of their intensive computational requirements, It is not capable of IOT (Internet of thing) environment. Advanced Artificial Intelligence / Chung-Ang University / Ho Han

SAPT SAPT Fortunately, MI Lab have already proposed a smartphone-assisted pronunciation learning technique. (we call this SAPT) Pronunciation training application should be a hand-held devices such as smartphones to improve their availability. Advanced Artificial Intelligence / Chung-Ang University / Ho Han

SAPT SAPT Word recommendation technique Recommendation function However, conventional CAPT systems are unable to execute on such lightweight devices, because require significant computational resources. So we propose word recommendation technique and a series of built-in functions to reduce the computational burden. Advanced Artificial Intelligence / Chung-Ang University / Ho Han

SAPT SAPT Tired… cloth …clothclothcloth… In conventional pronunciation systems, users practice the same words or sentences repeatedly until acceptable. The system bored users to death. To avoid this situation, SAPT to provide diverse and effective feedback. Advanced Artificial Intelligence / Chung-Ang University / Ho Han

SAPT vase valance When a person pronounces a word incorrectly, there is a high probability that the person will pronounce word with similar pronunciations incorrectly. This concept improve our SAPT system. Advanced Artificial Intelligence / Chung-Ang University / Ho Han

SAPT vase various valance For example, suppose that a person mispronounces the words “vase” and “valance”. Then there is high probability also mispronounce the word “various”. Advanced Artificial Intelligence / Chung-Ang University / Ho Han

SAPT various vase valance Feedback In this case, our system can recommend the word ``various'' for a user. Advanced Artificial Intelligence / Chung-Ang University / Ho Han

SAPT Bag of phoneme model To accomplish this, the word pronunciation is represented as a bag of phonemes. Using this bag of phonemes, the relationship of phonemes with error words is determined. Advanced Artificial Intelligence / Chung-Ang University / Ho Han

SAPT Let’s know how to process. Step 1, Users pronounce testing words displayed on the smartphone. In this example, test word set is consisted of the three words – “vase”, “let”, and “valance”. Advanced Artificial Intelligence / Chung-Ang University / Ho Han

SAPT Step 2, the system tests how the spoken words are actually recognized. In this case, “vase” is recognized as “base” because of mispronunciation. As same, “valance” is recognized as “balance”. Advanced Artificial Intelligence / Chung-Ang University / Ho Han

SAPT Bag of phoneme model Step 3, The system represents the pronunciation of a word as a bag of phonemes and generates a dataset based on the bag of phoneme model. Advanced Artificial Intelligence / Chung-Ang University / Ho Han

SAPT Step 4, the system assesses correlation of the phonemes and mispronunciation. This shows that two phonemes /v/ and /s/ are highly correlated, because they include the error words “vase” and “valance”. Advanced Artificial Intelligence / Chung-Ang University / Ho Han

SAPT Step 5, the system assigns a selection probability to each word. This shows an example of the word recommendation based on correlation analysis. A circle with a thick line indicates that the phoneme is strongly correlated with the error pronunciation. Advanced Artificial Intelligence / Chung-Ang University / Ho Han

SAPT Phoneme /v/ and /s/ are included three words. But these two phonemes are only two words. Based on this, the system assigns a selection probability value to each word in order to recommend a word set. Advanced Artificial Intelligence / Chung-Ang University / Ho Han

SAPT Step 6. The system recommends selected words. This shows that the two words, “valance” and “various”, were selected because of their high selection probability. Advanced Artificial Intelligence / Chung-Ang University / Ho Han

SAPT At last Step 7. The user practices selected words. Some of the above steps is described in detail in the following sections. Advanced Artificial Intelligence / Chung-Ang University / Ho Han

Test word set 700 words A set of words for the pronunciation test was collected by seven experts. A total of 700 words were selected according to the use frequency of the words used in Korean elementary schools for English education. Advanced Artificial Intelligence / Chung-Ang University / Ho Han

Why Korean elementary schools ? Test word set Why Korean elementary schools ? 700 words It because the purpose of this system is a pronunciation correcting. For example If unknown words are included, then user will mispronounce the words Advanced Artificial Intelligence / Chung-Ang University / Ho Han

Word recommendation method Now, I will show you the word recommendation method. Here is a sample words and bag of phonemes. The value 1 for a test result indicates unacceptable pronunciation, 0 for otherwise. Advanced Artificial Intelligence / Chung-Ang University / Ho Han

Word recommendation method Let W denote a set of words that is composed of n words. A word 𝑤 𝑖 can be represented based on the occurrence of d phonemes in its pronunciation, thereby like this. Advanced Artificial Intelligence / Chung-Ang University / Ho Han

Word recommendation method If the value of the j-th element in 𝑤 𝑖 is 1 If the value of the j-th element in 𝑤 𝑖 is 1 denoted as like this, and then it indicates that the j-th phoneme is required for the pronunciation of 𝑤 𝑖 . (In another case, 𝑤 𝑖,𝑗 to 0) Advanced Artificial Intelligence / Chung-Ang University / Ho Han

Word recommendation method 1 From the perspective of each phoneme, 𝑃 𝑗 ∈ 0,1 𝑛×1 (where 1≤ j ≤ d ) is a column vector of the j-th phoneme in the word 𝑤 𝑖 , if the value of the i-th element is 1. Advanced Artificial Intelligence / Chung-Ang University / Ho Han

Word recommendation method 1 Next, the test result 𝑡 𝑖 ∈ 𝑇 represents acceptable / unacceptable pronunciation for 𝑤 𝑖 , Where 𝑇∈ 0,1 𝑛×1 . Advanced Artificial Intelligence / Chung-Ang University / Ho Han

Word recommendation method 1 we employed the Pearson correlation coefficient. which is one of the most widely used statistical measures, to calculate the correlation between two variables. Advanced Artificial Intelligence / Chung-Ang University / Ho Han

Word recommendation method 1 cov( 𝑃 𝑗 , T ) denotes the covariance between 𝑃 𝑗 and T. var( 𝑃 𝑗 ) and var(T ) are the variance of 𝑃 𝑗 and T , respectively. Advanced Artificial Intelligence / Chung-Ang University / Ho Han

Word recommendation method The lower and upper bound of C( 𝑃 𝑗 , T ) is given as like this. value is 1 in the case of a perfect correlation, -1 a perfect inverse correlation. Advanced Artificial Intelligence / Chung-Ang University / Ho Han

Word recommendation method Here is an example data set after performing the pronunciation test. The example data set shows that this user mispronounced all of the words including /g/. Advanced Artificial Intelligence / Chung-Ang University / Ho Han

Word recommendation method Correlation C(/g/,T) is 1, and C(/k/,T) is -1. Advanced Artificial Intelligence / Chung-Ang University / Ho Han

User interface After the first round is completed, 70 words are selected to train the user. This is a user interface in the practice phase. Advanced Artificial Intelligence / Chung-Ang University / Ho Han

Thank you Q&A Thank you. Do you have any questions? Advanced Artificial Intelligence / Chung-Ang University / Ho Han