Course Concept Expansion in MOOCs with External Knowledge and Interactive Game Jifan Yu, Chenyu Wang, Gan Luo, Lei Hou, Jie Tang, Juanzi Li and Zhiyuan Liu Tsinghua University Good Morning ladies and gentleman, I am Jifan from Tsinghua University. It's great pleasure indeed for me to be able to attend this conference. My job is to expand the content of MOOCs based on a small game. To be honest, I have been longing for this mode of playing digital games in class since my childhood.

Learning in a Game Find out the irrelevant orange candidate(s) ! Heap Christmas tree Binary Tree Tango Tree Huffman Tree So how do games expand course knowledge, and why do we need to expand knowledge in MOOCs? Maybe this simple little game can give us some inspiration. The rule of this game is simple. Given a few orange candidates, we need to find out which of them are irrelevant to a given blue concept. Okay now I need a volunteer, to help me find the irrelevant candidate.

Learning in a Game The game helps students to review old knowledge Find out the irrelevant orange candidate(s) ! Heap Data Structures The game helps students to review old knowledge and get to new knowledge in a relaxing way. Christmas tree ✔️ Binary Tree Tango Tree Huffman Tree “Tango Tree is a type of Binary Search Tree that a competitive ratio…” ---Wikipedia Nice job! Thank you, the answer is Christmas Tree. Because other candidates are all concepts from "data structures". In addition to the answer of this game, I think we've all noticed an unfamiliar concept, Tango Tree. Actually, I didn't know it before I Google it, which is an advanced binary search Tree structure.

Course: “Introduction to Psychology” In real MOOCs “Top-Student Game” in XuetangX Over 50,000 user operations Course: “Introduction to Psychology” Top-Student Game: Users delete irrelevant orange concepts and gain bonus. In fact, that‘s how we actually use it, on XuetangX, one of China’s largest MOOC education sites. It already has thousands of users interacting with it. The game which we called “Top-student” is designed below the videos and presents content related to this video.

Challenges A New Task: Course Concept Expansion Find out concepts related to the course. Interact with MOOC users via our Game. Set Expansion (Wang, 2007; Adrian and Manna, 2018) Courses are not typical “categories”.  Semantic Drifts How to find high-quality expansion candidates? How to involve user behavior in Top-Student Game? Typical Category MOOC Course “Psychology in Work” Cities Florence New York Work Paris Beijing … Therefore, we designed a new task named Course Concept Expansion, which aims at finding out concept related to the course from external source. And we present the expansion results for users using our game. The closest existing job to this, is Set Expansion. That is, given a set, use external resources to expand the entities of the set as many as possible. However, these methods cannot be used directly on MOOCs because MOOC course are often combinations of categories. And this may lead to a violent Semantic Drifts. Psychology

1-2. Prevent Semantic Drifts 3. Interact with Users Framework Input: MOOC, External Knowledge Base Models + Game Output: Expanded Concepts 1-2. Prevent Semantic Drifts 3. Interact with Users To overcome the challenges, we build a three stage workflow to expand using an external knowledge base. At first we extract course concepts from MOOC, and select the entities which have relations with them in KB as candidates. Finally we put them into game, to show them to MOOC users and collect feedback for further optimization.

Method: Candidate Generation First: Course Concept Extraction Assumption: A course is a concept space which contains one or more concept clusters. Each course, e.g. “Data Structure and Algorithm” Graph algorithms Trees Sort algorithms. Using Clusters to delineate the semantic boundaries of the course The first step: Candidate Generation is very important, Because candidate generation directly determines the quality and quantity of the resulting extensions presented to the user, we want to minimize semantic drift in this step. Through observation, we find that the concepts of the course tend to cluster together when represented by vectors. For example, in the course “data structure and algorithm”, concepts can be roughly divided into three categories, Graph Algori

Method: Candidate Generation The concept space boundary is fitted while searching for new candidates (in KB). Confidence Score Each new found concept’s confidence score is provided by its nearest cluster. Once a potential cluster’s size reach to τ… (It can provide enough seeds)

Method: Candidate Generation Link the course concepts into KB Search for the concepts, entities of their neighbors. A part of KB in Candidate Generation Link the Course Concept Search the neighbors as candidates

Method: Classification Feature Engineering Confidence Score (S) Search Path Encoding (Cho et al., 2014) (PE) Prerequisite Features (Pan, 2017) (Ps) User Deletion Rate (From Game) (Dr) Corpus-based Feature Knowledge Base Feature Human Efforts Domain-Specific Feature

Method: Game-based Optimization “Top-Student Game” in XuetangX As a feedback collector As an online evaluation

Method: Game-based Optimization “Top-Student Game” in XuetangX Multi-level optimization For Candidate Generation Adjust the Confidence Score For Classification Perform as a feature

Experiment Dataset: MOOC data from XuetangX and Coursera For each courses, we select top 800 expanded concepts from 100,000 candidates Knowledge Base: XLORE (Jin et al., 2018)

Our Model achieve a good result with(without) game. Experiment Baselines PR (Graph Based Method) SEISA (He and Xin, 2011) EBM (Embedding Based Method) (Mamou et al., 2018) PUL PU-Learning methods(Wang et al., 2017) Evaluation Metrics MAP Mean Average Precision Results Our Model achieve a good result with(without) game.

Each of the Feature we design is useful. Experiments Parameter Analysis Feature Contribution Each of the Feature we design is useful.

Expansion results satisfied the real MOOC users. Experiments Online Evaluation Cr is the rate of user deletion.(Larger Cr indicates a lower expansion quality) Expansion results satisfied the real MOOC users.

MOOCdata: http://moocdata.cn We are THU MOOC Team! Data and Other Interesting Work!!All in http://moocdata.cn MOOC: A perfect platform of AI in Education Easy to interact with users Easy to build fancy functions Our Publications in AAAI, NIPS…

MOOCdata: http://moocdata.cn Our Data and Other Interesting Work!! Students’ behavior Course Recommendation QA system Prerequisite Relation Discovery Xiaomu: A learning assistant in XuetangX.

Thank you! Questions? Knowledge Engineering Group, Tsinghua University THU MOOC Team: http://moocdata.cn yujifan0326@gmail.com