1. Course Concept Expansion in MOOCs with External Knowledge and Interactive Game
Jifan Yu, Chenyu Wang, Gan Luo,
Lei Hou, Jie Tang, Juanzi Li and Zhiyuan Liu
Tsinghua University
Good Morning ladies and gentleman, I am Jifan from Tsinghua University. It's great pleasure indeed for me to be able to attend this conference.
My job is to expand the content of MOOCs based on a small game. To be honest, I have been longing for this mode of playing digital games in class since my childhood.

2. Learning in a Game Find out the irrelevant orange candidate(s) ! Heap
Christmas tree
Binary Tree
Tango Tree
Huffman Tree
So how do games expand course knowledge, and why do we need to expand knowledge in MOOCs?
Maybe this simple little game can give us some inspiration.
The rule of this game is simple. Given a few orange candidates, we need to find out which of them are irrelevant to a given blue concept.
Okay now I need a volunteer, to help me find the irrelevant candidate.

3. Learning in a Game The game helps students to review old knowledge
Find out the irrelevant orange candidate(s) !
Heap
Data Structures
The game helps students to review old knowledge
and get to new knowledge in a relaxing way.
Christmas tree ✔️
Binary Tree
Tango Tree
Huffman Tree
“Tango Tree is a type of Binary Search Tree that a competitive ratio…”
---Wikipedia
Nice job! Thank you, the answer is Christmas Tree. Because other candidates are all concepts from "data structures".
In addition to the answer of this game, I think we've all noticed an unfamiliar concept, Tango Tree. Actually, I didn't know it before I Google it, which is an advanced binary search Tree structure.

4. Course: “Introduction to Psychology”
In real MOOCs
“Top-Student Game” in XuetangX
Over 50,000 user operations
Course: “Introduction to Psychology”
Top-Student Game:
Users delete irrelevant orange concepts and gain bonus.
In fact, that‘s how we actually use it, on XuetangX, one of China’s largest MOOC education sites. It already has thousands of users interacting with it. The game which we called “Top-student” is designed below the videos and presents content related to this video.

5. Challenges A New Task: Course Concept Expansion
Find out concepts related to the course.
Interact with MOOC users via our Game.
Set Expansion (Wang, 2007; Adrian and Manna, 2018)
Courses are not typical “categories”.  Semantic Drifts
How to find high-quality expansion candidates?
How to involve user behavior in Top-Student Game?
Typical Category
MOOC Course
“Psychology in Work”
Cities
Florence
New York
Work
Paris
Beijing …
Therefore, we designed a new task named Course Concept Expansion, which aims at finding out concept related to the course from external source. And we present the expansion results for users using our game.
The closest existing job to this, is Set Expansion. That is, given a set, use external resources to expand the entities of the set as many as possible.
However, these methods cannot be used directly on MOOCs because MOOC course are often combinations of categories.
And this may lead to a violent Semantic Drifts.
Psychology

6. 1-2. Prevent Semantic Drifts 3. Interact with Users
Framework
Input: MOOC, External Knowledge Base
Models + Game
Output: Expanded Concepts
1-2. Prevent Semantic Drifts Interact with Users
To overcome the challenges, we build a three stage workflow to expand using an external knowledge base.
At first we extract course concepts from MOOC, and select the entities which have relations with them in KB as candidates.
Finally we put them into game, to show them to MOOC users and collect feedback for further optimization.

7. Method: Candidate Generation
First: Course Concept Extraction
Assumption: A course is a concept space which contains one or more concept clusters.
Each course, e.g. “Data Structure and Algorithm”
Graph algorithms
Trees
Sort algorithms.
Using Clusters to delineate the semantic boundaries of the course
The first step: Candidate Generation is very important,
Because candidate generation directly determines the quality and quantity of the resulting extensions presented to the user, we want to minimize semantic drift in this step.
Through observation, we find that the concepts of the course tend to cluster together when represented by vectors. For example, in the course “data structure and algorithm”, concepts can be roughly divided into three categories, Graph Algori

8. Method: Candidate Generation
The concept space boundary is fitted while searching for new candidates (in KB).
Confidence Score
Each new found concept’s confidence score is provided by its nearest cluster.
Once a potential cluster’s size reach to τ…
(It can provide enough seeds)

9. Method: Candidate Generation
Link the course concepts into KB
Search for the concepts, entities of their neighbors.
A part of KB in Candidate Generation
Link the Course Concept
Search the neighbors as candidates

10. Method: Classification
Feature Engineering
Confidence Score (S)
Search Path Encoding (Cho et al., 2014) (PE)
Prerequisite Features (Pan, 2017) (Ps)
User Deletion Rate (From Game) (Dr)
Corpus-based Feature
Knowledge Base Feature
Human Efforts
Domain-Specific
Feature

11. Method: Game-based Optimization
“Top-Student Game” in XuetangX
As a feedback collector
As an online evaluation

12. Method: Game-based Optimization
“Top-Student Game” in XuetangX
Multi-level optimization
For Candidate Generation
Adjust the Confidence Score
For Classification
Perform as a feature

13. Experiment Dataset: MOOC data from XuetangX and Coursera
For each courses, we select top 800 expanded concepts from 100,000 candidates
Knowledge Base: XLORE (Jin et al., 2018)

14. Our Model achieve a good result with(without) game.
Experiment
Baselines
PR (Graph Based Method)
SEISA (He and Xin, 2011)
EBM (Embedding Based Method) (Mamou et al., 2018)
PUL PU-Learning methods(Wang et al., 2017)
Evaluation Metrics
MAP Mean Average Precision
Results
Our Model achieve a good result with(without) game.

15. Each of the Feature we design is useful.
Experiments
Parameter Analysis
Feature Contribution
Each of the Feature we design is useful.

16. Expansion results satisfied the real MOOC users.
Experiments
Online Evaluation
Cr is the rate of user deletion.(Larger Cr indicates a lower expansion quality)
Expansion results satisfied the real MOOC users.

17. MOOCdata: http://moocdata.cn
We are THU MOOC Team!
Data and Other Interesting Work!!All in
MOOC: A perfect platform of AI in Education
Easy to interact with users
Easy to build fancy functions
Our Publications in
AAAI, NIPS…

18. MOOCdata: http://moocdata.cn
Our Data and Other Interesting Work!!
Students’ behavior
Course Recommendation
QA system
Prerequisite Relation Discovery
Xiaomu: A learning assistant in XuetangX.

19. Thank you! Questions? Knowledge Engineering Group, Tsinghua University
THU MOOC Team: