NETWORK-BASED MODEL OF LEARNING Akanksha Bapna, Aabhas Chauhan 10th January 2018 EVALDESIGN Twitter: @evaldesign www.evaldesign.com
Outline Objective Methodology Education Network Analysis Data Preprocessing Node and Relation Extraction Algorithm Relation Filtering Education Network Analysis Limitations & Next Steps Applications Outline www.evaldesign.com
Understand the complexity of education and learning processes OBJECTIVE www.evaldesign.com
Methodology www.evaldesign.com
Database From 1965 onwards Data also consists of citations, keywords, full text, abstract Only Abstracts Full text: Over 400 GB data www.evaldesign.com
Methodology www.evaldesign.com
Preprocessing Peer reviewed article abstracts only (859,821) Sentence Segmentation Parallel creation of Test dataset of 5000 abstracts randomly selected Reference dataset Stanford CoreNLP toolkit analyzes individual sentences Part of Speech (POS) tagging Named Entity Recognition (NER) Lemmatization (thinking, thought, thinks – think) Dependency mapping www.evaldesign.com
Sample Sentence “Formatting magazines help students learn new computer skills and promote creativity.” -Johnstone, C., Figueroa, C., Attali, Y., Stone, E., and Laitusis, C. (2013)* * Results of a Cognitive Interview Study of Immediate Feedback and Revision Opportunities for Students with Disabilities in Large Scale Assessments. Synthesis Report 92. www.evaldesign.com
Preprocessing: CoreNLP output www.evaldesign.com
Develop Node Extraction Algorithm Common Nouns Magazines Students Creativity Adjectives + Common Nouns New Computer Skills Noun Compounds Formatting Magazines www.evaldesign.com
Preprocessing: CoreNLP output www.evaldesign.com
Develop Relation Extraction Algorithm Relation – “Linking word(s)” between “subject” and “object” Heuristic Rules Avoid self loops Avoid phrases like “author says”, “objective of the paper” Check for conjunction and negations Output Triplet (subject, linking word(s), object) (object, linking word(s), subject) www.evaldesign.com
Node Extraction and Relation Extraction Algorithm output S.No. Triplet 1 (Formatting magazines, help, students) 2 (students, learn, new computer skills) 3 (Formatting magazines, help learn, new computer skills) 4 (students, promote, creativity) 5 (Formatting magazines, help promote, creativity) www.evaldesign.com
Methodology www.evaldesign.com
Relation Filtering Deep Learning-based Word2Vec model Cosine Similarity Threshold Based on F-measure Increase/Decrease Used sample Dataset B www.evaldesign.com
Methodology www.evaldesign.com
Relation Filtering Output Triplet Direction Cosine Similarity (Threshold 0.04) (Formatting magazines, help, students) -0.0045 (students, learn, new computer skills) 0.0500 (Formatting magazines, help learn, new computer skills) 0.0459 (students, promote, creativity) 0.2922 (Formatting magazines, help promote, creativity) 0.1706 www.evaldesign.com
Output: Network of 88,411 nodes www.evaldesign.com
Education is Complex! www.evaldesign.com
The Network Complexity S.No. Degree of node 𝒅 Number of nodes 1 𝑑>100 147 2 100≥𝑑>50 210 3 50≥𝑑>10 1,604 4 𝑑≤10 86,450 www.evaldesign.com
The Network Output www.evaldesign.com
(Student, Teacher) Relation No. of intermediate Nodes No. of Nodes 1 182 2 3,248 3 120,776 www.evaldesign.com
www.evaldesign.com
Summary ERIC Database: 1,596,398 article abstracts Peer reviewed article abstracts: 859,821 Testing over Dataset B Recall: 80% Precision: 77% F-measure: 79% Cosine Similarity Threshold: (0.04, 0.15) Final network: 1,25,522 (node, edge, node) triplets www.evaldesign.com
Limitations & Next Steps Specific to current corpus Noise exists Triplets like (students, promote, creativity) couldn’t be filtered out Does not analyze inter-sentence relation Does not take geography into account www.evaldesign.com
Applications Design learning experiences Design interventions Identify gaps in research Self-evaluation platform for schools School leaders - Efficient allocation of school resources Teachers - Determine best topics for new class Students – Explore learning concepts at own pace www.evaldesign.com
THANK YOU! www.evaldesign.com