Kadupitiya JCS Dr. Surangika Ranathunga Prof. Gihan Dias Department of Computer Science and Engineering University of Moratuwa Sri Lanka.

Slides:



Advertisements
Similar presentations
An Introduction to Computer- assisted Assessment Joanna Bull and Ian Hesketh CAA Centre Teaching and Learning Directorate.
Advertisements

Course assessment: Setting and Grading Tests and Examinations By Dr C. Bangira Chinhoyi University of Technology Organized by the Academy of Teaching and.
Xiaomeng Su & Jon Atle Gulla Dept. of Computer and Information Science Norwegian University of Science and Technology Trondheim Norway June 2004 Semantic.
EngageNY.org Scoring the Regents Examination in Algebra I (Common Core)
Algorithmic e-Assessment with DEWIS Dr Rhys Gwynllyw Dr Karen Henderson Senior Lecturers in Mathematics and UWE Learning and Teaching Fellows Department.
MASTERS THESIS DEFENSE QBANK A Web-Based Dynamic Problem Authoring Tool BY ANN PAUL ADVISOR: PROFESSOR CLIFF SHAFFER JUNE 2013 Computer Science Department.
Evaluation and analysis of the application of interactive digital resources in a blended-learning methodology for a computer networks subject F.A. Candelas,
Aplication of CAS in automatic assessment of math skills Przemysław Kajetanowicz ( Jędrzej Wierzejewski
Computational Methods to Vocalize Arabic Texts H. Safadi*, O. Al Dakkak** & N. Ghneim**
Science Fair How To Get Started… (
The role of CAA in Helping Engineering undergraduates to Learn Mathematics David Green, David Pidcock and Aruna Palipana HELM Project Mathematics Education.
PSAT/NMSQT th Grade Advisement. What is the PSAT? The Preliminary SAT/National Merit Scholarship Qualifying Test (PSAT/NMSQT) The Preliminary.
INTRODUCTION OF THE ECTS AT THE B&H UNIVERSITIES UNIVERSITY OF EAST SARAJEVO Faculty of Electrical Engineering WORKLOAD, GRADES AND ECTS Case Study Prof.
Research Methodology Class.   Your report must contains,  Abstract  Chapter 1 - Introduction  Chapter 2 - Literature Review  Chapter 3 - System.
Moodle Quizes Staff Guide. Creating Quizzes Click Add an Activity or Resource With the course in editing mode...
Virtual Tutor Application v1.0 Ruth Agada Dr. Jie Yan Bowie State University Computer Science Department.
Instance Discovery and Schema Matching With Applications to Biological Deep Web Data Integration Tantan Liu, Fan Wang, Gagan Agrawal {liut, wangfa,
Mater Gardens Middle School MATHEMATICS DEPARTMENT WHERE LEARNING HAS NO FINISH LINE ! 1.
Parent Testing Info. Session (March, 16, 2016). FSA Reading and Listening Test Grades 6-10  Students in grades 6,7,8 will have two sessions of 85 minutes.
Algebra 1 Foundations, pg 150 Focus Question How do you write inequalities?  You can use the symbol ______________ to compare two expressions.  Students.
AP Exam Development and Grading The AP Physics exams are developed by a committee of high school and college physics faculty After the exams are administered,
B1 Exam Tips Feedback from the examiners The examiner says…
ORAL ADMINISTRATION GUIDELINES The following slides contain information for Test Administrators providing an oral administration for students taking a.
Computer Representation of Venn and Euler Diagrams Diunuge B. Wijesinghe, Surangika Ranathunga, Gihan Dias Department of Computer Science and Engineering,
TIPS FOR TESTING SUCCESS
Oral Administration Guidelines
Teachers’ evaluation by the Petroleum – Gas University of Ploiești
Automatic Writing Evaluation
Obstacles in combining the use of various tools in solving mathematical problems – why is Copy/Paste often useless Matija Lokar Faculty of mathematics.
SAT.
Queensland University of Technology
New Developments in NYS Assessments
3.1 Fundamentals of algorithms
3rd – 5th Grade Gregory Drive Elementary
Math Field Day Meeting #2 October 28, 2015
Different Types of Testing
Structured Browsing for Unstructured Text
Bay High School, Retired
ICS103 Programming in C Lecture 1: Overview of Computers & Programming
Presented by: Hassan Sayyadi
Grades 6-8 Florida Standards Assessment in English Language Arts
The Florida Standards Assessments: What Every Parent Should Know
Natural Language Processing (NLP)
Video 3: Scoring Policies and
HRM 326 Possible Is Everything/tutorialrank.com
HRM 326 TUTORIALS Lessons in Excellence -- hrm326tutorials.com.
HRM 326 Education for Service/tutorialrank.com
Milwee Middle School Math Night
2.2 Linear Equations Learn about equations and recognize a linear equation Solve linear equations symbolically Solve linear equations graphically Solve.
Office of Education Improvement and Innovation
Key Stage 2 SATs Earsham Primary School Presentation to Parents
9th Grade Literature & Composition
AP CLASS POLICIES One of the objectives of taking AP Calculus is to pass the AP test, thus the following classroom policies will be followed to align with.
Understanding New Expectations for Rigor in Mathematics
Test Design - Mathematics
Python Camp Session 4: GCSE Computing CA Analysis Design Development
F.S.A. Computer-Based Item Types
Key Stage 2 SATs.
Resource Recommendation for AAN
Flowcharts and Pseudo Code
Writing a Free Response Essay
Natural Language Processing (NLP)
Y540 Pure Core Annotated sample assessment materials
Equations and Inequalities Part 1: Expressions vs Equations and Inequalities Essential Question: What is an equation and how is it similar and/or different.
Review of Previous Lesson
Continuous assessment An overview
Key Stage 2 SATs Earsham Primary School Presentation to Parents
Claim 1: Concepts and Procedures
Chapter 1: Creating a Program.
Natural Language Processing (NLP)
Presentation transcript:

Kadupitiya JCS Dr. Surangika Ranathunga Prof. Gihan Dias Department of Computer Science and Engineering University of Moratuwa Sri Lanka

Introduction This is part of an initiative to implement automatic grading of secondary school level mathematical answers. Specifically focus on automatic grading of multi- step answers to word type problems. –Written in Sinhala. 2

Motivation O/L Exam is a major turning point for a student 40%-50% Students fail Mathematics. O/L Mathematics is identified as a subject that needs “Special Attention” by Ministry of Education. “Making Mathematics a favorite Subject” is an educational goal of

Motivation Contd. 4

The most common way of preparing for an exam – working on questions Lack of feedback due to –Large classroom size –Inability to afford individual tuition classes Solution – Having a platform where students can get their solutions automatically assessed 5

Focus of the research 1.Assess answers (written in Sinhala) of word-type problems using a marking rubric 2.Award full/ partial credit to students 3.Give feedback to students 6

Word type problems Short sentences and numerical expressions are included in the answers to this type of questions. Format : – –etc. Examples for these type of questions. Percentage calculations Interest Calculations Measurement calculations(Perimeter, Area, Volume) 7

Example Question %. = *8/100 = = ( )*8/100 = = =

Existing Work Overview of automatic grading A. Bennett - Categorization of Automatic grading of mathematics –Those calling for equations or expressions, where the problem has a single mathematically correct answer that can take many different surface forms –Those calling for one or more instances from a potentially open-ended set of numeric, symbolic, or graphical responses that meet a given set of conditions –Those requiring the student to show the symbolic work leading to the final answer –Those asking for a short text explanation 9

Existing Works Contd. Mathematical Expression Evaluation A. Lan - Mathematical Language Processing (MLP) –It supports partial automatic grading of mathematical expressions. –They have used a clustering based approach using affinity propagation algorithm. –They have not considered the questions that have answers with natural language phrase and a expression separated by equality sign M. Badger - A System implemented to support automatic grading for Mathematics questions in GCSE examination UK. –They have used a limited interface to get the solution from the student in a step- wise manner. –It only focuses on mathematical expressions and numerical calculations 10

Existing Work Contd. Existing Auto grading Systems M-rater –M-rater scoring engine is used for scoring open-ended mathematical responses, such as mathematical expressions, equations and graphs. –System is limited to take only string or numeric responses via single or multiple text boxes. OpenMark –Evaluates multi-step answers –Uses a CAS to evaluate each step against correct answer IMathAS –Uses a Computer Algebra System –Evaluates steps of an answer Doesn’t support Mathematical Word Type Problems 11

Existing Work Contd. Computer Representation of Marking Rubric A. Li - Mathematics Assessment Grid To integrate the mathematical question resources on Internet into a very large, open and virtual questions library. XML+ MathML + SVG = Mathematics Assessment Markup Language (MAML) We have adopted their research with necessary changes 12

Existing Work Contd. Short Sentence Similarity Needed to validate the students final answer units A. Wael - String Based and Corpus Based Similarity –Natural language processing methods such as raw, stop, stem and stopstem used for pre processing –String Similarity (Levenshtein, Jaro, Jaro–Winkler, N-gram, etc.) –Corpus-Based Similarity- Latent Semantic Analysis (LSA) –We have adapted these techniques for Sinhala language M. Mohler-Semantic Similarity –Text-to-text Semantic Similarity for Automatic Short Answer Grading –Used knowledge-based measures and corpus-based measures with Word Net –Supervised approach to improve their earlier results obtained using WordNet based approaches. –We cant use these approaches as “Sinhala” wordnet is still under the implementation 13

Methodology Expression Evaluation Data Preparation Computer Representation of the Marking Rubric, Question, Unit Categorization and the Answer Implementation –Expression Validation Module –Units Validation Module –Automatic Grading Module 14

Data Preparation Interest Calculation Question Sub questions = 05 Percentage Calculation Question Sub questions = Students ( secondary school level ) 300 Answers Tuning Data Set = 100 Answers Testing Data Set = 500 Answers Answers

Computer Representation of the Marking Rubric, Question, Unit Categorization and the Answer Documents used for manual grading are in paper based documents, thus cannot be directly used for automatic grading. Use XML and MathML based representation for these paper based documents 16

Implementation Expression Validation Module Automatic Grading Module Units Validation Module Overview of the system 17

Expression Validation Module Students may write some random expressions that will finally get evaluated to the final answer. Example Question : A person allocate 30% of Rs for a stock purchase. Prove that the amount he allocated was Rs Student answer: Allocations= *600000*0.3 = Rs

Expression Validation Module contd. 19

Units Validation Module To offer full marks for a particular question it should be correct numerically as well as unit wise. Numerical part grading is done by the automatic grading module while unit checking is done in the unit validation module. Partial marks are awarded for the final answer if both numerical grading and unit validation modules return true. Levenshtein (edit) distance based approach is used. 20

Units Validation Module contd. 21

Automatic Grading Module This is the main module of the implemented system. Incorporates expression validation module to grade the expression of the students answer. According to the marking rubric, system decides whether to award marks to an expression or a numeric answer. For final answer grading, it only checks for numerical value equality and unit similarity. 22

Automatic Grading Module contd. 23

Results and discussion Grading module was tested with 500 answers for two word type questions, each having five sub-questions. These two questions were perfectly graded and gave the accuracy of 99%. When considering the sub questions, the accuracy was 99.8%. Manual grading vs. automatic grading for question 01Manual grading vs. automatic grading for question 02 24

Results and discussion contd. Reason for the accuracy drop A person spent 30% of Rs to buy the shares of an organization for Rs 24 per share. The nominal value of a share is Rs. 25. Show that the Nominal value of the bought shares is Rs Correct Answer: Amount of shares=180000/24 = shares 7500 Nominal value of the Bought shares=7500*25 =Rs Ambiguous Answer: Value of a Bought share=180000/24 =Rs Nominal value of the Bought shares = Rs =Rs

Future Work Complete the Sinhala phrase validation module. Identify the type of the errors which students make. Give specific feedback on the error. 26

Thank You 27

28 Questions?