Professor Diane Litman

Slides:



Advertisements
Similar presentations
Discussion (20 points) and Final Assignment (200 points)
Advertisements

COMP-400 Introduction and Orientation Winter 2006 January 19, 2006 School of Computer Science McGill University.
Natural Language Processing for Writing Research: From Peer Review to Automated Assessment Diane Litman Senior Scientist, Learning Research & Development.
Styles of Leadership LET II. Introduction Leadership styles are the pattern of behaviors that one uses to influence others. You can influence others in.
Writing: 3rd Grade by Shira Ackerman, MAShira Ackerman, MA Third graders grow as writers as they write more structured and complex pieces. They continue.
Résumé / C.V. 3 rd Lecture. What is a Résumé? A résumé is a brief document that summarizes your education, employment history, and experiences that are.
Experiences with Undergraduate Research (Natural Language Processing for Educational Applications) Professor Diane Litman University of Pittsburgh.
Computational Models of Discourse Analysis Carolyn Penstein Rosé Language Technologies Institute/ Human-Computer Interaction Institute.
Malaria: How it affects the lives on Earth By: Nyah Walters.
Introduction to CSCI 1311 Dr. Mark C. Lewis
Natural Language Processing for Enhancing Teaching and Learning
Welcome to Alpha Pod We’re glad you’re here!.
Reading, Analyzing, and Writing the SAT Essay
A.M. Kulp…Grade 5… A New Adventure
Texas Success Initiative (TSI)
D I S C O V E R Y Challenge.
Texas Success Initiative (TSI)
*The claim is your topic/main idea of essay
SMARTER BALANCED Student Overview
Mrs. Roque Fifth Grade English Language Arts
Natural Language Processing for Enhancing Teaching and Learning
BCR DAY!.
Writing Paper Three Monday, November 2.
FACILITATOR: Professor Pandeli Glavanis (PhD) Associate Director, CLT
The ability to communicate effectively in the workplace is essential
Thesis-based Writing.
Writing Workshop: Courage & heroism
Naviance: Do What You Are Personality Survey
Activity 2.8: Gathering evidence from news articles
Persuasive Writing.
Texas Success Initiative (TSI)
Activity 2.11: Understanding argumentative elements
CSCI 5582 Artificial Intelligence
PURPOSE/FOCUS/ORGANIZATION
PURPOSE/FOCUS/ORGANIZATION
How to become a Crime Scene Investigator
PURPOSE/FOCUS/ORGANIZATION
Writing and Feedback.
Peer Reviews Tips for the author.
9th Grade Literature & Composition
LIFTING LIVES FOR GOOD This presentation is about how Oxfam works.
RELIGIOUS ATTITUDES TO WORLD POVERTY.
Lecturer: Geoff Hulten TAs: Kousuke Ariga & Angli Liu
Classroom Newsletter May 7, 2018
PURPOSE/FOCUS/ORGANIZATION
Developing 21st Century Classrooms: Connecting the Dots IV
You are more than a score.
Taking the guess work out of answering questions:
Creating Our CommonWealth… in western Mass
I will understand why people create government.
Mrs. Burrell’s Class Classroom Newsletter April 23, 2018
Information for Students and Families
Final exam project guidance
College Prep for Freshmen
My Performance Appraisal How to write SMART objectives
Argumentative writing
SMARTER BALANCED Student Overview
SCARAB.
CBC An overview.
Presentation and project
NC Tenth Grade Writing Test
Building Body Paragraphs
Task Criteria – Text-based Argument Rubric
How “Appeals” and Exam Estimates work
High School 101: Promotion and Retention Policy, GPA, Diploma of Distinction, and Graduation Requirements.
Argumentative writing
Welcome! to Ms. O’Connor’s Grade 7 English Class 8/26/2019.
Course and Placement Information
Engagement Planning - Communications
Something to Think About
Presentation transcript:

Professor Diane Litman RoboGrading Professor Diane Litman

An Example Writing Task Time for Kids

Rubric for Grading Use of Evidence 1 2 3 4 Features one or no pieces of evidence Features at least 2 Features at least 3 Selects inappropriate or little evidence from the text; may have serious factual errors and omissions Selects some appropriate but general evidence from the text; may contain a factual error or omission Selects appropriate and concrete, specific evidence from the text Selects detailed, precise, and significant evidence from the text Demonstrates little or no development or use of selected evidence Demonstrates limited development Demonstrates use of selected details from the text to support key idea Demonstrates integral use of selected details from the text to support and extend key idea Summarize entire text or copies heavily from text Evidence provided may be listed in a sentence, not expanded upon Attempts to elaborate upon Evidence Evidence must be used to support key idea / inference(s)

Audience Participation: Evidence Scoring (1 or 4?) Student 1: Yes, because even though proverty is still going on now it does not mean that it can not be stop. Hannah thinks that proverty will end by 2015 but you never know. The world is going to increase more stores and schools. But if everyone really tries to end proverty I believe it can be done. Maybe starting with recycling and taking shorter showers, but no really short that you don't get clean. Then maybe if we make more money or earn it we can donate it to any charity in the world. Proverty is not on in Africa, it's practiclly every where! Even though Africa got better it didn't end proverty. Maybe they should make a law or something that says and declare that proverty needs to need. There's no specic date when it will end but it will. When it does I am going to be so proud, wheather I'm alive or not.

Audience Participation: Evidence Scoring (1 or 4?) Student 2: I was convinced that winning the fight of poverty is achievable in our lifetime. Many people couldn't afford medicine or bed nets to be treated for malaria . Many children had died from this dieseuse even though it could be treated easily. But now, bed nets are used in every sleeping site . And the medicine is free of charge. Another example is that the farmers' crops are dying because they could not afford the nessacary fertilizer and irrigation . But they are now, making progess. Farmers now have fertilizer and water to give to the crops. Also with seeds and the proper tools . Third, kids in Sauri were not well educated. Many families couldn't afford school . Even at school there was no lunch . Students were exhausted from each day of school. Now, school is free . Children excited to learn now can and they do have midday meals . Finally, Sauri is making great progress. If they keep it up that city will no longer be in poverty. Then the Millennium Village project can move on to help other countries in need.

RoboGrading Student 1: Yes, because even though proverty is still going on now it does not mean that it can not be stop. Hannah thinks that proverty will end by 2015 but you never know. The world is going to increase more stores and schools. But if everyone really tries to end proverty I believe it can be done. Maybe starting with recycling and taking shorter showers, but no really short that you don't get clean. Then maybe if we make more money or earn it we can donate it to any charity in the world. Proverty is not on in Africa, it's practiclly every where! Even though Africa got better it didn't end proverty. Maybe they should make a law or something that says and declare that proverty needs to need. There's no specic date when it will end but it will. When it does I am going to be so proud, wheather I'm alive or not. (SCORE=1)

RoboGrading Student 2: I was convinced that winning the fight of poverty is achievable in our lifetime. Many people couldn't afford medicine or bed nets to be treated for malaria . Many children had died from this dieseuse even though it could be treated easily. But now, bed nets are used in every sleeping site . And the medicine is free of charge. Another example is that the farmers' crops are dying because they could not afford the nessacary fertilizer and irrigation . But they are now, making progess. Farmers now have fertilizer and water to give to the crops. Also with seeds and the proper tools . Third, kids in Sauri were not well educated. Many families couldn't afford school . Even at school there was no lunch . Students were exhausted from each day of school. Now, school is free . Children excited to learn now can and they do have midday meals . Finally, Sauri is making great progress. If they keep it up that city will no longer be in poverty. Then the Millennium Village project can move on to help other countries in need. (SCORE=4)

Automatic Scoring Approach Natural language processing on the essays Machine learning using the NLP output Experts derive a decision tree from the rubric Define some features based on the tree and extract them from the essays Perform supervised machine learning using extracted features

Natural Language Processing (NLP) Getting computers to perform useful and interesting tasks involving human languages languages such as English, Spanish, Chinese, etc. as opposed to computer languages such as Python

Why is NLP needed? An enormous amount of machine readable text, audio, and video is now available Conversational agents such as Siri and Alexa are becoming an important form of human-computer communication

Litman Lab Undergraduate Research Connecting I recruit (class performance, email to colleagues) I advertise (Pitt’s First Experiences in Research Program) RoboGrader is a listed project, deadline soon! Another project is also available Students initiate contact Ideal prerequisites: one or more of Artificial Intelligence, Natural Language Processing (via CS or linguistics), Machine Learning, Vision Implementing Employment or Credit Individual and/or group meetings Interaction with graduate students

Selected Prior Projects / Outcomes First-authored publications CS Day Award for Best Undergraduate Student Poster Honorable Mention in the Computing Research Association's Outstanding Undergraduate Award National Science Foundation Graduate Research Fellowship Accepted for PhD programs in Computer Science

Teams Project Process conversations using NLP Applications Conversational agents Browsers for (un)successful teams

Data Experimental Design Audio-Video Questionnaires 47 hours 63 teams 216 individuals