Department of Humanities College of Sciences and Liberal Arts Writing Program Assessment at New Jersey Institute of Technology Carol Siri Johnson Associate.

Slides:

Advertisements

Similar presentations

The SUNY Assessment Initiative: Best Practices for Mapping Program Objectives to Curricular Activities Presentation to Middle States Commission on Higher.

Advertisements

SLO Course Assessment in 5 Easy Steps Vivian Mun, Ed.D.

Daniel Peck January 28, SLOs versus Course Objectives Student Learning Outcomes for the classroom describe the knowledge, skills, abilities.

Interpreting Data & “Closing the Loop” Margaret Kasimatis, PhD VP for Academic Planning & Effectiveness.

PEER REVIEW OF TEACHING WORKSHOP SUSAN S. WILLIAMS VICE DEAN ALAN KALISH DIRECTOR, UNIVERSITY CENTER FOR ADVANCEMENT OF TEACHING ASC CHAIRS — JAN. 30,

Holyoke Public Schools Professional Development By, Judy Taylor

Student Evaluations. Introduction: Conducted: Qualtrics Survey Fall 2011 o Sample Size: 642 o FT Tenured: 158, FT Untenured: 59 o Adjunct: 190 o Students:

Faculty Evaluation Systems: Student Evaluations of Faculty What are we measuring? “The evaluation of teachers is a mark of a good college.” Ernest Boyer,

CWU Writing Assessment. Why? To improve student learning.

Dr. Barbara M. Wheeling Associate Professor College of Business Director of Accreditation and Assessment MSUB Coordinator of Institutional Assessment.

Accreditation Strategy for the BYU CE En Dept. Presentation to External Review Board October 20, 2000.

Program Evaluation Data Department of Education SUNY Cortland.

What assignments do university professors ask Nursing students to write? Roger Graves Director, Writing Across the Curriculum Professor, English and Film.

Report to External Review Board Brigham Young University Civil & Environmental Engineering October 14, 2005.

 Evolving Assessment & ePortfolio at LaGuardia Community College J. Elizabeth Clark LaGuardia Community College--CUNY.

Outcomes, Assessment and Improvement Student Learning Outcomes Implementation at Crafton Hills College.

UNF Writing Program and Center Linda Howell, Director UNF Writing Program David MacKinnon, Coordinator UNF Writing Center.

 8 years teaching English at SMSU  7 ½ years mentoring high school CEP teachers  Professional awareness of disconnect in high school and college standards.

Academic Assessment Report for the Academic Year Antioch University New England Office of Academic Assessment Tom Julius, Ed.D., Director Submitted.

Yanling Sun, Ph.D Carolyn Masterson, Ed. D. Implementing ePortfolio among Pre-service Teachers and Interns Montclair State University.

Graduate Program Review Where We Are, Where We Are Headed and Why Duane K. Larick, Associate Graduate Dean Presentation to Directors of Graduate Programs.

PhD Research Seminar Series: Writing the Method Section Dr. K. A. Korb University of Jos.

Joint Advisory Board/Working Group Meeting November 18, 2010 School of Education.

MASTER’S IN EDUCATION DEGREE ADULT & CAREER EDUCATION BUSINESS EDUCATION AND INFORMATION TECHNOLOGY OPTION 9/17/2012.

Overview of the Department’s ABET Criterion 3 Assessment Process.

Central Texas College College Prep Mathematics and English Language Arts Courses

 “…the systematic collection, review, and use of information about educational programs undertaken for the purpose of improving student learning and.

Move On When Ready Standard Diploma Provision May 8, 2014.

Susan Agre-Kippenhan, Portland State University Professor, Art Department Evaluating the Effectiveness of Service Learning.

Three Levels of Assessment: Portfolio Data, Curriculum Inventory, Standardized Assessment Results Dr. Kathleen DaBoll-Lavoie, PhD Department of Inclusive.

2004 Southern Business Administration Association Meeting, Myrtle Beach, South Carolina The UCF College of Business Approach to Assessing Student Learning.

The Basics of February 25, 2012 EDTN.  The ACCJC requires it for accreditation  To make course, degree, certificate, and GE outcomes more relevant 

Department of English and Foreign Languages School of Liberal Arts B.A. in English Education CIP Code Program Code Program Quality Improvement.

Evidence of Student Learning Fall Faculty Seminar Office of Institutional Research and Assessment August 15, 2012.

IS Faculty Perceptions of ABET Accreditation Presented by Bruce White and Wendy Ceccucci.

ASSESSMENT OF CORE SKILLS/ GENERAL EDUCATION OUTCOMES Angelina Hill, PhD Associate Director, Office of Academic Assessment.

Full-Time Faculty In-Service: Program and Student Learning Outcomes Fall 2005.

Joanna Fulbright Jeremy Nicholson Chris Lorch Ozarka College, Melbourne, Arkansas Based on a process created by the Des Moines Area Community College System.

At Oconomowoc High School.  Aims/Goals  Curriculum  Benefits  Cost  Preparation  Scheduling.

Preparing Future Faculty in Engineering & Applied Science A program for graduate students who: are in the College of Engineering and Applied Science are.

Everything You Need to Know for Spring 2010 S. Kashima.

Valuing E- Portfolios: Formative and Summative Assessment Jane Moore, National-Louis University David Wicks, Seattle Pacific University.

From the IR Office To the Classroom: The Role of Assessment in Student Learning Dr. John W. Quinley Dr. Brett Parker.

Assessing Your Assessments: The Authentic Assessment Challenge Dr. Diane King Director, Curriculum Development School of Computer & Engineering Technologies.

Telecommunications Engineering The University of Texas at Dallas The Erik Jonsson School of Engineering and Computer Science Jonsson School Industrial.

Connecting Course Goals, Assignments, and Assessment Faculty Development for Student Success at Prince George’s Community College William Peirce

Holistic Rating Training Requirements Texas Education Agency Student Assessment Division ©2010 TEXAS EDUCATION AGENCY. ALL RIGHTS RESERVED.

THE SLO PROCESS #1 create/update SLO’s, rubrics, & assessment methods #2 assess all students in all sections of all courses #3 maintain SLO assessment.

Sharon M. Livingston, Ph.D. Assistant Professor and Director of Assessment Department of Education LaGrange College LaGrange, GA GaPSC Regional Assessment.

English Department Glendale Community College English Assessment Program Documents are available online on the Assessment Program webAssessment Program.

Assessment Committee. Communicate what we Think about Assessment. Learn to Integrate assessment into our teaching practice and Act on it toward continuous.

Assessment of Course-Level Learning Outcomes in Psychology.

1 Learning Outcomes Assessment: An Overview of the Process at Texas State Beth Wuest Director, Academic Development and Assessment Lisa Garza Director,

ARCADIA UNIVERSITY UNDERGRADUATE CURRICULUM Get Ready…..For the Journey of Your Life Andrea Crivelli-Kovach Associate Professor & Director Community Health.

Challenges of Quantitative Reasoning Assessment Donna L. Sundre Center for Assessment and Research Studies

PROGRAM ASSESSMENT BASICS Alan Kalish, PhD Teresa Johnson, PhD Director Assistant Director University Center for the Advancement of Teaching.

An Institutional Writing Assessment Project Dr. Loraine Phillips Texas A&M University Dr. Yan Zhang University of Maryland University College October 2010.

Transfer Course Credit – Institutions of Higher Education Credit for Prior Learning Industry Recognized Credentials/Test Credit AGC – April 2016.

Lessons Learned. Communication, Communication, Communication Collaborative effort Support of all stakeholders Teachers, Principals, Supervisors, Students,

MUS Outcomes Assessment Workshop University-wide Program-level Writing Assessment at The University of Montana Beverly Ann Chin Chair, Writing Committee.

Transition to Kean (T 2 K): A Comprehensive Program to Assist Students in Transition Presented by Terry Y. Fung, Ph.D. Director of General Education and.

Mary Ann Roe e-Colorado Portal Coordinator Colorado Department of Labor and Employment Jennifer Jirous Computer Information Systems Faculty Pikes Peak.

Information Literacy at NJIT: Toward Validity ETS ICT National Advisory Committee July 17-19, 2006 The NJIT Team Davida Scharf, Director of Reference,

Consider Your Audience

Phyllis Lynch, PhD Director, Instruction, Assessment and Curriculum

The NJIT Team Davida Scharf, Director of Reference, Van Houten Library

Institution Name.

Department of Computer Science The University of Texas at Dallas

NCAA Eligibility Center Academic requirements

Developing a Rubric for Assessment

Presentation transcript:

Department of Humanities College of Sciences and Liberal Arts Writing Program Assessment at New Jersey Institute of Technology Carol Siri Johnson Associate Professor and Chair

Outline of Presentation Personal Introduction New Jersey Institute of Technology ABET Writing / Technical Writing Assessment Using Online Portfolios Using Analytic Scoring Examples of Data Analysis Results

Personal Introduction Mount Holyoke College English Ph.D., but few jobs New – Technical Writing Field Tenure-track Position at NJIT Chair, Research EA and TC History

My Assessment History Knew nothing about it Disliked standardized tests Was unconvinced of its value Then I saw a best-paper reading The Power of Assessment

Assessment at NJIT Department of Humanities active in assessment for ~20 years Previous Chair, NJCBSPT Norbert Elliot and ETS Course / Program Assessment Analytic Online Portfolio Assessment

On a Scale – Norbert Elliot

What is Assessment? ●Giving a student a grade? ●End-of-semester course evaluations ? ●Faculty review of portfolios? ●Student surveys? ●The chair’s opinion?

What is Assessment? ●For Teachers, GRADES ●For Departments, COMMON EXAMS ●For Administration, COURSE EVALUATIONS ●For Learning, OUTCOMES – stating and reaching curricular goals in Courses and Programs

Assessment Can Be: Program assessment Curricular assessment Method of automatic updating Method of teaching collaboration Gathering statistics for multiple purposes.

ABET Accreditation Board for Engineering and Technology

ABET’s 8 General Criteria:

#4 Continuous Improvement “The results of these evaluations must be systematically utilized as input for the continuous improvement of the program.”

Outcomes Assessment Cycle

NJIT Core Curriculum – GURs (General University Requirements) ● English Composition – 6 credits ● Cultural History – 6 credits ● Humanities Electives – 6 credits ● Senior Seminar – 3 credits

GURs Outcomes Assessment ●English Composition – every semester ●Cultural History – every other spring ●Humanities Electives – every other fall ●Senior Seminar – every semester

Assessment Basics – Validity The issue of “validity” means we are measuring what we want to measure; the test is appropriate to the subject The problem with writing assessment has been using multiple choice tests only – they alone are not valid as a measure of writing Multiple measures are often the best

Assessment Basics – “Associative” Validity Correlating Multiple Inputs: Machine-scored essay scores Portfolio scores Grade in course Student GPA

Assessment Basics – Reliability Can we be sure that the measurement, if taken again, would yield the same results? Inter-reader Reliability (scored by two separate readers, discrepancies resolved by a third) Content Reliability (portfolio contains work from entire semester)

Two Important Steps: Decide on and define goals or criteriaDecide on and define goals or criteria Develop method to assess outcomes (goals/criteria)Develop method to assess outcomes (goals/criteria)

Large Scale Writing Assessment Initiatives ●1950sETS – Holistic Scoring ●1970sCalifornia (Edward White) ●1980sNew York (CUNY) ●1990sNew Jersey (NJCBSPT) ●2000sNJIT – Program Assessment

Example: Technical Communication Analytic Online Portfolio Assessment Established Criteria by “Online Delphi” – exchanges by facultyEstablished Criteria by “Online Delphi” – exchanges by faculty 2004 – Failed effort (network down)2004 – Failed effort (network down) – Successful Analytic Online Portfolio Assessment, every semester – Successful Analytic Online Portfolio Assessment, every semester 2009-Present – once per year2009-Present – once per year

Technical Communication Cycle

Using Online Portfolios

Procedure 1.Have Students Make Online Portfolios with Examples of the Main Assignments 2.Collect URLs 3.Select Random Sampling (~60-100) 4.All Instructors Meet 5.~1 Hour Calibration with Sample Portfolios where instructors discuss scoring processes 6.~3 Hours Scoring 7.Assessment Director Conducts Necessary Adjudications 8.Enter Scores into Excel or SPSS Dataset

Old Humanities Rubric

Old Score Sheets

Revised Score Sheet

Sample Data Collection

A Community of Assessment Shared Instructional Content Self-Evaluation and Norming Curricular Goals Evaluation Continual Improvement (ABET) Interesting Statistics