EDIT 6900: Research Methods in Instructional Technology UGA, Instructional Technology Spring, 2010 If you can hear audio, click If you cannot hear audio, click If you have a question, click Lloyd Rieber Co-Instructor Greg Francom Graduate Assistant TJ Kopcha Co-Instructor

To avoid chaos, if you have a question or comment, click on the “Raise Hand”, but don’t send/speak your message until prompted by me.

Be sure to have the Online Survival Guide by your side.

Preparing for & Attending the Weekly Online Class BEFORE CLASS: –Do all activities by due date in order to be prepared for the live class (i.e. read chapters, view pre-recorded presentations, listen to podcasts, etc.) Check Log on to “Horizon Wimba Live Classroom” –link is on our course’s eLC home page Enter “Research Methods in Instructional Technology” Room Have “Online Survival Guide” by your side

Three Topics for Today Lessons from the “LSAT Logic in Everyday Life” Podcast Considering the Tools of Research –Sources of Information –Measurement as a Tool for Research –Limiting the data –Scales of measurement –Reliability and validity Breakout Room Discussion

Reminder My web hosting company is doing a major upgrade today, starting at 6 pm and lasting about 12 hours NowhereRoad.com may be down for up to 4 hours at some point during this time.

Informal Activity SDC Systematic Data Collection An informal, (hopefully) enjoyable activity designed to give you first-hand experience collecting research data Your Task: Go and research something of interest to you! Report on it informally in writing Give 5 minute oral report 10%, Due: April 14

Informal Activity SDC Ideas Systematic Data Collection Personal Finance Family Health Gardening Nature Pets Sport

Literature Searching Workshop February 17, 2010, 5:00 pm Michael Law, UGA Gwinnett Librarian

First Graded Research Design Activity Begins Next Week! Topic: Identifying a Researchable Question

Questions? Go ahead and enter question in message field, or… Click and wait for my prompt to speak.

“Unchartered Territory”

Let’s choose the person to briefly summarize this week’s podcast…

“Unchartered Territory” Take away points ETS study showed that 4 th graders at 150 (out of 3600) charter schools performed lower than 4 th graders at public schools. Vindication for public schools, or “faulty data…sloppy government stuff”? Is the sample representative? –When controlled for urban areas, no difference in test scores –Perhaps high performing public schoolers stay put? –Comparing only 4 th graders is problematic Need to track improvement over time –Problem of generalizing conclusions to ALL public schools or ALL charter schools in ALL areas

Tools of Research

Sources of Information Beware!

But I heard it on Oprah!

Never use the following as sources of information in your course project Wikipedia, blogs, and most other non- filtered Internet sources Articles from newspapers and magazines like Time and Newsweek Television and radio broadcasts Podcasts Get the idea?

Measurement as a Tool of Research  “Measurement is limiting the data … so that those data may be interpreted and, ultimately, compared to an acceptable qualitative or quantitative standard.”  Substantial - having an obvious basis in the physical world (e.g., an engineer measures the span of a bridge)  Insubstantial - existing only as concepts, ideas, opinions, feelings, or other intangible entities (e.g., the degree to which students have learned)  Data are limited by:  Measurement construct  Instrument capability  Amount of raw information we are prepared to deal with

Isolating meaningful data from most research studies is like _____________.

Husband: I lost a shirt button in the bedroom. Wife: Why are you looking for it in the kitchen? Husband: The light is better in here.

Data Sampling: How Well Do Participants in EDIT 6900 Know Mathematics?

Data Sampling: Is giving this problem to solve sufficient to answer that question?

Interpretation of the Data

Four Scales of Measurement:  Nominal Scale  Ordinal Scale  Interval Scale  Ratio Scale Let’s check our understanding…

When Is It OK to Compute… NominalOrdinalIntervalRatio Frequency distributions (i.e. counting; finding the mode) Yes Median and percentilesNoYes Add or subtractNo Yes Mean, standard deviation, standard error of the mean No Yes RatioNo Yes

What’s your favorite color? ColorCode Blue1 Red2 Yellow3 Green4 Purple5

What’s your favorite color? PersonColorCode 1Blue Red2 4Yellow3 5Blue1 Average color = 1.6

What’s your favorite color? PersonColorCode 1Blue Red2 4Yellow3 5Blue1 Average color = 1.6 This result makes absolutely no sense because the data are nominal. Therefore, we can’t average them!

Two Important Types of Statistics Measures of Central Tendency Measures of variability

Measures of Central Tendency Mean –The average of a set of numbers. Median –Given a set of number arranged in descending order, the median is the number at the midpoint. Mode –Given a set of numbers, the mode is the number with the greatest frequency. Given a normal distribution, these are all the same number.

The Normal Distribution

When they are not the same Illustration taken from “

Examples of when to use median instead of mean Median income –Imagine a neighborhood of 999 extremely poor people and 1 billionaire. Is this a rich neighborhood? Median price of homes

Characteristics of Good Assessment Instruments  Validity –Does the instrument assess what it is supposed to assess  Reliability –People who ‘know the material’ do well, those who don’t do poorly; consistency  Practicality –The instrument can be implemented with relative ease  Efficiency –The instrument takes as little time as necessary to get valid and reliable results

Understanding Reliability & Validity Are your measurements on target?

Understanding Reliability & Validity Reliable, but not valid

Understanding Reliability & Validity Not reliable and not valid

Understanding Reliability & Validity Reliable and valid

Understanding Reliability & Validity Can a test be valid but not reliable?

Break-Out Room Discussion Dr. Kopcha

Questions? Go ahead and enter question in message field, or… Click and wait for my prompt to speak.

To do list Follow the Course Learning Plan!