1. Two Types of Measurement: a) Substantial b) Insubstantial
“Nothing exists that the researcher cannot measure” (Some are just more defined)
Two Types of Measurement:
a) Substantial
b) Insubstantial

2. Substantial
Substantial measurements are things being measured that have an obvious basis in the physical world.
Using Quantities: (a number and a unit)
The table is 15 inches long
Unbiased

3. Insubstantial
Abstract data that exist only as concepts, ideas, opinions, or feelings.
Example: asking someone for their opinion of something by asking them their feelings on the subject.
Very subjective and biased

4. Measurement Defined (pg. 24)
“Limiting the data of any phenomenon-substantial or insubstantial-so that those data may be interpreted and ultimately compared to an acceptable qualitative or quantitative standard”

5. Goal for Researcher
Measurement and collection of data is to be as OBJECTIVE as possible.
If data that is measured is not concrete or substantial, then QUANTIFY abstract or insubstantial data

6. 4 Scales of Measurement
1. Nominal
2. Ordinal
3. Interval
4. Ratio

7. Nominal Scale You assign names to data in order to measure it Example
Measuring a group of children
Divide into 2 groups: Girls and Boys
Each subgroup is thereby measured by a girl’s name or a boy’s name
Only a few statistics are appropriate for analyzing this kind of data: (frequencies, modes, % …Chi square)

8. Ordinal Scale
Compare pieces of data in terms of being greater > or less < than the others.
Example
Grades of proficiency
Skilled
Unskilled
Overskilled

9. Ordinal Scale Measurements are relative
Type of statistics used expands beyond nominal
Examples: Median, percentile rank; Spearman’ rank of Correlation

10. Interval Scale
Uses equal units of measurement
Its zero point is established arbitrarily
Example
Measuring temperature using Fahrenheit
Intervals between degrees reflect equal changes in temperature
The zero point is not a total absence of heat
Example: 0 degress Fahrenheit does not indicate absence of heat

11. Interval Scale
Uses any type of statistic using addition or subtraction because of the “equal “ interval
Uses means, standard deviations, Pearson Product Moment Correlation

12. Ratio Scale Equal measurement units
Absolute zero point: Absence of property
Can express values in terms of multiples or fractional parts
Example
Yardstick Kelvin Scale
3 feet = 1 yard Absolute Zero

13. Validity and Reliability of Measurement

14. Validity and Reliability
Validity and Reliability are the two terms most often used with measurement.

15. Validity
Validity is whether or not a tool of measurement has the ability to properly measure what it is suppose to measure.
Example: A test may be intended to measure a certain characteristic, and it may be called a measure of that characteristic, but these things don’t necessarily mean that the test actually measures what its authors say it does.
Example” Does an IQ test accurately measure all types of IQ’s? (academic IQ, social IQ, mechanical IQ, etc…

16. Reliability
When the conditions for measurement are consistent for each measurement.
Instruments used to measure insubstantial data are less reliable than substantial
Ex: On a teacher availability scale a student rates the same teacher a score of 60 one day when the teacher is less available and 95 a different day when the teacher is more available

17. Relationship Between Validity and Reliability
Reliability without validity: archer hitting a target consistently by 1’ away from the bulls-eye (predictable but not accurate)
Validity without reliability: archer’s hits centered and near bulls-eye but haphazard (but some reliability
Easier to increase the validity of a reliable measure than to increase the reliability of a valid measure

18. Conclusion
Both reliability and validity reflect the degree to which we may have error in our measurements.
Validity errors are usually due to the instrument itself, and reliability errors are usually due to the use of the instrument.

19. Testing Validity and Reliability
Which do you think is easier?
How would you test validity?
How would you test reliability?

20. Measurements made on you
SAT/ACT
Final Exam
College/Scholarship Interview
Height and weight
BMI
Others?

21. Gifted Program Placement
Pencil version of CogAT, 1st grade
Torrance Test for Creativity and Motivation
Are these valid?
Are these relaible?
What is the error if they are not?

22. Blood Test for Cancer Recurrence
Antibody level
How do you test to see if this is valid?
Reliable?
What is the error if the test is wrong?
False positives vs False negatives

23. Optical Scanning for NDT
New method that relies on smaller, more inexpensive system
Is this valid?
Is this reliable?
What are the potential errors and cost of error?

24. Survey and Pre-and-Post Test for Teacher Performance
Student surveys
Delta between pre and post test as a % of total possible
Quantized at 4 levels of students
Is this valid?
Is this reliable?
What are the possible errors and cost of errors?

25. Testing the validity of an instrument
Face Validity: yep, looks like it’s valid
Content Validity: extent to which it is representative of the content area or domain being measured
Criterion Validity
Construct Validity

26. Criterion Validity
Extent to which a measurement instrument’s result correlates with another related measure

27. Construct Validity
extent to which instrument measures an inferred characteristic (e.g., motivation, creativity)
should have evidence that test measures construct
What kind of evidence do you think demonstrates construct validity?

28. Testing Reliability
Interrater reliability: extent to which two or more persons evaluating give same judgement
Internal consistency reliability: extent to which all items in a single instrument yield similar results

29. More Testing Reliability
Equivalent Forms Reliability: extent to which two different versions of the same instrument yield similar results
Test-retest Reliability: extent to which same instrument yields same results on different occasions

30. Assignment
Find a group of 3-5 people in the same broad type of internship
Create a list of possible measurement tools you might use in your research in ppt
Under each tool, comment on validity and reliability and possible methods of testing such
We will return to these ppts in the future

31. Example Social Media Marketing Internships FB statistics
Customer survey
Expert Quality Measurement
Accounting Stats (total income, sales, donations, etc.)

32. Example
Biology Lab Internships
PCR
Western Blot
Necropsy