Two Types of Measurement: a) Substantial b) Insubstantial “Nothing exists that the researcher cannot measure” (Some are just more defined) Two Types of Measurement: a) Substantial b) Insubstantial

Substantial Substantial measurements are things being measured that have an obvious basis in the physical world. Using Quantities: (a number and a unit) The table is 15 inches long Unbiased

Insubstantial Abstract data that exist only as concepts, ideas, opinions, or feelings. Example: asking someone for their opinion of something by asking them their feelings on the subject. Very subjective and biased

Measurement Defined (pg. 24) “Limiting the data of any phenomenon-substantial or insubstantial-so that those data may be interpreted and ultimately compared to an acceptable qualitative or quantitative standard”

Goal for Researcher Measurement and collection of data is to be as OBJECTIVE as possible. If data that is measured is not concrete or substantial, then QUANTIFY abstract or insubstantial data

4 Scales of Measurement 1. Nominal 2. Ordinal 3. Interval 4. Ratio

Nominal Scale You assign names to data in order to measure it Example Measuring a group of children Divide into 2 groups: Girls and Boys Each subgroup is thereby measured by a girl’s name or a boy’s name Only a few statistics are appropriate for analyzing this kind of data: (frequencies, modes, % …Chi square)

Ordinal Scale Compare pieces of data in terms of being greater > or less < than the others. Example Grades of proficiency Skilled Unskilled Overskilled

Ordinal Scale Measurements are relative Type of statistics used expands beyond nominal Examples: Median, percentile rank; Spearman’ rank of Correlation

Interval Scale Uses equal units of measurement Its zero point is established arbitrarily Example Measuring temperature using Fahrenheit Intervals between degrees reflect equal changes in temperature The zero point is not a total absence of heat Example: 0 degress Fahrenheit does not indicate absence of heat

Interval Scale Uses any type of statistic using addition or subtraction because of the “equal “ interval Uses means, standard deviations, Pearson Product Moment Correlation

Ratio Scale Equal measurement units Absolute zero point: Absence of property Can express values in terms of multiples or fractional parts Example Yardstick Kelvin Scale 3 feet = 1 yard Absolute Zero

Validity and Reliability of Measurement

Validity and Reliability Validity and Reliability are the two terms most often used with measurement.

Validity Validity is whether or not a tool of measurement has the ability to properly measure what it is suppose to measure. Example: A test may be intended to measure a certain characteristic, and it may be called a measure of that characteristic, but these things don’t necessarily mean that the test actually measures what its authors say it does. Example” Does an IQ test accurately measure all types of IQ’s? (academic IQ, social IQ, mechanical IQ, etc…

Reliability When the conditions for measurement are consistent for each measurement. Instruments used to measure insubstantial data are less reliable than substantial Ex: On a teacher availability scale a student rates the same teacher a score of 60 one day when the teacher is less available and 95 a different day when the teacher is more available

Relationship Between Validity and Reliability Reliability without validity: archer hitting a target consistently by 1’ away from the bulls-eye (predictable but not accurate) Validity without reliability: archer’s hits centered and near bulls-eye but haphazard (but some reliability Easier to increase the validity of a reliable measure than to increase the reliability of a valid measure

Conclusion Both reliability and validity reflect the degree to which we may have error in our measurements. Validity errors are usually due to the instrument itself, and reliability errors are usually due to the use of the instrument.

Testing Validity and Reliability Which do you think is easier? How would you test validity? How would you test reliability?

Measurements made on you SAT/ACT Final Exam College/Scholarship Interview Height and weight BMI Others?

Gifted Program Placement Pencil version of CogAT, 1st grade Torrance Test for Creativity and Motivation Are these valid? Are these relaible? What is the error if they are not?

Blood Test for Cancer Recurrence Antibody level How do you test to see if this is valid? Reliable? What is the error if the test is wrong? False positives vs False negatives

Optical Scanning for NDT New method that relies on smaller, more inexpensive system Is this valid? Is this reliable? What are the potential errors and cost of error?

Survey and Pre-and-Post Test for Teacher Performance Student surveys Delta between pre and post test as a % of total possible Quantized at 4 levels of students Is this valid? Is this reliable? What are the possible errors and cost of errors?

Testing the validity of an instrument Face Validity: yep, looks like it’s valid Content Validity: extent to which it is representative of the content area or domain being measured Criterion Validity Construct Validity

Criterion Validity Extent to which a measurement instrument’s result correlates with another related measure

Construct Validity extent to which instrument measures an inferred characteristic (e.g., motivation, creativity) should have evidence that test measures construct What kind of evidence do you think demonstrates construct validity?

Testing Reliability Interrater reliability: extent to which two or more persons evaluating give same judgement Internal consistency reliability: extent to which all items in a single instrument yield similar results

More Testing Reliability Equivalent Forms Reliability: extent to which two different versions of the same instrument yield similar results Test-retest Reliability: extent to which same instrument yields same results on different occasions

Assignment Find a group of 3-5 people in the same broad type of internship Create a list of possible measurement tools you might use in your research in ppt Under each tool, comment on validity and reliability and possible methods of testing such We will return to these ppts in the future

Example Social Media Marketing Internships FB statistics Customer survey Expert Quality Measurement Accounting Stats (total income, sales, donations, etc.)

Example Biology Lab Internships PCR Western Blot Necropsy