1. Selecting a Sample

2. Sampling Select participants for study Select participants for study Must represent a larger group Must represent a larger group Picked from a population Picked from a population

3. It’s all about convenience Total population too large Total population too large Would be too costly Would be too costly Sample done well, results can become a generalization Sample done well, results can become a generalization

4. Defining Populations Target – entire group (ex. All Maryland 5 th grade students) Target – entire group (ex. All Maryland 5 th grade students) Accessible – realistic number of participants Accessible – realistic number of participants

5. Simple random sampling All individuals in population All individuals in population –Have equal and independent chance of being selected –Best way to obtain a representative sample of your population –Table of random numbers may be used

6. Stratified Sampling Examples in text Examples in text Equal-sized stratified groups Equal-sized stratified groups Proportional stratified Proportional stratified

7. Cluster sampling Intact groups are selected Intact groups are selected Example – instead of a sample of all 5 th graders, one class would be used Example – instead of a sample of all 5 th graders, one class would be used Similar characteristics and abilities Similar characteristics and abilities Systematic Sampling Systematic Sampling –Rarely used (every Kth name) see pg. 108

8. Sample size? No agreement here No agreement here 30 is usual guide for some quantitative 30 is usual guide for some quantitative 60 needed however for causal- comparative and experimental 60 needed however for causal- comparative and experimental

9. Qualitative sampling Select small number Select small number Persons selected must help researcher understand certain phenomenon Persons selected must help researcher understand certain phenomenon Intensity, homogenous, snowball sampling are a few examples from text Intensity, homogenous, snowball sampling are a few examples from text

10. MeasuringInstruments

11. Data All types of research require the collection, analysis, and interpretation of data. All types of research require the collection, analysis, and interpretation of data. Data are pieces of evidence used to examine a research topic or test a hypothesis. Data are pieces of evidence used to examine a research topic or test a hypothesis.

12. Constructs Mental abstractions such as personality, creativity, and intelligence that cannot be measured or observed directly. Mental abstractions such as personality, creativity, and intelligence that cannot be measured or observed directly. Constructs become variables when different levels or scores can be used to measure the constructs (example: Personality / introverts vs. extroverts). Constructs become variables when different levels or scores can be used to measure the constructs (example: Personality / introverts vs. extroverts).

13. Variables/Instruments Variable Variable –A concept that can assume any one of a range of values, e.g., intelligence, height, aptitude. Instrument Instrument –Tool used to collect data (Iowa Test of Basic Skills).

14. Characteristics of Measuring Instruments Three main ways to collect data: Three main ways to collect data: –Administer an existing instrument. –Construct an instrument. –Observation(record naturally occurring events). –Collect existing data.

15. Measurement Scale Group of several related statements that participants select to indicate degree of agreement or lack of agreement. Group of several related statements that participants select to indicate degree of agreement or lack of agreement. There are four types of measurement scales and associated variables. There are four types of measurement scales and associated variables.

16. Nominal Variables Also called categorical variable. Also called categorical variable. Represents lowest form of measurement. Represents lowest form of measurement. Simply classifies persons. Simply classifies persons. –Female; male –Public or private school Represented by a number (1-female; 2- male) – doesn’t mean one category is higher or lower than another. Represented by a number (1-female; 2- male) – doesn’t mean one category is higher or lower than another.

17. Ordinal variables Classifies persons or objects. Classifies persons or objects. In addition, it ranks them In addition, it ranks them –Ex. – height, #1 = 6’ 5” ; #2 = 6’ 0”

18. Interval variables Has all characteristics of nominal and ordinal variables. Has all characteristics of nominal and ordinal variables. Also has equal intervals. Also has equal intervals. –The difference between a score of 70 and 80 is basically the same as the difference between a 90 and 100. –A score of zero does not mean a total lack of knowledge.

19. Ratio variables Represents highest level of measurement. Represents highest level of measurement. In addition to having all characteristics of the three previous variables, it has a true zero point. In addition to having all characteristics of the three previous variables, it has a true zero point. Mainly used for physical measures. Mainly used for physical measures. –One person is 5 feet tall, and her friend is two thirds as tall as she.

20. Qualitative and Quantitative Variables Nominal variables Nominal variables –Show qualitative differences only –i.e. religion, gender, political party Ordinal, interval, and ratio variables Ordinal, interval, and ratio variables –Show quantitative differences –i.e. test scores, heights, age, class size

21. Dependent and Independent Dependent variable Dependent variable –Depends on or caused by: Independent variable Independent variable –Manipulated by researcher

22. Testing Instruments It is more time efficient and less skillful to select an existing instrument than to create a new one that measures the same thing. It is more time efficient and less skillful to select an existing instrument than to create a new one that measures the same thing.

23. Standardized Tests There are thousands of standardized tests that measure a broad range of variables. There are thousands of standardized tests that measure a broad range of variables. A standardized test is one that is administered, scored, and interpreted in the same way no matter when and where it is administered. A standardized test is one that is administered, scored, and interpreted in the same way no matter when and where it is administered.

24. Tests Most tests are paper and pencil. Most tests are paper and pencil. Common formats are: Common formats are: –true/false –multiple choice –essay/short answer

25. Interpreting Test Results Raw scores indicate the number of items a person got right. Raw scores indicate the number of items a person got right. They can be transformed into derived scores (percentile ranks, stanines, and standard scores). They can be transformed into derived scores (percentile ranks, stanines, and standard scores). Derived scores are most often used with standardized tests. Derived scores are most often used with standardized tests.

26. Norm Reference Scores Compares a students test performance to the performance of other test takers. Compares a students test performance to the performance of other test takers.

27. Criterion-Reference Scores Compares a student’s test performance to predetermined standards of performance. Compares a student’s test performance to predetermined standards of performance.

28. Self Reference Scores Compares a students performance over time. Compares a students performance over time. Looks for improvement or regression Looks for improvement or regression –Similar or equal assessment is repeated over time

29. Types of Tests Achievement tests measure the current status of individuals on school taught subjects. Achievement tests measure the current status of individuals on school taught subjects. Aptitude tests are used to predict how well a test taker is likely to perform in the future. Aptitude tests are used to predict how well a test taker is likely to perform in the future.

30. Types of Tests Affective tests measure characteristics such as interest, values, attitude, and personality. Affective tests measure characteristics such as interest, values, attitude, and personality. Projective tests are not self-reporting instruments. They present an ambiguous situation to the test taker that requires him or her to “project” his or her true feelings on the ambiguous situation. Projective tests are not self-reporting instruments. They present an ambiguous situation to the test taker that requires him or her to “project” his or her true feelings on the ambiguous situation.

31. Validity of Measuring Instruments Most important quality of a test. It is the degree to which a test measures what it is supposed to measure, and consequently, permits appropriate interpretations of test scores. Most important quality of a test. It is the degree to which a test measures what it is supposed to measure, and consequently, permits appropriate interpretations of test scores.

32. A test is not valid per se; it is valid for a particular interpretation and for a particular group. Each intended score interpretation requires its own validation. A test is not valid per se; it is valid for a particular interpretation and for a particular group. Each intended score interpretation requires its own validation.

33. Validation Validation is a matter of degree; tests are not valid or invalid, they are highly valid, moderately valid, or generally invalid. Validation is a matter of degree; tests are not valid or invalid, they are highly valid, moderately valid, or generally invalid.

34. The three main forms of validity are content, criterion-related, and construct. They are viewed as interrelated, not independent aspects of validity. The three main forms of validity are content, criterion-related, and construct. They are viewed as interrelated, not independent aspects of validity.

35. Content Validity The degree to which a test measures an intended content area. It requires both item validity and sampling validity. The degree to which a test measures an intended content area. It requires both item validity and sampling validity. Item validity is concerned with whether the test items are relevant to the intended content area. Item validity is concerned with whether the test items are relevant to the intended content area.

36. Content Validity Content validity is concerned with how well the test sample represents the total content area. Content validity is concerned with how well the test sample represents the total content area. Content validity is of prime importance for achievement tests. Content validity is of prime importance for achievement tests. Content validity is determined by expert judgment of item and sample validity, not statistical means. Content validity is determined by expert judgment of item and sample validity, not statistical means.

37. Criterion-Related Validity Concurrent Validity Concurrent Validity –the degree to which the scores on a test are related to scores on another test administered. Predictive Validity Predictive Validity –the degree to which scores can determine how well someone will do in a future situation. –

38. Construct Validity The most important form of validity. The most important form of validity. Seeks to determine whether the construct underlying a variable is actually being measured. Seeks to determine whether the construct underlying a variable is actually being measured.

39. Construct Validity Determined by a series of validation studies that can include content and criterion-related approaches. Determined by a series of validation studies that can include content and criterion-related approaches.

40. Construct Validity The validity of any test can be diminished by factors such as: The validity of any test can be diminished by factors such as: –unclear test directions –inappropriate vocabulary –subjective scoring –failing to follow administration procedures.

41. Reliability of Measuring Instruments The degree to which a test measures whatever it measures. The degree to which a test measures whatever it measures. It is expressed numerically, usually as a coefficient ranging from 0.0 to 1.0; a high coefficient indicates high reliability. It is expressed numerically, usually as a coefficient ranging from 0.0 to 1.0; a high coefficient indicates high reliability.

42. Reliability Scores obtained when administered in November 2005 would be similar to results from the same test in December 2005 Scores obtained when administered in November 2005 would be similar to results from the same test in December 2005 Same test-takers would be used Same test-takers would be used