Data Collection Population Sampling

Comments on literature summary  Chapters 1, 2, 3,4 are not yours  Strong remark  Use of phrases from another article  Citing secondary sources  Block quotation and Page reference  In text citation  Colloquial – “In my opinion….”  Connecting ideas – on the other hand, moreover, in support of, This is supported by.., in contrast to, in addition, nonetheless….

Where we are now… Observation Background survey Broad area of research Literature review Problem definition Problem statement Research questions Theoretical framework Variables clearly identified Hypothesis generation Research design Methods Data collection Analysis Interpretation Deduction Hypothesis substantiated? RQ answered?

Lesson Objectives  Describe procedures  Obtaining permissions for data collection  Selecting participants for data collection  Identifying data options  Recording and administering data collection

Let’s move on to data collection What is data? Collection of facts from which conclusions may be drawn

Key ideas  Who will you study (unit, sampling, sample size)?  What permissions will you need? (levels,MOE)  What information will you collect? (types of data, links to questions/variables)  What instrument(s) will you use? (selecting an instrument, scales of measurement, validity, reliability)  How will you administer the data collection? (standardization, ethical issues)

Who will you study?  Unit of analysis is the level (e.g. individual, family, school, school district) the data will be gathered.  There may be different units of analysis  one for the dependent variable  one for the independent variable

Procedures for Collecting Quantitative Data (1) Obtain permissions  secure permissions  obtain informed consent from participants

Obtaining Permissions  Institutional or organizational (e.g. school district)  Site-specific (e.g. secondary school)  Individual participants or parents  Campus approval (e.g. university or college)

Obtaining Informed Consent from EPRD,MOE  Obtain Approval via State Education Department  Have participants sign an informed consent form

Procedures for Collecting Quantitative Data (2) Select participants  specify a population and sample  use probability and non- probability sampling  choose a sample size

Procedures for Collecting Quantitative Data (3) Identify data options  specify variables  operationalize variables  select scales of measurement  choose types of data measures

Procedures for Collecting Quantitative Data (4) Record and administer data collection  locate or develop an instrument  obtain reliable and valid data  develop administrative procedures for data collection

Population and Sampling

POPULATION and SAMPLING  A population is a group of individuals that comprise the same characteristics  A sample is a sub-group of the target population that the researcher plans to study

Select Participants: Specify a Population and Sample  Samples  for the purpose about making generalizations about the target population (quantitative research).  samples are only estimates  the difference between the sample estimate and the true population is the “sampling error.”

Populations and Samples Sample Target Population Sample Population All science teachers in secondary schools in Kuantan College students in all community colleges Adult educators in all faculties of education Sample All sec. school biology teachers in Kuantan Students in one community college Adult educators in 2 faculties of education in the East Malaysia

Probability and Non-Probability Sampling  Probability sampling is the selection of individuals from the population so that they are representative of the population  Non-probability sampling is the selection of participants because they are available, convenient, or represent some characteristic the investigator wants to study.

Types of Sampling Sampling Strategies Probability/Random Sampling Non-Probability/Purposeful Sampling Simple StratifiedCluster Random Sampling Sampling Sampling Convenience Snowball Sampling

Differences Between Random and Purposeful Sampling Random “Quantitative” Sampling Select Representative individuals To generalize from sample to population To make claims about the population To build/test “theories” that explain the population Purposeful “Qualitative” Sampling Select people/sites who can best help us understand our phenomenon To develop detailed understanding That might be “useful: information That might help people “learn” about the phenomenon That might give voice to “silenced” people

Types of Probability Sampling  Simple Random: selecting a sample from the population so all in the population have an equal chance of being selected  Systematic: choosing every “nth” individual or site in the population until the desired sample size is achieved

Types of Probability Sampling  Stratified sampling: stratifying the population on a characteristic (e.g. gender), then sampling from each stratum. Boys N=6000 Girls N=3000 Population (N=9000).66 of pop of pop100 Sample = 300

Types of Probability Sampling  Cluster Sampling:  Selects groups, not individuals  All members in the groups have similar characteristics  Useful when the population is large or spread over a wide geographical area

Example: Cluster Sampling  Population : All primary school teachers in Klang valley (5000)  The desired sample : 400  Cluster: School  No. of primary schools in Klang valley: 150  Average number of teachers per school : 40  Number of cluster : 400/ 40  10 out of 150 schools are randomly selected  All teachers in the selected schools make up the sample

Try one  You want to study the resilience of UiTM students.  Population?  Sampling?

Select Participants: Choose a Sample Size  Select a sample size as large as possible from the individuals available  Select a sufficient number of participants for the statistical tests you will use  Calculate the sample size using a sample size formula

Select Participants: Choose a Sample Size  A rough estimate:  15 participants in each grp in an expt  30 participants for a correlational study  350 individuals for a survey study but depend of several factors

Calculating sample size  Krejcie and Morgan (1970) Krejcie and Morgan

EXAMPLE  Gender difference in vocational interest of post matriculation at UiTM  Independent variable:  Dependent variable:  Research Design:  Population:  Sample size:  Instrument:

EXAMPLE Gender difference in vocational interest of post matriculation at UiTM  Independent variable: Gender  Dependent variable: Vocational interest  Research Design: Survey  Population: Post matriculation students  Sample size: 350  Instrument: Self developed

Calculating sample size  Use web calculator  ml ml  calc.htm

Non Probability Sampling

Types of Purposeful Sampling When Does Sampling Occur? After Data Collection has started What is the intent? To develop many perspectives Extreme Case Sampling To describe particularly troublesome or enlightening cases Typical Sampling To describe what is “typical” to those unfamiliar with the case What is the intent? To take advantage of whatever case unfolds Opportunistic Sampling To locate people or sites to study Snowball Sampling To explore confirming or disconfirming cases Confirming/ Disconfirming Sampling Maximal Variation Sampling To generate a theory or concept Theory or Concept Sampling To describe some sub-group in depth Homogenous Sampling Before Data Collection

One may sample..  Maximal variation  Most hardworking/ Highest achiever  Lowest achiever  Extreme case  ??

Examples of Non-Probability Samples  Convenience Sampling: participants are selected because they are willing and available to be studied  Snowball Sampling: the researcher asks participants to identify other participants to become members of the sample.

Non Probability sampling example  Study delinquent behaviour during recess.  I selected 1 school out of 4 - Interviewed all 4 principals and toured all 4 schools. I chose school to which I was given most access with fewest restrictions. Also school that reported the widest variations in delinquent behaviour during recess (very high to virtually no display).  Then sampled different locations with camera to find most varied activity and least self- conscious/guarded behavior. Where?  Turned out to be behind the surau.  Later used snowball approach in choosing children to interview.

Your sample?

Proceed to collecting quantitative data

Collecting quantitative data

Flow of Activities in Collecting Data 1.Identify the variable 2.Operationally define the variable 3.Locate data (measures, observations, documents with questions and scales) 4.Collect data on instruments yielding numeric scores

Identify Data Options: Specify Variables  Independent Variables  Dependent Variables  Intervening  Moderating

Identify Data Options: Operationalize Variables  Operational Definition: The specification of how the variable will be defined and measured  typically based on the literature  often found in reports under “definition of terms”  Sometimes the researcher must construct it

Flow of Activities in Collecting Data Identify the variable Operationally define the variable Locate data (measures, observations, documents with questions and scales) Collect data on instruments yielding numeric scores Self-efficacy for learning from others Level of confidence that an individual can learn something by being taught by others 13 items on a self-efficacy attitudinal scale from Bergin (1989) Scores of each item ranged from with 10 being “completely confident.” Flow of ActivitiesExample

Flow of Activities in Collecting Data Identify the variable Operationally define the variable Locate data (measures, observations, documents with questions and scales) Collect data on instruments yielding numeric scores Learning motivation of adult learners Interest and level of engagement of an individual items on a questionnaire developed Scores of each item ranged from 1-5 with 5 being “most interested” Flow of ActivitiesExample

Scales of measurement

Identify Data Options: Select Scales of Measurement  Nominal (Categorical): categories that describe traits or characteristics  participants can check  Female  Male  Ordinal: participants rank the order of a characteristic, trait or attribute

Identify Data Options: Select Scales of Measurement  Interval: provides “continuous” response possibilities to questions with assumed equal distance ; scale with no true zero  Discrete (SD SA)  Metric ( o C)  Ratio: a scale with a true zero and equal distances among units

Practice Identify the level of measurement

Measurement level?  Age  Religion  Gender  Income bracket  Test scores  CGPA  Frequency of asking questions  Time spent on task  Level of acceptance  (0 – never, 5 – all the time)  Activity  ( 0 –not active, 5 – very active) Ratio ? Nominal? Ordinal? Interval?

Measurement level?  Age  Religion  Gender  Income bracket  Test scores  CGPA  Frequency of asking questions  Time spent on task  Level of acceptance  (0 – never, 5 – all the time)  Activity  ( 0 –not active, 5 – very active)  Ratio  Nominal  Ordinal  Ratio  Ordinal  Ratio  Interval/ Ordinal

Identify Data Options: Choose Types of Data Measures  An instrument is a tool for measuring, observing, or documenting quantitative data  Types of Instruments  Performance Measures (e.g. test performance)  Attitudinal Measures (measures feelings toward educational topics)  Behavioral Measures (observations of behavior)  Factual Measures (documents, records)

Record and Administer Data Collection: Locate or Develop an Instrument  Develop your own instrument  Locate an existing instrument  Modify an existing instrument

Record and Administer Data Collection: Locate or Develop an Instrument  Strategies to use  Look in published journal articles  Run an ERIC search and use the term “instruments” and the topic of the study  Go to ERIC web site for Evaluation and Assessment  Examine guides to commercially available tests

Developing a questionnaire

Obtain Reliable and Valid Data  Reliability: individual scores from an instrument should be nearly the same or stable on repeated administrations of the instrument  Bathroom scale

Reliability  Types of reliability  Test-retest (scores are stable over time)  Internal consistency (consistent scores across the instrument) Cronbach coefficient alpha if items are scored as continuous variables (SA—SD)  Inter-rater reliability (similarity in observation of a behavior by two or more individuals)

Validity  Validity: the ability to draw meaningful and justifiable inferences from the scores about a sample or a population

Types of validity  Content (representative of all possible questions that could be asked)  Criterion-referenced (scores are a predictor of an outcome or criterion they are expected to predict  Construct (determination of the significance, meaning, purpose and use of the scores)

Let’ s look at a School Based Assessment ResearchSchool Based Assessment Research The instrument

Collecting Qualitative Data

Key Ideas  Gaining site permission  Purposive sampling  Types of qualitative data  Protocols and Issues regarding administering and recording qualitative

Gaining Permission  Gain permission from Institutional Review Board  Gain permission from “gatekeepers” at the research site  Gatekeepers: individuals at the site who provide site access, helps researcher locate people and identifies places to study  The gatekeeper may require written permission about the project

Information for the gatekeeper  Why their site was chosen  What time and resources are required  What will be accomplished at the site  What potential there is for your presence to be disruptive  What individuals at the site will gain from the study  How will you use and report the results

Types of data to Collect  Observations  Interviews  Documents  Audio-Visual Materials

Sources of Qualitative Data  From People:  Interviews  Surveys  Focus Groups  Participant Observation (field notes)  From Things:  Agency case records  Miscellaneous documents  Historical Artifacts  Media  Published materials

Observation Interviews Documents Audio Visuals

Types of Data to Collect: Observations  An observation is the process if gathering first-hand information by observing people and places at a research site.  Observational roles  Participant observer  Non-Participant observer  Observational roles can be changed

Types of Data to Collect: Observations  Conduct multiple observations  Record both descriptive and reflective field notes during the observation  Descriptive field notes describe the events, activities and people  Reflective field notes record personal reflections that relate to their insights, hunches or broad themes that emerge

Administering and Recording Data: Observational Protocols  The header: essential information about the observation  Left column to record descriptive notes  Right column to record reflective notes  A picture of the site may be sketched

Types of Data to Collect: Interviews  Types of Interviews  Individual  Focus group  Telephone 

Structured, Unstructured, and Semi-Structured Interviews Approach to Data Collection Type of Response Options to Questions Types of Interviews Leading to Data Quantitative Close- Ended Structured/ semi-structured Interviews Scores to answers Qualitative Open- Ended Unstructured Interviews Transcription of words

Types of Data to Collect: Interviews  General open-ended questions are asked  allows the participant to create options for responding  participants can voice their experiences and perspectives  Information is recorded then transcribed for analysis

Administering and Recording Data: Interview Protocols  The header: essential information about the interview  Open-ended questions include  “ice-breaker”  ones that address major research questions  probes that clarify and elaborate  Closing comments thanking the participant

EXAMPLE: Semi structured Interview protocols  Name:  Date:  Time:  Venue:  Experience – please describe  Questions 1.Is the method a valid approach to addressing the learning needs of adult learners? Please describe your thoughts of the method 2.What do you think are the barriers to the utilization of the method in UiTM? 3.How would you rate the quality of learning using the method? 4.What overall suggestions do you have for improving the method?

Types of Data to Collect: Documents  Public and private records  Good source for text data  You must obtain permission before using documents  Scan documents when possible

Types of Data to Collect: Audio-Visual Materials  Determine the material that can provide evidence to address your research questions  Determine if the material is available and obtain permission to use it  Check the accuracy and authenticity of the material if you do not record it yourself  Collect the data and organize it

Administering and Recording Data: Field Issues  Time needed for data collection  Limit initial collection or one or two observations or interviews  Time is needed to establish a substantial data base

Administering and Recording Data: Field Issues  Obtaining permission to use materials  Ethical issues  Anonymity of participants  Convey true purpose of study without deception

Let’s gather data  Fill in the Qnaire  Tabulate results  Answer research Qs  Qualitative data  What do you do to overcome distress?

NEXT Quantitative Data Analysis