Learning from errors and Feedback: Statistics Efficacy Dissertation Proposal CUI Department Simin Hall

Organization Background Statement of the problem Statement of purpose Previous Research Significance of this research Research Design Questions of the study Analysis Utility/Results

Background Increase of 62% between 1976 to 2004 in enrollment in graduate programs career development and personal enrichment UNC- Long_range_planning, UNC- Long_range_planning These graduate students are required to take statistics & quantitative research courses, Their background may be far removed from the field of statistics, They experience anxiety, What interventions are needed?

What does this intervention intend to accomplish? To reduce anxiety in learning statistics. Cultivate self-efficacy in graduate students in using statistics and research methods.

Environmental Scanning The population of the study The nature of the problem

Characteristics of adult learners problem-centered results –oriented self-directed seek education that relates or applies directly to their perceived needs accept responsibility for their own learning, if learning is perceived timely and appropriate Time/Distance are issues of concern

Nature of the problem students may experience anxiety due to internal factors such as low efficacy perceptions in statistics or external factors such as poor instruction. Instruction fragments theory, statistical analysis, and empirical reality. Anxiety due to lack of background in quantitative reasoning or problem solving skills.

Statement of Purpose Explore how students’ learning can be enhanced through instruction that builds self- efficacy. Test the social cognitive theoretical prediction that self efficacy is enhanced by feedback that fosters problem solving skills. Examine these predictions using a network- portfolio or a web-based feedback technology such as a wiki.wiki

Previous Research Wang and Lin (2006): Social Cognitive theory Technology Liu et al. (2001): Web-based peer review Seifert and Hutchins (1992): errors as opportunity for instruction & learning Navigation Navigation Mejias (2006):Network Portfolio system & practical research skills Teamwork Teamwork Network-technology Lovette et al. (2000): cognitive theory, statistics instruction, individual practice, real-time feedback Lin et al. (2001):Web-based peer assessment NetPeasNetPeas McKendree (1990): feedback content Mayer, R.E. (2002, 1996) Cognitive theory; Examples & Statistics word Problems, Models for understanding Fredrickson et al. (2005) web- supported vs lecture-based: course in statistics & research methodology for graduate students in education Larreamendy-Joerns et al. (2005)Six on-line Statistics courses Penn State

Features of this Research/Significance of this Study Monitoring errors (Recovery Strategy) Explicit Demonstration of how to provide feedback Pattern recognition:Problems,Mathematical- steps, Soft-wareProblemsMathematical- stepsSoft-ware Decision tree Map Map Use of Published research articles researchresearch Control & treatment groups

Social Cognitive Theory Social learning theory explains human behavior in terms of a continuous reciprocal interaction between cognitive, behavioral and environmental determinants (Bandura, 1986). Students are capable to self-regulate, self- reflect, to plan alternative strategies, to learn by doing, and by modeling.

Social Cognitive Model of learning

Network Technology Allows continuous reciprocal interaction between cognitive, behavioral and environmental. Reduces individual cognitive loads Allows students to monitor and integrate the information with the help of peers. interactive technology accommodates students’ differences in regard to their information processing rate.

Feedback/Efficacy Hypothesis: four sources of feedback impact students’ efficacy positively through their direct link to metacognitive strategies of planning, monitoring and regulating. Planning, monitoring, and regulating are steps to problem solving strategies.

Sources of Efficacy

Theoretical Prediction According to social cognitive model of learning, students monitor their steps and errors, they note their progress toward the solution, and progress indicators convey to students that they are capable of solving statistical problems. This reciprocity enhances self-efficacy for continued learning and impacts students’ emotion positively (Pintrich & Degroot, 1990; Pintrich & Schunk, 2002).

Research Design Quasi-experimental design with two groups, one control and one treatment group. The control group will receive all instructions via Blackboard system Course, and the treatment group will use the Web-Based feedback system (WBF- wiki). Coursewiki

Intervention (a) The teacher posts the homework assignment. (b) Each student prepares the homework and uploads it to the wiki. (c) For each assignment, the teacher assigns randomly three reviewers for each student. (d) Each student views other works and must comment on the steps of how the solution is derived. (e) Each student must revise the original assignment in line with the comments received. The final solution has a brief description from each student to summarize their initial understanding of the problem and their final understanding.

What do students achieve during the feedback process? In completing the assignments, a student may plan the steps to the solution, the procedures to be shown in the solution, and finally execute the plan. In reviewing peer homework, student must read, compare, or question ideas, suggest modification, or even reflect how well student’s work is compared with others. These cognitive processes involve monitoring the adequacy of steps adopted. However, if student receives a message that a step is not adequate, then the student must regulate the cognitive function and employ other alternatives.

Control Group Students receive the same assignment as the treatment group; however, it is in the assignment format of Black Board for which students can not view each other’s works. Moreover, students can not retrieve and correct errors since they only receive feedback from the teacher on their final submissions. Students are encouraged to use the Discussion Board and s to interact among themselves and with the teacher. They are assigned into groups to facilitate communication and learning. However, there is no grade associated with these activities.

Question-1 Is there a significant difference in the means of the two achievement grades across the groups (WFB vs. BB)? Students receive scores on assignments with peer feedback and the final is without the peer feedback. The assignment score will be compared with the final score using a repeated measure analysis. The dependent variables are the assignment and review scores combined and the final score and the independent variable is the method which is the web-based feedback system (treatment) and the Blackboard (control). The covariates are the students’ characteristics. It is hypothesized that students’ scores are higher with peer feedback. H0: μt = μc ; Ha: μt > μc.

Question-2 Is there a significant difference in the means of pre/post of efficacy scores across the groups? The pre and post test scores of students on a questionnaire that measures students’ confidence/competence with statistics and feedback system are compared using a repeated measure analysis. The dependent variables are the pre and post efficacy scores and the independent variable is the method with students’ characteristics as the covariates. H0: μt= μc, the null hypothesis states that the difference in efficacy scores is the same for both groups. Ha: μt> μc

Question-3 Is there a significant difference in the means of pre/post of anxiety scores across the groups? The pre and post test scores on statistics anxiety for both control and treatment groups are analyzed using a repeated measure analysis with the test scores as within subject effect (dependent variable) and the intervention system as the between subject effect (independent variable) with students’ characteristics as covariates. H0: μt= μc, the null hypothesis states that the difference in anxiety scores is the same for both groups. The alternative hypothesis is that the treatment group manage to reduce anxiety more than the control group. Ha: μt < μc

Question-4 How do students cluster based on the variables in the study? In cluster analysis, there is no distinction between dependent variables and independent variables. A cluster analysis is performed to determine the grouping of students that have similar characteristics in terms of the variables measured in this study. By looking at the combination of variables in each cluster, I hope to define each cluster.

Question-5 Is there a significant difference in the means of satisfaction between the two groups? A one-way analysis of variance with satisfaction scores as dependent variable and the method as the independent variable with students’ characteristics as covariates is analyzed. H0: μt= μc, the null hypothesis states that the mean of satisfaction scores is the same for both groups and Ha: μt > μc.

Advantages & Disadvantages: web-based feedback Advantages Continuous interaction Learning from modeling and errors Planning, monitoring, regulating nonintimidating characteristics, such as anonymity in asking questions Disadvantages potential losses due to free riding, social loafing, and diffusion of responsibility to lack focus or willingness to participate or the discipline for the web-based environment some students experienced greater peer pressures in the peer assessment dislike peer assessment because raters were at the same time competitors

What benefits/Who would use these results? The prevalence of quantitative methods in social science literature (Vijverberg, 1997). The decisions for doctoral students to persist may vary widely, however, the peer mentoring in the area of research were identified as a predictor for retention. Mastery of quantitative techniques opens up topics for thesis and dissertations and could potentially impact career choices. Understanding efficacy beliefs should be helpful for teaching courses that traditionally have produced anxiety in students This research may provide empirical evidence in regard to the effect of sources of feedback on efficacy in learning quantitative courses and thus research. Students’ satisfaction with this intervention may lead universities instructors to adopt the more advanced form of this technology, which provides instructors with tools for assessments (Wang & Lin, 2006).

Recommendation for Future Research Researchers and engineers who construct web-based learning systems tend to overlook the reciprocal interactions between three influences in social cognitive theory. Interactive web-based system provide scaffolding to learning. Need more empirical results to improve functions such as assessment tools. Design better Psychometric measures of students’ learning.

Conclusion In this study, it is hypothesized that monitoring and learning from errors on the network technology lead to successful problem solving skills. My main interest is not just limited to determine whether providing a learning assisted system promote active knowledge construction but also the effective learning strategies supported by the system is of high priority. Finding the grouping of students who do not adapt well in the feedback system (are they not effective team players? What cognitive strategies do they lack?) are important findings for further studies to examine the effectiveness of web-based feedback system in different fields.