SIG Writing Conference, Liverpool – July 2016

The interaction between writer characteristics, writing process and text quality when writing synthesis texts Nina Vandermeulen & Brenda van den Broek Sarah Bernolet, Elke Van Steendam, Gert Rijlaarsdam

Problem statement Synthesis task Integrating different source texts into a new meaningful text Hybrid task with high cognitive load Reading – understanding – selecting – organising – integrating – writing –revising Complex writing process ~ text quality ~ writer characteristics

Previous studies Relation process – product More complex writing processes lead to better texts For synthesis texts: recursive process Mateos & Solé (2009), Martínez, Mateos, Martín & Rijlaarsdam (2015) Distribution of the different cognitive activities during the process: importance of moment (when do they take place?) and duration (how long do the activities last?) Breetvelt, Van den Bergh & Rijlaarsdam (1994)

Previous studies Relation writer characteristics - process Writing processes differ according to the different types of writers (engineers plan & sculptors revise) Tillema, Van den Bergh, Rijlaarsdam & Sanders (2011) Kieft et al. (2006) Relation writer characteristics – product Self-efficacy: Belief in one’s own writing abilities has an influence on the writing performance (remark: current European studies don’t come to a clear-cut conclusion) Villalón, Mateos, Cuevas (2013)

Previous studies Synthesis writing Few studies on process – product relation Small-scale (not generalisable) Think aloud protocols (interrupt the normal writing process)

Purpose of this study An explorative, empirical study Larger group of university students With Inputlog (keystroke logging software) Multiple texts to measure text quality better generalisability Van Steendam, Tillema, Rijlaarsdam & Van den Bergh (2012) 3-fold aim: relation process – product – writer characteristics

Method Participants University of Antwerp (Belgium) Premaster students Multilingual Professional Communication Diverse study background Course on Academic Writing 45 students (75% women, 25% men) Tasks Students wrote 3 synthesis texts during the course 3 or more sources 500 to 600 words

Method Questionnaires: Valid questionnaires (used in previous studies) To measure writer characteristics Writing beliefs Writing attitude Writing process style: degree of planning (Engineer) and revising behaviour (Sculptor) Self-efficacy: measuring belief in one’s own writing abilities Bruning & White (2005); Rijlaarsdam & Van den Bergh (1987); Kieft et al. (2007); Braaksma (2002)

Method Assessing text quality: Benchmarking (scale of texts in increasing order, average text has a score of 100 ) 1 score for overall text quality for each of the 3 tasks (taking into account content, structure, style and language use) For some analyses: generalised score over the 3 texts

Method Inputlog (process analysis) Keystroke logging software Registers all the key strokes, mouse movements and window changes Analyses: general, pauses, fluency, source use … Multiple variables that provide insight in different aspects of the writing process

Method Inputlog – Variables Strokes Number of key strokes during the whole writing process Number of key strokes per minute SD (standard deviation): shows variance in the typing speed Pauses Number of pauses

Results Relation writing process – writing product Relation writer characteristics – writing process Relation writer characteristics – writing product Relation writing process – writing product – writer characteristics Correlations Hayes Process Analysis (Hayes, 2013)  moderators

Results Relation writing process – writing product Interaction between variation in writing speed (key strokes) – pause time in first half of process – text quality Variation in writing speed

Results Relation writing process – writing product Interaction between variation in writing speed (key strokes) and pause time in first half of process – text quality Variation in writing speed

Results Relation writing process – writing product Interaction between variation in writing speed (number of key strokes) and pause time in first half of process – text quality Variation in writing speed

Results 2. Relations writer characteristics – writing process Score sculptor – variation key strokes Score sculptor Variation number of key strokes 0,335 Variation key strokes per minute 0,351

Results 3. Relations writer characteristics – writing product (text quality) Score sculptor/ engineer – text quality Text quality Score Sculptor 0,509 Score Engineer 0,586

Results 3. Relation writer characteristics – writing product (text gain) Effect sculptor and engineer on gain from text 1 to 2

Results 3. Relation writer characteristics – writing product (text gain) Effect sculptor and engineer on gain from text 1 to 2

Results 3. Correlation writer characteristics – writing product (text gain) Effect sculptor and engineer on gain from text 1 to 2

Results 4. Relations writing process – writing product – writer characteristics Variation in speed moderates the effect of self-efficacy on text quality

Conclusion and discussion Relation between writing process, writer characteristics and writing product

Conclusion and discussion Relation between writing process, writer characteristics and writing product Relation process – product variation key strokes & pause time – text quality

Conclusion and discussion Relation between writing process, writer characteristics and writing product Relation process – product variation key strokes & pause time – text quality Relation writer characteristics – process score sculptor – variation key strokes

Conclusion and discussion Relation between writing process, writer characteristics and writing product Relation process – product variation key strokes & pause time – text quality Relation writer characteristics – process score sculptor – variation key strokes Relation writer characteristics – product score sculptor/ engineer – text quality score sculptor and engineer – text gain

Conclusion and discussion Relation between writing process, writer characteristics and writing product Relation process – product variation key strokes & pause time – text quality Relation writer characteristics – process score sculptor – variation key strokes Relation writer characteristics – product score sculptor/ engineer – text quality score sculptor and engineer – text gain Relation process – writer characteristics – product variation key strokes – self-efficacy – text quality

Conclusion and discussion Relation between writing process and product is complicated (moderator variables) Take into account the individual writer characteristics. Important to gain insight in interaction process –product – writer characteristics: in order to use insights in education (design instruction and feedback on synthesis writing)

Thank you! Questions?

Contact brenda.vandenbroek@uantwerpen.be nina.vandermeulen@uantwerpen.be www.inputlog.net Want to know more about our project? www.liftwritingresearch.com Synthesis writing in grade 10 – bachelor 2: national baseline study – intervention studies – instruction and feedback on product and process Contact

References Breetvelt, I., Van den Bergh, H., & Rijlaarsdam, G. (1994). Relations between writing processes and text quality: When and how? Cognition and Instruction, 12 (2), 103-123. Hayes, A. (2013). Introduction to Mediation, Moderation, and Conditional Process Analysis. A Regression-Based Approach. New York, NY: Guilford Press. Kieft, M., Rijlaarsdam, G., & Van den Bergh, H. (2006). Writing as a learning tool. Testing the role of students’ writing strategies. European Journal of Psychology of Education, 21(1), 17–34. Raedts, M. (2008). De invloed van zelfeffectiviteitsverwachtingen, taakkennis en observerend leren bij een nieuwe en complexevschrijftaak. Ongepubliceerde doctoraatsthesis, Universiteit Antwerpen, België. Tillema, M., Van den Bergh, H., Rijlaarsdam, G., & Sanders, T. (2011). Relating self reports of writing behaviour and online task execution using a temporal model. Metacognition and Learning, 6(3), 229-253 Veenman, M. V. (2011). Alternative assessment of strategy use with self-report instruments: a discussion. Metacognition and Learning, 6(2), 205-211. Van Steendam, E., Tillema, M., Rijlaarsdam, G., & van den Bergh, H. (Eds.).(2012). Measuring Writing: Recent Insights into Theory, Methodology and Practice. Brill. Villalón, Mateos & Cuevas (2013). High school boys ‘ and girls’ writing conceptions and writing self-efficacy beliefs: what is their role in writing performance? Educational Psychology: An International Journal of Experimental Educational Psychology.