1. SIG Writing Conference, Liverpool – July 2016

2. The interaction between writer characteristics, writing process and text quality when writing synthesis texts
Nina Vandermeulen & Brenda van den Broek
Sarah Bernolet, Elke Van Steendam, Gert Rijlaarsdam

3. Problem statement Synthesis task
Integrating different source texts into a new meaningful text
Hybrid task with high cognitive load
Reading – understanding – selecting – organising – integrating – writing –revising
Complex writing process
~ text quality
~ writer characteristics

4. Previous studies Relation process – product
More complex writing processes lead to better texts
For synthesis texts: recursive process Mateos & Solé (2009), Martínez, Mateos, Martín & Rijlaarsdam (2015)
Distribution of the different cognitive activities during the process: importance of moment (when do they take place?) and duration (how long do the activities last?) Breetvelt, Van den Bergh & Rijlaarsdam (1994)

5. Previous studies Relation writer characteristics - process
Writing processes differ according to the different types of writers (engineers plan & sculptors revise) Tillema, Van den Bergh, Rijlaarsdam & Sanders (2011) Kieft et al. (2006)
Relation writer characteristics – product
Self-efficacy: Belief in one’s own writing abilities has an influence on the writing performance (remark: current European studies don’t come to a clear-cut conclusion) Villalón, Mateos, Cuevas (2013)

6. Previous studies Synthesis writing
Few studies on process – product relation
Small-scale (not generalisable)
Think aloud protocols (interrupt the normal writing process)

7. Purpose of this study An explorative, empirical study
Larger group of university students
With Inputlog (keystroke logging software)
Multiple texts to measure text quality better generalisability Van Steendam, Tillema, Rijlaarsdam & Van den Bergh (2012)
3-fold aim: relation process – product – writer characteristics

8. Method Participants University of Antwerp (Belgium)
Premaster students Multilingual Professional Communication
Diverse study background
Course on Academic Writing
45 students (75% women, 25% men)
Tasks
Students wrote 3 synthesis texts during the course
3 or more sources
500 to 600 words

9. Method Questionnaires: Valid questionnaires (used in previous studies)
To measure writer characteristics
Writing beliefs
Writing attitude
Writing process style: degree of planning (Engineer) and revising behaviour (Sculptor)
Self-efficacy: measuring belief in one’s own writing abilities
Bruning & White (2005); Rijlaarsdam & Van den Bergh (1987); Kieft et al. (2007); Braaksma (2002)

10. Method Assessing text quality:
Benchmarking (scale of texts in increasing order, average text has a score of 100 )
1 score for overall text quality for each of the 3 tasks (taking into account content, structure, style and language use)
For some analyses: generalised score over the 3 texts

11. Method Inputlog (process analysis) Keystroke logging software
Registers all the key strokes, mouse movements and window changes
Analyses: general, pauses, fluency, source use …
Multiple variables that provide insight in different aspects of the writing process

12. Method Inputlog – Variables Strokes
Number of key strokes during the whole writing process
Number of key strokes per minute
SD (standard deviation): shows variance in the typing speed
Pauses
Number of pauses

13. Results Relation writing process – writing product
Relation writer characteristics – writing process
Relation writer characteristics – writing product
Relation writing process – writing product – writer characteristics
Correlations
Hayes Process Analysis (Hayes, 2013)  moderators

14. Results Relation writing process – writing product
Interaction between variation in writing speed (key strokes) – pause time in first half of process – text quality
Variation in writing speed

15. Results Relation writing process – writing product
Interaction between variation in writing speed (key strokes) and pause time in first half of process – text quality
Variation in writing speed

16. Results Relation writing process – writing product
Interaction between variation in writing speed (number of key strokes) and pause time in first half of process – text quality
Variation in writing speed

17. Results 2. Relations writer characteristics – writing process
Score sculptor – variation key strokes
Score sculptor
Variation number of key strokes
0,335
Variation key strokes per minute
0,351

18. Results
3. Relations writer characteristics – writing product (text quality)
Score sculptor/ engineer – text quality
Text quality
Score Sculptor
0,509
Score Engineer
0,586

19. Results
3. Relation writer characteristics – writing product (text gain)
Effect sculptor and engineer on gain from text 1 to 2

20. Results
3. Relation writer characteristics – writing product
(text gain)
Effect sculptor and engineer on gain from text 1 to 2

21. Results
3. Correlation writer characteristics – writing product
(text gain)
Effect sculptor and engineer on gain from text 1 to 2

22. Results
4. Relations writing process – writing product – writer characteristics
Variation in speed moderates the effect of self-efficacy on text quality

23. Conclusion and discussion
Relation between writing process, writer characteristics and writing product

24. Conclusion and discussion
Relation between writing process, writer characteristics and writing product
Relation process – product
variation key strokes & pause time – text quality

25. Conclusion and discussion
Relation between writing process, writer characteristics and writing product
Relation process – product
variation key strokes & pause time – text quality
Relation writer characteristics – process
score sculptor – variation key strokes

26. Conclusion and discussion
Relation between writing process, writer characteristics and writing product
Relation process – product
variation key strokes & pause time – text quality
Relation writer characteristics – process
score sculptor – variation key strokes
Relation writer characteristics – product
score sculptor/ engineer – text quality
score sculptor and engineer – text gain

27. Conclusion and discussion
Relation between writing process, writer characteristics and writing product
Relation process – product
variation key strokes & pause time – text quality
Relation writer characteristics – process
score sculptor – variation key strokes
Relation writer characteristics – product
score sculptor/ engineer – text quality
score sculptor and engineer – text gain
Relation process – writer characteristics – product
variation key strokes – self-efficacy – text quality

28. Conclusion and discussion
Relation between writing process and product is complicated (moderator variables)
Take into account the individual writer characteristics.
Important to gain insight in interaction process –product – writer characteristics: in order to use insights in education (design instruction and feedback on synthesis writing)

29. Thank you! Questions?

30. Contact brenda.vandenbroek@uantwerpen.be
Want to know more about our project?
Synthesis writing in grade 10 – bachelor 2: national baseline study – intervention studies – instruction and feedback on product and process
Contact

31. References
Breetvelt, I., Van den Bergh, H., & Rijlaarsdam, G. (1994). Relations between writing processes and text quality: When and how? Cognition and Instruction, 12 (2), Hayes, A. (2013). Introduction to Mediation, Moderation, and Conditional Process Analysis. A Regression-Based Approach. New York, NY: Guilford Press. Kieft, M., Rijlaarsdam, G., & Van den Bergh, H. (2006). Writing as a learning tool. Testing the role of students’ writing strategies. European Journal of Psychology of Education, 21(1), 17–34. Raedts, M. (2008). De invloed van zelfeffectiviteitsverwachtingen, taakkennis en observerend leren bij een nieuwe en complexevschrijftaak. Ongepubliceerde doctoraatsthesis, Universiteit Antwerpen, België. Tillema, M., Van den Bergh, H., Rijlaarsdam, G., & Sanders, T. (2011). Relating self reports of writing behaviour and online task execution using a temporal model. Metacognition and Learning, 6(3), Veenman, M. V. (2011). Alternative assessment of strategy use with self-report instruments: a discussion. Metacognition and Learning, 6(2), Van Steendam, E., Tillema, M., Rijlaarsdam, G., & van den Bergh, H. (Eds.).(2012). Measuring Writing: Recent Insights into Theory, Methodology and Practice. Brill. Villalón, Mateos & Cuevas (2013). High school boys ‘ and girls’ writing conceptions and writing self-efficacy beliefs: what is their role in writing performance? Educational Psychology: An International Journal of Experimental Educational Psychology.