1. Brain signatures of game play and language processing
Robert V. Reichle
Department of Foreign Languages
Center for the Interdisciplinary Study of Language and Literacy
Northern Illinois University

2. L2 PROCESSING For late L2 learners, L2 processing is difficult
Corresponding features or constructions in the native language do not ensure success in the L2
Variability is typical
Native-like attainment is not typical
Previously thought to relate to putative critical period for L2 acquisition

3. L2 PROCESSING
Among other explanations for divergence in L1/L2 processing, Ullman’s (2001, 2004) Declarative/Procedural model
Two domain-general memory systems
Declarative: facts, associations between form and meaning, non-compositional words
Procedural: rule-governed processes, morphosyntactic composition, cognitive-motor skills
Some L2 learners become more nativelike as they come to rely more on procedural memory in the L2

4. EVENT-RELATED POTENTIALS (ERPs)
Lexical/semantic processing: N400
Morphosyntactic/grammatical processing:
LAN
Differential ERPs for L1 syntax/semantics interpreted as evidence for DP model (Ullman, 2004)
(e.g., graphs from Osterhout et al., 2006, p. 204 and Neville et al., 1991; Friederici, Steinhauer, & Frisch, 1999; Friederici, 2002; Hahne & Friederici, 1999, 2001; Hagoort & Brown, 1999; Kaan et al., 2000; Kutas & Hillyard, 1980; Osterhout, Bersick, & McLaughlin, 1997; Osterhout & Holcomb, 1992; Osterhout & Mobley, 1995; Steinhauer, Halter, & Friederici, 1999; Friederici, Pfeifer, & Hahne, 1993; Kluender & Kutas, 1993; Münte, Heinze, Matzke, Wieringa, & Johannes, 1998)

5. EVENT-RELATED POTENTIALS (ERPs)
Proficiency effects on sensitivity to inflectional morphology and syntax
Artificial language learning: with language training, N400 LAN/P600
(e.g., Friederici, Steinhauer, & Pfeifer, 2002; Morgan-Short, Sanz, Steinhauer, & Ullman, 2010; Morgan- Short, Steinhauer, Sanz, & Ullman, 2012; Morgan-Short, Finger, Grey, & Ullman, 2012)
Real language learning: with increasing proficiency, N400 LAN/P600
(e.g., Bowden et al., 2007; Hahne et al., 2006; Osterhout et al., 2006; Rossi et al., 2006; Steinhauer et al., 2006; Gillon Dowens, Vergara, Barber, & Carreiras, 2010; McLaughlin, Osterhout, & Kim, 2004; McLaughlin, Tanner, Pitkänen, Frenck-Mestre, Inoue, Valentine, & Osterhout, 2010; Tanner, Osterhout, & Herschensohn, 2009; Ojima, Nakata, & Kakigi, 2005)

6. EVENT-RELATED POTENTIALS (ERPs)
Proficiency effects on sensitivity to inflectional morphology and syntax
Artificial language learning: with language training, N400 LAN/P600
(e.g., Friederici, Steinhauer, & Pfeifer, 2002; Morgan-Short, Sanz, Steinhauer, & Ullman, 2010; Morgan- Short, Steinhauer, Sanz, & Ullman, 2012; Morgan-Short, Finger, Grey, & Ullman, 2012)
Real language learning: with increasing proficiency, N400 LAN/P600
(e.g., Bowden et al., 2007; Hahne et al., 2006; Osterhout et al., 2006; Rossi et al., 2006; Steinhauer et al., 2006; Gillon Dowens, Vergara, Barber, & Carreiras, 2010; McLaughlin, Osterhout, & Kim, 2004; McLaughlin, Tanner, Pitkänen, Frenck-Mestre, Inoue, Valentine, & Osterhout, 2010; Tanner, Osterhout, & Herschensohn, 2009; Ojima, Nakata, & Kakigi, 2005)

7. brocanto training game
Artificial language (BROCANTO)
Fulfills syntactic requirements of natural language
Contains morphosyntactic features that differ from L1 of participants
Participants trained in lexicon of artificial language
One group trained to proficiency in BROCANTO via a two-player computer board game
Played against each other in pairs
Orally expressed all game moves in BROCANTO
Received feedback on valid game moves and grammatical BROCANTO utterances
The structure of the language not dependent on the structure of the game
(Friederici, Steinhauer, & Pfeifer, 2002; for similar BROCANTO2 design, see also Morgan-Short, Sanz, Steinhauer, & Ullman, 2010; Morgan-Short, Steinhauer, Sanz, & Ullman, 2012; Morgan-Short, Finger, Grey, & Ullman, 2012)

8. brocanto training game
aaf trul prez nöri aak füne plox
the trul-piece captures horizontally the round plox-piece
aaf trul prez nöri *rix füne ploxthe trul-piece captures horizontally *buy round plox-piece
ERPs
ERP data collection using syntactically felicitous and anomalous items took place after participants in the training group reached a 95% accuracy threshold.
Training group had biphasic response to violations that resembled LAN/P600 of L1 processing

9. brocanto training game
Why was this a successful training paradigm?

10. brocanto training game
Why was this a successful training paradigm?
Gee (2003): active, critical learning
cooperation
competition
a requirement of successful learning in order to progress
feedback
meta-awareness of gameplay mechanics

11. brocanto training game
Why was this a successful training paradigm?
Declarative/Procedural model
nativelike syntactic processing should be handled by procedural memory
learning a cognitive-motor skill (like the rules of the training game) should also be handled by this system
Did these two types of learning engage similar cognitive processes? Does “words and rules” have an equivalent “sprites and rules” (Reichle, 2012)?
P600 not necessarily fully language-specific — also observed for structural revision in music (e.g. Patel et al., 1998)

12. Research questions
Does the execution of gameplay skills engage the same processes as syntactic processing?
Specifically, examining violations of syntax and violations of expected gameplay behavior.
If the same processes, then could aid learning, but could also present confound to interpretation of results.
Does processing linguistic stimuli in the context of a game lead to a task effect compared to linguistic stimuli in isolation?

13. Hypotheses
Violations of expected gameplay behavior will elicit LAN and P600 effects comparable to those seen for violations of morphosyntax.
Processing violations of morphosyntax in the context of a game causes a task effect compared to violations of morphosyntax in isolation.

14. ERP Experiment 1 23 participants (5 rejected due to artifacts)
native speakers of English
pre-screened for knowledge of rules of movement in chess
EEG data were recorded from nine sites (F3, Fz, F4, C3, Cz, C4, P3, Pz, P4)Four conditions
A: Control. Legal moves followed by grammatical sentences
B: Morphosyntax violation. Legal moves followed by sentences with a morphosyntax violation
C: Gameplay mismatch. A chess move that is legal for a piece other than the one shown in the image, followed by a grammatical sentence
D: Gameplay violation. A chess move that is never legal for any piece, followed by a grammatical sentence

16. A1. Control: The queen takes the pawn.

17. A: The white queen takes the black pawn.
A1. Control: The queen takes the pawn.
A: The white queen takes the black pawn.

18. B: The white queen *taking the black pawn.
A1. Control: The queen takes the pawn.
B: The white queen *taking the black pawn.

20. F1. Game Rule Violation: The queen takes the pawn.

21. C: The white queen takes the black pawn.
F1. Game Rule Violation: The queen takes the pawn.
C: The white queen takes the black pawn.

23. F1. Game Rule Violation: The queen takes the pawn.

24. D: The white queen takes the black pawn.
F1. Game Rule Violation: The queen takes the pawn.
D: The white queen takes the black pawn.

25. ERP Experiment 1 Stimulus presentation each image onscreen for 1 sec
600ms pause
word-by-word presentation: 290 ms plus 30 ms per letter, up to a maximum of 590 ms, with a 150 ms interval between words
Analysis
At fourth word (takes/*taking)
Mean amplitude data in ms and ms windows
Repeated-measures ANOVAs: 4 (Condition) x 3 (Anteriority) x 3 (Laterality)

26. F3 Fz F4 C3 Cz C4 P3 Pz P4

28. F3 Fz F4
Omnibus
ms: Cond * Lat, p < .05 C3 Cz C4 P3 Pz P4

29. F3 Fz F4 C3 Cz C4 P3 Pz P4 Morph. vs Control
ms: Cond * Lat, p < .05 C3 Cz C4 P3 Pz P4

30. F3 Fz F4 C3 Cz C4 P3 Pz P4 Game rule mismatch vs Control 200-500ms: NS

31. F3 Fz F4 C3 Cz C4 P3 Pz P4 Game rule violation vs Control
ms: Cond * Ant * Hem, p = .022 C3 Cz C4 P3 Pz P4

32. Results P600 effect for morphosyntax violations
No effect for game rule mismatch
Game rule violation
Negativity effect, strongest in left and mid central and frontal sites
Interpreted as possible variation of LAN
Extended into later time window than typical biphasic LAN
But, not biphasic — no P600 effect followed

33. Discussion
Hypothesis: Violations of expected gameplay behavior will elicit LAN and P600 effects comparable to those seen for violations of morphosyntax.
Not fully supported — no P600. Consistent with generally language-specific nature of P600.
Partially supported by presence of LAN. But work still remains on reproducibility and functional significance of LANs (e.g. Steinhauer, Drury).

34. Discussion Implications
Possibility remains that processing gameplay rules engages similar cognitive processes as processing morphosyntax (LAN)
No evidence that gameplay elicits P600
Doubtful that P600s seen in BROCANTO studies were indices of gameplay-related processing

35. Hypotheses
Violations of expected gameplay behavior will elicit LAN and P600 effects comparable to those seen for violations of morphosyntax.
Processing violations of morphosyntax in the context of a game causes a task effect compared to violations of morphosyntax in isolation.

36. ERP Experiment 2
Show morphosyntactic anomalies within and without game context
2 groups
Image group: Saw chess move after reading sentence
NoImage group: Did not see chess moves
12 participants (6 per group)
native speakers of English
pre-screened for knowledge of rules of movement in chess
EEG data were recorded from nine sites (F3, Fz, F4, C3, Cz, C4, P3, Pz, P4)

37. ERP Experiment 2 2 Conditions
Control: The white queen takes the black pawn.
Morphosyntax violation: The white queen taking the black pawn.
2 fillers
Progressive filler: The queen is taking the black pawn.
Game rule violation: The black queen takes the white pawn.

38. ERP Experiment 2 Stimulus presentation
word-by-word presentation: 290 ms plus 30 ms per letter, up to a maximum of 590 ms, with a 150 ms interval between words
for Image group, presentation of game images 1 sec after final word
Analysis
At fourth word (takes/taking)
Mean amplitude data in ms and ms windows
Repeated-measures ANOVAs: 2 (Condition) x 3 (Anteriority) x 3 (Laterality)

39. NoImage group
Control Morphosyntax violation

40. Image group
Control Morphosyntax violation

41. Between-group omnibus: 400-900ms: Cond * Group, p = .26
Image group
Control Morphosyntax violation

42. Image group: 400-900ms: Cond, p = .25 400-900ms: Cond * Lat, p = .19
ms: Cond * Lat * SRTT, p = .2
Image group
Control Morphosyntax violation

43. Results Trend toward between-group difference (task effect)
Trend toward P600 for Image group
Data collection still in progress — more participants needed

44. Discussion
Hypothesis: Processing violations of morphosyntax in the context of a game causes a task effect compared to violations of morphosyntax in isolation.
Preliminary results trend toward support of hypothesis.
Attributable to “motivation” in the broadest sense. Additional studies needed to separate out Gee’s factors (competition, meta-awareness, etc.) from simpler explanations (i.e. more attentive due to presence of images)

45. CONCLUSIONS
Evidence of an LAN for the processing of gameplay violations suggests gameplay may engage cognitive processes also used in language processing.
Lack of P600 for gameplay processing indicates that L2 studies using game training paradigms are not likely to confound indices of gameplay with indices of morphosyntactic processing.
Preliminary data suggest trend toward game-related task effect, suggesting increased motivation or attention within game context.
Additional studies needed to further tease apart variables.

46. Acknowledgments
This research has been supported by the NIU Division of Research and Graduate Studies, the NIU Center for the Interdisciplinary Study of Language and Literacy, and the NIU Center for the Study of Family Violence and Sexual Assault.
The author thanks Cody Happ and Regina Hiraoka for their assistance with materials creation and data collection, and all participants for their cooperation.

47. Thank you