1. Reconstructive processes in working memory
Kim Uittenhove & Pierre Barrouillet
University of Geneva

2. Why do we need reconstructive processes ?
Maintaining information in a state appropriate for processing = key to cognition
We need reconstructive processes to keep information in a state appropriate for processing. Indeed, information in working memory is lost in a very fast way, and without processes designed to maintain this information we would never be able to achieve the kind of processing of information over time that forms the cornerstone of human cognition

3. Forgetting in working memory
Limited capacity and duration
Four chunks (Cowan, 2001; 2015)
Fast forgetting (timecourse of seconds)
Nature of forgetting:
Decay, when attention is no longer on the information
Interference
as we all know, the nature of forgetting has been a matter of huge debate in the literature. Two main views oppose as to why information is lost over time.
One view is that loss occurs through decay of memory traces when attention is no longer on these traces.
An opposing view is that loss does not occur with the mere passage of time, but instead occurs through interfering information, which may superpose itself on the target information.
The term "decay theory" was first coined by Edward Thorndike in his book "The Psychology of Learning" in 1914.[3] This simply states that if a person does not access and use the memory representation they have formed the memory trace will fade or decay over time. This theory was based on the early memory work by Hermann Ebbinghaus in the late 19th century.[4] The decay theory proposed by Thorndike was heavily criticized by McGeoch and his interference theory.[5] This led to the abandoning of the decay theory, until the late 1950s when studies by John Brown and the Petersons showed evidence of time based decay by filling the retention period by counting backwards in threes from a given number. This led to what is known as the Brown-Peterson Paradigm.[6][7] The theory was again challenged, this time a paper by Keppel and Underwood who attributed the findings to proactive interference.[8] Studies in the 1970s by Reitman[9][10] tried reviving the decay theory by accounting for certain confounds criticized by Keppel and Underwood. Roediger quickly found problems with these studies and their methods.[11] Harris made an attempt to make a case for decay theory by using tones instead of word lists and his results are congruent making a case for decay theory.[12] In addition, McKone used implicit memory tasks as opposed to explicit tasks to address the confound problems. They provided evidence for decay theory, however, the results also interacted with interference effects.[13][14] One of the biggest criticisms of decay theory is that it cannot be explained as a mechanism and that is the direction that the research is headed.

4. How do we keep information alive ?
Protection from interference
Distractor removal (Lewandowsky, Oberauer, & Brown, 2009)
Protection from decay
Attentional refreshing (Barrouillet, Bernardin, Portrat, Vergauwe, & Camos, 2007)
Domain-specific rehearsal(i.e., phonological loop, Baddeley & Hitch, 1974: Baddeley, 1986)
In both scenarios, the key is to keep the target information alive long enough to be able to efficiently process it.
In the case of interfering information, one may suppose that interference could be removed to protect the important information.
In the case of protection from decay, those memory traces at risk of time-related decay may be strengthened.
One could refresh these traces by focusing attention on them.
Many models also postulate the existence of domain-specific rehearsal mechanisms, independent from attention, that could help protect certain types of memory traces.
Alternatively, selective attentional refreshing or reconstructive processes could also serve to bias our system towards the relevant information instead of the interfering information.

5. Loss and Maintenance Loss Time-based decay Interference Maintenance
Attentional refreshing
Rehearsal
Reconstruction
Distractor removal
So to summarize, working memory functioning is defined by two opposing forces, loss of information on the one hand, and the continuous maintenance of target information on the other hand,
So today we are going to focus on evidence for the role of these several possible maintenance mechanisms

6. Time-based Resource Sharing
Processing
Maintenance
Cognitive load =
Duration of attentional capture by processing / Total time
Rapid switching
Many researchers have attempted to model this interaction between loss and maintenance. In our lab in geneva, we proposed the time-based resource sharing model,
which assumes that loss and maintenance of information is a simple function of the time limited attentional resources are occupied by distractors and the time they are used to refresh target information,
the model defines this interplay by the cognitive load, which is the duration of attentional capture by concurrent processing divided by the total time attention is available
Alternatives have been proposed that model the interplay between loss and maintenance as interference building and distractor removal
Barrouillet, Bernardin, Portrat, Vergauwe, & Camos, 2007

7. The interplay between distractors and free time: Complex span
Memoranda
Free time ➔ Reconstruction / Refreshing / Rehearsal
➔ Distractor removal
One way the interplay between loss and maintenance can and has studied is by the complex span paradigm. The complex span paradigm alternates target information, distractors, and free time.
We can manipulate the duration of distractors, which would increase decay, and also interference when distractors involve more interferering representations
We can manipulate the duration of free time, which increases opportunities for maintenance activities
And finally, we can manipulate the number of processing episodes L K S
Concurrent processing
Decay
Interference
Memoranda recall ?

8. Observing loss 4 3 6 2 Duration of distractors (processing episodes)
- Increased time-related decay
- Increased interfering representations ?
quatre trois
six deux
Constant free time
600 ms
In the complex span paradigm, loss can be observed by manipulating the duration of the distractors. According to the TBRS, the longer the distractor captures the attention, the more the relevant information will have undergone time-related decay, since it does not benefit from the focus of attention at this time.
Alternatively, the longer a distractor last, the more interference it may generate, and thus also lead to more loss.
One way to try and disentangle both possibilities is to manipulate the duration of the distractor without augmenting the number of interfering representations. We tried to accomplish this in our lab by xxxx
Ofcourse, it remains hard to be absolutely certain that two distractors that are different like this would generate exactly the same amount of interference. However, when looking at the data we collected over the last years, an overall picture does emerge. We will start by looking at some results with this first manipulation
Barrouillet, Uittenhove, Langerock & Lucidi

9. Expt. 1: Increasing distractor duration increases forgetting
Very systematic relation between time and recall
➔ Time-based Resource Sharing
Numbers
Words
Numbers
These are the results we obtained with 47 participants in two different experiments. In one experiment the task was to add the red numbers and compare the sum to the sum of the other numbers, in the second expt the task was much simpler and consisted of saying whether one number was smaller or not than the others. What we see on the x-axis is the average processing times of these distractors in the two experiments. Now when we plot the recall of letters on this, we see a relationship between processing time and recall that is very much linear, as if the passing of time were directly related to recall. The same pattern of results was observed when looking at the recall of spatial locations.
Although these data do not dispel a role for interference, the simplest model in this case is to see loss of information as a function of the passing of time, and as such is very reminiscent of the functioning proposed by TBRS.
Words
Addition
Comparison

10. Number of distractors
TBRS: No increase in time-related decay under constant CL
Interference-only (SOB-CS) : Accumulation of interference under high CL (> .60)
Interaction with nature of the memoranda ?
Manipulating the number of processing episodes, or distractors, is a very interesting manipulation, because it leads to different predictions in different models
according to the tbrs, the number of pe should not have an effect, since it is the ratio between free time and concurrent processing time which will define the amount of material that can be maintained, which will then remain constant over successive PE.
Alternatively, every novel processing episode could add interference, and if the cognitive load is high enough, distractor removal processes won’t be able to remove all of it and interference will accumulate, leading to the degradation of target information over successive PE.
However, both models do not currently consider how efficiency of maintenance processes may depend on the type of information to be maintained, and how the nature of this information may potentially alter the interplay between loss and maintenance.
For example, previous research has hinted at different maintenance efficiency for verbal and visuospatial information. . Lilienthal et al found a degradation of visual but not verbal info over unfilled retention intervals, (Ricker and Cowan, 2010): unconventional visual characters degrade over unfilled delays.
Suggesting that maintenance processes act differently on verbal and visuospatial information and may influence the balance between loss and maintenance
within the TBRS, where refreshing or reconstruction may depend on the refreshability or underlying representation of memoranda. In the case of distractor removal, it is less clear how the type of memoranda should play a role, since distractor removal will especially depend on the nature of the distraction, and it’s quantity.
note that all distractors in the trials were new items, and their succession was fast enough to prevent complete removal

11. The paradigm 4 3 6 2 1200 ms 600 ms CL = 1200 / 1200 + 600 = .66
Constant free time
Vary number
Consonants
Spatial locations

12. Expt. 1: Number of processing episodes
Verbal material
No influence of number of PE
CL remained constant (= balance of loss and maintenance)
Visuospatial material
- Influence of number of PE
Maintenance processes seem less efficient with visuospatial memoranda
Difficult to reconcile with interference-only theories
n.s. **
These are the results from 24 participants that xxx
Pretty straightforward graph
whatever the protection processes at play, they seem to be highly dependent on the type of memoranda
Uittenhove & Barrouillet

13. Domain-specific rehearsal mechanisms
Observing maintenance
Domain-specific rehearsal mechanisms
Can counteract temporal decay
Role in distractor removal ?
Free time duration
Increased refreshing / reconstruction
Increased distractor removal
Nature of memoranda
Discriminability, semantic richness
We then set out to explore the roots of the differential maintenance efficiency of verbal and visuospatial information over successive PE. Multiple avenues are possible to achieve this, all of which we explored.
We used all of these manipulations in conjunction with manipulating the number of PE, to see how all these different factors contribute to how information maintenance evolves over successive PE, and possibly get to the root of the differences between verbal and visuospatial memoranda.
Refreshability: Attentional refreshing could be more efficient for verbal than for visual information, in line with what Majerus said about attention-driven segmentation being more efficient for verbal information, which was visible in the dorsal attentional network activation being different between these two types of information

14. Expt. 2: Role of the phonological loop
n.s.
Role of the phonological loop in the resistance of verbal information over successive processing episodes
***
N = 24

15. Expt. 3: Free time 600 ms -> 1200 ms
- When specific rehearsal mechanisms (phonological loop) are unavailable
***
n.s.
Extra time benefited verbal information maintenance, making it stable over successive PE
This was not observed for visuospatial information
Maintenance mechanisms other than the phonological loop are more efficient for verbal memoranda
whereas the number of processing episodes did no longer significantly affect verbal recall under AS (mean spans of 3.21 and 3.00 for 2 and 6 processing episodes, respectively), F (1, 25) = 1.41, p = 0.25,
visuospatial recall performance still decreased between 2 and 6 processing episodes (mean spans of 3.63 and 2.77 for 2 and 6 processing episodes respectively), F (1, 25) = 16.35, p < .001.
N = 24

16. Expt. 4: Role of LTM representations
Words vs non-words
Selective strengthening of words over successive PE
Independent from rehearsal mechanisms
LTM representations influence maintenance efficiency
N = 48
Our results (see Figure 5) show that the effect of the number of processing episodes interacted with the type of memoranda, F (1, 46) = 11.78, p < .01, and this did not differ with the presence or absence of AS, F (1, 46) < 1. Whereas increasing the number of processing episodes had a positive effect on the recall of words (mean spans of 3.54 and 3.84 for 2 and 6 processing episodes, respectively), F (1, 46) = 5.43, p < .05, the degrading effect of increasing numbers of processing episodes on non-words just fell short of significance (mean spans of 2.35 and 2.19 for 2 and 6 processing episodes, respectively), F (1, 46) = 2.86, p = .097.
-Number of processing episodes x type of memoranda (p < .01)
-Increasing number of PE had a significant positive effect for words, and a tendential negative effect for non-words
-No interaction with AS

17. Expt. 5: Discriminable visual memoranda
More features compared to spatial locations w/ single feature
Memory traces of figures resisted the succession of PE
The extent to which memoranda can be discriminated plays a role in how much they can be fortified to overcome successive distractors
Beyond the lack of long-term memory knowledge and domain-specific systems of maintenance for the visuospatial information, another peculiarity of the spatial locations in our study is their high level of similarity, differing only by the single feature of their location in the grid.
N = 24, articulatory suppression
Results showed no effect of the number of processing episodes on the recall of figures (mean spans of 1.50 and 1.52 for 2 and 6 episodes, SD = 0.65 and 0.75, respectively), F < 1.
Mean spans of 1.50 and 1.52 for 2 and 6 episodes , F < 1

18. Maintenance mechanisms ?
Rehearsal: Domain- specific (phonological loop)
Refreshing ?
Reconstruction (elaborative rehearsal):
LTM representations (words vs non-words)
Discriminability: Distinguishing features (abstract objects vs spatial locations)
Here: availability of rehearsal mechanisms (no specific visual store, candice morey meta-analysis)
Refreshability: Role of differential attentional segmentation (steve majerus) ?
Attentional refreshing could be more efficient for verbal than for visual information, in line with what Majerus said about attention-driven segmentation being more efficient for auditory information, which was visible in the dorsal attentional network activation being different between these two types of information
Reconstruction (different from mere refreshing, like eddy davelaar mentioned in his talk)
-semantic richness (LTM): Marlène gist and verbatim representations
-discriminability (Zhang and Luck, sudden death of information when the sole distinguishing feature getslost, or even higher confusion between memoranda when only one feature discriminates them, Simmering et al. 2008)
The fact that different memoranda are differentially affected by the increase in the number of processing episodes whereas they decline in the same way when these processing episodes last longer suggests that WM representations undergo a passive decay during processing, but are actively restored by refreshing mechanisms, the efficiency of this restoration depending on the availability of these refreshing mechanisms and on the nature of the WM representations. Depending on the nature of the information to be maintained and its semantic richness, reconstruction either overcomes loss, just contains it, or fails to counteract it.

19. In conclusion Words Letters Abstract objects Non-words Locations
Maintenance: When CL is constant, effects of augmenting number of PE vary with the nature of the memoranda
Differential maintenance of different types of memoranda based on semantic richness, discriminability, and availability of rehearsal mechanisms
Points at active reconstructive processes
The interplay between loss and maintenance is altered by the nature of target information to be maintained, any models attempting to characterize this interplay would thus need to factor in this variable.
Words
Letters
Abstract objects
Non-words
Locations
Positive effects
Number of PE
Negative effects

20. jonides (2008) de-sycnhronization of neuronal circuits

21. Expt. 6: Control experiment w/ parity judgment
It could be argued that the negative effect on visuospatial maintenance of the number of processing episodes systematically observed in our study was due to the use of distracting tasks involving spatial attention. The number comparison task required participants to consecutively attend to numbers displayed on four spatially distinct locations on screen. To discard this possibility, we conducted a control experiment involving a distracting task with no obvious spatial competent (i.e., a parity judgment task).
Results show an interaction between the number of processing episodes and the type of memoranda, F (1, 23) = , p < .01. Whereas increasing the number of processing episodes had no significant effect on verbal recall (mean spans for letters of 4.90 and 5.21 for 2 and 6 processing episodes, respectively), F (1, 23) = 2.78, p = .11, it had a strong negative effect on the recall of spatial locations (mean spans of 4.38 and 3.79 for 2 and 6 processing episodes, respectively), F (1, 23) = 12.67, p < .01. Thus, this experiment replicated and extended to another distracting task the differential sensitivity of verbal and spatial memoranda to increasing numbers of intervening processing episodes.