Reconstructive Memory (vs. Storage and Retrieval) and "Experience Sampling"

In both cognitive science and folk psychology, the dominant metaphor for memory – a metaphor that both reflects and reinforces a certain way of thinking about it – is the metaphor of storage and retrieval (often with a search in the middle). There’s one particular aspect of this metaphor I want to highlight in this post: On the storage-and-retrieval picture, memory is a process that, once initiated, can and typically should operate largely independently of other cognitive processes. Other processes like inferring, imagining, and perceiving interfere with pure remembering. To the extent those processes influence one’s final judgment about some remembered fact or event, one isn’t really quite remembering it.

This isn’t to say, of course, that on such a model inferring, imagining, or perceiving couldn’t sometimes be helpful. When something is difficult to recall, they might help one recall it, perhaps by giving clues about where to look in one’s memory stores. (If the clue is specific enough, they might even turn a recall task into a recognition task.) They might appropriately increase or decrease one’s confidence in the results of the retrieval process. But if one’s aim is as purely and cleanly as possible simply to remember, there’s something problematic in allowing such processes to play anything but a secondary role. And one might worry that they’re as likely, perhaps more likely, to distort and corrupt the memory as to enable it.

Bartlett (1932), Neisser (1967), and Roediger (1980) have ably described the various infelicities of this storage-and-retrieval picture. When the task is to remember a complex event or a complex passage (as in Bartlett’s seminal research) the core problem with the retrieval metaphor is more evident than when the task is to recall, say, a list of numbers or nonsense syllables. If I tell you a story about a cricket match and ask you to recall it later, you will not reproduce the story verbatim. Nor will you reproduce gappy but verbatim pieces of the story. Rather, you will produce a new version of the story, in light of your general background knowledge of cricket. This half-inventive process is especially revealed by your plausible mistakes and interpolations, but there’s no reason to suppose that it would only be the mistakes and interpolations that show the heavy influence of background knowledge. Someone, for example, without that background knowledge would not do nearly so well remembering overall (even if certain mistakes are more likely). Nor is this simply a matter of a cricket-knowledgeable person encoding the story better in the first hearing and thus “storing” it differently (though no doubt hearing the story knowledgeably is very important to remembering it well later). Knowledge of cricket is also used to construct or reconstruct the story at the time of recall. If, in the intervening time, new knowledge of cricket is acquired, that will affect the reconstruction, probably for the better if the match was real and typical. (In my own case, I have particularly noticed the profound effect of new knowledge on my reconstructive memory of philosophical works I read as an undergraduate.)

Bartlett writes:

Remembering is not the re-excitation of innumerable fixed, lifeless and fragmentary traces. It is an imaginative reconstruction, or construction, built out of the relation of our attitude toward a whole active mass of organized past reactions or experience, and to a little outstanding detail which commonly appears in image or in language form. It is thus hardly even really exact, even in the most rudimentary cases of role recapitulation, and it is not at all important that it should be so (1932, p. 213).

From the fact that memory is reconstructive in this way – necessarily reconstructive, at least for complex events – it follows that imagination, inference, the application of pre-existing schemata, and other cognitive processes are not separable from the process of remembering but rather an integral part of it. They are not interfering or aiding forces from which an act of “pure” remembering could be isolated.

Let's apply this to an example, from "experience sampling" -- a topic close to my heart.

An event transpires in your stream of experience – an image of warplanes in flight, say – and then a randomly generated beep occurs, signaling that you are to try your best to recall that moment of experience, which is to say the last undisturbed moment of experience before the sampling beep. Russ Hurlburt (or someone else) will interview you about it later, trying to discover in this way the truth about randomly sampled moments of your everyday, lived experience. (Now that's pretty cool, don't you think?) Okay, so what's going to happen?

First, let's note the obvious: That target event is now gone. Furthermore, there’s no reason to think your brain would have stored a detailed and enduring record of that event as it was ongoing. As change blindness experiments have shown, as well as experiments about the forgetting of mundane everyday details (even details frequently seen like the layout of a penny), we almost instantly forget many, perhaps most, major features of the environment (Sanford 1917/1982; Nickerson and Adams 1979; Rensink, O’Regan, and Clark 1997, 2000; Simons and Levin 1998). You may try to retain that image of warplanes over the duration of the beep and the post-beep reflection, using that retained image as a model for the image as it existed the moment before the beep; but surely it’s plausible to suppose that the image might be transformed, elaborated, rendered artificial in the course of retention, and it may be very difficult to detect such changes reliably, accurately accounting for and subtracting them when reaching judgments about the target experience at the moment of the beep.

Or you may try to recreate the image, if it was momentarily lost, which would appear to invite all the same risks if not more.

Or you may try to recall the image without retaining or recreating it (perhaps purely linguistically?), but this too will be a constructive or reconstructive act, involving (for example) one’s knowledge of warplanes, how you take them generally to look, knowledge of the outward event that inspired the image (a passage in a book, say), and probably also one’s general opinions about imagery. It will not be the simple retrieval of a recorded trace, in high or low pixilation, but rather elaborative, constructive, and plausibility- and schemata-grounded, like Bartlett’s subjects’ recollections stories and passages of text.

Then, hours later, you are interviewed, and the reconstructive process begins again, with the target event less fresh, but – perhaps compensatingly – with more available bases for the reconstruction: all the general knowledge (or opinions), schemata, and skills that were available (except literal retention) in the first instance of recollection after the beep; plus also one’s knowledge of, or best recollection of, the judgments and other processes that occurred after the beep; plus one’s written notes; plus cues (maybe subtle) from the interviewer; plus one’s knowledge of the intervening beeps and interviews. From this confluence of forces issues an utterance, “they’re jet planes with a tapered nose and that kind of dark gray steel with a…”, which the interviewer interprets in accord with his own system of schemata and prejudices.

This, I think, is the cognitive process underlying interviews about sampled experiences – both in Hurlburt’s method and in related methods like Petitmengin's. You see, then, why I think there’s plenty of room for error.