Thursday, August 27, 2009

Reconstructive Memory (vs. Storage and Retrieval) and "Experience Sampling"

In both cognitive science and folk psychology, the dominant metaphor for memory – a metaphor that both reflects and reinforces a certain way of thinking about it – is the metaphor of storage and retrieval (often with a search in the middle). There’s one particular aspect of this metaphor I want to highlight in this post: On the storage-and-retrieval picture, memory is a process that, once initiated, can and typically should operate largely independently of other cognitive processes. Other processes like inferring, imagining, and perceiving interfere with pure remembering. To the extent those processes influence one’s final judgment about some remembered fact or event, one isn’t really quite remembering it.

This isn’t to say, of course, that on such a model inferring, imagining, or perceiving couldn’t sometimes be helpful. When something is difficult to recall, they might help one recall it, perhaps by giving clues about where to look in one’s memory stores. (If the clue is specific enough, they might even turn a recall task into a recognition task.) They might appropriately increase or decrease one’s confidence in the results of the retrieval process. But if one’s aim is as purely and cleanly as possible simply to remember, there’s something problematic in allowing such processes to play anything but a secondary role. And one might worry that they’re as likely, perhaps more likely, to distort and corrupt the memory as to enable it.

Bartlett (1932), Neisser (1967), and Roediger (1980) have ably described the various infelicities of this storage-and-retrieval picture. When the task is to remember a complex event or a complex passage (as in Bartlett’s seminal research) the core problem with the retrieval metaphor is more evident than when the task is to recall, say, a list of numbers or nonsense syllables. If I tell you a story about a cricket match and ask you to recall it later, you will not reproduce the story verbatim. Nor will you reproduce gappy but verbatim pieces of the story. Rather, you will produce a new version of the story, in light of your general background knowledge of cricket. This half-inventive process is especially revealed by your plausible mistakes and interpolations, but there’s no reason to suppose that it would only be the mistakes and interpolations that show the heavy influence of background knowledge. Someone, for example, without that background knowledge would not do nearly so well remembering overall (even if certain mistakes are more likely). Nor is this simply a matter of a cricket-knowledgeable person encoding the story better in the first hearing and thus “storing” it differently (though no doubt hearing the story knowledgeably is very important to remembering it well later). Knowledge of cricket is also used to construct or reconstruct the story at the time of recall. If, in the intervening time, new knowledge of cricket is acquired, that will affect the reconstruction, probably for the better if the match was real and typical. (In my own case, I have particularly noticed the profound effect of new knowledge on my reconstructive memory of philosophical works I read as an undergraduate.)

Bartlett writes:

Remembering is not the re-excitation of innumerable fixed, lifeless and fragmentary traces. It is an imaginative reconstruction, or construction, built out of the relation of our attitude toward a whole active mass of organized past reactions or experience, and to a little outstanding detail which commonly appears in image or in language form. It is thus hardly even really exact, even in the most rudimentary cases of role recapitulation, and it is not at all important that it should be so (1932, p. 213).
From the fact that memory is reconstructive in this way – necessarily reconstructive, at least for complex events – it follows that imagination, inference, the application of pre-existing schemata, and other cognitive processes are not separable from the process of remembering but rather an integral part of it. They are not interfering or aiding forces from which an act of “pure” remembering could be isolated.

Let's apply this to an example, from "experience sampling" -- a topic close to my heart.

An event transpires in your stream of experience – an image of warplanes in flight, say – and then a randomly generated beep occurs, signaling that you are to try your best to recall that moment of experience, which is to say the last undisturbed moment of experience before the sampling beep. Russ Hurlburt (or someone else) will interview you about it later, trying to discover in this way the truth about randomly sampled moments of your everyday, lived experience. (Now that's pretty cool, don't you think?) Okay, so what's going to happen?

First, let's note the obvious: That target event is now gone. Furthermore, there’s no reason to think your brain would have stored a detailed and enduring record of that event as it was ongoing. As change blindness experiments have shown, as well as experiments about the forgetting of mundane everyday details (even details frequently seen like the layout of a penny), we almost instantly forget many, perhaps most, major features of the environment (Sanford 1917/1982; Nickerson and Adams 1979; Rensink, O’Regan, and Clark 1997, 2000; Simons and Levin 1998). You may try to retain that image of warplanes over the duration of the beep and the post-beep reflection, using that retained image as a model for the image as it existed the moment before the beep; but surely it’s plausible to suppose that the image might be transformed, elaborated, rendered artificial in the course of retention, and it may be very difficult to detect such changes reliably, accurately accounting for and subtracting them when reaching judgments about the target experience at the moment of the beep.

Or you may try to recreate the image, if it was momentarily lost, which would appear to invite all the same risks if not more.

Or you may try to recall the image without retaining or recreating it (perhaps purely linguistically?), but this too will be a constructive or reconstructive act, involving (for example) one’s knowledge of warplanes, how you take them generally to look, knowledge of the outward event that inspired the image (a passage in a book, say), and probably also one’s general opinions about imagery. It will not be the simple retrieval of a recorded trace, in high or low pixilation, but rather elaborative, constructive, and plausibility- and schemata-grounded, like Bartlett’s subjects’ recollections stories and passages of text.

Then, hours later, you are interviewed, and the reconstructive process begins again, with the target event less fresh, but – perhaps compensatingly – with more available bases for the reconstruction: all the general knowledge (or opinions), schemata, and skills that were available (except literal retention) in the first instance of recollection after the beep; plus also one’s knowledge of, or best recollection of, the judgments and other processes that occurred after the beep; plus one’s written notes; plus cues (maybe subtle) from the interviewer; plus one’s knowledge of the intervening beeps and interviews. From this confluence of forces issues an utterance, “they’re jet planes with a tapered nose and that kind of dark gray steel with a…”, which the interviewer interprets in accord with his own system of schemata and prejudices.

This, I think, is the cognitive process underlying interviews about sampled experiences – both in Hurlburt’s method and in related methods like Petitmengin's. You see, then, why I think there’s plenty of room for error.


Dan said...

Isn't memory as reconstruction the standard view in cognitive psychology? That's how I learned it, at least.

The experience sampling method typically has people answer questions right after the beep goes off. One of the main reasons for doing beeper studies is so you can get people's reports while they're still active in working memory, so that there doesn't need to be as much reconstruction.

Swati said...

I am a layperson, and so unaware of the studies you cite. Just wanted to know if I have understood correctly: remembering is not just storing a piece of information and then searching and extracting it. At each level, there is one's own experience modifying - when storing, and when recounting. And you object to the interview method because the interviewer has his or her own biases or backgrounds, and so will not listen objectively?

Michael Metzler said...

Cool stuff Eric. It is interesting that you use for the main example the recollection of a story that was originally heard. Recollection takes play through the creative ("half-creative") on-line construction of narrative meaning based on background knowledge, etc. This seems right to me. But I would want to take this idea further: is it not true that it is precisely this creative on-line construction that occurs when one comprehends the story originally? During the original comprehension of the story, background knowledge, social context, immediately preceding cognitive state, and perceptual input of the story teller are all employed, in addition to the words spoken by the story teller, which themselves are but under-determining linguistic cues for on-line, creative construction of the narrative meaning (Turner, 1996; Coulson, 2001, 2006; Coulson & Oakley 2000, 2005). It seems likely that one would originally walk away from hearing 'the story' with little recollection at all of the actual linguistic cues the story teller used, since it would be the resulting, creative, on-line construction that was fresh in mind, not the linguistic cues employed to induce it. Accurate recall would therefore likely require that the listener use different linguistic cues. I like your conception of recall, but I think many are apt to assume something like the storage-and-retrieval model for the original decoding of the story teller's linguistic meaning, a "process that, once initiated, can and typically should operate largely independently of other cognitive processes," like "imagining." Perhaps the recall process is little different from the original comprehension process. No?

Eric Schwitzgebel said...

Thanks for the comment, Dan! The reconstruction view of memory may be standard in explicit discussions, but that's not reflected very well, I think, in many of the metaphors, models, and implicit suppositions. Also, I assuming that "working memory" is either very quickly dumped or must be constantly refreshed (which would itself involve reconstruction, I think); even the first few moments of reflection after the beep will already be reconstructive.

Eric Schwitzgebel said...

Swati: The first part of that is not a bad summary, although I'm not sure "modifying" is the right word -- as though there is some real, independent thing we could ideally get at and unfortunately always end up "modifying". On your last sentence: My view is rather more nuanced than that. I don't object to experience sampling. I don't think there's any *better* method (though I do think there are alternative methods with a roughly similar degree of promise and flaw), but I do think we should keep at the forefront of our minds, in thinking about experiencing sample, how much room there is for error to enter.

Eric Schwitzgebel said...

Thanks, Michael, that's a very nice thought, and I agree with you!