Why the Epistemology of Conscious Perception Needs a Theory of Consciousness

On a certain type of classical "foundationalist" view in epistemology, knowledge of your sensory experience grounds knowledge of the outside world: Your knowledge that you're seeing a tree, for example, is based on or derived from your knowledge that you're having sensory experiences of greens and browns in a certain configuration in a certain part of your visual field. In earlier work, I've argued that this can't be right because our knowledge of external things (like trees) is much more certain and secure than our knowledge of our sensory experiences.

Today I want to suggest that foundationalist or anti-foundationalist claims are difficult to evaluate without at least an implicit background theory of consciousness. Consider for example these three simple models of the relation between sensory experience, knowledge of sensory experience, and knowledge of external objects. The arrows below are intended to be simultaneously causal and epistemic, with the items on the left both causing and epistemically grounding the items on the right. (I've added small arrows to reflect that there are always also other causal processes that contribute to each phase.)

[apologies for blurry type: click to enlarge and clarify]

Model A is a type of classical foundationalist picture. In Model B, knowledge of external objects arises early in cognitive processing and informs our sensory experiences. In Model C, sensory experience and knowledge of external objects arise in parallel.

Of course these models are far too simple! Possibly, the process looks more like this:

How do we know which of the three models is closest to correct? This is, I think, very difficult to assess without a general theory of consciousness. We know that there's sensory experience, and we know that there's knowledge of sensory experience, and we know that there's knowledge of external objects, and that all of these things happen at around the same time in our minds; but what exactly is the causal relation among them? Which happens first, which second, which third, and to what extent do they rely on each other? These fine-grained questions about temporal ordering and causal influence are, I think, difficult to discern from introspection and thought experiments.

Even if we allow that knowledge of external things informs our sense experience of those things, that can easily be incorporated in a version of the classical foundationalist model A, by allowing that the process is iterative: At time 1, input causes experience which causes knowledge of experience which causes knowledge of external things; then again at time 2; then again at time 3.... The outputs of earlier iterations could then be among the small-arrow inputs of later iterations, explaining whatever influence knowledge of outward things has on sensory experiences within a foundationalist picture.

On some theories, consciousness arises relatively early in sensory processing -- for example, in theories where sensory experiences are conscious by virtue of their information's being available for processing by downstream cognitive systems (even if that availability isn't much taken advantage of). On other theories, sensory consciousness arises much later in cognition, only after substantial downstream processing (as in some versions of Global Workspace theory and Higher-Order theories). Although the relationship needn't be strict, it's easy to see how views according to which consciousness arises relatively early fit more naturally with foundationalist models than views according to which consciousness arises much later.

The following magnificent work of art depicts me viewing a tree:

[as always, click to enlarge and clarify]

Light from the sun reflects off the tree, into my eye, back to primary visual cortex, then forward into associative cortex where it mixes with associative processes and other sensory processes. In my thought bubble you see my conscious experience of the tree. The question is, where in this process does this experience arise?

Here are three possibilities:

Until we know which of these approaches is closest to the truth, it's hard to see how we could be in a good position to settle questions about foundationalism or anti-foundationalism in the epistemology of conscious perception.

(Yes, I know I've ignored embodied cognition in this post. Of course, throwing that into the mix makes matters even more complicated!)

Is Consciousness Sparse or Abundant? Five Dimensions of Analysis

Consciousness -- that is, "phenomenal" consciousness, the stream of experience, subjective experience, "what-it's-like"-ness -- might be sparse, or it might be abundant. There might be lots of it, or there might be very little. This might be so along at least five different dimensions of analysis.

Entity sparseness vs. entity abundance. Consciousness might be sparse in the sense that few entities in the universe possess it, or it might be abundant in the sense that many entities in the universe possess it. Someone who thinks that consciousness is entity-abundant might think that insects are conscious, and snails, maybe earthworms -- all kinds of entities with sensory systems. They might also think that computers could soon have conscious experiences, if designed in the right way. At the extreme, they might accept panpsychism -- the view that every entity in the universe is conscious, even isolated hydrogen ions in outer space. In contrast, someone who thinks that consciousness is entity-sparse is much more conservative, seeing consciousness on Earth as limited, for example, only to cognitively sophisticated mammals and birds, or in the extreme only to adult human beings with sophisticated self-reflective powers.

State sparseness vs. state abundance. An entity who is conscious might be conscious all the time or only once in a while. Someone who thinks that consciousness is state abundant might think that even when we aren't in REM sleep we have dreams or dreamlike experiences or at least experiences of some sort. They might think that when we're driving absent-mindedly and can't remember a thing, we don't really blank out completely. In contrast, someone who thinks that consciousness is state sparse would hold that we are often not conscious at all. Consciousness might disappear entirely during long periods of dreamless sleep, or during habitual activity, or maybe during "flow" states of skillful activity. Someone who holds to extreme state sparseness might say that we are almost never actually phenomenally conscious, except in rare moments of explicit self-reflection.

Modality sparseness vs. modality abundance. An entity who is currently state conscious might be conscious in lots of modalities at once or in only one or few modalities at a time. For example, a minute ago, as you were reading the previous paragraph, did you have visual experience of the computer screen? Did you also have auditory experience of the hum of the fan in your room (to which I assume you weren't attending)? Did you have tactile experience of your feet in your shoes? Did you hear the words of that paragraph in inner speech? Did you have relevant visual imagery? Were you also simultaneously having emotional experience, a lingering feeling of hunger, etc., etc.? Someone who accepts modality abundance thinks that we have many types or modalities of experience ongoing most of the time when we are conscious. In contrast, someone who accepts modality sparseness thinks that we have only one or a few modalities of experience at any one time -- for example, no experience at all of the feeling of your feet in your shoes when your attention is elsewhere.

Modality width. Within a modality that is currently conscious in an entity at a time, the stream of experience might be broad or it might be narrow. Consider visual experience as you are reading. Do you have visual experience only of a few degrees of visual arc, whatever is near the center of your attention? Or do you have visual experience all the way out to the edge, including the rim of your computer screen, the rims of your glasses, the tip of your nose, almost 180 degrees of arc? Or somewhere in between -- maybe the whole computer screen and its rim, but little beyond that? Someone with a sparse view about visual modal width thinks that visual experience is usually (maybe until you think to attend to the periphery) only of a few degrees of arc. Someone with an abundant view might think that we basically experience the full 180 degrees of the visual field all the time when we have any visual experience at all. Analogous issues arise for other modalities. Do you have auditory experience only of the conversation to which you are attending, or also of the background noise? Is your visual imagery sketchily minimal or full of detail? Is your emotional experience a minimal valence and label or is it a very specifically embodied feeling?

Property sparseness vs. property abundance. Consider visual experience again. You are looking at a tree. According to one view, all you visually experience are low-level properties like color, shape, and orientation. You know it's a tree, but the "treeness" of it isn't part of your visual experience. (Maybe you have a cognitive experience of thinking "tree!" but that's a different modality.) According to another view, your visual experience is not just of simple low-level properties but instead of a wealth of properties. It's part of your visual experience, for example, that it's a tree, and that it looks ancient, and that it invites climbing, and that the leaves look about ready to fall.

Philosophers and consciousness scientists have recently been arguing in all kinds of interesting ways about various of these issues in isolation, but I can't recall seeing a good structuring of the landscape of options here which both captures the many different mix-and-match possibilities here, while also recognizing that all of the "abundant" views have something in common (consciousness is widespread and multifarious; it's easy to generate lots of types of it) and all of the "sparse" views have something in common (consciousness is rare and limited in its forms).

[image source]

Minds Online Conference

is cooking along. You might want to check out this week's lineup, on Perception and Consciousness:

Nico Orlandi (UC Santa Cruz): "Bayesian Perception Is Ecological Perception" (KEYNOTE)

Derek H. Brown (Brandon University): “Colour Layering and Colour Relationalism” (Commentators: Mazviita Chirimuuta and Jonathan Cohen)

Jonathan Farrell (Manchester): “‘What It Is Like’ Talk Is Not Technical Talk” (Commentators: Robert Howell and Myrto Mylopoulos)

E.J. Green (Rutgers University): “Structure Constancy” (Commentators: John Hummel and Jake Quilty-Dunn)

Assaf Weksler (Open University of Israel and Ben Gurion University): “Retinal Images and Object Files: Towards Empirically Evaluating Philosophical Accounts of Visual Perspective” (Commentators: René Jagnow and Joulia Smortchkova)

How to Make Van Gogh's "Starry Night" Undulate

Not sure the original source of this one (maybe notbecauseitsironic on Reddit?).

First, look at the center of the image below for about 30 seconds.

Look at the center of this image for 30sec, then watch Van Gogh's *Starry Night* come to life

Then look at Van Gogh's "The Starry Night".

The technique also achieves interesting results when applied to Kincade:

[HT Mariano Aski]

"A" Is Red, "I" Is White, "X" Is Black -- Um, Why?

This is just the kind of dorky thing I think is cool. Check out this graph of the color associations for different letters for people with grapheme-color synesthesia.

[click on the picture for full size, if it's not showing properly]

This is from a sample of 6588 synesthetes in the US, reported in Witthoft, Winawer, and Eagleman 2015. Presumably, they're not talking to each other. But there's a pretty good agreement that "A" is red, "X" is black, and "Y" is yellow. But you knew that already, right?

Now some of these results seem partly explicable: "Y" is yellow, maybe, because of the word "yellow" starts with "Y". That might also work for "R" red, "B" blue, and "G" green. For "A" I think of the big red apple with the "A is for apple" posters that ubiquitously decorate kindergarten classrooms. But "O" is not particularly associated with orange in this chart, nor "W" with white. And why are "X" and "Z" black? Because we're tired because it's near the end of the alphabet and our eyelids are starting to droop doesn't seem like a good answer. (Does it?)

You might wonder whether it's only synesthetes who have this consensus of associations, and how stable such associations are over time or between countries.

You're in luck, then, because here's another cool chart, from Australia in 2005!

[again, click for clearer view]

The colored bars are synesthetic respondents and the hatched bars are non-synesthetic respondents. The patterns are similar between synesthetes and non-synesthetes, but maybe with the non-synesthetes tending toward stronger associations between the color and the initial letter of the color word. Furthermore, again "A" is red, "I" is white, and "X" and "Z" are black. US and Australian synesthetes seems to agree that "O" is white, but the Australian non-synesthetes like their "O" orange. For some reason, "D" is now brown (47%!).

There are some older US data from the underappreciated early introspective psychologist Mary Whiton Calkins in her classic 1893 paper on synesthesia. [Pop quiz: Who are the only three people to have been president of both the American Psychological Association and the American Philosophical Association? Answer: William James, John Dewey, and Mary Whiton Calkins.] She reports that synesthetes tend to associate "I" with black and "O" with white. "O" being white matches the synesthete reports from the US and Australia in 2015 and 2005, but Calkins's black "I" is different. Calkins reports this possible explanation for the whiteness of "O", from one of her participants, seeming to find it plausible: O "= cipher = blank = sheet of white paper".

Witthoft et al. 2015 found that almost a sixth of their participants born in the US in the late 1970s (but not those born before 1967) seem to have letter-color associations that match much better than chance with the colors of the letters of this then-popular magnet toy:

[image source]

Neat finding. Of course, the darned toy has "X" purple and "Z" orange, so it's all wrong!

Brang, Rouw, Ramachandran and Coulson 2011 find a weak tendency for similarly-shaped letters to associate to similar colors in US sample. Irish-based Barnett et al. 2008 and British-based Simner et al. 2015 find broadly similar patterns to the other recent English-language populations.

Spector and Maurer 2011 find that even pre-literate English-speaking Canadian toddlers associate "O" and "I" with white and "X" and "Z" with black, though they do not share older participants' associations of "A" with red, "B" with blue, "G" with green, and "Y" with yellow. They hypothesize that jagged shapes ("X" and "Z") might be more likely to have shaded portions in a natural environment than non-jagged shapes ("O" and "I"), and that other, later associations might be language based. However, color maps of Swiss research on German-language synesthetes (Beeli, Esslen, and Jaencke 2007) shows no such relationship (see the chart on p. 790) -- for example with more participants associating "X" with white or light gray than with black or dark gray (though Simner et al. have a German subset which do show black associations with "X" and "Z"). Beeli et al. find a weak tendency for higher frequency letters to be associated with higher saturation colors in a German-language sample. Rouw et al. 2014 found that Dutch and English-speaking non-synesthetic participants had similar associations for "A" (red), "B" (blue), "D" (brown), "E" (yellow), "I" (white), and "N" (brown). Hindi participants, with their different alphabet, had a rather different set of associations -- though the first letter of the Hindi alphabet was also associated with red. They speculate that the first letter in each alphabet gets a "signal" color.

Okay, so now you know!

Let me leave you then, with this highly unnatural thought:

Whoa.

MIT Press BITS

One reason I'm a fan of MIT Press (the publisher of both of my books) is that for an academic press their prices are very low (my 2011 book is currently $14.21 at Amazon) which means that a broader range of people can afford the book than if it were at another press. Another reason I'm a fan is that MIT has tended to be a leader in exploring new electronic media.

So it's very cool that they've chosen a chapter of my Perplexities of Consciousness for their BITS project, a new enterprise which allows people to electronically buy a portion of an MIT Press book for a low price ($2.99 in this case) and then later, if the reader wants, the whole book for 40% off list price. The chapter they've chosen is "When Your Eyes Are Closed, What Do You See?", which although it is the eighth and final chapter of my book, does not require that the reader know material from the previous chapters -- thus, a reasonable choice for a BIT.

What I'd really love to see down the road is a model where you can buy any selection of pages from a book for a nickel per page.

Striking Confirmation of the Spelunker Illusion

In 2010, I worked up a post on what I dubbed The Spelunker Illusion (see also the last endnote of my 2011 book). Now, hot off the press at Psychological Science, Kevin Dieter and colleagues offer empirical confirmation.

The Spelunker Illusion, well-known among cave explorers, is this: In absolute darkness, you wave your hand before your eyes. Many people report seeing the motion of the hand, despite the absolute darkness. If a friend waves her hand in front of your face, you don't see it.

I see three possible explanations:

(1.) The brain's motor output and your own proprioceptive input create hints of visual experience of hand motion.

(2.) Since you know you are moving your hand, you interpret low-level sensory noise in conformity with your knowledge that your hand is in such-and-such a place, moving in such-and-such a way, much as you might see a meaningful shape in a random splash of line segments.

(3.) There is no visual experience of motion at all, but you mistakenly think there is such experience because you expect there to be. (Yes, I think you can be radically wrong about your own stream of sensory experience.)

Dieter and colleagues had participants wave their hands in front of their faces while blindfolded. About a third reported seeing motion. (None reported seeing motion when the experimenter waved his hand before the participants.) Dieter and colleagues add two interesting twists: One is that they add a condition in which participants wave a cardboard silhouette of a hand rather than the hand itself. Under these conditions the effect remains, almost as strong as when the hand itself is waved. The other twist is that they track participants' eye movements.

Eye movements tend to be jerky, jumping around the scene. One exception to this, however, is smooth pursuit, when one stabilizes ones gaze on a moving object. This is not under voluntary control: Without an object to track, most people cannot move their eyes smoothly even if they try. In 1997, Katsumi Watanabe and Shinsuke Shimojo found that although people had trouble smoothly moving their eyes in total darkness, they could do so if they were trying to track their ("invisible") hand motion in darkness. Dieter and colleagues confirmed smooth hand-tracking in blindfolded participants and, strikingly, found that participants who reported sensations of hand motion were able to move their eyes much more smoothly than those who reported no sensations of motion.

I'm a big fan of corroborating subjective reports about consciousness with behavioral measures that are difficult to fake, so I love this eye-tracking measure. I believe that it speaks pretty clearly against hypothesis (3) above.

Dieter and colleagues embrace hypothesis (1): Participants have actual visual experience of their hands, caused by some combination of proprioceptive inputs and efferent copies of their motor outputs. However, it's not clear to me that we should exclude hypothesis (2). And (1) and (2) are, I think, different. People's experience in darkness is not merely blank or pure black, but contains a certain amount (perhaps a lot) of noise. Hypothesis (2) is that the effect arises "top down", as it were, from one's high-level knowledge of the position of one's hand. This top-down knowledge then allows you to experience that noisy buzz as containing motion -- perhaps changing the buzz itself, or perhaps not. (As long as one can find a few pieces of motion in the noise to string together, one might even fairly smoothly track that motion with one's eyes.)

Here's one way to start to pull (1) apart from (2): Have someone else move your hand in front of your face, so that your hand motion is passive. Although this won't eliminate proprioceptive knowledge of one's hand position, it should eliminate the cues from motor output. If efferent copies of motor output drive the Spelunker Illusion, then the Spelunker Illusion should disappear in this condition.

Another possibility: Familiarize participants with a swinging pendulum synchronized with a sound, then suddenly darken the room. If hypothesis (2) is correct and the sound is suggestive enough of the pendulum's exact position, perhaps participants will report still visually experiencing that motion.

Update, April 28, 2014:

Leonard Rosgole and Miguel Roig point out to me that these phenomena were reported in the psychological literature in Hofstetter 1970, Brosgole and Neylon 1973, Brosgole and Roig 1983. If you're aware of earlier sources, I'd be curious to know.

The Experience of Reading: Imagery, Inner Speech, and Seeing the Words on the Page

(by Alan T. Moore and Eric Schwitzgebel)

What do you usually experience when you read?

Some people say that they generally hear the words of the text in their heads, either in their own voice or in the voices of narrator or characters; others say they rarely do this. Some people say they generally form visual images of the scene or ideas depicted; others say they rarely do this. Some people say that when they are deeply enough absorbed in reading, they no longer see the page, instead playing the scene like a movie before their eyes; others say that even when fully absorbed they still always visually experience the words on the page.

Some quotes:

Baars (2003): “Human beings talk to themselves every moment of the waking day. Most readers of this sentence are doing it just now.”

Jaynes (1976): “Right at this moment… as you read, you are not conscious of the letters or even of the words, or even of the syntax or the sentences, or the punctuation, but only of their meaning.”

Titchener (1909): “I instinctively arrange the facts or arguments in some visual pattern [such as] a suggestion of dull red… of angles rather than curves… pretty clearly, the picture of movement along lines, and of neatness or confusion where the moving lines come together.”

Wittgenstein (1946-1948): While reading “I have impressions, see pictures in my mind’s eye, etc. I make the story pass before me like pictures, like a cartoon story.”

Burke (1757): While reading “a very diligent examination of my own mind, and getting others to consider theirs, I do not find that one in twenty times any such picture is formed.”

Hurlburt (2007): Some people “apparently simply read, comprehending the meaning without images or speech. Melanie’s general view… is that she starts a passage in inner speech and then “takes off” into images.”

Alan and I can find no systematic studies of the issue.

We recruited 414 U.S. mechanical Turk workers to participate in a study on the experience of reading. First we asked them for their general impressions about their own experiences while reading. How often -- on a 1-7 scale from "never" to "half of the time" to "always" -- do they experience visual imagery? Inner speech? The words on the page? (We briefly clarified these terms and gave examples.)

The responses:

[Note: For words on the page, we asked: "How often do you NOT experience the words on the page as you read? Example: your mind is filled with the ideas of the story and not the actual black letters against the white background". We have reversed the scale for presentation here.]

Now, if you're anything like me, you'll be pretty skeptical about the accuracy of these types of self-reports. So Alan and I did several things to try to test for accuracy.

Our general design was to give each person a passage to read, during which they were interrupted with a beep and asked if they were experiencing imagery, inner speech, or the words on the page. Afterwards, we asked comprehension questions, including questions about visual or auditory details of the story or about details of the visual presentation of the material (such as font). Finally, we asked again for participants' general impressions about how regularly they experience imagery, inner speech, and the words on the page when they read.

The comprehension questions were a mixed bag and difficult to interpret -- too much for this blog post (maybe we'll do a follow-up) -- but the other results are striking enough on their own.

Among those who reported "always" experiencing inner speech while they read, only 78% reported inner speech in their one sampled experience. Think a bit about what that means. Despite, presumably, some pressure on participants to conform to their earlier statements about their experience, it took exactly one sampled experience for 22% of those reporting constant inner speech to find an apparent counterexample to their initially expressed opinion. Suppose we had sampled five times, or twenty?

For comparison: 9% of those reporting "always" experiencing visual imagery denied experiencing visual imagery in their one sampled experience. And 42% did the same about visually experiencing the words on the page.

Participants' final reports about their reading experience, too, suggest substantial initial ignorance about their reading experience. The correlations between participants initial and final generalizations about reading experience were .47 for visual imagery, .58 for inner speech, and .37 for experience of words on the page. Such medium-sized correlations are quite modest considering that the questions being correlated are verbatim identical questions about participants' reading experience in general, with an interval of about 5-10 minutes between. One might have thought that if people's general opinions about their experience are well-founded, the experience of reading a single passage should have only a minimal effect on such generalizations.

What Are You Noticing When You Adjust Your Binoculars?

Maja Spener has written an interesting critique of my 2011 book, Perplexities of Consciousness, for a forthcoming book symposium at Philosophical Studies. My book is an extended argument that people have very poor knowledge of their own conscious experience, even of what one might have thought would be fairly obvious-seeming features of their currently ongoing experience (such as whether it is sparse or full of abundant detail, whether it is visually doubled, whether they have pervasive auditory experience of the silent surfaces around them). Maja argues that we have good introspective knowledge in at least one class of cases: cases in which our skillful negotiation with the world depends on what she calls "introspection-reliant abilities".

Maja provides examples of two such introspection-reliant abilities: choosing the right amount of food to order at a restaurant (which relies on knowledge of how hungry you feel) and adjusting binoculars (which relies on knowing how blurry the target objects look).

Let's consider, then, Maja's binoculars. What, exactly, are you noticing when you adjust your binoculars?

Consider other cases of optical reflection and refraction. I see an oar half-submerged in water. In some sense, maybe, it "looks bent". Is this a judgment about my visual experience of the oar or instead a judgment about the optical structure of my environment? -- or a bit of both? How about when I view myself in the bathroom mirror? Normally during my morning routine, I seem to be reaching judgments primarily about my body and secondarily about the mirror, and hardly at all about my visual experience. (At least, not unless we accept some kind of introspective foundationalism on which all judgments about the outside world are grounded on prior judgments about experience.) Or suppose I'm just out of the shower and the mirror is foggy; in this case my primary judgment seems to concern neither my body nor my visual experience but rather the state of the mirror. Similarly, perhaps, if the mirror is warped -- or if I am looking through a warped window, or a fisheye lens, or using a maladjusted review mirror at night, or looking in a carnival mirror.

If I can reach such judgments directly without antecedent judgments about my visual experience, perhaps analogously in the binoculars case? Or is there maybe instead some interaction among phenomenal judgments about experience and optical or objectual judgments, or some indeterminacy about what sort of judgment it really is? We can create, I'm inclined to think, a garden path between the normal bathroom mirror case (seemingly not an introspective judgment about visual experience) and the blurry binoculars case (seemingly an introspective judgment about visual experience), going either direction, and thus into self-contradiction. (For related cases feeding related worries see my essay in draft The Problem of Known Illusion and the Resemblance of Experience to Reality.)

Another avenue of resistance to the binoculars case might be this. Suppose I'm looking at a picture on a rather dodgy computer monitor and it looks too blue, so I blame the monitor and change the settings. Arguably, I could have done this without introspection: I reach a normal, non-introspective visual judgment that the picture of the baby seal is tinted blue. But I know that baby seals are white. So I blame the monitor and adjust. Or maybe I have a real baby seal in a room with dubious, theatrical lighting. I reach a normal visual assessment of the seal as tinted blue, so I know the lighting much be off and I ask the lighting techs to adjust. Similarly perhaps, then, if I gaze at a real baby seal through blurry binoculars: I reach a normal, non-introspective judgment of the baby seal as blurry-edged. But I know that baby seals have sharp edges. So I blame the binoculars and adjust. Need this be introspective at all, need it concern visual experience at all?

In the same way, maybe, I see spears of light spiking out of streetlamps at night -- an illusion, some imperfection of my eyes or eyewear. When I know I am being betrayed by optics, am I necessarily introspecting, or might I just be correcting my normal non-introspective visual assessments?

This is a nest of issues I am struggling with, making less progress than I would like. Maybe Maja is right, then. But if so, it will take further work to show.

Two Types of Hallucination

Oliver Sacks is one of the great essayists of our time. I have just finished his book Hallucinations.

Sacks does not, I think, adequately distinguish two types of hallucination. I will call them doxastic and phenomenal. In a phenomenal hallucination of A, one has sensory experience as of A. In a doxastic hallucination, one thinks one has sensory experience as of A. The two can come apart.

Consider this description, from page 99 of Hallucinations (and attributed to Daniel Breslaw via David Ebin's book The Drug Experience).

The heavens above me, a night sky spangled with eyes of flame, dissolve into the most overpowering array of colors I have ever seen or imagined; many of the colors are entirely new -- areas of the spectrum which I seem to have hitherto overlooked. The colors do not stand still, but move and flow in every direction; my field of vision is a mosaic of unbelievable complexity. To reproduce an instant of it would involve years of labor, that is, if one were able to reproduce colors of equivalent brilliance and intensity.

Here are two ways in which you might come to believe the above about your experience: (1.) You might actually have visual experiences of the sort described, including of colors entirely new and previously unimagined and of a complexity that would require years of labor to describe. Or (2.) you might shortcut all that and simply arrive straightaway at the belief that you are undergoing or have undergone such an experience -- perhaps with the aid of some unusual visual experiences, but not really of the novelty and complexity described. If the former, you have phenomenally hallucinated wholly novel colors. If the latter, you have merely doxastically hallucinated them.

The difference seems important -- crucial even, if we are going to understand the boundaries of experience as revealed by hallucination. And yet phenomenal vs. merely doxastic hallucinations might be hard to distinguish on the basis of verbal report alone, and almost by definition subjects will be apt to confuse the two. I can recall no point in the book where Sacks displays sensitivity to this issue.

Once I was attuned to it, the issue nagged at me again and again in reading:

Time was immensely distended. The elevator descended, "passing a floor every hundred years" (p. 100).

Then my whole life flashed in my mind from birth to the present, with every detail that ever happened, every feeling and thought, visual and emotional was there in an instant (p. 102).

I have had musical hallucinations (when taking chloral hydrate as a sleeping aid) which were continuations of dream music into the waking state -- once with a Mozart quintet. My normal musical memory and imagery is not that strong -- I am quite incapable of hearing every instrument in a quintet, let alone an orchestra -- so the experience of hearing the Mozart, hearing every instrument, was a startling (and beautiful) one (p. 213).

The possibility of merely doxastic hallucination might arise especially acutely when subjects report highly unusual, almost inconceivable, experiences or incredible detail beyond normal perception and imagery; but of course the possibility is present in more mundane hallucination reports too.

(A fan of Dennett might suggest that there's no difference between the phenomenal and doxastic hallucinations; but I don't know what Dennett himself would say -- probably something more complex than that.)

Does the World Look Upside-Down Through Inverting Lenses, Once You're Used to Them?

For eight days in 1897, psychologist George Stratton saw the world through a pair of glasses that rotated everything 180 degrees so that up was down and left was right. At first, everything looked upside down and backwards and he stumbled about, crashing into things. By the eighth day, he had largely adapted to the inverting lenses and was able to skillfully maneuver through his environment. Although Stratton's experiment is often cited by psychologists and philosophers of perception, few have effectively addressed the question that interests me most: After adaptation, did the world still look upside-down to Stratton, or at some point did it flip rightside up?

Since after adaptation the upside-down look might no longer seem wrong, in posing this question perhaps I should avoid the word "upside-down", with its implication of error or illusion. So maybe the stays-upside-down view is better expressed as follows. Before donning the inverting lenses, let's suppose, the visual field is toovy when Stratton looks at his room (where "toovy" picks out the phenomenal quality ordinarily associated with its looking like the light is hanging down from above, etc.) and after donning the lenses for the first time, the visual field turns teavy instead (where "teavy" picks out the phenomenal quality ordinarily associated with its looking like the light is jutting up from below). Here are two possibilities for what happens during adaptation: (1.) At some point (perhaps unsteadily and fragmentedly) Stratton's visual experience flips from teavy back to toovy. (2.) Stratton's visual experience stays teavy, but he learns to associate that teavy quality with the light's actually being above him.

These seem like very different phenomenal possibilities. But which is correct? Susan Hurley and Alva Noe (2003) say that things go back to looking toovy -- that post-adaptation, Stratton's visual pheomenology is just the same as it was before donning the glasses (minus the feeling of wearing glasses and other incidentals like the narrowing of the field). They say this partly based upon, and in defense of, a controversial general theory of the relationship between skillful action and conscious experience, and partly based upon James Taylor's 1962 reports in his replication of Stratton. But Taylor's reports, too, are situated within a theoretical framework to which he appears deeply committed, and Stratton's original reports are most naturally interpreted as suggesting the opposite. For example, Stratton writes:

The harmonization of the new experience and the suppression or subordination of insistent remnants of the old were always apparent during active operations in the visual surroundings, as has been described for several of the preceding days. While I sat passively the old localization of unseen parts of my body often came back, or perhaps was the usual form in which they appeared. But the instant I began to rock my chair the new position of these parts came prominently forward, and, except in the case of my shoulders and back, readily felt more real than the old. And in walking, when hands and feet rhythmically made their appearance in the visual field, the old representation, except perhaps for some faint inharmonious sensations in the back, was fully expelled without employing any device of will or of attention whatever (1897c, p. 469).

Stratton does not say that the new experience is the same as the old, after adaptation: Rather the new experience is the reverse of the old, and adaptation is the victory of the new in the fight between them.

I don't know which way it goes, but the two possibilities seem to differ, and different observers appear to be giving different reports. The question has implications for the general question of whether skillful action shapes our phenomenology to match the world or whether, instead, phenomenology tends to stay (approximately) the same across the development of skill, only gathering new associations and behavioral contingencies.

I discuss this issue with more everyday examples of mirror reflection in earlier blog posts on the apparent location of mirror images and on the U.S. Department of Transportation's advice that "objects in mirror are closer than they appear".

Similar questions arise, of course, for inverted spectrum cases.

To Australia, and Representational Realism

I'm off to Australia with my family for a week (to consider the ANU job offer), then to a conference in St. Louis, so it might be a couple weeks before I'm blogging again.

As a parting thought, it occurred to me to clarify why I'm so interested in mirror images, which I don't think I made explicit in either of my earlier posts on the topic. Such issues might seem merely an amusing irrelevance -- and I do find them (for whatever reason) amusing. But I also have some big game in my (warped, distorted, reflective) sights. If it turns out that the best thing to say about such cases is that (a.) they are veridical, and yet also (b.) our visual experience is different than in the light-straight-to-the-eye case, then I have a hunch that this is a thread I can pull on that might unravel the whole of representational realism -- by which I mean the view that the properties of observed external objects resemble the properties of our experiences of those objects, e.g., that there is something analogously squarish both about an outward square and about our normal visual experience of that square.

No guarantees that's where I arrive after fuller consideration, but that's the snark in the bush.

The Apparent Location of Mirror Images

When I gaze into a mirror, does it look like there's someone a few feet away gazing back at me? (Someone who looks a lot like me, though perhaps a bit older and grumpier.) Or does it look like I'm standing where I in fact am, in the middle of the bathroom? Or does it somehow look both ways? Suppose my son is sneaking up behind me and I see him in the same mirror. Does it look like he is seven feet in front of me, sneaking up behind the dope in the mirror and I only infer that he is actually behind me? Or does he simply look, instead, one foot behind me?

Suppose I'm in a new restaurant and it takes me a moment to notice that one wall is a mirror. Surely, before I notice, the table that I'm looking at in the mirror appears to me to be in a location other than its real location. Right? Now, after I notice that it's a mirror, does the table look to be in a different place than it looked to be a moment ago? I'm inclined to say that in the dominant sense of "apparent location", the apparent location of the table is just the same, but now I'm wise to it and I know its apparent location isn't its real location. On the other hand, though, when I look in the rear-view mirror in my car I want to say that it looks like that Mazda is coming up fast behind me, not that it looks like there is a Mazda up in space somewhere in front of me.

What is the difference between these cases that makes me want to treat them differently? Does it have to do with familiarity and skill? I guess that's what I'm tempted to say. But then it seems to follow that, with enough skill, things will look veridical through all kinds of reflections, refractions, and distortions. Does the oar angling into water really look straight to the skilled punter? With enough skill, could even the image in a carnival mirror look perfectly veridical? Part of me wants to resist at least that last thought, but I'm not sure how to do so and still say all the other things I want to say.

One resolution might be to strip away the specifically sensory, spatial sense of "looks" or "appears", assimilating to cases like "It looks like you're headed for a C" as I gaze over someone's final exam (cases which, in turn, might be assimilated to entirely non-visual cases like "It looks like we might be in Libya a long time" said after hearing a radio broadcast). That maneuver doesn't satisfy me, though: I want, if I can have it, some robustly visuospatial sense in which the Mazda looks like it's behind me and yet the restaurant table looks like it's in front of me.

(See also "Objects in Mirror Are Closer Than They Appear".)

Oliver Sacks: Through One Eye, the World Looks Flat; Through Two, The World Has Depth

In his 2010 book, Oliver Sacks writes:

The only way to actually perceive depth -- to see it rather than judge it -- is with binocular stereoscopy (p. 114).

Thus, he reports on a woman ("Stereo Sue") who gains stereo vision and with it, for the first time in her life, real experience of three-dimensional visual depth. And of his own loss of vision in one eye he writes:

Stereo vision, however, now that I am mostly monocular, is quite compromised -- completely missing in the upper half or two-thirds of my visual field, though partly intact at the bottom, where I retain some peripheral vision. So I see the lower halves of people in stereoscopic depth, while their upper halves are completely flat and two-dimensional (p. 173).

With one eye occluded, I have no sensation of distance or depth whatever (p. 157).

Although Sacks acknowledges the possibility of some individual variation, he also objects to contemporary accounts of depth perception that treat binocularity as only one among several depth cues:

Such [multifactorial] views [of depth perception], while wholly consistent with a behavioral or empirical theory of vision, give no weight to the qualitative and subjective aspects of stereoscopy. Here one needs inside narratives, personal accounts of what it is like to suddenly gain stereo vision after a lifetime of stereo blindness... or to suddenly lose it after a lifetime of seeing in stereo [as in Sacks's own case] (p. 140-1, fn 14).

I wonder, though: Why is permanent loss or gain of stereoscopy necessary to the subjective evaluation? Can't the binocular among us at least temporarily mimic monocular experience simply by closing one eye? Does doing so make the world go from three-dimensional to flat?

It doesn't seem that way to me. Nor to most other people I've interviewed. Maybe depth is a little richer and more striking binocularly than monocularly -- but the difference is nothing so radical as the difference between 2-D and 3-D. Some of my interviewees, though, do characterize the monocular-binocular difference as 2-D vs. 3-D; and so do, for example, Ernst Mach (1886/1959) and Brian O'Shaughnessy (2003). Call those who report radically different monocular vs. binocular experience, like Sacks and O'Shaughnessy, the PuffOuters. Call those who report pretty similar monocular vs. binocular experience, like me (see also Chapter 2 of my recent book), the StaySamers.

Question: Is the difference between PuffOuters and StaySamers a real difference in experience? That is, does the world actually seem to puff out and go flat for PuffOuters, depending on whether it is seen monocularly or binocularly, while it varies little for the StaySamers? Or is the group difference mainly a difference in report only, with everyone having pretty much the same monocular vs. binocular experience -- and some people mistaken about or misdescribing their experience (in line with some of my conjectures in Chapter 2)?

We might start to address this question empirically as follows.

Sacks mentions that people differ substantially in how they experience three-dimensionality when presented with stereoscopic pictures -- that is, when presented with slightly offset pictures, one to each eye, producing a three-dimensional effect (like in contemporary 3D movies). For example, when presented with a stereoscopic picture of an impossible M.C. Escher tuning fork, Sacks reports one person saying it looked like the top prong rose about 3-4 centimeters from the plane of the page, while two others reported seeing the prong as 12 cm above the plane and Sacks himself saw it as 5 cm higher still (p. 134-5).

Conjecture: If the PuffOuter-StaySamer difference is a real difference in experience, the PuffOuters should, as a group, report much larger effects of that sort than the StaySamers. That could be tested empirically. Presumably PuffOuters should also better detect subtle stereoscopic effects -- which is even more appealing experimentally because it would permit catch trials to corroborate any differences in subjective report.

Yes, this is totally do-able.

"Objects in Mirror Are Closer Than They Appear"

... so it says, at least, on my passenger side mirror.


(image from http://amchurchadultdiscipleship.net)

I've been worrying though, are they closer than they appear?  This might seem a strange thing to worry about, but I refuse to be thus consoled.

Here's a case for saying that objects in the mirror are closer than they appear: The mirror is slightly convex so as to give the driver a wider field of view.  As a result, the expanse of mirror reflecting light from the object into my eye is smaller than the expanse would be if the mirror were flat.  Thus, the size of the object "in the mirror" is smaller than it would be in a flat mirror.  If we assume that flat mirrors accurately convey size, it seems to follow that the size of the object in the mirror is inaccurately small.  Finally, apparent distance in a mirror is determined by apparent size in a mirror, smaller being farther away.

The argument for the other side is, at first blush, much simpler: Objects in the mirror are no closer than they appear, at least for me, because as an experienced driver I never misjudge, or am even tempted to misjudge, their distance.

Now both of these arguments are gappy and problematic.  For example, on the first argument: Why should flat mirrors be normative of apparent size?  And why shouldn't we say that the object is larger than it appears (but appearing the right distance away), rather than closer than it appears (but perhaps appearing the right size)? That is, why does it look like a distant, full-sized car rather than a nearby, smallish car?

You might be tempted to mount a simpler argument for the "closer than they appear" claim: A naive mirror-user will misjudge the distance of objects seen in a slightly convex mirror.  The naive mirror-user's misjudgments are diagnostic of apparent size -- perhaps they are based primarily on "appearances"? -- and this apparent size does not change with experience.  The experienced mirror-user, in contrast, makes no mistakes because she learns to compensate for apparent size.  But this argument is based on the dubious claim that the experience of a novice perceiver is qualitatively the same as the experience of an expert perceiver -- a claim almost universally rejected by contemporary philosophers and psychologists.  It's also unclear whether the naive mirror-viewer would make the mistake if warned that the mirror is convex.  (Can apparent size in a mirror be contingent upon verbally acquired knowledge of whether the mirror is slightly convex or concave?)

Should we, then, repudiate the manufacturers' claim, at least as it applies to experienced drivers?  Should we, perhaps, recommend that General Motors hire some better phenomenologists?  Well, maybe.  But consider carnival mirrors: My image in a carnival mirror looks stretched out, or compressed, or whatever, even if I am not for a moment deceived.  Likewise, the lines in the Poggendorff Illusion look misaligned, even if I have seen the illusion a thousand times and know exactly what is going on.  Things look rippled through a warped window, no matter how often I look through that window.  Perhaps you, too, want to say such things about your experience.  If so, how is the passenger-side mirror case different?

Here is one way it might be different: It takes a certain amount of intellectual stepping back to not be taken in by the carnival mirror or the Poggendorff Illusion or the warped window.  The visual system, considered as a subset of my whole cognitive system, is still fooled.  And maybe this isn't so for the passenger-side mirror case.  But why not?  And does it really take such intellectual stepping back not to be fooled in the other cases?  Perhaps there's a glass of water on my table and the table looks warped through it.  I'm not paying any particular attention to it.  Is my visual system taken in?  Am I stepping back from that experience somehow?  It's not like I just ignore visual input from that area: If the table were to turn bright green in that spot or wiggle strangely, I would presumably notice.  Is my father's visual system fooled by the discontinuity between the two parts of his bifocals?  Is mine fooled by the discontinuities at the edge of my rather strong monofocals as they perch at the end of my nose?  And what if, as Dan Dennett and Mel Goodale others have suggested, there are multiple and possibly conflicting outputs from the visual system, some fooled and some not?

Can we say both that objects are farther than they appear in passenger-side mirror (in one sense) and that they aren't (in some other sense)?  I'm inclined to think that such a "dual aspect" view in this case only doubles our problems, for it's not at all clear what these two senses would be: They can't be the same two senses, it seems, in which a tilted penny is sometimes said to look in one way round and in another way elliptical -- for what would we then say about the tilted penny viewed in a convex mirror?  We would seem to need three answers.

Hey, wait, don't drive off now -- we've only started!

How Big the Moon Is, According to One Three-Year-Old

A conversation I had with last night my daughter Kate, three years and seven months old:

Me: Which is bigger, the moon or the house?

Kate: The house.

Me: Which is bigger, the moon or a tree?

Kate: A tree.

Me: Which is bigger, the moon or a quarter?

Kate: No.

Me: No? What?

Kate: They're little.

Me: Which is smaller, the moon or a quarter?

Kate: A quarter.

Me: Which is smaller, the moon or a peanut butter jar?

Kate: The moon.

Me: So the moon is between a quarter and a peanut butter jar?

Kate: That's right, Daddy, you got it!

(See also my posts Development of the Moon Illusion? and How Far Away Is the Television Screen of Visual Experience?)