Monday, May 24, 2010

Qualia Inversion: Sound and Color (A Contest with a "Valuable Prize"!)

If you're the kind of person who reads philosophy blogs, you've probably heard of inverted qualia thought experiments. The most famous example is red-green inversion: The red-green invert is someone who has reddish color experiences when she looks at green things (grass, leaves) and has greenish color experience when she looks at red things (blood, ripe tomatoes). Since the invert, like the rest of us, learns the meaning of color terms by example, her language and behavior is entirely, or at least virtually, indistinguishable from anyone else's. She uses the English word "red" to refer to the color of blood and "green" to refer to the color of grass, despite the difference in her color experiences of those things.

In a talk at UC Riverside a couple of weeks ago, Saul Kripke asserted that no philosopher had ever suggested the possibility of sound-color qualia inversion -- that is, the possibility of a person who experiences sound qualia when stimulated by light and color qualia when stimulated by sonic vibrations. Let's be clear that Kripke was not denying the possibility of synaesthesia. He was not denying that people sometimes (for example) experience colors alongside sounds when stimulated by sounds. Kripke's claim, rather, was that no philosopher had contemplated a true sound-color qualia invert -- someone who normally experiences sound rather than color when stimulated by light and color rather than sound when stimulated by sonic vibration and whose language and behavior from the outside is indistinguishable from that of non-inverts.

Kripke said this twice. I told him I was pretty sure he was wrong and that I had read such a discussion. Kripke challenged me to send him the citation. After a little research, I turned up my source: an unpublished essay by one of my graduate students, Nathan Westbrook. When I asked Nathan whether he knew of any precedents, he said he didn't. I also tried asking someone who had recently published a review of the qualia inversion literature; he too said he didn't know of anyone who had advanced that type of example.

But given the huge number of philosophy articles published each year and the prominence of qualia inversion examples, I feel sure someone must have discussed this kind of case somewhere. Therefore, I offer a challenge: Find a published discussion of sound-color qualia inversion. The winner will receive a "valuable prize" -- hm, what can I offer? How about: a drink of your choice (coffee, beer, whatever) on me, next time we are in the same city. (Okay, maybe that's no so valuable.)

Rules:

* The contest is open until June 14th or until someone delivers a satisfactory example, whichever comes later.

* To be satisfactory, the discussion must be published in a reputable philosophy journal or press.

* To be satisfactory, the discussion need not ultimately endorse the possibility of color-sound qualia inversion, just take it seriously.

* If more than one satisfactory example is submitted, the person who submits the best example will be declared the winner, where I will judge "best" impressionistically, criteria including but not limited to: the length and quality of the discussion, the prominence of the writer or venue, and how seriously the possibility is taken.

* If more than one person submits the best example, the first person to submit the best example will be declared the winner.

* Submit your example as a comment on this post or by email to me.
For the record, I lean toward thinking that sound-color qualia inversion is possible in a conceptual/metaphysical/pulling-it-out-of-my-a-priori-hindquarters-for-the-little-that's-worth sense of possibility.

Color experiences famously differ along three dimensions: hue, saturation, and lightness or brightness. They also differ in egocentric, subjective location. To work an inversion with sound, we need a one-to-one mapping of dimensions of variation. Subjective location would appear to be easy, since both colors and sounds have subjective location. Since brightness and saturation are both unidimensional, we might be able to map them one-to-one onto pitch and volume, which are also unidimensional. Hue varies in a bit more complex a way, with red-green as opposites and blue-yellow as opposites, but perhaps patterns in the overtone series could be used. Probably the hue-overtone series mapping would require some tweaking to work, but presumably human-like beings with a slightly different set of visual and auditory capacities could exhibit a clean mapping (e.g., if the beings were only capable of discriminating certain patterns in overtone variation). Another complication is our much higher sensitivity to variation in visual as opposed to auditory position. But I don't see why, for the purposes of the thought experiment, we shouldn't be liberal about such matters: The sound-color invert, for example, might be an invert only of a relatively poor-sighted person and/or might have an exquisite appreciation of subtle variations in the position of sound sources subjective location of sound qualia.

25 comments:

Jeremy Goodman said...
This comment has been removed by the author.
Jeremy Goodman said...

Adam Wager. 1999. "The extra qualia problem: synaesthesia and representationism" Philosophical Psychology


"the problem of cross-mo- dal qualia is one of intermodal inversion. The cases I have in mind would be ones in which a subject perceives an object through sensory modality M1 but has qualia of modality M2 in place of qualia from M1 (note that I make no claims here about there being actual cases of this sort in human subjects) [9]. One way to imagine such a case would be to consider a synaesthetic subject (a case of extra qualia) and imagine his brain altered in such a way as to cause him to lose the qualia of modality M1 (e.g. in a colored-hearing synaesthete, damage to the auditory cortex might be able to produce such a result). One also might imagine cases where hearing produces visual qualia and vice versa, perhaps by equipping the subject with auditory± visual and visual± auditory transducers in place of the inverting lenses often used in inverted spectrum cases [10]."

Jeremy Goodman said...
This comment has been removed by the author.
Jeremy Goodman said...

Also, see the penultimate paragraph of David Cole's entry on the inverted spectrum.

http://books.google.com/books?id=6M3NSNm6MlkC&lpg=PA509&ots=2qLwKrl3Dk&dq=auditory%20visual%20inversion%20qualia&pg=PA507#v=onepage&q=auditory%20visual%20inversion%20qualia&f=false

Jeremy Goodman said...

Re: Mapping colors onto tones.

Colors vary in two linear dimensions (brightness, saturation) and one circular dimension (hue). Although not recognized by folk psychology, sounds also vary along two linear dimensions (loudness, pitch-height) and one circular dimension (chroma; i.e. A-#G). If this is right, then it should be pretty straightforward to map colors onto tones (although it would leave out timbre, and saturation is arguably bounded in a way that pitch-height isn't.)

[Folk psychology lumps chroma and pitch-height together into one variable we call pitch, but Shepard tones and related phenomena arguably show that a tone's chroma can change without its pitch-height changing. At least, many perceptual psychologists have drawn this conclusion.]

Larry Rosenblum said...

Maybe search for sound-induced 'photisms'. Also, there may be instances in which synesthetes only experience the induced sensation, not the stimulus, if the latter is presented subliminally. Finally, might be worth looking at the auditory hallucinations literature to see if the experiences are ever visually-induced.

Ayoob said...

I would have searched if the prize had had its worth. a long journey from Iran to U.S just for a cup of tea or coffee?
Anyway, I've studied on the issue for a long time but I've not come across such an inversion. Maybe you graduate student is the first one.

Fredrik Stjernberg said...

there was a paper in BBS perhaps ten years ago, on possible colour qualia inversions beyond red-green. The paper ran through all the obvious alternatives (blue-yellow etc.), only to discard most of them for structural reasons - such alternative inversions would be detectable by external behaviour. For example, blue is typically darker than yellow, so a blue-yellow invert would be detectable. I think that paper also discussed some possible other inversions, and sound could be one of them.

Fredrik Stjernberg said...

I found the BBS paper:
BEHAVIORAL AND BRAIN SCIENCES (1999) 22, 923–989
Color, consciousness, and the isomorphism constraint
Stephen E. Palmer

there are a few mentions of sounds scattered through the paper and commenting notes.

Jonathan Jenkins Ichikawa said...

Is it permissible to publish a discussion oneself, then cite it for the purpose of the contest?

Eric Schwitzgebel said...

Thanks for all the comments, folks!

Jeremy: Thanks for both the citations and the thoughts! I haven't had a chance to chase down the citations yet, but the quote from Wager is certainly satisfactory, so the contest will end June 14th, with your citations as the bar to beat.

Also, your thoughts about chroma and pitch-height are neat. I guess the worry is that even chroma and pitch-height can be distinguished, they are a bit too entangled with each other. The other worry, which you mention, is now it looks like there might be too many dimensions to sound experience for the necessary one-to-one mapping.

Eric Schwitzgebel said...

Larry and Fredrik: Thanks for those tips. I will check them out. I read the Palmer article long ago; maybe that was one of things in the back of my mind.

Jonathan: Sure thing! But can you have it out before June 14? :)

IanOlasov said...

I'm not sure if this counts, but there's a relevant discussion of the relation between vision and hearing in Strawson's (1959) Individuals:

"[R]elations between elements in respect of the spatial dimension are presented simultaneously, all at once... But relations between elements in respect of the auditory analogue of the spatial dimension cannot be presented simultaneously, all at once. They turn essentially on change. Roughly, two visual elements can be seen all at once as at a certain visual distance from one another; whereas two auditory elements cannot be heard all at once as at a certain auditory distance from one another. Or, to put it another way,: the momentary states of the colour-patches of the visual scene visibly exhibit spatial relations to each other at a moment; whereas the momentary states of the sound-patches of the auditory scene do not audibly exhibit the auditory analogue of spatial relations to each other at a moment. Not in their momentary states, but only in swelling or fading... could the particular sounds exhibit such relations." (72-3)

Norm Nason said...

Eric, Douglas Hofstadter has a discussion about this in his book, "I Am a Strange Loop," chapter 23. For instance he says:

"What is sacrosanct about the idea of shuffling colors inside a spectrum? Why not shuffle all sorts of experiences arbitrarily? Maybe your private inner experience of redness is the same as my private inner experience of hearing very low notes on a piano."

I must not take credit for finding this, however. It was first noted by one of the readers of your post on machineslikeus.com.

Adam Wager said...

Any chance you want to add a prize for the author of the winning citation? A cup of coffee if I'm ever out your way would at least be some consolation for the crushing news that Kripke never read my paper.

Jeremy Goodman said...

Thanks Eric! Chroma and pitch-height are certainly entangled in normal experience, but I'm not sure there are any necessary connections between them. Shepard tones are pretty suggestive of their dissociability, I think. Hue and saturation are also entangled in normal experience, but come apart in special cases (e.g., supersaturated red).

I don't see why the existence of timbreal residue (or, for that matter, uniqueness residue left over on the color side; i.e. the uniqueness of unique green) is an objection to the possibility of the kind of inversion I described. Why can't cross-modal inversions be partial? Why can't there be colors that sound like violins? Why couldn't there be unique D-flat?

Ian: I think Strawson is wrong about this. We do hear sounds as being higher and lower than each other. See discussion of pitch relations in 3.2.5 of http://plato.stanford.edu/entries/perception-auditory/

But I do think Strawson's discussion suggests a much more counterintuitive sort of visual/auditory inversion: an inversion with space rather than with color. That is, rather than a brightness-saturation-chroma/loudness-height-chroma inversion, what about a z-r-theta/loudness-height-chroma inversion, where z, r, and theta are cylindrical coordinates of the egocentric positions of objects in our environment? Could hearing a loud, low C feel like seeing a certain timbre instantiated above me, far away, and to my right? Could looking at something red in front of my face me feel like hearing a red, medium-volume, low A? As Hofstadter wonders, "Why not shuffle all sorts of experiences arbitrarily?"

Jeremy Goodman said...

It's probably more intuitive to invert r with loudness and z with height, in which case: Could hearing a loud, low C feel like seeing its timbre instantiated near my right foot? Could looking at something red at arm's length and eye-height in front of me feel like hearing a red, medium-volume, middle-height A?

Eric Schwitzgebel said...

Thanks for the continuing comments, folks!

Ian: Thanks for the reminder about Strawson. His discussion is certainly relevant, though not quite satisfactory for the prize.

Norman: Thanks for that suggestion (and for the plug on Machines Like Us).

Adam: Sure thing! I suspect there is a large number of papers Kripke has never read, so I wouldn't take it too hard. ;) (I say this without meaning to take a stand on whether the number of papers Kripke hasn't read exceeds or does not exceed the number of papers any other eminent philosopher hasn't read.)

Thanks, Jeremy, for your very well informed comments! Are you in contact with Chris Hill at Brown? I'm inclined to agree with your in-principle dissociable point. However, I still think overtones potentially create trouble. The invert needs something in her visual experience to correspond to the overtonish aspect of the non-invert's auditory experience, otherwise she either won't be behaviorally indistinguishable or there will be an implausible dissociation between experience and behavior. Now maybe we can imagine the invert being an invert of someone with normal auditory experience *except* for an insensitivity to overtones -- but I think that's probably bending the rules of the game a bit too far.

Jeremy Goodman said...

Eric: If the rules of the game are that there has to be a total sound-color inversion, then timbreal residue is a problem. But I still think it's fruitful to consider the possibility of partial sound-color inversions.

Visual texture might be a good candidate for inversion with timbre though. Just as timbre is a presumably a function of the amplitudes of different frequency components of the time-series of sound pressure levels, visual texture is plausibly a function of the magnitudes of different spatial-frequency components of a surface's depth profile.

The first philosophy class I took was Chris' class on consciousness. We were actually just talking the other week about your and Hurlburt's stuff on introspection.

praymont said...

This is a 'close but no cigar' case, but Dominic Lopes has a possibly relevant critique of representationalism in the March, 2000 issue of Philosophy & Phen Research called "What is it like to see with your ears?"

Lopes sets aside colour inversion objections as being too speculative and then introduces some data concerning actual cases of echolocation in humans. He cites several neat-sounding(-looking?) papers about human echolocation from the 1940's to the 1970's.

He doesn't ask us to imagine an inversion of shape-seeing and shape-hearing qualia, but he does say things like, "[W]hat it is like to echo-discriminate the shape of that post box is not what it is like to (dimly) see the shape of that post box" (p. 450), even though the very same feature (allegedly) is represented by both experiences.

Another close-but-no-cigar paper that may yet be of interest to readers of this post is Neil Campbell's 2004 paper in Erkenntnis ('Generalizing Qualia Inversion', pp. 27-34). Campbell considers the possibility of qualia inversions within (but not between) different perceptual modalities.

praymont said...

JG mentioned an entry by David Cole. Here's an earlier paper by Cole (from 2000), but I think it's only on-line and not published:

http://www.d.umn.edu/~dcole/inverted_spectrum.htm

See esp. the section called 'Radical Synesthesia and Transesthesia', where Cole says, "Imagine a modality transposing radical synesthete, with all qualia swapped between two modalities, a condition we might call "transesthesia". The vision-audition transesthete will 'see" sounds and "hear" sights, that is, experience (only) visual qualia when hearing, and experience (only) auditory qualia when seeing."

Dan said...

It's not exactly what you're looking for, but here's a story about a (very partial) smell-sight inversion, in fruitflies.

Eric Schwitzgebel said...

Thanks Praymont and Dan!

praymont said...

I know the contest is over, but this is interesting: Richard Dawkins seems to affirm the possibility of such drastic inversions.

He says: "A bat, when echolocating an insect, might use the subjective sensation that we call ‘red’ as a convenient label for the furry texture of a moth, and might use ‘blue’ as an internal label for the leathery texture of a locust. These qualia are just conveniences, to be pressed into service in the way that is most useful for the species concerned. Since the mammalian brain has the capacity to construct the qualia that we call hues, and use them as internal labels to facilitate sensory distinctions, why wouldn’t bats, as fully paid-up mammals, press into sonar service the labels that we call red and blue? By the same token, I went on, perhaps rhinoceroses smell in colour."

Dawkins connects this with views for which he has "long argued". I wonder if he's entertained this possibility before in print.

Here's the link for Dawkins' post: http://richarddawkins.net/articles/479563-sky-blue-pink-a-colour-never-before-seen

Eric Schwitzgebel said...

Praymont: Nice quote, thanks!