Monday, January 14, 2008

Peripheral Vision: Mapping Color onto Shape

It's well established in vision science that perception of shape and color declines precipitously outside the central one or two degrees of visual arc (about the size of your thumbnail held at arm's length). Most people find it surprising, actually, how poor their peripheral vision is, and how narrow the region of clarity (despite the number of hours each of us has logged expertly seeing outward things). Dennett's playing card experiment is a nice way to test yourself: Take a playing card from a normal deck, without looking at it. Hold it at arm's length to one side, just beyond the field of view. Keeping your eyes fixed on a single point ahead of you, slowly rotate the card toward the center, noting when you can discern its color, its suit, and its value. It won't be until almost dead center that you'll be able to tell if it's a Jack or a King.

But if shape and color perception are bad individually in the periphery, Peter Neri and Dennis M. Levi (2006) suggest that things are even worse when you combine them.

Suppose you are presented a red square and a blue circle. Regions of your brain specializing in color will register a red thing and a blue thing. Other regions specializing in shape will register a square and a circle. If everything works right, you'll also know that it's the square that's red and the circle that's blue -- but that is a bit of extra work, an additional thing that must go right. In some situations, people will get the colors and shapes right, but they won't know which color went with which shape (Triesman and Schmidt 1982).

Neri and Levi's experiments suggest that this sort of "feature binding" goes especially badly in the periphery. Even at resolutions where their subjects could make out color and shape individually, they could not accurately put those colors and shapes together.

Now I have some picky complaints about the methodology of their experiment -- having to do with Gestalt principles for shape detection and possible single-feature computing shortcuts (if you go to their article, note the "7"-like figure in the left hand image in Fig. 1C which could not occur in a non-target image; HT: Ryan Robart) -- but there's also the phenomenological question. Peripheral vision seems blurry, and the color of unknown objects can be surprisingly indistinct or inaccurate, but does it additionally seem on introspection that the colors out there get mapped onto the wrong objects, or that the colors and shapes don't coherently fit together?

Billock and Tsou (2003) describe the phenomenology of binding failure thus:

In other cases, all sense of object and surface can be lost and the target is perceived as a ‘jumble of lines’ or ‘extremely confusing and hard to describe’. Moreover, the contrast of equiluminant images can seem unstable – Gregory describes such images as ‘jazzy’. In 1927, Liebmann reported that there is a ‘critical zone [where] everything flows…glimmers…most everything is soft, jelly-like, colloidal. Often…parts which belong together in the normal figure now have nothing to do with one another. [It is] a world without firm things, without solidity.’
Is that how peripheral vision seems? I've been walking around today trying to notice, and it just doesn't strike me that way. Indistinct, yes. But binding failure is different.

Furthermore, I've finally found a use for all those silly business cards they gave me when I was promoted to Associate Professor. On the backs, I've written letters and shapes right next to each other in arbitrary colors. I made about 80 cards, each holding two of eight shapes in two of four possible colors. Rotating them in from the periphery, I don't find myself making many binding errors. As soon as the cards are clear enough to distinguish shape and color, I know which shape goes with which color.

9 comments:

Christine said...

Another great post!

Yeah, the idea that the brain would somehow separate the shape and color of an object and confuse either one of those properties with those belonging to another, physically distinct, object seems really strange to me. While perception is not an exact reflection of physical reality, there's still a pretty tight relationship between the two, and it seems odd that the brain would assign some property to an object based on something else it's "seeing" at the same time.

Eric Schwitzgebel said...

Thanks for the comment, Christine!

Josh Weisberg said...

Eric-

Thanks for linking to the cool articles, and for the interesting post! Not sure if I really got what the tasks were in the articles, but here are some thoughts:

I feel like some things are being mixed up here. The phenomenon described in Neri and Levi seems fully compatible with what I think I see peripherally. On the sorts of stimuli they present, I find it difficult to tell what's what on the test cases. Kind of like peripheral vision. So if they're correct that that's binding failure, it does seem like what goes in in the Dennett-type cases, at least.

But the odd phenomena that Billock and Tsou describe may not be the same sorts of binding failures. They were using "retinally stabilized" images to get their effects, and that may not be the sort of thing that Neri and Levi had in mind. (Though they do sound very cool, and I would love to experience them...) So, the class of feature binding errors may include a range of cases, but only the more mellow one noted in Neri and Levi correspond to what's going on in peripheral vision.

Also, I was thinking about Treisman's illusory conjunctions. I can't recall that subjects themselves realized they had made the errors. Rather, they reported what they thought they saw, and the researchers could tell if they described stimuli that were not present. So, maybe you can't tell phenomenologically when feature binding gets mixed up in peripheral vision either.

Another worry: maybe even the minimal attention you're paying to your peripheral vision acts to bind features. Or maybe your attempt to categorize things and report them distorts how things seem. That is, prior to the attempt, things are not bound, but top-down influences from the attempt to categorize act to bind, or at least to make it seem to you that things are bound, even if they are not.

Cheers.

Eric Schwitzgebel said...

Hi Josh -- Thanks for that very thoughtful comment! I agree that the inferences are tenuous at best. Yes (for example), as you say, the act of attending may aid the binding process.

I'm not sure that I agree with your reasoning about the illusory conjunctions, though. True, Triesman's subjects (probably) didn't notice them -- but if I were subject to lots of illusory conjunctions in the periphery (including when I attend to the periphery), then I should repair those conjunctions when I foveate in that direction, so my opinion about the colors, for example, of peripherally attended objects of unknown color should not only change (as it does) but nearby objects should seem sometimes to swap colors. I don't think I have that experience very often -- not even when I tried to induce it by writing on the backs of business cards as described in the post.

Josh Weisberg said...

Eric--

I wonder if change blindness might be an issue here. Given the difficulty of making out just what we're seeing in the periphery (even when we're attending to it) maybe we don't notice the changes. There are striking color changes that can be missed in even foveal vision.

And, as you note, there's still the worry about attention working to bind what we experience in the periphery. We may not notice a change between pre-attended and post-attended peripherally perceived objects, even if post-attended and foveal perceptions do not appear to change (or don't change).

(I'm not nearly as practiced as you at trying to pick out my peripheral perceptions, though I did find Dennett's card trick quite surprising. Also, is there a worry that this sort of peripheral vision is particularly theory-laden? I'm reminded of a nice Dennett article called "Wondering Where the Yellow Went" wherein Dennett considers looking at Jerry Fodor's yellow rain slicker in the pre-dawn light on his sailboat--does it look yellow, or does he simply know that it's yellow, even though it looks gray?)

Anonymous said...

one interesting thing I found out in an eye test is that my peripheral vision in my weakest eye (there is only peripheral vision) is able to read eye tests as if I had 20/20 vision much better than my strong eye- but a much greater strain.

GNZ

Eric Schwitzgebel said...

Yes, Josh, change blindness could easily imply that we wouldn't notice the color swapping of incorrectly bound peripheral objects -- so that could explain the daily impression that objects in the periphery don't swap colors when we turn to focus on them.

I did try to avoid that concern -- and also the theory-ladenness worry -- in my business-cards self-test, by announcing out loud during the test which color I thought went with which shape. But of course that's just one very informal test done under poorly controlled conditions!

I do suspect the point you make about the yellow slicker and theory-ladenness is very much in play in peripheral perceptions of color. It's really hard to shake the impression that I see fairly specific colors deep into the periphery, though I know that our color perception deep in the periphery is quite weak. Shape, somehow, seems vaguer -- less "filled in" with what I know.

Or is it just that the idea of a vague, indeterminate shape is more familiar to me and easier to recognize than the idea of a vague, indeterminate color?

Eric Schwitzgebel said...

Funny, GNZ, I hadn't heard of that. Peripheral vision sometimes exceeds foveal vision in detection of motion and faint lights. My impression was the shape detection is generally far worse (even if one strains); but there may well be eye conditions in which that is reversed.

There's so much to learn about vision. What an intimidating literature!

Genius said...

I think that my main eye might be worse due to too much computer use (or something like that) but still it was a surprise.

Maybe the cells good at shape detection had to grow somewhere...