Wednesday, September 30, 2009

Visual Imagery of Sensory Impossibilities

There was a time when I could visualize the obverse, and then the reverse. Now I see them simultaneously. This is not as though the Zahir were crystal, because it is not a matter of one face being superimposed upon another; rather, it is as though my eyesight were spherical, with the Zahir in the center.

– Jorge Luis Borges (1949/1962)

My mental field of vision is larger than the normal one. In the former I appear to see everything from some commanding point of view, which at once embraces every object and all sides of every object.

– a questionnaire respondent in Galton (1880)

In conversation, a couple people have told me that they can visually imagine four spatial dimensions -- a hypercube, for example. (Probably someone has said this in print, too, but I can't recall any instances. Pointers welcome.)

I find this rather hard to imagine, and (I confess) believe. But maybe that's just my own narrowness? If there are or could be four actual, physical spatial dimensions, then presumably there are or could be aliens who could see (and thus presumably visually imagine) in all those dimensions. The unpicturability of such imagery by me doesn't imply its impossibility. Maybe the dimensionality of some people's visual imagining outruns the dimensionality of their vision. (I assume here that no human being can actually outwardly see in four dimensions or with such apparently impossible points of view. At least I've heard no such reports.)

Such impossibilities of visual perspective, if they are possible in visual imagination, appear to strain against two common claims about imagery. One is Hume's "faint perceptions" view, according to which images are just faint copies of ordinary sensory impressions. (One needn't of course be a fully orthodox Humean on this point -- even Hume probably wasn't -- to think that visual imagery is very structurally similar, experientially, to visual sensation.) Another is Stephen Kosslyn's view, related to Hume's, that imagery differs from perception mainly in being the activation of visual brain areas and visual representational forms by central cognitive causes rather than from peripheral sensory stimulation. The visual image, Kosslyn says, is a "simulation" or "emulation" of what we visually perceive. Though not perhaps impossible, it would be odd, on such a view, if the imagistic simulation could have radically different perspectival and spatial properties from the perceptions simulated.

So probably: Either visual imagery of such sensory impossibilities is itself impossible, despite some people's reports, or Hume and Kosslyn (and lots of other people) are wrong. Is there a way to go about settling this?


Michael Metzler said...


After some thought, I am inclined to agree with your conclusion. But I still wonder - in view of moving towards a way of settling this - if the imagination described in the 1962 and 1880 segments should be fully constrained by the idea of emulation. Simulation is imagination that must mimic perception, and emulation, a narrower concept than simulation, requires that the *processes* of a perceived event be mimicked. Kosslyn wants to equate this second order simulation with all mental imagery (S.T. Moulton & S.M. Kosslyn, 2009 . . . .thanks for the link!)

I am not sure that the mind's creative powers should be limited in this way. It seems we typically take for granted the complex creativity of simple perception and the simulation that mimics it. Moulton and Kosslyn note, for example, that even simple tasks of visual imagery - involving the same brain functions as the perception it mimics - often rely on motor representation and "is affected by body position." Similarly, if I must 'perceive' time - an important element of a simulation's 'content' it would seem - in terms of embodied space (Lakoff & Johnson), then perhaps creative binding, mapping, and blending (Lakoff, Johnson, Fouconnier, & Turner) could create effects not describable in terms of prototypically literal terms.

Thus, a more objective point of imaginative view might be "as though" our eyesight were spherical (1962), or "appear" as if we see all objects with a View From NoWhere (1880). Perhaps this is in part a function of compressing sensory-motor experience with - for example - objects on the beach off of 101 (walking on the beach, swimming with the waves, picking up seaweed, observing acts of surfing through emulation) and mapping this to more . . . er, now I have to pull out my old file box. Ah ha!: map this to a "survey perspective", a "stationary viewpoint above the environment" that is "perspective free" (Tversky, 'Narratives of Space, Time, and Life', Mind & Language, 2004).

Autumnal Harvest said...

Despite the fact that we live in a three-dimensional world, we only get a two-dimensional projection of it on the back of our eye. We demonstrate that we have an understanding of the three-dimensional world when we show that we can reconstruct it from the two-dimensional projections, and navigate it with relative ease. e.g. walk easily around obstacles, identify a box as the same objects from two different viewpoints, despite the different cross-sections generated, etc. . .

So if you want to settle the dispute, write a computer program which populates a four-dimensional world with various objects, simulates photons radiating from those objects, and then calculates the cross-section of photon lines onto a two-dimensional plane (simulating your eye). Let your claimiants play a video game where they can move around this world and manipulate objects, and see how well they do. A simple task would be to give them objects A, B, and C, let them rotate each for short amounts of time, and see if they can figure out which two are the same shape.

Not sure if you were wondering about the theoretical possibility of setting the dispute, or if you wanted a practical solution. This is, admittedly, quite a lot of work.

dioscuri said...

Very interesting topic. I was once pondering this question so asked two Phd mathematician friends of mine, both of whom familiar with advanced geometry. One was adamant that he could visualise a fourth spatial dimensional, while the other, equally vehemently, insisted that he couldn't. I probably shouldn't have been surprised - this seems to chime with a lot of the stuff you've done on imagery and introspection - responses variable and difficult to assess, etc....

Unknown said...

Autumnal Harvest: "Despite the fact that we live in a three-dimensional world, we only get a two-dimensional projection of it on the back of our eye"

Yes but due to stereoscopic vision we have two such projections, and our brains combine the images such that we do have actual three-dimensional perception. And one can easily see the possibility of, if not directly imagine, more commanding or complete fields of view, without having to consider the possibility of four-dimensional perception. Birds, for example, have a near 360 degree field of vision, and I feel I can to some extent imagine what that would be like - instead the single forward facing viewport that we have, it would be an all-round window. And I do believe that the kind of experience described in the first two quotes is possible - simultaneous visual experience of the front and back of an object, even of the inside and outside. It just needs appropriately positioned eyes, or cameras, and the right kind of neural wiring.

Eric Schwitzgebel said...

Thanks for the comments, folks!

Michael: I'm inclined to think you're right about the complexity of this. It's not impossible that imagery, even if implemented by ordinary perceptual representational devices, is integrated temporally or cross-modally in a way that ordinary, real-time perception is not. (Or seems not to be?)

Cool suggestion, Autumnal! One complication is that even for those who claim ordinary three-dimensional imagery, the relationship is quite unsteady between self-declared imagery skill or use and actual performance on shape-transformation or shape-matching tasks.

Eric Schwitzgebel said...

Ledge: Yeah, if only I could pop one eye out of this limited three-dimensional space I seem to be stuck in! (I picture the guy in Flatworld being lifted off the plane.) I'm not sure how my brain would integrate the data from that eye with the data coming in the other eye, but maybe over time I could start to make sense of it....

Michael Metzler said...

Hmmm: seems not to be? I now wonder if Kosslyn's constraint is more accommodating than I first thought. Three problems I find when exploring the content of consciousness: 1) the sophistication of the unconscious mind 2) the indeterminate fluctuating line between what I consider the kinds of items above or below the line of consciousness, and 3) the dependence of one item of conscious experience on the whole (irreducible unity). I recall my intense gaze on Saturday, when I was tempting a seagull to get 18 inches away from me with some treats. The social context was implicit and certainly not fully unconscious. And I was not just observing a seagull, but seeing a seagull as a seagull (wow, how do we do that?). Seeing the seagull as a seagull is largely an unconscious phenomena, yet, certainly, also part of the conscious experience itself in some sense. I must have had some kind of conscious employment of sensory-motor simulation, since I could perceive the softness of the feathers around the entire seagull-object. I watched the seagull turn its head back and forth, apparently more concerned about what was going on all around more than with me. I could therefore in some sense consciously grasp the seagull's motions through my own embodied emulation mechanisms. Further, this perception must have involved a conscious aspect of the mapping of this embodied emulation to a "survey" view, since part of the emulation was an immediate awareness of seagull's 360 degree tracking of the environment.

Unknown said...

This might be relevant:
(See the 11 minute video.)

Kapitano said...

We've all seen that shot in The Matrix where a character (Trinity) jumps up high in the air...then time stops and it's like the camera pans all around her. We've also seen plenty of immitations or spoofs of that architypal shot.

So now it's not difficult to imagine (say) a falling drop of water, arrested in time while the viewpoint flies around it - or any number of similar scenes.

Slow motion photography is even more familiar, so it's not hard to visualise the drop falling at 1/100th speed, while the viewpoint circles.

But could we imagine an image frozen in time with circling viewpoint before we'd seen it multiple times on a screen? I don't think so. And 100 years ago, people couldn't imagine an event in slow-mo. The same screen wipes, dissolves and other common film effects.

So what I'm saying is: If we've seen a visual effect created artificially a few times, we can generalise it to other imagined scenes. And if a multi-viewpoint view can be created with camera and editing trickery - we should be able to imagine with multiple viewpoints.

All of which has implications for how we do philosophy. Philosophy relies a great deal on metaphors taken from life, and the palette televisual special effects adds to our stock of available metaphors.

Joel Marsh said...

Personally I am not a fan of trying to explain a hypothetical, imaginative "visualization" of impossible circumstances, using anything relating to physics, mathematics, etc. Like much philosophy-of-science, it seems closer to "deciding" what is believable rather than "proving" what is true.

The nature of the question is that we are wondering if the mind can move past concrete experience.

Moreover, I tend to think we all accomplish the meantal feat in question on a daily basis. If you're a chair-maker you imagine the chair before you create it based on a multitude of chair-related properties, even if the chair you're making has never been created before. If you're Michaelangelo you look at a block of stone and see the "statue hidden within". The person who created this website designed information architecture in their mind, which is the categorical organization of non-existing "things" into one structure rather than any of the other possible structures, all of which are visually and physically non-existent.

The mere idea of using a stone as a tool requires that an animal to "see" the properties of something being applied in an unprecedented way.

Seeing two sides or surfaces of a sphere may only require a level of familiarity with spheres that one can combine and understand multiple points of view in a mental environment. The first quote even implies a progression of "skill" in multi-faceted visualization.

Trying to decide whether it is possible or not seems like an irrelevant argument of philosophers; figuring out how we do it seems like an interesting question of science.

Eric Schwitzgebel said...

Thanks for the comments, Kapitano and Joel.

Kapitano: I mean to be considering something more radical than a fast-sweeping viewpoint -- something that could never be adequately presented with current cinematic technology (or perhaps not in any visual way, ever). It's one thing to sweep around quickly. It's another not to sweep at all but to have 360 degrees simultaneously. That said, I completely agree with you about how much media metaphors affect our thinking about imagery and consciousness!

Joel: I'm all for using science to attempt to figure this out, rather than declaring the possibility or impossibility for 4-D imagination beforehand. But I'm not sure you're right that we all engage in such visualization. Truly *visual* and *simultaneous* imagery of objects from more than one viewpoint is not the kind of thing most people report.

Arnold Trehub said...

Hi Eric,

I wonder what you and your readers would make of this experiment:

Seeing-More-Than-is-There (SMTT)

If a narrow vertically oriented aperture in an otherwise occluding screen is fixated while a visual pattern is moved back and forth behind it, the entire pattern may be seen even though at any instant only a small fragment of the pattern is exposed within the aperture.


1. Subjects sit in front of an opaque screen having a long vertical slit with a very narrow width, as an aperture in the middle of the screen. Directly behind the slit is a computer screen, on which any kind of figure can be displayed and set in motion. A triangular-shaped figure in a contour with a width much longer than its height is displayed on the computer. Subjects fixate the center of the aperture and report that they see two tiny line segements, one above the other on the vertical meridian. This perception corresponds to the actual stimulus falling on the retinas (the veridical optical projection of the state of the world as it appears to the observer).

2. The subject is given a control device which can set the triangle on the computer screen behind the aperture in horizontal reciprocating motion (horizontal oscillation) so that the triangle passes beyond the slit in a sequence of alternating directions. A clockwise turn of the controller increases the frequency of the horizontal oscillation. A counter-clockwise turn of the controller decreases the frequency of the oscillation. The subject starts the hidden triangle in motion and gradually increases its frequency of horizontal oscillation.


As soon as the figure is in motion, subjects report that they see, near the bottom of the slit, a tiny line segment which remains stable, and another line segment in vertical oscillation above it.

As subjects continue to increase the frequency of horizontal oscillation of the almost completely occluded figure there is a profound change in their experience of the visual stimulus.

At an oscillation of ~ 2 cycles/sec (~ 250 ms/sweep), subjects report
that they suddenly see a complete triangle moving horizontally back and forth instead of the vertically oscillating line segment they had previously seen. This perception of a complete triangle in horizontal motion is strikingly different from the tiny line segment oscillating up and down above a fixed line segment which is the real visual stimulus on the retinas.

As subjects increase the frequency of oscillation of the hidden figure, they observe that the length of the base of the perceived triangle decreases while its height remains constant. Using the rate controller, the subject reports that he can enlarge or reduce the base of the triangle he sees, by turning the knob counter-clockwise (slower) or clockwise (faster).

3. The experimenter asks the subject to adjust the base of the perceived triangle so that the length of its base appears equal to its height.


As the experimenter varies the actual height of the hidden triangle, subjects successfully vary its oscillation rate to maintain approximate base-height equality, i.e. lowering its rate as its height increases, and increasing its rate as
its height decreases.

T. said...

Conc. visualizations of higher dimensional geometries: IMO every mathematician can do that, when he works on themes related to that. But the really interesting point is that one does not need such visualizations at all. E.g. much of mathematics involving geometry depends on general and abstract features of spaces and relations between them, whose visualizations need no "higher dimensional" imaginations. OTOH "familiar" three dimensional space quickly goes beyond the average visual powers, e.g. these "sphere eversions" created by a blind mathematician.

Conc. "mental paradoxa": Here a very strange story I one heard.