The Role of the Visual Cortex

The visual cortex isn't one structure doing one job. It's a network of dozens of interconnected areas at the back and sides of the brain, each specialising in some aspect of seeing, such as edges, colour, motion, faces or layout (Felleman & Van Essen, 1991). Together they take up a large share of the cortex, a measure of how much work vision demands.

V1: the way in

Signals from the eyes, relayed through the thalamus, arrive first at the primary visual cortex, called V1 or the striate cortex, at the very back of the brain. Its neurons are famously particular. In the 1950s and 1960s, David Hubel and Torsten Wiesel showed that individual V1 cells respond best not to plain light but to edges and bars at a particular angle in a particular place (Hubel & Wiesel, 1959), work that won them a Nobel Prize. V1 keeps a rough map of the retina, and its cells are arranged in columns that cycle systematically through the different angles. In effect, V1 breaks every scene into a huge catalogue of small edge fragments before anything like an "object" exists in the brain.

Beyond V1: a division of labour

From V1, signals spread to a series of further areas, labelled V2 to V5 (also called MT) and beyond. V2 continues working on edges and contours, and some of its cells respond to illusory contours, edges inferred where none is drawn (von der Heydt, Peterhans & Baumgartner, 1984). That's the brain activity behind the Kanizsa triangle. Area V4 is heavily involved in colour, including colour constancy, the stable perception of an object's colour under changing light that the checker shadow and the dress exploit.

Area V5/MT specialises in motion. Damage to it can cause akinetopsia, a rare condition in which moving things are seen as a series of still snapshots: Josef Zihl and colleagues described a patient who found pouring tea difficult because the liquid seemed frozen in mid-air (Zihl, von Cramon & Mai, 1983). The motion aftereffect is thought to come from adaptation in motion-sensitive areas like this one.

Further along, in the temporal lobe, are regions that respond to particular categories. The best known, the fusiform face area, responds much more to faces than to other objects (Kanwisher, McDermott & Chun, 1997). That's part of why face illusions such as the Thatcher effect feel different from geometric ones: they engage dedicated face circuitry rather than general edge and motion detectors.

Contrast everywhere

A theme running through all these areas is the receptive field: the patch of visual space, or the kind of feature, a neuron responds to. Many receptive fields are organised with a centre and a surround that work against each other, so neurons throughout the visual system respond to differences rather than absolute levels. The classic explanation of the Hermann grid rests on this arrangement, although, as that page explains, the smudges seem to come from cortical processing rather than the retinal cells first blamed for them.

Feedback: not a one-way climb

It's easy to picture the visual cortex as a strict ladder, from simple features in V1 to whole objects further on. The wiring says otherwise. Higher areas send large numbers of connections back down to V1 and the thalamus, so expectation, attention and context can change how early areas respond to the same input. That feedback is central to how the brain handles ambiguous images. For figures like the Necker cube and the spinning dancer, imaging studies link each switch between interpretations to activity spread across visual, parietal and frontal areas, rather than to a single "decision" in one place (Sterzer, Kleinschmidt & Rees, 2009).

Illusions as probes

Because different illusions target different stages, vision scientists use them as probes. An illusion built from contrast and edges tests early processing; one built from depth cues, like the Ponzo illusion, tests how cues are combined into a layout; one that depends on grouping, as discussed in Gestalt principles of perception, tests how features are organised into objects. A striking example: people whose V1 has a smaller surface area tend to experience stronger Ebbinghaus and Ponzo illusions (Schwarzkopf, Song & Rees, 2011), a finding you can set against your own scores on those pages' tests.

Sources and further reading

1. Hubel, D. H., & Wiesel, T. N. (1959). Receptive fields of single neurones in the cat's striate cortex. Journal of Physiology, 148(3), 574–591. Find this reference on Google Scholar
2. Felleman, D. J., & Van Essen, D. C. (1991). Distributed hierarchical processing in the primate cerebral cortex. Cerebral Cortex, 1(1), 1–47. Find this reference on Google Scholar
3. Zihl, J., von Cramon, D., & Mai, N. (1983). Selective disturbance of movement vision after bilateral brain damage. Brain, 106(2), 313–340. Find this reference on Google Scholar
4. Kanwisher, N., McDermott, J., & Chun, M. M. (1997). The fusiform face area: A module in human extrastriate cortex specialized for face perception. Journal of Neuroscience, 17(11), 4302–4311. Find this reference on Google Scholar
5. Sterzer, P., Kleinschmidt, A., & Rees, G. (2009). The neural bases of multistable perception. Trends in Cognitive Sciences, 13(7), 310–318. Find this reference on Google Scholar
6. von der Heydt, R., Peterhans, E., & Baumgartner, G. (1984). Illusory contours and cortical neuron responses. Science, 224(4654), 1260–1262. Find this reference on Google Scholar
7. Schwarzkopf, D. S., Song, C., & Rees, G. (2011). The surface area of human V1 predicts the subjective experience of object size. Nature Neuroscience, 14(1), 28–30. Find this reference on Google Scholar