Demonstration 4.1

Take a look at the image below. In the bottom section, you see a gradient smoothly changing from black to white. In the top, you see the same gradient, but now the shades of grey change in discrete steps, with each successive column. Closely inspect one of the grey columns. Look at the edges and at the middle. How does the coloration of the column appear to you?

 

Do the columns in the demonstration above appear to be a uniform shade of grey throughout its entire width? Or do you perceive, like I do, that the left edge of the column (and every column) appears lighter while the right edge appears darker? These so-called Mach bands indeed look like they have lighter and darker edges. In fact though, each column is a uniform shade of grey across its whole extent. Your visual system has perceptually exaggerated the differences to make the changes in the visual field more obvious! This phenomenon is called contrast enhancement.

Contrast Enhancement

One of the more fascinating and far-reaching consequences of the connectivity of neural networks in our brain is how they tend to exaggerate differences (enhance contrasts). And this happens almost immediately!

Consider the retina. Located in the back of our eyes (see figure below), the retina contains three layers of different cells in the so-called vertical pathway: photoreceptors, bipolar cells, and ganglion cells. Our photoreceptors (rods and cones, in purple and blue below) transduce electromagnetic energy into neural signals. As discussed in Chapter 2, we can think of photoreceptors as representing “pixels” of the visual field. Remarkably, the visual receptive fields take a dramatic leap in representational complexity at just the next level of the pathway.

Bipolar cells have what’s known as a “center-surround” receptive field. Here, a bipolar cell receives direct input from photoreceptors falling within a “central” region (shown below in light purple) for that cell and indirect input (via horizontal cells, shown in grey above) from photoreceptors surrounding the central region (shown in darker purple below). The net effect of this direct and indirect input is that the bipolar cell responds differently to visual information falling within the center of their receptive field compared to information falling within the surrounding area.

Below you can see a diagram which shows the receptive field of an “on-center off-surround” bipolar cell. This cell will be stimulated whenever light falls on the center of the receptive field and inhibited whenever light falls on the surrounding area. Let’s imagine two hypothetical situations. In the first, the photoreceptors which project to both the central and surrounding areas are being stimulated. This might happen if you were just looking at the center of a blank document on your computer or tablet. The photoreceptors in the center area will cause the bipolar cell to become more active via direct input, while the photoreceptors in the surrounding area will cause the same cell to become less active via indirect input. The inhibitory effect of the surround will largely counteract the excitatory effect of the center, with very little net activity change in the bipolar cell. In the second situation, imagine looking at the edge of your computer or tablet. For the sake of this example, imagine that the edge of your computer was black and fell on the right edge of the inhibitory-surround, as shown below, while everything to the left of the edge was your blank document. In this case, the bipolar cell would receive less inhibition, because part of the inhibitory-surround will not be stimulated. As a result, the bipolar cell will be much more active.

Let’s rephrase the results of these two scenarios. When there was no contrast in the visual field, there was very little net activity difference in the bipolar cell. In contrast (pun intended), when there was a contrast, because of an edge, the cell became much more active. Activity in just the second layer of cells in the retina thus reflects a contrast in the incoming information stream.

Lateral inhibition is principally responsible for the creation of center-surround receptive fields that are sensitive to contrasts. In the retina, lateral inhibition is implemented by the cells of the so-called horizontal pathway—amacrine cells and horizontal cells—which have an inhibitory effect on information flowing through the vertical pathway (from photoreceptors to bipolar cells to ganglion cells). In other words, without lateral inhibition, bipolar and ganglion cells would simply be on-cells or off-cells. Converging input from upstream cells in the vertical pathway (e.g., photoreceptors or bipolar cells) along with input from cells in the horizontal pathway creates these more dynamic response profiles. Consider again the Mach band illusion at the start of this chapter, and in particular an on-center off-surround bipolar cell whose center area is just to the left of an edge. This cell would receive lateral inhibition from cells with receptive fields to the left of the center area and to the right of the center area. The right of the center area (which is to the right of the edge) is more illuminated, which means the cells with receptive fields in that area may be more stimulated. If they are more stimulated, they will exert greater lateral inhibition on the bipolar cell, resulting in both decreased activity and the perception that the area just to the left of the edge is darker than it actually is. The same logic explains why the area to the right of the edge appears lighter than it actually is, with a bipolar cell receiving less inhibition from the darker left side of the edge. By making the edge seem darker on one side and lighter on the other, the presence of the edge is very clear to you and the rest of your visual system!

More generally, local (horizontal) inhibition of neurons that are excited by ascending and descending pathways is pervasive throughout the cortex of the brain. This is also illustrated in the predictive coding schematic from Chapter 3. This push-and-pull of excitatory and inhibitory dynamics means that not only the retina, but also the cortex of the brain, excel at detecting differences and enhancing contrasts.

Visually, contrast enhancement is important for picking out the contours (edges) of objects. This in turn facilitates object recognition. If you were to draw an object, such as a house, how would you begin? By outlining all the edges of the house. Edges are thus key components of what defines and differentiates an object. That this differentiation happens already at the second layer of cells in your visual system suggests that contrast enhancement is both a critical and central feature of information processing in your brain!

It also means that, from just the second layer of cells in the retina, we’re prone to illusions.

Demonstration 4.2

Look at the Hermann Grid below. What do you notice as you move your eyes across the display?

 

In the demonstration above, grey or black dots seem to appear and disappear at the intersections! This illusion can be explained by neurons with center-surround receptive fields. Overlaid on the Hermann Grid are the receptive fields of two such neurons. In both cases, the excitatory center will be stimulated by the white portions of the grid at those locations. Both cells will also have parts of their inhibitory surrounds stimulated by the white areas falling within those regions. Note that the inhibitory surround will be more stimulated in the top cell than in the bottom cell. As a result, the top cell will be more inhibited than the lower cell and you will perceive that area (and all similar areas at the intersections) as darker!^[1]

Bipolar cells are not the only cells to have a center-surround receptive field. The next layer of cells, the ganglion cells, also have them. And so do the next layers of cells in the thalamus.

These thalamic cells, in the lateral geniculate nucleus region, help to implement “opponent processes” in color perception. Take a look at the figure below. On the left is a “R+G-” cell, which will be stimulated by the color red and inhibited by the color green. It’s also possible for a cell to be stimulated by green and inhibited by red (in this case, the cell would be R-G+). In either case, green and red are “opponent colors” because of the connectivity between color-sensing cones in your retina and downstream neurons. There are also blue-yellow opponent cells and black-white opponent cells, as shown below.

These opponent color cells in the lateral geniculate nucleus also give rise to fun illusions. The most famous, and a key driver of the theory of opponent process theory, are afterimages. Here, when you stare at a color for awhile (30 seconds to a minute) and then look at a white surface, you’ll see its opposite color!

What did you see?!? Did you see a brief flash of a circle with a blue center and yellow surround?

What’s happening here is that you are fatiguing those neurons in an opposing-color circuit that represent particular colors. This means that a) neurons that are stimulated by that color will not be stimulated as much and b) neurons that are inhibited by that color will not be inhibited as much. When you moved your eyes to the white space, you weren’t able to normally represent that white light (which is a combination of red, blue, and green light). If the neurons involved in processing blue in a region of space are fatigued, you’ll have less activity in your B+Y- neurons and more activity in your B-Y+ neurons. As a result, you’ll briefly see yellow. Likewise, if the neurons involved in processing yellow are fatigued, you’ll have less activity in your B-Y+ neurons and more activity in your B+Y- neurons, resulting in the perception of blue!

There are many examples of afterimages online that are fun to play with. I’ve included one particularly wacky one here:

Demonstration 4.4

Stare at the center of the image below for at least 30 seconds, sitting a bit further back from the screen than you would normally. After at least 30 seconds, look to the white space at the right. Who do you see?

 

The interaction between excitatory and inhibitory inputs becomes even more complex in the first cortical area for visual input, appropriately called V1, in the occipital lobe. Sets of these neurons have so-called “double-opponent” cells. Here, the cells respond to a pair of opponent colors differently in the center of their receptive field compared to the surround of their receptive field. This is illustrated below where, for example, a V1 cell would be stimulated by the color red if it appeared in its center, and stimulated by the color green if it appeared in the surround. These same colors would inhibit the cell if they covered a different region of the cell’s receptive field.

The effect of double-opponent cells in the primary visual cortex is to enhance color contrast. This can be appreciated by considering the photo below of red berries hanging from a green-leaved plant. A red-green double-opponent cell like the one shown above will be maximally activated when the center of its receptive field is stimulated by a red berry and the surround is stimulated by a green leaf. As a result, the red berries will “pop out” more. This makes it much easier to pick out the potentially nutritious red berries compared to if there were no such contrast enhancement.

Now let’s look at a different kind of effect.

Demonstration 4.5

Look at the box in the center of the image below. This time, instead of just staring, covertly move your attention from one circle to the next. That is, try to focus on one circle at a time, without moving your eyes from the center box. What do you notice as you do this?

 

What did you notice? Did the circle you were focusing on appear to get darker?!?

This is a very interesting demonstration showing that attention can change the perceived brightness of an object relative to the others. Or in other words, attention can enhance perceived contrast. What’s remarkable about this demonstration is that the stimulus itself isn’t changing, but your perception of it does as you mentally interact with it!

In fact, one of the ways we can think of voluntary attention is as another system for doing contrast enhancement. It turns up the “gain” on whatever you are focusing on, so that it is easier to pick that feature out. Imagine trying to find a friend in a crowded street. If you knew your friend was wearing a red hat, you could turn up the gain on your red-processing-neurons to give those neurons a competitive advantage in processing the scene. As you scan the crowd, focusing on “red”, you’ll be more likely to quickly find your friend. The advantage of such a top-down attention system is that you can selectively and flexibly enhance the processing of stimuli in a goal-directed way.

Attentional Capture

So far, we’ve seen how our brains can automatically perform contrast enhancement on visual stimuli and how we can use our attention to temporarily enhance the contrast of visual objects in the space around us.

In addition to object and spatial contrast enhancement, we can also talk about temporal contrast enhancement. In this form of contrast enhancement, our brains are tuned to the differences in successive moments in time. One example of this is seen in attentional capture.

In attentional capture, stimuli grab our attention, forcing an “alerting and orienting” response to behaviorally-relevant occurrences in the world. For example, sudden, loud changes in sound will grab our attention, mediated by the inferior colliculi in the midbrain of our brainstem. This happens when we receive a “ping” alerting us to a new message on our phones, or when commercial breaks on TV seem louder than the accompanying program. A similar phenomenon occurs with light, mediated by the superior colliculi. A flash on your phone automatically grabs your attention, as does the sudden appearance of a notification of a message on your desktop. Contemporary films have a shorter “shot length”: the amount of time a particular perspective of a scene is shown. This means there are a greater number of transitions between shots, and this increase in temporal differences keeps your attention yoked to the screen. To give one final example of attentional capture, angry or fearful faces can also automatically grab our attention in a scene. This is thought to be mediated by the amygdala. We can readily imagine why this would be beneficial, as such faces indicate the possibility of a threat (either from the angry person or from an event occurring near a fearful person).

More generally, surprising and novel occurrences tend to grab our attention and coordinate large-scale brain changes to attend to and remember those occurrences. Our discussion of contrast enhancement and difference detection now converges with our discussion of predictive processing in the last chapter. There we saw how our brains’ internal models generate predictions about incoming information, and compares incoming information against these models. When there is a difference between the incoming information and the model, that difference will be coded as “prediction error”. This can then be used to either find or create a better fitting model of the data. In information theory, prediction error is called “surprisal”. Surprisal corresponds to the subset of information which can be considered informative, since it deviates from what you already knew (as predicted by your internal models). We also saw in the last chapter how model-data mismatches (surprisal) can trigger an “alert and orient” response.

In these ways, contrast enhancement and predictive processing can be thought of as two different sides of the same coin. Our discussion of predictive processing highlighted the central role of model-based prediction in our perceptual and cognitive lives. We experience things as closer to how we expected to experience them than as unbiased representations of the world as it is now. This has important implications which you were encouraged to consider. Contrast enhancement and difference detection highlight the tendency for sudden changes (spatially or temporally) to “pop out”. We notice them more. Perhaps more brightly, perhaps more vividly, or perhaps as accompanied by a feeling of surprise. This invites a consideration of the kinds of things we tend to notice, or the way in which our noticing can be manipulated, which we turn to next.

Perceptual Blindness

In the previous section, we saw how our brains are tuned to differences in successive moments in time. As a result, we tend to notice, attend to, and remember surprising and novel occurrences. But we don’t always notice differences. In this last section, we’ll examine a few causes of such perceptual blindness.

Inattentional blindness refers to being unaware of stimuli or large-scale occurrences that are outside our current attentional window. For example, as you are reading now, you are likely unaware of the tactile sensations associated with your sitting posture, such as the feeling of the backs of your legs on a chair. Now that I’ve directed your attention to these sensations, though, you now perceive them.

Perhaps the most famous example of inattentional blindness is the one illustrated in the video demonstration below.^[2]

Did you see the gorilla?!? Many people miss it the first time they watch this video. Why?

In the task given, you’re asked to count the number of passes between players wearing white shirts. This is made somewhat difficult because the players in white are constantly moving around, while other players are also in motion. To help accomplish the task, you can selectively attend to the color white in order to focus on the players wearing white. Perhaps you’ll recognize similarities with examples I gave involving voluntary attention in the first section on contrast enhancement: we can flexibly use our attention to enhance the processing of, or turn up the gain on, the object of our attention.

But this has a cost. When we turn up the gain on an object of attention, we also inhibit the processing of other objects. This dynamic between facilitation and inhibition was noted very early in the history of psychology, by the often-prescient William James, in the first textbook of psychology ever written:

Every one knows what attention is. It is the taking possession by the mind, in clear and vivid form, of one out of what seem several simultaneously possible objects or trains of thought. Focalization, concentration, of consciousness are of its essence. It implies withdrawal from some things in order to deal effectively with others….

– William James, Principles of Psychology (1890), Chapter 11 (Attention).

In the gorilla video, you must “withdraw” from processing the color black, in order to focus on white. Note that the gorilla has the same color black as the players that need to be ignored. By inhibiting the processing of the players in black, you also then are less likely to register the gorilla. This effect is likely further enabled because, as we saw in the discussion of color contrast, white is an opponent color of black.

Selective attention (for example, to a color) is often thought of as a spotlight. Attention “shines light on”, or makes more perceptible, what falls within its reach. At the same time, objects outside of the spotlight are rendered less perceptible because of this focus. Indeed, we can think of the spotlight of attention as being like a flexibly directed on-center off-surround receptive field! More accurately, we can think of selective attention as an on-center off-surround biasing field. This analogy with center-surround fields brings us back full circle (pun intended) to where we started this chapter.

Selective attention and inattentional blindness are two dimensions of the same process. Whereas selective attention highlights our ability to enhance the processing of particular objects, inattentional blindness highlights the inhibitory and perceptual consequences of this enhancement. If we have habitual patterns of attention, we then also have habitual patterns of inattention. If our attention is being pulled toward some objects (as examined in Activity 4.1 on the Attention Economy), awareness is also being pushed away from other objects. It’s worth considering what our attentional habits might reveal about our perceptual neglect. We’ll return to this topic later in Activity 4.2.

Another type of perceptual blindness is change blindness. There are two types of situations where we often fail to notice perceptual changes: either when there is a sensory interruption between the changes or when the change happens too slowly.

The first cause of change blindness is vividly demonstrated in the following video, called “The color changing card trick”.

How many changes did you notice?

In this demonstration, your awareness of scene changes was impaired by the selective presentation of apertures of the whole scene through successive film shots. To notice these kinds of interrupted changes requires using a different type of memory system, which has a much lower capacity. We can certainly notice some changes, particularly those in objects we happened to be attending to, but we are less likely to notice them when the immediacy of change is not available. This type of change blindness can be very difficult to prevent, as evidenced by a variety of editing failures in television programs and movies that escaped the awareness of the editors meant to notice them! And, as suggested by the video, magicians regularly use misdirection so that you are blind to the changes responsible for executing a magic trick.

Another cause of change blindness is when changes occur too slowly. For example, we don’t notice ourselves aging in the mirror from day to day. But if we look at photos separated by a number of years, the changes become obvious! A dramatic example of this kind of change blindness is shown in the demonstration video below.

The video above illustrates a very unusual situation in which an object slowly materializes into space. Not only is that an unusual phenomenon “in the real world”, but it’s also not one that has behavioral consequences requiring a reaction. This is what is meant by “too slow”. Our brains have evolved under selection pressures (e.g., predation) that tend to operate over particular time spans. Think of a lion attacking! We are less sensitive, if not entirely insensitive, to changes occurring with much slower dynamics.

Of course, there are changes happening very slowly that we perhaps ought to act upon, and therefore should try to notice. Climate change is one example. Gradual escalations leading toward war is another. These types of changes can require real effort to perceive. More generally, change blindness suggests there are features of our worlds that we may not be sensitive to, either because they effectively don’t change, or change too slowly to be noticed. In the final activity of this chapter, I invite you to try to perceive something to which you may normally be blind.