“When the moon hits the eye…”

Not only beauty but also reality lies in the eye of the beholder – according to two recent papers by Institute of Science and Technology Austria (IST Austria) professor Gašper Tkačik with his collaborators. Two quite similar visual scenes can be perceived very differently first by the retina and then by the brain. Tkačik together with Einat Granot-Atedgi, Ronen Segev and Elad Schneidman from the Weizmann Institute of Science and the Ben Gurion University, developed a theoretical tool that allows them to define the difference between two seemingly very similar visual scenes. This opens up exciting possibilities for researching sensory systems.

The eye is our visual gateway to the world. When we view a scene, the light signals coming into our eyes are transformed into sequences of nerve impulses, which are sent to the brain via the optic nerve. Anything the eye does not signal through the optic nerve cannot be perceived by the brain and therefore does not exist for it.

The retina is the first piece of neural tissue that transforms visual inputs into action potentials recognizable by the brain. For a long time, it has been thought that this process is comparable to the workings of electronic cameras: each neuron is responsible for encoding a certain spot in the visual field and reporting to the brain the light intensity in this spot. This view has been challenged in the past decade in several ways. For example, one surprising observation has been that unlike in a camera, each spot in the visual field does not map in a one-to-one fashion to a single neuron in the retina. Instead, each spot is jointly encoded by the activity of a double-digit number of neurons.

The scientists have been asking themselves what the purpose of such an over-representation is: for example, does it serve to compensate for the unreliability of single neurons? Or do different neurons report different types of events that can happen in the visual scene? As a retina is unlike a camera, its workings need to be analyzed carefully; in particular, since they are wired together, a population of neurons encoding a single spot in the visual scene does not do so independently, but using a combinatorial set of activity patterns. Understanding how such patterns represent visual information is thus much like breaking a code.

In their paper “Stimulus-dependent maximum entropy models of neural population codes”, published in the March 2013 edition of PLoS Computational Biology, Gašper Tkačik together with his collaborators recorded the simultaneous output of about 100 retinal neurons. They then developed a new mathematical model to analyze the “code” that these neurons send to the brain while being exposed to controlled visual stimulation. In this way, they could effectively monitor to what the neurons are conveying and what not.

This development laid the groundwork for their paper “Retinal metric: a stimulus distance measure derived from population neural responses”, published in Physics Review Letters 110, 058104 (January 28), [with accompanying viewpoint article]. Their aim was to see which visual inputs can be distinguished by the brain, and how well. Mathematically, this led the scientists to the following question: if we take two images, how can we precisely define how different these two images are from each other? There are many possible mathematical definitions of such difference, for example by considering – pixel by pixel – how different the light intensity is in the two pictures. These mathematical definitions may, however, not correspond to how differently the brain experiences the two images. As the retina acts as a filter, conveying some information but not all, images that are mathematically very different by some measure may be experienced by the brain as actually very similar. Alternatively, images that appear mathematically quite similar may be distinguished by the brain because of the heightened sensitivity of the retina to very particular image features, or differences, perhaps ones that to us are behaviorally important.

Gašper Tkačik and his collaborators therefore wanted to find a definition of similarity and difference that is biologically relevant for the retina and for the perception by the brain. In their Physics Review Letters publication, they presented salamander retina with pairs of simple images as stimuli and analyzed the response of a large number of retina neurons. The scientists introduced a new definition of stimulus difference, defining it as the difference in the response the stimuli can reliably induce. Tkačik and his collaborators show that groups of recorded retinal ganglion cells are jointly sensitive only to certain features of the stimulus. While this feature sensitivity was previously documented for single neurons, it was unknown how sensitive large groups of neurons react together in encoding visual inputs. The results of this study explicitly demonstrate that some stimuli which are mathematically very different, cannot be discriminated by the retina very well, while other pairs of mathematically different stimuli reliably cause different and distinguishable neural responses.

This research opens up a more general framework for investigating and understanding sensory stimuli. Beyond the proof-of-concept application that appeared in their two recent publications, the researchers are designing new experiments in the retina that will be guided by the concept of “retinal metric”. These newly developed mathematical tools can be applied to other sensory systems as well, especially those where there is a lack of any clear concept of similarity between stimuli. For example, in the olfactory system, two chemicals may give rise to two scents, or odor percepts. But there is no unique way of defining the difference between two molecules. However, the difference in neural responses to the two molecules is easy to define and measure. Using the framework of the researchers, it should be possible to thus define the difference between odors and even mixtures of odors – a project the researchers are excited to embark on.