ESI News

#Sciart!😊Two @maxplanckpress researchers, Hermann Cuntz & Marvin Weigand, will showcase their work on computer-generated #neurons at the Centre #Pompidou in #Paris @ the exhibition "Neurons | Simulated intelligence" from 26 Feb - 20 Apr 2020 https://t.co/2Isg8Wk3dQ pic.twitter.com/zEbAbddrJg

— Max Planck Society (@maxplanckpress) February 26, 2020

Hermann, exhibiting at a world famous art museum as an artist is exeptional as a researcher even more so. How did this come about?

Hermann Cuntz: It was a whole chain of events that led us to contribute to this exhibition at the Centre Pompidou. It actually started when my PostDoc boss Michael Häuser submitted one of the pictures of these artificial nerve cells for the Wellcome Image Awards competition. The theme was “most beautiful scientific picture of the year” or something along those lines. We actually won the prize and the picture was exhibited at the Wellcome Collection in London. This but also others of our pictures distributed pretty well around the internet and probably that’s why science journalists starting coming up to me. One thing led to another: Someone knew someone who knew someone and one day the ZKM (Centre for Art and Media Technology) in Karlsruhe contacted me and I made an exhibit for them. It was a 360-degree film in their PanoramaLabor, and they liked it so much that they asked me to do something for a biennial the ZKM organized in 2015. Marvin had just started his doctoral thesis with me and he came up with the idea of translating the whole thing into virtual reality. That made the whole thing extra cool and apparently also made an impression on people in the art scene. In any case, the curator of the exhibition at the Centre Pompidou knew our neurons through the ZKM Biennial. And indeed, one has to say that our visualization fits the theme of the exhibition very well.

The exhibition at the Centre Pompidou is called “Neurons - simulated intelligence” and is intended to be something like a journey through the history of artificial intelligence from an artistic perspective. But there are also quite biological exhibits - a brain in formaldehyde, for example. What does your piece look like?

Marvin Weigand: Basically, we show the cortical neurons that are produced by Hermann’s model. When visitors put on the VR glasses, they get to fly through 150 of those simulated neurons. You can control the direction of flight by moving your head. Occasionally a neuron fires an action potential - then it flashes. The whole thing is accompanied by rather spherical music, which helps to shut out the real world and completes the immersive experience that one wants to achieve with Virtual Reality.

Is it a strange feeling to contribute to an art exhibition as a scientist? And do you think your work actually has artistic value?

Hermann Cuntz: I think that if we have now made it to the ZKM and the Centre Pompidou, then what we do is art. That means, the science we do is a part of contemporary art. Of course, there are artists who have nothing to do with science, but we are obviously on a border or rather a transition between disciplines. Art and science blend into each other, you can’t really distinguish between them.

Even though you are exhibiting and therefore somehow also an artist, you are actually scientists and make pictures and videos as part of your research - what scientific knowledge do you gain from your artificial neurons?

Marvin Weigand: Strictly speaking, we do not really exhibit our research. The visualization, that is these artificial neurons that look like real biological preparations, are the byproduct of what we’re actually researching. We are interested in the general rules of the architecture of the nervous system. If we think we have discovered a possible rule, such as how neurons grow their dendrites, then we put this rule into a model that predicts what a dendrite that grows according to these rules would look like. And the more realistic the artificial dendrite looks later, the better we understood the general rule behind it.

Hermann Cuntz: The pyramidal cells that we are exhibiting at the Centre Pompidou for example, come from an article I published a few years ago. In that study, I looked at dendrites not only of pyramidal cells, but from all kinds of neurons and from many different species. The paper is called “One Rule to Grow Them All” and we show that a basic rule for the growth of dendrites is to keep the cable length as short as possible and the signal transmission as fast as possible. So if you know where the cells are a dendrite has to get its inputs from, then you can calculate with our model how the optimal dendrite should look like. And the same rules can be applied over different species, over different cell types. This works in all the cases we have tried. And one of the nice things about this model is that it passes the Turing test right from the beginning. You can give these cells - the visualization of the cell - to an expert and he can no longer differentiate: Is this a real neuron or is it artificial. In fact, it once happened that a colleague gave a lecture at our institite and said in the introduction: Of course, it will never be possible to reproduce this beautiful variety of real nerve cells in a computer model. And then he showed my picture of artificially created neurons. Obviously this is a fantastic confirmation for us that we did something right. Of course, it’s not enough to count as reliable evidence, but it is an indication.

So, the visualization of the neurons is a test whether you have made correct assumptions. But the VR element is just a joke that you thought up for the artwork without any practical, scientific use for it?

Hermann Cuntz: I wouldn’t say so. Marvin’s implementation in VR allows for people who work with similar models or approaches as we do, to have the possibility to visualize the artificial cells immersively in three dimensions. And not just individual neurons, but within a network of cells. The more cells you want to look at in context, the more difficult it is to keep track. And if you can so to say experience the constellations in VR spatially, it definitely helps to gain a better understanding. Marvin may have been the first to translate neurons into VR five years ago, but there are now other groups that are doing it simply because they are coming up against the same limits of visual perception as we are.

Marvin Weigand: In fact, I would like to publish the VR implementation, not necessarily as a paper, but as a program for virtual reality glasses. Free to download for everyone. I think it could have a pretty high impact because it is still very new. And regardless of whether it is now relevant for scientific work, we could certainly reach people who have nothing to do with research and increase interest in neuroscience in general. Just now, while I was setting up the exhibition in the Centre Pompidou, a security guard approached me and said that generally speaking, he wasn’t really interested in modern art, but that he thought our piece to be very nice and he wanted to know more about it.

What’s next on your agenda, art or research?

Hermann Cuntz: Both. We’re currently talking with an Israeli artist couple about a new, interactive project. But of course most of our time and energy goes into research. One of our next projects is to study cell types not in isolation, but in a cell compound. We are now in a position to no longer let the cells grow alone in empty space, but also packed together in a piece of cortex for example. So, we are now trying to make all the cell types that we now of grow together in one simulation. And in the end, of course, we hope that we will be able to simulate electron microscopic images taken from real tissue. This will enable us to say: We expect our model to look like this - does it? If not, we still have to change something. If so, then we have already understood a lot about neural architecture.

Ever since the beginning of brain research, scientists have tried to infer the functions of the neural system from its building blocks: single nerve cells, or “neurons”. Neurons use sudden, electric discharges to transmit information. These so called action potentials or “spikes” are, so to say, the language of the brain. By sorting spikes according to shape and signal-strength, individual neuron “voices” can be identified in isolation from a choir of active neurons.

In this way, neurons can be divided into groups of cells with similar characteristics, so called functional cell classes. In the primary visual cortex, which is the first brain area that process things we see, neurons fall in two different classes: 1) inhibitory neurons with fast, narrowly shaped spikes, which silence the activity of other cells, and 2) excitatory neurons with slower, more broadly shaped spikes, which enhance the activity of other cells.

While this categorisation may be a good rule of thumb, a recent finding shows that notable exceptions exist: “Looking at the spiking activity from neurons in the primary visual cortex, we identified a third class of cells that was previously unknown,” says Irene Onorato, first author of the study that was published in the print edition of neuron today. The researchers discovered the newly described cell type in two different kinds of monkeys, but not in mice.

Bursting Neurons - more common than expected

The new type of cells are characterized by properties which in part resemble inhibitory neurons: they have fast, narrow shaped spikes. Yet, their activity patterns are clearly distinct from those of inhibitory neurons, namely, they break into burst-like bouts of spiking activity. Moreover they seem to be excitatory. Looking at the ratio of cell types, the researchers found that almost a third of the recorded neurons fell in the narrow-spike, bursting class. A fraction that is surprisingly large for a cell-type that has until now been overlooked.

Martin Vinck who supervised the study sees one possible explanation in the choice of animal model: “There are very few studies that have looked into functional cell classes in the primate brain. Most studies of that kind are done in mice. And as we could show by comparing data between species, mice simply don’t have these neurons.” Currently, mice are the most popular model organism in neuro science and with good reason. There are many cutting edge techniques, particularly genetic tools that can be used in mice but not in monkeys. This advantage however is compromised by the less developed cognitive abilities of the small rodents. Considering the differences between species, it’s no surprise to the researchers to also find differences in their brains.

Rhythmic brain waves and bursting neurons provide good timing

One of the striking differences between brains of monkeys and mice is that in monkey visual cortex there is a lot of rhythmic network activity. By contrast, in the visual cortex of mice, the rhythmic activity is much weaker. Brain rhythms, also called oscillations, reflect the synchronized activity of many neurons at the same time. Neuroscientists suspect oscillations to have a major impact on neural communication and memory formation. Particularly synchronized activity in the so called gamma band between 30 and 80 HZ is thought to play a role in higher cognitive functions, like attention and predictive processing.

“Knowing that particularly the monkey visual cortex basically flows over with rhythmic activity, and then discovering a type of neuron with burst of rhythmic activity, again in monkeys but not mice, it is kind of obvious to suspect a connection,” thinks Martin Vinck. He and his colleagues developed mathematical tools in order to investigate of the exact relationship between neurons and brain waves.

Indeed, the data analyses suggest that the newly described neurons, through interactions with inhibitory neurons, may act as a pacemaker to produce gamma-waves. Previous research has shown that when spikes are synchronized in gamma waves, they are transmitted more effectively to higher brain areas. It is as if synchronized activity provides a schedule for neurons at what time to speak up in order to be heard well. The new cell-type seems to act as a director for this schedule. It gets the timing right. Accordingly, it seems very probably that the new cell-type could transmit its information in a more powerful and efficient way to distant neuronal populations. These cells could therefore be very important for visual processing.

Original publication: Onorato, I., Neuenschwander, S., Hoy, J., Lima, B., Rocha, K.S., Broggini, A.C., Uran, C., Spyropoulos, G., Klon-Lipok, J., Womelsdorf, T., Fries, P., Niell, C., Singer, W., Vinck, M. (2020). A Distinct Class of Bursting Neurons with Strong Gamma Synchronization and Stimulus Selectivity in Monkey V1. Neuron 105(1), P180-197.e5. https://doi.org/10.1016/j.neuron.2019.09.039

We continue to learn throughout life. That doesn’t necessarily mean that we study for another university degree every month, but there are small changes in our environment all the time that we have to adjust to. Like when your colleague moves the coffee mugs to another shelf, or when you go on holiday to Switzerland and have to deal with a foreign currency. The basis of lifelong learning is a process called neuronal plasticity, which is the brain’s ability to change the wiring of its neurons. Connections between neurons are strengthened or weakened depending on how often they are active at the same time. “Neurons that fire together, wire together” is how scientists like to describe this simple but effective rule.

Now, neurons can be quite busy being excited about all sorts of input. Accordingly, rewiring is a very flexible process. It can happen very rapidly, but be reversed just as quickly. However, to introduce long-lasting changes, just firing together is not enough. The stimulus causing the excitation also has to be relevant; it has to fulfil certain criteria of validity. According to Wolf Singer, Senior Director at the Ernst Strüngmann Institute, this makes a lot of sense: “This way the brain makes sure not to introduce non-sensical modifications that would render it dysfunctional.” But how does the brain do that? How does it make sure all conditions in favour of a long lasting change are met?

Creating long lasting connections between neurons

Wolf Singer and his colleagues Ralf Galuske and Matthias Munk suspected local network dynamics in the so-called gamma-band between 20 and 48Hz as a promising candidate mechanism, which pushes short lived “wiring together” into long lasting neural connections. In order to test this idea, the scientists needed to set up a stable experimental condition, in which the only thing that would change was the presence or absence of precise synchrony of neuronal discharges. Such precise synchronisation of firing occurs when neuron populations engage in gamma oscillations. In previous experiments they found a way to do exactly that: In cats the occurrence of gamma oscillations in the primary visual cortex (V1) can be evoked by pairing visual stimulation with electrical stimulation of a particular area in the midbrain, the midbrain reticular formation. This stimulus configuration often but not always induces gamma oscillations in V1. If synchronized network activity is indeed a significant factor in defining a relevant stimulus, long term changes should happen only when oscillations are present.

This was exactly what the scientists found to be true in their experiment. Parallel to the electrical stimulation of the midbrain, they visually stimulated orientation selective neurons in V1 with gratings of different orientations. By measuring the orientation selectivity of these neurons both with optical and electrical recording techniques, they found that with time, the orientation preferred by the neurones started to drift towards that of the presented stimulus. If oscillations were weak or absent, the responses of the stimulated neurons became weaker but did not change their orientation preferences.

These results indicate that synchronized population responses at gamma-frequencies could be an important mechanism to regulate neuronal plasticity. “The idea is not new” summarizes Wolf Singer. “But we never had direct experimental evidence to back it up.” With the latest results at hand, scientists might have to expand the notion of “firing together” to include gating by precise temporal synchronisation.

Original publicatrion: Galuske, R.A.W., Munk, H.J.M., Singer, W. (2019). Relation between gamma oscillations and neuronal plasticity in visual cortex. Proc Natl Acad Sci USA 116(46), 23317-23325. https://doi.org/10.1073/pnas.1901277116