Reconsidering Dopamine

Maybe it's not all just about reward and movement

Sometimes I forget about the two years of my college education that I spent doing nothing but thinking about neurobiology. There’s a vague sense in psychiatry that we’re supposed to know about the basic science aspects of the brain, but in practical terms it’s not really necessary most of the time. You might contend that, hey, knowing that excess dopaminergic activity in the mesolimbic pathway is thought to underlie the positive symptoms of schizophrenia is useful to know, isn’t it? That knowledge should cause you to avoid medications that increase dopaminergic tone, like stimulants, right? And you’d think so, but it turns out that maybe giving amphetamine to people with psychosis is actually fine? There are many such examples where theoretical understandings of neurobiology turn out to be useless in clinical practice.

At its very worst, this reality cultivates a sort of “who cares?” mentality in psychiatrists when it comes to neurobiology — the results in the patient in front of you are all that really matter, right? It can be hard to argue with this sometimes, particularly when patients get better or worse for no apparent reason; especially when you do something that you’re confident couldn’t possibly work and probably makes no sense and then it works and you almost wish you hadn’t just so you don’t feel so confused. I find myself slipping towards that purely pragmatic, “who really cares about neurobiology?” attitude at times, too.

Now and then, though, I get introduced to something that reminds me that I am 100% right that neurobiology is amazingly cool and definitely matters… probably. Maybe. Look, it’s really cool, ok?

That “something” lately has been predictive processing, which I have primarily been learning about through Andy Clark’s phenomenal book Surfing Uncertainty: Prediction, Action, and the Embodied Mind. For those of you not familiar with predictive processing, Scott Alexander puts it best in his own review of Surfing:

Unbeknownst to me, over the past decade or so neuroscientists have come up with a real theory of how the brain works – a real unifying framework theory like Darwin’s or Einstein’s – and it’s beautiful and it makes complete sense.

Here though, I want to go through a paper that’s about some of the basic science aspects of predictive processing, which I talked about in collaboration with a science communication Vtuber who goes by Phy the Neutrophil. Check his stuff out!

The paper is Dopamine Modulates Subcortical Responses to Surprising Sounds, published in PLoS Biology in 2020 by Valdés-Baizabal et al.

Some Background on Predictive Processing

I’ll try and summarize the predictive processing paradigm a little bit (please note that this is a gross oversimplification).

The most common idea about how the brain works is a purely “bottom-up” model. This model tells us that we are constantly absorbing and integrating every little bit of sensory data that we have available to us. We take that data, pass it forward in our neural processing hierarchy, and gradually build up more and more complex pictures of the world around us until we end up with our conscious perception. In other words, we attempt to create a perfect internal representation of the external world through sensory information, so we can then determine behavior. Importantly, this model implies that every time you perceive a new stimulus, your brain has to interpret and integrate the raw sensory information de novo every time¹.

Cognitive scientists thought about this for a few decades and some of them eventually pointed out that it would be really energetically expensive to integrate and process all of that information all the time. Wouldn’t it make more sense, they asked, for the brain to develop some sort of predictive model about what the current environment is like? That way you could save time and energy by only dealing with “relevant” bottom-up information.

In other words, what if the brain had a top-down and bottom-up set of processing pathways?

Enter Predictive Processing (PP)

PP demands that we go back to first principles and remember that the bottom-up information our brains have access to is nothing more (or less!) than electrochemical signals being passed between neurons. Just as the LEDs in my display do not “know” that they are receiving an electrical signal from a graphics card in order to display the color white in a middling essay about Predictive Processing, an individual bipolar neuron in the retina does not “know” that the action potential flowing through it was triggered by a cone cell that optimally detects light with a wavelength of 564–580nm and just so happened to be struck by a photon with a wavelength of around 532nm which we will eventually perceive as roughly this shade of the color green.

PP asks how a system that only has access to such simple pieces of information might organize itself. The answer is with a collection of top-down and bottom-up processing hierarchies, where the top-down systems try to organize themselves in a way that will accurately predict the behavior of the systems below them in the hierarchy. In other words, top-down systems attempt to explain away “expected” inputs so that they don’t get (needlessly) passed further up the processing hierarchy. This seems like a tough job, but fortunately they have the perfect training data — the bottom-up information sourced straight from our sense organs!

To provide a illustration of this idea I’ve thrown together a very simple neural network schematic (THIS IS NOT AN ACCURATE REPRESENTATION OF THE CURRENT UNDERSTANDING OF THESE MODELS, JUST A REALLY QUICK WAY TO UNDERSTAND THE CONCEPT):

Green = Excitatory - Red = Inhibitory - Black = It’s complicated

The “goal” in this simplified model is for the Top Down Predictor to drive the GABAergic Interneuron strongly enough to totally cancel out any excitatory input from the Sensory Neuron onto the Error Detector. The output is thus whatever sensory input wasn’t accurately predicted. This output/unpredicted input is often referred to as “prediction error, “surprise,” or “surprisal.”

Predictive processing argues that this sort of processing happens at every level of our brains, from the neurons in our retinas (yes, before you’re even into the visual cortex!) all the way up to the prefrontal cortex. At every level, top-down models are working to filter out expected information, leaving the error signals to be passed upwards. As error moves upwards through the processing hierarchy, different top-down models compete to provide the best account of these errors. Predictive processing contends that once the system has settled on a top-down state that adequately minimizes ascending error signals², we experience that state as our conscious reality.

Attention

Ok, so let’s say that PP has this right, and our brain is a very complex set of hierarchical networks that work to suppress expected inputs and only pass along error signals. Unfortunately we are left with the problem that not all error signals are created equal.

Consider a situation in which you’re out on a hike with your dog. You decide to let him off the leash to run around a bit and he bounds out of view. A few minutes later you hear a howl and see a dog-shaped form on the ridge above you. That howl sure sounded like Fido, but that shape on the ridge doesn’t quite look like a good boy and you know that there are wolves in this forest.

In this situation your top-down model needs to try and resolve the uncertainty to help guide action, but how does it decide which stream of sensory error to pay attention to? The answer is the stream that is deemed most likely to resolve the largest amount of error most reliably (i.e. which streams will be both precise and accurate). This process is referred to as precision-weighting.

In this situation, your top down model predicts that listening closely to more howls won’t reliably provide you with more information — you’re not sophisticated enough to differentiate the slight tonal differences between Fido’s howls and a wolf’s — so your top-down model dampens whatever error signals are coming from your auditory cortex. However, your top-down model predicts that visual information will be very useful here — it’s pretty confident that you can tell the difference between Fido and a wolf — so your top-down model amplifies incoming visual error (i.e. you direct your attention towards it).³

We can dive a bit into the multi-level aspects of “attention” in the PP model even more.

Fido is about the size and the shape of the wolves you’ve seen, but you know that Fido is wearing a bright orange collar and you’re very confident that wolves don’t wear bright orange collars (your top-down model knows they prefer earth tones). So, a top-down model might suppress some of the error signal from pathways that primarily process information from your rod cells (which don’t sense color), and amplify error signals from cone cells (which do).

Go one step further, and you would expect your top-down model to suppress error signals from cone cells that capture light in the blue part of the spectrum and amplify error signals from the other two subtypes of cone cells which are predicted to capture the expected orange signal better.

This modulation of error signals — adjusting the “gain” if you will — is what we mean when we talk about ‘attention’ in the predictive processing model.

Clear as mud, right?

Cool, let’s move on.

Dopamine

Probably everyone knows dopamine as the “reward” neurotransmitter. If you’re in the neurosciences you probably also think of it as playing a key role in positive reinforcement and movement.

The classic model of dopamine release from the midbrain conceptualizes it acting as a sort of reward mismatch signal. When the reward is better than expected, dopaminergic neurons increase their burst firing activity. When the reward is worse than expected, they reduce their tonic output. It’s a very neat little story that was supported by experimental data and it serves as the cornerstone for a lot of theories about addiction.

There are some problems though. First, there is now a substantial amount of experimental data that indicates that dopaminergic neurons in the midbrain respond to a whole lot of different information. Yes, there are some populations that are excited by reward-predicting information, but there are others that are excited by cues that anticipate noxious stimuli. Even more damning for the “dopamine as reward predictor” hypothesis, is the existence of populations of dopaminergic neurons that adjust their firing in response to totally surprising and unanticipated stimuli, to which they should theoretically not respond at all.⁴

The Paper

Valdés-Baizabal et al. make all of these points about dopamine, and propose an alternative hypothesis within the predicting processing framework:

What if dopamine’s role is to modulate the gain on bottom-up error signaling based on top-down predictions? In other words, what if its role is to encode expected precision?

The authors argue that this would solve a lot of the problems between the reward theory of dopamine release and contradicting observations. It would explain why we see dopamine release in response to both positive and negative anticipatory cues, as well as why we see its release in response to unpredicted events.

How might you test this?

Well, a simple way is to use the auditory oddball paradigm; you play a series of unchanging tones (STD) and randomly intersperse an oddball/deviant tone (DEV) in there from time to time.

One of the best places to record the effects of the oddball paradigm is in an area of the brain where inputs from sensory organs and higher cortical areas meet. In this study, the authors chose the inferior colliculus (IC) of the rat, mainly in its nonlemniscal portion, which they say “plays a key role in processing surprising sounds and shaping the auditory context.” This region of the brain is also confirmed to receive dopaminergic inputs from the thalamic subparafascicular nucleus, and mRNA analysis indicates that neurons in the IC express D2-like receptors.

Previous experiments in rats have shown that neurons in the IC display a type of short-term plasticity called “selective stimulus adaptation” (SSA) in response to the oddball paradigm, which is quantified as the difference between neuronal population activation in response to STD and DEV tones. Over time, population SSA increases due to dampening of the response to STD tones, while response to DEV tones is preserved.

The authors talk a lot about SSA and a standardized measure of the SSA called the Common SSA Index (CSI). I think this is really just a way to be able to talk about changes in firing patterns across different neurons that might have different firing patterns. I suspect that there might be a bigger reason to care about them. I’m guessing that the SSA is thought to encode some type of information on its own. Unfortunately, the paper doesn’t make it clear and I’m not smart enough to figure it out. If you know or are smart enough to figure it out, please let me know.

Drug Delivery and Neuron Recording

I’m not going to spend a ton of time on the technical aspect of the drug delivery and recording because this is way, way outside of my area of expertise.

Basically they used glass micropipettes that are hand-crafted to have very fine points that are able to record electrical activity for single units (i.e. single neurons) and deliver drugs locally.

They confirmed that they were in the right part of the brain by histologic examination. You might wonder how they did that when these pipette tips are so small that they are able to record from single neurons - surely there’s no way you could see those small piercings.

Correct!

So instead this is what they do:

At the end of each experiment, we inflicted electrolytic lesions (5 μA, 5seconds) through the recording electrode. Animals were euthanized with a lethal dose of pentobarbital, after which they were decapitated, and the brains immediately immersed in a mixture of 1% paraformaldehyde and 1% glutaraldehyde in 1M PBS.

In other words, they burn holes into their brains with electricity, euthanize them, decapitate them, pop their brains out, and put them into a fixative solution ASAP.

Fun times.

Frequency Response

Figure 2 shows us frequency response ranges and intensity of response, for two different neurons before and after dopamine administration. A & B represent one neuron, C & D represent a second. The pair of graphs on the left (A and C) show the frequency response and intensity, while the ones on the right (B and D) show the delta between the control and dopamine graphs.

You can see a differential response to dopamine here, which will be a running theme through the paper. The neuron on the top has its response enhanced across the frequency spectrum and at lower volume levels, while you see the opposite on the bottom.

This seems like a sort of throw-away thing, but before you can get anywhere you’ve first gotta show that dopamine is actually modulating these cell’s response to auditory stimuli in some way.

Oddball Response with Dopamine

Next the authors looked at how delivery of dopamine and a D2 antagonist (eticlopride) modulate response to the oddball paradigm (schema below):

This is where things start to get interesting! In individual neurons, dopamine consistently (though not universally) caused the neurons to reduce their response to the DEV tone.

E and F represent two different neurons. You can see how the red DEV trace decreases in magnitude.

Across the population of 104 neurons they sampled, a reduction in the DEV response was by far the most common change seen with dopamine administration. Changes in STD response were mixed and dependent on the individual neuron being recorded; at a population level, dopamine did not alter firing in response to STD tones.

You can see this illustrated in the above graphs. In the violin chart (B) you can see that only response to the DEV stimuli was statistically different — a 26% drop in the median spike frequency response to DEV (p < 0.001).

Scatterplots C and D here are just another way of showing the data from the DEV and STD plots in chart B. The dashed line represents a perfect 1:1 ratio between the control and dopamine conditions. Dots that fall below the dashed line indicate that their firing rate with dopamine treatment was lower than in the control condition.

So far, so good! This is pretty much what we would expect to see if our hypothesis was correct that dopamine’s role is to attenuate surprising bottom-up signaling. A significant decrease in firing in response to DEV tones, without changes to STD tone response.

Why don’t we see a consistent response across all neurons? Well, given that we’re literally just sticking electrodes into the surface of the brain and advancing them until we get a clear signal, it seems likely that we’re just sampling different functional populations of neurons.

Oddball Response with Eticlopride

The eticlopride trials were not so conclusive, as we can see in Figure 4

There was no population level effect of eticlopride in the 43 neurons recorded. The authors point out, however, that eticlopride did appear to modulate the firing responses to STD and DEV relative to one another (i.e. the CSI) when individual neurons were analyzed. Only 11 units (the grey dots in scatterplot A) did not show any change in their CSI, indicating the eticlopride was having some effect by blocking D2-like receptors.

Some thoughts on what could be going on here:

First, this might just be bad luck. The authors used a smaller sample size, got unlucky with what neurons they recorded from and got a nonsignificant result. Always possible.

Second, it’s important to consider the difference in what is trying to be detected between the dopamine and eticlopride experiments. With dopamine administration the authors are looking to see if exogenous dopamine — delivered in what are probably supraphysiological quantities and in a fashion that is not analogous to synaptic release — produces a change in DEV response.

Now consider the eticlopride experiment. In this case our dopamine release is coming from the brain itself, through normal neurons releasing normal amounts of dopamine at their normal synapses. Maybe the effect of endogenous dopamine is more subtle relative to the effect observed with exogenous dopamine and the sample size was too small?

Also important to note that the rats were anesthetized, which is absolutely going to alter the endogenous response to external stimuli. That’s… the whole point of anesthesia.

Are We Just Detecting Change?

Ok - so let’s say that those results are good enough that we buy the hypothesis that dopamine is playing some modulatory role in bottom-up signaling. Cool. Good. So far, though, we’ve only shown that dopamine is modulating the neuronal response to change, not necessarily to surprise. Experimentally, it’s actually pretty easy to separate these things out when you’re using auditory stimuli.

Many of you are probably already thinking, “Yeah just use musical scales, duh!” Indeed, they just played evenly spaced ascending or descending tones without any deviation from the expected pattern, which they refer to as the CAS (CAscade Sequence) condition.

In the violin plot you can see two interesting things:

Elevated Spiking in the CAS Condition

First, the neurons in the CAS condition increase their spike count just like in the DEV condition - so it’s clear that these neurons are conveying some information about changing tones, even if the changes are expected. This was actually a bit of a surprise when I first thought about it - shouldn’t we expect these neurons to behave more like in the STD condition where predictable stimuli produces a low spike count?

My speculation on why the CAS condition doesn’t behave that way has to do with the difference in how and where a brain would “notice” “surprising” changes in the CAS vs. STD conditions⁵.

Think about tracking a single, repeated tone. “Surprise” is any change to that tone. “Surprise” in a scale, however, means that you need to continually track the scale on some level to detect a deviation. My guess is that this part of the brain is at a low enough level of the hierarchy that the system that detects deviations in “sequences of tones” is located farther upstream, and so still needs the neurons at this level to transmit their data forwards.

It would’ve been really interesting to see if dopamine would alter error signaling in the context of a scale, but that’s an experiment for another time.

No Effect of Dopamine in the CAS Condition

At a population level, dopamine didn’t change the spike rate of the neurons in the CAS condition. The authors contend that this result indicates that dopamine is doing something beyond just modulating the response to change, it’s modulating the response to surprising change. This is exactly what you would expect to see if dopamine was modulating the response to surprise, specifically.

Honestly, I find myself pretty convinced.

Interesting Speculation About Psychopathology Follows

Warning: This is me being very speculative about how this relates to psychiatry. I am not an expert in PP and all of the neurobiological research that has been done on it, so expect some of this to be flat out wrong and much of it to be only loosely correct.

The thing that fascinates and excites me most about this paper is the invitation to think about dopamine as having a single role in the CNS: modulating error signaling (maybe primarily?) by dampening bottom-up error signals.

I’d like to run with that idea, because the more I think about it, the more it makes so much sense to me when I start to apply it to various psychiatric illnesses.

ADHD

Why stimulants “work” for ADHD has always perplexed me to some degree. Why does giving someone a drug that dumps a bunch of dopamine into their synapses improve focus and “executive function?” Why does that seem to improve impulsivity in people with ADHD, but not so much in people without it?

Thinking of dopamine as modifying the gain on error signaling answers a lot of these questions for me! If a major component of the poor attention in ADHD is caused by an inability to turn down the volume on irrelevant bottom-up signals (i.e. distractions), it would make sense that adding more dopamine to the system would improve this.

Subjectively, this is exactly matches my experience with my prescribed stimulants. I just seem to be able to “tune out” distractions better. It also explains a sort of cognitive rigidity that I’ve noticed while on stimulants. For example, when I play chess while not on stimulants, my focus is so poor that I just can’t compute more than 2-3 moves ahead and I will often forget the last line I calculated. Stimulants improve my ability to calculate, but I consider alternative lines much less frequently and find myself returning repeatedly to the one or two lines that I originally fixated on.⁶

It’s the same with video games - I actually seem to perform worse on stimulants in games that require me to constantly check up on multiple different systems or adjust my strategy on the fly.

Schizophrenia

If dopamine suppresses bottom-up error signals, and there is too much mesolimbic dopamine activity in schizophrenia… then you have inappropriate suppression of bottom-up error signals and top-down predictions can’t be properly updated/adjusted.

This provides a nice explanation for hallucinations. Remember that in the PP model perceptions are experienced when top-down models explain away ‘enough’ of the bottom-up signal. As long as the top-down model is ‘confident’ enough, and the bottom-up error signals are weak enough (i.e. ‘well explained’), there is no reason that top-down models can’t produce totally hallucinatory experiences. Incidentally, this is part of how PP proposes that we are able to engage in imagination and have such vivid experiences in dreams.

Delusions could be thought of as incredibly distorted top-down models, and you can see how excessively suppressed bottom-up signaling could make them difficult to overcome. In some cases, delusions may also develop as reasonable explanatory models in response to hallucinated bottom-up signals. However, this account of delusions does not adequately explain the sort of intense, sudden-onset delusions described by individuals who have recovered from their psychosis. Harding et. al in their paper A new predictive coding model for a more comprehensive account of delusions suggest that these sorts of delusions come about as the product of “amortized inferences” in which top-down systems in certain situations will produce a strong initial set of explanations that are then gradually discounted when subjected to opposing bottom up stimuli, as opposed to “iterative inferences” which are gradually built up over time from bottom-up information.

Movement

Not a psychiatric illness, but belongs in this speculative section. I’ve long wondered why the hell dopamine has ended up being so central to movement (via the nigrostriatal pathway) and somehow got co-opted into being the “reward” neurotransmitter. PP and the conceptualization of dopamine in this paper offer an answer!

Briefly, PP says that there are two ways to resolve error. You can either select a better top-down model that better explains the error, or act in ways that help to resolve the error. Usually we’re doing some mix of both, but in the case of movement PP suggests that we use this second option really hard. Essentially, movement is initiated by the selection of a series of top-down predictions of proprioceptive inputs - of course at the moment that these top-down selections happen, you now have a bunch of mismatch between the actual inputs and what has been predicted. The resolution thus comes by moving your muscles through the series of motions that match the top-down predictions and viola the PP paradigm now explains movement⁷.

So, how does this help resolve my confusion about dopamine’s disparate roles? Well, if dopamine’s job is really about modulating the gain on error signals, and movement is just the product of an attempt to explain away error by action… then maybe dopamine’s role in the movement pathways is no different than its role elsewhere in the brain’s neurocircuitry! It’s a beautifully simple explanation — which probably means I’m missing something — but it’s just so compelling that I can’t help but think it’s true.

A final piece of rank speculation:

The pathology of Parkinson’s disease results in insufficient dopamine release. Some develop REM behavior disorder prior to developing full-blown Parkinson’s, a condition in which your body doesn’t paralyze you like it’s supposed to while in REM sleep, and you act out your dreams. PP speculates that the ability to quash bottom-up error signals is what allows us to dream without acting — remember that movement is conceptualized as a way to reduce that bottom-up error — so it would make sense to me that a disease of impaired dopamine release also results in impairments with preventing movements during sleep!

---

1. Yes, I know that this is a gross oversimplification, please don’t write me angry comments.

2. No state is ever error free

3. The information flow need not be from a single sensory modality, it’s just easier to talk about it that way.

4. There are some arguments that this is because “surprise” should be inherently rewarding which would encourage novelty seeking - but the spectrum of how good/bad surprises tend to be are so wide that I think this is a poor argument.

5. Of course, they didn’t introduce any surprising tones in the CAS condition, but I think it’s a good explanation anyway.

6. This is also a time to remind ourselves that the “bottom up” signals are not just pure sensory stimuli, but can be higher-order concepts as well as we move upward through the processing hierarchy.

7. As someone else put it, you move by imagining that you’re moving really hard