Murphy PR, Wilming N, Hernandez-Bocanegra DC, Prat-Ortega G, Donner TH (2021). Adaptive circuit dynamics across human cortex during evidence accumulation in changing environments. Nature Neuroscience.
By Peter Murphy & Tobias Donner.
The neurobiology of perceptual decision-making has witnessed an impressive convergence between theory and experiment. Theorists have developed normative algorithms for making optimal decisions in certain contexts, which rely on the accumulation of evidence over time. Experimentalists have shown that these algorithms tend to provide excellent accounts of choice behavior measured in many experiments. They have also identified circuits in the brain that are prime candidates for implementing the associated computations.
Even so, much remains to be understood about how even the most elementary decisions are formed in the brain. One reason for this is that most of the recent progress came from studies using tasks in which the state of the environment remains constant while the agent is accumulating the evidence. In this situation, decision difficulty arises solely from noise in the stimuli and maximizing decision accuracy requires the perfect (i.e., lossless) integration of all the evidence. In natural settings, however, the state of the environment can change, which requires more sophisticated, adaptive evidence accumulation schemes (Ossmy et al, 2013; Glaze et al., 2015). Moreover, the canonical neurobiological model for perceptual decision-making that has emerged from this previous work postulates the linear integration of evidence along purely feedforward information processing pathways. These features are difficult to reconcile with some fundamental properties of cortical anatomy and function. Cortical networks are massively recurrent, locally within regions, as well as across regions. This organization gives rise to non-linear dynamics, and it may preclude a straightforward segregation of evidence and decision representations in the brain as assumed by a serial processing account.
We reasoned that exactly the recurrent organization and non-linear dynamics of cortical networks might help the brain make good decisions in changing environments. We started from a normative algorithm for evidence accumulation in two-alternative forced choice (2AFC) tasks with changing environments, which had been developed by Josh Gold’s lab (Glaze et al., 2015). This model entails a non-linear form of evidence accumulation with a non-linearity that depends on the volatility (or hazard rate) of the hidden states of the environment that generate the evidence. Our hunch was that this normative, non-linear accumulation might be well-approximated by well-known features of neural decision circuits.
We asked 51 human participants across three experiments (~160,000 trials in total!) to perform a task in which they viewed sequences of stimuli with different spatial locations. The stimuli came from one of two noisy sources that could undergo hidden switches at any time during a trial. The participants needed to report which source was ‘in play’ at the end of each sequence.
The participants were by and large very good at this task. They achieved accuracies that were quite close to those produced by the normative algorithm, albeit limited by internal noise and some apparent biases in their estimates of the task statistics (as indicated by fits of computational models to their choice behavior). Indeed, the normative algorithm subject to noise and biases fit the behavior of the participants better than several alternative models that we considered. Critically, participants also showed two key qualitative signatures of the non-linear accumulation process prescribed by the algorithm (Figure panel a): a strong sensitivity to the surprise associated with a newly observed evidence sample (reflecting the inconsistency of the sample with the existing decision state), and weaker but still reliable sensitivity to the uncertainty under which a new sample is encountered (reflecting the absolute accumulated evidence before sample onset).
We next found that an established cortical circuit model for evidence accumulation and decision-making developed by XJ Wang (2002) produced choices with precisely these same qualitative signatures in our task – and activity that closely approximated the dynamics of the normative evidence accumulation. This model had previously been used to reproduce single neuron activity observed in standard perceptual choice tasks without change points. It relies on a balance of recurrent excitation within populations of choice-selective neurons, and inhibition between them (Figure panel b), to form decisions. We found that it required only minimal tuning (of the strength of the recurrence, and the stimulus input) to work for our task. In other words, our hunch turned out to be correct!
What’s more, we found that the normative algorithm as well as the circuit model closely matched the dynamics of action-selective motor preparatory activity of our participants’ brains in our task, measured with magnetoencephalography (MEG; Figure panel c). More detailed source analyses showed that this activity was generated by regions in posterior parietal, premotor, and primary motor cortex, all of which are involved in action planning. Taken together our findings indicate that premotor cortical activity can be explained by recurrent cortical circuit dynamics, which in turn implements the normative non-linear evidence accumulation process required by changing environments.
We weren’t finished there though. The normative model that guided our research was adaptive only for a restricted setting (a fixed hazard rate that had already been learned). Likewise, the circuit model described a single cortical area with fixed circuit properties (e.g. recurrence and E/I balance) that could produce normative evidence accumulation only for restricted setting. The real brain is equipped with a large mosaic of cortical areas with heterogenous properties, which are subject to dynamic neuromodulation. These considerations indicate that (i) the problem faced by the brain is much broader and (ii) it is equipped with machinery that may endow it with the flexibility to generate appropriate decisions in different environmental contexts. Inspired by these insights, and capitalizing on recent technical developments in our lab, we dug deeper into the data. We dissected decision-related signals across the entire visuo-motor pathway transforming sensory input into behavioral choice; and we also quantified the dependence of these dynamics on pupil-linked arousal signals reflecting neuromodulatory input to cortex.
The first of our additional analyses characterized the impact of the dense feedback projections from higher-tier areas encoding decision states (including parietal and premotor cortex), to early sensory areas that process the incoming sensory information. One function of feedback could be to consolidate emerging, distributed decision states in the brain (Nienborg & Cumming, 2009; Wimmer et al., 2015), which may be beneficial in stable environments (where decision states do not need to change often) but not in highly volatile environments (where decision states need to change frequently). In keeping with other recent work from our lab (Wilming et al., 2020), we found clear signatures of decision-related feedback in early visual cortex in a relatively stable task context (Figure panel d). And critically, this feedback adapted to the level of volatility, becoming suppressed when changes in task state were very frequent. Thus, the brain might exploit a flexible, adaptive feedback mechanism to stabilize evolving decision states when it’s beneficial to do so.
A second possible source of flexibility that we investigated is that provided by pupil-linked arousal systems, which we tracked through changes in pupil size measured during MEG. Pupil size increased specifically in response to evidence samples that were surprising or encountered under uncertainty (Figure panel e) – the two factors strongly modulating evidence accumulation in the normative and circuit models, and human behavior. Larger pupil responses were also associated with a stronger influence of the eliciting evidence samples on choice; and with a stronger encoding of the sensory evidence in visual cortex. This broadly supports ideas that pupil-linked neuromodulation is engaged to rapidly adjust the weight that new evidence exerts on evolving decision states in the brain (Dayan & Yu, 2006); and suggests that, in concert with tunable feedback signals, neuromodulatory systems may be key players in imbuing cortical decision circuits with the flexibility to appropriately adapt to varied environmental contexts. This project was significant for us in just about every sense: challenging, long in preparation, hugely instructive, and ultimately extremely satisfying. And as it should be, it left us with many more questions than we answered. Do the circuit mechanisms we identified generalize to other task settings where behavior and learning are similarly shaped by surprise and uncertainty, but over much slower timescales (e.g. Nassar et al., 2010)? How does the final choice behavior result from the collective activity of these interconnected cortical circuits with heterogenous circuit properties? What distinct roles might different neuromodulatory systems play in the non-linear accumulation process we characterized here? What can our approach tell us about psychological disorders that are characterized by aberrant weighting of evidence in decision-making, and/or a lack of adaptability to different contexts? We look forward to tackling these and other questions in near future. Thanks for reading, and check out the paper for many more details and interesting results!
References
Dayan, P. & Yu, A. J. Phasic norepinephrine: a neural interrupt signal for unexpected events. Network 17, 335–350 (2006).
Glaze, C. M., Kable, J. W. & Gold, J. I. Normative evidence accumulation in unpredictable environments. eLife 4, e08825 (2015).
Nassar, M. R., Wilson, R. C., Heasly, B. & Gold, J. I. An approximately Bayesian delta-rule model explains the dynamics of belief updating in a changing environment. J. Neurosci. 30, 12366–12378 (2010).
Nienborg, H. & Cumming, B. G. Decision-related activity in sensory neurons reflects more than a neuron’s causal effect. Nature 459, 89–92 (2009).
Ossmy, O. et al. The timescale of perceptual evidence integration can be adapted to the environment. Curr. Biol. 23, 981–986 (2013).
Wang, X. J. Probabilistic decision making by slow reverberation in cortical circuits. Neuron 36, 955–968 (2002).
Wilming, N., Murphy, P. R., Meyniel, F. & Donner, T. H. Large-scale dynamics of perceptual decision information across human cortex. Nat. Commun. 11, 5109 (2020).
Wimmer, K. et al. Sensory integration dynamics in a hierarchical network explains choice probabilities in cortical area MT. Nat. Commun. 6, 6177 (2015).