Active Inference and its Application to Empirical Data

Thursday, November 4, 2021 – Part I (background): 2:00 - 3:15 pm ; Part II (applications to data): 3:30 – 5:00 pm

Online
Passcode: MAPs

Dr. Ryan Smith, Associate Investigator, Laureate Institute for Brain Research, Tulsa, OK

What?

Active inference, and in particular its recent application to partially observable Markov decision processes, offers a unified mathematical framework for modelling perception, learning, and decision making. This framework treats each of these psychological processes, and their interactions, as interdependent forms of Bayesian inference.

Active interference models can be fit to participants' choices in behavioral tasks. This provides estimates of computational model parameter values for each participant, which can subsequently be used as individual difference measures. Often these parameters include the precision of prior expectations, the precision afforded to sensory input, learning rates, and levels of risk-seeking vs. exploratory drives. For example, one could test the hypothesis that some individuals do not seek out sufficient information before settling on a strategy for maximizing desired outcomes; or one could examine the possibility that some individuals rely too heavily on prior expectations and/or update their beliefs too slowly in the face of new evidence. If participants perform tasks during neuroimaging, these models can also predict trial-by-trial changes in neural responses that can be tested against empirical data.

This talk will introduce the active inference approach to modelling partially observable Markov decision processes (POMDPs). It will cover the structure of these models, their mathematical underpinnings, and the types of other psychological and neuroscientific research questions they may be helpful in answering. The main aim is to provide the audience with a foundation to build from in applying this approach in their own research.

Why?

Particular benefits of active interference models:

• They include perception, learning, and decision-making parameters in a single model.
• They offer a unique approach to modelling ‘explore-exploit’ trade-offs (i.e., when individuals choosing to seek information vs. reward).
• They include a neural process theory that supports the theory’s biological plausibility and can make precise neuroscientific predictions.

The active inference approach is a leading approach in computational psychiatry. It also has gained a large following in several research circles in psychology, neuroscience, and machine learning. However, this approach is mathematically complex. Some people prefer simpler (e.g., reinforcement learning) approaches over the added complexity of active inference models.

How?

Prerequisites: Fluency in MATLAB. Specifically, ability to with scripts that allow fitting models to behavioral data. A certain comfort level with model specification in terms of vectors and matrices. Fluency in model comparison.

Software: MATLAB and SPM. The former requires a license; the latter is freely available. There are Python versions being created that don't require a license.

Datasets: We have provided example scripts that generate simulated data for learning. Otherwise, our approach could be used on any dataset that includes behavioral tasks amenable to modelling approaches.

Tutorial: https://www.sciencedirect.com/science/article/pii/S0022249621000973

• Smith R, Badcock PB, Friston KJ. Recent advances in the application of predictive coding and active inference models within clinical neuroscience. Psychiatry and Clinical Neurosciences. 2020.

• Schwartenbeck, P., FitzGerald, T. H., Mathys, C., Dolan, R., & Friston, K. (2015). The dopaminergic midbrain encodes the expected certainty about desired outcomes. Cerebral cortex, 25(10), 3434-3445.
• Mirza, M. B., Adams, R. A., Mathys, C., & Friston, K. J. (2018). Human visual exploration reduces uncertainty about the sensed world. PloS one, 13(1), e0190429.
• Smith R, Schwartenbeck P, Stewart JL, Kuplicki R, Ekhtiari H, Investigators T, et al. Imprecise Action Selection in Substance Use Disorder: Evidence for Active Learning Impairments When Solving the Explore-exploit Dilemma. Drug and Alcohol Dependence. 2020;215:108208.
• Smith R, Kuplicki R, Feinstein J, Forthman KL, Stewart JL, Paulus MP, et al. A Bayesian computational model reveals a failure to adapt interoceptive precision estimates across depression, anxiety, eating, and substance use disorders. PLoS Computational Biology. 2020;16(12):e1008484.
• Smith R, Kirlic N, Stewart JL, Touthang J, Kuplicki R, Khalsa SS, et al. Greater decision uncertainty characterizes a transdiagnostic patient sample during approach-avoidance conflict: a computational modeling approach. Journal of Psychiatry & Neuroscience. 2021;46(1):E74-E87.
• Smith R, Kirlic N, Stewart J, Touthang J, Kuplicki R, McDermott T, et al. Long-term Stability of Computational Parameters During Approach-avoidance Conflict in a Transdiagnostic Psychiatric Patient Sample. Scientific Reports. 2021;11.
• Smith R, Mayeli A, Taylor S, Al Zoubi O, Naegele J, Khalsa SS. Gut inference: A computational modelling approach. Biol Psychol. 2021:108152.

Registration Information

Please subscribe to the MAPs workshop via google group: yale-maps@googlegroups.com