Biological Sciences Training Program (BSTP): "Neural Mechanisms of Model-Based Planning in the Rat"
Dr. Kevin Miller, Ph.D., Postdoctoral Associate at University College London in the Cortex Lab and Research Scientist at Deep Mind, will be giving the BSTP seminar. His talk is entitled, "Neural Mechanisms of Model-Based Planning in the Rat". The seminar is hosted by the Department of Molecular Psychiatry.
Abstract
Neural Mechanisms of Model-Based Planning in the Rat
Planning can be defined as the use of an internal model, containing knowledge of action-outcome contingencies, to guide action selection. Recently, we adapted for rodents a multi-step decision task widely used to study planning in human subjects, allowing the experimental toolkit available for rodents to be brought to bear on this problem in a new way. We found that rats adopt a strategy of model-based planning to solve the task, and that silencing neural activity in either the orbitofrontal cortex or the dorsal hippocampus was sufficient to impair this strategy (Miller, Botvinick, & Brody, 2017). Here, I will describe data from new experiments designed to reveal the computational role in model-based cognition played by each region. In the orbitofrontal cortex, neurons encode information about expected outcomes in a manner specifically suitable for a role in model-based learning, but not for a role in model-based choice. Trial-by-trial optogenetic inactivations similarly reveal a pattern of impairment that is consistent with impaired learning, but not with impaired decision-making. These data suggest that rodent OFC acts as a "model-based critic" (Schoenbaum, et al., 2009), signaling expected outcomes to a process which updates choice mechanisms residing elsewhere in the brain. In the dorsal hippocampus, neural activity does not seem to encode information about expected outcomes, but instead indexes the various behavioral states of the task in a manner reminiscent of “place cell” coding. Ongoing work seeks to test computational proposals that hippocampal activity supports planning via predictive coding (Stachenfeld, et al., 2017), via representations of latent environmental state (Gershman & Niv, 2010), or via prospective activity during either theta sequences (Johnson & Redish, 2007) or sharp-wave ripple events (Mattar & Daw, 2018).