Yale Psychiatry Grand Rounds - March 5, 2021

Transcript

• 00:00Because my introduction is much less
• 00:02important than Doctor Calipari's talk,
• 00:04which will be eagerly awaiting.
• 00:07So it's it's my absolute pleasure
• 00:10to introduce Doctor Erin Calipari
• 00:13who's today's grand round speaker.
• 00:16She is someone who has really been
• 00:19a pioneer and incredibly prolific
• 00:22scientists already in the area of
• 00:25addiction and using a rodent models to
• 00:30understand the neurobiological basis
• 00:32of important constructs underlying
• 00:35addiction that are relevant to.
• 00:38Human subjects she did her
• 00:40undergraduate degree at the University
• 00:42of Massachusetts in Amherst,
• 00:44where she has got her BS in both psychology
• 00:47and biology and already began working
• 00:50with Doctor Gerald Meyer using rodent models.
• 00:53And then she did her graduate work at Wake
• 00:57Forest School of Medicine with Sarah Jones,
• 01:00where she began to.
• 01:03It really immerse herself and neurochemistry,
• 01:06an other aspects of in vivo manipulations
• 01:09and measurements related to behavior.
• 01:12She then went on as a postdoctoral fellow
• 01:15and then a instructor at the Mount Sinai's
• 01:19Icahn School of Medicine at Mount Sinai,
• 01:23where she worked with Eric Nestler.
• 01:26And there she began to develop
• 01:28a number of lines of research,
• 01:31including examination of.
• 01:33Particularly molecular consequences
• 01:35of drug addiction,
• 01:37and particularly intracellular
• 01:39signaling as a consequence of drug
• 01:43addiction that may maintain long term
• 01:46structural and behavioral adaptations
• 01:49to drugs of abuse and also was really
• 01:52key in a number of important papers
• 01:55on sex specific effects of addictive,
• 01:58addictive substances like cocaine and
• 02:01those experiments are particularly
• 02:04notable because they.
• 02:05Involved many, many levels of evaluation,
• 02:07not only in rodents but also in
• 02:10human subjects.
• 02:11She has won a number of awards already.
• 02:14I want to note a few notable ones.
• 02:17In particular she is an awardee of the DP.
• 02:21One Avenir Award in genetics,
• 02:23and epigenetics from the National
• 02:25Institute on Drug Abuse,
• 02:27and that's someone who the director
• 02:29of Naida pulls out as having research
• 02:32that is extremely innovative and
• 02:34at the edge of the.
• 02:36Molecular basis of addiction.
• 02:38She's an associate member of the AC NP.
• 02:41Oh I should have noted that she's
• 02:43also an assistant professor in
• 02:45the department's particularly in
• 02:46the Department of Pharmacology
• 02:48at Vanderbilt University,
• 02:50but also has appointments in
• 02:52the departments of Psychiatry
• 02:53and Behavioral Sciences.
• 02:55Of course relevant to our Department
• 02:57and the Department of Molecular
• 02:59Physiology and Biophysics.
• 03:00So again, even with her appointments,
• 03:03you can see how her work spans areas of
• 03:06investigation from the very molecular
• 03:08and cellular through the pharmacological too.
• 03:11The area of drug addiction
• 03:13relevant to psychiatric illness.
• 03:15So back to her awards.
• 03:17Just because they're so notable she is,
• 03:20has been awarded a Whitehall Foundation
• 03:23research grant and are said Young
• 03:25Investigator Award and a K99 to
• 03:28R00 pathway to Independence Award,
• 03:30and even back in her days
• 03:33as a graduate student.
• 03:34The Knighted Director's Award,
• 03:36which acknowledged her innovative
• 03:38work from the beginning,
• 03:40an addiction.
• 03:41She's an editorial boards relevant
• 03:43to this Department.
• 03:44Both the editorial board of
• 03:46Neuropsychopharmacology and
• 03:46the Journal of Neuroscience,
• 03:48and she's been incredibly
• 03:49prolific at every stage of her
• 03:52career, showing how which I love that
• 03:54not only does she have great ideas,
• 03:57but she carries them through all the way
• 03:59to really publishing beautiful papers,
• 04:02and it's been exciting to watch
• 04:04the evolution of the work.
• 04:06Do you want to say something that
• 04:08is not exactly science related,
• 04:10as she was recounting during?
• 04:12The first discussion she does have one area
• 04:15that she's interested in outside of science,
• 04:17or that she's able to devote time
• 04:19to as an assistant professor,
• 04:21and that is she is a powerlifter and
• 04:24I have seen a video of her doing.
• 04:27Plus, which I'm very, very envious of,
• 04:29but I do want to highlight that her most
• 04:32recent achievement in powerlifting is
• 04:34that she has just deadlifted 300 pounds.
• 04:36Now if that isn't an achievement,
• 04:38I don't know what is.
• 04:39So thank you very much for being
• 04:42with us here.
• 04:42Erin Dr.
• 04:43Calipari,
• 04:43it is a pleasure to welcome you
• 04:45to the Department and I'm looking
• 04:47forward to hearing we're all looking
• 04:49forward to hearing your talk.
• 04:51Thank you so
• 04:52much. That was so nice.
• 04:53And yeah, I know I tried it.
• 04:55You know the pandemic has
• 04:57made me start to try to find.
• 04:59Things to occupy my time that aren't
• 05:01just sitting at home and working and
• 05:03so that's that's what I've been doing.
• 05:06But I'm I'm actually really excited
• 05:07to present for a number of reasons.
• 05:09One, because I get to see some people
• 05:12outside of my immediate family in my lab.
• 05:14But to this, this work is some kind
• 05:17of work that's been developing in
• 05:19my lab of the last couple of years.
• 05:22That focuses on kind of what
• 05:24dopamine is doing,
• 05:25and so I think you know for somebody who
• 05:28studies addiction and psychiatric disease.
• 05:31The reason this is so important
• 05:33is because domains at the core
• 05:35of a lot of these disorders,
• 05:37and you know specifically addiction
• 05:39where you see deficits in dopamine,
• 05:41and I think it's really important
• 05:43to understand for people what
• 05:45those deficits mean.
• 05:46You know if dopamine is encoding
• 05:48reward and reduction in dopamine
• 05:50may mean something very different.
• 05:52Then if it don't mean is encoding
• 05:54some other aspect of learned behavior,
• 05:56which I'm probably going to show you today.
• 06:00So you know,
• 06:00I'll kind of show some stuff in the weeds,
• 06:03but also kind of tide into big
• 06:05picture and so please like stop
• 06:07me if things aren't clear or you
• 06:10have thoughts or comments as we go.
• 06:12OK,
• 06:13so the focus of my lab is really
• 06:15understanding if you can see my slides right.
• 06:18OK,
• 06:18good is understanding how neural
• 06:20experience or how neural circuits
• 06:22integrate experiences to drive
• 06:23behavior and so you know in life
• 06:26or in animals you know we have
• 06:28experiences that have negative outcomes.
• 06:30Things that have positive outcomes
• 06:32and what happens is that these
• 06:34experiences change the way our brain
• 06:36response to stimuli in the future to
• 06:38increase the probability of behaviors
• 06:40that result in good outcomes.
• 06:41And decrease the probability of
• 06:43behaviors that result in negative outcomes.
• 06:46And the reason I'm so interested
• 06:49in this is because this is kind of,
• 06:52you know,
• 06:53the fundamental core of how we
• 06:55make decisions.
• 06:56But it's also dysregulated,
• 06:57and almost every psychiatric disease state,
• 07:00and so you know for somebody
• 07:02who studies addiction,
• 07:04you know drug associated stimuli,
• 07:06or overvalued relative to
• 07:07negative consequences.
• 07:08An alternative reinforcers depression.
• 07:10You have reduced motivation.
• 07:11Valuation of rewards.
• 07:12Reward learning or things like
• 07:14anxiety and stress disorders where
• 07:16these negative outcomes may be
• 07:18overgeneralized to neutral cues and contexts,
• 07:20which is,
• 07:21you know things like PTSD,
• 07:23and so you know this kind of
• 07:25fundamental process by which we
• 07:27attribute value to things that our
• 07:29environment is really a core of
• 07:31how we should be thinking about
• 07:33treating people with disorders
• 07:35where this is dysregulated.
• 07:37So you know, I'm, you know,
• 07:39going way back to the simple,
• 07:41you know half of my lab studies.
• 07:43You know how to drug change the brain,
• 07:45but the other half of my
• 07:47labs that he's just
• 07:48kind of. How do these same systems
• 07:50work in a normal situation?
• 07:51And so the first thing is kind of,
• 07:54you know, how do we learn to
• 07:56make these adaptive decisions?
• 07:57And so we use things in our
• 07:59Virat environment to do this.
• 08:01So maybe there's contextual cues
• 08:02that help you figure out when things
• 08:05are dangerous and when they aren't,
• 08:06and so if you have something like the
• 08:09sound of a helicopter in a hometown
• 08:11may be very different than the cell of
• 08:14sound of a helicopter in a war zone.
• 08:16We have things like discrete cues,
• 08:18which would be the sound of a
• 08:19helicopter itself with drug addiction.
• 08:21It's these cues that are associated
• 08:23with the drug taking experience,
• 08:25and so we learn to associate those.
• 08:27And then there's also, you know.
• 08:29What we do in response to these stimuli,
• 08:31and so you know my background is
• 08:33really focused on reinforcement
• 08:34learning and how these stimuli in
• 08:36the environment drive animals to
• 08:37make decisions in different context.
• 08:39So what are they going to do?
• 08:41Are they going to, you know,
• 08:43are they reinforce?
• 08:43Are they going to do something or
• 08:45they going to avoid something?
• 08:47And how we can understand the neural
• 08:50circuitry that underlies this
• 08:51decision to kind of seek out or avoid
• 08:54different things in the environment?
• 08:56And so you know,
• 08:57I think this is a really important thing,
• 08:59right?
• 09:00Is that we learn to make predictions and
• 09:02so our actions have some sort of outcome.
• 09:05It changes the state of our
• 09:07environment and basically what we
• 09:09do is we learn to do something and
• 09:11so this is kind of guiding these.
• 09:13These associations aren't just there,
• 09:14they're guiding how we navigate
• 09:16an environment. And so.
• 09:18How do we do this?
• 09:21Well,
• 09:21we need to encode the value or
• 09:24salience salience is kind of like how
• 09:26attention grabbing something is of
• 09:28unexpected outcomes and so you get ice cream.
• 09:31That's great.
• 09:32It's awesome.
• 09:33We need to know whether it's good
• 09:35or bad and how good or bad it is.
• 09:38How attention, yes, are you advancing slides?
• 09:40Yes, sorry,
• 09:41this has happened to me before and
• 09:44I have no idea why it does this.
• 09:47Let me try this again.
• 09:49It's OK if you guys followed them.
• 09:51Yeah, I thought the intro was all very clear,
• 09:54but I thought maybe you were dancing.
• 09:56We didn't know. Yeah, this is
• 09:57this is happened to me before. I have no.
• 10:01Idea when or why this does this?
• 10:04Let me try this one more time.
• 10:06So where I made it 'cause it's on,
• 10:09zoom on the shorter side because
• 10:11I don't think people love watching
• 10:13zoom for two hours. So OK.
• 10:15So now if I move the slides they move OK.
• 10:19Well, there were pictures you guys
• 10:21have experience with all of this stuff,
• 10:23so that's fine.
• 10:24So now we're into the important
• 10:25bit so it's good you saw this. OK,
• 10:28so nothing like this is like pandemic level.
• 10:30Like everything,
• 10:30something has to go wrong every time.
• 10:32Otherwise, like you know, it's not.
• 10:34It's not real, so OK,
• 10:35so you have to encode some information.
• 10:37We need to know whether it's good
• 10:39or bad and how intense it is.
• 10:41Is this something we should
• 10:43really pay attention to?
• 10:44Or is this something that's not as important?
• 10:46We need to make predictions
• 10:48about when that's going to occur.
• 10:50And so you know you have an ice cream truck.
• 10:52You predict whether the ice
• 10:54cream will be there or not.
• 10:55But not only do we need to make predictions,
• 10:58we need to be able to update
• 11:00these when they change.
• 11:01So when something no longer is associated,
• 11:03we need to be able to adapt.
• 11:05If the update this so that we can change
• 11:07our behavior when the environment is
• 11:09not the same as we learned previously,
• 11:11and so this is a really, really critical
• 11:14aspect of learning and behavior.
• 11:15So I'm going to kind of go.
• 11:17There's going to be some computation in here,
• 11:19but what I'll tell you is most of
• 11:21it is is more of a framework for
• 11:23how people think about how these
• 11:25these computations are being done,
• 11:27and if you don't care about the computation,
• 11:29which I've met, many people who say,
• 11:31oh,
• 11:31whatever we've used these to
• 11:32design experiments,
• 11:33and so it's not like you
• 11:34need to know the math.
• 11:36It's more of a kind of framework
• 11:38for how we designed experiments.
• 11:40So this kind of prediction
• 11:42based learning was formalized.
• 11:43You know,
• 11:44originally by Rescorla Wagner in 1972,
• 11:46and there's been a bunch of kind
• 11:48of adaptations of this and allow
• 11:51the model to do other things.
• 11:53But really,
• 11:53what this is is it's a mathematical
• 11:56model that allows us to kind
• 11:58of formalize how animals learn,
• 12:00and So what happens in this model
• 12:02is that if you have something
• 12:04like an unexpected outcome,
• 12:06that is an error in prediction,
• 12:08you predicted nothing, something was there.
• 12:10You made an error and what happens
• 12:13overtime is your prediction gets
• 12:15better and then there's less error.
• 12:17So essentially what happens is the
• 12:20associative strength or how well
• 12:22you how well something predicts
• 12:24something goes up and the error in
• 12:26that prediction goes down and so
• 12:28basically the way the model works is
• 12:31that as you learn the prediction of
• 12:34that Q and the outcome increases.
• 12:36But any errors you make go down
• 12:39and so essentially
• 12:40you get this. Increase in the
• 12:42predictive response and a decrease in
• 12:44the error or the mistakes from that,
• 12:46and so this is kind of how animals learn.
• 12:49It can map learning rates in
• 12:51a lot of different contexts.
• 12:53You know learning about
• 12:54Accuen award extinction.
• 12:55All of these and so people have really
• 12:58been searching for what is a circuit
• 13:01in the brain that does this math.
• 13:03And that's been a kind of really big
• 13:06focus of specifically the dopamine field.
• 13:09And other fields too.
• 13:10I think a lot of people are
• 13:11starting to see that these kinds of
• 13:13computations are done in a variety
• 13:15of circuits across the brain.
• 13:17So the dopamine system is important
• 13:19for any a lot of reasons.
• 13:21These neurons you know that we focus on
• 13:24originate in the ventral tegmental area,
• 13:27so we're focusing on more reward
• 13:29associated circuits rather than things
• 13:31that are associated with motor.
• 13:33So we're looking in for this
• 13:35particular project.
• 13:36The nucleus accumbens,
• 13:37the core region,
• 13:38and So what these dopamine neurons
• 13:41are really important for survival.
• 13:43Lesioning them present prevents
• 13:44this kind of associative learning.
• 13:46An also reinforcement learning.
• 13:49And the thing people been kind of really
• 13:52focus on is that this domain neurons
• 13:54respond in a fashion that mimics this.
• 13:57This mathematical model I just showed you,
• 13:59and so essentially this kind of
• 14:01originated and within a lots of other
• 14:04people have shown these patterns.
• 14:05So this originative with Wolfram Schultz
• 14:07and I'm just showing the the original.
• 14:10But people within the domain
• 14:12field have done this with all
• 14:14kinds of other approaches as well.
• 14:16But essentially what they see
• 14:18is this kind of same math.
• 14:20We went hoping that this didn't
• 14:23just move because OK,
• 14:24the slides are still advancing, right?
• 14:27Yeah, OK, OK.
• 14:28So essentially what happens is
• 14:29you have an unexpected reward.
• 14:32Dopamine firing goes up.
• 14:34You predict that reward dopamine firing
• 14:37now goes up to the queue that predicts it,
• 14:40but not the reward,
• 14:41because the prediction of that
• 14:43reward is basically perfect.
• 14:44And now if the reward is omitted,
• 14:47what happens is the dopamine
• 14:49response to the queue goes up,
• 14:51but there is now a decrease in that
• 14:54domain response when it's omitted,
• 14:56signaling the negative error that,
• 14:58uh, from that prediction.
• 14:59And so this is they.
• 15:02You know, originally this,
• 15:03this first paper.
• 15:04They said, wow,
• 15:05that looks a lot like reward prediction,
• 15:07error learning,
• 15:08and so this is kind of formed the
• 15:10basis of the domain field dopamine
• 15:13does reward prediction learning.
• 15:15So here's the kind of maybe issue with that.
• 15:19If you do stress work,
• 15:21anything else you know the domain
• 15:23does not only respond to rewards
• 15:26and reward predictions,
• 15:28it's involved in things like punishment,
• 15:30which is an aversive learning parameter.
• 15:33Motivation, fear, safety transitions,
• 15:35aversive learning.
• 15:37All kinds of there's been a
• 15:38lot of Association,
• 15:39aversive learning and these
• 15:40fields have been kind of.
• 15:42It was a separate,
• 15:43but there's kind of the reward.
• 15:44Prediction people and then the
• 15:46people who studied anxiety,
• 15:47depression looking at Microdialysis,
• 15:48showing that dopamine does
• 15:49go up to aversive stimuli,
• 15:50and so these kind of have been a
• 15:52little bit at odds with each other.
• 15:55But they kind of are,
• 15:56you know, different fields,
• 15:57so people haven't really looked
• 15:59at them in the same context.
• 16:01And so essentially,
• 16:02I think some of the disconnect also
• 16:04comes from this kind of fundamental
• 16:06process of about domain neurons.
• 16:08That's actually my favorite
• 16:09part of GOP neurons.
• 16:11Many studies have looked
• 16:12at VTA cell body firing.
• 16:14They use electrophysiology.
• 16:15They say we don't need it goes up.
• 16:18It goes down.
• 16:19And there's this inference that
• 16:21that means dopamine release.
• 16:23That is,
• 16:23projection targets is going to be the
• 16:26same as what the firing looks like.
• 16:29But dopamine terminals are so cool
• 16:31because they're regulated at the
• 16:33terminal level by **** synaptic.
• 16:34So things that are regulated by Domi
• 16:37itself but also header, synaptic,
• 16:39regulators things like glutamate,
• 16:40GABA.
• 16:41A favorite of this Department,
• 16:43acetylcholine and so these.
• 16:44These things actually can elicit
• 16:46dopamine release from the terminals,
• 16:47independent of cymatic firing.
• 16:49And so if you want to understand
• 16:51what dopamine release another
• 16:52projection target is doing,
• 16:54you need to actually record
• 16:56dopamine and the ultimate wrists.
• 16:57You have a few of those as well,
• 17:00have been doing this for a really long time,
• 17:03but there's a lot of kind of
• 17:06limitations to voltammetry and we'll
• 17:08kind of talk about those as we go,
• 17:10but our goal was really too.
• 17:13Record dopamine,
• 17:13but be able to dissociate
• 17:15these kind of things.
• 17:16People have seen in the aversive
• 17:18field with the things people
• 17:20have seen in the reward fields.
• 17:22Why is dopamine look like it's doing
• 17:24both of these at the same time?
• 17:27So my background isn't reinforcement
• 17:29learning and what we did is we we
• 17:31like to develop behavioral tasks to
• 17:33parse the things that we're interested.
• 17:35So we developed this task,
• 17:37which is not really the task
• 17:38itself is an innovative behavioral
• 17:40pharmacology and reinforcement learning.
• 17:42People have been doing this for years.
• 17:45Essentially what we do is we
• 17:46have a queue that comes on.
• 17:48In one phase,
• 17:49that tells animals if they know
• 17:50spoke during this Q and in
• 17:52this example is white noise.
• 17:54But we counterbalance and change
• 17:55that they will get sucrose.
• 17:57This is like normal positive reinforcement.
• 17:59You know you treat your teacher dog,
• 18:01that's it. They get a reward.
• 18:03What we taught the animals in
• 18:05the other face is that a separate
• 18:07queue comes on and they have
• 18:09the same behavioral response.
• 18:10They know spoke,
• 18:11but they know spoke to prevent a
• 18:13series of shocks from being delivered,
• 18:15so it's called negative reinforcement.
• 18:18The reason that using these is so cool
• 18:20is because they have the exact same action.
• 18:23So if dopamine just encodes
• 18:24the motivated response,
• 18:25these will look the same.
• 18:28They have the same outcome value.
• 18:30The outcome is positive.
• 18:31Avoiding something negative is positive.
• 18:32Getting something positive is positive,
• 18:34but there's different stimuli maintaining
• 18:36these behavioral events and so
• 18:38essentially what we've associated here is
• 18:40the kind of motivated action from this
• 18:42stimulus value in the outcome value,
• 18:43and the question is in this sounds
• 18:46more complicated than it is,
• 18:47and I'm going to tell you the story
• 18:50is that we're going to be able to see
• 18:52if dopamine responds to just rewards
• 18:55if it's just involved in motivation,
• 18:57or if it's doing something maybe slightly.
• 19:00It's a more complicated,
• 19:01but it's actually simpler.
• 19:03We need a way to record dopamine
• 19:06during this task.
• 19:07Aversive foot shocks are electrical signals.
• 19:09All of the previous domain recording
• 19:11techniques on fast time scales
• 19:13were used on electrical systems,
• 19:14and so the problem with this is all
• 19:17the voltammetry techniques people
• 19:18use before you couldn't record
• 19:20responses to foot shocks,
• 19:22and So what we've been using is
• 19:25a fluorescent dopamine sensor.
• 19:26This one is called delight.
• 19:28It was developed at UC Davis by
• 19:30Lindsay Angela and what this is is
• 19:33it's a modified D1 dopamine receptor
• 19:35that when it binds to dopamine it.
• 19:38Laura says.
• 19:38And so this fluorescent sensor is
• 19:40really great because we can inject
• 19:42it in with a virus into the brain.
• 19:44We build fiber, photometry, systems.
• 19:46I know a lot of people are using these,
• 19:49but what it allows us to do is in awake
• 19:51and behaving animals during these
• 19:53discrete aspects of this behavioral task,
• 19:55is record fluctuations in joking that
• 19:57happened through this kind of fluorescent
• 19:59response that isn't interfering?
• 20:00Electrical signals and also the
• 20:02great thing about these optical
• 20:03sensors is they have really great
• 20:05signal to noise and so you can get
• 20:07single trial responses which with
• 20:08a lot of voltammetry in the past,
• 20:10which is my backgrounds you didn't get,
• 20:12you had to average responses and
• 20:14what I'll show you is a lot of what
• 20:16we see is these really rapid changes
• 20:18in dopamine that are happening on
• 20:19the trial by trial basis that we
• 20:21think are really critical for
• 20:23this behavioral response.
• 20:26So we started with kind
• 20:27of what everyone is done.
• 20:29Before this you know it's always good
• 20:30when you start with like new tools to
• 20:33make sure you see what everybody else's.
• 20:35And So what we did is we recorded domain
• 20:37responses during the pre training session.
• 20:39The first time the animals had been
• 20:41in these operating chambers and post
• 20:42training after the animals had learned
• 20:44and so not surprising animals learn to
• 20:46know spoke during a queue for sucrose we
• 20:48can change the length of the queue to
• 20:50make the task more or less difficult.
• 20:52We kind of did this so we had some
• 20:55dynamic range of whether they did the.
• 20:57Miss trials or not,
• 20:58and so we did some machine learning.
• 20:59I won't show you then.
• 21:01This was actually a really
• 21:02great tool for that,
• 21:03but what we do is we see kind of the
• 21:05same thing everyone else is seen.
• 21:07Early on,
• 21:08when the animals go into the sucrose port,
• 21:11so red in here this is more
• 21:13domain response over trials.
• 21:14When they go into the Super sport,
• 21:16you get this robust domain
• 21:18response to the sucrose.
• 21:20Overtraining this signal
• 21:21moves back to the cube.
• 21:23That's very predictive.
• 21:23So now you get this really robust
• 21:26domain response to the queue,
• 21:27but not as much of a domain
• 21:30response to the sucrose.
• 21:32Great, it looks just like that equation.
• 21:34I showed you.
• 21:35Dopamine goes up to the Q goes
• 21:37down to the error signal.
• 21:40All is well in the reward domain
• 21:43does reward based learning field.
• 21:45But then we moved on to this
• 21:47other behavioral task.
• 21:48So again,
• 21:49the animals can know spoke
• 21:50during this Q and they do it to
• 21:53avoid a series of foot shocks.
• 21:54So if they don't press during
• 21:56the Q they get shocked.
• 21:58There's a series of shocks they
• 21:59can press anytime during this
• 22:01series to terminate the shocks,
• 22:02and so we did the same thing
• 22:04we recorded early in Leanne.
• 22:06Learning animals actually learn.
• 22:07When I started my lab a bunch of behavior.
• 22:10People told me that animals
• 22:11will never do this.
• 22:12Mice do this really great.
• 22:14They'll learn really rapidly
• 22:15to press on the nose,
• 22:17poke that active know spoke
• 22:18to avoid the shocks.
• 22:19And they actually at the end
• 22:21of these trials are doing this,
• 22:23that they are not getting shots at all,
• 22:25and so they learn this very fast.
• 22:27And it's actually really robust task
• 22:29for generating motivated behavior.
• 22:30Um?
• 22:31OK,
• 22:31so the first thing we saw which goes
• 22:34in the face of dopamine encoding
• 22:36rewards is that you get this really
• 22:39robust positive response to a foot shock.
• 22:43So dopamine goes up when
• 22:45aversive stimuli are encountered.
• 22:47Um,
• 22:48other people have seen this,
• 22:49but what's really interesting is I'll
• 22:51remind you of what that model looked like.
• 22:53If dopamine is doing reward based learning.
• 22:56We didn't get this robust
• 22:58response to the Q like we did with
• 23:02sucrose overlearning.
• 23:03We did get a decrease in the response.
• 23:05We have the safety queue
• 23:07that came on when the
• 23:08animals avoided the negative consequences.
• 23:10So at the end of the trial.
• 23:13It did go down over learning like
• 23:15you'd expect of an error signal,
• 23:17but here's the problem.
• 23:18The safety queue domain response
• 23:19was the biggest on the first
• 23:21trial they ever encountered it
• 23:22before they could know its value,
• 23:24and so we were kind of a little
• 23:26bit hesitant about that.
• 23:27But we said OK, but maybe this looks kind of.
• 23:30Maybe it fits.
• 23:31But then what we found was that Adobe
• 23:34response to the foot shot during
• 23:36these trials was not only positive,
• 23:38it actually increases.
• 23:40Animals got better at the task and so.
• 23:43We've looked and we said,
• 23:45OK, this doesn't really fit.
• 23:46People had seen this safety Q response
• 23:48before and said look doping doesn't
• 23:50work of RP in aversive contexts.
• 23:52But we looked at the other parameters
• 23:55that this doesn't really make sense.
• 23:57So now we have this situation
• 24:00where dopamine responses in the
• 24:02nucleus accumbens track these
• 24:05prediction error based computations.
• 24:07But only in contexts that are reward
• 24:10based and so everyone has really kind
• 24:13of design these experiments to parse
• 24:15weather domain does RP not does all
• 24:18of what doping does fit these computations.
• 24:21So we had a problem.
• 24:24This reward based Association model
• 24:26was just too simplistic to account
• 24:28for what domain was doing in the same
• 24:31animals in his behavioral tasks.
• 24:33And what we started looking through
• 24:35the literature is a lot of people.
• 24:37What they did is once we have this RP
• 24:39hypothesis reward prediction error.
• 24:41Apophysis people started saying OK,
• 24:42well reward fish based dictionary does.
• 24:44This. Does dopamine look like this?
• 24:46And the issue with that is that
• 24:48dopamine does a lot of stuff that
• 24:50reward prediction error cannot do,
• 24:52and so we ended up doing all
• 24:54this broad prediction error.
• 24:55Modeling those that math cannot
• 24:56do negative reinforcement.
• 24:57So we ended up at this problem where
• 24:59if we wanted to understand what domain
• 25:01was doing from a computational model.
• 25:03These models didn't even make
• 25:06the computations we needed.
• 25:07So.
• 25:08We decided to I have a
• 25:10postdoc who's fantastic,
• 25:12who's a computational psychologist.
• 25:13I will not take credit for
• 25:15developing the model.
• 25:16This is not my backgrounds,
• 25:18but we've had a really great
• 25:20synergistic relationship.
• 25:21And So what we did is we created a
• 25:23complex model of learning and memory.
• 25:26You have the theoretical components of
• 25:28this model that are developed from site.
• 25:30Many years of psychology research
• 25:32and then what we can do is we can
• 25:35record domain responses and many many
• 25:37Contacts and we can map the domain responses.
• 25:40On to the parameters of this model
• 25:42that we know is capable of modeling
• 25:45the behavioral outputs we have.
• 25:47So I'm not going to go into
• 25:49details like super details,
• 25:50but I'm really happy for people if
• 25:52they have questions to talk more
• 25:54essentially with the model does is
• 25:56it models the behavioral responses,
• 25:58the outcomes,
• 25:59the predictions like people have done before.
• 26:01We actually have those prediction
• 26:02based learning algorithms.
• 26:03But one thing it has that's actually
• 26:05was been has been found over years.
• 26:08So you really critical component of learning.
• 26:10Is this perceived salience term?
• 26:11And so this is kind of you know how
• 26:14attention grabbing something is and
• 26:15so it's really highly influenced by
• 26:17things in the environment like novelty.
• 26:20The first time you experience an unexpected
• 26:22aversive foot shock or something,
• 26:23it's it's more attention grabbing
• 26:25the 10th time you present it,
• 26:26and so this that with this term does
• 26:28is it influences how we learn based
• 26:31on these other factors that are
• 26:33not included in these other models
• 26:34and what it does is it's able to
• 26:37make really accurate predictions of
• 26:39what animals will do in the future.
• 26:42Again, we use this model to figure out
• 26:44what experiments would dissociate these
• 26:45different factors from one another.
• 26:47So after I show you this,
• 26:48you can ignore the model stuff
• 26:50if you want and just look at the
• 26:52experiments we run to parse.
• 26:54Kind of what's going on.
• 26:56The really important thing about this
• 26:58model is the simulations from the model.
• 27:00The behavioral output
• 27:01simulations are in grey,
• 27:03and the behavioral data itself
• 27:04is in blue and you can see it can
• 27:07start to model these things that
• 27:09couldn't be modeled before,
• 27:11so things like the animals to train
• 27:13to know spoke for sucrose and then
• 27:15we have we introduce an aversive
• 27:17foot shock all the sudden.
• 27:19The animals responding goes down.
• 27:20That's what the model suggests.
• 27:22We can model how animals will learn
• 27:24and negative reinforcement Contacts
• 27:26the removal of an aversive stimulus.
• 27:28And so we're now able to model the
• 27:31behavioral outcomes of these more
• 27:33complex behaviors but use these same
• 27:35kind of computations to not punch to me so.
• 27:38What I'm going to show you is that
• 27:40dopamine does not track reward
• 27:42prediction error.
• 27:42It tracks perceived salience,
• 27:44which is just kind of like how
• 27:46attention grabbing a stimulus is.
• 27:48Perceived salience is the perception
• 27:51of housing question.
• 27:52Yes,
• 27:53just a quick question.
• 27:55How different is this model from
• 27:58Pierce Hall Macintosh model of?
• 28:02It's it's actually very similar,
• 28:04so it includes the.
• 28:05So the thing about the Pearsall Macintosh
• 28:07models for the people who don't know is
• 28:09they include these attentional terms,
• 28:11which are really actually important
• 28:12for things like Leighton ambition,
• 28:14things that these other models can't do.
• 28:16It includes the same kind of
• 28:18computational terms, and this perceived
• 28:19salience term is essentially kind
• 28:21of what was added to that model.
• 28:23It's based on a neural net model,
• 28:25so it's a little bit different in the math,
• 28:27but it's the same kind of idea,
• 28:30and that's it that the
• 28:31attentional value of things.
• 28:32Are going to influence the associative
• 28:34strength and how animals learn,
• 28:35and so that's that's what we're adding
• 28:38that allows us to do these things.
• 28:40And then on top of that, we've added these.
• 28:42They operate probabilistic responses because
• 28:44pavlovi and learning is not probabilistic,
• 28:45and so we've added that to that.
• 28:47To that kind of framework.
• 28:49So it's very, very similar to that.
• 28:51Thank you for that question.
• 28:52I don't go into too much detail 'cause
• 28:54people don't always know about these models,
• 28:57but that's actually kind of the
• 28:58framework by which we're using it.
• 29:00More appears Hall that the Macintosh,
• 29:02though.
• 29:04OK, so we can model the behavior we need,
• 29:07so so again,
• 29:08this perceived salience is kind of
• 29:10like a driving animals attention
• 29:11towards States and so it's really
• 29:13highly affected by the in physical
• 29:15intensity of a stimulus and the
• 29:17novelty and environment that
• 29:18changes how you attend to stimulate.
• 29:21And So what we did is we started.
• 29:24We said OK,
• 29:25if it's tracking just the saliency,
• 29:27it should increase with physical
• 29:29intensity of a stimulus.
• 29:30Whether it's positive or negative.
• 29:32And so this is a increasing
• 29:34series of foot shocks on the left.
• 29:37That is,
• 29:38the simulations from that perceived
• 29:40salience term of our model,
• 29:41and on the right is the actual
• 29:44dopamine recorded responses to
• 29:46increasing intensities of foot shocks
• 29:47and what you see is that dopamine
• 29:50increases with foot shock intensity.
• 29:52It also increases when we
• 29:55increase the volume of sucrose.
• 29:57So a better than expected appetitive.
• 30:00Reward or the volume of quiet Night,
• 30:02which is a bitter taste scent,
• 30:04and so everyone always asks.
• 30:06And it's a great question.
• 30:07Foot shocks are kind of
• 30:09weird aversive stimuli,
• 30:10so for many of these tasks we have
• 30:12worked in other things that are aversive.
• 30:15But not, you know,
• 30:16painful.
• 30:16We can argue all day about whether
• 30:18foot shock is pain or something else,
• 30:20but we see the same pattern with
• 30:22other types of aversive stimulation,
• 30:24and so this is ruling out simple,
• 30:26rewarding coding by dopamine,
• 30:28because dopamine in the incumbents
• 30:29is going up to both appetitive
• 30:31and aversive stimuli.
• 30:32So it cannot be just rewards.
• 30:36So the other thing that influences
• 30:39perceived salience is novelty.
• 30:40So how much experience
• 30:42you have with something.
• 30:43So what we did is we took foot
• 30:46shocks of the same intensity.
• 30:49Intensity is not changed and we repeated
• 30:51them in a series on a fixed interval,
• 30:54so every every 60 seconds
• 30:56the animals got a foot shot.
• 30:59What we found is that doping goes down
• 31:02to the foot shots even though the
• 31:04intensity of the foot shock stays.
• 31:07Constant, so it's also not just
• 31:10encoding only the intensity,
• 31:12it's encoding other aspects
• 31:14like novelty as well,
• 31:16and so you get decreased domain
• 31:19responses to repeated exposures
• 31:22of stimulate both aversive.
• 31:24And neutral simulate,
• 31:25and so this is an auditory tone,
• 31:28so we've done this with tones,
• 31:30lights, and white noise.
• 31:32This is an example from white noise,
• 31:34but what you see which is
• 31:37important is the first exposure
• 31:39of a stimulus that is neutral,
• 31:41elicits dopamine and repeated exposures
• 31:44of that stimulus go down overtime.
• 31:47So you're getting decreased dopamine
• 31:49when animals are exposed repeatedly
• 31:51to stimuli in the environment,
• 31:53regardless of whether they have
• 31:55positive value, negative value,
• 31:57or are what we think of as neutral.
• 32:01I'm.
• 32:02So another aspect that I think,
• 32:05and this is really important experiment
• 32:07because it really rules out this
• 32:09reward prediction error based learning.
• 32:11So what we did is we trained animals to
• 32:13know spoke during a discriminative cue.
• 32:16So auditory cue comes on.
• 32:17If they respond they get sucrose.
• 32:21Without any signal,
• 32:22we now switch it to the same auditory cue.
• 32:25If they press,
• 32:27they get shocked,
• 32:28so we're switching that the Q is
• 32:30the same and it now represents
• 32:33a worse than expected outcome.
• 32:36And so.
• 32:36What should happen in a task
• 32:38like this is that the animals
• 32:41in this grezar simulation,
• 32:42the animal should reduce
• 32:44their behavioral responding.
• 32:46They do not surprisingly,
• 32:47this is a traditional kind
• 32:48of punishment task.
• 32:49Animals will reduce their behavior.
• 32:52But what our model predicts,
• 32:54this perceived salience model.
• 32:55Is that because there's unexpected
• 32:57information and it's novel?
• 32:59There should be increase
• 33:01in dopamine to this Q.
• 33:03Prediction error responding with
• 33:05say it should be decreased because
• 33:07it's a worse than expected outcome.
• 33:10A reward prediction error, excuse me.
• 33:12So what we're going to look
• 33:14at here is this Q,
• 33:16which is the last Q that predicted sucrose,
• 33:18so this is before the animals
• 33:20got gotten shocked.
• 33:21So this Q still has that predict sucrose,
• 33:24and then we're going to look at the next
• 33:27Q right after the first foot shock.
• 33:30And what we find is first,
• 33:33this foot shock causes a
• 33:35positive domain response.
• 33:36So it doesn't matter what kind of task,
• 33:40but shocks are being presented and
• 33:42they're always words resulting
• 33:44in positive domain responses.
• 33:46What the dopamine response to this
• 33:48discriminative cue actually goes up,
• 33:50even though it's it represents a
• 33:53worse than expected outcomes and
• 33:55so dopamine is increasing anytime
• 33:58information is novel or salient
• 34:00to the animal.
• 34:01And it's increasing even if
• 34:03the outcome is worse than
• 34:05expected or better than expected,
• 34:07and one of the key aspects of this
• 34:10experiment is dopamine is going up,
• 34:12even though the animal's
• 34:14behavior is going down,
• 34:15so increases in dopamine don't just
• 34:17mean motivated behavior or approach
• 34:19because we're getting increases in
• 34:21dopamine here that correlate with animals
• 34:24inhibiting a behavioral response,
• 34:25and so this kind of saliency.
• 34:27What it'll do, is it helps animals make
• 34:31adaptive updating of responses were.
• 34:33List of what the context of those responses
• 34:36are or the behavioral response necessary.
• 34:39So I'm gonna show I think
• 34:41I've one more experiment,
• 34:43so this experiment is
• 34:45actually really important,
• 34:46because it's kind of shows how much these
• 34:49kind of novel salients events that don't
• 34:53necessarily acquire value R to animals.
• 34:56So what we did is we did an experiment
• 34:59where we associated ECU with a foot shock,
• 35:02so just fear conditioning.
• 35:04But what we did is on some of
• 35:07the trials we just put a random
• 35:10irrelevant house light on.
• 35:12And what the model predicts is that because
• 35:14there is novelty in the environment,
• 35:16there will be an increase in dopamine
• 35:18response on these trials where novel
• 35:20information is added even though previous
• 35:23work has shown that novel irrelevant
• 35:25information will not acquire value.
• 35:27So the Q that we're adding this random
• 35:29irrelevant light won't acquire value
• 35:31because the animals already associated
• 35:34the previous Q with the foot shot.
• 35:36And So what we find is on trials
• 35:38where we add this novel light.
• 35:41There is a very large increase
• 35:43in the domain response,
• 35:44even though that novel light
• 35:46won't acquire value itself.
• 35:47And So what this does is it rules
• 35:50out the simple Attribution of some
• 35:52sort of balance to a queue or
• 35:55associative strength of that Q.
• 35:57It means that you're getting
• 35:59increases in dopamine responses.
• 36:00They don't necessarily correspond with
• 36:02the animal making Association between
• 36:05that queue and the outcome in some cases.
• 36:11The last thing, which again for
• 36:12people who are are really deep
• 36:14in the prediction based field.
• 36:16One thing that people can
• 36:17say at this point is, well,
• 36:19maybe doping is doing prediction error
• 36:21but not reward prediction error.
• 36:22So it's going up every time
• 36:24there is an error in prediction.
• 36:26This is actually really good good thought
• 36:28and we thought this too and we said OK.
• 36:30Well let's let's see if that is the
• 36:33case and we were kind of Gnostic here.
• 36:35We were saying, OK,
• 36:36let's just figure out what it does.
• 36:38We're not trying to.
• 36:40Pusha theory we're saying, well,
• 36:41how does the data fit together?
• 36:44So saliency or perceived salience?
• 36:46What it would suggest is that when
• 36:48you have a stimulus like a foot shock,
• 36:52you should have the biggest opening response,
• 36:55because when the stimulus is
• 36:56present and there it's the most
• 36:59salient 'cause most intense.
• 37:00But if you have a prediction of
• 37:03that during extinction or omission,
• 37:05there should still be a positive
• 37:08doping response,
• 37:08but it should be lower than when the
• 37:11stimulus is physically there a prediction.
• 37:14Error hypothesis would be that
• 37:16when you have an omission,
• 37:17this response should be higher
• 37:19than when the stimulus is there
• 37:21because it signals an error.
• 37:25We did this experiment.
• 37:27What we found was that there's a
• 37:29positive domain response at the time
• 37:31of the predicted foot shock when
• 37:33it's omitted, so it's not there,
• 37:35but the response of the foot shock
• 37:38itself is higher than when it's omitted.
• 37:40And so this also rules out
• 37:43other competing theories,
• 37:44which is that domain does
• 37:45prediction error learning,
• 37:46but it's not reward based.
• 37:50So. And I showed you a lot of stuff and,
• 37:54well, I kind of rule kind of come
• 37:56back and say like why should you care?
• 37:59So essentially what we did is we did
• 38:01a number of experiments to rule out
• 38:03these kind of competing factors of what
• 38:05dopamine is doing in learning and memory.
• 38:07Don't mean release is doing.
• 38:08I'm not saying the VTA cell
• 38:10bodies don't do this.
• 38:11Maybe they do,
• 38:11but there's integration of information
• 38:13at the level of the terminal that
• 38:15dictates how doing is actually
• 38:16releasing these brain regions and
• 38:17what I am saying is that dopamine
• 38:19release in the nucleus accumbens,
• 38:21core Maps on true perceived.
• 38:22Salience not prediction error or value.
• 38:27Our models are modeled behavior,
• 38:28we just use it to generate experiments
• 38:30we should run to parse different
• 38:32aspects of domain encoding,
• 38:33and So what you can do is use
• 38:35these kind of predictions to make
• 38:37experiments with other circuits as well,
• 38:39which I think is kind of an interesting
• 38:42way to approach the question.
• 38:45But what I'm showing you is that
• 38:47even in reinforcement context,
• 38:48pavlovi in context,
• 38:49this isn't a value based prediction signal,
• 38:51and these same signals are
• 38:53there in punishment tasks in
• 38:54negative reinforcement tasks,
• 38:55and so it's actually really
• 38:57interesting that you're seeing
• 38:58this kind of dopamine signal.
• 39:00It's very critical in driving behaviors,
• 39:02just not in the way that I
• 39:04think we predicted before.
• 39:07So why should we care?
• 39:08I think understanding what domain is
• 39:11doing is really important for disease,
• 39:13and so if you want to understand what
• 39:15dopamine is doing and what deficits in
• 39:18dopamine in a patient mean for that patient,
• 39:21it really requires a kind of
• 39:23holistic understanding.
• 39:24What domains doing across contexts and
• 39:26internal States and things like that,
• 39:28and so you know when you have a model and
• 39:32you say doesken dopamine fit this model,
• 39:35the answer might be yes.
• 39:37But it kind of leaves out that aspect of.
• 39:39But what is domain doing in other
• 39:42contexts of the model can't fit and
• 39:44so understanding the components that
• 39:46dry of these behaviors is really
• 39:49critical to understanding this.
• 39:51But I think that maybe more
• 39:53important thing for for kind of
• 39:55human health is is from, you know,
• 39:57my primary field which is
• 39:59addiction an it's understand.
• 40:00The difference between a dopamine
• 40:02signal that signals were worn
• 40:04and what a salience signal does.
• 40:05So a reward signal.
• 40:07If you have a deficits in a reward signal,
• 40:09you may say you know we don't
• 40:11want to increase those and people
• 40:13suffering from substance use disorder
• 40:15because if we do that may increase
• 40:17the rewarding value of stimuli
• 40:19in the environment like drugs.
• 40:21But the issue is with the salience signal.
• 40:24If you have deficits,
• 40:26it's going to slow the rate of learning
• 40:29for everything in the environment,
• 40:31so it could explain why people
• 40:34are compulsive because they don't
• 40:36respond to negative outcomes.
• 40:38It would explain why they have trouble
• 40:41learning. The adaptive alternatives.
• 40:42Are there an?
• 40:43It would explain why extinguishing
• 40:46drug associations is much slower,
• 40:48and so in if it's a saliency signal,
• 40:51we may want to.
• 40:52Increased opening so that people can
• 40:54learn adaptively in all of these contexts.
• 40:57And so again,
• 40:58I'm not saying that like you know,
• 41:00this is the end all be all dopamine is
• 41:02in lots of projection targets and it
• 41:04does lots of things in different areas.
• 41:07And we're in one single area,
• 41:08but I think kind of taking a step
• 41:10back and thinking about what these
• 41:12domains signatures really mean
• 41:14and what those deficits would
• 41:15mean to a behaving individual,
• 41:17as I think it's really important
• 41:19component of conceptualizing
• 41:20what these you know psychiatric
• 41:22deficits mean to people and how
• 41:24to best treat them.
• 41:25Anyway, so with that I'll
• 41:27end with thanking my lab,
• 41:29so Ganesh clue his background
• 41:30is in computational psychology,
• 41:31so he's like the modeler and he's
• 41:34really like driven this you know.
• 41:35Together he's a Pavlovian guy.
• 41:37I was a reinforcement person.
• 41:39I think this was like one of
• 41:40those projects that was this great
• 41:42synergism between two people who
• 41:44just we're really excited about.
• 41:46Kind of figuring out what's going on.
• 41:48Jennifer, Zachary and Patrick and
• 41:50Stephanie were are grad students
• 41:52that were working on this project
• 41:54and put a lot of time into it.
• 41:56Cody Siciliano and Lindsay Ann Lynn
• 41:58was was really nice and was helping us
• 42:01get the delight these optical sensors
• 42:03up and running in the lab fairly
• 42:06early in Cody's at optical engineer.
• 42:07So he helps a lot at Vanderbilt with
• 42:10getting these working correctly,
• 42:12validating them an my funding and I
• 42:14can take any questions you may have.
• 42:17So thank you so much.
• 42:22Thank you so much Aaron.
• 42:23If anybody doesn't want to pipe
• 42:25up with the just asking questions,
• 42:27please put them in the chat and
• 42:30I can read them out for Aaron.
• 42:33Like can I start with one?
• 42:35Go for it. So in terms of those
• 42:38projections you were discussing
• 42:40and you're looking in the core,
• 42:42do you think that just with predictions
• 42:44of what the core versus shell of the
• 42:47incumbents does in activations of the
• 42:49core versus Shell does to behavior?
• 42:51Do you think that one might be
• 42:54more important than the other in
• 42:56the prediction error?
• 42:57This is actually really
• 42:59great question an I should
• 43:00have pasted this.
• 43:01Actually I can do it now.
• 43:04I we have shell data so so.
• 43:06The answer is probably yes in some contexts,
• 43:09although when we we started to
• 43:12look through the shell data.
• 43:15So I just wanted to side when
• 43:17we started to look through the
• 43:20new data or the shell data.
• 43:22It did not look like what
• 43:24I would expect either.
• 43:28Basically. It still doesn't
• 43:31look like prediction error.
• 43:33So we still get a positive
• 43:36domain response to the shock.
• 43:38We get some scaling with stimulus intensity.
• 43:41Not quite as much.
• 43:44People have shown decreases in dopamine
• 43:46and to aversive stimuli and we have
• 43:49been working out why that would be when
• 43:51we aren't seeing them and we think.
• 43:53And so basically we did show
• 43:55decreases in domain in some contexts.
• 43:57Dopamine goes down when animals don't
• 44:00have to do anything or they have to wait.
• 44:02So what we did is we design
• 44:05this other experiment that I'm
• 44:06like really excited about.
• 44:08What we did is we trained animals to know
• 44:11smoker sucrose and then we switched the
• 44:13contingency so that they had to with.
• 44:16Hold a response and wait to get sucrose.
• 44:18So this is kind of like the same reciprocal
• 44:20thing to fear conditioning right?
• 44:22You have a queue.
• 44:23The animal just waits to get shocked.
• 44:25There's nothing they can
• 44:26do during that period.
• 44:27They wait,
• 44:28we see decreases in dopamine to
• 44:30fear conditioning Q and two AQ,
• 44:31where the animal gets sucrose at the end,
• 44:34but they have to wait to do it.
• 44:37And So what we think is happening is a
• 44:39lot of these decreases in domain people
• 44:42have seen or not necessarily just value.
• 44:45They have to do with what animals
• 44:47are doing and what novelty in
• 44:49salience do in an environment
• 44:51is they increase exploration.
• 44:53So if you need to decrease exploration
• 44:55and just wait for something to happen,
• 44:58domain goes down.
• 44:59So we get doping reductions even when
• 45:02the outcome is positive when the task
• 45:04design mimics that of the aversive
• 45:06tasks where people have seen reductions.
• 45:09And a lot of the like you know,
• 45:11there's a lot of great work from
• 45:13like Mitchell White Men looking at
• 45:15aversive like wine in the mouth.
• 45:17It's unavoidable.
• 45:17The animals are just waiting there as well.
• 45:20And so I think there's also
• 45:21differences in relative,
• 45:22you know,
• 45:23in his designs he has positive and
• 45:25negative stimuli in the same task,
• 45:27which are a little bit different
• 45:28than having an animal behaves.
• 45:30So obviously there's there is some
• 45:31sort of value based computation,
• 45:33but we think they're done and
• 45:35really specific context.
• 45:38Thank you.
• 45:39I think the other question go ahead.
• 45:44But they are zoom etiquette.
• 45:46Go ahead, I'll jump in after you.
• 45:49OK, thanks for bringing talker
• 45:50and that was really really cool.
• 45:52So one thing that you can do with
• 45:55these models is sort of see if they
• 45:57can predict particular phenomena
• 45:58and one of the ideas I think that's
• 46:01becoming more and more prevalent
• 46:02about a sort of teleological idea
• 46:04about what the dopamine system
• 46:06might be for is not rewards,
• 46:08not punishments as you've argued,
• 46:09but actually the causal
• 46:10structure of the world.
• 46:12Yeah, and so does your model
• 46:14predict things like sensory
• 46:15preconditioning where?
• 46:16There was
• 46:16no value at all.
• 46:18Initially you use that information
• 46:19later to imbue or impede value.
• 46:21Yes, yes, so actually this is one
• 46:23of the other powers of this model is
• 46:25it can do sensory preconditioning,
• 46:28Layton addition,
• 46:28so these are two things that that even
• 46:31the temporal difference models cannot do,
• 46:33and the problem is dopamine does them.
• 46:36So if dopamine does these
• 46:37in a computational model,
• 46:39can't that cannot be the
• 46:41computation domain is doing,
• 46:42and so we we have our next at
• 46:45once we get this out the door.
• 46:47I'm trying to find my.
• 46:49We haven't started doing sensory
• 46:50preconditioning yet because of the fact that
• 46:53it's a little bit more of a pain in mice,
• 46:56and I think we're going to
• 46:58have to switch to rats.
• 47:00Mice aren't like the best set
• 47:02like attending to things so,
• 47:04so lame.
• 47:05Inhibition is something that
• 47:06our model does do.
• 47:08So late inhibition is actually
• 47:10this really interesting novelty
• 47:11based learning constructs were
• 47:13essentially pre exposed stimuli,
• 47:14acquire values slower than simulated.
• 47:16Have not been pre exposed so familiar
• 47:18stimuli take longer to because you're
• 47:20basically unlearning the no Association.
• 47:23So a lot of these different models
• 47:25you know we brought this up earlier.
• 47:28I was asked about Pierce Hall Macintosh.
• 47:30Like all of these models have
• 47:33added these components to do this
• 47:35and our model does this and the
• 47:37sensory preconditioning.
• 47:39I showed you the dobine goes
• 47:41down to repeated shocks.
• 47:42Um?
• 47:42We can get pre exposed stimuli to
• 47:45have less associative value and
• 47:47what's really interesting is that
• 47:50the dopamine response to these pre
• 47:53exposed stimuli is much lower and it
• 47:56also tracks over the pre exposure period.
• 47:59So these these kind of non value based
• 48:01learning constructs were previous
• 48:03experience is changing the way that
• 48:05stimuli can drive future behavior
• 48:07or sensory preconditioning were two
• 48:10irrelevant simulate form associations
• 48:12that can then be associated later.
• 48:14Our model does it,
• 48:15and dopamine still Maps onto that
• 48:17perceived salience term in those contexts.
• 48:19And this is actually why we were so set on.
• 48:21You know,
• 48:22the first experiments really well
• 48:23it could be other things too,
• 48:25and then we started going
• 48:26into these latent addition and
• 48:27sensory preconditioning ideas,
• 48:29because those really can't be other things.
• 48:31I mean,
• 48:31it could be there's other components of it,
• 48:34but I think it is more strong
• 48:35with the other stuff.
• 48:37It does too, that that's what it's doing.
• 48:39But it's that's a great, I think.
• 48:41Those are like the killer like knife
• 48:43in the coffin experiments, right?
• 48:44Because they just.
• 48:45Those other models cannot do them,
• 48:47so yeah, that's a great great point.
• 48:51I think we had a question,
• 48:52thanks. Those are really great talk here
• 48:54and thanks for taking us through all that.
• 48:56And you have touched on my question
• 48:58a little bit because you started.
• 49:00I mean, even with the default
• 49:02Pomeranians when he starts to
• 49:03look at nucleus accumbens shell.
• 49:04But one thing I was curious about with your
• 49:07model is how how global do you think it is?
• 49:10And you hinted as we talked about the
• 49:12VTA component as well. So you know,
• 49:14going back to some of the Bromberg,
• 49:16Martin work about different VTA
• 49:17neurons responding differently
• 49:18to balance versus silence.
• 49:19You think this Maps onto multiple regions
• 49:21using this sub regions within the.
• 49:23The common score is a global number
• 49:25as a discrete to specific places.
• 49:27Yeah, so this is a great question,
• 49:29so I'm lucky to have married very well in
• 49:32my life and my partner is a two photon guide.
• 49:35When he does,
• 49:36he does 2 photon imaging through grin lenses,
• 49:39cranial windows, you name it and
• 49:40so one thing we're working on now.
• 49:43The really power of of these kind
• 49:45of optical imaging approaches is
• 49:47that you can record as small as
• 49:49you want or as big as you want,
• 49:51and so depending on your.
• 49:53Microscope and resolution so
• 49:54we're moving into these.
• 49:55Either I love slice work.
• 49:57This is like my background so
• 49:59we're moving into these kind of.
• 50:00In vivo in slice imaging approaches
• 50:02to understand better domain
• 50:04regulation across big and small areas.
• 50:06Because the thing about domain
• 50:07neurons that so you know,
• 50:09like kind of weird about
• 50:11them is their projection.
• 50:12Like you know, Arborization is insane.
• 50:14If you fill a single domain on the PTA
• 50:17and look at the field that it populates,
• 50:20it's like half the straight up.
• 50:22But then if you look at these specific
• 50:24release sites on these neurons,
• 50:26it's not releasing dopamine
• 50:27at everywhere every time.
• 50:29It's depending on all these different things.
• 50:31So this.
• 50:32Release structure is so complicated and
• 50:33I think part of the reason people have
• 50:36been so like oh volume transmission
• 50:38is our ability to really look at these
• 50:40granularities between these components.
• 50:42And so we're starting to go,
• 50:44you know, start big.
• 50:45We're just saying,
• 50:46OK,
• 50:46if we do image in a bigger field or
• 50:49with multiple sites at the same time,
• 50:51are we seeing differences?
• 50:52We do see differences so I think there
• 50:55are differences in these VTA neurons in
• 50:57what what they're doing in different areas.
• 50:59So I don't think like all dopamine is.
• 51:02This I think don't mean to the core is this,
• 51:05but it also makes sense that domain
• 51:06in the core that's been tide more to
• 51:08instrumental responding than like the shell.
• 51:10That's like these acquisition.
• 51:11And like Valeant space kind of learning
• 51:13would look like a perceived salience term,
• 51:15right?
• 51:15That makes way more sense for something
• 51:17that is involved in punishment,
• 51:18negative reinforcement,
• 51:19positive reinforcement,
• 51:19which are the same motivated responses,
• 51:21independent violence.
• 51:22So we're doing some more stuff in the
• 51:24shell. You know, I'm not sold that the
• 51:26shell doesn't do value because I don't think
• 51:28foot shocks are the best way to do stuff.
• 51:30I think foot shocks are weird.
• 51:32Stimulus that are really powerful initially,
• 51:34but we didn't really evolve
• 51:36to respond to foot shocks,
• 51:37so we're starting to go in
• 51:39more with things like Quy 9.
• 51:41We've been developing.
• 51:42You can make them liquor lic ometer hot,
• 51:45so we've been doing like thermal,
• 51:47like not pain, but thermal sensitivity
• 51:49curves so that you can look at thermal
• 51:51stimuli that reduced responding.
• 51:53But without this, like foot shot component,
• 51:55so we're trying to parse this out.
• 51:57I'm not sold that,
• 51:58it's just like every dopamine responses
• 52:00that I think it's more complicated,
• 52:02but I think we need better resolution
• 52:05techniques to really parse that.
• 52:07And hopefully over the next I don't know.
• 52:09However long my career last
• 52:10will see will start to get it.
• 52:11Some of those questions and
• 52:12other people are doing that too.
• 52:14I mean,
• 52:14there's some really great work
• 52:15coming out where people are
• 52:16using those like single synapse.
• 52:18You know,
• 52:18you know Uncaging and Eli and
• 52:19all kinds of crazy stuff,
• 52:21so I'm excited to see where the field goes.
• 52:23Yeah,
• 52:23that's really excited how
• 52:24great you are looking
• 52:25at. I'm glad you're looking
• 52:26into it. Sounds like
• 52:27you thought about it already. Definitely
• 52:28I went a little bit. Now
• 52:29the question is just like do we
• 52:31have the tools and then the next
• 52:33thing is do we have the month of
• 52:34money and the people that do it?
• 52:36And so it's like you know.
• 52:38You you see what you can do and
• 52:39with the resources you have so.
• 52:44There is a question in the chat by
• 52:47from Denise, Baghdad and Denise.
• 52:48Do you want to read it out or
• 52:50would you like me to ask it?
• 52:53Good morning, great talk,
• 52:56so I just wanted to understand something.
• 53:00Maybe it's not a great question.
• 53:03Let me just say it.
• 53:06So nicotine reinforcement is
• 53:09generally considered as positive
• 53:11reinforcement enforcement.
• 53:12However, it's also discussed
• 53:14about like the weather.
• 53:17Nicotine reinforcement is
• 53:18negative reinforcement.
• 53:20Becausw of nicotine withdrawal.
• 53:22The compost.
• 53:23Open intake and taking is actually
• 53:27contributes to nicotine reinforcement,
• 53:29so I am interested in whether your
• 53:33model could dissect the positive or
• 53:36negative reinforcement for the nicotine.
• 53:39It's I know it's your.
• 53:44Shock, but this is the one molecule.
• 53:47You know, so could have both.
• 53:49So how we put the fact
• 53:51that's a great question.
• 53:53So one of the things in the addiction
• 53:55field is that there are all these
• 53:57series of negative reinforcement, right?
• 53:59Like opioid withdrawal alcohol,
• 54:01withdrawal.
• 54:01All of these are negative
• 54:03reinforcement concepts,
• 54:03but no one actually does
• 54:05negative reinforcement.
• 54:06We make the inference that is negative
• 54:08reinforcement from the fact that it
• 54:10causes withdrawn animals are taking
• 54:12it with during the withdrawal period.
• 54:15It's OK,
• 54:15so it's a hard question.
• 54:17I think the first step would
• 54:19be to look at how you know if
• 54:22negative reinforcement processes,
• 54:23like for avoiding shocks,
• 54:25are changed after nicotine.
• 54:26So one of the things that we're
• 54:28working with with Cody Siciliano is
• 54:30looking at how alcohol changes animals
• 54:32motivation for negative reinforcers,
• 54:34and so that's like the first step.
• 54:37I think this is like it's
• 54:39a hard thing to parse.
• 54:41Nicotine is also, I know it's like I do.
• 54:45A cholinergic regulation
• 54:46of dopamine terminals.
• 54:47So nicotine is like in my mind,
• 54:49but like we don't do nicotine
• 54:51reinforcement stuff.
• 54:51It's also this really interesting
• 54:53molecule because it regulates
• 54:54like how dopamine is released
• 54:56in a really interesting way.
• 54:57That's not just like up,
• 54:59it's changing like phasic responses
• 55:00to stimuli in the environment.
• 55:02And so thinking about the
• 55:04interaction between those,
• 55:05it's like we're doing some work
• 55:07with sex differences in that
• 55:08system is like so much more
• 55:10complicated than I want it to be.
• 55:12Like with cocaine,
• 55:13it's like it binds to the transporter.
• 55:15Show me goes up.
• 55:17Can we reduce that nicotine is like Oh
• 55:19well in some cases domain goes down
• 55:22some cases it goes up and so it's
• 55:24just such a complicated question.
• 55:25I think the behavioral stuff we do
• 55:27can start to parse how processes and
• 55:29animals are changing by exposure,
• 55:31and I think that's the first step
• 55:33and then the next step is trying.
• 55:35We're trying to develop task
• 55:36to figure out how to do this.
• 55:38So one thing we've been thinking
• 55:41about is doing.
• 55:42Old school drag discrimination so
• 55:44animals will actually press before
• 55:46like to tell you an internal state
• 55:48is X or Y and what we want to do
• 55:50is we've been thinking about doing
• 55:52this with optogenetics, right?
• 55:53Does an optical stimulation of a
• 55:55circuit substitute for X drug X
• 55:57state and I think that some of these
• 55:58withdrawal effects you could see if,
• 56:00like nicotine withdrawal,
• 56:01substituted for some of these other things,
• 56:03and if that was a critical component
• 56:05of reinforcement isn't hard question.
• 56:07I think that that's it can
• 56:09start answering that question,
• 56:10but I've been thinking about this a lot
• 56:12and I'm not sure how to specifically.
• 56:14Parse when an animal is doing something
• 56:17for two things at the same time.
• 56:18What component is what I?
• 56:20I wish I had better answer.
• 56:22I'm excited about the question,
• 56:23but I don't have the answer for you.
• 56:29Is there time for one more Marina
• 56:32or do we directly is that Beth?
• 56:34No, it's less less sorry.
• 56:37I had one too. OK first Liz then Jane.
• 56:42So Aaron, that was such a beautiful
• 56:43talk and I love all the different
• 56:46behavioral experiments that
• 56:47were inspired by your model.
• 56:48And one of the powers of this
• 56:50model is obviously you could
• 56:52take that salience term out.
• 56:53And guess how behavior would be
• 56:55altered by it in the future.
• 56:57So I'm curious whether you're going
• 56:59to start looking at blocking these
• 57:00signals and seeing whether they
• 57:02match the expectations that the
• 57:04model would make in particular,
• 57:05that when you were showing the responses
• 57:07to the light cue during conditioning,
• 57:09which shouldn't be involving any learning
• 57:11like what's the point of that signal?
• 57:14Our behavior could come from it.
• 57:15I'm
• 57:16so excited to just ask me this one.
• 57:18OK, so there's.
• 57:21I'm lazy and I don't want to.
• 57:22Maybe I'll do this why I
• 57:24should have put these in here.
• 57:25I didn't think people were going to have.
• 57:27Not that I didn't think you
• 57:28would have great questions,
• 57:29but I didn't think you guys were to ask
• 57:32questions that I had like specific data for.
• 57:34OK, so two things. First thing first.
• 57:37Wow, that looks terrible.
• 57:39We did do experiments to
• 57:40eliminate this signal.
• 57:41You can see I'm like really crafty with this.
• 57:44This is not my OK so we First things first.
• 57:47Yes, I'll tell you what I think
• 57:49that signal is doing and then two.
• 57:51Well, two.
• 57:52I'll show you the optic Jenner.
• 57:54Other we're not.
• 57:55We're almost done with the
• 57:56Histology so take this with it.
• 57:58This is preliminary preliminary ish.
• 58:00We did two experiments where we
• 58:02inhibited the signal using what our
• 58:04model would predict as the condition
• 58:06response that would dissociate
• 58:07it from these other components.
• 58:09Injected Halo rhodopsin in TH
• 58:11positive neurons in the VTA and
• 58:13then inhibited the terminal.
• 58:14So we're only inhibiting
• 58:15dopamine releasing terminals.
• 58:16Any comments?
• 58:18We either inhibited during a Q
• 58:22predicting fear conditioning.
• 58:25Or we inhibited or?
• 58:27We know we stimulated during.
• 58:28Sorry this is my fault.
• 58:30We stimulated during these are
• 58:32two different things.
• 58:33We stimulated during a fear conditioning
• 58:36Q or we stimulated channelrhodopsin
• 58:38during an emitted but expected shock.
• 58:41If you stimulate and this gets
• 58:43your questions during a Q,
• 58:45that's a fair condition.
• 58:46Q You actually get less freezing,
• 58:49so this is the opposite of what you
• 58:51would expect from associative strength,
• 58:53but it kind of person.
• 58:55We basically points to this question when
• 58:58there's novel stimuli in the environment,
• 59:00you increase exploration.
• 59:01All of our data,
• 59:03like the novel Q.
• 59:04All of our data show that
• 59:06when you add novelty,
• 59:08you increase dopamine and the increased
• 59:10dopamine is associated with more exploration.
• 59:12And less freezing.
• 59:13And so these novelty terms,
• 59:15what they're doing is they're
• 59:17helping animals to adaptively
• 59:18learn by increasing exploration.
• 59:20And like here, decreasing freezing.
• 59:22So in the same animal,
• 59:23the queue that wasn't stimulated has just
• 59:26as much freezing is the wifey group,
• 59:28and when it's stimulated they freeze less
• 59:31and that's what our model would predict.
• 59:33The other thing we did is we show
• 59:36that we can prevent extinction,
• 59:38freezing extinction by
• 59:39stimulating dopamine to the Q,
• 59:41and so we both prevent extinction.
• 59:43We prevent extinction by basically
• 59:45like increasing the salience of that
• 59:47event so that it doesn't go away,
• 59:49which is not the same as you'd expect
• 59:52by these prediction error terms.
• 59:54So what we think is happening
• 59:55is that these novel stimuli
• 59:57are increasing dopamine that
• 59:58increase in dopamine promotes and.
• 01:00:00Exploration term,
• 01:00:01we also did some like deep lab cut based,
• 01:00:04you know,
• 01:00:05machine learning algorithms to
• 01:00:06look at orienting responses.
• 01:00:08It's not associated with
• 01:00:09general motor activity,
• 01:00:10it's associated with orientation
• 01:00:12towards the novel stimulus.
• 01:00:13And so we think that these these
• 01:00:15this saliency term in the Commons
• 01:00:17is changing the way the animals are
• 01:00:19interacting with the environment
• 01:00:21rather than just the associated value.
• 01:00:23But the problem is these are such
• 01:00:26complex things to dissociate that.
• 01:00:28I understand why people solve
• 01:00:29the data before instead.
• 01:00:31Oh, it's that balance goes up to rewards,
• 01:00:33goes down to a fear conditioning Q
• 01:00:35that looks like violence to me too.
• 01:00:37You only start to see that
• 01:00:38it can't be balanced.
• 01:00:39When you do these kind of really
• 01:00:41in the weeds like someone showed
• 01:00:43this in 1950 in psychology,
• 01:00:44we're gonna do this again with optogenetics
• 01:00:46kinds of experiments which I don't know.
• 01:00:48I think those are the fun experiments,
• 01:00:50but does that answer your question?
• 01:00:51Yes, thank you, awesome.
• 01:00:53I think James next and then Rick.
• 01:00:55Hi that was a great talk.
• 01:00:59So my question is.
• 01:01:02You talk about increasing dopamine,
• 01:01:05and in most of your experiments where
• 01:01:08you're actually measuring dopamine,
• 01:01:10you're looking at.
• 01:01:12Using Delight an if you go back to the
• 01:01:16not the old psychology experiments,
• 01:01:19but the 80s dopamine literature.
• 01:01:21People think about tonic versus phasic,
• 01:01:24dopamine and a lot of your experiments
• 01:01:26seem to be focused more on what
• 01:01:30the phasic dopamine signal is.
• 01:01:32An I'm wondering whether you have
• 01:01:35some way that you can simultaneously
• 01:01:38look at dopamine tone because it
• 01:01:40may be that things like novelty.
• 01:01:43Might actually be linked to some
• 01:01:45of those and you get you know
• 01:01:47interactions between the two,
• 01:01:49and you know when you're you're seeing
• 01:01:51your increase with the delight.
• 01:01:53What baseline is that on?
• 01:01:54This is,
• 01:01:55I'm like you guys
• 01:01:56are like making my day.
• 01:01:58I have like I'm this is
• 01:02:00such a great question.
• 01:02:02So this is like our first like thing.
• 01:02:04We're getting out the door.
• 01:02:06We have a bunch of extra data where what
• 01:02:09we've been doing and this is the thing.
• 01:02:12OK, so voltammetry.
• 01:02:13Is background subtracted?
• 01:02:14So you can't really get both tonan,
• 01:02:17phasic stuff in the same experiment.
• 01:02:19So what people historically done as
• 01:02:21they said, microanalysis histone,
• 01:02:23gigha Ruth and voltammetry is
• 01:02:24is phasic an my the 80s domain.
• 01:02:27Literatures like where I started my
• 01:02:29career so I'm very excited about that.
• 01:02:31Do you like this kind of nice because
• 01:02:34you have some photobleaching but you
• 01:02:36can control for that and you don't
• 01:02:39know what the the problem with it
• 01:02:41is that you don't know the number.
• 01:02:44So with microdialysis you get an amount.
• 01:02:46With voltammetry you calibrate your probe,
• 01:02:47you have an estimated amount with delight.
• 01:02:51I haven't found a great way to
• 01:02:53figure out what the number is,
• 01:02:54but you can look at relative
• 01:02:57changes over the session.
• 01:02:58I don't have this data up and is easier
• 01:03:01way because we're putting it into
• 01:03:02something so we have a manuscript that
• 01:03:05we're getting together and its focus
• 01:03:07on novelty based changes and joking signals,
• 01:03:10and so it's focused on Lane in addition,
• 01:03:12But what we're looking at is
• 01:03:14the phasic response relative
• 01:03:15to longer changes in dopamine,
• 01:03:17and I don't want to call it tone
• 01:03:20because it's over like 10 minutes.
• 01:03:22Not like ours,
• 01:03:23but it's definitely not what you
• 01:03:25would call a phasic fast response.
• 01:03:27What novelty in the environment does?
• 01:03:29Is it increases that phasic response?
• 01:03:31But then it does it.
• 01:03:32The baseline is much higher.
• 01:03:34So what you have is this shift and
• 01:03:36what we think is happening is that
• 01:03:38the novelty is changing the state of
• 01:03:41the system so that if the next thing
• 01:03:43that's encountered in that situation
• 01:03:45the domain response will be bigger.
• 01:03:47Now one of the things I've always
• 01:03:49been interested in over my career
• 01:03:51is what matters for the animal that
• 01:03:53change from baseline or the peak.
• 01:03:55And so we're trying to get into that.
• 01:03:57Now to say like, OK, that increasing.
• 01:04:00Slide does that just increase the peak?
• 01:04:03Or does that actually still amplify
• 01:04:05more this signal to noise and so these
• 01:04:09novelty terms are definitely changing.
• 01:04:11What I would call.
• 01:04:13I don't want to,
• 01:04:15but they are definitely changing
• 01:04:16these slower baseline fluctuations in
• 01:04:18dopamine over longer periods of time,
• 01:04:20and we think that's really important
• 01:04:22for the effects of novelty on
• 01:04:24other types of learning,
• 01:04:25and so this dopamine,
• 01:04:26perceived salience,
• 01:04:27perceived failings is influenced by novelty,
• 01:04:29so anything that changes novelty
• 01:04:31will change this as well,
• 01:04:32and so it's really important in
• 01:04:34these novelty based learning
• 01:04:36things on these both slow and fast
• 01:04:38timescales in a way that is,
• 01:04:40I think,
• 01:04:40consistent with what people have seen
• 01:04:42with microdialysis and voltammetry,
• 01:04:44but in a more you are able to more granularly
• 01:04:46relate them with this, But again,
• 01:04:48you don't have the amount, so I kind of,
• 01:04:50you know, it's hard because there's no
• 01:04:52like calibration like you can't say,
• 01:04:54oh, this is the amount,
• 01:04:55and I think that's where I hesitate
• 01:04:57a little bit to make these really
• 01:04:59strong conclusions about like what.
• 01:05:00It is, I know that it's changed from
• 01:05:02the minute one of this session.
• 01:05:04The question is like exactly what is that?
• 01:05:06But there are those changes and I think
• 01:05:08that's a great point that people.
• 01:05:10I think a lot of people have gone into
• 01:05:12these kind of optical measurements and they.
• 01:05:15That was ignored,
• 01:05:16but they're not necessarily rooted in these,
• 01:05:19like microdialysis.
• 01:05:19Will Tanistry fields,
• 01:05:21and so they don't understand that
• 01:05:23there has been a ton of work parsing
• 01:05:26what these tonic changes mean?
• 01:05:28How tonic dopamine is regulated relative
• 01:05:31to like fast release like the domain
• 01:05:33transporters? My favorite protein because
• 01:05:35I guess the question I guess I'm asking is,
• 01:05:39would you see some more reward evidence
• 01:05:42of more reward prediction error if
• 01:05:45you somehow subtracted what the?
• 01:05:47Basil Change is an are you missing some
• 01:05:50of that? Because your tonic salience?
• 01:05:52I mean, you started out talking a lot about
• 01:05:55how dopamine could be doing this and that,
• 01:05:58but it also could be doing all of it.
• 01:06:01You know it's not mutually exclusive.
• 01:06:04It could be.
• 01:06:05It could be let me.
• 01:06:07I do have a data that answers
• 01:06:09that question because actually a
• 01:06:11really astute reviewer asked us.
• 01:06:13Now I need to find it. It wasn't me.
• 01:06:17They they they asked us saying,
• 01:06:18oh, there it is, we did this,
• 01:06:21so this is actually a really,
• 01:06:22really good question.
• 01:06:23Why is this coming up?
• 01:06:25Oh I'm like not looking at the right file.
• 01:06:27Maybe if I can get it in like
• 01:06:302 seconds I'm going to write,
• 01:06:31But basically we did what you
• 01:06:33your question is a good one.
• 01:06:35I think one of the things that they ask,
• 01:06:37which is a great question.
• 01:06:39We'd already been thinking about
• 01:06:40this so we were like, OK, cool was.
• 01:06:43If these changes in baseline are the
• 01:06:45reason that we don't see changes.
• 01:06:47In in a, let's see file new release.
• 01:06:51All these will be in here now. Insert.
• 01:06:55Side OK. OK, So what we did here?
• 01:07:01Is we had that repeated shock
• 01:07:04experiment and their question
• 01:07:05was kind of what yours is like.
• 01:07:07Well, maybe the difference is the baseline,
• 01:07:09like the baseline is changing overtime
• 01:07:11and so we calculated the shocks in
• 01:07:14the original stuff I showed from like
• 01:07:16a 2 second window before the event
• 01:07:18and then we went back and calculated
• 01:07:20all the shocks from a global baseline
• 01:07:23at the very beginning of each trial.
• 01:07:25And what we found is that
• 01:07:27the data is correlated,
• 01:07:28like very highly correlated.
• 01:07:29And if we look at the baseline change.
• 01:07:32Over that trial it is not changing,
• 01:07:34so it can't explain all of it.
• 01:07:36Like I understand,
• 01:07:37I do agree that there is a lot of
• 01:07:39baseline stuff that's this where we
• 01:07:41are like kind of taking out here that I
• 01:07:43think does paint a more complex picture,
• 01:07:45but I think for a lot of these
• 01:07:47things that would say OK,
• 01:07:48this is definitely not our PE.
• 01:07:50The baseline is not the factor
• 01:07:52that's driving all of it,
• 01:07:53but I think that's something actually
• 01:07:54that people should be thinking
• 01:07:56about in these papers and everyone
• 01:07:57does imaging now and they're all
• 01:07:59doing everything is baseline,
• 01:08:00But there's like slower changes
• 01:08:01that are totally left out of that.
• 01:08:03They're going to change the way
• 01:08:05you interpret the data.
• 01:08:06But that's I mean it's a great question.
• 01:08:08It will look into this more
• 01:08:09'cause I think maybe it does.
• 01:08:11All of that is totally reasonable.
• 01:08:13Explanation and it might be like nice at
• 01:08:15a different synapses and the same area,
• 01:08:16and so like how to parse those.
• 01:08:18I think it's important.
• 01:08:21Last question from Rick.
• 01:08:23The outstanding thank you.
• 01:08:25Very much so this is I'm not
• 01:08:27sure exactly this question,
• 01:08:28but since there's so much regulation
• 01:08:30and release at terminals Strike and dip
• 01:08:33compared to mean cell body activity,
• 01:08:34do you think that other neurotransmitters
• 01:08:36responsible for the specific
• 01:08:38components of your learning model
• 01:08:39and like dopamine release the end
• 01:08:41result and integration of those?
• 01:08:43And can you speak this one?
• 01:08:45I can think of right off the bat.
• 01:08:47So one thing we think is really
• 01:08:49important for this is acetylcholine
• 01:08:52regulation of dopamine release.
• 01:08:54We we have started to do some of this
• 01:08:56where we're starting on the slice level
• 01:08:58trying to workout these parameters
• 01:09:00because it's like calling regulation
• 01:09:02of domain releases actually really
• 01:09:04kind of cool regulatory mechanism
• 01:09:06because it's one of the few that really
• 01:09:09robustly releases domain from terminals,
• 01:09:11totally independent of the semantic activity.
• 01:09:13And you can even get this in
• 01:09:15isolated terminals and slices.
• 01:09:17And so we think that there is probably,
• 01:09:19you know, maybe it's the.
• 01:09:21Maybe it's both questions.
• 01:09:23Maybe Domain was released.
• 01:09:24In response to RP signals,
• 01:09:26but also attentional signals
• 01:09:27that are coming from these,
• 01:09:29maybe like that's the first thing I think
• 01:09:32of acetylcholine is like attention arousal,
• 01:09:34but here's the thing.
• 01:09:36Those are all integrated.
• 01:09:37What the domain signature finally is,
• 01:09:39and so I think that it is
• 01:09:41probably both in some contexts,
• 01:09:43right?
• 01:09:44It's probably both acetylcholine
• 01:09:45regulated domain release and RPE
• 01:09:47regulated cymatic activity that leads
• 01:09:49to this kind of saliency based term.
• 01:09:51One thing I'll tell you some
• 01:09:53ants about it as we see pretty
• 01:09:56interesting sex differences.
• 01:09:57In terminal regulatory
• 01:09:58mechanisms through the A4 beta,
• 01:10:00two containing nicotinic receptors
• 01:10:02that are on those terminals,
• 01:10:04and So what we're trying to do first
• 01:10:06is outline the mechanism or how this
• 01:10:09is different and then take that
• 01:10:11model into this and say do those
• 01:10:14differences predict differences
• 01:10:15in these learning parameters
• 01:10:16because it's really hard to get it.
• 01:10:19Don't mean terminal regulation by
• 01:10:21acetylcholine in vivo because it's
• 01:10:22your calling regulates other cell
• 01:10:24types that also regulate dopamine release.
• 01:10:26So it regulates GABA release.
• 01:10:28That regulates terminals and so,
• 01:10:30like without some sort of like biological
• 01:10:33system that you know is different.
• 01:10:35It's really hard to isolate just
• 01:10:37the effects of cetyl choline from
• 01:10:40the effects of acetylcholine media
• 01:10:43dopamine terminal regulation.
• 01:10:44I hope tools come along.
• 01:10:45There's like darts and stuff
• 01:10:47which are awesome.
• 01:10:48Those are coming and like we'll see.
• 01:10:49But like anyway I'm like this is a that's
• 01:10:51a great question I'm excited about so.
• 01:10:55Well thank you were coming up to 11:30
• 01:10:57and we peppered you with questions
• 01:10:59we really appreciate the great talk
• 01:11:01and the time that you spent with us.
• 01:11:03Thank you to everyone for your attention
• 01:11:05and we hope to hear more about the
• 01:11:08next steps in the research. Yeah,
• 01:11:09thank you guys so much.
• 01:11:11These questions were awesome man.
• 01:11:12I had a great time
• 01:11:14chatting with everybody so
• 01:11:15thanks. Thanks Erin.