COVID-19 projections for Connecticut

May 28, 2020

Associate Professor of Biostatistics, Statistics & Data Science, Operation, and Ecology & Evolutionary Biology

5.25.2020 Biostatistics Seminar

Information

ID: 5250
To Cite: DCA Citation Guide

Download Transcript

00:00- From a colleague
00:05asking for help with planning for the intensive care unit
00:10and floor bed capacity at the Yale New Haven Hospital
00:14Health System and Yale New Haven in particular.
00:18Margret and Sohei had previously, or around the same time,
00:21been working with the statistics policy
00:25modeling an epidemiology collective on a queuing model
00:28or discussing the parameters of the queuing model
00:32for the dynamics of Covid-19 patient flow through hospitals.
00:37So we decided to use this model setup to make a concrete
00:41software product in the form of a web application
00:44that Yale New Haven Health System and other hospital systems
00:47could use for capacity planning.
00:50We wanted to respond to their very immediate need
00:54to know how full the hospital would get if Covid patients
01:00kept coming at the rates that they were seeing
01:02and how they might expand capacity to accommodate
01:06these new patients.
01:09So we created a Slack channel,
01:12a way of communicating directly in real time
01:15with the team members, who created a GitHub repository.
01:17Within, I think, only about two hours,
01:20we had a web application written in R,
01:22using Shiny framework,
01:25where you could sort of dial in the
01:30current bed capacity at a hospital system.
01:32You could enter parameters that govern the length
01:35of stay of Covid patients and how they move through
01:37the hospital from the emergency department
01:40to the floor to the ICU
01:41and then toward discharge or possibly death.
01:47So that product went live very, very quickly.
01:53There are many other collaborators and contributors
01:54to the application beyond just our group.
01:59Our goal here was to produce
02:03something very quickly and immediately useful.
02:08The structure of this model is shown
02:10in this very complicated diagram.
02:12It's not as complicated as it looks.
02:14The basic idea is that patients enter through
02:18to the emergency department.
02:21They move to the floor then to the ICU.
02:27There are many things that could happen
02:28if those places are full.
02:31Each of those parts of the hospital is treated as a queue.
02:34That is, it's essentially a pool of patients
02:37who are waiting to exit.
02:39One of the ways they can exit is to step up
02:41from the floor to the ICU.
02:43One of the ways they can exit is to die.
02:45Another is to be discharged
02:47if they are no longer acutely ill.
02:50So sort of taking into account all of this schematic, this
02:56stylized depiction of the way Covid patients
03:00would flow through a hospital,
03:02we wrote a system of ordinary differential equations,
03:05which describe formally, the dynamics of this system.
03:10It's a very simple type of modeling
03:13that is very useful when the number of patients is large
03:17and when you want sort of aggregate dynamics over time.
03:21So we're not modeling, it's not an agent-based model.
03:24We're not modeling individual patients trajectories
03:26through the hospital.
03:27Rather, this idea of patient flow through the hospital.
03:31So this model, depicted schematically here,
03:34is formalized in a system about ordinary differential
03:37equations with many parameters.
03:39Those parameters are calibrated to data that we have
03:41from the Yale New Haven Health System
03:46and to values from the literature.
03:49We wrote this web application, which is now live
03:53at the Shiny apps URL that you can see below.
03:56You can interact with it if you like.
04:01It basically allows the user to specify time horizon,
04:04how quickly or slowly they think new Covid patients
04:07will present to the emergency department
04:10and then on subsequent tabs, you can dial in the current
04:15hospital capacity at your institution.
04:17You can dial in capacity increases that you anticipate
04:20being able to implement into the future
04:23to see how dynamics would change if say,
04:25you could add 100 new ICU beds over the course of two weeks
04:30a month from now, for example.
04:32Then there are many, many input parameters.
04:34Things like the age-specific rates
04:39of death or of stepping up from the floor in the ICU,
04:42to the average length of stay in each of those compartments
04:47for patients who come to the hospital.
04:50You can generate reports, downloadable PDF reports.
04:54We sort of envisioned this tool being responsive
04:57to the needs of hospital decision makers
05:00who wanted to be able to add this planning capability
05:04to their existing bed management software applications
05:09and then to be able to generate reports for say,
05:12supervisors and higher up decision makers
05:14that would describe the scenario that the analysts
05:17was most interested in.
05:19The reports would also describe the consequences
05:22of a capacity expansion strategy that might be implemented
05:25by the system.
05:27So I think this tool was very useful
05:30to the Yale New Haven Health System.
05:33It was publicized kind of broadly and we got some interest
05:36from hospital systems throughout the U.S..
05:39I had spoke to some of them about the ways
05:41that they were making decisions, planning capacity increases
05:45and using this application and others
05:47that are also publicly available online,
05:50to help guide their decision making.
05:54This is an open source project.
05:56You can get all of the source code for the Shiny application
05:59on our GitHub repository here, shown below.
06:07So what are the next steps for this project?
06:10Fortunately, hospitalization in Connecticut is declining.
06:15This figure that I've shown here is kind of compressed.
06:17It's declining slowly.
06:21But it has been declining for I think,
06:23more than three weeks now.
06:25Yale New Haven Health System, along with hospitals
06:28heath systems throughout the state,
06:30are doing much better than they were in mid-April.
06:33They have enough bed capacity to accommodate
06:35all the Covid patients and many more who may arrive
06:39in the coming months.
06:40So this is very good news for the hospitals
06:42and for the state.
06:43It's one of the reasons that the governor initiated
06:48the first phase of the reopening plan on May 20 this week.
06:53However, a lot of the projections and some that I'll show
06:56in a few minutes, indicate a substantial risk of resurgence
06:58of new cases, hospitalizations and deaths
07:01following reopening the state.
07:03This resurgence is anticipated to occur in July,
07:06August, maybe September, depending on how things go
07:09with reopening.
07:11So I think that
07:13the model, the web application,
07:16and this work in general will unfortunately,
07:20become useful again and very relevant again
07:23later on in the summer if hospitalization
07:25of Covid patients increases again.
07:28So we want to maintain our capacity to continue developing
07:31this model and responding to the needs
07:34of decision makers within hospital systems.
07:37We're taking this down time though,
07:38to write a technical report and a lessons learned paper
07:42about the way that we interact with health systems
07:46and how we might improve the way
07:48that we do that in the future.
07:50This work is of course also gotten us very interested
07:52in the ways that hospitals manage Covid patients.
07:55We're very interested in comparative evaluation
07:57and comparative effectiveness in the evaluation
08:00of Covid-19 medical interventions.
08:03That's something that Margret Erlensdottir
08:06an MD PhD student in biostat is working on.
08:09- Forrest? - Yes?
08:10- Can you take a question?
08:12- Yes, please go ahead.
08:16- Have you only applied this to Yale New Haven?
08:19- We have, the model itself is generic.
08:22This is a good question, but we have calibrated many
08:26of the length of stay and probability parameters
08:30based on data that we received from Yale New Haven.
08:34So in that sense, the dynamics that we present by default
08:39are specific to Yale New Haven.
08:41The user has the ability to change all of those parameters,
08:45so we anticipate that this could be useful
08:47for hospital systems of any size
08:50with different patient demographics,
08:51different age distributions for example.
08:54So we want it to be as useful as possible,
08:57but having said all this, the customer in this case,
08:59was very clearly for us, Yale New Haven
09:02and they had a very specific need and--
09:04- Have you had a reaction?
09:06Did you have an ongoing reaction with the people
09:08at Yale New Haven who were using this product,
09:10whether or not it was helping them or was it accurate
09:13or did they have any complaints about it?
09:16I'm sure they did.
09:16Can you tell me about that interaction?
09:19- Sure, sure.
09:22I think that they made a few requests of us.
09:25Some of them were very qualitative.
09:27They wanted very early to be able to generate reports.
09:30A lot of the requests were for additional functionality
09:33rather than additional structure in the OD model
09:37but they really,
09:39I think many of the requests were about flexibility
09:41and granularity in the predictions.
09:44They wanted to be able to dial in the exact patient
09:46demographics and the care parameters that were actually
09:50being implemented at Yale New Haven.
09:52So we tried to give them that ability and that control.
09:58I think mostly, successfully.
09:59We retained some of the generality of the model,
10:02while allowing users to input the parameters
10:05that they felt were right for their system.
10:07In terms of the way it was used at Yale New Haven,
10:11I think that by the time they asked us for help,
10:15many of the actual capacity expansions
10:18had already been implemented.
10:19I'm talking about taking over high school gymnasia,
10:22changing the configuration of parking lots
10:25to provide drive through testing and turning,
10:30I guess parts of the hospital into ICUs.
10:33Many of those--
10:36- You might say that they over expanded a little bit
10:38since they quickly came not needed capacity.
10:41So did you help them, saying hey,
10:43you guys don't need to do that much?
10:47- I think that based on the projections for population
10:50level incidence that they were receiving in early
10:52to mid-April, the capacity expansion was appropriate.
10:59This model here did not provide
11:03population level projections,
11:05which I'll show in a few minutes.
11:06So we were not telling them
11:07that they had over expanded capacity.
11:11I think that at the state level,
11:13the total hospitalization in the state came very close
11:17to the preexisting capacity, as it was in early March.
11:22So I think that there was a big concern that it was unclear
11:26what the doubling rate of new cases would be.
11:29We had not yet seen some of the benefits of state lock down
11:31and closure of schools.
11:33So the hospital systems were expanding very aggressively,
11:37I think for good reason.
11:40- Okay, but they were just doing that by looking at
11:44the daily or maybe the weekly case counts right,
11:47and seeing what the doubling rate was and things like that.
11:50They were doing anything more subtle than that?
11:52- That is what they were doing when they called us on.
11:55We tried to give them projections under their own
12:00in-house assumed doubling rates.
12:03So we were very interested in showing them when the hospital
12:06would fill up and under what circumstance
12:08and how different parts of the hospital would fill up.
12:13- Okay.
12:14I'll let you go on.
12:16- In a few minutes I'll show state level projections
12:18that might answer some of your questions.
12:24All right.
12:28- By the way, I'm John Hardigen, by the way.
12:31Used to be in the statistics department.
12:33- Yes, I know, good to see you.
12:38All right, second project.
12:40On April 14, so just as we
12:47finished the most fundamental software development
12:50on the application that I just showed you,
12:52on April 14 we were asked to start producing projections
12:55for the governor's Reopen Connecticut Advisory Panel,
12:58which was charged with
13:01making recommendations to the state, to the government,
13:04to the Department of Public Health
13:06on how reopening should proceed and what the timeline
13:09should be and what business sectors could safely reopen
13:12at which times.
13:14The panel consisted of public health researchers,
13:17including Albert Ko and several other people from Yale
13:20and many business leaders in Connecticut.
13:23It was a mixed group.
13:27The panel needed projections at that time of Covid-19
13:31incidence, hospitalizations and deaths
13:33under future reopening scenarios, to plan testing expansion,
13:37seroprevalence studies and most importantly,
13:39to assess the risk of a second wave of infections.
13:43So this was in mid-April,
13:45around the time when hospitalization was peaking.
13:48Of course, nobody knew exactly at that time
13:51that the peak was occurring and there was a lot of concern
13:55that things would continue to get much worse,
13:58in terms of hospitalization in Connecticut.
14:03The work of that committee
14:07advised the governor in his reopening strategy,
14:09which we've all probably heard about,
14:11if you're following press releases from the state.
14:14The state began reopening on May 20th and there's now,
14:17I think, although the work
14:19of the advisory panel may be wrapping up,
14:22there's now an ongoing need for projections
14:24to inform decision making and epidemiological study design,
14:28that further informs decision making
14:32for the Connecticut response and reopening.
14:36This part that I'll talk about now is joint work
14:37with Olga Morozova and Richard Li.
14:42So at the beginning of this project, we had to explain
14:46to decision makers and members of the advisory panel,
14:50how data are different from model projections
14:54and what sort of...
14:59What the differences between these two products were.
15:01But I think there is a recognition at that time
15:03on the part of policy makers and committee members
15:05that the policy makers have access
15:07to a real-time data stream, which is very high quality.
15:11They have access to all sorts of state dashboards
15:14describing the current state of the Connecticut pandemic.
15:18They know about hospitalization and bed capacity
15:20information from the Connecticut Hospital Association.
15:23They know about test counts and nearly real-time
15:25case counts, number of tests positive at hospitals
15:28and in the community.
15:30They know how many deaths have occurred to attributable
15:33to Covid-19 or that are suspicious,
15:37that are possibly related.
15:39They might have information about excess deaths
15:42that are not attributed to Covid-19 but are above
15:45and beyond what you might normally expect in a typical year.
15:49They have access to all this information.
15:51They have access to very responsive staff
15:53and many very smart people working for the state
15:57Department of Public Health and other state agencies.
16:02So there might be a sense that policy makers have access
16:04to all the information and the most timely information
16:07they could possibly need to make good decisions
16:08for the state.
16:10We tried to argue that there was more information
16:12that they might be able to use constructively
16:16to guide reopening, and that was information that was not
16:19directly derived from contemporaneous data streams,
16:24but rather these would be projections from transmission
16:27models about possible futures.
16:30So projections here can tell us about what might
16:33happen in the future, possible hypothetical
16:34or counterfactual scenarios to be defined
16:39by the governor and the outcomes that would occur
16:42under those reopening scenarios.
16:44So I'm talking about phases, business sectors,
16:47reopening back-to-school, what might happen in late August,
16:50early September, as children go back to school
16:52or back to summer camp in June and July.
16:56What might happen under expanded testing
17:00and contact tracing or continue to modified
17:03social distancing guidelines.
17:04Things like wearing masks or keeping six feet apart
17:10and all of those things.
17:11So we tried to explain how projections from these types
17:15of models might be very different from simple plots
17:19of the data streams that policy makers have access to.
17:23This is a figure I showed them at the very beginning.
17:26On the left, we have the number of death,
17:28I think by early May, that had accumulated in Connecticut.
17:33These are the red dots on the left hand side.
17:36On the right hand side, we have a projection of what might
17:38occur in the future on this day and I think it was
17:40first week of May.
17:42Right, and I think this may seem silly as a projection
17:47exercise or it seems silly to
17:52make a distinction between data and predictions,
17:56but it may have useful in the setting to emphasize
17:59that the real-time data that policy makers were using
18:02was just the stuff on the left
18:04and that if one believed the assumptions underlying
18:07some of these dynamic transmission models,
18:10that they could be provided with the stuff on the right,
18:13which would be a projection
18:14of what might happen in the future.
18:16Here, I happen to have shown projections starting
18:19on March 1, just to emphasize sort of how the line follows
18:24the data points in the projection.
18:27But the idea is that these projections would come
18:29with some sort of uncertainty windows or sets
18:34that would represent, in some sense,
18:38the most likely possible futures under what we know today
18:41and what we believe may happen about the future.
18:44So the-- - May I stop you for a second?
18:47- Of course. - Forrest.
18:50First of all, I think you'll agree that the points
18:53on the line at the left are extremely highly correlated
18:55with each other, since they're just cumulatives.
18:59And that's not a good way to show what's happening,
19:01is to look at cumulatives.
19:02You have to kind of guess what the derivatives are
19:06and people aren't so good at that.
19:07You would be much better off trying to project
19:10and look at say, the weekly values.
19:13Certainly can't look at daily values because God knows
19:15what the daily values goes from,
19:17but you know, you see in a week they kind of catch up
19:19with the truth.
19:20So if you looked at weekly values, you would tell on
19:22what the present situation was.
19:24Surely, that's what the hospitals need to know.
19:27They don't need to know how many people they had
19:29a long time ago or what the total was.
19:31They want to know that the present charge is.
19:33So I would just suggest that the thing you should be
19:36working on is something closer.
19:38Can't use daily values, it's too small,
19:40but a weekly value and then that's what really matters.
19:43That's the present situation.
19:46- Certainly and have access to all that information.
19:48The State Department of Public Health produces
19:50weekly smoothed and unsmoothed count.
19:54In fact, daily counts as well.
19:56They're very volatile.
19:58They jump up--
19:59- The daily counts have a huge weekly effect.
20:02You just don't want to rely on them at all.
20:05The docs aren't bothered to do things on the weekends
20:07is my interpretation of it.
20:08But maybe it's someone not bothering, but whatever it is,
20:11it's a big weekly effect.
20:12It's something you don't want to have.
20:14But if you take a weekly value, that's always averaged out.
20:17I just think that projecting the future
20:20and I think you would find there's quite a lot
20:22more error in that.
20:23You're getting the benefit of the fact that all this
20:26is highly correlated but if you were trying to project
20:28the future, these things would be whoa, of stuff.
20:32- Yes, totally agree.
20:33This figure was generated in response to a very specific
20:36question, which is how many deaths will the state expect
20:39to have accumulated on a future date.
20:42- Okay.
20:45Thanks.
20:48- Okay, so I wanted to answer this question
20:50because I hope that you're all wondering about it.
20:54Does the world need another Covid-19 projection model?
20:58There are lots of them out there.
21:00Vary in quality, some from very experienced research groups
21:05and experienced epidemiologists, some from
21:09Silicon Valley software developers
21:11who just learned about regression.
21:13I don't think that the world needs another Covid-19 model
21:17at the national or international level.
21:19But I think Connecticut does for several reasons
21:23that I wanted to describe briefly here.
21:27We wanted to develop a scenario analysis tool
21:29that was responsive to specific questions
21:31from the Connecticut leadership,
21:33who were planning to reopen the state.
21:38We thought there were several reasons that we could add
21:39some value here, beyond what is provided by some of the more
21:42generic models that are available
21:47for national, state and also local projections.
21:50The first thing is access to epidemiologists
21:53at the School of Public Health
21:54and in the Public Health Modeling Unit.
21:57We have pretty unique access to data
21:59from the Connecticut Hospital Association
22:01on the bed capacity and bed occupancy throughout the state.
22:06We can use information on individual patient trajectories
22:10through the healthcare system from using data
22:12from Yale New Haven.
22:14We have access to empirical epidemiological studies
22:18from Yale emerging infections program and data streams
22:21from the Department of Public Health through Yale EIP.
22:25We have connection to the people who are running
22:28the testing and seroprevalence studies
22:30to be conducted in the future and the model projections
22:34that we produce will be very closely tied
22:38to the conduct of those studies.
22:39Some of them can give information that we can use
22:42for calibrating the model, and in turn,
22:44we can use model projections
22:48to provide preliminary estimates of say,
22:51cumulative incidence of Covid-19 for study planning,
22:56in order to do sample size calculations.
23:00And of course, we are hoping to be able to help
23:04with the Department of Public Health's
23:07implementation of optimal testing and sampling strategies
23:10as they look for new cases and try to control outbreaks
23:13that may occur in the future in Connecticut.
23:20So the modeling principle here,
23:22this is an infections disease model that I'm gonna show you.
23:25It's not a model for hospital
23:29patient flow through hospitals.
23:31But I think in introducing this to people who have not seen
23:35these models before, the operating principle
23:38is that of mass action.
23:41I think if mathematical infectious disease epidemiology
23:44has a central dogma or a single principle that governs
23:47the structure of quantitative models for infections,
23:51it's something like the Law of Mass Action,
23:54that in a small time interval, the number of new cases
23:57that accrue is proportional to the number of ways
24:01that susceptible individuals and infectious individuals
24:05can come together.
24:07This means that new cases or incidences
24:09is driven by the product of--
24:11Or sorry, I should have said the product of susceptibles
24:14and infectives or the number of ways that people
24:18susceptible individual can come into contact
24:20with an infected person.
24:23This general principle is what underlies all transmission
24:27models and many transmission models are compartmentalized
24:29or they are separated in space and geography
24:33or by age group or by different risk categories,
24:36but this is the essential principle.
24:38That new cases of a certain type and a certain place
24:41arise at a rate that is proportional to the product
24:45of the number of susceptibles and infectives.
24:47The number of ways that disease can be transmitted.
24:53So we have divided the population of Connecticut
24:56into many compartments.
24:59Those who have not had the disease,
25:01those who are susceptible,
25:03those who have been infected,
25:06they are exposed but not yet infectious.
25:08So they don't have symptoms.
25:11Those who are infectious but remain asymptomatic,
25:14those with mild symptoms.
25:16They know they're sick
25:17but they do not require hospitalization.
25:19Those with severe symptoms who do require hospitalization.
25:24Those who have mild symptoms but are successfully isolated
25:28because they realized they have symptoms or they got
25:33a viral test that told them that they are infected.
25:36So they successful isolate themselves.
25:38Those people with severe disease who are hospitalized,
25:42those who have severe disease but remain unhospitalized
25:45because there's no space for them.
25:47This is very important in projecting deaths in the future
25:50scenario, in which we run out of hospital capacity.
25:55Then we have severe institutionalized populations,
25:58who are not in the hospital,
25:59such as people in nursing homes, correctional institutions
26:03and other long-term care facilities.
26:06Those who have been infected but did not die
26:09and are now recovered or successfully isolated
26:12and recovering and those who have died.
26:15So the idea here is to divide up the population
26:18of Connecticut into a number of people
26:22in each of these compartments.
26:28The model that we put together is a variation
26:30on the susceptible exposed infected and removed model.
26:38We divide up the infectious individuals into three
26:42categories that I told you about, severe, mild
26:44and asymptomatic infections.
26:47We have two different types of patients
26:49who need hospitalization.
26:51We have unhospitalized patients.
26:55We can remove patients by isolating them
26:57and they can recover after some amount of time,
27:01if they do not die.
27:04This is the basic structure of the SEIR model.
27:09The usual model structure is just this linear part,
27:12SEI and then R.
27:15We divided up into these additional components,
27:19not because we believed that these components
27:21cover every possible scenario or every possible type
27:25of illness or state of the world or state of patients,
27:28but because this is the most parsimonious model
27:31that we can think of that captures the dynamics
27:34of infection that are most likely to lead to the outcomes
27:38that a state government cares most about.
27:40Those are state-level hospitalizations and deaths
27:44and possibly cumulative incidents.
27:48Right, so this model is not intended to capture
27:51every biological or epidemiological feature of Covid-19
27:55transmission in Connecticut.
27:57Rather, it is the simplest model that captures the features
28:00that policy makers care most about.
28:06It's also structured by geography.
28:10We found that the...
28:14We looked at information about travel and commuting patterns
28:17throughout the state to look at where people
28:18might be mixing, where they live, where they work,
28:21others things like that.
28:22But we found that that information did not give us
28:26much more information than simple adjacency matrix
28:28of counties in the state.
28:33We're well aware that many people in Connecticut
28:38work or commute or travel often to New York City area.
28:41We'll try to accommodate that in the model
28:45or in our interpretation of the model.
28:48Rather, the adjacency matrix of counties in Connecticut
28:54gives us much of the information that we use
28:55for the geographically dependent nature of transmission.
29:04Basic idea--
29:05- Rhode Island and Massachusetts
29:07aren't doing too good either.
29:09- They're not doing well, I agree.
29:12To avoid turning this into a very granular or national model
29:19we are going to treat the exogenous force of infection
29:22experience by Connecticut residents as something else.
29:28So we sort of imagined that it is
29:31subsumed into the force of infection experience
29:33by everyone in Connecticut.
29:35I agree, both a lot of infections
29:38and a lot of heterogeneity outside of Connecticut
29:42in bordering states.
29:45So most of this we don't specifically take into account.
29:50The basic idea here, I'm just showing two compartments
29:54of the ODE system, the basic idea is that in county I,
29:59the number of susceptibles or the rate of new infections
30:02is governed by the number of infectious individuals
30:07in that county and the number of infectious individuals
30:10in neighboring counties.
30:13Here in beta is the transmission rate of infection.
30:17So individuals who are susceptible transition to the exposed
30:22infectious state and then to other states down the road.
30:25But these are sort of the mass action equations
30:28for a heterogeneous population in which the force
30:30of infection is coming from outside and within
30:34individual counties.
30:39I'm not going to go into a great deal of detail
30:42about the system of ODs that is most useful here.
30:48I'll just say that we solve in numerically.
30:50It's a system of 11 differential equations
30:53given the parameters, which I'm just gonna bundle into
30:56a vector theta.
30:58Let Y of T given theta, be the solution to the OD system
31:01at time T with parameters theta.
31:03You can solve this system with pretty good accuracy
31:07using modern OD solvers.
31:10This solution--
31:11- They're just linear equations, are they?
31:13Linear right?
31:15- They're non-linear in the right hand side
31:18is non-linear in the other model compartments.
31:21Right, that's what mass action is.
31:24It's proportional to the product.
31:26So OD is proportional to the product of S and I.
31:31So it's--
31:33- On the other hand, you agree that S doesn't change much
31:35because unless you've got a very fully infected population,
31:39S doesn't change that much.
31:41You've got--
31:42- S is most quickly when infections are increasing
31:45most quickly and Connecticut right now,
31:48S is still pretty large.
31:50I think cumulative incidence is between 5% and 15%.
31:55So S has not changed.
31:57- S is 85% and is gonna change.
31:59I'm just making it linear for myself, that's all.
32:02- Sure, yeah.
32:04So right now S has not decreased that much.
32:08You know, between, it's still at 95% to maybe 85%,
32:13something like that.
32:15As the pandemic progresses and into the fall,
32:17if there's another resurgence of infections,
32:19we will expect S to change quite a lot more.
32:22If it changes a lot, then we'll be in herd immunity
32:25territory where depletion of susceptibles
32:28plays a prominent role in altering the dynamics
32:32of the pandemic, but we're not there yet.
32:34- But I am right in thinking that this is linear,
32:36it's really just a matrix problem isn't it,
32:38that we have to solve.
32:39- If it were linear it would be a matrix problem.
32:41- Yeah, okay.
32:44- Right. - Yeah.
32:47- So this system is a deterministic system.
32:52Engineers, mostly and some epidemiologists,
32:55have been thinking for a very long time about principled
32:58ways of estimating parameters for deterministic system.
33:02Unfortunately, for models of this type,
33:04which is generally the case in infectious disease
33:06epidemiology, there are some serious
33:11identifiability problems.
33:11Not all parameters can be uniquely estimated from the data
33:16or infinitely many combinations of parameters
33:18that appear to fit equally well.
33:23We only observe in this case,
33:25the hospitalization and death compartments.
33:27There's some information from PCR testing
33:30about the prevalence of infection at different times,
33:34but because the testing strategy in Connecticut
33:36and elsewhere has varied so dramatically
33:40over the last few months, we didn't feel like we could use
33:42any information from testing alone to inform the sizes
33:47of the currently infected compartments.
33:51So basically, we're trying to estimate many parameters
33:54for a system with 11 components using only the time series
33:59of hospitalizations and deaths.
34:02So it's quite challenging and in practice,
34:04this necessitates taking parameter values
34:08from the literature, from clinical studies,
34:10from our knowledge of how hospitals treat patients
34:14and also using a statistical estimation scheme
34:18to learn about elements of theta, of the unknown parameters.
34:22I wish that I could give you a more coherent statistical
34:26inference strategy in which all of the parameters
34:31were learned from the data and I could tell you
34:34that they were being consistently estimated
34:36and that as the epidemic went on, we would get more and more
34:39precise estimates of each of those parameters.
34:41Unfortunately, it's just not true.
34:43That the model structure that we need here to be able
34:46to accommodate
34:51the structure of the pandemic is more complicated
34:54than the model structure that we could possibly identify
34:59non-parametrically or semi-parametrically
35:01or even in this parametric model.
35:05So I just wanted to give you some examples
35:06of how people do this in practice.
35:08These are not exactly endorsements
35:10of statistical frameworks.
35:12The basic idea is that given theta,
35:14we can solve the ODE system, it gives us deterministic
35:17solutions at time points where we have an observation
35:21and then calibration or statistical inference
35:23essentially amounts to minimizing a loss criteria
35:26and are comparing the observed values to model predictions.
35:31The two frameworks that are most frequently used here
35:34are imposing a normal errors or gaussian errors,
35:37almost normal gaussian errors
35:41or equivalently minimizing at least squares type
35:44of loss function or doing this plus on maximum likelihood
35:49estimation for elements of theta
35:51that you can identify in this way.
35:53I think in this project we used
35:58the Poisson maximum likelihood.
36:01There are many things about this, one of which is that
36:03a Poisson random variable could take values
36:06that are larger than the size of the population.
36:08In practice here, that's not what occurs
36:12because the number of infections here is small,
36:15but this is basically a framework for doing a type
36:18of statistical inference or learning about
36:22a posterior distribution on parameters
36:25from a model, which gives deterministic predictions
36:27and which doesn't have any inherent stochasticity.
36:31The procedure that we used here, which I'm not gonna
36:33talk about in great detail here, was developed
36:36by postdoc Olga, is a hybrid approach that fixes
36:39some parameters and imposes uncertainty distributions
36:42on them from our prior knowledge and the literature
36:46and conducts Bayesian posterior inference
36:49on known parameters and initial conditions.
36:51So we try to learn jointly about parameter--
36:55Yes go.
36:56- Forrest, there's a question people always ask of this
36:58whenever I give a talk like this,
37:00how do you determine your prior distribution?
37:03- In this case, I would say we're in a very good position
37:06to interpret priors as literally being prior beliefs.
37:11We have for example, point estimates and confidence
37:14intervals from published studies.
37:19We also have parameters which are intrinsic to the model
37:22but for which we have very little information.
37:24So we assign to them, what we believe qualitatively,
37:27to be an appropriate representation of our uncertainty
37:31or ignorance about those parameters
37:33under the parametrization.
37:36But to your question--
37:38- What you believe to be true then, is that right?
37:42- Oh certainly.
37:43It is a mixture of what other people believe to be true
37:46and what we believe to be true as well.
37:48So I would take a subject of interpretation
37:49to the priors here.
37:52They are subjective in the sense that we believe
37:56these uncertainty distributions.
37:59They are quantitative in the sense in that
38:02some of them come from published studies.
38:06- Okay, I'm sorry, I do just a little bit longer.
38:11You know, I know you've got a lot of parameters in here,
38:13many of which I don't know anything about,
38:15but I suspect the very important one is parameter
38:18which says what is the ratio of new cases,
38:24assuming that susceptibility isn't changing
38:26to the infection rate, right.
38:28What's the...
38:30That's the number,
38:32that ratio is an important ratio.
38:34New cases against the number that are infected
38:38and that number out to extract is an important number
38:42because it changes a lot, according to the conditions
38:44that the government sets.
38:46Changes all the time because you're trying to reduce
38:48contacts and effectively reducing that contacts
38:50is to change that ratio.
38:52I assume that that's built into the model somehow,
38:55but I would think you probably don't know very much
38:57about how the government's policies and whatever
39:00are gonna change that ratio.
39:02So if you said you know, I know it's gonna be a month
39:04from now, I'd say no you don't.
39:06- Oh sure. - Yeah.
39:08So how do you handle it?
39:10- We certainly do parametrize that rate,
39:14that is the transmission rate that you were talking about.
39:16It's the parameter that multiples the product
39:19of the number of susceptibles
39:20and the number of infectious individuals.
39:23That's called beta in the model.
39:25Beta does change over time.
39:27It's parametrized as a sum of step functions.
39:33Those step functions change in their value
39:36around when the governor closes schools, which happened,
39:42I think on March 25th and when the governor--
39:45Or sorry, a little bit earlier, maybe March 20th,
39:48I can't remember.
39:49Then when the governor issued the stay at home order,
39:52the stay safe stay at home order,
39:56which I think took effect on the 23rd.
39:59So those step functions are in the model for historical
40:02interventions that were implemented by the state.
40:04For future interventions which are implemented by the state,
40:07we are guessing.
40:09Fortunately, we are guessing using information
40:11from the people who will actually make those decisions.
40:14So I will show how we assume that that transmission rate
40:20or contact rate might change in the future
40:23under guidelines expressed by the governor
40:26and policy makers.
40:28Right, so in the future of course,
40:29I don't know what going to actually occur.
40:31The best I can do is ask the people
40:33who will implement the change.
40:35- All right, well.
40:37I'm sorry, this is my last remark.
40:39I won't keep on doing this, but I would think that
40:42these rates that we're talking about,
40:43which seems to be are really critical to what happens
40:45in the model, that you and find invasion inferency
40:49you have to give a plausible, defensible probability
40:52for them, which I would find hard to do,
40:55and I also find it hard to do because I know that those
40:58rates differ huge amount in Connecticut
41:00between the different counties, that you can just see
41:03if you look at what's happening in different counties.
41:05Those rates are different because different
41:10amount of separation and different amount
41:12of personal contact.
41:13- Sure.
41:14- I think so kind of do that on an average way
41:16of all the counties, seven or eight of them,
41:19you'd think you at least got a vary among the counties
41:23and have some number among the counties.
41:25Then if there's a change of policy from the governor,
41:27there'd be a change in sum or expected you need
41:29to have that built in somehow here.
41:31- Certainly.
41:32In this work, I guess in all policy-relevant work,
41:37there is a constant tension between the need
41:41for parsimony and parametrization
41:45and the need for these rich ways
41:48of accommodating heterogeneity.
41:52What we have found in this setting is that we lacked
41:55the information or data to be able to separately
41:58parametrize transmission rates at the county level
42:04but that we can capture the aggregate number of cases,
42:07hospitalizations and other relevant outcomes
42:09at the state level by averaging over them.
42:13The reason is because the counties themselves
42:15have very different incidence, which actually does explain
42:19quite a lot in the differing trajectories
42:22of case counts and hospitalizations and deaths
42:25within the counties.
42:30- Hi Forrest thank you, this is very interesting.
42:32This is Donna.
42:33I have a question.
42:35Do you have, the para--
42:37Hi.
42:38Are the parameters identifiable without Bayesian priors
42:41or just from the data that we have or do you need
42:46the priors in order to estimate the parameters?
42:49- A subset of parameters is uniquely identifiable
42:52by maximum likelihood or is point identified.
42:55But really speaking, the answer to your question is no.
42:59There are infinitely many combinations of parameters,
43:04which fit any given loss function criteria equally well.
43:07So we do need parameters here.
43:09It is unfortunate and I think--
43:12Yeah, go ahead.
43:15- Priors you mean.
43:17Do you know like what's the simplest possible model
43:21that's just identifiable from the data
43:23and is that model useful at all or is it so simple
43:26that it's not even helpful?
43:29- Two parts to that question, the simplest model
43:31that is identifiable from the data is probably one in which
43:34there is no heterogeneity in types of infection,
43:39no asymptomatic infection.
43:40We just lump all those people together
43:42and there's only one kind of hospitalization
43:45and people just transition, a certain proportion
43:47of people transition to hospitalization.
43:49That model is probably, has all the parameters identified.
43:56And no, it's not useful.
44:00That seems to be what we have found.
44:03But I would say, I think there are two kinds
44:05of usefulness, right.
44:06One is answering the questions that policy makers have
44:09and the other one is what Charles Manski calls credibility,
44:12that there is a need to take into account
44:16known heterogeneity and known mechanisms
44:20when we construct these models.
44:21So if I produce a useful projection that a policy maker
44:24likes but I have not separated out asymptomatic infections,
44:28then the numbers that I'm producing may become less,
44:32regarded as less credible, right.
44:34There's always this rhetorical function of modeling
44:38beyond the numbers that are being produced,
44:40to being able to accommodate or capture known mechanisms
44:44by which data are generated
44:48is one way that we can produce more believable
44:50and actionable projections, right.
44:53So I think there's this balance right,
44:55between parsimony and richness and also this balance between
45:03simplicity and believability of the assumptions.
45:07So here we tried to you know, strike that balance.
45:09If you think we've done it wrong, then please let us know.
45:14- No, I definitely don't think you did it wrong,
45:16but it would be interesting to see how much you lose
45:20and sort of,
45:22sort of cross validated predictability
45:26by adding in priors, as opposed to just using the data
45:30itself in a very simple model.
45:32- Right, so--
45:33- I don't know if you know the answer to that or not
45:34but you should probably go on and I know other people
45:37are wanting you to go on and not spend time answering
45:40a lot of individual questions and we can always
45:42talk another time.
45:44- Okay, sounds good.
45:46The model fits pretty well, fits observe data pretty well.
45:49Here, I'm showing projections that start on March 1st,
45:51rather than at the current day or any intermediate day,
45:54just to emphasize that
45:58model projections and uncertainty intervals here,
46:00which are point-wise 95%, I call it--
46:04They are not proper confidence intervals.
46:07They're point-wise projections from draws,
46:10using draws of parameters and initial conditions
46:12from the posterior distribution over those quantities.
46:15They're not confidence intervals in the strict sense.
46:19But they do appear to
46:22match observed data quite well.
46:26So I think we're capturing dynamics that govern
46:29what has occurred already.
46:30We can learn quite a lot about the transmission rate
46:34and under historical circumstances because we know
46:37when those circumstances changed.
46:39So we can estimate for example,
46:41the percent decrease in transmission in Connecticut
46:45following closure of schools and implementation
46:48of the stay at home order.
46:50That is what causes actually,
46:51this downturn in hospitalizations and flattening
46:54of cumulative deaths in the state.
46:58So here, just to get a little bit more concrete,
47:01on the upper left-hand corner, we see what we call
47:05the contact intervention.
47:06This is a function that multiples that transmission rate
47:10parameter that we were discussing.
47:12So in early March, schools are closed,
47:15people start staying home
47:16and so this intervention drops down.
47:19The level to which it drops is a little more,
47:23it drops more than 85%, I think, or somewhere around 85%.
47:28That is an estimated quantity.
47:30So the drops in historical contact are estimated
47:35based on the changes in hospitalizations and deaths
47:39and the implied changes in new infections.
47:42Then what happens after the dotted line,
47:44that is after May 20th, this is just a scenario
47:49in which the amount of contact between individuals
47:53increases at, I think here, monthly intervals by 10%
47:57of the suppressed latent contact.
48:00Under this historical and hypothetical future scenario,
48:07we see cumulative incidence in the upper right-hand corner,
48:11projected from March 1st onward.
48:14Hospitalizations, with the dashed line,
48:17showing expanded hospital capacity in Connecticut.
48:27We see projections of deaths under this scenario,
48:31cumulative incidence as a proportion of the population size
48:34among people who are alive.
48:35So this is what you would get if you conducted
48:37a seroprevalence study in the future.
48:39We hope this is useful for planning those types of studies,
48:42and estimates of the affective reproduction number
48:46in Connecticut over time.
48:48There are two scenarios in particular that we want to show
48:54policy makers that correspond to slow and fast reopening.
48:57Really, this is not reopening scenarios.
48:59I'm not sure what happened with this green annotation.
49:03I don't know if you can see it.
49:04If I did that or somebody else did, but just ignore that.
49:09I'm not sure where it came from.
49:11Under slow reopening, we imagine that people
49:16release 10% of their latent suppressed contact
49:19every month and under a scenario like this,
49:22where everybody keeps distancing and everything goes
49:24very well in the state, new infections continue their drop
49:29and rise very slowly into the late summer and fall,
49:32hospitalization stays low throughout the summer.
49:39Deaths sort of begin to plateau and do not rise above
49:4310,000 by the end of the summer.
49:48Right, so this is the scenario that the state
49:50is really hoping for.
49:51It's a slow reopening that does not substantially increase
49:55new infections with very slow rise
49:59in new infections as the state reopens.
50:02In contrast, a more pessimistic scenario,
50:05which I think corresponds more to
50:12a fast reopening, is one in which contact increases by 10%
50:17or 10% of suppressed contact is released every two weeks.
50:22This results in a very fast resurgence of new cases,
50:26new hospitalizations and deaths by the end of the summer.
50:29This is what the governor would like to avoid
50:33when school children are scheduled
50:34to go back to school in the fall.
50:40There is a lot of interest right now in seroprevalence
50:42because of competing claims about herd immunity
50:45and how many people have been already infected
50:47and have evidence of prior infection.
50:50Under these scenarios, we can produce projections
50:53of the proportion of people in a random sample
50:56in the state, who might have evidence of prior infection.
51:01So this is very important for designing seroprevalence
51:04studies that we can use to further calibrate these models
51:07and that can be used to guide policy.
51:14I'm going to try to finish up very quickly here.
51:17There are a couple of key messages from this work
51:19that we tried to convey to policy makers.
51:21The first is that the state is doing pretty well,
51:23in terms of suppression of contact, closure of schools
51:26and the stay at home order have effectively reduce
51:28transmission and hospitalizations in Connecticut.
51:32If contact increases quickly, the state's at serious risk
51:37of big resurgence by later summer 2020.
51:40Real time metrics that policy makers have access to
51:42are really not going to serve as an early warning system
51:46for that resurgence.
51:49The state probably needs to be evaluating future projections
51:53under realistic contact scenarios for the state.
51:57We still have a lot of uncertainty that we tried to capture
52:01in model projections about cumulative incidence,
52:04asymptomatic fraction,
52:06how things are going to go with children,
52:10the effects of enhanced testing and contact tracing
52:13and how contact patterns may change following reopening.
52:18So we are issuing a series of reports, which you can read
52:22online and we will be updating them in real time
52:27as the summer goes on.
52:28You can find them at this URL.
52:29You can also email me and I'll point you to them.
52:34These are sort of continuously updated research products
52:36and I hope that they will represent
52:38the latest information from Connecticut
52:40and our latest predictions for the state as it reopens.
52:45Also, there's a document here which summarizes
52:47much more detail about the transmission model
52:50that I have given here in this presentation.
52:53I'm gonna skip over this stuff about our workflow.
52:56We can talk about it later, if anybody is interested,
52:59but this is just how we transition from regular research
53:03to doing this type of very active software development.
53:07I will end here.
53:08I want to thank all of the people in the group
53:11and beyond, who have been working on this tirelessly
53:13over the last couple of months.
53:15All of the products that I've told you about
53:17are publicly available.
53:18You can find the source code on Git
53:21on our Git repositories and you can find the web application
53:24and the reports online as well.
53:27So I'd be happy to take any questions.
53:29- Thanks, thanks Forrest for the last part.
53:32I think some people have some questions using the chat box.
53:39Ken asked, "Is the model used at currently proposing
53:43"used at hospital or by your medical group?"
53:50- The ICU planning app
53:53has been used, we know, and possibly is being used
53:56at Yale New Haven Hospital.
53:58The projections for Connecticut are not intended for use
54:01in any particular hospital systems,
54:03though I think they will be of interest
54:05to leaders of systems who are planning to accommodate
54:10a potential second wave of infections
54:12as it might occur later in the summer.
54:15I hope that as we get farther in the summer,
54:18if there is a second wave that appears to be coming,
54:21that the projections will be useful in planning
54:23capacity expansion efforts, possibly at or beyond levels
54:27that we already saw in April.
54:31So we will be generating any information
54:33that decision makers at those hospital systems
54:37think would be useful as they plan their response.
54:40That's a great question.
54:42- Thanks.
54:43And...
54:49Let me see and Sherry asked,
54:55"In the first reopening model, what amount was the reopening
54:58"assumed to start in?"
55:01- Exactly on May 20th,
55:03which is when the governor began the process of reopening.
55:08It is also true that the governor has been
55:13giving information about potential reopening plans
55:15for a very long time
55:17and that there is some change in contact as people
55:19begin to anticipate those changes in policy.
55:23I think that if you are looking at human mobility data
55:28from cell phones and other sources,
55:31you will see that people have been moving around
55:33for a while, increasing their level of activity
55:36outside of the home, even before May 20th in Connecticut.
55:41Whether that has actually resulted
55:46in a substantial increase in transmission remains to be seen
55:50but I don't think we should assume that just because
55:52people are moving around and possibly returning
55:54to some types of work that there will be a corresponding
55:58increase in transmission.
56:02- Okay thanks.
56:03Daniel asks, "Is the increase in incidence starting
56:07"in September a cumulative effect of prolonged increase
56:14"in contact."
56:15- Can I just ask the question directly?
56:16So I'm wondering, in the parts where you're showing
56:18the two reopening models, it looked like the curve
56:21starts to go back up around August,
56:23September in the slow one.
56:25I'm wondering if that's because you reach a threshold
56:28above a certain percentage of contact
56:30or if it's a cumulative effect?
56:32Like, if we were to keep contact at .2 for example,
56:36throughout all of this time and it weren't to increase
56:39above a threshold, is there a situation which you don't see
56:42that tail come up again?
56:45- Yes, great question.
56:47If you like to think in terms of the effective
56:49reproduction number, this increase just corresponds
56:52to a time about three weeks after
56:55that number goes above one.
56:58So there is a threshold effect and to answer your question,
57:00if contact were to remain below a level
57:06that would give that value of one,
57:08then you would not see this type of resurgence.
57:11I think as a practical matter, it is very unlikely
57:14that the state can avoid a situation where the effective
57:17reproduction number does above one.
57:21I think this is not the stated strategy of anyone
57:24and it's probably not, but I think it is the realistic
57:29expectation about what will happen in reality.
57:32The reality is that the state is going to try very hard
57:35to increase a level of contact just about to that level,
57:39where they would see some local outbreaks
57:41that can be extinguished but they will try to maximize
57:45the level of contact, meaning economic activity
57:49and social mobility
57:53that the state can achieve.
57:54So they'll try to get as much economic productivity
57:57and contact as they can without causing resurgence
58:03or large outbreak or an overrun of hospital capacity.
58:08- Thank you.
58:09- Thanks.
58:12- Akil here have two questions.
58:15So the first one is are there any assumptions
58:17of the proposed population who have Covid-19
58:20but have not been tested?
58:24- There are implicit and explicit assumptions
58:26about that proportion.
58:28I think we can produce predictions
58:31for the current prevalence and also cumulative incidence
58:37but those predictions depend quite a lot on our prior
58:39assumptions about the asymptomatic faction.
58:44We don't have very precise information about how many
58:48or what proportion of infections are totally asymptomatic
58:51and would go undetected by the healthcare system
58:54because people don't seek testing or seek care of any kind
58:58when they're not feeling sick.
59:02So certainly, we can try to learn about those things.
59:03There's some information in the available case counts
59:07and in hospitalizations and deaths about that stuff,
59:12but we still have a lot of uncertainty about current
59:15cumulative incidence.
59:17I think it's fair to say that currently prevalence
59:18is quite low in Connecticut.
59:22- Okay thanks.
59:22I guess I saw something new saying they test the people
59:26(unclear speaking).
59:29Because they can test other people that have the ability
59:32and then they have some estimate of the asymptomatic case,
59:37the rate of them?
59:38- Yes, that's true.
59:41In some very specific settings, like institutional settings
59:44like nursing homes and correctional institutions,
59:47you can test everybody and then you can learn how many
59:51infections are asymptomatic.
59:53The question then becomes of how representative
59:57those samples are compared to the rest of the state.
01:00:01Is it safe to take situations where people
01:00:05are living in very close proximity
01:00:08and possibly poor health conditions and to generalize
01:00:13all of that information to the state?
01:00:15I think there is some very good anecdotal evidence
01:00:17from prisons, from nursing homes
01:00:18and also testing systematic testing of healthcare workers
01:00:23that we can try to take into account,
01:00:26but it remains unclear how generalizable
01:00:29that information is.
01:00:30For example, healthcare workers may be immunologically
01:00:33somewhat unlike members of the general population
01:00:36who are not continuously exposed to different types
01:00:40of illness and to coronaviruses in particular.
01:00:43So I would hesitate to take large screening studies
01:00:45of nurses for example, and apply the asymptomatic fraction
01:00:51or prevalence or incidence in that sample
01:00:53to the general population.
01:00:56- Thanks.
01:00:58And the second question of Akil is can Covid-19 models
01:01:03from different states learn from each other?
01:01:06I have relay the question is because currently your model
01:01:10is most about stating the data
01:01:11and you can validate how good the model is.
01:01:15Because states, maybe they have their reopening plan
01:01:18at different times, can this provide useful information
01:01:21about how good the model is by learning
01:01:24from different states.
01:01:25- Yes, great question.
01:01:27It is always true that information from other contexts
01:01:30can be very useful if you know what is different
01:01:33in those other contexts.
01:01:35I would love to be able to use more granular information
01:01:37from neighboring states throughout the northeast
01:01:40to inform projections from Connecticut,
01:01:42'cause as we know, Connecticut is not an island
01:01:45and as soon as New York opens up and people start working
01:01:47in New York, then everything will change quite a lot,
01:01:50quite quickly in Connecticut.
01:01:53So I would like to share information.
01:01:56We have focused on Connecticut here because we have very
01:01:58detailed information about Connecticut but no special access
01:02:02in Massachusetts, Rhode Island and New York.
01:02:07So that's why we've done it, but I think it will become
01:02:09very important and I always thought
01:02:13it would be the job of the CDC and the US
01:02:16to synthesize a national and local projections
01:02:18and to gather all the granular local information
01:02:21and to put it all together.
01:02:22That has not happened in this particular pandemic.
01:02:26So I think everyone else is trying to scramble
01:02:31to aggregate information at the right levels
01:02:33to produce predictions that are actionable locally.
01:02:37But there's not coordination right now
01:02:39between groups that are doing state-specific
01:02:42reopening plans, unfortunately.
01:02:44As for whether the differences or staggered reopening
01:02:48can be used as a kind of instrument to identify
01:02:50the causal effects of reopening, I assume that's the subtext
01:02:54of the question, the answer is yes.
01:02:57I think people are very interested in doing that.
01:02:59The problem is that reopening is somewhat endogenous.
01:03:03The states to reopening as a function of the conditions
01:03:06currently in the states and also obviously,
01:03:09as a function of the political considerations
01:03:12of the leadership and of the population.
01:03:16Right now I don't think it's safe to say that reopening
01:03:19occurs randomly in some time interval
01:03:22and that we can exploit that randomness in a simple way
01:03:24to assess the effect of reopening.
01:03:27Certainly, some of the states that we observe
01:03:29reopening quickly, take Georgia for example.
01:03:34Those states are likely to see at least local
01:03:36and possibly very broad resurgences and outbreaks
01:03:42that may result in reversion to more restrictive movement
01:03:45conditions in those states.
01:03:48So I think really, there's this going to be a long,
01:03:50longitudinal sequence of treatments,
01:03:53meaning changes in state regulations and then outcomes,
01:03:57which the regulators will observe
01:04:00and then this kink of cat and mouse game,
01:04:01where decision makers try to tamp down on local outbreaks
01:04:06and then respond to ones that occur in the future.
01:04:11So we will try to learn about the effects of all those
01:04:13interventions and changes in policies
01:04:16but I think that there is cause for some skepticism
01:04:20in really learning a generalizable causable effects
01:04:24just from the time series.
01:04:27- Thanks.
01:04:29I guess one last very specific question about a talk.
01:04:33So Paul asked, "Have you considered how real time data
01:04:36"metrics, such as oxygen sensors from fitness trackers
01:04:39"could effect your predictions?"
01:04:41- Very interested in distributed measurements
01:04:44at the population level that could be helpful
01:04:46to inform some of these things.
01:04:49I think that we have not yet seen widespread adoption
01:04:53of mobile apps
01:04:58for self monitoring for contact tracing.
01:05:04There is some adoption of thermometers and oxygen sensors
01:05:10but as far as I know, there are no data streams
01:05:12that are publicly available.
01:05:14- This is Paul Forcher, I asked the question.
01:05:18There are some, there's--
01:05:20I'm participating in two studies.
01:05:22One that's run out of by Mike Snider,
01:05:26who use to be at Yale who's head of Stanford Genomics.
01:05:31The other one's institute
01:05:34and any of you can sign up for these things
01:05:37and if you have a fitness tracker that's tracking
01:05:41oxygen levels, there's emerging evidence that changing
01:05:46oxygen levels can be predictive of Covid infection
01:05:49before the patients are symptomatic and there's some...
01:05:54So I would, those are two studies that you could
01:05:57connect with and I wouldn't be surprised at all
01:05:59if they would share all of their realtime data
01:06:01that they're collecting with you.
01:06:02- Yeah, that is a great idea, thank you.
01:06:04With these--
01:06:05- Mike Snider's a former Yale person,
01:06:08so you already have an inroad with that guy.
01:06:12- Yeah, thank you, that's a great idea.
01:06:15- Okay thanks, I guess that's all questions for the talk.
01:06:20If you have any questions, I guess they can talk to you.
01:06:24Like the audience can talk to Forrest offline.
01:06:28- Please feel free to email me, anybody who has questions.
01:06:30- Some people want to hear more about the talks,
01:06:34like you didn't have time to cover,
01:06:35that I guess the interest you can talk to Forrest offline.
01:06:40Also, this talk will be recorded
01:06:41and will be publicly available.
01:06:44Also, on the previous talk are also recorded.
01:06:47I'll also send out a link to everyone
01:06:51in the School of Public Health,
01:06:54so if you want you can access it.
01:06:56Okay thank, thanks Forrest.
01:06:58- Thanks everyone.
01:06:59- And thanks for everyone. - Thanks everyone.