The Impact of Changes in Testing Practices on Estimates of COVID-19 Transmission

May 22, 2020

Information

Virginia Pitzer, ScD, Associate Professor of Epidemiology (Microbial Diseases)

ID5233

To CiteDCA Citation Guide

00:00I would now like to
00:02introduce our next speaker,
00:03doctor Virginia Pittser Doctor Pitts
00:05are joined the Yale School of public
00:08health as an assistant professor in 2012.
00:10Help you could see me, let's see yes um,
00:14her work focuses on mathematical
00:15modeling of the transmission dynamics
00:17of imperfectly immunizing infections and
00:19how interventions such as vaccination,
00:22improved treatments and progress
00:23in sanitation affect disease
00:25transmission at the population level.
00:27Doctor Pittser thank you for being here.
00:31Thank you, um,
00:32so hopefully everyone can see my slides now.
00:35Um, so I'm going to be talking about
00:38some of the recent work that we've
00:40been doing trying to look at how
00:43changes in testing practices may bias
00:46our ability to estimate important
00:48measures of transmission for Coed 19.
00:51Um and so just so that everyone
00:53is kind of familiar with some of
00:55the basic ways that we measure the
00:58transmission of any infectious disease.
01:01I'm going to introduce some of the
01:03two main measures of transmission
01:06that we're interested.
01:07The first measure that people may
01:10have heard about some of you I'm sure
01:13more familiar with is called the
01:16basic reproductive number or are not,
01:19and this is defined as the average
01:22number of secondary infections that
01:24are produced by a primary case in
01:26the fully susceptible population.
01:29So it's beginning of an epidemic
01:32when everyone is acceptable.
01:33How many people, on average,
01:36is that first case potentially
01:38going to infect?
01:39And the reason why this is an important
01:42measure is that it's closely related
01:44to the herd immunity threshold that
01:47is needed to completely interrupt
01:49transmission in the population and
01:51to eventually eliminate the pathogen
01:53from the population where you can
01:56get an estimate of that herd immunity
01:59threshold as 1 - 1 over are not,
02:01and so if you're randomly, for example,
02:04distributing vaccine within the population,
02:06then if you vaccinate 1 minus are
02:08not of the population.
02:10Then you should see the infection go away.
02:16Another important measure of
02:17transmission for infectious diseases,
02:19which is closely related to are
02:21not is the time varying affective
02:23reproductive number or RT,
02:26and this refers to the average
02:28number of secondary infections
02:30that are produced per primary case.
02:33Occurring through time at a particular
02:35time T and this accounts for both the
02:38buildup of munity within the population,
02:41which will serve to limit transmission as
02:43well as the impact of control measures,
02:46and so this is an important way in
02:48which we can kind of track transmission
02:51through time and see what impact
02:54control measures are having on transmission,
02:57and so both of these different measures
02:59and the methods that are available
03:02for estimating these different measures.
03:04Have been shown to be robust
03:06to under reporting of cases,
03:09and so it's generally assumed that
03:11only a fraction of true infections that
03:14are out there within the population
03:16are actually observed in detected.
03:19However,
03:19both methods for estimating both
03:21are not an arty.
03:23Assume that the fraction of
03:25infections that are detected and
03:27reported through time is constant
03:30such that there's no change in the
03:33reporting fraction through time.
03:35But we know particularly for the
03:37early stages of the COVID-19
03:39pandemic in the United States,
03:41that there has been a lot of
03:44variation in testing effort and
03:46reporting fractions through time,
03:48and this is just one example of data that
03:51comes from the Cove at tracking project,
03:54which was set up by.
03:57People at the Atlantic to digitize
03:59data coming from state public
04:01health Department websites on
04:03the confirmed number of code,
04:0619 cases in left in blue
04:08from Louisiana on in red.
04:10In the middle is the reported number
04:14of new tests per day in Louisiana and
04:17on the rights in purple and Gray is
04:21the fraction of those tests that are
04:24positive and you can see that there's.
04:27Some sort of important patterns that
04:29you're seeing in the data where early on
04:33when testing capacity was quite limited,
04:36the number of or the percentage of tests
04:39that were positive tended to be quite high,
04:43but Louisiana managed to ramp up
04:46its testing practices quite quickly
04:48in kind of mid March and eventually
04:51change their testing criteria sometime
04:53between March 15th and April 15th to go.
04:57Come from preferentially testing
04:59individuals who are health care workers.
05:03For example,
05:03or at high risk to allowing anyone with
05:06a fever to be eligible for a test.
05:09And you can see that this is potentially
05:12reflected in a drop in the percent
05:14of individuals that were testing
05:16positive within the population.
05:18And then there are other funny
05:20things in the data where they did an
05:22audit of the commercial labs that
05:24were testing for COVID-19 between
05:26April 20th and April 24th,
05:28and they revise their total test
05:31numbers down such that if you.
05:33Calculate a daily number of tests
05:35from the cumulative number of
05:37tests you actually see.
05:38A negative number of tests,
05:40which obviously we know is not true,
05:42and so given the data that's
05:45available becomes very difficult to.
05:46Make this assumption that testing
05:49effort has been constant through
05:51time that we need to measure our.
05:53Estimates of the transmission
05:55rate for COVID-19.
05:57And so one way that we've tried
06:00to get at this question of,
06:02well,
06:03how could these differences and
06:05changes in testing practices affect
06:07our ability to measure the transmission
06:09rates of a new infection like COVID-19
06:11is to simulate what might happen
06:14in the population when we have a
06:16new infection being introduced and
06:18then simulate sort of different
06:20changes in testing practices and so
06:23to do this we can use what's called
06:25the basic essay are type model.
06:28In this model,
06:29is based on the assumption that
06:31whenever it when people are born,
06:34everyone is susceptible to infection,
06:36and so before a new infection is introduced,
06:39everyone in the population is susceptible.
06:42When the new infection gets
06:44introduced into the population,
06:46susceptible individuals can
06:47get infected at some rate,
06:49Lambda and in turn these individuals
06:51are infectious and can
06:52infect other individuals.
06:54So the rate Lambda here is dependent
06:56both on the number of susceptible
06:59individuals in the population as well
07:01as the number of currently infected.
07:04An infectious individuals
07:06within the population.
07:07But after a certain amount of time,
07:10we know that individuals stop being
07:12infectious and stop shedding the
07:14particular virus and may recover
07:15and build up some level of immunity
07:18that prevents further infection.
07:19And then finally individuals can die
07:22both of the disease or of natural
07:25causes from all of these states.
07:27And then all of this gets summarized
07:29into a series of differential equations
07:32in which the number of individuals
07:34in each state within the population
07:37changes through time in proportion
07:39to these particular parameters,
07:41and the current state of number
07:44of individuals in each state.
07:46And so we can use a model like this.
07:50Uhm, to simulate an epidemic where
07:53instead of using the basic Sir model,
07:56we add an additional E compartment
07:58which models individuals who are
08:01infected but not yet infectious.
08:03And we stochastically simulate an
08:05epidemic occurring through time,
08:06and this is just one example on
08:09the left here of the results of
08:12this stochastic simulation where we
08:14introduce one infected individual at
08:17Time zero in a population of a million.
08:20And allow the infection to kind
08:22of slowly take off and then in Day
08:2550 we decided we're going to come
08:28in and we're going to reduce the
08:30transmission rate by some amount.
08:32Such the epidemic starts to decline
08:33and then we can make assumptions
08:36about the reporting process,
08:37where we model both the.
08:40Probability that a true case is
08:42detected an tested and the observed
08:45cases are then some fraction of
08:47the overall number of infections
08:50times the reporting fraction.
08:52And that's plotted in blue here as
08:55well as the number of uninfected
08:58individuals who are tested,
09:00which we assume is some occurs
09:02in some proportion to the overall
09:05number of infections out there.
09:08As testing capacity starts ramping up.
09:11And then we also assume that individuals
09:13are tested and reported with some delay,
09:15where we assume a median of five and
09:17a half days between the time the new
09:19infection becomes symptomatic and the
09:21time they actually get tested and reported.
09:23And this was based on some
09:26early data out of China.
09:28And then to estimate the basic
09:30reproductive number or not.
09:31The way we do this is based on
09:33the rate of exponential growth
09:35at the beginning of the epidemic,
09:37where if you take this equation for the
09:40rate of change of the number of new
09:42infected individuals within the population.
09:45You assume that everyone is
09:47acceptable in the first place.
09:49And you do some math to solve
09:51this differential equation.
09:52What you find is that the number of
09:55new infections through time should
09:56be equal to the number of infected
09:59individuals initially times E to the RT,
10:01where this little R is equal to
10:04the growth rate of the epidemic
10:06or the slope of the log in
10:08the number of cases through time and is
10:11equal to are not minus one over D and
10:13so you can estimate are not based on
10:16this knowledge of what the growth rate.
10:19Through the epidemic is through time and D,
10:22which is the generational or the
10:24serial interval between one case and
10:26the case that that individual impacts.
10:29And then we can also estimate Artie
10:32by our knowledge of the sort of.
10:36Or inference of the underlying infection
10:38tree within the population where if
10:41you have one individual say he was
10:43infected on day four of the epidemic,
10:45they could have been infected
10:47by any individual on Day 3,
10:49two or one of the epidemic,
10:51and the probability that this
10:53individual on day one infected this
10:55individual on day four is just going
10:57to be a function of how likely the
11:00generation interval is to be 3 days
11:03compared to all the other possible
11:05generation intervals that could
11:06have given rise to this infection.
11:09And then we can look back to this
11:11infection occurring on Day One and ask
11:14well how many individuals did this
11:16person likely infect by summing up the
11:18probability that all the individuals
11:20on subsequent days was infected by
11:22this particular individual on day
11:24one on Day 2 on day three, etc.
11:27And so when you put all of this together.
11:31Oops, sorry. Um?
11:32What we can do here is to estimate
11:35the impact of either an increase or
11:38decrease in the testing probability
11:41through time. We're on the top.
11:43Here we are assuming that the testing
11:45probability through time is constant
11:47and the number of true cases.
11:49The number of tests in the number of
11:52confirmed cases is plotted in black,
11:54red and blue on the left.
11:57The percent of tests that are positive
11:59is plugged in purple in the middle,
12:01and our estimate of the real time time
12:04bearing reproductive number is in green here.
12:07Based on the observed number
12:08of cases and in black,
12:10based on the true number of
12:12infections through time,
12:13and generally what we find is that
12:15when the probability of a true case
12:18being tested is increasing slightly
12:20through time plotted in the middle here,
12:22you'd expect to see a slight increase
12:24in the percent of individuals
12:26testing positive through time.
12:28As well as a slight overestimation of the
12:30value of the basic reproductive number,
12:33because the number of observed cases
12:36is growing faster than the number of
12:38two infections through time as well
12:40as a slight overestimation of the
12:43real time time varying reproductive
12:45number through time.
12:46Whereas if the probability of detecting a
12:49true cases slightly decreasing through time,
12:51we slightly underestimate
12:53the value of are not,
12:55and we slightly underestimate again
12:57the value of Artie through time.
13:01Um,
13:01however,
13:02this increase or decrease in the
13:04percent positive through time might
13:07also be occurring because individuals
13:09who are not infected are being
13:12becoming more likely to be tested.
13:14Perhaps because there's an
13:16increase in testing capacity.
13:18And so instead we assume that
13:20the number of individuals
13:22tested for every true cases
13:23just increasing through time.
13:25Again, we just expect to see potentially
13:27a decrease or an increase in the
13:29percent of the individuals that
13:31are testing positive through time.
13:33But in this case our estimates of
13:35are not an arty tend to be unbiased,
13:38so it's really important to
13:40understand the context in which
13:42these increases or decreases in the
13:45percent positive may be happening.
13:47Another possibility is that there is
13:49a change to the testing criteria which
13:52could lead to a sudden increase or
13:54decrease in the testing probability or
13:57the probability that a true case gets tested.
14:00And if this is the case,
14:02and you see a large increase in
14:04the probability that a true cases
14:06actually getting to test tested.
14:08We in this case the model estimates
14:10that there should be a slight
14:13bias in the estimate of are not,
14:15and they larger bias in your estimate
14:17of the time bearing reproductive
14:19number such that you see this sort of
14:22large increase that is not consistent.
14:24Slow decline in the true number of
14:28infections occurring through time.
14:30And similarly,
14:30if you see a decrease in the
14:33testing probability through time,
14:36you see a similar bias occurring.
14:39Again,
14:39however,
14:40this increase or decrease in the
14:42percent positive through time could
14:44just be due to a change in the number
14:47of tests that are being performed,
14:49or a change in the testing capacity.
14:51For example,
14:52if a new private lab starts
14:54testing individuals.
14:55So in this case,
14:56you'd see a chart start changing the
14:58number of tests occurring through time,
14:59but you would not expect there to be any
15:02bias in your estimates of are not or RT.
15:06And then finally we also looked at
15:08what would happen if there was a
15:10change in the reporting delay through
15:12time within either an increase or
15:14decrease in the reporting delay.
15:16In this case,
15:17it would be harder to accept that by
15:19looking at the percent of individuals
15:21testing positive through time,
15:23but we could potentially see
15:24a relatively large bias in our
15:26estimates of both are not and Artie,
15:29so this is a potentially more problematic
15:31change in the testing process.
15:34And so now we're looking at applying
15:36these methods to learn something
15:38about how our estimates of the real
15:41time and basic reproductive number
15:43of COVID-19 in the US may or may
15:45not be biased by these different
15:47changes in testing practices.
15:49And this is data for all of the
15:52US in which we have the number,
15:55total number of tests in the
15:57number of positive tests plotted
15:59on the log scale on the left here,
16:02as well as the percent of.
16:04Individuals testing positive through
16:06time for both daily data as well as
16:09kind of cumulatively overtime on
16:11it in the middle and then our best
16:14estimate of the real time time varying
16:16reproductive number through time.
16:18Where overall what we estimate
16:20is that the basic reproductive
16:22number before March 24th,
16:23when things start to flatten
16:25out is estimated to be
16:27around 3 1/2 with a time varying
16:29reproductive number of starting off
16:31around 4:00 and then kind of quickly.
16:34Decreasing and then kind of has been
16:38hovering just at or below one since around
16:42early to mid April in the entire US.
16:46And then we can look at this, uhm,
16:49broken down for each of the states where
16:52we start to see kind of more an more
16:56inconsistencies in reporting as well as
16:58low probabilities of individuals kind of
17:01being tested early on in in the epidemic,
17:04where this starts to kind of emerged
17:07as a greater potential bias in some of
17:10these estimates of the time varying
17:12reproductive number through time.
17:15Particularly, for example,
17:16in Washington,
17:17where there's this strong day
17:19of the week effect,
17:20you can see within the testing process,
17:23which is probably causing some of
17:25these kind of Wiggles in their
17:28time varying estimate of the
17:30reproductive number through time.
17:31In in California,
17:32generally what we see these kind of
17:35large increases in the number of tests
17:37'cause they had some inconsistencies
17:39and particularly the reporting of the
17:41negative test through time, which we
17:43don't think will bias estimates of RT.
17:46But this sort of lack of.
17:48Slow ramp up and recording early on
17:50may have led to these sort of larger
17:53estimates of the RT value early on,
17:55and similarly in New York West testing
17:58capacity kind of was limited early on.
18:00We think that this sort of initial
18:02peak in the estimated real-time
18:04reproductive numbers is based
18:06on this sort of large increase,
18:08but you can then see kind of our
18:11estimates of the most recent measures
18:13of Artie are probably not going
18:15to be biased by these sort of,
18:17for example.
18:18Slow decrease in the percentage of
18:21individuals testing positive in New York
18:23because this is mostly been associated with,
18:26UM,
18:26a ramp up the testing capacity in
18:29the number of tests conducted through
18:31time and these slow changes didn't
18:34seem to bias our estimates of RT.
18:37And so finally,
18:38what we've been doing more recently
18:40is to work on kind of incorporating
18:43some of this data to develop now casts
18:46of the current COVID-19 epidemic,
18:48where we can take information
18:50about the observed number of cases
18:53occurring in blue here and deaths
18:55occurring in green here and infer
18:58back based on our prior knowledge of
19:00the reporting process to estimate the
19:02number of new infections occurring
19:04through time within the population.
19:07And this is just one example of.
19:09Data from Connecticut where you
19:12can see that the number of new
19:15infections here is peaking quite
19:17a bit earlier than the observed
19:19number of cases from the UM,
19:22Connecticut Department of Public health,
19:24and this allows for more accurate estimates
19:26of the time bearing reproductive number,
19:29which corrects for the reporting
19:31delays that we know are going on.
19:34And now these time varying
19:36reproductive numbers can allow
19:38for more accurate assessment of
19:40the impact of interventions.
19:42For example,
19:43these changes in mobility
19:45that sod was talking
19:47about earlier. And so finally,
19:49I'd just like to thank some of
19:51my collaborators on this work,
19:53including a series of a number of
19:56individuals, both PhD students,
19:57postdocs, as well as other faculty
19:59from the school, public health,
20:01public health modeling unit,
20:03as well as Nick Menzies from
20:05Harvard School of public
20:06health and funding from NIH.
20:12Thank you very much Doctor Pittser.