"Consumer Sleep Technologies-Potentials, Pitfalls, and the Future of Ambulatory Sleep Tracking" Cathy Goldstein (01.06.2021)
January 11, 2021ID6068
To CiteDCA Citation Guide
- 00:20Alright hi everybody.
- 00:22Welcome, my name is Lauren Tobias.
- 00:26I'd like you to welcome to welcome
- 00:28you back to our Yale Sleep Seminar
- 00:30for the Winter Spring semester.
- 00:32I have a few quick announcements
- 00:34before I introduce today's speaker.
- 00:36So first, please take a moment to
- 00:39ensure that you're muted in order to
- 00:41receive CME credit for attendance,
- 00:43please see the chat room for instructions.
- 00:46You can text the unique ID for
- 00:48this conference anytime until 3:15
- 00:50PM Eastern Time if you're not
- 00:52already registered with DLC and me.
- 00:54You do need to do that.
- 00:561st, If you have any questions
- 00:58during the presentation,
- 00:59I encourage you to make use of the
- 01:01chat room throughout the hour and
- 01:03will also have an opportunity for you
- 01:05to unmute yourself and ask questions
- 01:07directly to Doctor Goldstein at the end.
- 01:09We do have recorded versions of
- 01:11these lectures which are going to
- 01:13be available online at the link
- 01:15provided in the chat and then finally,
- 01:17please feel free to share our
- 01:18announcement for a weekly lecture
- 01:20series to anyone else who you
- 01:21think may be interested,
- 01:23or contact Debbie Lovejoy to
- 01:24be added to our email list.
- 01:26So this afternoon I have the pleasure
- 01:29of introducing Doctor Kathy Goldstein.
- 01:31Doctor Goldstein received her medical
- 01:33degree from the Medical College of Georgia.
- 01:36Didn't ask her about her feelings
- 01:39about events of this morning yet.
- 01:41She completed a neurology residency
- 01:43at the University of Colorado and
- 01:46her sleep fellowship at Northwestern,
- 01:48followed by a Masters in clinical research,
- 01:51design,
- 01:51and statistical analysis at
- 01:53the University of Michigan.
- 01:55She's currently a clinical associate
- 01:57professor in neurology at the
- 01:59University of Michigan, Ann Arbor.
- 02:01Her research interest is in sleep and
- 02:04circadian interactions with health
- 02:05and for several years she's been
- 02:08interested in the role of consumer
- 02:11sleep technologies and Sleep Medicine,
- 02:13including the role of wearables and
- 02:15smartphones, sleep tracking acts, apps.
- 02:17She's an active clinical educator.
- 02:19She lectures nationally and and
- 02:21she is slated to be Co.
- 02:24Chair of the ASM Sleep Trends
- 02:26Conference for the upcoming year.
- 02:28She's authored ASM position statement
- 02:30on the role of consumer sleep tracking.
- 02:33And also artificial intelligence
- 02:34and Sleep Medicine.
- 02:35And just recently she published an
- 02:38article that encourage everybody to
- 02:40check out in JC SM that shares the
- 02:42results of a large national survey
- 02:44examining the impact of COVID-19
- 02:46stay at home orders and sleep,
- 02:48health and working patterns,
- 02:50which I found really interesting.
- 02:52So with that I am delighted to have
- 02:55Doctor Goldstein here to discuss
- 02:56a topic that I know many of us are
- 02:59eager to learn more about consumer
- 03:02sleep technologies potentials.
- 03:03Pitfalls in the future of ambulatory
- 03:05sleep tracking? Thank you.
- 03:07I will turn myself off Doctor
- 03:09Goldstein and turn it over to you.
- 03:12Awesome,
- 03:12thank you so much Lauren and big virtual.
- 03:15Hello to everybody.
- 03:16I'm going to go ahead and get started here.
- 03:20This is your code to text to get your CME.
- 03:24And I have no conflicts
- 03:27of interest to disclose.
- 03:29OK, so here's our road map
- 03:32for today's discussion.
- 03:34Casino what is to come?
- 03:36And I do want to disclose the fact that
- 03:39this is not gonna be a review of all
- 03:43the available consumer technologies.
- 03:45The entire landscape that
- 03:47I'm going to present to you,
- 03:50a general framework with which you can
- 03:53approach consumer sleep technologies
- 03:55in general and with the specific use
- 03:57case of wearable sleep estimation
- 04:00devices that use motion and heart rate.
- 04:03So first of all.
- 04:05Why do we even care bout tracking sleep?
- 04:09So we have this really big paradox
- 04:11in Sleep Medicine and research where
- 04:14we know that sleep deprivation and
- 04:16sleep disorders cause their detriment
- 04:18through chronic exposure every
- 04:20night to disturbances in sleep,
- 04:23quality, erratic sleep timing or
- 04:25reduce sleep duration.
- 04:26That how do we measure sleep with an
- 04:30overnight polysomnogram is our gold
- 04:32standard which is great to diagnose.
- 04:34Sleep apnea, certain parasomnias
- 04:36and movement Santa titrate.
- 04:38Happy,
- 04:38but this is really impractical
- 04:40beyond one to two nights or abuse.
- 04:44So how do we track sleep objectively
- 04:46so that we can get more than one
- 04:49to two nights of sleep recording?
- 04:52Well, we do have an option.
- 04:54I am the currently accepted option for
- 04:57clinical Sleep Medicine and research
- 05:00is actigraphy anac trigger fee in the
- 05:03traditional sense is cleared by the FDA,
- 05:05and it uses a wrist warm accelerometer
- 05:08to monitor motion during 24 hour period
- 05:11with the rationale that we move more than.
- 05:14When we're awake,
- 05:15then we do when we're asleep.
- 05:17So the way this works is you give
- 05:20an actigraph to your patient.
- 05:22They wear the device and there's
- 05:24a piezo electric accelerometer or
- 05:26Mens accelerometer in the device
- 05:28which collects motion,
- 05:29which is then digitized into
- 05:30activity counts the data from the
- 05:33devices then downloaded.
- 05:34And a software package uses an
- 05:37algorithm to determine sleep from wake,
- 05:40and we like actigraphy because Actigraphy
- 05:43performance has been compared to PSG,
- 05:45and the findings of that
- 05:48performance published in many,
- 05:50many research publications, so.
- 05:52What's the problem?
- 05:53If we have this small, unobtrusive
- 05:55way to measure sleep over days to weeks,
- 05:58well, there's quite a bit.
- 06:00As you can tell.
- 06:02So first of all, actigraphy,
- 06:04typically just measures movement.
- 06:05Some devices measure light.
- 06:07The data acquisition methods and storage
- 06:09methods are really outdated with actigraphy,
- 06:12so before you give the device to somebody,
- 06:15you have to initialize it
- 06:17and to recover the data.
- 06:19You typically have to plug
- 06:20it back in via USB ports,
- 06:23and so we can't really transmit an.
- 06:26Evaluate the data in real time.
- 06:28I've even seen researchers male actor
- 06:31graphs back and forth to each other,
- 06:33which is pretty prehistoric
- 06:35in this day and age.
- 06:37Always talk about Actigraphy is validated,
- 06:39it's validated it's validated,
- 06:41and yes,
- 06:42it's been compared to polysomnogram
- 06:44many times, but it's not that great,
- 06:47so it has a high sensitivity
- 06:49but a low specificity for sleep,
- 06:52which means that the algorithms
- 06:53applied to actigraphy data typically
- 06:55misinterpret nonmoving wakefulness,
- 06:57as sleep, and So what that means is
- 06:59these have very high accuracy and
- 07:02healthy sleepers during nighttime sleep.
- 07:05However, as sleep disruption
- 07:06increases throughout the night.
- 07:08There's more wake after sleep onset,
- 07:10increased sleep onset latency,
- 07:12the accuracy of the sleep metrics that are
- 07:15output by the software are going to go down.
- 07:19And Speaking of software that is
- 07:21another pitfall of actigraphy.
- 07:22The traditional actigraph software does
- 07:24not interface with the electronic health
- 07:27record or any other digital health platforms,
- 07:30so that makes it difficult to kind
- 07:32of track sleek sleep along with
- 07:35other health metrics in an easy,
- 07:37meaningful, practical way.
- 07:38The use of actigraphy requires
- 07:40quite a bit of data cleaning.
- 07:42Given that I'm in bed intervals need
- 07:44to be set for appropriate analysis,
- 07:47and this interpretation,
- 07:48as well as the set up the patient
- 07:50recording even though we have a CPT code.
- 07:53This is typically not reimbursed,
- 07:55so actigraphy is not going to be practical
- 07:58for a clinical workflow in most cases,
- 08:00and it's going to be burdensome
- 08:03on the research team.
- 08:05Additionally,
- 08:05actor graphs are expensive.
- 08:07There are at least $800 a device
- 08:09for the FDA cleared devices,
- 08:11so this is not really practical
- 08:13for large population studies,
- 08:15and it restricts use clinically,
- 08:17particularly if there is a sleep
- 08:19clinic in an underserved area.
- 08:21One big thing that people don't talk
- 08:23about is it these devices are owned by
- 08:26the health system where the research
- 08:28team they're not owned by the patient,
- 08:31so even though we consider actigraphy
- 08:33to be longitudinal sleep recording.
- 08:35These typically don't go beyond 1
- 08:37two weeks as opposed to devices
- 08:40that patients might own and use
- 08:42themselves that are worn over weeks,
- 08:44days, months, even years.
- 08:47So despite the strong wording
- 08:49that we see here,
- 08:51that Actigraphy should be used before
- 08:54the MSL T in circadian rhythm disorders.
- 08:58What happens when we stop following
- 09:01the icst three and start getting real?
- 09:04The real world Sleep Medicine clinic
- 09:07and research we ask about sleep
- 09:10cross sectionally.
- 09:11So for example,
- 09:13will say how many hours do you
- 09:16sleep on average?
- 09:18At face value,
- 09:19this seems like a very kind of easy,
- 09:21simple question,
- 09:22but obviously this can be
- 09:24interpreted in a multitude of ways.
- 09:26For example,
- 09:26how many hours do you sleep at night?
- 09:29How many hours do you sleep at night,
- 09:31plus how many hours do you sleep during naps?
- 09:34What time do you go to bed and when
- 09:37do you wake up and do the math?
- 09:40Is this during the work week during the
- 09:42weekend so we can see that this can
- 09:45be misconstrued if we use lanja tude?
- 09:47No self report?
- 09:48With the sleep diary.
- 09:50So daily self report recording of sleep.
- 09:52That might be a bit better,
- 09:54but there's problems with that as well.
- 09:56This is burdensome to our patients
- 09:58and is oftentimes incomplete,
- 09:59or the classic example of people filling
- 10:01these out in the waiting room before
- 10:03they come in for their PSG MSL T.
- 10:06This could be difficult for certain
- 10:08populations specifically like the very
- 10:09young or the older with cognitive problems.
- 10:11And again, even if your patients to
- 10:13fill this out, what did they do?
- 10:16Fax it to you or FS,
- 10:17send it as an email attachment.
- 10:19So not really relevant for the technical
- 10:23capabilities that we have in 2020.
- 10:25Now, so why is this important?
- 10:28Does this really affect
- 10:29our clinical practice?
- 10:31Well, we suspect it does,
- 10:33so the American Academy of Sleep Medicine
- 10:35did do a systematic review and found
- 10:38that the objective information from
- 10:40Actigraph recordings is beneficial.
- 10:42An provides distinct information over
- 10:44sleep logs in a variety of disorders,
- 10:47but the evaluation and management.
- 10:50Additionally,
- 10:51the group from Harvard has already
- 10:53asked researchers to stop querying
- 10:55self reported sleep duration.
- 10:57As you can see here,
- 10:59and that's because objective sleep
- 11:01recording is also incredibly
- 11:03relevant for research because the
- 11:05in accuracies that are possible
- 11:07with self report sleep duration
- 11:09can depend on patient demographics,
- 11:11whether or not they have comorbid
- 11:13sleep disorders.
- 11:14When the question is asked an way,
- 11:17the question is asked, so this can lead.
- 11:20Tobias in research.
- 11:22Another problem with using self
- 11:24report sleep duration and research
- 11:27is that the whole construct of
- 11:29sleep duration as an equivalent
- 11:32for sleep health which we know is
- 11:35probably not the case,
- 11:36and there's growing evidence to support this.
- 11:39For example,
- 11:40something called intraindividual variability.
- 11:42So the variation of sleep
- 11:44parameters around the mean,
- 11:46which has been shown relevant
- 11:48for a variety of outcomes,
- 11:50including psychiatric, metabolic,
- 11:51and cardiovascular.
- 11:52This is going to require
- 11:54longitudinal tracking which.
- 11:56Is difficult to do with self report and so
- 11:59a passive objective method is required.
- 12:03Additionally,
- 12:03as we said that sleep duration
- 12:06doesn't equate to sleep health eh 7
- 12:09domain definition has been proposed,
- 12:11and that's a combination of both self
- 12:14report and primarily objective measures,
- 12:17including in addition to duration,
- 12:19continuity, timing,
- 12:20rhythmicity regularity and
- 12:21subjective sleepiness and quality,
- 12:23as well as a composite of extreme values.
- 12:27And when you combine,
- 12:29combine all seven domains an.
- 12:31Additionally,
- 12:31the number of composite extreme values.
- 12:34We can see that this has a much
- 12:37bigger impact on mortality then
- 12:39just sleep duration alone,
- 12:41which I don't know if you
- 12:43can see my cursor here,
- 12:45but here are the seven domains combined
- 12:48as well as the number of extreme values,
- 12:51and then we have sleep duration is
- 12:54all the way over here when used as
- 12:57an isolated sleep variable to predict
- 12:59mortality in a random forest model.
- 13:02So it seems like longitudinal objective
- 13:05sleep measurements are pretty important.
- 13:07So if we can't use actigraphy
- 13:09because of the limitations,
- 13:10what are options?
- 13:11It seems like we sure have a lot
- 13:14with consumer sleep tracking technology.
- 13:16We have wrist worn devices,
- 13:18as you can see here from a variety
- 13:21of manufacturers.
- 13:22More and more rings are
- 13:24coming on to the market dry,
- 13:26e.g headbands mirabal's that are
- 13:28either at the bedside or under the bed.
- 13:32So before we can really talk about
- 13:35the possibilities and the problems,
- 13:36that is consumer sleep technologies hold.
- 13:38We have to understand a little bit
- 13:41more about how they work or how
- 13:44do we get from signal to sleep.
- 13:46And I'm not sure I can't see everybody,
- 13:49so it's hard to know Peoples ages.
- 13:51But for those of you that
- 13:53remember Schoolhouse Rock,
- 13:54the best schoolhouse rock is.
- 13:56How does a bill become a law?
- 13:58And here we have Bill,
- 14:00and he's just sitting here on Capitol Hill.
- 14:03And just like a bill, is not a law.
- 14:06Smartwatches,
- 14:06rings etc aren't direct measures of sleep.
- 14:09There are a bunch of steps in between,
- 14:11so this is an oversimplification,
- 14:13but this does give you some
- 14:15idea about how these.
- 14:17Actually work,
- 14:17so your device is going to
- 14:19or most risk form devices.
- 14:22At least they're going to typically
- 14:24include a Tri axial accelerometer,
- 14:26which is going to record motion
- 14:28in three directions, XY and Z,
- 14:30and that's going to give you raw signal
- 14:33to the motion we see pictured here,
- 14:36which is actually from an Apple Watch
- 14:39and then optical photoplus Mogra Fi,
- 14:41and that is a visual,
- 14:43light based way to estimate blood volume,
- 14:46which is then converted into
- 14:48pulse Anna variety.
- 14:49Of computed features from pulse,
- 14:50including things like heart rate variability
- 14:53and then your motion and your heart rate.
- 14:55Features are going to go into an
- 14:58algorithm which is going to even be there
- 15:00through an app or a cloud based server,
- 15:03and that algorithm is typically black box.
- 15:05As you can see from the figure and then
- 15:08the sleep information is returned to the
- 15:10user both visually and in summary metrics.
- 15:13As you can see in the example below.
- 15:16So it's a myth that your watch track sleep,
- 15:20but what the truth is is that algorithms
- 15:23estimate sleep from wearable sensor,
- 15:25acquired physiological
- 15:25signals and arrived features.
- 15:27But sometimes that's a bit difficult
- 15:29to discuss with their patients.
- 15:32Now, these algorithms that estimate sleep.
- 15:34Where do they come from?
- 15:36So let's start by taking a look
- 15:39at our traditional actigraphy,
- 15:41so the algorithms that are
- 15:43used by FDA cleared ACTIGRAPHY,
- 15:45and the associated software typically are
- 15:47ones that are more than 2025 years old.
- 15:50So on the top left you can see
- 15:53the Respironics actor algorithm,
- 15:55and on the bottom right that's the cool
- 15:58Kripke algorithm in the Respironics Act.
- 16:01Aware Algorithm E sub zeros
- 16:03the epic being scored.
- 16:05And same with cold Kripke,
- 16:06we have a sub zero,
- 16:08so those are the epic of interest.
- 16:10And then there's a weighted sum and
- 16:13the negative numbers are the epochs
- 16:15that are before the epic at hand,
- 16:17being scored and the positive
- 16:19numbers or the epics after the
- 16:21epic being scored and you can see
- 16:23these received lesser weights and
- 16:25then if the sum crosses a threshold
- 16:27it's going to get scored.
- 16:29That epic is going to get Squared's
- 16:31wake and if it is below that
- 16:33threshold it'll get scored as sleep.
- 16:35So weighted sounds.
- 16:36Now how is this changed?
- 16:38So we use much more sophisticated
- 16:41methods to classify,
- 16:42sleep and wake from wearable sensor data.
- 16:45At this point in time an I worked
- 16:47with a couple mathematicians at
- 16:49University of Michigan Humane,
- 16:51Olivia Walsh and Danny Forger and
- 16:54we wanted to know if we could
- 16:56actually make our own algorithms
- 16:58to train models to classify sleep
- 17:01versus wake from an off the shelf
- 17:04consumer sleep technology and we
- 17:06actually use the Apple Watch and
- 17:08the reason was is that was the.
- 17:11Only one that we could get our hands
- 17:13on where we had access to the motion.
- 17:16An heart rate data for modeling
- 17:18purposes and this was funded by
- 17:20our athletics Department because
- 17:21they are very interested in sleep
- 17:24and sleep technology to improve
- 17:26recovery and performance,
- 17:27and for those of you interested in
- 17:29college basketball, it was pretty fun.
- 17:31Because Jordan Poole and Isaiah
- 17:33Livers did the UI for our app.
- 17:35So really neat project that we're
- 17:38grateful we have the funding for.
- 17:40And what we did is we had people
- 17:43wear Apple Watch is for a seven
- 17:45day ambulatory period,
- 17:46and the reason we use that ambulatory
- 17:49period is Danny Anna Livia.
- 17:51They're computational biologists
- 17:52that are focused on the modeling
- 17:54of the circadian Clock,
- 17:55and there is a very well validated
- 17:58differential equations model that you
- 18:00can pass activity through to get a
- 18:03Clock proxy that we used as an input,
- 18:05and then on the final night
- 18:07of this seven day recording,
- 18:09we had our subjects come in and they slept.
- 18:12Simultaneously with the Apple Watch
- 18:14and synchronized with Polysomnogram,
- 18:16and we obtained the rock celebration data.
- 18:19And at PPG heart rate from their
- 18:22Apple Watch on that same night,
- 18:25we use that Clock proxy feature as
- 18:27well as activity counts derived from
- 18:30the acceleration and PPG heart rate
- 18:32and arrived standard deviation of
- 18:35heart rate to train algorithms against
- 18:38scored PSG to classify wake versus sleep,
- 18:41and we use regular old logistic regression.
- 18:44As well as machine learning
- 18:46techniques for our classifier,
- 18:48I'm not going to go into the details of that,
- 18:53but our final algorithm performed as
- 18:55well as actigraphy or what's been
- 18:58historically cited for actigraphy
- 19:00in regards to sensitivity and
- 19:02even improved with specificity.
- 19:04But you guys can read about that
- 19:07in the paper.
- 19:08But I do wanna show you is that there are
- 19:11a few other groups that have done this.
- 19:14When the access to raw signal from consumer
- 19:17sleep technologies to been iaccessible,
- 19:18so Zang used a Microsoft Band,
- 19:20Roberts,
- 19:21used an Apple Watch Ann
- 19:22Beattie at all used to Fitbit.
- 19:24I believe they worked with Fitbit,
- 19:26so they can have access to that data and I
- 19:29won't go through all the details of this.
- 19:32But what I want you to kind of
- 19:35recognize is the breath of the
- 19:37features that are used.
- 19:38So we went to previous algorithms with.
- 19:40FDA cleared actigraphy that were just
- 19:43using weighted sums of activity counts
- 19:46to a variety of features from motion,
- 19:48an heart rate, an various computed features.
- 19:51As you can see here and then,
- 19:54these are going into machine
- 19:57learning algorithms to train their
- 19:59capability to predict sleep and wake
- 20:01so much as this is changed overtime,
- 20:04and we're really looking at
- 20:06different sleep wake classifiers
- 20:08that were previously accustomed to.
- 20:10But what everyone wants to know is
- 20:13what about the algorithms that are
- 20:15actually associated with the apps
- 20:17that come with the devices that your
- 20:20patients or research subjects actually use.
- 20:22So these these algorithms that other
- 20:25people have come up with aren't
- 20:27typically available in a usable way.
- 20:29Ours is open source which
- 20:31can be obtained from GitHub,
- 20:33but it's not enough.
- 20:35Closed app right now,
- 20:36so the problem is we don't really know
- 20:39the the apps that are available now
- 20:42and be associated with the CST's that
- 20:45are available on the market market.
- 20:47We don't really have details
- 20:49about the features that are used,
- 20:52the populations that the algorithms
- 20:53were trained on, or the methodology.
- 20:56So it's pretty unclear.
- 20:58Which makes us dependent on the already
- 21:01processed sleep stage output from these
- 21:04devices and their associated algorithms.
- 21:06So the question is,
- 21:08how good is this data?
- 21:10Which brings us to the
- 21:13question on everybody's mind,
- 21:14which is validation.
- 21:16And this is the position statement that the
- 21:19American Academy of Sleep Medicine put forth
- 21:22in regards to consumer sleep technology.
- 21:25And you can see that we
- 21:28talk about validation a lot.
- 21:30So what is this validation that's so
- 21:33important for consumer sleep technologies?
- 21:35So what's validation in general?
- 21:37So validation is the act,
- 21:39process,
- 21:40or instance of validating,
- 21:41especially the determination of the
- 21:44degree of the validity of a device.
- 21:46So what's validity,
- 21:48the quality or state of being valid,
- 21:50such as the quality of being well grounded,
- 21:53sound or correct in regards
- 21:56to a new method of testing?
- 21:59So how do we validate consumer
- 22:01sleep technologies?
- 22:02So there was a group that was
- 22:06formed to discuss just that.
- 22:08An last year there was a manuscript
- 22:11that described the outcomes of
- 22:13that discussion and made some
- 22:16recommendations on how to quote unquote,
- 22:18validate,
- 22:19consumer sleep technologies which might
- 22:21be better described as determining
- 22:23performance in a specific situation,
- 22:26but will use the term validation for now.
- 22:30So PSG was cited as the gold standard
- 22:33to which the CST must be compared,
- 22:36so not actigraphy as the sole competitor
- 22:39must be. Sleep is defined by PSG.
- 22:42The scoring of PSG should be manual,
- 22:45not automated,
- 22:45right 'cause it's not just PSG signal,
- 22:48that's ground truth, it's scored PSG.
- 22:51That is our ground truth or gold standard,
- 22:54ideally double,
- 22:55scored portable PSG is acceptable
- 22:57as compared to full.
- 22:59These protocols also cited that the CS
- 23:02team needs to be recorded simultaneously.
- 23:05Obviously with the PSG so that you're
- 23:08comparing the same time interval
- 23:10on PSG to the same time in revolt
- 23:13that was scored by the Consumer
- 23:16sleep technology algorithm.
- 23:17Additionally, it was recommended that
- 23:20Epic by epic data from the Consumer
- 23:23Sleep technology outputs be sought
- 23:25out for the epic by epic analysis,
- 23:28and I just want to go into a little bit
- 23:31more detail about why that's important.
- 23:34So whenever you're reading a consumer
- 23:37sleep technology validation paper,
- 23:39you're aware of this,
- 23:40but what an epic by epic analysis
- 23:43is is that you're comparing the
- 23:46same 32nd increment of stage sleep.
- 23:48By the Consumer sleep technology
- 23:51algorithm to that exact 30
- 23:53seconds in time of polysomnogram
- 23:55that was scored by a technician.
- 23:59So you're comparing simultaneous data
- 24:01in the same increment of time and that
- 24:05supposed to comparing summary data
- 24:08over the course of the night to each.
- 24:11So how do you analyze these
- 24:14epic by epic comparisons?
- 24:16This takes us back to college
- 24:19or medical school.
- 24:204 by 4 tables where the typical use
- 24:24was assessing a new test against a
- 24:27gold standard for disease of interest.
- 24:30But in the context of validating
- 24:32consumer sleep technologies against PSG,
- 24:34sleep is the outcome of interest and
- 24:37the population is actually not people.
- 24:40It's the 32nd FX at hand.
- 24:43So how does that look?
- 24:45So let's look at our 4 by 4 table here.
- 24:49So we have horizontally or gold
- 24:52standard which is scored PSG.
- 24:54And then vertically we have the new test
- 24:57which is the output of the CST algorithm.
- 25:00So if you have a CST epic that
- 25:03is scored as sleep and that
- 25:05same epic is actually PSG sleep,
- 25:07that's going to be a true positive.
- 25:10If you have CST output for a given
- 25:1330 seconds that scored his wake,
- 25:15but it's actually sleep,
- 25:17that would be considered a false negative.
- 25:19If you have a PSG wake,
- 25:22epic that the CST interprets is sleep.
- 25:24That's going to be a false positive,
- 25:27and if you have a CST epic or CST
- 25:30scored Wake Epic, that's actually wake.
- 25:33That's going to be considered
- 25:34a true negative.
- 25:36And again, these are true positive,
- 25:38false positive,
- 25:39false negative,
- 25:40true negative,
- 25:40because the outcome of interest is sleep,
- 25:43so the negative state is not sleep or wake,
- 25:47and so then you can derive sensitivity and
- 25:50specificity from this epic by epic analysis.
- 25:52So sensitivity here is the fraction
- 25:55of PSG sleep.
- 25:56Epics that are correctly designated
- 25:58as sleep by the CST while specificity
- 26:01is the fraction of PSG wake ethics
- 26:04that are correctly designated as wake
- 26:07by the Consumer sleep technology.
- 26:09Oftentimes will also get a report
- 26:11on accuracy and what accuracy
- 26:13is in this setting.
- 26:15Is is the number of epochs correctly
- 26:18identified by the CST as a correct
- 26:20state divided by all the epics
- 26:22being considered and accuracy
- 26:24can be very deceiving and healthy
- 26:27sleepers because these studies
- 26:28typically take place at night and
- 26:31what a healthy sleepers do at night,
- 26:33they sleep and they have
- 26:35high sleep efficiency.
- 26:36So say you have an algorithm
- 26:38and the developers.
- 26:40They just want to have a startup company,
- 26:42so the algorithm goes through and scores.
- 26:45Everything is sleep if you test it algorithm
- 26:47in the lab and you determine accuracy in
- 26:50somebody with a sleep efficiency of 90%,
- 26:53the accuracy is going to be 90% despite
- 26:56the fact that the algorithm was arbitrary.
- 26:58So that can be a very deceiving
- 27:01metric in these studies.
- 27:03So how does performance of consumer
- 27:06sleep technologies look overall?
- 27:07So in the left panel you'll see
- 27:10sensitivity and in the right
- 27:12panel you'll see specificity.
- 27:14And the black bars here are
- 27:17consumer sleep technology,
- 27:18wearable devices and then the
- 27:21Gray bars are standard actigraphy,
- 27:23so CST's tend to have sensitivity
- 27:26greater than 90%.
- 27:28Pretty similar to Actigraphy,
- 27:30and specificity's that are more
- 27:33widely ranging about 30 to 60%.
- 27:35So again, similar to standard Actigraphy,
- 27:38sometimes outperforming and
- 27:40sometimes underperforming.
- 27:41So, just like actigraphy,
- 27:43these consumer sleep technologies
- 27:46overall tend to not miss any sleep,
- 27:48but they may misclassify awake
- 27:50during the sleep period and sleep,
- 27:53which results overall,
- 27:55and an overestimation of sleep and
- 27:58decreased accuracy when people have more
- 28:01wake during the time in bed period.
- 28:04So if we have validation, are we good to go?
- 28:08I mean based on this American Academy
- 28:11of Sleep Medicine position statement,
- 28:13this is really the main barrier, Sir.
- 28:16I'm just checking my time here.
- 28:18So once a consumer sleep technology
- 28:20is validated,
- 28:21can we start implementing it in
- 28:24clinical practice and research?
- 28:26Well,
- 28:26there unfortunately a lot of threats
- 28:30to extrapolation of these in lab
- 28:34validation studies to the real
- 28:37life environment. Quite a few.
- 28:39So for example we have problems
- 28:42with our ground truth,
- 28:43so these algorithms are trained
- 28:46to predict sleep as defined by
- 28:48scored PSG and is squared PSG.
- 28:50Really a gold standard.
- 28:52It is our best gold standard now,
- 28:54but there are a lot of problems with it.
- 28:58How many people do we need to score the PSG?
- 29:02What is an experienced technologist?
- 29:04We know that even good RP SG teams
- 29:07have imperfect interscore reliability.
- 29:09So again,
- 29:09making her ground truth not
- 29:11necessarily perfect.
- 29:12Automated scoring is something that
- 29:14could be beneficial in the future
- 29:16to make this more more systematic.
- 29:18And you know we have differences
- 29:20in laboratory,
- 29:21we might have better quality of our data,
- 29:24but out of center is going to
- 29:26allow us to have subject sleep in
- 29:28the home environment,
- 29:30which may be more reflective
- 29:32of sleep overtime.
- 29:34Another problem with the current
- 29:36validation processes we have it is we
- 29:39don't really consider the fact that
- 29:41these consumer sleep technologies
- 29:43are probably measuring an entirely
- 29:45different construct of sleep.
- 29:47So in addition to the fact that with
- 29:50the CST you have an algorithm that's
- 29:53determining the state as opposed to a person,
- 29:57we're also measuring entirely different
- 29:59physiological metrics with the
- 30:01consumer sleep technology typically
- 30:03being based in motion and heart rate.
- 30:06And a PSG being based in EG.
- 30:09And we accept these different contract
- 30:11the constructs for sleep logs versus PSG,
- 30:14but we don't practice graffiti and
- 30:18consumer sleep technologies which
- 30:20we hold for this to the same
- 30:23standard as polysomnogram.
- 30:24Another problem with taking those
- 30:26validation studies done in the lab
- 30:29and extrapolating them to use in
- 30:31the wild is that I'm in bed issue,
- 30:34which is another under
- 30:36recognized threat to validity.
- 30:37And when these studies are done in
- 30:40the sleep lab that I'm in bed is set
- 30:44manually by lights out to lights on.
- 30:47However, that there not is not how
- 30:49these devices work when used in reality.
- 30:52So previously with some of the older
- 30:56consumer sleep technologies you did
- 30:58had to put the device into sleep mode.
- 31:00So then the sleep detection
- 31:03classifiers would be diploid.
- 31:05However, now this is all passive,
- 31:07so even if you have a consumer
- 31:10sleep technology algorithm that
- 31:12performs awesome in the sleep lab,
- 31:14you have high sensitivity without
- 31:17sacrificing too much specificity.
- 31:18If that time in bed detection
- 31:21algorithm isn't good,
- 31:22those sleep wait classifiers are not
- 31:25going to be deployed at the correct time,
- 31:28and that's going to result in sleep
- 31:31summary metrics that are off.
- 31:33So real world utility.
- 31:35Of the consumer,
- 31:36sleep technology is not only based
- 31:39on the performance of the sleep
- 31:41classifier into sleep lab against PSG,
- 31:44but also if those algorithms are
- 31:46deployed in the correct window,
- 31:48which is something that's really study.
- 31:52What about environment?
- 31:53So we have our external environment
- 31:56and things that are not going
- 31:58to be problems in the lab.
- 32:00For example, bed partners, pets.
- 32:02Those aren't going to be in the labs when
- 32:05the sleep period is placed, so most CST.
- 32:08PSG comparison studies are going
- 32:10to be during the night time,
- 32:12so not during naps dot during daytime.
- 32:15Sleep in shift workers,
- 32:16not during this time.
- 32:18Sleep and circadian rhythm,
- 32:19sleep wake disorders,
- 32:21and these are situations.
- 32:22Where we can really benefit from lanja tude.
- 32:25No sleep tracking.
- 32:26Additionally,
- 32:27things like substance use alcohol,
- 32:28most people aren't going to use alcohol
- 32:31or drugs if they are enrolled in his
- 32:34study of their scent tracking device
- 32:36against PSG is often times these are
- 32:39exclusion criteria's for the study,
- 32:41so this might change the performance
- 32:44when we look at these devices.
- 32:46In the real world and the
- 32:48other thing is reliability,
- 32:49so we've already said that
- 32:51the beauty of these devices,
- 32:52their whole need,
- 32:53is the fact that they can be used night
- 32:56after night after night overtime.
- 32:58And we don't know if the performance
- 33:00is reliable overtime given the
- 33:02validation studies typically
- 33:03only take place over one night,
- 33:05and there's a possibility that
- 33:06overtime the sensors aren't as good.
- 33:08That's something we don't know as well.
- 33:12We also can't verify what we don't
- 33:14understand and many consumer sleep
- 33:16trackers come out with these
- 33:18numbers or scores that they don't
- 33:21disclose the definition of.
- 33:23And I know our patients really like
- 33:25these an you know this manufacture
- 33:28on the right probably should get
- 33:30the Nobel Peace Prize because they
- 33:33somehow know what an individual's
- 33:35exact sleep need is.
- 33:37Here cited at 6 hours and
- 33:3951 minutes so we don't know.
- 33:42The algorithms behind these metrics
- 33:44there's really no way we can compare them
- 33:48to anything that is based in science.
- 33:51Population is also a big problem here,
- 33:54so once a consumer sleep technology
- 33:56is validated in a certain age group,
- 33:59that doesn't mean that that
- 34:02performance will translate to
- 34:03different age groups or individuals
- 34:05with other comorbid sleep disorders,
- 34:08particularly as we've seen
- 34:10in hypersomnia and insomnia.
- 34:12We also don't know if heart conditions
- 34:14or certain medications could potentially
- 34:16be problematic given that these
- 34:19algorithms are highly dependent on heart
- 34:21rate and derived metrics from the heart rate,
- 34:24like heart rate variability.
- 34:26Additionally, user also algorithms
- 34:27that are based in motion,
- 34:29so how about individuals with spinal
- 34:32cord injuries strokes patients
- 34:34in the ICU who sleep?
- 34:35We'd like to track.
- 34:37We probably can't extrapolate the performance
- 34:40data that we see to these individuals.
- 34:43Now, problems with accuracy can even
- 34:45or validity can even extend past.
- 34:48Just looking at in accuracies
- 34:50in the device output.
- 34:52So this is a patient of mine who
- 34:54wore an Apple Watch the same night
- 34:57of his home sleep apnea test.
- 35:00I'm sorry,
- 35:01trying to get everything on one page so
- 35:04you can't see these oximetry values here,
- 35:07but with these high pop Mia's
- 35:10he's desatting as low as 88% and.
- 35:13None of these were captured
- 35:15by his Apple Watch oximeter,
- 35:17and this is an individual with
- 35:19dark skin and we now know that
- 35:23these pulse oximetry in general,
- 35:25and these pulse oximeters,
- 35:27particularly on CST,
- 35:28which were unsure of the general accuracy of,
- 35:31could be discrepant an that could augment.
- 35:35Problems that we see are ready
- 35:37with P in individuals with poor
- 35:39social determinants of health.
- 35:41If were collecting massive amounts
- 35:43of data from these devices and
- 35:45using this as a basis for research,
- 35:48we might augment some major
- 35:50health disparities.
- 35:52Now another problem,
- 35:53when we compare a consumer sleep
- 35:55technology to polysomnogram and
- 35:56there's some performance metrics cited,
- 35:58it takes time to do the study.
- 36:01Then the studies written up and
- 36:03it goes through a peer review
- 36:05process for publication,
- 36:07and then someone will say,
- 36:09Oh yes,
- 36:10that device has been validated against PSG.
- 36:13Um,
- 36:13that performance metrics that are
- 36:16output from that study are going
- 36:18to be specific to the sensors
- 36:21of the device that was used,
- 36:23plus the algorithm of the current
- 36:26firmware and software iteration,
- 36:27and these are things that change,
- 36:30and they change with actigraphy as well,
- 36:33so that's not something you,
- 36:35but oftentimes with consumer
- 36:37sleep technologies,
- 36:38the hardware of the device is updated.
- 36:41There's new models that come out or.
- 36:44They could spend all this time on
- 36:46a validation study and the company
- 36:48goes out of business and so that's
- 36:50another threat to generalizing
- 36:52the results of these studies.
- 36:54And what we really need to kind of
- 36:58move passed through these oneof.
- 37:01You know small convenience samples
- 37:03and lack of transparent data in
- 37:06regards the performance of these
- 37:09consumer sleep technologies is
- 37:11really large datasets with some
- 37:14disclosure of the methodology behind how
- 37:17this sleep classifiers are developed.
- 37:19An also information about the population.
- 37:23And we are getting closer to that.
- 37:26So the Somnia project is a project
- 37:29in the Netherlands where in their
- 37:31individuals who are receiving clinical
- 37:34polysomnogram there recording a variety
- 37:36of other sensors and these are sensors
- 37:39that could be incorporated into consumer
- 37:41sleep technologies or that already are,
- 37:44for example, respond PPG,
- 37:46an actigraphy under the bed sensor,
- 37:48microphones, and what this is
- 37:50going to establish is a data set
- 37:53that includes Co recorded.
- 37:55Polysomnogram with all of this different
- 37:58sensor signal so that the best
- 38:01predictors of sleep can be determined
- 38:03an also that these can be tested.
- 38:06These algorithms can be tested
- 38:09on different population subsets.
- 38:12So what does our future hold in regards
- 38:15to consumer sleep technologies?
- 38:17So although it's tempting to
- 38:19just Subs in for actigraphy,
- 38:21I believe these could be much more valuable.
- 38:25I think that they could change the
- 38:27logistics of Sleep Medicine and
- 38:29completely transform our practice.
- 38:31So instead of seeing a patient and
- 38:34taking this leap history that's
- 38:36primarily retroactive,
- 38:37and then maybe sending them home
- 38:40with an actigraph or sleep Diaries,
- 38:42we can have a long term recordings
- 38:45that were performed prior to their
- 38:48first appointment.
- 38:48Because they own the consumer sleep
- 38:50technology and have those available
- 38:52at our initial evaluation.
- 38:54So what I've heard this compared to
- 38:56that you would never hear a woman come
- 38:59to her first obstetric appointment
- 39:01without an over the counter pregnancy test.
- 39:04So this could be something equivalent
- 39:06to that where even if it isn't
- 39:08an FDA cleared device,
- 39:10we have some idea of the sleep wake
- 39:12patterns from over the counter source.
- 39:15Also,
- 39:15this can be used in between visits
- 39:18and given our move to Tele medicine.
- 39:20The fact that the patient's own these
- 39:23devices really makes us a completely no
- 39:26contact way to practice Sleep Medicine.
- 39:29Consumer sleep technologies would
- 39:30allow us to personalize interventions
- 39:32to patients actual sleep patterns
- 39:34as opposed to what we think they are
- 39:36or what they try and remember them
- 39:38to be at the clinic visit and those
- 39:41interventions could be made in real time.
- 39:44The API with the apps associated with
- 39:46CST might also make it easier for us
- 39:49to leave link sleep metrics to other things.
- 39:53So,
- 39:53for example,
- 39:54mobile apps really lend
- 39:55themselves to something called
- 39:56ecological momentary assessment,
- 39:58and that's when you evaluate self
- 40:00report symptoms in real time in
- 40:03response to a push notification,
- 40:04which might give us a better understanding
- 40:07of how sleep interacts with other symptoms.
- 40:10Additionally,
- 40:10we can get health and non health information.
- 40:13Things like Geo location, social media use.
- 40:16Alongside sleep through other apps,
- 40:18potentially use of smart devices
- 40:21like glucometers scales,
- 40:22blood pressure,
- 40:23cuffs with our consumer sleep
- 40:25technology to help see how sleep
- 40:28affects those objective parameters,
- 40:30and then obviously ideally these
- 40:32would interface with our electronic
- 40:35health records so the sleep provider
- 40:37could see this data
- 40:39in a seamless fashion and hopeful
- 40:42that one day sleep with what we've
- 40:45learned longitudinally could even.
- 40:46Act as a vital sign and we'd be able
- 40:49to see changes in the signature of
- 40:51consumer sleep technologies before
- 40:53an acute health event occurs,
- 40:55so we can see an impending deterioration
- 40:57in our patients and potentially intervene
- 40:59before they end up in the hospital.
- 41:01And that's the case.
- 41:03I hope with Pap data as well.
- 41:06So this is 1 example of a consumer sleep
- 41:09technology that has been FDA cleared
- 41:11up for personalized treatment and you
- 41:14guys have probably heard of this.
- 41:16So this is the night where app and this
- 41:19is an app for individuals with nightmares,
- 41:22primarily individuals with PTSD
- 41:23and what this entails is you charge
- 41:26your Apple Watch during the day
- 41:28and then you wear it at night.
- 41:31This device doesn't.
- 41:32Outright claim to track sleep.
- 41:34It says it kind of learns the
- 41:37the cardiac and motion patterns
- 41:39during sleep over the course of
- 41:42days and then based on that,
- 41:44it identifies departures from that
- 41:46that the app thinks that the algorithm
- 41:49thinks are nightmares and there's
- 41:51a stimuli of vibration or that goes
- 41:54through the watch to arouse the patient
- 41:57just slightly from the nightmare
- 41:59interfere with the nightmare without
- 42:01completely waking them for sleep.
- 42:03And there's trials going on
- 42:05right now with this device.
- 42:07Now we talked about the need for this
- 42:11truly longitudinal recording of sleep
- 42:14to really understand sleeping disease
- 42:16an when you look at the data that we
- 42:20currently have, it's pretty short,
- 42:22so active graphic recordings in studies
- 42:25that associate sleep timing duration,
- 42:27intraindividual variability with
- 42:29health outcomes have been 5 seven days.
- 42:32And if we're looking at contribution
- 42:34of sleep to chronic disease,
- 42:37we're going to learn much more.
- 42:40If a consumer sleep technology
- 42:42that's worn for weeks, months,
- 42:45seasons, years can be used.
- 42:48But the other thing is that these devices
- 42:51they are inexpensive and they're pretty
- 42:54widespread in regards to their use,
- 42:57and that gives us the ability to perform
- 43:00sleep research with objective data at scale.
- 43:03Colleagues at U of M used an app to globally
- 43:06quantify sleep in circadian rhythms.
- 43:09So across the world.
- 43:11Additionally,
- 43:11there's studies that have been
- 43:13done in interns who have a really
- 43:16unique sleep challenge,
- 43:17and because of the taxing nature
- 43:20of the medical training program,
- 43:22it would be difficult to assess
- 43:24their sleep otherwise.
- 43:25And there was really unique study
- 43:28by Eric to pollen colleagues where
- 43:30Fitbit data was looked at in a state.
- 43:34Aggregate level Anne used to
- 43:36determine the likelihood of influenza
- 43:38like illness outbreaks,
- 43:39so these are things that couldn't
- 43:42really be easily conducted with use
- 43:45of actigraphy and are very promising.
- 43:48And of course, what about Covid?
- 43:50So Covid has been now used in a few studies.
- 43:54These are the three big ones.
- 43:57Excuse me,
- 43:57CST's have been used in Covid studies
- 44:00are the three big ones to determine
- 44:03if consumer wearable technologies
- 44:05can help predict covid both with
- 44:07and without the incorporation
- 44:09of symptoms, and these have been promising
- 44:12these primarily used resting heart rate
- 44:14and heart rate variability and activity,
- 44:16and some of them you sleep.
- 44:19To predict covid, they've also been found
- 44:22to predict other viral illnesses and has
- 44:24shown that the data from the sensors is
- 44:27beneficial above and beyond the symptom data.
- 44:30And here is a nice picture from one
- 44:33of these investigations and you can
- 44:35see that symptom onsets day zero here,
- 44:38and your blue tracing is respiratory rate.
- 44:41Your red tracing is heart rate,
- 44:43and your green and pink are
- 44:46heart rate variability measures.
- 44:47This is all believe this one is.
- 44:50Yep, this ones from Fitbit and if you
- 44:53look at Day Zero which is symptoms start,
- 44:57you're already seeing a change from
- 44:59baseline and these consumer sensor
- 45:01parameters and so this could be
- 45:04very helpful in identifying people
- 45:05who are going to have a viral
- 45:08illness specifically coded.
- 45:09And if all else fails,
- 45:11you can put your wearable
- 45:13tracker on your pet.
- 45:14So I thank you for your attention
- 45:16today and I'm happy to take any
- 45:19questions and thank you for having me.
- 45:23Thank you so much for a fantastic
- 45:26overview of that topic.
- 45:28Doctor Goldstein, that was really
- 45:30just a really great in depth
- 45:32perspective on all of these devices,
- 45:35how they work algorithms,
- 45:36the data, everything. Thank you.
- 45:38So I'll open it up for questions.
- 45:41And while people are getting ready for that,
- 45:44I'll just start with one.
- 45:46I love your optimism vote kind of
- 45:48embracing the practical advantages
- 45:50of consumer sleep technology
- 45:52over traditional actigraphy,
- 45:53and obviously in the setting of research.
- 45:56I think you've shown great
- 45:58examples of how this can be used,
- 46:01especially for collecting large
- 46:02amounts of data that are just not
- 46:05possible with with actigraphy.
- 46:07What I'm struggling with.
- 46:08Is what the mechanism is that we
- 46:10as asleep community could use
- 46:12to actually encourage companies
- 46:14to be transparent about their
- 46:17algorithms to facilitate validation.
- 46:19And I am not sure how I could see
- 46:21that there ever will really be a great
- 46:25financial incentive for validation
- 46:27to hold companies accountable.
- 46:29I mean, by what means?
- 46:31Does Apple I mean have to to
- 46:34actually broadcast that these
- 46:36things are accurate to consumers?
- 46:38Pay attention to this,
- 46:40how can?
- 46:41How can we put pressure on them or what
- 46:43is their their ill and incentive to do
- 46:46this? Yeah, and I you know
- 46:47I I don't know the answer.
- 46:49There's obviously a lot of suggestibility,
- 46:51so once the patient starts wearing the
- 46:53device but we often see is that they take
- 46:56the information from that's reported
- 46:57obviously by the app more than they
- 46:59take their own perception of their sleep.
- 47:01So there becomes, you know,
- 47:03I would think that most patients would
- 47:05want these to be as accurate as possible,
- 47:07but there also interacting with the
- 47:09device in that way that they are
- 47:11taking into account what it says.
- 47:13So they might take that above their own.
- 47:16Own subjective impression of their sleep,
- 47:18and so it's hard to know.
- 47:20I don't know if it will be on the
- 47:22order of the customer because of that.
- 47:25That pushes the companies to more validation.
- 47:27I don't think that we can do.
- 47:30We can currently continue on with
- 47:32the validation framework as it is if
- 47:34we want to engage with corporations,
- 47:36it just doesn't.
- 47:37It doesn't work.
- 47:38It's too slow to wait till these
- 47:41are companies come out in a peer
- 47:43reviewed publication.
- 47:44I don't know if they'll have,
- 47:46you know, I know some of them.
- 47:48You know there might be occupational
- 47:50programs and insurance company programs
- 47:52and fitness for duty programs that
- 47:54might be more attractive than healthcare.
- 47:57You know,
- 47:57we already don't get reimbursed
- 47:59for FDA cleared Actigraphy,
- 48:01so I don't think that you know
- 48:03from the hospital from the Medical
- 48:05Center standpoint we're really
- 48:07going to move the needle on that.
- 48:10I would hope that they are
- 48:12interested in what we have to say,
- 48:14and they seem like they are.
- 48:17I mean, I've had.
- 48:18Positive interactions with industry,
- 48:20you know, and we don't need to have.
- 48:22You know when you're using
- 48:24machine learning in general,
- 48:25it's you're not going to know
- 48:27everything about the algorithm, right?
- 48:29But we do need some trends,
- 48:31so that can you know,
- 48:32maintain trade secret,
- 48:33but we do need some information
- 48:35about the population.
- 48:36And just like we were going to
- 48:38recommend with AI based sleep stage,
- 48:40and we need to understand what
- 48:42population it was trained on
- 48:44hasn't been independently tested.
- 48:45Things like that,
- 48:46or things that we would like to know.
- 48:49From the companies.
- 48:50Before we use these and you know,
- 48:53rely on these, but you know,
- 48:55I don't really know what's going
- 48:57to push the needle. That's that.
- 48:59I don't have any answer to it's
- 49:02a great question.
- 49:03Thanks.
- 49:05Anyone else who wants to ask a
- 49:07question to Doctor Goldstein?
- 49:10Hi, it's Brian. I have a question.
- 49:16So yeah, I agree with Lauren
- 49:19that's so great talk an I myself,
- 49:21I'm sort of wondering about using various
- 49:24different technologies in my own research,
- 49:26so one one comment and then one question just
- 49:29based on what you guys were just saying.
- 49:32I wonder if. I mean if they if the this
- 49:35you know ASM sleep research society,
- 49:38could we have any leverage with these
- 49:41companies to say you know the more
- 49:43transparent you are about your technology,
- 49:45it how it works,
- 49:47the more likely researchers already
- 49:49use it in their own research.
- 49:51I mean what I've thought about is,
- 49:53I've looked at these different technologies
- 49:56is am I going to have access to the raw data?
- 49:59I think that's really important
- 50:01from a reach research perspective,
- 50:03and so perhaps some companies you know.
- 50:06They want uptake of these devices.
- 50:08Might be receptive to that kind of argument.
- 50:11Just a thought.
- 50:12My question is Doctor Goldstein,
- 50:14have you whether you've looked at some
- 50:17of these consumer technologies that
- 50:19actually look at EEG as opposed to you,
- 50:23know actigraphy,
- 50:23and if you have any thoughts or
- 50:26comments about those technologies.
- 50:28So I haven't in detail, I'll say it.
- 50:31Probably many people miss in this room
- 50:34zoom room whatever we call it now.
- 50:37Have tried them.
- 50:38I mean,
- 50:39I haven't been impressed physically
- 50:40trying those myself and we don't have.
- 50:43There's very limited,
- 50:44limited data on those as well
- 50:47an I haven't seen much.
- 50:48I haven't personally as far as
- 50:50like the dry e.g head mansized and
- 50:53that's where you're talking about.
- 50:55I haven't seen them interact with
- 50:58academia as much as the heart rate and.
- 51:01Motion based wearable devices.
- 51:03You would think that they would be better,
- 51:07but I'm not sure.
- 51:08There's definitely a disconnect and
- 51:09they want to make it so patient
- 51:11can just put it on,
- 51:12but that it also records the EG
- 51:14and I just don't know if the dry
- 51:16e.g technology is there yet.
- 51:18You should try some of these
- 51:19out and I think you'll agree.
- 51:21I looked like I was.
- 51:22I was in some kind of like I was
- 51:25awake and I was like in Delta.
- 51:27I mean they're just they don't.
- 51:28I mean 'cause you when you think
- 51:30about the work that guys end
- 51:32up putting on e.g leads,
- 51:33it just doesn't translate.
- 51:35I mean you can have like.
- 51:36Single lady e.g.
- 51:37But these are embedded in the headband,
- 51:39so I don't know.
- 51:41I just don't think we're
- 51:42quite there yet with that.
- 51:49I'm just going to scan the chat for any
- 51:52question I had actually question then
- 51:54this is that you know where how you
- 51:56doing Kathy good to see you again.
- 51:58At Norwalk hospital.
- 51:59So yeah, we talk and it's a very,
- 52:02you know, sort of hot topic issue.
- 52:04You know when I think about this
- 52:06I I think about how in terms of
- 52:08population health is probably where
- 52:10it may have its biggest effect and
- 52:12I think of sleep deprivation, which
- 52:14is really an epidemic that
- 52:16we don't talk much about in
- 52:17Sleep Medicine as much as we probably should.
- 52:20And so when people track their sleep,
- 52:22you know one thing I find similar
- 52:24to like the step trackers you know
- 52:26they'll look at and they'll say,
- 52:28oh, I've done, you know?
- 52:305000 steps OK, Great
- 52:31and you don't do anything
- 52:33with the information,
- 52:34so I guess the question is can we
- 52:36can we use these sleep trackers?
- 52:38ANO Validation is important,
- 52:39but maybe sort of the more
- 52:41important issue is can the
- 52:43individual person use it to
- 52:44sort of improve their health,
- 52:46get more sleep and
- 52:47you know? And so is there.
- 52:49I think that maybe a sense of
- 52:52kind of focus in on a population
- 52:54in because a lot of people have
- 52:56these these trackers now so you
- 52:58aware of anyone who is using.
- 53:00Consumer technology in an algorithm
- 53:01to try to help people either get
- 53:03more sleep or you know similar
- 53:05to maybe diagnosis or like
- 53:06a screening for sleep apnea.
- 53:07I can't do what they
- 53:09have for the Apple Watch,
- 53:10which is this A-fib detection,
- 53:11which is quite interesting. Where
- 53:13that would sort of clue them in that
- 53:15they may have a sleep problem.
- 53:16So I guess the question is,
- 53:18are you aware of any studies
- 53:20that are looking at using the
- 53:21zoom technologies to increase
- 53:22total sleep time in a population?
- 53:24So this is it.
- 53:25This is a great question and a great point.
- 53:27And as of this time you know
- 53:29both for fitness and sleep.
- 53:31There there's not good evidence to
- 53:33suggest that tracking improves behavior.
- 53:35There is some work that's going on though,
- 53:38so Kelly Baron and colleagues
- 53:39at University of Utah I think.
- 53:41Or maybe she did this back in Chicago,
- 53:44but they used a consumer sleep technology.
- 53:47They developed an app and they did
- 53:49sleep extension in people with short
- 53:51sleep duration and hypertension,
- 53:53and they did improve sleep times
- 53:55and they did improve blood pressure
- 53:57control in those individuals.
- 53:59So an I think that's the key.
- 54:01In how we use these is there
- 54:03never gonna be exactly like e.g
- 54:06defined sleep like they will just.
- 54:08It's your measuring something
- 54:10totally different.
- 54:10Magni Eunice told me once he was like
- 54:12I don't understand he's like these
- 54:15sleep trackers there monitoring things
- 54:17that we don't stay asleep with that
- 54:19they're totally different, right?
- 54:21So but maybe it's not,
- 54:23you know,
- 54:23getting to this perfection level of accuracy.
- 54:26Maybe it's using it as a completely
- 54:28different construct of sleep and seeing
- 54:31how interventions applied to that.
- 54:32Can change your still both your
- 54:34sleep outcome in other health
- 54:36outcomes totally agree.
- 54:40Thank you, ******
- 54:46Hi Kathy, this is Andres
- 54:47and Chuck really nice talk.
- 54:49Hey, thank you hear hear your thoughts
- 54:51and agree with you and Ian about you
- 54:54know these are probably not going to
- 54:57be perfect matches to what we measure
- 54:59in the PSG and I wonder if you know
- 55:02we can use the metrics from these
- 55:04devices as biomarkers and correlate
- 55:06them or establish a relationship with
- 55:08meaningful outcomes and overtime.
- 55:09They might be actually more relevant
- 55:11than e.g signals that we get.
- 55:13I mean sleep is defined by e.g but we're
- 55:16only looking at superficial layer.
- 55:18Of the cells and you know that some of
- 55:20that sleep occurs supportively, right?
- 55:22And so.
- 55:23It's hard to say what we what
- 55:26is the truth here,
- 55:28but I was going to ask you question about the
- 55:32what do we do with all of this information.
- 55:35So for example,
- 55:36if we're tracking patients over days,
- 55:39weeks, months,
- 55:39how do you analyze that
- 55:41data in a meaningful way?
- 55:43And how do you relate it to an outcome?
- 55:46I think it's been a challenge 'cause so far.
- 55:49We've really mostly been used
- 55:51to metrics that are very.
- 55:52Sort of Abacus like where we count
- 55:55things overtime or we you know measure
- 55:58some intensity area under the curve
- 56:00and so the data is being collected.
- 56:03Now at these devices is vastly
- 56:05different and so even with just sleep
- 56:08duration or sleep period written ecity
- 56:11like have you seen any unique ways
- 56:13that that data has been analyzed?
- 56:16And so not from consumer wearables
- 56:18and not for extended amounts of time.
- 56:21And that's where we'll obviously need
- 56:23people with your level of expertise
- 56:25in AI to help distill the data,
- 56:28because there's going to be a
- 56:30lot of Co dependencies,
- 56:31there's going to be just a vast number of,
- 56:34and this is not even.
- 56:36The machine learning algorithms
- 56:37to determine sleep from wake
- 56:39for everyone who's listening.
- 56:40This is the once the post process
- 56:43data is just so much bigger
- 56:45and faster than anything else.
- 56:47So we will need more flexible
- 56:49models to deal with that.
- 56:50I think the closest that we
- 56:52get is the work that Meredith.
- 56:54I think there's no.
- 56:55I don't think pen is here that
- 56:58Meredith Wallace has done,
- 56:59but that was using actigraphy for
- 57:01about 5 days, but they were able.
- 57:03At least you know it's a start
- 57:05for looking at these different
- 57:07parameters of sleep,
- 57:08at least at the same time simultaneously,
- 57:11duration rhythmicity timing,
- 57:12but this is going to be even
- 57:14way more than that,
- 57:15so it'll be interesting to see
- 57:17how that data is dealt with.
- 57:19When we get to that point.
- 57:23Great, thank you so much again,
- 57:26Doctor Goldstein.
- 57:26I think will will close there.
- 57:29I just want to let everybody know
- 57:31that our talk next week we were
- 57:34going to have as we have on the 2nd
- 57:37Wednesday of each month adjoint Yale,
- 57:40Harvard Sleep Conference and we're
- 57:42having a kind of unique talk.
- 57:44James Nestor is an author of the book
- 57:47Breathe the new science of a Lost Art,
- 57:50and so he's going to be giving
- 57:53next week stock so please.
- 57:55Join us for that and again,
- 57:57thank you so much Doctor Goldstein
- 57:59and stay well everybody will see you
- 58:01next week. Sounds good. Thanks,