Skip to Main Content

INFORMATION FOR

"Consumer Sleep Technologies-Potentials, Pitfalls, and the Future of Ambulatory Sleep Tracking" Cathy Goldstein (01.06.2021)

January 11, 2021

"Consumer Sleep Technologies-Potentials, Pitfalls, and the Future of Ambulatory Sleep Tracking" Cathy Goldstein (01.06.2021)

 .
  • 00:20Alright hi everybody.
  • 00:22Welcome, my name is Lauren Tobias.
  • 00:26I'd like you to welcome to welcome
  • 00:28you back to our Yale Sleep Seminar
  • 00:30for the Winter Spring semester.
  • 00:32I have a few quick announcements
  • 00:34before I introduce today's speaker.
  • 00:36So first, please take a moment to
  • 00:39ensure that you're muted in order to
  • 00:41receive CME credit for attendance,
  • 00:43please see the chat room for instructions.
  • 00:46You can text the unique ID for
  • 00:48this conference anytime until 3:15
  • 00:50PM Eastern Time if you're not
  • 00:52already registered with DLC and me.
  • 00:54You do need to do that.
  • 00:561st, If you have any questions
  • 00:58during the presentation,
  • 00:59I encourage you to make use of the
  • 01:01chat room throughout the hour and
  • 01:03will also have an opportunity for you
  • 01:05to unmute yourself and ask questions
  • 01:07directly to Doctor Goldstein at the end.
  • 01:09We do have recorded versions of
  • 01:11these lectures which are going to
  • 01:13be available online at the link
  • 01:15provided in the chat and then finally,
  • 01:17please feel free to share our
  • 01:18announcement for a weekly lecture
  • 01:20series to anyone else who you
  • 01:21think may be interested,
  • 01:23or contact Debbie Lovejoy to
  • 01:24be added to our email list.
  • 01:26So this afternoon I have the pleasure
  • 01:29of introducing Doctor Kathy Goldstein.
  • 01:31Doctor Goldstein received her medical
  • 01:33degree from the Medical College of Georgia.
  • 01:36Didn't ask her about her feelings
  • 01:39about events of this morning yet.
  • 01:41She completed a neurology residency
  • 01:43at the University of Colorado and
  • 01:46her sleep fellowship at Northwestern,
  • 01:48followed by a Masters in clinical research,
  • 01:51design,
  • 01:51and statistical analysis at
  • 01:53the University of Michigan.
  • 01:55She's currently a clinical associate
  • 01:57professor in neurology at the
  • 01:59University of Michigan, Ann Arbor.
  • 02:01Her research interest is in sleep and
  • 02:04circadian interactions with health
  • 02:05and for several years she's been
  • 02:08interested in the role of consumer
  • 02:11sleep technologies and Sleep Medicine,
  • 02:13including the role of wearables and
  • 02:15smartphones, sleep tracking acts, apps.
  • 02:17She's an active clinical educator.
  • 02:19She lectures nationally and and
  • 02:21she is slated to be Co.
  • 02:24Chair of the ASM Sleep Trends
  • 02:26Conference for the upcoming year.
  • 02:28She's authored ASM position statement
  • 02:30on the role of consumer sleep tracking.
  • 02:33And also artificial intelligence
  • 02:34and Sleep Medicine.
  • 02:35And just recently she published an
  • 02:38article that encourage everybody to
  • 02:40check out in JC SM that shares the
  • 02:42results of a large national survey
  • 02:44examining the impact of COVID-19
  • 02:46stay at home orders and sleep,
  • 02:48health and working patterns,
  • 02:50which I found really interesting.
  • 02:52So with that I am delighted to have
  • 02:55Doctor Goldstein here to discuss
  • 02:56a topic that I know many of us are
  • 02:59eager to learn more about consumer
  • 03:02sleep technologies potentials.
  • 03:03Pitfalls in the future of ambulatory
  • 03:05sleep tracking? Thank you.
  • 03:07I will turn myself off Doctor
  • 03:09Goldstein and turn it over to you.
  • 03:12Awesome,
  • 03:12thank you so much Lauren and big virtual.
  • 03:15Hello to everybody.
  • 03:16I'm going to go ahead and get started here.
  • 03:20This is your code to text to get your CME.
  • 03:24And I have no conflicts
  • 03:27of interest to disclose.
  • 03:29OK, so here's our road map
  • 03:32for today's discussion.
  • 03:34Casino what is to come?
  • 03:36And I do want to disclose the fact that
  • 03:39this is not gonna be a review of all
  • 03:43the available consumer technologies.
  • 03:45The entire landscape that
  • 03:47I'm going to present to you,
  • 03:50a general framework with which you can
  • 03:53approach consumer sleep technologies
  • 03:55in general and with the specific use
  • 03:57case of wearable sleep estimation
  • 04:00devices that use motion and heart rate.
  • 04:03So first of all.
  • 04:05Why do we even care bout tracking sleep?
  • 04:09So we have this really big paradox
  • 04:11in Sleep Medicine and research where
  • 04:14we know that sleep deprivation and
  • 04:16sleep disorders cause their detriment
  • 04:18through chronic exposure every
  • 04:20night to disturbances in sleep,
  • 04:23quality, erratic sleep timing or
  • 04:25reduce sleep duration.
  • 04:26That how do we measure sleep with an
  • 04:30overnight polysomnogram is our gold
  • 04:32standard which is great to diagnose.
  • 04:34Sleep apnea, certain parasomnias
  • 04:36and movement Santa titrate.
  • 04:38Happy,
  • 04:38but this is really impractical
  • 04:40beyond one to two nights or abuse.
  • 04:44So how do we track sleep objectively
  • 04:46so that we can get more than one
  • 04:49to two nights of sleep recording?
  • 04:52Well, we do have an option.
  • 04:54I am the currently accepted option for
  • 04:57clinical Sleep Medicine and research
  • 05:00is actigraphy anac trigger fee in the
  • 05:03traditional sense is cleared by the FDA,
  • 05:05and it uses a wrist warm accelerometer
  • 05:08to monitor motion during 24 hour period
  • 05:11with the rationale that we move more than.
  • 05:14When we're awake,
  • 05:15then we do when we're asleep.
  • 05:17So the way this works is you give
  • 05:20an actigraph to your patient.
  • 05:22They wear the device and there's
  • 05:24a piezo electric accelerometer or
  • 05:26Mens accelerometer in the device
  • 05:28which collects motion,
  • 05:29which is then digitized into
  • 05:30activity counts the data from the
  • 05:33devices then downloaded.
  • 05:34And a software package uses an
  • 05:37algorithm to determine sleep from wake,
  • 05:40and we like actigraphy because Actigraphy
  • 05:43performance has been compared to PSG,
  • 05:45and the findings of that
  • 05:48performance published in many,
  • 05:50many research publications, so.
  • 05:52What's the problem?
  • 05:53If we have this small, unobtrusive
  • 05:55way to measure sleep over days to weeks,
  • 05:58well, there's quite a bit.
  • 06:00As you can tell.
  • 06:02So first of all, actigraphy,
  • 06:04typically just measures movement.
  • 06:05Some devices measure light.
  • 06:07The data acquisition methods and storage
  • 06:09methods are really outdated with actigraphy,
  • 06:12so before you give the device to somebody,
  • 06:15you have to initialize it
  • 06:17and to recover the data.
  • 06:19You typically have to plug
  • 06:20it back in via USB ports,
  • 06:23and so we can't really transmit an.
  • 06:26Evaluate the data in real time.
  • 06:28I've even seen researchers male actor
  • 06:31graphs back and forth to each other,
  • 06:33which is pretty prehistoric
  • 06:35in this day and age.
  • 06:37Always talk about Actigraphy is validated,
  • 06:39it's validated it's validated,
  • 06:41and yes,
  • 06:42it's been compared to polysomnogram
  • 06:44many times, but it's not that great,
  • 06:47so it has a high sensitivity
  • 06:49but a low specificity for sleep,
  • 06:52which means that the algorithms
  • 06:53applied to actigraphy data typically
  • 06:55misinterpret nonmoving wakefulness,
  • 06:57as sleep, and So what that means is
  • 06:59these have very high accuracy and
  • 07:02healthy sleepers during nighttime sleep.
  • 07:05However, as sleep disruption
  • 07:06increases throughout the night.
  • 07:08There's more wake after sleep onset,
  • 07:10increased sleep onset latency,
  • 07:12the accuracy of the sleep metrics that are
  • 07:15output by the software are going to go down.
  • 07:19And Speaking of software that is
  • 07:21another pitfall of actigraphy.
  • 07:22The traditional actigraph software does
  • 07:24not interface with the electronic health
  • 07:27record or any other digital health platforms,
  • 07:30so that makes it difficult to kind
  • 07:32of track sleek sleep along with
  • 07:35other health metrics in an easy,
  • 07:37meaningful, practical way.
  • 07:38The use of actigraphy requires
  • 07:40quite a bit of data cleaning.
  • 07:42Given that I'm in bed intervals need
  • 07:44to be set for appropriate analysis,
  • 07:47and this interpretation,
  • 07:48as well as the set up the patient
  • 07:50recording even though we have a CPT code.
  • 07:53This is typically not reimbursed,
  • 07:55so actigraphy is not going to be practical
  • 07:58for a clinical workflow in most cases,
  • 08:00and it's going to be burdensome
  • 08:03on the research team.
  • 08:05Additionally,
  • 08:05actor graphs are expensive.
  • 08:07There are at least $800 a device
  • 08:09for the FDA cleared devices,
  • 08:11so this is not really practical
  • 08:13for large population studies,
  • 08:15and it restricts use clinically,
  • 08:17particularly if there is a sleep
  • 08:19clinic in an underserved area.
  • 08:21One big thing that people don't talk
  • 08:23about is it these devices are owned by
  • 08:26the health system where the research
  • 08:28team they're not owned by the patient,
  • 08:31so even though we consider actigraphy
  • 08:33to be longitudinal sleep recording.
  • 08:35These typically don't go beyond 1
  • 08:37two weeks as opposed to devices
  • 08:40that patients might own and use
  • 08:42themselves that are worn over weeks,
  • 08:44days, months, even years.
  • 08:47So despite the strong wording
  • 08:49that we see here,
  • 08:51that Actigraphy should be used before
  • 08:54the MSL T in circadian rhythm disorders.
  • 08:58What happens when we stop following
  • 09:01the icst three and start getting real?
  • 09:04The real world Sleep Medicine clinic
  • 09:07and research we ask about sleep
  • 09:10cross sectionally.
  • 09:11So for example,
  • 09:13will say how many hours do you
  • 09:16sleep on average?
  • 09:18At face value,
  • 09:19this seems like a very kind of easy,
  • 09:21simple question,
  • 09:22but obviously this can be
  • 09:24interpreted in a multitude of ways.
  • 09:26For example,
  • 09:26how many hours do you sleep at night?
  • 09:29How many hours do you sleep at night,
  • 09:31plus how many hours do you sleep during naps?
  • 09:34What time do you go to bed and when
  • 09:37do you wake up and do the math?
  • 09:40Is this during the work week during the
  • 09:42weekend so we can see that this can
  • 09:45be misconstrued if we use lanja tude?
  • 09:47No self report?
  • 09:48With the sleep diary.
  • 09:50So daily self report recording of sleep.
  • 09:52That might be a bit better,
  • 09:54but there's problems with that as well.
  • 09:56This is burdensome to our patients
  • 09:58and is oftentimes incomplete,
  • 09:59or the classic example of people filling
  • 10:01these out in the waiting room before
  • 10:03they come in for their PSG MSL T.
  • 10:06This could be difficult for certain
  • 10:08populations specifically like the very
  • 10:09young or the older with cognitive problems.
  • 10:11And again, even if your patients to
  • 10:13fill this out, what did they do?
  • 10:16Fax it to you or FS,
  • 10:17send it as an email attachment.
  • 10:19So not really relevant for the technical
  • 10:23capabilities that we have in 2020.
  • 10:25Now, so why is this important?
  • 10:28Does this really affect
  • 10:29our clinical practice?
  • 10:31Well, we suspect it does,
  • 10:33so the American Academy of Sleep Medicine
  • 10:35did do a systematic review and found
  • 10:38that the objective information from
  • 10:40Actigraph recordings is beneficial.
  • 10:42An provides distinct information over
  • 10:44sleep logs in a variety of disorders,
  • 10:47but the evaluation and management.
  • 10:50Additionally,
  • 10:51the group from Harvard has already
  • 10:53asked researchers to stop querying
  • 10:55self reported sleep duration.
  • 10:57As you can see here,
  • 10:59and that's because objective sleep
  • 11:01recording is also incredibly
  • 11:03relevant for research because the
  • 11:05in accuracies that are possible
  • 11:07with self report sleep duration
  • 11:09can depend on patient demographics,
  • 11:11whether or not they have comorbid
  • 11:13sleep disorders.
  • 11:14When the question is asked an way,
  • 11:17the question is asked, so this can lead.
  • 11:20Tobias in research.
  • 11:22Another problem with using self
  • 11:24report sleep duration and research
  • 11:27is that the whole construct of
  • 11:29sleep duration as an equivalent
  • 11:32for sleep health which we know is
  • 11:35probably not the case,
  • 11:36and there's growing evidence to support this.
  • 11:39For example,
  • 11:40something called intraindividual variability.
  • 11:42So the variation of sleep
  • 11:44parameters around the mean,
  • 11:46which has been shown relevant
  • 11:48for a variety of outcomes,
  • 11:50including psychiatric, metabolic,
  • 11:51and cardiovascular.
  • 11:52This is going to require
  • 11:54longitudinal tracking which.
  • 11:56Is difficult to do with self report and so
  • 11:59a passive objective method is required.
  • 12:03Additionally,
  • 12:03as we said that sleep duration
  • 12:06doesn't equate to sleep health eh 7
  • 12:09domain definition has been proposed,
  • 12:11and that's a combination of both self
  • 12:14report and primarily objective measures,
  • 12:17including in addition to duration,
  • 12:19continuity, timing,
  • 12:20rhythmicity regularity and
  • 12:21subjective sleepiness and quality,
  • 12:23as well as a composite of extreme values.
  • 12:27And when you combine,
  • 12:29combine all seven domains an.
  • 12:31Additionally,
  • 12:31the number of composite extreme values.
  • 12:34We can see that this has a much
  • 12:37bigger impact on mortality then
  • 12:39just sleep duration alone,
  • 12:41which I don't know if you
  • 12:43can see my cursor here,
  • 12:45but here are the seven domains combined
  • 12:48as well as the number of extreme values,
  • 12:51and then we have sleep duration is
  • 12:54all the way over here when used as
  • 12:57an isolated sleep variable to predict
  • 12:59mortality in a random forest model.
  • 13:02So it seems like longitudinal objective
  • 13:05sleep measurements are pretty important.
  • 13:07So if we can't use actigraphy
  • 13:09because of the limitations,
  • 13:10what are options?
  • 13:11It seems like we sure have a lot
  • 13:14with consumer sleep tracking technology.
  • 13:16We have wrist worn devices,
  • 13:18as you can see here from a variety
  • 13:21of manufacturers.
  • 13:22More and more rings are
  • 13:24coming on to the market dry,
  • 13:26e.g headbands mirabal's that are
  • 13:28either at the bedside or under the bed.
  • 13:32So before we can really talk about
  • 13:35the possibilities and the problems,
  • 13:36that is consumer sleep technologies hold.
  • 13:38We have to understand a little bit
  • 13:41more about how they work or how
  • 13:44do we get from signal to sleep.
  • 13:46And I'm not sure I can't see everybody,
  • 13:49so it's hard to know Peoples ages.
  • 13:51But for those of you that
  • 13:53remember Schoolhouse Rock,
  • 13:54the best schoolhouse rock is.
  • 13:56How does a bill become a law?
  • 13:58And here we have Bill,
  • 14:00and he's just sitting here on Capitol Hill.
  • 14:03And just like a bill, is not a law.
  • 14:06Smartwatches,
  • 14:06rings etc aren't direct measures of sleep.
  • 14:09There are a bunch of steps in between,
  • 14:11so this is an oversimplification,
  • 14:13but this does give you some
  • 14:15idea about how these.
  • 14:17Actually work,
  • 14:17so your device is going to
  • 14:19or most risk form devices.
  • 14:22At least they're going to typically
  • 14:24include a Tri axial accelerometer,
  • 14:26which is going to record motion
  • 14:28in three directions, XY and Z,
  • 14:30and that's going to give you raw signal
  • 14:33to the motion we see pictured here,
  • 14:36which is actually from an Apple Watch
  • 14:39and then optical photoplus Mogra Fi,
  • 14:41and that is a visual,
  • 14:43light based way to estimate blood volume,
  • 14:46which is then converted into
  • 14:48pulse Anna variety.
  • 14:49Of computed features from pulse,
  • 14:50including things like heart rate variability
  • 14:53and then your motion and your heart rate.
  • 14:55Features are going to go into an
  • 14:58algorithm which is going to even be there
  • 15:00through an app or a cloud based server,
  • 15:03and that algorithm is typically black box.
  • 15:05As you can see from the figure and then
  • 15:08the sleep information is returned to the
  • 15:10user both visually and in summary metrics.
  • 15:13As you can see in the example below.
  • 15:16So it's a myth that your watch track sleep,
  • 15:20but what the truth is is that algorithms
  • 15:23estimate sleep from wearable sensor,
  • 15:25acquired physiological
  • 15:25signals and arrived features.
  • 15:27But sometimes that's a bit difficult
  • 15:29to discuss with their patients.
  • 15:32Now, these algorithms that estimate sleep.
  • 15:34Where do they come from?
  • 15:36So let's start by taking a look
  • 15:39at our traditional actigraphy,
  • 15:41so the algorithms that are
  • 15:43used by FDA cleared ACTIGRAPHY,
  • 15:45and the associated software typically are
  • 15:47ones that are more than 2025 years old.
  • 15:50So on the top left you can see
  • 15:53the Respironics actor algorithm,
  • 15:55and on the bottom right that's the cool
  • 15:58Kripke algorithm in the Respironics Act.
  • 16:01Aware Algorithm E sub zeros
  • 16:03the epic being scored.
  • 16:05And same with cold Kripke,
  • 16:06we have a sub zero,
  • 16:08so those are the epic of interest.
  • 16:10And then there's a weighted sum and
  • 16:13the negative numbers are the epochs
  • 16:15that are before the epic at hand,
  • 16:17being scored and the positive
  • 16:19numbers or the epics after the
  • 16:21epic being scored and you can see
  • 16:23these received lesser weights and
  • 16:25then if the sum crosses a threshold
  • 16:27it's going to get scored.
  • 16:29That epic is going to get Squared's
  • 16:31wake and if it is below that
  • 16:33threshold it'll get scored as sleep.
  • 16:35So weighted sounds.
  • 16:36Now how is this changed?
  • 16:38So we use much more sophisticated
  • 16:41methods to classify,
  • 16:42sleep and wake from wearable sensor data.
  • 16:45At this point in time an I worked
  • 16:47with a couple mathematicians at
  • 16:49University of Michigan Humane,
  • 16:51Olivia Walsh and Danny Forger and
  • 16:54we wanted to know if we could
  • 16:56actually make our own algorithms
  • 16:58to train models to classify sleep
  • 17:01versus wake from an off the shelf
  • 17:04consumer sleep technology and we
  • 17:06actually use the Apple Watch and
  • 17:08the reason was is that was the.
  • 17:11Only one that we could get our hands
  • 17:13on where we had access to the motion.
  • 17:16An heart rate data for modeling
  • 17:18purposes and this was funded by
  • 17:20our athletics Department because
  • 17:21they are very interested in sleep
  • 17:24and sleep technology to improve
  • 17:26recovery and performance,
  • 17:27and for those of you interested in
  • 17:29college basketball, it was pretty fun.
  • 17:31Because Jordan Poole and Isaiah
  • 17:33Livers did the UI for our app.
  • 17:35So really neat project that we're
  • 17:38grateful we have the funding for.
  • 17:40And what we did is we had people
  • 17:43wear Apple Watch is for a seven
  • 17:45day ambulatory period,
  • 17:46and the reason we use that ambulatory
  • 17:49period is Danny Anna Livia.
  • 17:51They're computational biologists
  • 17:52that are focused on the modeling
  • 17:54of the circadian Clock,
  • 17:55and there is a very well validated
  • 17:58differential equations model that you
  • 18:00can pass activity through to get a
  • 18:03Clock proxy that we used as an input,
  • 18:05and then on the final night
  • 18:07of this seven day recording,
  • 18:09we had our subjects come in and they slept.
  • 18:12Simultaneously with the Apple Watch
  • 18:14and synchronized with Polysomnogram,
  • 18:16and we obtained the rock celebration data.
  • 18:19And at PPG heart rate from their
  • 18:22Apple Watch on that same night,
  • 18:25we use that Clock proxy feature as
  • 18:27well as activity counts derived from
  • 18:30the acceleration and PPG heart rate
  • 18:32and arrived standard deviation of
  • 18:35heart rate to train algorithms against
  • 18:38scored PSG to classify wake versus sleep,
  • 18:41and we use regular old logistic regression.
  • 18:44As well as machine learning
  • 18:46techniques for our classifier,
  • 18:48I'm not going to go into the details of that,
  • 18:53but our final algorithm performed as
  • 18:55well as actigraphy or what's been
  • 18:58historically cited for actigraphy
  • 19:00in regards to sensitivity and
  • 19:02even improved with specificity.
  • 19:04But you guys can read about that
  • 19:07in the paper.
  • 19:08But I do wanna show you is that there are
  • 19:11a few other groups that have done this.
  • 19:14When the access to raw signal from consumer
  • 19:17sleep technologies to been iaccessible,
  • 19:18so Zang used a Microsoft Band,
  • 19:20Roberts,
  • 19:21used an Apple Watch Ann
  • 19:22Beattie at all used to Fitbit.
  • 19:24I believe they worked with Fitbit,
  • 19:26so they can have access to that data and I
  • 19:29won't go through all the details of this.
  • 19:32But what I want you to kind of
  • 19:35recognize is the breath of the
  • 19:37features that are used.
  • 19:38So we went to previous algorithms with.
  • 19:40FDA cleared actigraphy that were just
  • 19:43using weighted sums of activity counts
  • 19:46to a variety of features from motion,
  • 19:48an heart rate, an various computed features.
  • 19:51As you can see here and then,
  • 19:54these are going into machine
  • 19:57learning algorithms to train their
  • 19:59capability to predict sleep and wake
  • 20:01so much as this is changed overtime,
  • 20:04and we're really looking at
  • 20:06different sleep wake classifiers
  • 20:08that were previously accustomed to.
  • 20:10But what everyone wants to know is
  • 20:13what about the algorithms that are
  • 20:15actually associated with the apps
  • 20:17that come with the devices that your
  • 20:20patients or research subjects actually use.
  • 20:22So these these algorithms that other
  • 20:25people have come up with aren't
  • 20:27typically available in a usable way.
  • 20:29Ours is open source which
  • 20:31can be obtained from GitHub,
  • 20:33but it's not enough.
  • 20:35Closed app right now,
  • 20:36so the problem is we don't really know
  • 20:39the the apps that are available now
  • 20:42and be associated with the CST's that
  • 20:45are available on the market market.
  • 20:47We don't really have details
  • 20:49about the features that are used,
  • 20:52the populations that the algorithms
  • 20:53were trained on, or the methodology.
  • 20:56So it's pretty unclear.
  • 20:58Which makes us dependent on the already
  • 21:01processed sleep stage output from these
  • 21:04devices and their associated algorithms.
  • 21:06So the question is,
  • 21:08how good is this data?
  • 21:10Which brings us to the
  • 21:13question on everybody's mind,
  • 21:14which is validation.
  • 21:16And this is the position statement that the
  • 21:19American Academy of Sleep Medicine put forth
  • 21:22in regards to consumer sleep technology.
  • 21:25And you can see that we
  • 21:28talk about validation a lot.
  • 21:30So what is this validation that's so
  • 21:33important for consumer sleep technologies?
  • 21:35So what's validation in general?
  • 21:37So validation is the act,
  • 21:39process,
  • 21:40or instance of validating,
  • 21:41especially the determination of the
  • 21:44degree of the validity of a device.
  • 21:46So what's validity,
  • 21:48the quality or state of being valid,
  • 21:50such as the quality of being well grounded,
  • 21:53sound or correct in regards
  • 21:56to a new method of testing?
  • 21:59So how do we validate consumer
  • 22:01sleep technologies?
  • 22:02So there was a group that was
  • 22:06formed to discuss just that.
  • 22:08An last year there was a manuscript
  • 22:11that described the outcomes of
  • 22:13that discussion and made some
  • 22:16recommendations on how to quote unquote,
  • 22:18validate,
  • 22:19consumer sleep technologies which might
  • 22:21be better described as determining
  • 22:23performance in a specific situation,
  • 22:26but will use the term validation for now.
  • 22:30So PSG was cited as the gold standard
  • 22:33to which the CST must be compared,
  • 22:36so not actigraphy as the sole competitor
  • 22:39must be. Sleep is defined by PSG.
  • 22:42The scoring of PSG should be manual,
  • 22:45not automated,
  • 22:45right 'cause it's not just PSG signal,
  • 22:48that's ground truth, it's scored PSG.
  • 22:51That is our ground truth or gold standard,
  • 22:54ideally double,
  • 22:55scored portable PSG is acceptable
  • 22:57as compared to full.
  • 22:59These protocols also cited that the CS
  • 23:02team needs to be recorded simultaneously.
  • 23:05Obviously with the PSG so that you're
  • 23:08comparing the same time interval
  • 23:10on PSG to the same time in revolt
  • 23:13that was scored by the Consumer
  • 23:16sleep technology algorithm.
  • 23:17Additionally, it was recommended that
  • 23:20Epic by epic data from the Consumer
  • 23:23Sleep technology outputs be sought
  • 23:25out for the epic by epic analysis,
  • 23:28and I just want to go into a little bit
  • 23:31more detail about why that's important.
  • 23:34So whenever you're reading a consumer
  • 23:37sleep technology validation paper,
  • 23:39you're aware of this,
  • 23:40but what an epic by epic analysis
  • 23:43is is that you're comparing the
  • 23:46same 32nd increment of stage sleep.
  • 23:48By the Consumer sleep technology
  • 23:51algorithm to that exact 30
  • 23:53seconds in time of polysomnogram
  • 23:55that was scored by a technician.
  • 23:59So you're comparing simultaneous data
  • 24:01in the same increment of time and that
  • 24:05supposed to comparing summary data
  • 24:08over the course of the night to each.
  • 24:11So how do you analyze these
  • 24:14epic by epic comparisons?
  • 24:16This takes us back to college
  • 24:19or medical school.
  • 24:204 by 4 tables where the typical use
  • 24:24was assessing a new test against a
  • 24:27gold standard for disease of interest.
  • 24:30But in the context of validating
  • 24:32consumer sleep technologies against PSG,
  • 24:34sleep is the outcome of interest and
  • 24:37the population is actually not people.
  • 24:40It's the 32nd FX at hand.
  • 24:43So how does that look?
  • 24:45So let's look at our 4 by 4 table here.
  • 24:49So we have horizontally or gold
  • 24:52standard which is scored PSG.
  • 24:54And then vertically we have the new test
  • 24:57which is the output of the CST algorithm.
  • 25:00So if you have a CST epic that
  • 25:03is scored as sleep and that
  • 25:05same epic is actually PSG sleep,
  • 25:07that's going to be a true positive.
  • 25:10If you have CST output for a given
  • 25:1330 seconds that scored his wake,
  • 25:15but it's actually sleep,
  • 25:17that would be considered a false negative.
  • 25:19If you have a PSG wake,
  • 25:22epic that the CST interprets is sleep.
  • 25:24That's going to be a false positive,
  • 25:27and if you have a CST epic or CST
  • 25:30scored Wake Epic, that's actually wake.
  • 25:33That's going to be considered
  • 25:34a true negative.
  • 25:36And again, these are true positive,
  • 25:38false positive,
  • 25:39false negative,
  • 25:40true negative,
  • 25:40because the outcome of interest is sleep,
  • 25:43so the negative state is not sleep or wake,
  • 25:47and so then you can derive sensitivity and
  • 25:50specificity from this epic by epic analysis.
  • 25:52So sensitivity here is the fraction
  • 25:55of PSG sleep.
  • 25:56Epics that are correctly designated
  • 25:58as sleep by the CST while specificity
  • 26:01is the fraction of PSG wake ethics
  • 26:04that are correctly designated as wake
  • 26:07by the Consumer sleep technology.
  • 26:09Oftentimes will also get a report
  • 26:11on accuracy and what accuracy
  • 26:13is in this setting.
  • 26:15Is is the number of epochs correctly
  • 26:18identified by the CST as a correct
  • 26:20state divided by all the epics
  • 26:22being considered and accuracy
  • 26:24can be very deceiving and healthy
  • 26:27sleepers because these studies
  • 26:28typically take place at night and
  • 26:31what a healthy sleepers do at night,
  • 26:33they sleep and they have
  • 26:35high sleep efficiency.
  • 26:36So say you have an algorithm
  • 26:38and the developers.
  • 26:40They just want to have a startup company,
  • 26:42so the algorithm goes through and scores.
  • 26:45Everything is sleep if you test it algorithm
  • 26:47in the lab and you determine accuracy in
  • 26:50somebody with a sleep efficiency of 90%,
  • 26:53the accuracy is going to be 90% despite
  • 26:56the fact that the algorithm was arbitrary.
  • 26:58So that can be a very deceiving
  • 27:01metric in these studies.
  • 27:03So how does performance of consumer
  • 27:06sleep technologies look overall?
  • 27:07So in the left panel you'll see
  • 27:10sensitivity and in the right
  • 27:12panel you'll see specificity.
  • 27:14And the black bars here are
  • 27:17consumer sleep technology,
  • 27:18wearable devices and then the
  • 27:21Gray bars are standard actigraphy,
  • 27:23so CST's tend to have sensitivity
  • 27:26greater than 90%.
  • 27:28Pretty similar to Actigraphy,
  • 27:30and specificity's that are more
  • 27:33widely ranging about 30 to 60%.
  • 27:35So again, similar to standard Actigraphy,
  • 27:38sometimes outperforming and
  • 27:40sometimes underperforming.
  • 27:41So, just like actigraphy,
  • 27:43these consumer sleep technologies
  • 27:46overall tend to not miss any sleep,
  • 27:48but they may misclassify awake
  • 27:50during the sleep period and sleep,
  • 27:53which results overall,
  • 27:55and an overestimation of sleep and
  • 27:58decreased accuracy when people have more
  • 28:01wake during the time in bed period.
  • 28:04So if we have validation, are we good to go?
  • 28:08I mean based on this American Academy
  • 28:11of Sleep Medicine position statement,
  • 28:13this is really the main barrier, Sir.
  • 28:16I'm just checking my time here.
  • 28:18So once a consumer sleep technology
  • 28:20is validated,
  • 28:21can we start implementing it in
  • 28:24clinical practice and research?
  • 28:26Well,
  • 28:26there unfortunately a lot of threats
  • 28:30to extrapolation of these in lab
  • 28:34validation studies to the real
  • 28:37life environment. Quite a few.
  • 28:39So for example we have problems
  • 28:42with our ground truth,
  • 28:43so these algorithms are trained
  • 28:46to predict sleep as defined by
  • 28:48scored PSG and is squared PSG.
  • 28:50Really a gold standard.
  • 28:52It is our best gold standard now,
  • 28:54but there are a lot of problems with it.
  • 28:58How many people do we need to score the PSG?
  • 29:02What is an experienced technologist?
  • 29:04We know that even good RP SG teams
  • 29:07have imperfect interscore reliability.
  • 29:09So again,
  • 29:09making her ground truth not
  • 29:11necessarily perfect.
  • 29:12Automated scoring is something that
  • 29:14could be beneficial in the future
  • 29:16to make this more more systematic.
  • 29:18And you know we have differences
  • 29:20in laboratory,
  • 29:21we might have better quality of our data,
  • 29:24but out of center is going to
  • 29:26allow us to have subject sleep in
  • 29:28the home environment,
  • 29:30which may be more reflective
  • 29:32of sleep overtime.
  • 29:34Another problem with the current
  • 29:36validation processes we have it is we
  • 29:39don't really consider the fact that
  • 29:41these consumer sleep technologies
  • 29:43are probably measuring an entirely
  • 29:45different construct of sleep.
  • 29:47So in addition to the fact that with
  • 29:50the CST you have an algorithm that's
  • 29:53determining the state as opposed to a person,
  • 29:57we're also measuring entirely different
  • 29:59physiological metrics with the
  • 30:01consumer sleep technology typically
  • 30:03being based in motion and heart rate.
  • 30:06And a PSG being based in EG.
  • 30:09And we accept these different contract
  • 30:11the constructs for sleep logs versus PSG,
  • 30:14but we don't practice graffiti and
  • 30:18consumer sleep technologies which
  • 30:20we hold for this to the same
  • 30:23standard as polysomnogram.
  • 30:24Another problem with taking those
  • 30:26validation studies done in the lab
  • 30:29and extrapolating them to use in
  • 30:31the wild is that I'm in bed issue,
  • 30:34which is another under
  • 30:36recognized threat to validity.
  • 30:37And when these studies are done in
  • 30:40the sleep lab that I'm in bed is set
  • 30:44manually by lights out to lights on.
  • 30:47However, that there not is not how
  • 30:49these devices work when used in reality.
  • 30:52So previously with some of the older
  • 30:56consumer sleep technologies you did
  • 30:58had to put the device into sleep mode.
  • 31:00So then the sleep detection
  • 31:03classifiers would be diploid.
  • 31:05However, now this is all passive,
  • 31:07so even if you have a consumer
  • 31:10sleep technology algorithm that
  • 31:12performs awesome in the sleep lab,
  • 31:14you have high sensitivity without
  • 31:17sacrificing too much specificity.
  • 31:18If that time in bed detection
  • 31:21algorithm isn't good,
  • 31:22those sleep wait classifiers are not
  • 31:25going to be deployed at the correct time,
  • 31:28and that's going to result in sleep
  • 31:31summary metrics that are off.
  • 31:33So real world utility.
  • 31:35Of the consumer,
  • 31:36sleep technology is not only based
  • 31:39on the performance of the sleep
  • 31:41classifier into sleep lab against PSG,
  • 31:44but also if those algorithms are
  • 31:46deployed in the correct window,
  • 31:48which is something that's really study.
  • 31:52What about environment?
  • 31:53So we have our external environment
  • 31:56and things that are not going
  • 31:58to be problems in the lab.
  • 32:00For example, bed partners, pets.
  • 32:02Those aren't going to be in the labs when
  • 32:05the sleep period is placed, so most CST.
  • 32:08PSG comparison studies are going
  • 32:10to be during the night time,
  • 32:12so not during naps dot during daytime.
  • 32:15Sleep in shift workers,
  • 32:16not during this time.
  • 32:18Sleep and circadian rhythm,
  • 32:19sleep wake disorders,
  • 32:21and these are situations.
  • 32:22Where we can really benefit from lanja tude.
  • 32:25No sleep tracking.
  • 32:26Additionally,
  • 32:27things like substance use alcohol,
  • 32:28most people aren't going to use alcohol
  • 32:31or drugs if they are enrolled in his
  • 32:34study of their scent tracking device
  • 32:36against PSG is often times these are
  • 32:39exclusion criteria's for the study,
  • 32:41so this might change the performance
  • 32:44when we look at these devices.
  • 32:46In the real world and the
  • 32:48other thing is reliability,
  • 32:49so we've already said that
  • 32:51the beauty of these devices,
  • 32:52their whole need,
  • 32:53is the fact that they can be used night
  • 32:56after night after night overtime.
  • 32:58And we don't know if the performance
  • 33:00is reliable overtime given the
  • 33:02validation studies typically
  • 33:03only take place over one night,
  • 33:05and there's a possibility that
  • 33:06overtime the sensors aren't as good.
  • 33:08That's something we don't know as well.
  • 33:12We also can't verify what we don't
  • 33:14understand and many consumer sleep
  • 33:16trackers come out with these
  • 33:18numbers or scores that they don't
  • 33:21disclose the definition of.
  • 33:23And I know our patients really like
  • 33:25these an you know this manufacture
  • 33:28on the right probably should get
  • 33:30the Nobel Peace Prize because they
  • 33:33somehow know what an individual's
  • 33:35exact sleep need is.
  • 33:37Here cited at 6 hours and
  • 33:3951 minutes so we don't know.
  • 33:42The algorithms behind these metrics
  • 33:44there's really no way we can compare them
  • 33:48to anything that is based in science.
  • 33:51Population is also a big problem here,
  • 33:54so once a consumer sleep technology
  • 33:56is validated in a certain age group,
  • 33:59that doesn't mean that that
  • 34:02performance will translate to
  • 34:03different age groups or individuals
  • 34:05with other comorbid sleep disorders,
  • 34:08particularly as we've seen
  • 34:10in hypersomnia and insomnia.
  • 34:12We also don't know if heart conditions
  • 34:14or certain medications could potentially
  • 34:16be problematic given that these
  • 34:19algorithms are highly dependent on heart
  • 34:21rate and derived metrics from the heart rate,
  • 34:24like heart rate variability.
  • 34:26Additionally, user also algorithms
  • 34:27that are based in motion,
  • 34:29so how about individuals with spinal
  • 34:32cord injuries strokes patients
  • 34:34in the ICU who sleep?
  • 34:35We'd like to track.
  • 34:37We probably can't extrapolate the performance
  • 34:40data that we see to these individuals.
  • 34:43Now, problems with accuracy can even
  • 34:45or validity can even extend past.
  • 34:48Just looking at in accuracies
  • 34:50in the device output.
  • 34:52So this is a patient of mine who
  • 34:54wore an Apple Watch the same night
  • 34:57of his home sleep apnea test.
  • 35:00I'm sorry,
  • 35:01trying to get everything on one page so
  • 35:04you can't see these oximetry values here,
  • 35:07but with these high pop Mia's
  • 35:10he's desatting as low as 88% and.
  • 35:13None of these were captured
  • 35:15by his Apple Watch oximeter,
  • 35:17and this is an individual with
  • 35:19dark skin and we now know that
  • 35:23these pulse oximetry in general,
  • 35:25and these pulse oximeters,
  • 35:27particularly on CST,
  • 35:28which were unsure of the general accuracy of,
  • 35:31could be discrepant an that could augment.
  • 35:35Problems that we see are ready
  • 35:37with P in individuals with poor
  • 35:39social determinants of health.
  • 35:41If were collecting massive amounts
  • 35:43of data from these devices and
  • 35:45using this as a basis for research,
  • 35:48we might augment some major
  • 35:50health disparities.
  • 35:52Now another problem,
  • 35:53when we compare a consumer sleep
  • 35:55technology to polysomnogram and
  • 35:56there's some performance metrics cited,
  • 35:58it takes time to do the study.
  • 36:01Then the studies written up and
  • 36:03it goes through a peer review
  • 36:05process for publication,
  • 36:07and then someone will say,
  • 36:09Oh yes,
  • 36:10that device has been validated against PSG.
  • 36:13Um,
  • 36:13that performance metrics that are
  • 36:16output from that study are going
  • 36:18to be specific to the sensors
  • 36:21of the device that was used,
  • 36:23plus the algorithm of the current
  • 36:26firmware and software iteration,
  • 36:27and these are things that change,
  • 36:30and they change with actigraphy as well,
  • 36:33so that's not something you,
  • 36:35but oftentimes with consumer
  • 36:37sleep technologies,
  • 36:38the hardware of the device is updated.
  • 36:41There's new models that come out or.
  • 36:44They could spend all this time on
  • 36:46a validation study and the company
  • 36:48goes out of business and so that's
  • 36:50another threat to generalizing
  • 36:52the results of these studies.
  • 36:54And what we really need to kind of
  • 36:58move passed through these oneof.
  • 37:01You know small convenience samples
  • 37:03and lack of transparent data in
  • 37:06regards the performance of these
  • 37:09consumer sleep technologies is
  • 37:11really large datasets with some
  • 37:14disclosure of the methodology behind how
  • 37:17this sleep classifiers are developed.
  • 37:19An also information about the population.
  • 37:23And we are getting closer to that.
  • 37:26So the Somnia project is a project
  • 37:29in the Netherlands where in their
  • 37:31individuals who are receiving clinical
  • 37:34polysomnogram there recording a variety
  • 37:36of other sensors and these are sensors
  • 37:39that could be incorporated into consumer
  • 37:41sleep technologies or that already are,
  • 37:44for example, respond PPG,
  • 37:46an actigraphy under the bed sensor,
  • 37:48microphones, and what this is
  • 37:50going to establish is a data set
  • 37:53that includes Co recorded.
  • 37:55Polysomnogram with all of this different
  • 37:58sensor signal so that the best
  • 38:01predictors of sleep can be determined
  • 38:03an also that these can be tested.
  • 38:06These algorithms can be tested
  • 38:09on different population subsets.
  • 38:12So what does our future hold in regards
  • 38:15to consumer sleep technologies?
  • 38:17So although it's tempting to
  • 38:19just Subs in for actigraphy,
  • 38:21I believe these could be much more valuable.
  • 38:25I think that they could change the
  • 38:27logistics of Sleep Medicine and
  • 38:29completely transform our practice.
  • 38:31So instead of seeing a patient and
  • 38:34taking this leap history that's
  • 38:36primarily retroactive,
  • 38:37and then maybe sending them home
  • 38:40with an actigraph or sleep Diaries,
  • 38:42we can have a long term recordings
  • 38:45that were performed prior to their
  • 38:48first appointment.
  • 38:48Because they own the consumer sleep
  • 38:50technology and have those available
  • 38:52at our initial evaluation.
  • 38:54So what I've heard this compared to
  • 38:56that you would never hear a woman come
  • 38:59to her first obstetric appointment
  • 39:01without an over the counter pregnancy test.
  • 39:04So this could be something equivalent
  • 39:06to that where even if it isn't
  • 39:08an FDA cleared device,
  • 39:10we have some idea of the sleep wake
  • 39:12patterns from over the counter source.
  • 39:15Also,
  • 39:15this can be used in between visits
  • 39:18and given our move to Tele medicine.
  • 39:20The fact that the patient's own these
  • 39:23devices really makes us a completely no
  • 39:26contact way to practice Sleep Medicine.
  • 39:29Consumer sleep technologies would
  • 39:30allow us to personalize interventions
  • 39:32to patients actual sleep patterns
  • 39:34as opposed to what we think they are
  • 39:36or what they try and remember them
  • 39:38to be at the clinic visit and those
  • 39:41interventions could be made in real time.
  • 39:44The API with the apps associated with
  • 39:46CST might also make it easier for us
  • 39:49to leave link sleep metrics to other things.
  • 39:53So,
  • 39:53for example,
  • 39:54mobile apps really lend
  • 39:55themselves to something called
  • 39:56ecological momentary assessment,
  • 39:58and that's when you evaluate self
  • 40:00report symptoms in real time in
  • 40:03response to a push notification,
  • 40:04which might give us a better understanding
  • 40:07of how sleep interacts with other symptoms.
  • 40:10Additionally,
  • 40:10we can get health and non health information.
  • 40:13Things like Geo location, social media use.
  • 40:16Alongside sleep through other apps,
  • 40:18potentially use of smart devices
  • 40:21like glucometers scales,
  • 40:22blood pressure,
  • 40:23cuffs with our consumer sleep
  • 40:25technology to help see how sleep
  • 40:28affects those objective parameters,
  • 40:30and then obviously ideally these
  • 40:32would interface with our electronic
  • 40:35health records so the sleep provider
  • 40:37could see this data
  • 40:39in a seamless fashion and hopeful
  • 40:42that one day sleep with what we've
  • 40:45learned longitudinally could even.
  • 40:46Act as a vital sign and we'd be able
  • 40:49to see changes in the signature of
  • 40:51consumer sleep technologies before
  • 40:53an acute health event occurs,
  • 40:55so we can see an impending deterioration
  • 40:57in our patients and potentially intervene
  • 40:59before they end up in the hospital.
  • 41:01And that's the case.
  • 41:03I hope with Pap data as well.
  • 41:06So this is 1 example of a consumer sleep
  • 41:09technology that has been FDA cleared
  • 41:11up for personalized treatment and you
  • 41:14guys have probably heard of this.
  • 41:16So this is the night where app and this
  • 41:19is an app for individuals with nightmares,
  • 41:22primarily individuals with PTSD
  • 41:23and what this entails is you charge
  • 41:26your Apple Watch during the day
  • 41:28and then you wear it at night.
  • 41:31This device doesn't.
  • 41:32Outright claim to track sleep.
  • 41:34It says it kind of learns the
  • 41:37the cardiac and motion patterns
  • 41:39during sleep over the course of
  • 41:42days and then based on that,
  • 41:44it identifies departures from that
  • 41:46that the app thinks that the algorithm
  • 41:49thinks are nightmares and there's
  • 41:51a stimuli of vibration or that goes
  • 41:54through the watch to arouse the patient
  • 41:57just slightly from the nightmare
  • 41:59interfere with the nightmare without
  • 42:01completely waking them for sleep.
  • 42:03And there's trials going on
  • 42:05right now with this device.
  • 42:07Now we talked about the need for this
  • 42:11truly longitudinal recording of sleep
  • 42:14to really understand sleeping disease
  • 42:16an when you look at the data that we
  • 42:20currently have, it's pretty short,
  • 42:22so active graphic recordings in studies
  • 42:25that associate sleep timing duration,
  • 42:27intraindividual variability with
  • 42:29health outcomes have been 5 seven days.
  • 42:32And if we're looking at contribution
  • 42:34of sleep to chronic disease,
  • 42:37we're going to learn much more.
  • 42:40If a consumer sleep technology
  • 42:42that's worn for weeks, months,
  • 42:45seasons, years can be used.
  • 42:48But the other thing is that these devices
  • 42:51they are inexpensive and they're pretty
  • 42:54widespread in regards to their use,
  • 42:57and that gives us the ability to perform
  • 43:00sleep research with objective data at scale.
  • 43:03Colleagues at U of M used an app to globally
  • 43:06quantify sleep in circadian rhythms.
  • 43:09So across the world.
  • 43:11Additionally,
  • 43:11there's studies that have been
  • 43:13done in interns who have a really
  • 43:16unique sleep challenge,
  • 43:17and because of the taxing nature
  • 43:20of the medical training program,
  • 43:22it would be difficult to assess
  • 43:24their sleep otherwise.
  • 43:25And there was really unique study
  • 43:28by Eric to pollen colleagues where
  • 43:30Fitbit data was looked at in a state.
  • 43:34Aggregate level Anne used to
  • 43:36determine the likelihood of influenza
  • 43:38like illness outbreaks,
  • 43:39so these are things that couldn't
  • 43:42really be easily conducted with use
  • 43:45of actigraphy and are very promising.
  • 43:48And of course, what about Covid?
  • 43:50So Covid has been now used in a few studies.
  • 43:54These are the three big ones.
  • 43:57Excuse me,
  • 43:57CST's have been used in Covid studies
  • 44:00are the three big ones to determine
  • 44:03if consumer wearable technologies
  • 44:05can help predict covid both with
  • 44:07and without the incorporation
  • 44:09of symptoms, and these have been promising
  • 44:12these primarily used resting heart rate
  • 44:14and heart rate variability and activity,
  • 44:16and some of them you sleep.
  • 44:19To predict covid, they've also been found
  • 44:22to predict other viral illnesses and has
  • 44:24shown that the data from the sensors is
  • 44:27beneficial above and beyond the symptom data.
  • 44:30And here is a nice picture from one
  • 44:33of these investigations and you can
  • 44:35see that symptom onsets day zero here,
  • 44:38and your blue tracing is respiratory rate.
  • 44:41Your red tracing is heart rate,
  • 44:43and your green and pink are
  • 44:46heart rate variability measures.
  • 44:47This is all believe this one is.
  • 44:50Yep, this ones from Fitbit and if you
  • 44:53look at Day Zero which is symptoms start,
  • 44:57you're already seeing a change from
  • 44:59baseline and these consumer sensor
  • 45:01parameters and so this could be
  • 45:04very helpful in identifying people
  • 45:05who are going to have a viral
  • 45:08illness specifically coded.
  • 45:09And if all else fails,
  • 45:11you can put your wearable
  • 45:13tracker on your pet.
  • 45:14So I thank you for your attention
  • 45:16today and I'm happy to take any
  • 45:19questions and thank you for having me.
  • 45:23Thank you so much for a fantastic
  • 45:26overview of that topic.
  • 45:28Doctor Goldstein, that was really
  • 45:30just a really great in depth
  • 45:32perspective on all of these devices,
  • 45:35how they work algorithms,
  • 45:36the data, everything. Thank you.
  • 45:38So I'll open it up for questions.
  • 45:41And while people are getting ready for that,
  • 45:44I'll just start with one.
  • 45:46I love your optimism vote kind of
  • 45:48embracing the practical advantages
  • 45:50of consumer sleep technology
  • 45:52over traditional actigraphy,
  • 45:53and obviously in the setting of research.
  • 45:56I think you've shown great
  • 45:58examples of how this can be used,
  • 46:01especially for collecting large
  • 46:02amounts of data that are just not
  • 46:05possible with with actigraphy.
  • 46:07What I'm struggling with.
  • 46:08Is what the mechanism is that we
  • 46:10as asleep community could use
  • 46:12to actually encourage companies
  • 46:14to be transparent about their
  • 46:17algorithms to facilitate validation.
  • 46:19And I am not sure how I could see
  • 46:21that there ever will really be a great
  • 46:25financial incentive for validation
  • 46:27to hold companies accountable.
  • 46:29I mean, by what means?
  • 46:31Does Apple I mean have to to
  • 46:34actually broadcast that these
  • 46:36things are accurate to consumers?
  • 46:38Pay attention to this,
  • 46:40how can?
  • 46:41How can we put pressure on them or what
  • 46:43is their their ill and incentive to do
  • 46:46this? Yeah, and I you know
  • 46:47I I don't know the answer.
  • 46:49There's obviously a lot of suggestibility,
  • 46:51so once the patient starts wearing the
  • 46:53device but we often see is that they take
  • 46:56the information from that's reported
  • 46:57obviously by the app more than they
  • 46:59take their own perception of their sleep.
  • 47:01So there becomes, you know,
  • 47:03I would think that most patients would
  • 47:05want these to be as accurate as possible,
  • 47:07but there also interacting with the
  • 47:09device in that way that they are
  • 47:11taking into account what it says.
  • 47:13So they might take that above their own.
  • 47:16Own subjective impression of their sleep,
  • 47:18and so it's hard to know.
  • 47:20I don't know if it will be on the
  • 47:22order of the customer because of that.
  • 47:25That pushes the companies to more validation.
  • 47:27I don't think that we can do.
  • 47:30We can currently continue on with
  • 47:32the validation framework as it is if
  • 47:34we want to engage with corporations,
  • 47:36it just doesn't.
  • 47:37It doesn't work.
  • 47:38It's too slow to wait till these
  • 47:41are companies come out in a peer
  • 47:43reviewed publication.
  • 47:44I don't know if they'll have,
  • 47:46you know, I know some of them.
  • 47:48You know there might be occupational
  • 47:50programs and insurance company programs
  • 47:52and fitness for duty programs that
  • 47:54might be more attractive than healthcare.
  • 47:57You know,
  • 47:57we already don't get reimbursed
  • 47:59for FDA cleared Actigraphy,
  • 48:01so I don't think that you know
  • 48:03from the hospital from the Medical
  • 48:05Center standpoint we're really
  • 48:07going to move the needle on that.
  • 48:10I would hope that they are
  • 48:12interested in what we have to say,
  • 48:14and they seem like they are.
  • 48:17I mean, I've had.
  • 48:18Positive interactions with industry,
  • 48:20you know, and we don't need to have.
  • 48:22You know when you're using
  • 48:24machine learning in general,
  • 48:25it's you're not going to know
  • 48:27everything about the algorithm, right?
  • 48:29But we do need some trends,
  • 48:31so that can you know,
  • 48:32maintain trade secret,
  • 48:33but we do need some information
  • 48:35about the population.
  • 48:36And just like we were going to
  • 48:38recommend with AI based sleep stage,
  • 48:40and we need to understand what
  • 48:42population it was trained on
  • 48:44hasn't been independently tested.
  • 48:45Things like that,
  • 48:46or things that we would like to know.
  • 48:49From the companies.
  • 48:50Before we use these and you know,
  • 48:53rely on these, but you know,
  • 48:55I don't really know what's going
  • 48:57to push the needle. That's that.
  • 48:59I don't have any answer to it's
  • 49:02a great question.
  • 49:03Thanks.
  • 49:05Anyone else who wants to ask a
  • 49:07question to Doctor Goldstein?
  • 49:10Hi, it's Brian. I have a question.
  • 49:16So yeah, I agree with Lauren
  • 49:19that's so great talk an I myself,
  • 49:21I'm sort of wondering about using various
  • 49:24different technologies in my own research,
  • 49:26so one one comment and then one question just
  • 49:29based on what you guys were just saying.
  • 49:32I wonder if. I mean if they if the this
  • 49:35you know ASM sleep research society,
  • 49:38could we have any leverage with these
  • 49:41companies to say you know the more
  • 49:43transparent you are about your technology,
  • 49:45it how it works,
  • 49:47the more likely researchers already
  • 49:49use it in their own research.
  • 49:51I mean what I've thought about is,
  • 49:53I've looked at these different technologies
  • 49:56is am I going to have access to the raw data?
  • 49:59I think that's really important
  • 50:01from a reach research perspective,
  • 50:03and so perhaps some companies you know.
  • 50:06They want uptake of these devices.
  • 50:08Might be receptive to that kind of argument.
  • 50:11Just a thought.
  • 50:12My question is Doctor Goldstein,
  • 50:14have you whether you've looked at some
  • 50:17of these consumer technologies that
  • 50:19actually look at EEG as opposed to you,
  • 50:23know actigraphy,
  • 50:23and if you have any thoughts or
  • 50:26comments about those technologies.
  • 50:28So I haven't in detail, I'll say it.
  • 50:31Probably many people miss in this room
  • 50:34zoom room whatever we call it now.
  • 50:37Have tried them.
  • 50:38I mean,
  • 50:39I haven't been impressed physically
  • 50:40trying those myself and we don't have.
  • 50:43There's very limited,
  • 50:44limited data on those as well
  • 50:47an I haven't seen much.
  • 50:48I haven't personally as far as
  • 50:50like the dry e.g head mansized and
  • 50:53that's where you're talking about.
  • 50:55I haven't seen them interact with
  • 50:58academia as much as the heart rate and.
  • 51:01Motion based wearable devices.
  • 51:03You would think that they would be better,
  • 51:07but I'm not sure.
  • 51:08There's definitely a disconnect and
  • 51:09they want to make it so patient
  • 51:11can just put it on,
  • 51:12but that it also records the EG
  • 51:14and I just don't know if the dry
  • 51:16e.g technology is there yet.
  • 51:18You should try some of these
  • 51:19out and I think you'll agree.
  • 51:21I looked like I was.
  • 51:22I was in some kind of like I was
  • 51:25awake and I was like in Delta.
  • 51:27I mean they're just they don't.
  • 51:28I mean 'cause you when you think
  • 51:30about the work that guys end
  • 51:32up putting on e.g leads,
  • 51:33it just doesn't translate.
  • 51:35I mean you can have like.
  • 51:36Single lady e.g.
  • 51:37But these are embedded in the headband,
  • 51:39so I don't know.
  • 51:41I just don't think we're
  • 51:42quite there yet with that.
  • 51:49I'm just going to scan the chat for any
  • 51:52question I had actually question then
  • 51:54this is that you know where how you
  • 51:56doing Kathy good to see you again.
  • 51:58At Norwalk hospital.
  • 51:59So yeah, we talk and it's a very,
  • 52:02you know, sort of hot topic issue.
  • 52:04You know when I think about this
  • 52:06I I think about how in terms of
  • 52:08population health is probably where
  • 52:10it may have its biggest effect and
  • 52:12I think of sleep deprivation, which
  • 52:14is really an epidemic that
  • 52:16we don't talk much about in
  • 52:17Sleep Medicine as much as we probably should.
  • 52:20And so when people track their sleep,
  • 52:22you know one thing I find similar
  • 52:24to like the step trackers you know
  • 52:26they'll look at and they'll say,
  • 52:28oh, I've done, you know?
  • 52:305000 steps OK, Great
  • 52:31and you don't do anything
  • 52:33with the information,
  • 52:34so I guess the question is can we
  • 52:36can we use these sleep trackers?
  • 52:38ANO Validation is important,
  • 52:39but maybe sort of the more
  • 52:41important issue is can the
  • 52:43individual person use it to
  • 52:44sort of improve their health,
  • 52:46get more sleep and
  • 52:47you know? And so is there.
  • 52:49I think that maybe a sense of
  • 52:52kind of focus in on a population
  • 52:54in because a lot of people have
  • 52:56these these trackers now so you
  • 52:58aware of anyone who is using.
  • 53:00Consumer technology in an algorithm
  • 53:01to try to help people either get
  • 53:03more sleep or you know similar
  • 53:05to maybe diagnosis or like
  • 53:06a screening for sleep apnea.
  • 53:07I can't do what they
  • 53:09have for the Apple Watch,
  • 53:10which is this A-fib detection,
  • 53:11which is quite interesting. Where
  • 53:13that would sort of clue them in that
  • 53:15they may have a sleep problem.
  • 53:16So I guess the question is,
  • 53:18are you aware of any studies
  • 53:20that are looking at using the
  • 53:21zoom technologies to increase
  • 53:22total sleep time in a population?
  • 53:24So this is it.
  • 53:25This is a great question and a great point.
  • 53:27And as of this time you know
  • 53:29both for fitness and sleep.
  • 53:31There there's not good evidence to
  • 53:33suggest that tracking improves behavior.
  • 53:35There is some work that's going on though,
  • 53:38so Kelly Baron and colleagues
  • 53:39at University of Utah I think.
  • 53:41Or maybe she did this back in Chicago,
  • 53:44but they used a consumer sleep technology.
  • 53:47They developed an app and they did
  • 53:49sleep extension in people with short
  • 53:51sleep duration and hypertension,
  • 53:53and they did improve sleep times
  • 53:55and they did improve blood pressure
  • 53:57control in those individuals.
  • 53:59So an I think that's the key.
  • 54:01In how we use these is there
  • 54:03never gonna be exactly like e.g
  • 54:06defined sleep like they will just.
  • 54:08It's your measuring something
  • 54:10totally different.
  • 54:10Magni Eunice told me once he was like
  • 54:12I don't understand he's like these
  • 54:15sleep trackers there monitoring things
  • 54:17that we don't stay asleep with that
  • 54:19they're totally different, right?
  • 54:21So but maybe it's not,
  • 54:23you know,
  • 54:23getting to this perfection level of accuracy.
  • 54:26Maybe it's using it as a completely
  • 54:28different construct of sleep and seeing
  • 54:31how interventions applied to that.
  • 54:32Can change your still both your
  • 54:34sleep outcome in other health
  • 54:36outcomes totally agree.
  • 54:40Thank you, ******
  • 54:46Hi Kathy, this is Andres
  • 54:47and Chuck really nice talk.
  • 54:49Hey, thank you hear hear your thoughts
  • 54:51and agree with you and Ian about you
  • 54:54know these are probably not going to
  • 54:57be perfect matches to what we measure
  • 54:59in the PSG and I wonder if you know
  • 55:02we can use the metrics from these
  • 55:04devices as biomarkers and correlate
  • 55:06them or establish a relationship with
  • 55:08meaningful outcomes and overtime.
  • 55:09They might be actually more relevant
  • 55:11than e.g signals that we get.
  • 55:13I mean sleep is defined by e.g but we're
  • 55:16only looking at superficial layer.
  • 55:18Of the cells and you know that some of
  • 55:20that sleep occurs supportively, right?
  • 55:22And so.
  • 55:23It's hard to say what we what
  • 55:26is the truth here,
  • 55:28but I was going to ask you question about the
  • 55:32what do we do with all of this information.
  • 55:35So for example,
  • 55:36if we're tracking patients over days,
  • 55:39weeks, months,
  • 55:39how do you analyze that
  • 55:41data in a meaningful way?
  • 55:43And how do you relate it to an outcome?
  • 55:46I think it's been a challenge 'cause so far.
  • 55:49We've really mostly been used
  • 55:51to metrics that are very.
  • 55:52Sort of Abacus like where we count
  • 55:55things overtime or we you know measure
  • 55:58some intensity area under the curve
  • 56:00and so the data is being collected.
  • 56:03Now at these devices is vastly
  • 56:05different and so even with just sleep
  • 56:08duration or sleep period written ecity
  • 56:11like have you seen any unique ways
  • 56:13that that data has been analyzed?
  • 56:16And so not from consumer wearables
  • 56:18and not for extended amounts of time.
  • 56:21And that's where we'll obviously need
  • 56:23people with your level of expertise
  • 56:25in AI to help distill the data,
  • 56:28because there's going to be a
  • 56:30lot of Co dependencies,
  • 56:31there's going to be just a vast number of,
  • 56:34and this is not even.
  • 56:36The machine learning algorithms
  • 56:37to determine sleep from wake
  • 56:39for everyone who's listening.
  • 56:40This is the once the post process
  • 56:43data is just so much bigger
  • 56:45and faster than anything else.
  • 56:47So we will need more flexible
  • 56:49models to deal with that.
  • 56:50I think the closest that we
  • 56:52get is the work that Meredith.
  • 56:54I think there's no.
  • 56:55I don't think pen is here that
  • 56:58Meredith Wallace has done,
  • 56:59but that was using actigraphy for
  • 57:01about 5 days, but they were able.
  • 57:03At least you know it's a start
  • 57:05for looking at these different
  • 57:07parameters of sleep,
  • 57:08at least at the same time simultaneously,
  • 57:11duration rhythmicity timing,
  • 57:12but this is going to be even
  • 57:14way more than that,
  • 57:15so it'll be interesting to see
  • 57:17how that data is dealt with.
  • 57:19When we get to that point.
  • 57:23Great, thank you so much again,
  • 57:26Doctor Goldstein.
  • 57:26I think will will close there.
  • 57:29I just want to let everybody know
  • 57:31that our talk next week we were
  • 57:34going to have as we have on the 2nd
  • 57:37Wednesday of each month adjoint Yale,
  • 57:40Harvard Sleep Conference and we're
  • 57:42having a kind of unique talk.
  • 57:44James Nestor is an author of the book
  • 57:47Breathe the new science of a Lost Art,
  • 57:50and so he's going to be giving
  • 57:53next week stock so please.
  • 57:55Join us for that and again,
  • 57:57thank you so much Doctor Goldstein
  • 57:59and stay well everybody will see you
  • 58:01next week. Sounds good. Thanks,