CBDS Digital Health Seminar: “Challenges in Developing Online Learning Algorithms to Dynamically Personalize Mobile Health Treatments” by Susan Murphy, PhD, Harvard University

“Challenges in Developing Online Learning Algorithms to Dynamically Personalize Mobile Health Treatments”

Susan Murphy, PhD
Professor of Statistics, Computer Science and a Radcliffe Alumnae Professor
Harvard University

Zoom: https://yale.zoom.us/j/91570712568

There are a variety of formidable challenges to reinforcement learning when used to design just-in-time adaptive mobile health interventions for individuals with chronic disorders. Challenges include settings in which most treatments delivered by a mobile device have immediate nonnegative (hopefully positive) effects but longer term detectable effects tend to be negative due to user burden. Furthermore, after a trial in which a reinforcement learning algorithm has been used, the data must be amenable to conducting a variety of statistical analyses, including causal inference as well as monitoring analyses. Other challenges include an immature domain science concerning the system dynamics but the need to incorporate some domain science due to low signal to noise ratio as well as non-stationary and limited data on individuals with chronic disorders. Here we describe how we confront these challenges including our use of low variance proxies for the delayed effects to the reward (e.g. proximal outcome) in the learning algorithm.

Dr. Murphy’s lab develops data analysis methods and experimental designs to improve real time sequential decision-making in mobile health. In particular, her lab develops algorithms, deployed on wearable devices, to deliver and continually optimize individually tailored treatments. She developed the micro-randomized trial for use in constructing mobile health interventions; this trial design is in use across a broad range of health related areas. In these trials each participant can be randomized or re-randomized 100’s of times. Some examples of micro-randomized trials that are completed or are in the field can be found at https://methodology.psu.edu/ra/adap-inter/mrt-projects#proj.

Dr. Murphy is a member of the National Academy of Sciences and of the National Academy of Medicine, both of the US National Academies. In 2013 she was awarded a MacArthur Fellowship for her work on experimental designs to inform sequential decision making.