QuNex, a neuroimaging processing suite

Name: QuNex, a neuroimaging processing suite
Uploaded: 2020-10-05T16:23:40.77Z
Duration: 57 min 33 s
Description: Dr. Alan Anticevic , Associate Professor, Department of Psychiatry, Division of Neurocognition, Neurocomputation, and Neurogenetics (N3) , Yale School of Medicine

October 05, 2020

Dr. Alan Anticevic, Associate Professor, Department of Psychiatry, Division of Neurocognition, Neurocomputation, and Neurogenetics (N3) , Yale School of Medicine

Information

ID: 5707
To Cite: DCA Citation Guide

Download Transcript

00:00Each of us is an associate
00:03professor of psychiatry at Yale.
00:05He's a native of Croatia and received
00:08graduate training at Washington University
00:10in Saint Louis with doctors Deanna Barton,
00:14David Van Essen at Yale.
00:16Allen runs the end three division,
00:19which stands for neurocognition,
00:21neural computation and neurogenetics,
00:23and he is broadly focused on using
00:26computational methods to examine cognition
00:28and genetics in psychiatric illness.
00:31His work has been funded by the Ann.
00:34I am age,
00:35the brain and Behavior Research Foundation,
00:37industry collaborators and
00:39the Simons Foundation.
00:40Allen is a leader in the field
00:42of psychiatric neuroimaging
00:43and computational psychiatry,
00:45and today he'll be speaking
00:47to us about Q next,
00:49which is a state of the art
00:51neuroimaging processing and analytic
00:53platform that was created by his lab
00:56and collaborators to flexibly and
00:58powerfully integrate cutting edge
00:59image Ng and analytic techniques.
01:01For imaging researchers,
01:02and I'm personally very pleased to
01:05be introducing Alan and to have him
01:07kick off our summer stuff Alan,
01:09I'll turn it over to you now.
01:11Thanks so much for being here.
01:15So I want to just make sure
01:18everybody can hear me. Yep, OK so.
01:20Thank you all. Thank you, Helen.
01:23Thank you Sarah,
01:24and thank you young son so I.
01:27It's it's. I'm thrilled to be here.
01:29It's really an honor to
01:31present in this context.
01:32I think that.
01:34It speaks volumes that you guys rallied
01:37folks around this really important topic,
01:40and so I'm just really thrilled to see
01:43the field of comp psych flourish and
01:45that you are really making this effort.
01:48And thank you for the kind introduction,
01:51Sarah,
01:52the one minor correction I would say,
01:54is that I code directly in three divisions,
01:58so my close colleague and collaborator John
02:01Murray was recently promoted in the Co.
02:03Director and so.
02:05He is also at the helm of this effort,
02:09and I'll be frequently mentioning
02:12his name so.
02:14The current presentation sort
02:16of a different beast than what
02:18I'm typically used to.
02:20It's it's really this methods
02:22and primer for Compsci Canoeing,
02:24Comics, and from what I understand,
02:27the goal is really to introduce
02:29you to a methodology in a tool that
02:32would that has practical use in
02:35computational psychiatry and so.
02:37I was asked to feature our Q Nix
02:42platform which is in the final
02:45stages of being.
02:47Prep for public release and this will
02:49be a good test run and I was also
02:52told that there's a time for Q&A at the end,
02:55so I'll see if I could time the talk in
02:58such a way that people can ask questions.
03:01So.
03:02So the first thing that I want to
03:04make sure is that I acknowledge
03:06folks who have participated,
03:08so this is I'm the slide isn't
03:10comprehensive and I forgive me those
03:12who I'm leaving off this slide.
03:14But really,
03:15the key point that I want to highlight
03:18is that this is a multidisciplinary.
03:20Multi year project Kunik started
03:22basically as some scripts at wash U
03:25when I was at first year grad student
03:28with my close colleague and friend
03:30Gregor Epub shoe is full professor
03:33at University of Ljubljana in
03:35Slovenia and so his lab is.
03:37Also, I would I would consider
03:40foundational code developer of this effort.
03:43And I wouldn't be here talking
03:46to you without his support and.
03:49Vision also.
03:50Our lab really contributes to this
03:52effort by continuously using refining
03:55and stress testing the platform
03:58so it's iterative and we now have
04:01folks who are students and graduated
04:04to become junior developers in
04:06their own research careers of the
04:09platform and also the marine lab
04:11with who has really enriched the
04:15platform substantially through a
04:17computational psychiatry focus so.
04:19So what is this right?
04:21So that's one of the questions
04:23that I was asked to answer, and.
04:25I'm sorry I'm going to interrupt your pro.
04:30Second, we only still see the first slide.
04:34On Is that right? Yes, let's try it.
04:39Let's try it this way.
04:42I think I know
04:51what's happening.
05:03What about now? You are awesome awesome.
05:07Thank you for that.
05:09So this was my acknowledgement slide.
05:11I acknowledge people so let's move on.
05:14So one of the questions that was
05:16posed to me and young son really
05:19helped draft this in the focus of
05:22the seminar is what's the purpose?
05:25What? What are we trying to answer?
05:27And so I think that are really
05:30common challenge that we have
05:32faced is that research projects
05:34have to be able to integrate.
05:37Pipelines Multimodal information
05:38across data sets across scale,
05:41both within a laboratory and across
05:43publicly available data sets,
05:45and so that common challenge really
05:47will be echoed throughout the talk,
05:50and so that really prompted the
05:52development of this research platform.
05:55So QX is what is it?
05:57It's a framework for integrating
06:00multiple image Ng,
06:01processing analytical pipelines into a
06:03single unified ecosystem which provides a
06:07flexible and powerful platform and engine.
06:09For Multi Molnar imaging,
06:10so hopefully as we deep dive
06:13more into what is connects,
06:14this will become more apparent.
06:16So why did we build it?
06:19So.
06:21To our knowledge,
06:22or at least to my knowledge,
06:24when we were starting this process,
06:26this journey is that we found that
06:28there was not a single platform that
06:31unites new image Ng pipelines and
06:33thereby allowing folks to really have.
06:35Uh.
06:36Easy and I truly mean easy access
06:39to these resources to analyze
06:42and deploy their data sets.
06:45And So what really prompted?
06:47Q Nixon why we developed it,
06:50is because we were able to garner
06:53support around multiple projects
06:54that then we're taking advantage of
06:57this software development effort,
07:00and this iteratively provided an
07:02organic growing interest for this
07:04platform which we're very excited too soon.
07:08Featuring the community and so how
07:10do you use it and what do you need
07:13to know to deploy it is some basic
07:16knowledge of Linux and scripting
07:18will be helpful to you if you know
07:20how to do it and you'll understand
07:23as I move through the talk.
07:26Why? This this matters,
07:28and the key point here is that.
07:33Kionix allows.
07:34I've given research laboratory, junior,
07:38researcher, or even an entire team.
07:412.
07:44Utilized this framework to
07:46process and analyze their data,
07:48so having some understanding or some
07:50some background as to what analytic
07:53approaches you want to deploy is is
07:56helpful before you start using it.
07:58And also it's an open source framework
08:01that deliberately integrates
08:03other Open source tools such that
08:05it can straddle both academic
08:07and commercial applications of
08:09necessary and so it can be used
08:12with various modes in your image Ng.
08:15And it's been stress tested
08:16across various data sets,
08:18and so we'll talk about that this
08:21publication that's relevant here.
08:22This is a paper by my former students.
08:25Now research faculty at, you know, Lisa G.
08:28Who published this inner image?
08:30It's an example of Q Nixon in action, so.
08:34Now if we zoom out 10,000 feet
08:37sort of from from the motivation
08:41for today's talk so.
08:43I think that it's fair to say that
08:46a core problem in computational
08:48psychiatry is for us to develop
08:51a system and to deploy a system.
08:554. Neurobiologically grounded
08:58multimodal biomarkers.
09:01Capacity, which I would call features.
09:04To support. A high throughput,
09:07scalable, secure,
09:08regulatory compliant implementation
09:10which is the informatics piece
09:12to enable data driven and
09:14reproducible discovery Science,
09:15which is the analytics piece
09:18and to be actionable.
09:19Useful for personalized to population
09:22level neuro behavioral markers
09:25in mental health which is our
09:27end point and so I see this as.
09:30A field wide problem,
09:32not a single laboratories problem
09:35and this cartoon picture here kind
09:38of illustrates this vision that
09:40hopefully we can automate through
09:43statistical learning approaches.
09:45More and more of our decision making
09:48in a irrationally guided way for
09:50personalized treatment delivery in
09:52psychiatry that is rationally implemented.
09:54For that I would argue we need an
09:58entire ecosystem in a re framing
10:00of how we approach this.
10:02And I would argue that we can exploit
10:05human or imaging methods to meet this goal.
10:09Which is what Kleenex is designed
10:10to enable and help support.
10:12So just kind of a trip down memory Lane.
10:15So if we know it is human image
10:18in good enough,
10:19is it even a tool that's useful
10:21for us to leverage?
10:22So why?
10:23Why build an ecosystem around it if we
10:26don't trust the tool and so for for those
10:29of you who are not imagers in the crowd,
10:32or having you know thought
10:34about this modality as much.
10:36So one example ner maging modality that
10:38provides a very good use cases bold fMRI,
10:41and in particular a method that
10:43a lot of people resting state,
10:45and so it's.
10:46It's a nice example of how our
10:48field has matured and how these
10:50tools become really deployable.
10:52So this is a picture from 1995.
10:54Well,
10:55actually it's a picture from a review paper.
10:58Published by Randy Buckner that
11:01recreates the 9095 effect and so.
11:03This is an example of sort of where
11:07the field was in some years ago,
11:10and this phenomenon that we
11:12have all come to noahs resting
11:14states is a nice example where.
11:17Tools,
11:17they even the basic intuitions
11:19in 95 were missing.
11:21So what this is this shows is if
11:24you place a seed which is now kind
11:26of a common term in our vernacular.
11:29In the left Motor Cortex and you
11:32quantify what is the spatial temporal
11:34variation of the signal from that
11:36region with all the other parts of the brain,
11:40and you plot a statistical map of that.
11:43You get this picture of bilateral motor
11:45cortex which highlights that these are.
11:47Coal bearing,
11:49spatial temporal signals,
11:51and so this term,
11:54called functional connectivity
11:56was really actually believed to be
12:00spurious for a good part of the next
12:0510 years until some group at wash U.
12:09Started to notice that this effect
12:11also coincided with some other
12:13task evoked in pet findings,
12:15and it's fair to say the rest is history.
12:19So after the Fox Babe bring 2006 to
12:22feel jumped on board and this is one
12:25of many illustrations from Thomas Joses
12:28work while he was still in Randy's lab,
12:31where where they showed very elegantly
12:33that you can image the human brain at
12:36scale and arrive in large scale functional
12:39networks reliably across thousands of people.
12:42And these kind of watershed moments
12:44prompted the human connectome project,
12:46which is again it was just one of
12:50many initiatives now in human images.
12:52Which in this case derive the comprehensive,
12:55not a definitive,
12:56but a comprehensive partition
12:58of the human cerebral cortex,
13:00multi modally, and so we're now in 2020,
13:04and the advances continue,
13:06and really the methods are developing
13:09so rapidly I can't even keep up
13:12with stuff that's coming out.
13:14But for us to really continue to innovate
13:17this now highly accelerated pace,
13:19we need to really.
13:21Strive towards standards and those
13:24standards need to allow junior folks
13:27to enter the field really rapidly
13:29and effectively around methods that
13:32can be reproducible and provided
13:35teaching the platform.
13:36And so that's one of the
13:39motivations behind building QX.
13:41Another neural biological motivation
13:44for building Q Nix is really
13:48to adopt and facilitate.
13:50The ability to bring legacy data
13:53sets to the cortical surface as
13:55well as to support cortical surface
13:58imaging of State of the art.
14:00Next generation data sets,
14:02and so one of the elements of clinics
14:05that will keep talking about is that
14:07it's built around the human connectome.
14:10Project pipelines as one of the flagship
14:13flagship examples of pipeline implementation,
14:15but.
14:16This is a key property you don't have
14:18to do surface based Analytics Inc Unix.
14:22If you don't want to,
14:23you can stay in the volume,
14:25but really this is a feature that
14:28drastic will improve your S&R
14:29and your registration quality.
14:31If you actually derive cortical
14:33surface is at the single subject level
14:35across your modality, so this is.
14:37A method in a mechanism that is
14:40implemented in Q Nix natively across
14:44modalities and neurobiologically very,
14:46very important.
14:48If you want to multi modally compare
14:51individual subject data sets with.
14:54That level of anatomical precision.
14:57So. Our field I hope you know.
15:00We can agree that we need an integrated
15:02framework for both research and development.
15:07So and for development of that ecosystem such
15:09that it can meet production requirements
15:12and that iterative framework needs to
15:14exist with certain standards in place.
15:16And currently we don't have one.
15:18We have a lot of bespoke or one
15:20off or focused software platforms,
15:23but we don't have an ecosystem
15:25for growth and so why build 1?
15:28So this goes back to the first set of slides,
15:31which is why do we need a turn Keener
15:34imaging ecosystem to begin with?
15:36And so. Really,
15:38when I was starting my group in 2012,
15:41but even as a grad student,
15:43and really there were many pipelines or
15:46even before that there weren't no pipelines.
15:49But even with many pipelines we have few
15:53integrated quantitative environments for
15:55their stitching across modalities tools.
15:57And computational settings,
15:59and so this is really a problem.
16:01And it's also a little emphasis is
16:03placed on personalized endpoints
16:05across health and disease.
16:06With these new imaging ecosystem ecosystem.
16:09So in other words,
16:10you don't hold them always to the
16:12standard of personalized reproducibility.
16:15And this also highlights that
16:17we need a flexible,
16:18extensible multi modal throughput ecosystem.
16:20And what I mean by flexible that
16:22it can support multiple use case
16:24scenarios and what I mean by extensible
16:27is that a grad student from another
16:30laboratory or postdoc or another
16:32lab can easily build on top of it,
16:34and so that there's a software
16:37development kit that that exists so.
16:40The all those reasons really
16:42motivated to Nicks and the team is
16:46supported by my group here at Yale.
16:49The mind and brain laboratory,
16:51led by Gregory Poch who is Unix,
16:54gets into the public domain.
16:56You'll probably run into a lot
16:58and the marine lab here at Yale
17:01who has really pushed us too.
17:03Make Q Nix.
17:05Practical and usable in the context
17:07of specific computational psychiatry
17:10applications and so that's that's our team.
17:14So what does clinics do today?
17:17What is it really designed to do so?
17:22Unix has a clear DevOps model,
17:25so we've built it such that.
17:29I always ask myself if I was the first year
17:31assistant professor starting my laboratory.
17:33Would I use this thing?
17:35And to use it,
17:36it has to have a clear development
17:38system such that if somebody wants to
17:40build on top of it or they don't like a
17:44certain method in QX and want to replace it,
17:47it has to have that modular structure
17:49so will hit on that morning,
17:51or it has to be portable.
17:53So in other words,
17:54it has to be deployable across
17:56multiple environments with these.
17:58Versioning has to be clear,
18:00so you have to be able to revert
18:02to prior tag of Q.
18:04Next,
18:04rerun it in any environment and
18:06get reproducible results that
18:08occurred with that tag.
18:09So it has to be sustainable,
18:11and so one example is that we've
18:13built a partnership with the
18:15connectome coordinating facility,
18:16and with Dan Marcus and David Vanessa,
18:18and so currently the CCF team
18:20runs older HCP pipelines
18:21through tunics.
18:22And I'm really proud of that.
18:24And the fact that we worked
18:27with them for now close to.
18:29Think Costa decade actually
18:31to make this possible.
18:33So it has to support training and teaching,
18:36so if you're a student learning image in,
18:38you have to be able to
18:40learn around this thing.
18:42Otherwise it's not going to be useful.
18:44So we teach Unix at the HCP
18:46course we taught in 2019.
18:48Suddenly this year with Covid has
18:50made things more challenging.
18:52Another thing that's really important
18:53to us is that its pipeline agnostic.
18:56So while we do make Unix.
18:59A Welk Unix and its current
19:01implementation and its original
19:03implementation was built to support
19:05human connectome project pipelines.
19:07Also, it can easily embed fMRI prep.
19:10We've tried it.
19:12And also it can support nonhuman bye
19:15bye points. And we've done this.
19:18Already with the nonhuman primate
19:20version of the human connectome.
19:23Project pipelines. So basic.
19:24Lastly,
19:25pipe on agnostics.
19:26You can embed your pipeline into
19:29its framework and and run it.
19:32It's multi format so we support
19:34both 50 and nifty workflows.
19:37And one of the things that we really
19:40pay attention to is that it can be
19:42cloud deployable in cloud secure,
19:44because if you're going to use this thing
19:46for potential clinical imaging down the line,
19:49you have to be able to go that route.
19:52It can't just be a desktop client application
19:54on your computer has to be deployable.
19:57Another element that I'll really hit on a
20:00little bit later is it has to be automated.
20:02So if you have to baby sit
20:05this thing and run.
20:07Pipelines by hand.
20:07Every step of the way and write
20:09Bespoke Code and it doesn't
20:10have a one liner with which you
20:12can execute the whole thing.
20:14Then it's not going to be useful
20:15for folks who want to scale,
20:17so it has a turn key functionality
20:19and I'll show you how it works.
20:21It is also embeddable in high
20:24performance computing environments,
20:25so you can use it on your local
20:29high performance computing cluster.
20:31It supports slurm,
20:32PBS LSF schedulers so it's scheduler engine.
20:35It's compliant with high performance
20:38computing scheduling systems.
20:39It has no dependencies,
20:41which means that when you download it
20:45you get everything so you don't have
20:48to then go and request a license from.
20:51Source A and source beans or
20:54C and recompile this and that.
20:56In other words,
20:57when you download the conex container,
20:59everything is embedded and this is
21:02possible thanks to the support of
21:04Bruce Fish on Steve Smith who I want
21:07to highlight have been tremendous.
21:09Lee supportive of both.
21:10Fso in Freesurfer being too next
21:13distributable and this is done by
21:15auto agreeing to their license terms
21:17when you get the container so it's
21:20basically unlicensed propagation system,
21:22so it's.
21:22A very important.
21:24Feature of how to build
21:27interoperable software like this?
21:29And we deeply stress tested it.
21:31I think this number 10,000
21:33has now probably scaled three
21:35times since I made this slide.
21:37But these are just some large scale
21:39data set that have been Q Next.
21:42So Naples Pier at Yale Abidi
21:45one 2B Snip paying Cam can.
21:48HPN can see Rockland,
21:49the Embarq data set,
21:51the entire city data set, CMP.
21:53PNC actually is not here,
21:55so I know it is so basically,
21:58we've tested it on both legacy
22:00and HCP compliant data sets,
22:02and one recent stress test that we did
22:05was actually on the CMP data set in
22:09its current implementation, and it.
22:11Achieved a success rate on CMP out
22:14of the box of around 99.2% I believe,
22:18and the failed cases were just missing
22:21data which was pretty cool to see that
22:24it works with that level of Robustness.
22:27At this stage.
22:29So again,
22:30what does it enable specifically so?
22:34Flexible onboarding of data sets
22:36so your data sets can be in very
22:40different forms and very different.
22:42Origins, I suppose,
22:44and you can onboard them
22:46through the QX ecosystem.
22:49It enables data set prep,
22:51so setting up a data set
22:55for further analytics.
22:57It enables running pipelines,
22:58so in particular I'll talk about how we run.
23:02Human connectome,
23:03minimal processing pipelines through QX.
23:06It also enables additional task
23:08and functional connectivity
23:10preprocessing so it's commands
23:12natively also support GL N task
23:15analytics and functional connectivity.
23:17That's what we call our GM engine.
23:21It is capable of supporting task and
23:24functional connectivity at the session level,
23:26which means at the level of a single
23:29acquisition within a person as well as
23:32across all the sessions within a person,
23:34as well as across people at the group level.
23:37So you can get group analytics and
23:40the way that this part works is.
23:43Thanks to Anderson Winkler who
23:44generously also has supported
23:46the palm being embedded in Unk,
23:48Unix, and we've built a palm.
23:51Functionality around in QX,
23:53and this is a Greg's team,
23:56has really spearheaded this,
23:58so if you're a palm user you actually get.
24:02Our our ipam single command line
24:05feature through Q Next that you
24:07can use and then it also enables
24:09QC reporting visualization.
24:11So in other words you can go soup
24:14to nuts from getting your raw
24:16data all the way to group level
24:19analytics in the same ecosystem.
24:21Its features.
24:22And there there are several that
24:25I really want to highlight,
24:27so it has a command line interface,
24:30so that's important to us, so I'm I.
24:33This is my philosophy,
24:34but I'm a strong believer that
24:36dinner imager of the next generation
24:39or imager needs to be fluent in
24:42computational and Informatics.
24:43Tools.
24:43In other words,
24:44you have to be able to work at
24:46the command line.
24:47If you can't work at the command line,
24:49it's the equivalent of working in
24:51a wet lab and not knowing how to
24:53pipette so we don't have a gooey
24:55and we don't have a GUI on purpose,
24:57so it's a philosophy that we've
24:59really adhered to that if you
25:01want to use this thing for real,
25:02you have to be familiar with how
25:05the Linux Command Line Works.
25:06It has an API though,
25:08and So what I mean by that is an application
25:10programming interface and this enables
25:12interoperability and scalability.
25:14So if you really don't want to
25:16learn the command line and you
25:18just want to hit the API,
25:20you can do that too.
25:22It's very flexible in terms
25:24of its parameterisation.
25:25I'll show you some stuff in a second.
25:28And both in terms of study
25:31and session level parameters,
25:32but also the metadata specification
25:34for how you want to run it.
25:37One thing that we worked very
25:39hard is everything is logged.
25:41Everything is logged in
25:43terms of what you ran,
25:44how you ran it to the full command
25:47line is echoed and you get every
25:49single line of runtime code printed
25:52to a log for debugging purposes.
25:54So that's a feature.
25:56So if you're doing comp psych research and
25:58you're executing these things at scale,
26:01it's really important that you
26:03can clearly capture your logs.
26:05One thing that I'm really
26:08proud of that was led by.
26:10The director of our entry
26:12technologies here is only Tamayo.
26:14Is containerization,
26:15so container deployment.
26:17And also the thing that we use routinely,
26:19but some people may not.
26:21But again, it's really useful is robust high
26:24performance computing scheduler capacity.
26:25So what I really want to highlight is the
26:29containerization piece in particular.
26:30So it's really hard to deploy
26:33or imaging pipelines rapidly.
26:34So the harder way is,
26:36you know you can go to the.
26:39GitHub or Bitbucket repository
26:41of a given pipeline.
26:42Read the Readme file,
26:44cloned the thing locali.
26:45Make sure your computer is has
26:48all the necessary dependencies.
26:49Try to run, doesn't run debug iterates.
26:52Tried around debug,
26:53iterating so on and so this
26:56deployment for those who are
26:58computationally savvy and Dan have
27:00computer skills could be you know,
27:02relatively pain free.
27:03And since I showed this to
27:05my brother actually,
27:07who this is a shameless advertisement,
27:09so he he runs a computer gaming
27:12company and in San Francisco,
27:14which is releasing the first
27:16official PlayStation 5 release.
27:17And so he knows a lot more about
27:20computer science than I do.
27:22So I show them this and he kills
27:24me and mess with me saying,
27:27like this is absurd.
27:28You guys, how is this so hard?
27:31How come you takes you so long
27:33to deploy these pipelines?
27:35And so he said,
27:36you know,
27:37my standard is if a 13 year old
27:39can't play their computer game in 10
27:41seconds then then I lose my job like that.
27:44That's the standard.
27:44So I was like, Alright, well,
27:46I can't really do 10 seconds.
27:48But can we compromise like like 30 minutes?
27:50Can you get this up and running
27:53in 30 minutes?
27:54Sounds like great fine,
27:55so that was the Standard and
27:57so I worked with Saline.
27:59Really hard to Containerize the thing,
28:00and so our standard was can we get
28:03it up and running in 30 minutes?
28:05Right from the point when
28:07somebody goes into our website.
28:08Doesn't know what Unixes and goes through
28:11the container deployment and gets it gets
28:13it up and running on their environment,
28:15and so that less hard way is really
28:17facilitated because of container,
28:19a container which is based on a docker
28:22infrastructure in this case or singularity.
28:24Encompass is every single application that
28:27is necessary to be embedded in Punics,
28:30and so this is an informatics feature of the
28:33system that really makes your life easier.
28:37But our life was hard.
28:39Before you figure this out.
28:41So basically the two docker implementation
28:43scenarios are singularity or to
28:45container implementation scenarios.
28:47Are there docker or singularity?
28:49There's some pros and cons to both.
28:52We support both so singularity is secure for
28:54your high performance computing environment.
28:57Because you're a system,
28:58administrators are not going
29:00to let you use docker.
29:01And Docker has wider support and is isolated,
29:04but it does give you root access
29:06onto the system.
29:07So if you're going to run the Unix Docker
29:10container you want to make sure that
29:12it's in a secure isolated environment,
29:14not in high performance computing left.
29:17So, but both are supported.
29:18You can get both of the images docker
29:21singularity and so the container itself
29:23supports an integrated deployment framework.
29:25What we mean by that is there's a complete
29:28operating system layer in the container,
29:31so the Q Nix.
29:34Ecosystem is actually a cent OS operating
29:36system, so you get a cent OS layer.
29:38So when you go into the conex
29:41container you can have a computer.
29:43So you have a full cent OS under the hood.
29:46You have a software dependency layer,
29:49so that's free.
29:51Surfer are octave.
29:53SL human connectome pipelines.
29:54Palm and so on and so forth, right?
29:56So the software dependencies
29:58are built as a separate layer,
29:59which allows us speed and nimbleness.
30:01So if we need to rebuild a
30:03layer for deployment,
30:04we can do that pretty quickly.
30:06And then there's a kunik
30:07sweet layer in the QX.
30:09Sweet layer is very lightweight,
30:10did only has two NICs on top of it,
30:13so that we don't rebuild these
30:15first 2 layers all the time,
30:17so we can iterate inversion very quickly,
30:19and we use the semantic version and
30:21framework for both the container
30:22tagging and the software itself.
30:24So that you always know that the system.
30:28That's deployed through the
30:29container and the actual code stack.
30:31The source code in the container our version
30:34matched, so there there's no mismatch.
30:37So what's the architecture?
30:38You could?
30:39You're sort of wondering.
30:40OK, So what does this thing look like?
30:43So it really is built around
30:45a module based architecture,
30:47so when you are, if you're coding,
30:50if you've coded anything or if you use git.
30:53As a tool in your work,
30:55what when you create a repository
30:58of any kind, you can make a choice.
31:02If it's your your overall.
31:04Uh,
31:04basically super ordinate
31:06repository and then you can create
31:08submodules of that repository.
31:11This architectural choice was made such that,
31:14say, Sarah wants to create a module for
31:17Q Next specifically for deep phenotypic,
31:20computational psychiatry worker.
31:22Helen wants to come in and say,
31:25and I really want to make a new
31:29ring conomiques plugin or module.
31:31She can make it interoperable with Q Nix.
31:36Based on the software
31:38development kit that we provide.
31:41Create a pull request.
31:42We can run our acceptance test,
31:45make sure everything works and
31:47then we embed a new module from
31:49a community contributor into the
31:52architecture and this is this is a
31:54strategic choice to allow anybody
31:56and everybody to contribute to the
31:58system and so the three the four
32:01modules that really exists are.
32:03So there's a connector module
32:05and it's basically bash code,
32:07so it's a bash code base that allows us in.
32:11Interoperability without any Linux system
32:13underneath that is the Anna Utilities Module,
32:17which is Python based,
32:18and it's really some of the heart
32:22and the engine that drives the.
32:25The large scale analytic
32:27pipeline implementation.
32:28Then there's any tools which is standalone
32:31functions written for either Mac,
32:34Matlab or octave.
32:35So there are 10 files and library
32:38is precisely that it's just
32:40supportive supporting materials such
32:42as pipeline templates or various
32:45Jason files and things like that.
32:48So with this architecture,
32:50you can imagine that you can implement
32:53something pretty trivially in connector.
32:56In turn,
32:56you can then embed it in my utilities later
32:59through more stable Python code base,
33:02and then if you want to run just
33:04standalone function in same matlab or
33:06octave for some signal processing,
33:08you can just go straight and deploy
33:11tonight tools and it'll work
33:12automagically through the rest of the.
33:15The system, because it's built that way,
33:18and so it also has the ability
33:21to Mount a file system onto the
33:24container so that way you can perform
33:27IO operations on any file system,
33:31and again it organically supports
33:33the way it's built.
33:35It supports pipeline integration,
33:37so in other words.
33:38The Connector Annina Utilities code
33:41stack is built such that we can take
33:45the flagged input from a given pipeline,
33:47such as the human connectome project,
33:50and create.
33:51An API adjustment in our code such that
33:54it interoperates with that pipeline,
33:57and that's that's a really
33:59important feature of ecosystem,
34:00so that we can build in pipelines.
34:03Its execution can happen either through
34:05individual processing and analytic commands,
34:07literally at the command line,
34:09so just.
34:10You get on a terminal and you can
34:13run things that way,
34:14or you can automate them through
34:16what we call the run turnkey workflow.
34:19So the difference here is that if
34:21you have some analysis,
34:22you want a test or run and you don't
34:24yet want to automate the whole thing
34:27across the batch of processing
34:28for an entire study,
34:30you can go the individual route with
34:32the specific command parameterized.
34:33Try it out this way or that way
34:36and then once you're done you can
34:38embed it in the run
34:40turnkey workflow and have it scale.
34:42And so all this is supported by very,
34:45you know, this is this is the
34:47part that I really worry about.
34:49And sometimes my colleagues.
34:51Get mad at me because I'm so.
34:56Such a stickler about documentation,
34:58but I really think that for this
35:00to be usable to the community,
35:02we need a documentation base
35:04that is robust both in the wiki
35:06side and the command line side.
35:09So that means that you should be able
35:11to go into the wiki and Conceptually
35:14Review and get instructions and user
35:16guides on how to use this thing,
35:18and then on the command line side.
35:22When you call help or a given function,
35:24you should be able to see exactly how
35:27to run it in a unified way across
35:29the ecosystem such that there isn't
35:32a surprise so that some command
35:34works differently than another,
35:35and so on,
35:37and so the other thing that we really
35:39worked hard is to make it cloud integratable.
35:42So we have an X net system here at Yale
35:45that al is actually collaboratively
35:47helping a stress test that runs
35:49connects natively on top of it, so.
35:52Concretely,
35:52this system can take in various
35:55kinds of data in a secure fashion,
35:58and it has a storage in a
36:00computer attached to it,
36:02and then it has a specific
36:04plugin in the X net.
36:06Implementation that enables containers
36:08to run in the cloud and that container
36:11plugin is then used to deploy Q Nixon.
36:13What you're seeing is an old slide
36:15we used to call it M nap actually,
36:18and it's beta, named and renamed.
36:20It took Unix 'cause one of our
36:23colleagues couldn't pronounce it.
36:24So it it was,
36:26that's an old data name that you're saying.
36:30But basically it's plugin
36:32enabled and this is another.
36:34Kind of view on the architecture
36:36as it's evolved so it supports
36:38data flows across multiple sites.
36:41So this is this is also a very
36:43important feature for multisite
36:44imaging studies where multiple apps
36:46and their data into the ecosystem,
36:49and so again,
36:50what we've implemented at Yale is a
36:52campuswide excellent platform that's
36:54it's both in a local lab environment
36:56in a cloud based environment,
36:58and on the high performance
37:00computing systems here,
37:01and all of them run Q Necks as as a plugin.
37:06And that then a user can go into
37:08these various environments depending
37:10on their use model and deploy this
37:12as an scalable analytics solution.
37:15And in the system,
37:16the way it works actually is
37:18you need a Jason Specification,
37:21so you just tell the system what
37:24exactly you want Q Next to execute.
37:27Then you'd enable your processing dashboard,
37:29which is available in your ex set system,
37:32and then you can launch your containers
37:34using a simple launch button.
37:36You tell it what you wanted to do
37:38based on the Jason Specification
37:40it deploy at scale,
37:42and then you can get individualized
37:44outputs that are reviewing just second,
37:46and so this is this kind of soup
37:48to nuts
37:49workflow that is cloud based is
37:51already implemented in running
37:53and close collaboration with
37:54Dan Marcus is lab at wash U so.
37:57Really, an important colleague in that sense,
37:59and So what we really strive for is an
38:02integrated pipeline support framework.
38:04So if you start from dot coms, parekh,
38:06bids, HCP, Lifespan Compliant Inputs,
38:08any kind of input that we could think of,
38:11and if you start from the scanner,
38:13you should be able to generate
38:16those kinds of Inputs.
38:18The example that I want to highlight
38:20here is the how we run the HCP pipelines,
38:24so the HCP pipelines in Q Necks are
38:27enabled through these multiple steps which
38:29map onto what they call pre freesurfer.
38:32Post Re surfer.
38:33Sorry pre free post and then
38:35their volume processing,
38:37surface mapping and diffusion processing.
38:39All of these are implemented in
38:41Q Next to that Unix API's calls
38:44which we call HCP 12345 and HCPD.
38:47And so when you run them, You basically can.
38:51On top of them,
38:53enable QX processing auxiliary steps
38:55that then take the minimal pre
38:57process steps and achieve additional
39:00QC image movement correction,
39:02derived signal statistics,
39:03nuisance extraction,
39:04nuisance signal removal, task analytics.
39:06See based global data driven
39:09connectivity as well as Parcellation's.
39:11So basically this entire
39:13workflow is automated in Q Mix,
39:15so in fact you can now connect your.
39:20Scanner platform if you have a
39:22Siemens directly to your ex.
39:24Net and auto trigger your conex run
39:26and this is something that we're
39:28actually stress testing now with our
39:31colleague Chris Pittenger at Yale,
39:33who study is ongoing and one of the
39:36first active studies post kovid Here at Yale.
39:39And so we're excited to test
39:42this functionality.
39:43And so again,
39:44those the pipelines actually
39:46are fully implemented in Q.
39:48Nix natively in this multi pipeline support
39:51engine is something that can be scaled.
39:54This is a detailed slide,
39:56sorry,
39:56running back detailed slide,
39:58but it's important because it highlights
40:00one key point which is that to run Q
40:04Nix you really need only two files,
40:06just two files to get it up and running
40:09you need your batch file which is
40:12a set of parameters that tell kenix
40:15what to do in each one of these has
40:18documentation explanations in the
40:20wiki and a mapping file which enables
40:23you to have a flexible input structure.
40:25And get into the QX folder hierarchy,
40:28which is bits compliant fairly easily,
40:30so this mapping krammer.
40:31Can be as trivial is nothing if you're
40:33mapping is already organically supported,
40:36or it can be,
40:37or it can be quite complicated as you see
40:40here where the data set isn't compliant.
40:42And the batch parameter file again allows
40:45you to really deeply parameterise your run,
40:47and so now you can start to test
40:50various values in your parameters.
40:52So concretely here for instance
40:54the pre freesurfer step of the HCP
40:56pipelines has the following parameters.
40:58The post freesurfer step has
41:01the following parameters.
41:02And concretely,
41:03here where we have multiple T1 and T2
41:05images in this data set which were
41:08than declaring the mapping for it,
41:10so only these two files there grammar
41:12behind them is not trivial torn.
41:14But once you know it then then
41:16you have tremendous flexibility
41:17and what you can onboard into QX.
41:20And I promised I'd show you
41:22actually how the command works.
41:24So this is truly AQ Nix Command,
41:26a single command that can run
41:28the whole thing on the command
41:30line in a fully automated way.
41:32So what I want to highlight is this piece,
41:35which are the the steps.
41:36So this is 1 type.
41:38This is 1 specification of the command
41:40where we're basically saying We want to
41:42process all the bolds we want to take
41:45the raw data input from this location.
41:47We are data format is HCP lifespan.
41:49In this case we want to batch
41:51it using this batch file.
41:53We want to map it using this mapping file.
41:55Our working directory.
41:56Is this so the Outputs will go here we
41:59want to our project this the path for it.
42:02Maps onto this.
42:03This is a feature that matters for X,
42:06not we want to run it locally.
42:09We want to name our file
42:10outputs in a specific way.
42:12We don't want to override anything
42:13we want to work only in a single
42:15session from a single person.
42:17And this is the ID for it which matches,
42:20and then these are the steps that
42:21we want to run turnkey through.
42:24So if we don't have would
42:25never created this study ever,
42:27then we can do that create study
42:29so it will populate the entire
42:30hierarchy in this location,
42:32even if there's no folder there ever on
42:34the file system and then just continue.
42:37So this goes all the way through extraction
42:39of data driven Bolt connectivity features,
42:41and this is a function supported
42:43by a command,
42:43for instance computable DFC,
42:45which you can actually run in isolation,
42:47can just go and execute that.
42:49By typing in QX space and then this command,
42:52and then the parameters behind it.
42:55But the run turnkey engine
42:57will just do it all for you.
42:59So what is the outlook like?
43:02So the outputs are fairly
43:04reliable and scalable,
43:05so this is an example of Q Nix run
43:08that we did actually years ago on two
43:11people scan twice on two different
43:14sites here at Yale and Mount Sinai,
43:17and we ran it through the pipeline.
43:19And just to show qualitatively,
43:21just to give you a sense of what
43:23are the outputs that you get so.
43:26It supports this kind of multi site
43:28multi modal turnkey image incapacity
43:31that that I think is highly useful
43:33if you're doing clinical imaging and
43:36I think that this is the last slide
43:39in the series of slides where I want
43:41to highlight that gives you really
43:44multimodal automated QC functionality,
43:46so this is important because you
43:49are able to truly inspect the
43:51quality of your data in in.
43:54A sense where you stay very close to the.
43:58Modalities that are connected at the
44:00single subject level for that session,
44:02and inspecting those becomes really
44:04important both in terms of quantitative
44:07in visual steps in sick Unix supports this
44:10so this is actual kenix outputs for what?
44:13AT one weighted QC picture looks like Weiling
44:16bold functional connectivity and diffusion.
44:18We also actually support pet now
44:20things to Karafun to know who is
44:23research scientists in our group we
44:26don't have a QC picture for it yet.
44:28But this gives you a flavor of what you
44:31can actually get out of it once you run it.
44:34And once you execute it on your data set,
44:37so I'll stop here.
44:39In the interest of time and see if there
44:41are questions and if we can open up for
44:44discussion and without wants to jump in.
44:47So thank you so much for your attention.
44:50Thanks Alan, This is really,
44:52really exciting.
44:53I think there's a lot of lot
44:56of uses for this.
44:57We do have one question so far in
45:00the chat function, which is Cancun
45:03X access data stored in the cloud.
45:058 WS like the Abcd data set.
45:09That's a great question.
45:10So the ABC data set is stored within
45:14the national data archive awf system,
45:18and so the way that that works.
45:21So the answer is sort of yes,
45:25but not not easily.
45:27And the answer that answer is
45:30because nothing can access.
45:32The NBA data archive easily, So what?
45:37So there's a grant, actually.
45:40I think it's funded now through.
45:44Damage to the human connectome
45:46project team to make X net
45:47interrupt interoperable with NDA,
45:49and so if that were to happen,
45:51then it becomes trivial,
45:52so the other way that we could do it
45:55we haven't implemented yet is too,
45:57so we actually have an AWS plug in.
46:00Believe it or not,
46:01in Q Next so you can query a WS
46:04through to next if you want to.
46:07But you need to write a script probably,
46:10so you probably need to write
46:12a module to query a WS,
46:14get the data on some local
46:17platform and then run Q Nix.
46:19Alternatively,
46:19what you would need is a dedicated cluster
46:23that the NDA supports that runs Unix.
46:26There in some version,
46:27which then can be requested for
46:29credit and then you could run it.
46:32The former implementation already exists.
46:34You could do it the latter.
46:36I'd have to call up Greg Farber and ask him
46:40to fund a grant so that we can build that.
46:44But yeah, it's a great question.
46:45I'd love to be able to do that, yeah?
46:48Elizabeth Olsen asks this is very
46:51exciting and does it handle Mrs data?
46:55So it in principle yes.
46:57So if it doesn't have an
47:00Mrs Pipeline implemented,
47:01it's in one of the reasons why
47:04we haven't yet implemented it,
47:07because we're trying to stay.
47:10We're trying to make Unix surface
47:12based compatible,
47:13and as you know Mrs.
47:15The resolution is still evolving right?
47:17Even though you can get your
47:19voxels to be able
47:21to certain size.
47:22So the way I would handle Mrs
47:25is if you have a pipeline in
47:28analytic framework for Emerson.
47:30If you get your say.
47:32Glutamate Spectra and the then you
47:34can easily embed the Mrs measure
47:36into the QX hierarchy and regress
47:39it onto another neural measure
47:41or even map it onto a location,
47:44but it's a great question,
47:46so it's natively Q Nix doesn't
47:48have an MRI's capacity at present.
47:53Got it. Great. Other
47:56questions from the audience.
48:00But I should say if you want to
48:02write a Unix plugin will take it.
48:04Great. Peter ass disk units allow for
48:08safety and 50 files to be compared.
48:12Yes it does actually, yeah,
48:14so that's that's we wanted to
48:16explicitly make Q Nix format
48:18agnostic so it supports both,
48:19so you can get your data.
48:22It's really up to you, right?
48:24So you can stay in volume or go
48:26to the surface and vice versa.
48:28So projecting back and forth is supported
48:31by the Workbench commands that are
48:33also embedded as a dependency in Q Mix.
48:36You get the Workbench Command
48:37as a Q Next feature in there.
48:40So you can definitely do it.
48:43Great. More rain for any other
48:47questions from the attendees.
48:49Any other panelists have questions for Alan?
48:52Does Q Next fly now?
48:54It doesn't take any breakfast.
48:56I don't know can I serve you breakfast?
48:59Peter actually also has a follow up question
49:02can nifti files be used for Mylan mapping?
49:05Yes, yes they could in principle.
49:07So I do think it's it's really
49:09kind of a methodological,
49:11scientific question, right?
49:12Because you could if you
49:14have your volume Maps,
49:16T1T2 high resolution volumes.
49:17Their registration is basically
49:19high parameter nonlinear
49:20registration to the Mnay Atlas,
49:22and so in principle you could get the
49:24ratio in the volume representation and
49:27not mapped onto the surface ribbon.
49:30So you have those two images and they
49:32come out of the pipeline organically,
49:35so you can just compute.
49:37In my own map,
49:38using the same algorithm in the volume.
49:43Great. And another question asks,
49:46is it possible to use Q Necks to calculate
49:49functional connectivity entropy? Yes,
49:52so the way I would do it is so there's a.
49:57There's a command bold FC compute
50:00in which we have multiple.
50:03Derived metrics from the bold time series,
50:08so it's. Will you can stay dance?
50:12You can take all the vertices and order
50:14all the boxes if you're in volume and
50:17work with those or you can parcel ate
50:20it so we have personalization support.
50:22And then when we typically,
50:24partially because it gives us,
50:26you have more powerful data reduction.
50:29Output just literally get more statistical
50:31power when you average the signal
50:33in a functional contiguous region,
50:34but then you can get an FC matrix
50:36out of it and then that FC matrix.
50:39What analytics you do want it is completely
50:41up to you and Kunik says are embedded in it,
50:44so R is. Uh, supported dependency because
50:47we need are for certain analytics.
50:50So if you wanted to.
50:53Take your code and run entropy
50:55on that derived FC Matrix.
50:57You could.
50:58I don't know that we currently have native.
51:03FC entropy feature output explicitly
51:04supported after check, but I don't think so.
51:07I think that we have various kinds of
51:10derived metrics from that EPS Matrix,
51:12but again,
51:13if you wanted to code and entropy equation,
51:15we can easily easily embedded in
51:18that bold FC compute function.
51:20Because really quickly it sounds like
51:22we have another question I think would
51:24be really useful for everyone to hear
51:26the answer to which is from Maxwell Elliot.
51:28Can you comment on your experience
51:30with QC failures when running Q Next
51:33out of the box in a typical data set,
51:35how many subjects typically need to
51:37be run and what are common places
51:39where things might fail?
51:40That's a great question,
51:41so I think that you know I'll tell
51:44you what was the easiest and what was
51:46the hard as that is set to run and
51:49then I'll kind of compare notes so.
51:51So for instance.
51:54Be Snip was pretty easy for a multi
51:56site study and I'll tell you why in
51:58a second whereas abide it was not.
52:00So by D1 and two was was really
52:04challenging to run in period and be.
52:07Snip was easy so.
52:10Another one that was fairly easy to run was.
52:16I don't want to let's stay with those right?
52:19So so here's what happens.
52:21So if you if your data organization on
52:23the front end and the data ingestion step.
52:26So if you have a data hierarchy or data
52:28format across these studies that just
52:31isn't compliant or is inconsistent,
52:33that's going to be your biggest
52:35source of failure.
52:36So the major thing will be your dad
52:38organization and Ingestion steps so that
52:40data orchestration on the front end.
52:42So if you if you get that right,
52:45and if you really have.
52:47Your dad I prepped for kunik suggestion.
52:49So Q Nixes bids compliance.
52:51If you have bids sidecars,
52:52then it's going to work OK.
52:54So it's going to work pretty
52:56pretty automatically and fine.
52:58So assuming that you've taken care of that
53:00which Abidi wanting to our nightmare,
53:02and so that's a good example where
53:05you're going to have if you just try to
53:07run it on a downloaded a body data set.
53:11Pretty much it's going to
53:12be a miserable failure,
53:14so you have to do that.
53:16That orchestration part initially.
53:17On a data set that Spits combined or doesn't
53:20have much this harmonization across sites,
53:22it becomes a lot easier.
53:24So let's make the assumption that
53:26your data is bid side cars and that
53:28you have a T1 image and that you
53:31have some bold and some diffusion.
53:33Now if you have just that then
53:35the next rate limiting factor
53:37will be how good is your T1.
53:40If your T1 is no good and it's really
53:42low quality, then you're going to.
53:45Any pipeline will fail.
53:46So will Q next.
53:48Right, you're gonna get Fuzzy T1 images.
53:50Your segmentations are not going
53:52to look good, you're free,
53:53surfer will fail or be stuck in topology,
53:56correction forever.
53:56And then you're going to,
53:58you know it's gonna choke.
54:00So you're going to get some
54:02percent fail there.
54:03But assuming your T1 is solid so I
54:05would say 1 millimeter isotropic high
54:07quality or lower resolution than that.
54:09Then it pretty much works.
54:12All the time,
54:13we don't see any issues like the success
54:16rate is almost 100% and then really
54:19the next question is your bold modality,
54:22right?
54:22If it's the if you have legacy data
54:25set with no spin echoes collected,
54:28it'll work pretty much all the time
54:31unless your registration fails miserably
54:33to your T1 somehow and that happens
54:35rarely unless the there's some kind
54:38of real miss orientation between
54:40the nifty's or something like that,
54:42those are.
54:43We see that rarely.
54:45And and those are really the two key
54:49fail points for your data, right?
54:51Is that registration of your bold onto
54:54the T1 and your T1 being solid diffusion?
54:57Five points are pretty robust.
54:59Typically, the more angular directions
55:01you have it kind of doesn't matter.
55:04So you just want to make sure that.
55:07That that you keep track of your
55:11diffusion tables and so on but.
55:14The four for legacy data set to just
55:16in terms of how good the T ones are,
55:19I would say we see fail rates
55:21anywhere between 5%,
55:22maybe a little lower than that.
55:24Sometimes up to 10%,
55:25but it's usually a data quality issue.
55:27It's not literally stuff that data
55:29that you wouldn't want to use anyway,
55:31so if you saw it like like you don't
55:34want to touch that T1 with a 10 foot pole,
55:37so it's really that's what's going
55:38to cause your biggest failure.
55:40And then after that really,
55:41it's pretty smooth sailing because
55:43you're dealing with derived features.
55:44Well, thank you Alan.
55:46I do want to be respectful.
55:48The time it's 501 would you
55:50be comfortable with people
55:51contacting you in case they
55:53have further questions?
55:55Yeah, absolutely.
55:55Also I mentioned this briefly,
55:57but Q&X is featured on so it's Q
56:00Next at yale.edu so and we have a
56:03forum in a wiki and if you have
56:06questions about it by all means,
56:08and probably I'm going to ask
56:10how I like to retweet when we're
56:12public releasing so that we can.
56:15Update the link on the website so
56:16people can get to it, so just stay tuned.
56:19Where were resolving some final blockers
56:21before we stage the public release?
56:23Probably within the next month or two?
56:25Yeah, we can definitely do that.
56:27Excellent and I think Sarah is about
56:29to introduce our next week speaker.
56:31Yes, so great great talk.
56:33Thank you so much.
56:34It was really fun to hear about
56:36the new work and also I think
56:38I'm sure lots of people share my
56:41enthusiasm about hoping to be able
56:43to use this resource going forward.
56:45I'm excited that we're having another
56:47end three speaker in November,
56:48so John Murray will be.
56:50Joining us to talk about his work,
56:53the title will be a flexible
56:55framework for simulating an fitting
56:56generalized drift diffusion model.
56:58So for people who've been thinking
57:01about these kinds of data,
57:02that can be really helpful resource an,
57:05I'll say that that talk will be again
57:08on the 1st Thursday of the month,
57:11November 5th at 4:00 PM.
57:13We may change sometimes,
57:15but will anticipate generally
57:16running on the 1st Thursday of
57:18each month at 4:00 PM so.
57:20Hope to see people back for the talk next
57:24month and thanks again to Alan for today.
57:27Thank you everybody.
57:28Thanks for coming.
57:29See you next month.
57:32Alan Um?