Introduction to NeuroStack

Transcript

• 00:00NeuroStack is an AWS CloudFormation-based
• 00:03tool designed to aid neuroimaging researchers
• 00:05working with AWS cloud computing.
• 00:08It is ideal for neuroimaging researchers
• 00:10transitioning their workflows to AWS or
• 00:13anyone working with neuroimaging at scale.
• 00:15By installing NeuroStack
• 00:16into your AWS account,
• 00:18you construct a pipeline of AWS
• 00:20resources specialized for neuroimaging.
• 00:23NeuroStack is built with the NITRC
• 00:25computational environment, or NITRC-CE,
• 00:27allowing users access to all of the
• 00:29programs pre-installed in the NITRC-CE.
• 00:32In this tutorial, we'll go over how to
• 00:35install NeuroStack into your AWS account,
• 00:37and two examples of using NeuroStack
• 00:39using two different software packages,
• 00:41AFNI and FreeSurfer.
• 00:43To begin using NeuroStack,
• 00:45first sign into your AWS account.
• 00:48Navigate to the NeuroStack page on
• 00:51the NITRC website and go to Downloads.
• 00:53The download is a URL that will bring you
• 00:56to the AWS CloudFormation stack page.
• 01:02Once on the Stack Creation page,
• 01:04everything you need is already preset.
• 01:06Just continue choosing next.
• 01:11On the final page, check the box to
• 01:14accept that CloudFormation will create
• 01:16IAM resources before creating the stack.
• 01:19NeuroStack will take several minutes
• 01:21to set up your AWS resources.
• 01:23When it says "Create Complete", it's finished.
• 01:27Let's run a simple job through
• 01:29NeuroStack to see how it works.
• 01:30We're going to convert an MRI file
• 01:32from one file type to another.
• 01:33We're going to use an AFNI program to convert
• 01:36from a NIFTI filetype to an AFNI filetype.
• 01:39The job should only take a few
• 01:40minutes to run, so you can see the
• 01:42process from beginning to end.
• 01:43I've downloaded the template
• 01:45script and a test imaging file from
• 01:47the NeuroStack GitHub Repo.
• 01:48I'll navigate to the AWS S3 console where I
• 01:51can see that NeuroStack has configured 3 S3
• 01:53storage buckets and input,
• 01:55output, and script buckets.
• 02:00I'll upload the script to
• 02:02the NeuroStack script S3 bucket.
• 02:09Let's take a look at the script.
• 02:11The top section contains commands that
• 02:13aren't intended to be altered by the user.
• 02:15The bottom section is meant
• 02:16to be modified by the user.
• 02:18There are three basic parts to the section:
• 02:21copying data from our input S3 bucket,
• 02:24performing an operation on the data,
• 02:26in this case using an AFNI command
• 02:28to convert file types, and copying our
• 02:30processed data to our output S3 bucket.
• 02:33An AFNI file consists of two
• 02:35files a .BRICK and a .HEAD.
• 02:36So we're actually copying 2 files here.
• 02:40The variable aSub is defined as the
• 02:42filename before the first period.
• 02:44So in this example we have
• 02:47a test file called test,
• 02:49which means that aSub
• 02:50will take the value "test".
• 02:54I'll upload the test imaging file to
• 02:56the NeuroStack input bucket. When I
• 02:58upload a file into the input bucket,
• 03:00it will immediately begin processing the file
• 03:02according to my script in the script bucket.
• 03:05It will take several minutes
• 03:07for the job to complete.
• 03:08When it has finished successfully,
• 03:09we will see our processed data in
• 03:11the NeuroStack output S3 bucket.
• 03:14In the meantime,
• 03:15let's watch our job status.
• 03:16We can do that by navigating to
• 03:18the AWS batch console.
• 03:19The job will move from "Submitted" to
• 03:22"Running". When it has finished running,
• 03:24it will either move to "Succeeded" or "Failed".
• 03:35Great, we can see that our job succeeded.
• 03:38We can see more details about
• 03:39our job by clicking on it.
• 03:41And we can see that the job's total
• 03:43running time was 12 seconds,
• 03:44which is the computation
• 03:45time that we're charged for.
• 03:47We can also look at the log of
• 03:49our job under Log Stream Name.
• 03:57And when we navigate to our output
• 03:59S3 bucket, we can see that our
• 04:00two processed files are there.
• 04:04In this example,
• 04:05we've uploaded a single subject file,
• 04:07but we can upload multiple files at
• 04:09a time for simultaneous processing
• 04:11by uploading the files either
• 04:13manually or through the AWS CLI.
• 04:15If you have many files to process,
• 04:17the CLI will be more efficient.
• 04:20Let's use NeuroStack to process
• 04:22the subject through FreeSurfer.
• 04:23To do this, we'll need to change our
• 04:26script in the user modified section.
• 04:28I have the FreeSurfer setup commands.
• 04:30I copied the data from our input S3 bucket.
• 04:33Freesurfer requires a license file to run,
• 04:36so I'll copy my license file as well.
• 04:39I'll then do recon-all, the command
• 04:41to process through FreeSurfer, and once
• 04:43that process is done I'll copy my
• 04:46output files to my output S3 storage.
• 04:48Now that we've modified our script,
• 04:50we can upload our script and the
• 04:52FreeSurfer license file to the
• 04:54script bucket and our test data to
• 04:56our input bucket. Again, by uploading
• 04:58to our input bucket,
• 04:59we started our process running.
• 05:02FreeSurfer will take several hours to run.
• 05:11It's the next day and I can see our
• 05:14FreeSurfer output in the output S3 bucket.
• 05:16I can go to the Batch dashboard and see
• 05:18that FreeSurfer took six hours to run.
• 05:21We hope you will find NeuroStack useful
• 05:23for your neuroimaging work on AWS.
• 05:25If you have questions about
• 05:26how to use NeuroStack,
• 05:27please visit the NeuroStack Forum
• 05:29on the NITRC website. Thank you.