Deep Project Management with XNAT Vanderbilt University 2013-08-20 Benjamin Yvernault, Bennett Landman, Brian Boyd,

Requirements At this point, you should know how to : Register and Log in to XNAT See your projects and join projects Create project / sharing data Manage access permissions Browse data and view image previews Search for projects, subjects, and data Download imaging data, processed, and stored meta-data Delete scans / sessions / projects Use the Pre-Archive Upload scans through DicomBrowser Quality Assurance

Set up your computer Xnatsetup (basic) will on your computer : Install Python package to run the command tools : Instructions on the wiki Install lxml and httplib2 using easy_install or pip Install Pyxnat from a git version : git clone git://github.com/sburns/PyCap.git PyCap cd PyCap python setup.py install Set up your .bashrc or .profile : Set up the XNAT variables : export XNAT_USER = bob (your xnat username) export XNAT_PASS = bob (your xnat password) export XNAT_HOST = http://xnat.vanderbilt.edu/xnat Add the tools to PATH : export PATH=/PathToMasimatlab/trunk/xnatspiders/Xnat_tools:$PATH Remember! Xnatsetup

After this short course, you will be able to: Get our code from Masimatlab Set up your computer (python packages, …) Download data from XNAT using command line tools Upload data using XNAT command tools Dcm4chee to send big DICOM database to XNAT Use XNAT command line tools Project mirroring XNAT->XNAT / full

Download from XNAT You can download using what you learn from the previous course : Download file Manage file You can use the Xnatdownload commandline tool developed by our lab that downloads resources from scans or/and assessors.

Xnatdownload Python commandline tool developed by our lab Present in the masimatlab svn repository: trunk/xnatspiders/Xnat_tools/Xnatdownload Different options to precise your download Use: Xnatdownload -h to see the options

XNAT view

Upload Data to XNAT DICOM can be uploaded using: DicomBrowser (previous course) dcm4che (next slides) Xnatupload is a commandline tool developed by our lab to upload any resources to XNAT using three methods.

Dcm4che (1) Open Source clinical Image and object management Programed in Java Tutorial on how to upload : http://xnat.vanderbilt.edu in Advanced resources. Join Masimatlab on NITRC to obtain the code for Dicom4che Dcmsnd present in the masimatlab/trunk/xnatspiders/dcm4che_tools/dcm4che- 2.0.27/bin/ E.G: dcmsnd VandyXNAT@xnat.vanderbilt.edu:8104 /FullPathTemporatoryFolder/*

XNAT reading DICOM XNAT provides a DicomServer: Remember! XNAT provides a DicomServer: Read information from the header Add the Dicom to the PreArchive with the right project/subject/session XNAT uses tags from DICOM to identify where it should be: Usually tag (0010,4000) Patient Comments Need to be: “Project:ProjectIDXNAT;Subject:SubjectIDXNAT;Session:SessionIDXNAT” E.G: Project:VUSTP;Subject:VUSTP4;Session:VUSTP4a Check this link to know the tags : http://mindhive.mit.edu/node/1352

Dcm4che (2) Use an IDE for java developers like Eclipse Create a new project and configure the java build path to add all the externals JARs and Class folder from masimatlab Change the SNARLdcmEdit.java file that will change in all of your DICOM the tag of identification for XNAT (0010,4000) Follow the steps in http://xnat.vanderbilt.edu “Sending DICOM to Vanderbilt XNAT” link Send the DICOMs to XNAT using dcmsnd dcmsnd VandyXNAT@xnat.vanderbilt.edy:8104 /path/*

Xnatupload Python commandline tool developed by our lab Present in the masimatlab svn repository: trunk/xnatspiders/Xnat_tools/Xnatupload It can upload any resources (not only DICOM) Three methods in the option: Files in the same folder, using specific names Files spread into a specific tree hierarchy Upload only on existing subjects/sessions/scans/assessors

First Method of Upload This method uploads only resources for scans. Options : --up1 Conditions: all the file are in the same folder Name the files like : Project-x-subject-x-experiment-x-scan-x-scantype-x-series_description-x- resource.extension E.G: VUSTP-x-VUSTP5-x-VUSTP5a-x-301-x-T1-x-3W_T1-x-NIFTI.nii.gz The project needs to exist first on XNAT You need to select a type of session (MR/CT/PET) Call the script : Xnatupload -p VUSTP -d /tmp/vustp_data/ --up1 --sess MR

Second Method of Upload This method uploads only resources for scans. Options : --up2 Conditions: One Folder with a specific structure The structure needs to be : Folder/subject/experiment/scan-x-scantype-x-series_description/resource/file.extension E.G: MainFolder/VUSTP5/VUSTP5a/301-x-T1-x-3W_T1/NIFTI/file.nii.gz The project needs to exist first on XNAT You need to select a type of session (MR/CT/PET) Call the script : Xnatupload -p VUSTP -d /tmp/vustp_data/ --up2 --sess MR

Third Method of Upload This method uploads only (does not create any subject/experiment/scan or assessor on XNAT) Options : -o/--onlyUpload Conditions: One Folder with all the files in it (multi project) The files needs to have a name like : Project-x-subject-x-experiment-x-scan-x-resource.extension E.G: VUSTP-x-VUSTP5-x-VUSTP5a-x-301-x-NIFTI.nii.gz Use the option -T to specify the kind of data (scans or assessors) The location needs to exist on XNAT Call the script : Xnatupload -p VUSTP -d /tmp/vustp_data/ --only -T 1

XNAT view

XNAT command tools Xnatdownload and Xnatupload (already presented) Xnatreport Creates report on project(s) from XNAT Xnatquery Queries and displays XNAT at any level Xnatcheck Checks if a resource exists on specific assessors or scans Checks the status of assessors XnatSwitchProcessStatus Xnatsetup

Xnatreport Use Xnatreport -h to see the options E.G: Xnatreport –p VUSTP

Xnatquery Help: Xnatquery –h Option --info for a report more focus on the processes. E.G: Xnatquery –p VUSTP --info Help: Xnatquery –h Xnatquery will display on the terminal all the objects at the level you asked.

Xnatcheck Xnatcheck allows you to check the existence of a resource on a project Help: Xnatcheck -h

XNAT view

XnatSwitchProcessStatus The status on an assessor gives information about the job status of this process. Our system use this status to launch jobs. XnatSwitchProcessStatus allows you to change the status of several assessors at a time. Help: XnatSwitchProcessStatus -h

XNAT view

Xnatsetup Command tool to Setup your computer Different setups: Basic : Install python packages: httplib2 / lxml / pyxnat Set up your .bashrc or .profile Advance: Set up path for spiders Redcap: Install python packages: PyCap and Pandas Some specific spiders set up to check if you have everything to run them.

xnat_copy Mirrors a Project from one XNAT instance to another Copies files & attributes for all Subjects/Sessions/Scans xnat_copy [options] project -cf recheck existing files -ca recheck existing attributes

Interactive Assignment Create a NITRC account and join the Masimatlab team to get the last version of masimatlab svn repository. Download the code from masimatlab on NITRC Set up your computer (manually or with Xnatsetup): Download all the NIFTI from VUSTP using Xnatdownload Upload the VUSTP data into a new project using Xnatupload and one of the method Check if a fMRIQA process in VUSTP doesn’t have a PDF Xnatquery --info on VUSTP

Next time Create a project on REDCap and ask for API What is a spider: source code Which spiders already exist? Run spiders from your computer ACCRE: run spiders (status / failed / ….)