Agenda: - Hippocampal Label Expansion project - Certification Platform: performance/criteria - Questionnaire - Congress presentations - Papers XV PMT meeting – Jan 30, 2013

Label Expansion - Design VariableValue rangeLevelsN Subjects x level N Hippos x level MTA ScannerSiemens, GE, Philips 3~ 4590 Magnet field strength 1.5 T, 3T DiagnosisCTRL, MCI, AD3~ 45~ 90 Age60-70, 70-80, 80+3~ 45~ 90 SidesRight, Left2135 Subjects = 135 Hippocampi = 270 4

Hippo Label Expansion USD AA (50%), Bioclinica, Brain Image Analysis, Ixico, Roche, Synarc, True Positive Medical Devices (Louis Collins, Jan 29) (50%) Laval: OK managing labels for qualification of Algorithms Fund from Can AA will better enable that, and completion of current platform/certification criteria for human tracers (Wide logistical analysis for certification criteria can be carried out by Abderazzak at Laval) Deadline: 3 months from $ receipt Tracers: 1) Masami (Jaccard=83 in the whole Certification phase) 60 images (overlap with his beta tester project for Jap VSRAD algorithm) 2) Martina 19 images Contacted: Rossana (Master; 28 img?), Dominik (Master; 28 img?)

Publication policy AVAILIABILITY OF DELIVERABLES: Needs being changed? NO for the qualification labels (they should not be delivered anyway) Maybe for the labels from the validation. Anyway, SD notes that these labels are FEW, and from few subjects: developers will have all what they need from the Expansion project,  no need to change policy on availability of labels

Publication policy AVAILIABILITY OF DELIVERABLES The Harmonized Protocol (i.e. the text description and illustrative material), the digital master tracer masks, and digital training sets (masks developed by the 5 best naïve tracers) will be available only to beta-test users in the period (about 1 year) between the end of the Delphi Panel and the publication of the full validation resultsOr ADFEQUATE QC (i.e. the analysis of the whole set of hippocampal tracings by all naïve tracers). To qualify as beta-testers, users will submit proposals to the SC; if accepted, beta-testers will be granted access to deliverables under conditions dictated by a written cooperation agreement that will be signed by the user and the Chair of the SC. Assuming mutual benefit, the cooperation agreement will include provisions related to scientific publications, donations, and feedback that beta users will provide on their exploitation of the Harmonized Protocol.

ADNI Publication policy AVAILIABILITY OF LABELS ON THE “SODIUM” WEBSITE This would soon be an issue for the Expansion Expanded labels may be back-transformed to native space: users will only need the ADNI code

Platform: Training Phase 2 Images (Scheltens=0,3) Feedback 6 Images (Scheltens=0,1,2,3,4,4) Feedback 10 Images (2 x Scheltens=0,1,2,3,4) Feedback CERTIFICATION VALIDATION TRAINING

Training Performance & Validation

Emma, round 1 (1.5T), Jaccard: 0.69 left and 0.65 right

Yawu, round 1 (1.5T), tot Jaccard: 0.71 does not distinguish amy from hippo; includes fornix

Melanie, img: bq_f_15T, total Jaccard: 0.76 She does not separate properly the indusium griseum, and sometimes includes fornix. These errors are repeated in more slices where the criterion applies. All the rest is ok or really minor.

Masami, img: bq_a_3T, total Jaccard: 0.82 Minor mistake in applying the HP criteria, or small inconsistencies in potentially ambiguous boundaries These are all of his divergence points. We can not really say that he is “not compliant” with the HP

Michel, subject bq_F_3T (Qualific) Jaccard: left 0.873, right 0.870

Michel 0,85 Masami 0,83 Marileen 0,81 Tim 0,81 Kristian 0,80 Corinna 0,80 Yawu 0,80 Travis 0,79 Enrica 0,79 Mariangela 0,78 Jaccard indices in Qualification

Reliability in III round

Certification Criteria (to be defined) Hypothesis so far (generated versus quality check) is that: Jaccard < 70 denotes major problems in hippocampal segmentation Jaccard < 80 denotes incomplete compliance with the HP criteria for normal subjects (may be admissible for atrophic subjects: Enrica, bq_r_15T, J=0.724, tracings ok) Combination of values needed (Wide statistical analysis needed. See slide 3, Abderazzak at Laval and Can AA funds for Laval)

Questionnaire How long does it take to you to segment one hippo based on the HP? Do you believe that your comprehension of the hippocampus and your way to segment has improved after the training based on HP? Which are the parts of the HP that were less clear to you? Were there regions in the Hippocampus where you did not know how to segment due to lack of instruction for that specific point or case? (if yes, please attach figures illustrating them) Questions regarding the platform as required by SD We can use SD’s Survey Monkey’s account

- Biomarkers for Brain Disorders (Cambridge, February 3-5, 2013) Definition of harmonized protocol for hippocampal segmentation - AD/PD (Florence, March 6-10, 2013) Definition of Harmonized Protocol for Hippocampal Segmentation - AAN (San Diego, March 16-23, 2013) EADC-ADNI Benchmark Images of Harmonized Hippocampal Segmentation - IEEE Medical Measurement and Applications conference (Gatineau –Quebec-, May 4-5, Platform for Training and Qualification - AIC-AAIC (Boston, July 13-18, 2013) Training of Naïve Tracers; Statistics for Certification Platform -(SINDEM (Ita Soc for Neurology of Dementias) Perugia March Definition of harmonized protocol for hippocampal segmentation Congress Presentations

Papers describing the project Survey of protocols (preliminary phase; Published, JAD 2011) Operationalization (preliminary phase; Accepted, Alzheimer’s & Dementia, MS n. ADJ-D ) Axes check short report (Brescia Team, ADJ-D ) Delphi consensus (Brescia Team, in progress) Master tracers’ practice and reliability (Brescia Team, in progr) Development of certification platform (Duchesne and coll) Validation data (Brescia Team – companion paper 1) Protocol definition (Brescia Team – companion paper 2) Validation vs pathology (TBD) DONE IN PROGRESS PLANNED

VALIDATION VS CURRENT PROTOCOLS ASSESSMENT OF SOURCES OF VARIANCE TRAINING SET DEVELOPMENT VALIDATION VS PATHOLOGY GOLD STANDARD Harmonized Protocol ADNI scans: 2 x 5 Scheltens’s atrophy score x 2 sides x 2 magnet strengths (1.5-3T) Total per rater: 40 hippos Harmonized Protocol ADNI scans: 2 sides x 5 Scheltens’s atrophy scores x 3 time points (bl-1y-2y) x 3 scanners (+ bl) x 2 magnet strengths (1.5-3T) Total per rater: 240 hippos Assessment of variance due to rater and center Local Protocol ADNI scans: 2 x 5 Scheltens’s atrophy scores x 2 sides x 2 magnet strengths (1.5-3T) Harmonized Protocol ADNI scans: 2 x 5 Scheltens’s atrophy score x 2 sides x 2 magnet strength (1.5-3T) Total per rater: 40 hippos Harmonized Protocol: Pathological datasets: Mayo Clinic and NYU Total: about 40 hippos Training ADNI scans: 10 at 1.5T x 2 sides x 7 SUs x 2 tracing rounds Total per rater: 40 hippos 20 naïve tracers5 master tracers 1 tracer REFERENCE PROBABILISTIC MASKS with 95% C.I. QUALIFICATION Best 5 naïve tracers Assessment of variance due to side, trace-retrace, atrophy, time, scanner, rater TRAINING SET Assessment of agreement with volume on pathology or ex vivo MRI and correlation with neuronal density

GANTT