Event Classification & Background Rejection

Slides:



Advertisements
Similar presentations
Bill Atwood, Dec GLAST 1 GLAST Energy or Humpty-Dumpty’s Revenge A Statement of the Problem Divide and Conquer strategy Easiest: Leakage (Depth)
Advertisements

Bill Atwood, August, 2003 GLAST 1 Covariance & GLAST Agenda Review of Covariance Application to GLAST Kalman Covariance Present Status.
Bill Atwood, SCIPP/UCSC, May 8, 2006 GLAST 1 Partial validation of v9r3 - Bkg. Rej. Comparison 2000 sec. of backgrounds were run using v9r3 (heros: Tom.
Assessment and Proposal W. B. Atwood UCSC, April 6, 2005.
Bill Atwood, SCIPP/UCSC, Apr. 3, 2006 GLAST 1 Pass 4 and Preparing for the LAT Handoff to NASA Why we can't go with the DC2 Analysis (Pass 3) 1) Background.
Bill Atwood, July, 2003 GLAST 1 A GLAST Analysis Agenda Overarching Approach & Strategy Flattening Analysis Variables Classification Tree Primer Sorting.
Bill Atwood, SCIPP/UCSC, Jan. 30, 2006 GLAST 1 Closer to Reality From Last Talk ( 7-Jan-06) – things seemed Great! However... 1) In estimating the SRD.
Bill Atwood, SCIPP/UCSC, Oct, 2005 GLAST 1 DC2 Discussion – What Next? 1)Alternatives - Bill's IM analysis - ???? 2)Backing the IM Analysis into Gleam.
February 2004GLAST - DC1 Closeout Meeting GRB Detection & spectral analysis in DC1 Data Nicola Omodei Francesco Longo, Monica Brigida INFN Pisa.
T. Burnett: IRF status 30-Jan-061 DC2 IRF Status Meeting agenda, references at:
Analysis Meeting 31Oct 05T. Burnett1 Classification, etc. status at UW Implement “nested trees” Generate “boosted” trees for all of the Atwood categories.
Hardware Failure Impacts Exercises S. Ritz. Issues Recent results from design reliability analysis (system engineering, Thurston et al.) Probabilities.
Bill Atwood, Core Meeting, 9-Oct GLAS T 1 Finding and Fitting A Recast of Traditional GLAST Finding: Combo A Recast of the Kalman Filter Setting.
GLAST LAT Project 1S. Ritz Discussion: To Where From Here? DC1 Closeout Meeting February 13, 2004 S. Ritz Gamma-ray Large Area Space Telescope.
Bill Atwood, SCIPP/UCSC, Dec., 2003 GLAST 1 DC1 Instrument Response Agenda Where are we? V3R3P7 Classification Trees Covariance Scaled PSF Pair Energies.
GLAST Science Support Center February 12, 2004 DC1 Closeout Detecting Gamma-Ray Bursts in the DC1 Data David Band (GSSC)
Bill Atwood, SCIPP/UCSC, August, 2005 GLAST 1 Energy Method Selections Data Set: All Gamma (GR-HEAD1.615) Parametric Tracker Last Layer Profile 4 Methods.
Bill Atwood, SCIPP/UCSC, Dec., 2003 GLAST ALL GAMMAs using the New Recons First runs of ALL GAMMA – from glast-ts and SLAC Unix farm glast-ts sample: no.
Onboard Filter Update Performance after updated cuts David Wren 26 January 2004.
Bill Atwood, Rome, Sept, 2003 GLAST 1 Instrument Response Studies Agenda Overarching Approach & Strategy Classification Trees Sorting out Energies PSF.
Surplus Hits Ratio, etc. Leon R. C&A Meeting &
GLAST LAT Project IA Workshop 4 July 15, 2005 E. do Couto e Silva 1/7 Overview of Data Taking Anders W. Borgland Science Verification, Analysis and Calibrations.
Bill Atwood, Nov. 2002GLAST 1 Classification PSF Analysis A New Analysis Tool: Insightful Miner Classification Trees From Cuts Classification Trees: Recasting.
Bill Atwood, SCIPP/UCSC, Jan. 30, 2006 GLAST 1 Post Processing & Pass4 The time for DC2 is drawing to a close. However we're still only part ways up the.
1 Using Insightful Miner Trees for Glast Analysis Toby Burnett Analysis Meeting 2 June 2003.
Bill Atwood, SCIPP/UCSC, DC2 Closeout, May 31, 2006 GLAST 1 Bill Atwood & Friends.
Data-based background predictions using forward events Victor Pavlunin and David Stuart University of California Santa Barbara July 10, 2008.
T. Burnett: IRF status 5-feb-061 DC2 IRF Status Meeting agenda, references at:
GLAST LAT ProjectIA Workshop 6 – Feb27,2006 S. Ritz Backgrounds and Trigger Rates 1 Backgrounds and Trigger Rates on Orbit S. Ritz with many contributions.
Classification and Regression Trees for Glast Analysis: to IM or not to IM? Toby Burnett Data Challenge Meeting 15 June 2003.
Bill Atwood, SCIPP/UCSC, Oct, 2005 GLAST 1 Back Ground Rejection Status for DC2 a la Ins. Miner Analysis.
Bill Atwood, SCIPP/UCSC, Jan., 2006 GLAST 1 The 3 rd Pass Back Rejection Analysis using V7R3P4 (repo) 1)Training sample: Bkg: Runs (SLAC) v7r3p4.
Analysis meeting 11July05 - T. Burnett 1 Status of the Background Review DC2 prep meeting: Switch to CRflux package by Mizuno for charged particles, Earth.
Tracker Reconstruction SoftwarePerformance Review, Oct 16, 2002 Summary of Core “Performance Review” for TkrRecon How do we know the Tracking is working?
Ensemble Learning (2), Tree and Forest
R.Dubois Sim/Recon Overview 1/10 GLAST LAT ProjectSoftware Workshop July, SLAC Sim/Recon Overview Elements of Simulation/Reconstruction Development,
DC2 at GSFC - 28 Jun 05 - T. Burnett 1 DC2 C++ decision trees Toby Burnett Frank Golf Quick review of classification (or decision) trees Training and testing.
Pass 6: GR V13R9 Muons, All Gammas, & Backgrounds Muons, All Gammas, & Backgrounds Data Sets: Muons: allMuon-GR-v13r9 (14- Jan-2008) AG: allGamma-GR-v13r9.
A statistical test for point source searches - Aart Heijboer - AWG - Cern june 2002 A statistical test for point source searches Aart Heijboer contents:
Training of Boosted DecisionTrees Helge Voss (MPI–K, Heidelberg) MVA Workshop, CERN, July 10, 2009.
Bill Atwood, SCIPP/UCSC, May, 2005 GLAST 1 A Parametric Energy Recon for GLAST A 3 rd attempt at Energy Reconstruction Keep in mind: 1)The large phase-space.
Feb. 7, 2007First GLAST symposium1 Measuring the PSF and the energy resolution with the GLAST-LAT Calibration Unit Ph. Bruel on behalf of the beam test.
GLAST/LAT analysis group 31Jan05 - T. Burnett 1 Creating Decision Trees for GLAST analysis: A new C++-based procedure Toby Burnett Frank Golf.
Bill Atwood, 6-Feb-2008 Pass 6: GR V13R9 Muons, All Gammas, & Backgrounds Muons, All Gammas, & Backgrounds Overview Overview Data Sets: Muons: allMuon-GR-v13r9.
Feb. 3, 2007IFC meeting1 Beam test report Ph. Bruel on behalf of the beam test working group Gamma-ray Large Area Space Telescope.
Gamma-ray Large Area Space Telescope -France -Germany -Italy -Japan -Sweden -USA Energy Range 10 keV-300 GeV. GLAST : - An imaging gamma-ray telescope.
Tree and Forest Classification and Regression Tree Bagging of trees Boosting trees Random Forest.
Background Rejection Activities in Italy
Background Rejection Prep Work
Measuring polarization at gamma-rays with Fermi
Time Independent Analysis
Comparison of GAMMA-400 and Fermi-LAT telescopes
Gamma-ray Large Area Space Telescope
GLAST Large Area Telescope:
Optimizing Foam, A Monte Carlo Event Generator
Data Challenge II Logo by Stefano Ciprini.
Week 5 Lecture 2 Chapter 8. Regression Wisdom.
Robert Johnson W. Atwood, M. Ziegler, B. Baughman U.C. Santa Cruz and
DC1 Energy Evaluation.
Point Sources Jacob Feintzeig WIPAC − May 21, 2014
Fun with DC1: GRB Triggers, Localizations, Etc.
A First DC2 Analysis with New Recons
Calibration and Analysis Methods Group Report
Testing some CAL specs Level 3 CAL requirement 5.5.5:
Imaging crystals with TKR
Tower A: A First Look Finding the Data: Login: THIS ONE! user: glast
Searching for photons in the LAT
CR Photon Working Group
Studies of the Time over Threshold
Overview of Beam Test Benoit, Ronaldo and Eduardo Nov 8 , 2005 thanks to Steve and Bill for helping to consolidate all the information.
Presentation transcript:

Event Classification & Background Rejection Bill Atwood & Friends

Recap Since DC1 DC1 Closeout February 2004 Begin Recon-Rewrite, May 2004 First up: TkrRecon Participants: Tracy Usher, Leon Rochester, Johann Cohen-Tanugi, WBA Next up: CalRecon Participants: Tracy Usher, Philippe Breul, Pol d'Avezac, Fred Piron, Eric Nuss, David Chamont, WBA Next up: AcdRecon Participants: Heather Kelly, Eric Charles, WBA Development of Alternative High Level Analysis Toby Burnett & Riccardo Rando et al Begin DC2 Analysis, May 2005 First Energy Analysis

Energy Methods Method % Computed % Best Est. Parametric 100 63.6 Profile 49.9 24.7 Last Layer 23.4 5.5 Tracker 16.5 6.3 4 Methods* 3 Only cover a part of Glast Phase Space *Pol d'Avezac has combined Tracker & Last Layer Parametric Tracker Last Layer Profile Only Parametric Available: 37.7% This tends to be the Local Land Fill (City Dump!) Unfortunately there are too many events here to simply throw out. Compare each Method against a "standard" defining: Energy Resolution Model Select Method giving the Highest CT Prob. Data Set: All Gamma (GR-HEAD1.615) sModel = .02+.6/(McLogEnergy)2.5 + .005*(McLogEnergy-2.)2

E4: Unbalanced CTs - 2s on s=.05+.72/(log(E))3 Data Set: AG1617-mod

How to Tell What You Have: In an analysis you may wish to check which energy method your "signal" is relying on. In the IM environment this is captured by a set of CATAGORICAL Variables and life is easy... however this was not captured in the translation to GLEAM. Here's how to tell: Compare: CTBParamProb CTBProfileProb CTBLastLayerProb CTBTrackerProb CTBBestEnergyProb The one which matches is the method used for the event in question. CTBBestEnergy, CTBLogEnergy, and CTBBestEnergyProb summarize the final results

PSF Analysis IM "glue" to put events bact together and produce Split according to Thick and Thin Layers: Tkr1FirstLayer 2-5 (Thick) or 6-17 (Thin) IM "glue" to put events bact together and produce CTBBestXDir,YDir,ZDir and CTBCORE (the "good" PSF probability variable) CTs: Of Events with Valid VTX choose to use 1Tkr or VTX Solution: CTBVTX Split according to VTX Solution (VtxAngle > 0 && CTBVTX > .5 CTs: PSF Image Sharpening Produces CTBCORE Variable

PSF Analysis Results What CTBCOR Does On Axis can vary PSF by ~ 30% (at the expense of Aeff) The 95/68 Ratio improves significantly cos(q) < -.95 cos(q) < -.2

How to Tell What You Have: To tell which analysis branch you have - THICK or THIN? THICK == Tkr1FirstLayer < 5 THIN == !THICK Note: Tracks can not originate in Layers 0 and 1 - VTX or 1TKR ? VTX == VtxAngle > 0. && CTBVTX > .5 1TKR == !VTX Note: We could have used VtxStatus bits instead of VtxAngle CTBBestXDir,YDir,ZDir, CTBCORE summarize the results THIN Layers THICK Layers Cuts: CTBGAM > .5 && CTBBestZDir < -.95 && CTBBestEnergyProb > .35

Back Ground Rejection Evolution - Use of only Classification Trees not adequate (Typ. Rejection Power 10 -40) - Improve Input to CTs via "typical" HEP Style cuts (Implemented as PreFilters) - Divide sample according type 1) Topology (VTX – 1Tkr) 2) Obs. Cal. Energy (use 6 ~log(E) bins) 3) Location (Lowest energy only) (Gives 14 Separate catagories) What's left is what CTs are poor at finding or Irreducible (start here) Results in a g Bkg. Rejection Prob. CTBGAM Class 3

First: Irreducible Back Grounds Correlated Uncorrelated e+ Blanket Conversion Proton - Blanket Interaction > 60% Remaining background are from gs produced locally They are Irreducible

Second: Weakness in CT Analysis Subsequent Search for "Hot Spots" in GLAST Phase space revealed - Correlated Events in the Top Elimination of Events which project back to Ribbons Cut Summary Bkg: 964 - 4.7% All Gam: 3046 - 2.9% Top Ribbons More Ribbons Holes at the Top of Side Ribbons

Correlations among Variables Heavy Ion Filter CTBBestEnergy > 1000 & ((CalTransRms - 1.5*Tkr1ToTTrAve) < 5) & CTBGAM > .5 What's Eliminated Cut Summary Bkg: 43 - .22% All Gam: 14 - .014% Scrambled Tracks Filter (Tkr1FirstLayer - Tkr2FirstLayer) <= 0 & Tkr2FirstLayer > 2 & Tkr2TkrHDoca > 10 & (CTBGAM+.16*CTBBestLogEnergy) < 1.32 Cut Summary Bkg: 1284 - 6.6% All Gam: 435 - .42%

These Post-Processing Cuts can be found at: https://confluence.slac.stanford.edu/display/DC2/Trigger+and+Filter+settings+and+Event+classes This leaves an Aeff vs Energy with features (not good...). Smooth Response by "boosting" low energy CTBGAM : CTBGAM = min(1., CTBGAM*min(2.0, (3.5/min(CTBBestLogEnergy, 3.5))^2)) Base Class 3 – No Post-Processing Filters CTBGAM > .5 Base Class 3 – Post-Processing Filters & CTBGAM Boost at Low Energy CTBGAM > .5

Class 3 Class 3 Post Process Cuts + Careful: Order of Plots different Top to Bottom

Demonstrations with DC2 Data from Galactic Anti Center Region (thanks to Julie) Ooppsss - Almost forgot – The Limb g issue.... Rocked Backgrounds What's cut out: CTBGAM > .5 2216 evts Evts. in Common 277 evts Evts. Missed by BestZDir > -.3 127 evts Julie's Cut Bill's Original Cut Cut Used + ZDir < -.3 Evts. Missed by ZenithTheta > 100 32 evts. Residual Bkg. Evts. 436 Total Evts.

On with the show... Galactic Anti-Center: Class A – No Post Proc. Cuts

Class A Events + Post-Proc. Cuts

What was killed by Post-Proc. Cuts The Fog of Back Ground...

Image Resolution The Geminga Pulsar Low Res. Good Res. Best Res.

Estimating Residual Backgrounds The ~20k Background events remaining after PreFilters As with previous experiments – look for orbit associated modulation in signal. Use this as a measure of remaining contamination. Seems to be ~ 10% Need to "fold" all the orbits on top of each other – Hard due to multiple components and SAA (holes). Steve's suggestion: Profile residuals events vs the Trigger Rate

Summary DC2 Background Rejection gets us close... however There is no factor of 2 left – the Irreducible Component Dominates Background Rejections is a "work-in-progress" We just ran out of time. We have made 3 complete passes so far and a Pass 4 is in the works. Most Analysis are not that sensitive to Background Contamination Thankfully residual backgrounds are essentially spread ~ uniformally across the sky. As such contamination in a few sq. degrees is small (part in 10-4) Work will continue and perhaps a "reprocessed" DC2 Dataset will appear prior to the DC2 Closeout (similar to real life) The instrument "hand-off" is approaching and we need to meet the Science Requirements by then.