Some Thoughts on Analysis Blindness Steve Brice Ex-Analysis Coordinator for MiniBooNE

When to Use a Blindness Scheme A blindness scheme blocks human biases from affecting an analysis A blindness scheme typically comes at a cost – Maybe a more complex analysis – Maybe a slower analysis Therefore….If your analysis is not at risk from human biases then don’t use a blindness scheme Blindness is a tool not a religion – Not every analysis benefits from it – If you’re going to use it then it needs to be for a good reason

541 meters downstream of target 3 meter overburden of dirt 12 meter diameter sphere Filled with 800 t of pure mineral oil (CH 2 --density 0.86, n=1.47) Fiducial volume: 450 t 1280 inner 8” phototubes-10% coverage, 240 veto phototubes (Less than 2% channels failed during run) The MiniBooNE Detector

Muons: Produced in most CC events. Usually 2 subevent or exiting. Electrons: Tag for   e CCQE signal. 1 subevent  0 s: Can form a background if one photon is weak or exits tank. In NC case, 1 subevent. Particle ID

Why Did MiniBooNE Use A Blindness Scheme? The main MiniBooNE analysis was all about using particle ID to identify electron candidates in the data This particle ID depended sensitively to the fine details of what electrons looked like in the MiniBooNE detector These details were affected by the poorly known optical model of the detector This optical model needed to be tuned using calibration and some neutrino data We wanted to avoid the situation where we were tuning the optical model whilst simultaneously picking out electron candidates in the beam data The solution – A blindness scheme that prevented us looking at electron candidates until after the detector Monte Carlo tuning was complete

The MiniBooNE Blindness Scheme Initially all beam events were held blind People made proposals to open up certain parts of the data (e.g. events with a Michel electron) – There proposals were scrutinized to make sure they could yield at most a <1  indication of a signal – No proof was needed that the sample was useful – just that it had essentially no signal In addition certain (not PID useful) information was available from all events

MiniBooNE Blindness progression As time went on more and more samples (“boxes”) were opened and available for study Eventually the MC tuning was complete enough that a conservative cut could be used to open up everything and keep only loose electron candidates in the box. Soon thereafter the analysis was complete and this final box was opened.

Notes on MiniBooNE Blindness The scheme worked because everyone analyzed the data using the same analysis framework – Furthermore, the fully reconstructed, open datasets were automatically produced and made available to people Circumventing the scheme was possible, but couldn’t be done accidentally. – We trusted our collaborators The scheme was labor intensive, bulky, restrictive, annoying, …. … but absolutely vital to producing a trustworthy result