G4ConvergenceTester and ExN03Con KOI, Tatsumi SLAC National Accelerator Laboratory 1Geant4 Collaboration Workshop

"G4ConvergenceTester“ and “ExN03Con” G4ConvergenceTester provides several information assisting user understanding of convergence level of his/her result, like MCNPs. “ExN03Con” is an example of the usage of G4ConvergenceTester. – examples/extended/analysis/N03Con/ They have been included in Geant4 since v9.0 (2007-Jun) 2Geant4 Collaboration Workshop

How to Use #include "G4ConvergenceTester.hh“ void ExN03RunAction::BeginOfRunAction(const G4Run* aRun) {,,,,,, Eabs_tally = new G4ConvergenceTester(); Egap_tally = new G4ConvergenceTester(); Labs_tally = new G4ConvergenceTester(); Lgap_tally = new G4ConvergenceTester(); } void ExN03RunAction::fillPerEvent(G4double EAbs, G4double EGap, G4double LAbs, G4double LGap) {,,,,, Eabs_tally->AddScore( EAbs ); Egap_tally->AddScore( EGap ); Labs_tally->AddScore( LAbs ); Lgap_tally->AddScore( LGap ); } void ExN03RunAction::EndOfRunAction(const G4Run* aRun) {,,,,, Eabs_tally->ShowResult(); Eabs_tally->ShowHistory(); Egap_tally->ShowResult(); Egap_tally->ShowHistory(); Labs_tally->ShowResult(); Labs_tally->ShowHistory(); Lgap_tally->ShowResult(); Lgap_tally->ShowHistory(); } 3Geant4 Collaboration Workshop

Output ShowResult() values and logical tests EFFICIENCY = MEAN = VAR = SD = R = SHIFT = VOV = FOM = THE LARGEST SCORE = and it happend at 1425th event Affected Mean = and its ratio to orignal is Affected VAR = and its ratio to orignal is Affected R = and its ratio to orignal is Affected SHIFT = and its ratio to orignal is Affected FOM = and its ratio to orignal is 1 MEAN distribution is not RANDOM r follows 1/std::sqrt(N) r is monotonically decrease 1 r is less than 0.1. r = VOV follows 1/std::sqrt(N) VOV is monotonically decrease 1 FOM distribution is not RANDOM SLOPE is large enough This result passes 6 / 8 Convergence Test. 4Geant4 Collaboration Workshop

Output ShowHistory() trend i/16 till_ith mean var sd r vov fom shift e r2eff r2int e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e e-06 5Geant4 Collaboration Workshop

Following slides are my presentation at 2005 Geant4 Collaboration workshop 6Geant4 Collaboration Workshop

A general assistant tool for the checking results from Monte Carlo simulations Koi, Tatsumi SLAC/SCCS 7Geant4 Collaboration Workshop

Contents Motivation Precision and Accuracy Central Limit Theorem Testing Method Current Status of Development Summary 8Geant4 Collaboration Workshop

Motivation After a Monte Carlo simulation, we get an answer. However how to estimate quality of the answer. What we must remember is Large number of history does not valid result of simulation. Small Relative Error does not valid result of simulation 9Geant4 Collaboration Workshop

Motivation (Cont.) To provide “ statistical information to assist establishing valid confidence intervals for Monte Carlo results ” for users, something like MCNPs did. 10Geant4 Collaboration Workshop

Subject of this study Precision of the Monte Carlo simulation Accuracy of the result is NOT a subject of this study At first we have to define Precision and Accuracy of simulations 11Geant4 Collaboration Workshop

True Value Mote Carlo Results Accuracy Precision Precision and Accuracy Precision: Uncertainty caused by statistical fluctuation Accuracy: Difference between expected value and true physical quantity. 12Geant4 Collaboration Workshop

Subject of this study (Cont.) Precision of the Monte Carlo simulation is subject of this study. To state accuracy of simulations, we should consider details of simulation, i.e., uncertainties of physical data, modeling of physical processes, variance reduction techniques and so on. To make a generalized tool, we have to limit subjects only for precision. Accuracy is a subject for most of presentations in this workshop. 13Geant4 Collaboration Workshop

Principal of this study is Central Limit Theorem 14Geant4 Collaboration Workshop

Central Limit Theorem Every data which are influenced by many small and unrelated random effects has normally distribution. The estimated mean will appear to be sampled from normal distribution with a KNOWN standard deviation when N approaches infinity. 15Geant4 Collaboration Workshop

Central Limit Theorem (Cont.) Therefore, We have to check that N have approached infinity in the sense of the CLT, or not. This corresponds to the checking the complete sampling of interested phase space has occurred, or not. 16Geant4 Collaboration Workshop

This is not a simple static test but check of results from nature of Monte Carlo simulations 17Geant4 Collaboration Workshop

Checking Values Mean Variance and Standard Deviation Relative error Variance of Variance 18Geant4 Collaboration Workshop

Checking Values (Cont.) Figure of Merit Scoring Efficiency R intrinsic and R efficiency Shift SLOPE Fit to the Largest history scores 19Geant4 Collaboration Workshop

What we check? Behavior of MEAN Values of R Time profile of R Values of VOV Time profile of VOV Time profile of FOM Behavior of FOM Value of SLOPE Value of SHIFT Effect of the largest history score occurs on the next history. –MEAN –R (R intrinsic and R efficiency) –VOV –FOM –SHIFT Boolean Answer Numeric Answer 20Geant4 Collaboration Workshop

Of cause, Boolean check is carried out mathematically (statistically) value behavior time profile 21Geant4 Collaboration Workshop

For behaviors and time profiles check Derive Pearson ’ s r from data (results and theoretical values) –r=1(-1), perfect positive (negative) correlation –r=0, uncorrelated null hypothesis is set to uncorrelated Then, follows student t distribution of degree of freedom Checking significance of r with null hypothesis. Rejection level of null hypothesis is 68.28% (1σ) 22Geant4 Collaboration Workshop

Example Checking value: Observable Energy of Sampling Calorimeter. Material –Pb (Lead)-Scinitillator Thickens –Pb: 8.0 mm/layer, Sci: 2.0 mm/layer Layers –120 layers –1 m x 1 m – interaction surface Beam –Electon 4 GeV Range Cuts –1 mm Pb 8mm 2mm Sci. ・・・・・・・・ e- 23Geant4 Collaboration Workshop

Example 100 histories SD VOV MEAN R Does not pass most of Boolean tests 24Geant4 Collaboration Workshop

Example 1,000 histories SD VOV MEAN R Does not pass some of Boolean tests 25Geant4 Collaboration Workshop

Example 10,000 histories SD VOV MEAN R Does not pass one of Boolean tests (SLOPE check) 26Geant4 Collaboration Workshop

How to apply Energy Spectrum estimation etc. Checking each confidence level of P1, P2, P3, P4,,,, Of course, scoring efficiency becomes low. P1 P2 P3 P4 E V/E 27Geant4 Collaboration Workshop

Unfortunately, this tool does not work well with some deterministic variance reduction techniques. This is come from limitation of CLT (means some variance are required for distribution), so that we can not over come. 28Geant4 Collaboration Workshop

And some simulations becomes deterministic without awaking of users. Please check your simulation carefully. 29Geant4 Collaboration Workshop

Current Status of Development Most part of developments has been done. Following items are remained under development. –Output of testing result –Class or function for minimization of multi dimensional functions 30Geant4 Collaboration Workshop

We want to include this tool in Geant4 but what category is suite for this tool? Run, SD, Hits and its collections, Tally?? 31Geant4 Collaboration Workshop

Summary We have successfully developed a general assistant tool for the checking the results from Monte Carlo simulations like MCNPs. Through this tool, users easily know the confidence intervals for Monte Carlo results. 32Geant4 Collaboration Workshop