A General Approach to Sensitivity Analysis Darryl Fenwick, Streamsim Technologies Céline Scheidt, Stanford University

Role of sensitivity analysis Model calibration (such as in history matching) Model identification (which models best represent a physical phenomenon) Model reduction (which parameters can be removed from the model) Model quality (is a model valid) May 9, 2012SCRF 25th Annual Meeting2

SA in reservoir modeling Sensitivity analysis in reservoir modeling has been dominated by the use of experimental design (ED) and response surface methods (RSM) Why ED and RSM? Efficiency and strong mathematical basis Implemented in commercial software May 9, 2012SCRF 25th Annual Meeting3

Regression Sensitivity analysis & ED Typically applied to: Continuous, “deterministic” parameters Single response 10 years) May 9, 2012SCRF 25th Annual Meeting4

ED & RSM: limitations Limitations: ED ignores prior PDF Single response Smooth response Discrete parameters Fault interpretations Facies proportion cubes Stochastic “noise” in response Spatial uncertainty ED & RSM is one technique within global SA approaches May 9, 2012SCRF 25th Annual Meeting5

A general SA approach Spear and Hornberger – study of growth of nuisance alga Divided output of model into two classes, behavior B, and behavior B’ Analyzed how model parameters influenced the classification May 9, 2012SCRF 25th Annual Meeting6

Proposed generalized SA 1.Define model and the set of uncertain input parameters. 2.Assign prior PDFs to the input parameters. 3.Generate an ensemble of models through sampling of prior PDFs. 4.Evaluate the models, creating output for the responses of interest. 5.Classify the model ensemble based upon the responses of interest. 6.Analyze the sample distributions of the input parameters within each class. 7.Asses the influence of each input parameter based upon distributions May 9, 2012SCRF 25th Annual Meeting7

Example: f(x,y,z) = x + y 50 samples created of x,y,z ϵ U[0,1] Classify 50 models into 3 clusters using value of response May 9, 2012SCRF 25th Annual Meeting8 Compare the distributions of x, y, z in three clusters with initial sample

Example: f(x,y,z) = x + y Cumulative distribution functions Initial 50 models (black) 3 clusters (red, blue, green) May 9, 2012SCRF 25th Annual Meeting9 x y z

Basic idea Influential parameter will distinguish the models into the separate classes (clusters) Evident when comparing the distributions Non-influential parameters will have no impact on the classification The distributions will be similar between classes May 9, 2012SCRF 25th Annual Meeting10

Advantages Account for any type of parameter distribution Classification is not limited to a single response Responses can be stochastic in nature Model responses are used only for classification Proxy models can be employed Accuracy of the response itself is inconsequential. What is important is that the responses correctly classify the models May 9, 2012SCRF 25th Annual Meeting11

Classification in metric space  Classify models by clustering in metric space Automatic clustering using iterative k-medoids May 9, 2012SCRF 25th Annual Meeting12 Clustering algorithm Metric Space Next step: Analysis

Techniques for analysis Kolmogorov-Smirnov test – for continuous distributions D n = maximum vertical distance May 9, SCRF 25th Annual Meeting

Techniques for analysis Cramer-von Mises test (similar to K-S test) Chi-Squared Test – for categorical or binned data May 9, 2012SCRF 25th Annual Meeting14

General method – L1-Norm Statistical tests require a large number of samples We can also calculate a “distance” between sample distribution F(x) and distribution in cluster F(x|C) May 9, SCRF 25th Annual Meeting

Normalization of L1-norm L1 distance normalized using a resampling procedure to estimate the statistical significance of distance Attempts to resolve problem of small sample sizes May 9, SCRF 25th Annual Meeting

Parameter interactions Response surfaces model them using “interaction” terms In the general SA approach, we are interested in how the distribution of x is influenced by y in C  F(x|y,C) (More of a “dependency” instead of an interaction) May 9, 2012SCRF 25th Annual Meeting17

Parameter interactions Measures of parameter dependency Correlation coefficient (ρ xy ) L1-norm: Bin y values (min, med, max) Construct F(x|y,C) Compare to F(x|C) May 9, 2012SCRF 25th Annual Meeting18 F(x|y,C) F(x|y) X Y

Application - WCA field Offshore turbidite 20 producers 8 injectors 78 x 59 x 116 ~ 100,000 active grid blocks 3-1/2 years production Uncertainty Depositional scenario 19May 9, 2012SCRF 25th Annual Meeting Scheidt, C. and J.K. Caers, “Uncertainty Quantification in Reservoir Performance Using Distances and Kernel Methods – Application to a West-Africa Deepwater Turbidite Reservoir”, SPEJ 2009

One realization – TI1 Upper SectionLower Section May 9, 2012SCRF 25th Annual Meeting20

Depositional scenarios TI 1TI 3 May 9, 2012SCRF 25th Annual Meeting21

Depositional scenarios TI 8TI 9 May 9, 2012SCRF 25th Annual Meeting22

Depositional scenarios TI 10TI 13 May 9, 2012SCRF 25th Annual Meeting23

Applications First application – 9 continuous parameters in the flow simulation, specifically: Swc for the levee and channel sands (2 parameters) Sorw for the levee and channel sands (2) Maximum water and oil rel perm values (2) Water and oil Corey exponents (2) Kv/Kh ratio (1) Single response: final cum. oil production Goal: compare general SA with traditional methods 60 runs created using Latin hypercube sampling May 9, 2012SCRF 25th Annual Meeting24

Traditional SA methods RSMCorrelation Coefficient May 9, 2012SCRF 25th Annual Meeting25

Kolmogorov-Smirnov test Cluster 1Cluster 2Cluster 3 KvKh0.88 (0)0.85 (0)0.99 (0) SOWCR levee0.95 (0)0.99 (0)0.97 (0) SOWCR sand0.93 (0)0.98 (0)0.89 (0) SWCR levee0.38 (0)0.57 (0)0.58 (0) SWCR sand0.82 (0)0.35 (0)0.14 (0) kroMax0.97 (0)0.69 (0)0.90 (0) krwMax0.34 (0)0.01 (1)0.001 (1) oilExp0.97 (0)0.90 (0)0.99 (0) watExp0.98 (0)0.63 (0)0.08 (0) May 9, 2012SCRF 25th Annual Meeting26

Cramer – von Mises test Cluster 1Cluster 2Cluster 3 KvKh0.42 (0)0.12 (0)0.05 (0) SOWCR levee0.02 (0)0.007 (0)0.03 (0) SOWCR sand0.12(0)0.05 (0)0.16 (0) SWCR levee0.61(0)0.37 (0)0.58 (0) SWCR sand0.36 (0)0.84 (0)0.90 (0) kroMax0.12 (0)0.64 (0)0.25(0) krwMax0.71 (0)0.99 (1)1 (1) oilExp0.05(0)0.12 (0)0.02 (0) watExp0.21 (0)0.71 (0)0.98 (1) May 9, 2012SCRF 25th Annual Meeting27

General SA May 9, 2012SCRF 25th Annual Meeting28 Normalized L1-Norm Distance

Summary of comparison Results of tornado plot, RSM, correlation coefficient, and general SA all in general agreement Influential parameters RSM: watExp, oilExp, krwMax, kroMax, SWCR_sand CC: krwMax, watExp, SWCR_sand General SA: krwMax, watExp, SWCR_sand K-S test and C-vM test concur with general SA, but are more conservative RSM indicates that Corey oil exponent (oilExp) is very influential parameter Correlation coefficient and general SA does not May 9, 2012SCRF 25th Annual Meeting29

Parameter interactions Comparison using RSM, correlation coefficient, and normalized L1-norm distance May 9, 2012SCRF 25th Annual Meeting30

May 9, 2012SCRF 25th Annual Meeting31 Normalized L1-norm Correlation coefficient Experimental design

A more sophisticated application Same 9 continuous parameters as before + training image TI’s represent uncertainty of depositional scenario Distance -  of difference over all TS of: Oil production rate for 20 producers Water injection for 8 injectors. 60 runs created using Latin hypercube sampling May 9, 2012SCRF 25th Annual Meeting32

A more sophisticated application Challenges for traditional SA methods 1.Discrete parameter  TI For 6 training images, would require building 6 response surfaces 2.Multiple responses (oil & water rates) 3.Stochastic response Seed for geostatistical algorithm changes for each run May 9, 2012SCRF 25th Annual Meeting33

Results – focus on TI CDF for TI in 3 clusters Black – initial sample Red = Cluster 3 Green = Cluster 2 Blue = Cluster 1 Chi-Squared test Cluster 2 and Cluster 3 show that TI is influential hchi2stat Cluster Cluster Cluster May 9, 2012SCRF 25th Annual Meeting34

Normalized L1-norm distance krwMax, SOWCR_sand, and TI are influential  Strong variation due to spatial uncertainty May 9, 2012SCRF 25th Annual Meeting35

Conclusion of SA for WCA Training image has (slight) influence on classification of models Parameter dependencies difficult to analyze Parameters chosen such that many dependencies are possible Analysis of results is very difficult for complex applications Different SA techniques can give different results Requires interpretation by modeling team May 9, 2012SCRF 25th Annual Meeting36

Conclusion General SA approach has been developed Idea: Use model classification and parameter distributions as basis for SA Addresses some limitations in traditional approach to SA in reservoir modeling RSM & ED are still very powerful  general SA is a complimentary approach New approach – ideas/concepts are new and still developing May 9, 2012SCRF 25th Annual Meeting37