1. Petroleum Reservoir Management Based on Approximate Dynamic Programming Zheng Wen, Benjamin Van Roy, Louis Durlofsky and Khalid Aziz Smart Field Consortium, Stanford University Motivation  Petroleum Reservoir can be modeled as a nonlinear dynamic system  State x : Pressure or Saturation  Control u : Bottom-Hole Pressure or Well Flow Rate  L(x,u): Instantaneous cost  System dynamics are determined by  Fluid dynamics  Mass balance equation  Geological model  Objective: Reservoir Management as an Optimization Problem Basis Function Selection Compute Weights: Smoothing Reduced Linear Programming (RLP) with Regularization  Basis functions include:  Global variables: constant, total oil, total water, oil/water ratio  Basis functions constructed by Proper Orthogonal Decomposition (POD): ADP Algorithm (Fixed Basis Function) Case 1: General Primary Production  Classical model in Petroleum Engineering  Two phase, two component model (oil, water)  Produce oil (with some water) by injecting water  The dynamic model is highly nonlinear  Geological Model is also a modified portion of SPE-10  35×40×1 blocks, 4 production wells and 1 injection well  Constraints on controls: BHPs are bounded  We use fixed basis functions in this case  Improvement is  15% compared with myopic control  8% compared with myopic-then-stop control  water saturation is more symmetric after optimization  Shortage of energy resources calls for better petroleum reservoir management policies  Optimizing decision-making in reservoir management is challenging  Large-scale nonlinear dynamic optimization problem  Complicated constraints on control  Current optimization techniques have various limitations  We propose optimization algorithms based on Approximate Dynamic Programming (ADP) to solve this problem State x Instantaneous cost L(x,u) Control Control Policy System (Petroleum Reservoir) Brief Overview of ADP  Dynamic Programming (DP) potentially achieves global optimum but suffers from the “curse of dimensionality ”  ADP tries to keep DP’s merits but overcome the dimensionality curse  Approach: approximate the cost-to-go function as a linear regression of a set of basis functions:  Question: how to choose basis functions and weights?  Many ADP algorithms could be used to compute weights for given basis functions;  Choice of basis functions is highly problem dependent  For reservoir production problem, basis functions should contain “enough information” about future total cost  An implementable approximation of LP approach in DP  Linear dynamics with complex cost function  Penalize on low BHP  Geological model is a modified portion of SPE 10  35×35×1 blocks, 4 production wells  We use fixed basis functions in this case  Baseline is chosen as the best result of 500 constant- control simulations and 500 randomized-control simulations Run Simulations and Get “Snapshots”, then Normalize them Characterize a Subspace Based on SVD Select Polynomials in that Subspace as Basis Functions L 1 Regularization Slack Variable (Smoothing) Sampled States Weights Sample States Based on Randomized Baseline Strategy Construct Basis Functions by POD Compute Weights Control Strategy Approximate Cost-to-go Function ConstraintsInput for SVD Solve Sub-Problem Basis Fun Weights ADP Algorithm (Adaptive Basis Function) Sample States Based on Randomized Baseline Strategy Construct Basis Functions by POD Compute Weights Control Strategy Approximate Cost-to-go Function ConstraintsInput for SVD Basis Fun Weights Basis Function Selection Strategy Evaluation Case 2: Black Oil Model with Simple Constraints Case 3: Comparison with Gradient-Based Method  We apply both ADP (with fixed basis functions) and Gradient-Based Algorithm to an example and compare their performances:  Black-oil Model  40×40×1 blocks, 4 production wells and 4 injection wells  Constraints on control: Bounded BHPs  We rerun gradient-based method for 200 different starting points, and record the best and the worst local optimum Case 4: Black-Oil Model with Complicated Constraints  Black-oil Model  40×40×1 blocks, 2 production wells and 2 injection wells  Constraints on controls:  Bounded BHP  Maximum water cut of each production well  Maximum liquid injection/production rates  Minimum oil production rate  Apply ADP with adaptive basis functions  Baseline: best result of 100 feasible control strategies  Result: 19% improvement over baseline (see Figure above) Conclusion and Future Work  We have applied ADP to reservoir optimization problems and compared its performance with baseline/other optimization algorithms  ADP has been shown to be a promising approach for reservoir production problems  Tentative future works include:  Test ADP extensively for realistic 3D examples  Compare with other optimization techniques  Model uncertainty of geological parameters