Lecture 1 Linear Variational Problems (Part I)

1. Motivation For those participants wondering why we start a course dedicated to nonlinear problems by discussing a particular family of linear problems let us say that: (i) The Newton’s method provides a systematic way to reduce the solution of some nonlinear problems to the solution of a sequence of linear problems.

1. Motivation For those participants wondering why we start a course dedicated to nonlinear problems by discussing a particular family of linear problems let us say that: (i) The Newton’s method provides a systematic way to reduce the solution of some nonlinear problems to the solution of a sequence of linear problems. (ii) During this course, we will show that there are other methods than Newton’s which rely on linear solvers to achieve the solution of nonlinear problems. An example of such methods will be provided by the solution of nonlinear problems in Hilbert spaces by conjugate gradient algorithms.

1. Motivation For those participants wondering why we start a course dedicated to nonlinear problems by discussing a particular family of linear problems let us say that: (i) The Newton’s method provides a systematic way to reduce the solution of some nonlinear problems to the solution of a sequence of linear problems. (ii) During this course, we will show that there are other methods than Newton’s which rely on linear solvers to achieve the solution of nonlinear problems. An example of such methods will be provided by the solution of nonlinear problems in Hilbert spaces by conjugate gradient algorithms. We will start our discussion with the Newton’s method.

2. The Newton’s method in Hilbert spaces Let V be a Hilbert space (real for simplicity); we denote by V’ the dual space of V and by the duality pairing between V and V’. Consider now an operator A (possibly nonlinear) mapping V into V’ and the following equation (E) A(u) = 0. Suppose that operator A is F-differentiable in the neighborhood of a solution u of (E); if v is sufficiently close to u we have then

0 = A(u) = A(v + u – v) = A(v) + A’(v)(u – v) + o(||v – u||), with A’(v) (  L ( V, V’)) the differential of A at v. The abo- ve relation suggest the following algorithm, classically known as the Newton-Raphson method: (1) u 0 is given in V ; then, for n ≥ 0, u n  u n+1 by (2) u n+1 = u n – A’(u n ) –1 A (u n ).

Suppose that A’(u)  Isom(V, V’) and that A’ is locally Lipschitz continuous, then if u 0 is sufficiently close to u we have lim n  +∞ u n = u, the convergence being super-linear. From a ‘practical’ point of view it may be more appropriate to write (2) as (3) A’(u n )(u n+1 – u n ) = – A(u n ), or even better (and equivalently) as: u n+1 – u n  V, (VF) = –,  v  V.

Many remarks are in order, but the only one we will make is that (VF) is a linear variational problem (in the sense of, e.g., J.L. LIONS) since the bilinear functional {v, w}  : V×V  R and the linear functional v  : V  R are both continuous. 3. On a family of linear variational problems in Hilbert spaces: The Lax-Milgram Theorem We are going to focus our discussion on a particular family of linear problems, namely to those problems defined by

considering a triple {V, a, L} verifying the following conditions: (i) V is a real Hilbert space for the scalar (inner) product (.,.) and the associated norm ||. ||. (ii) a: V×V  R is bilinear, continuous and V-elliptic (this last property meaning that there exists  > 0 such that a(v,v) ≥  ||v|| 2,  v  V ). (iii) L: V  R is linear and continuous. Remark 1: We do not assume that the bilinear functional a is symmetric.

Remark 2: If V is a finite dimensional space, the V- ellipticity of a  the positive definiteness of a (not necessarily true if dim V is infinite). To {V, a, L} we associate the following family of linear (variational) problems: Find u  V such that (LVP) a(u,v) = L(v),  v  V.

Why do we consider such a family of linear problems ? Among the many reasons, let us mention the main ones:

1) Many applications in Mechanics, Physics, Control, Image Processing,…….. lead to such problems once the convenient Hilbert space has been identified (exemples of such situations will be encountered during this course).

Why do we consider such a family of linear problems ? Among the many reasons, let us mention the main ones: 1) Many applications in Mechanics, Physics, Control, Image Processing,…….. lead to such problems once the convenient Hilbert space has been identified (exemples of such situations will be encountered during this course). 2) Let us return to the solution of A(u) = 0 by the Newton’s method and suppose that A = J’ where J is a strongly convex twice differentiable functional (i.e.,   > 0, such that ≥  ||w – v|| 2,  v, w), then the linear problem that we have to solve at each iteration to obtain u n +1 belongs to the (LVP) family *.

Strictly speaking, a problem is (was) variational if it can be viewed as the Euler-Lagrange equation of some problem from the Calculus of Variations. In the particular case of (LVP) this supposes that a is symmetric; indeed, if a is symmetric, then (LVP) is equivalent to Find u  V such that (MP) J(u)  J(v),   V, with J(v) = ½ a(v,v) – L(v). Following the influence of J.L. Lions and G. Stampacchia (mid 1960’s) many authors use the terminology variational equations (and of course inequalities) even when a(.,.) is not symmetric.

The Lax-Milgram Theorem Suppose that the triple {V,a,L} verifies (i)-(iii) then (LVP) has a unique solution. For a proof, see, e.g., Atkinson & Han (Springer). There are several proofs of the LM Theorem, our favorite one relies on the Banach fixed point theorem and can be generalized to variational inequalities. Actually, the LM Theorem can be generalized to complex Hilbert spaces by assuming that a(.,.) is sesquilinear, continuous and verifies Re a(v,v) ≥  ||v|| 2,  v  V.

Among the many comments which can be associated with the Lax-Milgram Theorem, we will focus on the following one, considering its computational implications : The fixed point based proof of the LM Theorem is based is based on the following observations:

Among the many comments which can be associated with the Lax-Milgram Theorem, we will focus on the following one, considering its computational implications : The fixed point based proof of the LM Theorem is based on the following observations: (1) From the Riescz representation Theorem, there exists a unique pair {A, l } such that: A  L (V,V), (Av,w) = a(v,w),  v, w  V,   ||A||, ( l,v) = L(v),  v  V.

Among the many comments which can be associated with the Lax-Milgram Theorem, we will focus on the following one, considering its computational implications : The fixed point based proof of the LM Theorem is based on the following observations: (1) From the Riescz representation Theorem, there exists a unique pair {A, l } such that: A  L (V,V), (Av,w) = a(v,w),  v, w  V,   ||A||, ( l,v) = L(v),  v  V. (2) If 0 < ρ < 2  ||A|| – 2 the mapping v  v – ρ(Av – l ) is a uniformly strict contraction of V.

Among the many comments which can be associated with the Lax-Milgram Theorem, we will focus on the following one, considering its computational implications : The fixed point based proof of the LM Theorem is based on the following observations: (1) From the Riescz representation Theorem, there exists a unique pair {A, l } such that: A  L (V,V), (Av,w) = a(v,w),  v, w  V,   ||A||, ( l,v) = L(v),  v  V. (2) If 0 < ρ < 2  ||A|| – 2 the mapping v  v – ρ(Av – l ) is a uniformly strict contraction of V. (3) (LVP) and Au = l are equivalent.

It follows from observations (1)-(3) that if ρ verifies 0 < ρ < 2  ||A|| – 2 we have geometric convergence of the following algorithm,  u 0  V : (1) u 0 is given in V, n ≥ 0, u n  u n+1 by (2) u n+1 = u n – ρ(Au n – l ). The practical interest of (1)-(2) as written above is limited by the fact that in general A and l are unknown.

A more practical form of (1)-(2) is obtained by replacing relation (2) by the following equivalent one: u n+1  V, (2)’ (u n+1,v) = (u n,v) – ρ[a(u n,v) – L(v)],  v  V. Problem (2)’ is also from the (LVP) family, with the role of a(.,.) played by the V-scalar product (.,.). The fact that ρ is not known in general can be overcome by using various techniques using a sequence { ρ n } n instead of a fixed ρ. If a(.,.) is symmetric, conjugate gradient provides a more efficient alternative to algorithm (1)-(2)’, at little extra cost. Conjugate gradient will be discussed next.