User Tools

Site Tools


Tikhonov Regularization

1. Operator Approach to Tikhonov Regularization

Let $H$ be a mapping from some normed space $X$ into a normed space $Y$. When solving an equation of the form \begin{equation} H x = y \end{equation} with an ill-posed operator $H$, Tikhonov Regularization replaces the unstable inverse $H^{-1}$ by a family of stable mappings $R_{\alpha}$ define dy \begin{equation} \label{Ralpha def} R_{\alpha} := (\alpha I + H^{\ast}H)^{-1}H, \end{equation} where $\alpha>0$ is known as regularization parameter. Here, the unbounded operator $(H^{\ast}H)^{-1}$ which appears in the Moore-Penrose Pseudo-Inverse $(H^{\ast}H)^{-1}H^{\ast}$ is replaced by the bounded operator $(\alpha I + H^{\ast}H^{\ast})^{-1}$. For $y = Hx$ we have $$ \begin{array}{cc} (\alpha I + H^{\ast}H)^{-1}H^{\ast} y & = &(\alpha I + H^{\ast}H)^{-1}H^{\ast} Hx \\ & = & (\alpha I + H^{\ast}H)^{-1}( \alpha I + H^{\ast} H - \alpha I ) x \\ & = & x - \alpha (\alpha I + H^{\ast}H)^{-1} x \\ & \rightarrow & x \end{array} $$ for $\alpha \rightarrow 0$, where the convergence \begin{equation} \alpha (\alpha I + H^{\ast}H)^{-1} x \rightarrow 0, \;\; \alpha \rightarrow 0, \end{equation} is shown by spectral arguments. Please note that this convergence is a pointwise convergence in $X$ and does not hold in norm!

The invertibility of $\alpha I + H^{\ast}H$ is obtained by $$ \begin{array}{cc} \langle x, (\alpha I + H^{\ast}H) x \rangle & = & \alpha \langle x,x \rangle + \langle Hx, Hx \rangle \\ & \geq & \alpha || x ||^2 \end{array} $$ according to the Lax-Milgram Lemma, which also yields $$ || (\alpha I + H^{\ast}H)^{-1} || \leq \frac{1}{\alpha}. $$ The sharper result $$ || (\alpha I + H^{\ast}H)^{-1}H^{\ast} || \leq \frac{1}{2\sqrt{\alpha}} $$ is again shown by spectral arguments and the arithmetic-geometric mean $$ \frac{\mu}{\alpha + \mu^2} \leq \frac{1}{2\sqrt{\alpha}}. $$

2. Optimization Approach to Tikhonov Regularization

We can reformulate the equation $Hx=y$ into minimizing \begin{equation} J(x) = || y - Hx ||^2, \;\; x \in X, \end{equation} which is the Moore-Penrose pseudo inverse. A stabilization is given by adding a term to the functional \begin{equation} J(x) := \alpha || x ||^2 + ||y - Hx||^2, \;\; x \in X. \end{equation} First order optimality conditions for the minimizer lead to $$ 0 = \nabla_x J(x) = 2\alpha x + 2 H^{\ast}(y - H x), $$ i.e. $$ (\alpha I + H^{\ast}H )x = H^{\ast} y. $$ Thus, the minimizer is obtained by \begin{equation} x_{\alpha} := (\alpha I + H^{\ast}H)^{-1} H^{\ast} y, \end{equation} which coincides with (\ref{Ralpha def}).

3. Spectral Approach to Tikhonov Regularization

The operator $H^{\ast}H$ is a self-adjoint operator. If $H$ is compact, then $H^{\ast}H$ is compact as well, and there is an orthonormal system $\varphi_{j}, j \in \mathbb{N},$ of eigenvectors of $H^{\ast}H$ with eigenvalues $\mu_{j}^2$, such that $$ H^{\ast}H \varphi_{j} = \mu_{j}^{2} \varphi_{j}, \;\; j \in \mathbb{N}. $$ Then, the pseudo inverse of $H$ can be written as $$ (H^{\ast}H)^{-1}H^{\ast} y = \sum_{j=1}^{\infty} \frac{1}{\mu_{j}} \langle\varphi_{j},y\rangle $$ When $H$ is compact and $\mu_j$ is sorted in descending order, we know that $\mu_j \rightarrow 0$ for $j\rightarrow \infty$. Here, the ill-posedness of $H^{\ast}H$ is reflected by the unboundedness of $1/\mu_j^2$. Stabilization can be achieved by bounding this unbounded term. A spectral damping scheme is achieved by \begin{equation} R_{\alpha} y := \sum_{j=1}^{\infty} \frac{\mu_j}{\alpha + \mu_{j}^2} <\varphi_{j},y>, \end{equation} which for $\alpha \rightarrow 0$ tends to $H^{-1}y$ for every fixed $y \in H(X)$. Using the spectral representation, this is readily to be identical to the above inverse $(\alpha I + H^{\ast}H)^{-1} H^{\ast}$.


We refer to the following literature for more detail about Tikhonov Regularization:

  1. Groetsch: Inverse problems in the mathematical sciences, 1993
  2. Rainer Kress: Linear Integral Equations, Springer, 1989
  3. David Colton and Rainer Kress: Inverse Acoustic and Electromagnetic Scattering Theory, 1993
  4. Engl, Hanke and Neugebauer: Regularization of Inverse Problems, 1996
  5. Andreas Kirsch: Introduction into the mathematical theory of inverse problems, Springer 1996
regularization/tikhonov_regularization.txt · Last modified: 2012/12/27 14:40 by potthast