Warning: "continue" targeting switch is equivalent to "break". Did you mean to use "continue 2"? in /homepages/1/d352993642/htdocs/inverseproblems/inc/parser/handler.php on line 1552

Warning: Declaration of syntax_plugin_shortcut_anchor::handle($match, $state, $pos, &$handler) should be compatible with DokuWiki_Syntax_Plugin::handle($match, $state, $pos, Doku_Handler $handler) in /homepages/1/d352993642/htdocs/inverseproblems/lib/plugins/shortcut/syntax/anchor.php on line 106

Warning: Declaration of syntax_plugin_shortcut_anchor::render($mode, &$renderer, $data) should be compatible with DokuWiki_Syntax_Plugin::render($format, Doku_Renderer $renderer, $data) in /homepages/1/d352993642/htdocs/inverseproblems/lib/plugins/shortcut/syntax/anchor.php on line 106

Warning: Cannot modify header information - headers already sent by (output started at /homepages/1/d352993642/htdocs/inverseproblems/inc/parser/handler.php:1552) in /homepages/1/d352993642/htdocs/inverseproblems/inc/actions.php on line 38

Warning: Cannot modify header information - headers already sent by (output started at /homepages/1/d352993642/htdocs/inverseproblems/inc/parser/handler.php:1552) in /homepages/1/d352993642/htdocs/inverseproblems/lib/tpl/dokuwiki/main.php on line 12
regularization:tikhonov_regularization [Inverse Problems and Data Assimilation Wiki and International Community]

User Tools

Site Tools


regularization:tikhonov_regularization

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

regularization:tikhonov_regularization [2012/12/27 14:33]
potthast created
regularization:tikhonov_regularization [2012/12/27 14:40] (current)
potthast
Line 1: Line 1:
 ===== Tikhonov Regularization ===== ===== Tikhonov Regularization =====
 +
 +
 +=== 1. Operator Approach to Tikhonov Regularization ===
 +
 +Let $H$ be a mapping from some normed space $X$ into a normed space $Y$. 
 +When solving an equation of the form
 +\begin{equation}
 +H x = y
 +\end{equation}
 +with an [[ip:​ill-posed]] operator $H$, **Tikhonov Regularization** replaces the
 +unstable inverse $H^{-1}$ by a family of stable mappings $R_{\alpha}$ define dy
 +\begin{equation}
 +\label{Ralpha def}
 +R_{\alpha} := (\alpha I + H^{\ast}H)^{-1}H,​
 +\end{equation}
 +where $\alpha>​0$ is known as //​regularization parameter//​. Here, the unbounded operator
 +$(H^{\ast}H)^{-1}$ which appears in the Moore-Penrose Pseudo-Inverse $(H^{\ast}H)^{-1}H^{\ast}$
 +is replaced by the bounded operator $(\alpha I + H^{\ast}H^{\ast})^{-1}$. For $y = Hx$ we have
 +$$
 +\begin{array}{cc}
 +(\alpha I + H^{\ast}H)^{-1}H^{\ast} y & = &​(\alpha I + H^{\ast}H)^{-1}H^{\ast} Hx \\
 +& = & (\alpha I + H^{\ast}H)^{-1}( \alpha I + H^{\ast} H - \alpha I ) x \\
 +& = & x - \alpha (\alpha I + H^{\ast}H)^{-1} x \\
 +& \rightarrow & x
 +\end{array}
 +$$
 +for $\alpha \rightarrow 0$, where the convergence
 +\begin{equation}
 +\alpha (\alpha I + H^{\ast}H)^{-1} x \rightarrow 0, \;\; \alpha \rightarrow 0,
 +\end{equation}
 +is shown by spectral arguments. Please note that this convergence is a //​pointwise//​
 +convergence in $X$ and does not hold in norm!
 +
 +The invertibility of $\alpha I + H^{\ast}H$ is obtained by
 +$$
 +\begin{array}{cc}
 +\langle x, (\alpha I + H^{\ast}H) x \rangle & = & \alpha \langle x,x \rangle + \langle Hx, Hx \rangle \\
 +& \geq & \alpha || x ||^2
 +\end{array}
 +$$
 +according to the Lax-Milgram Lemma, which also yields ​
 +$$
 +|| (\alpha I + H^{\ast}H)^{-1} || \leq \frac{1}{\alpha}.
 +$$
 +The sharper result
 +$$
 +|| (\alpha I + H^{\ast}H)^{-1}H^{\ast} || \leq \frac{1}{2\sqrt{\alpha}}
 +$$
 +is again shown by spectral arguments and the arithmetic-geometric mean
 +$$
 +\frac{\mu}{\alpha + \mu^2} \leq \frac{1}{2\sqrt{\alpha}}. ​
 +$$ 
 +
 +=== 2. Optimization Approach to Tikhonov Regularization ===
 +
 +We can reformulate the equation $Hx=y$ into minimizing ​
 +\begin{equation}
 +J(x) = || y - Hx ||^2, \;\; x \in X, 
 +\end{equation}
 +which is the **Moore-Penrose pseudo inverse**. ​
 +A stabilization is given by adding a term to the functional
 +\begin{equation}
 +J(x) := \alpha || x ||^2 + ||y - Hx||^2, \;\; x \in X. 
 +\end{equation}
 +First order optimality conditions for the minimizer lead to 
 +$$
 +0 = \nabla_x J(x) = 2\alpha x + 2 H^{\ast}(y - H x),
 +$$
 +i.e.
 +$$
 +(\alpha I + H^{\ast}H )x = H^{\ast} y. 
 +$$
 +Thus, the minimizer is obtained by 
 +\begin{equation}
 +x_{\alpha} := (\alpha I + H^{\ast}H)^{-1} H^{\ast} y, 
 +\end{equation}
 +which coincides with (\ref{Ralpha def}). ​
 +
 +=== 3. Spectral Approach to Tikhonov Regularization ===
 +
 +The operator $H^{\ast}H$ is a self-adjoint operator. If $H$ is compact, then $H^{\ast}H$ is
 +compact as well, and there is an orthonormal system $\varphi_{j},​ j \in \mathbb{N},​$ of 
 +//​eigenvectors//​ of $H^{\ast}H$ with eigenvalues $\mu_{j}^2$,​ such that
 +$$
 +H^{\ast}H \varphi_{j} = \mu_{j}^{2} \varphi_{j},​ \;\; j \in \mathbb{N}. ​
 +$$
 +Then, the pseudo inverse of $H$ can be written as
 +$$
 +(H^{\ast}H)^{-1}H^{\ast} y = \sum_{j=1}^{\infty} \frac{1}{\mu_{j}} \langle\varphi_{j},​y\rangle
 +$$
 +When $H$ is compact and $\mu_j$ is sorted in descending order, we know that $\mu_j \rightarrow 0$
 +for $j\rightarrow \infty$. Here, the ill-posedness of $H^{\ast}H$ is reflected by the 
 +unboundedness of $1/​\mu_j^2$. Stabilization can be achieved by bounding this unbounded term. 
 +A **spectral damping scheme** is achieved by 
 +\begin{equation}
 +R_{\alpha} y := \sum_{j=1}^{\infty} \frac{\mu_j}{\alpha + \mu_{j}^2} <​\varphi_{j},​y>,​
 +\end{equation}
 +which for $\alpha \rightarrow 0$ tends to $H^{-1}y$ for every fixed $y \in H(X)$. Using the spectral
 +representation,​ this is readily to be identical to the above inverse $(\alpha I + H^{\ast}H)^{-1} H^{\ast}$. ​
 +
 +=== Literature ===
 +
 +We refer to the following literature for more detail about Tikhonov Regularization: ​
 +
 +   - Groetsch: Inverse problems in the mathematical sciences, 1993
 +   - Rainer Kress: Linear Integral Equations, Springer, 1989
 +   - David Colton and Rainer Kress: Inverse Acoustic and Electromagnetic Scattering Theory, 1993
 +   - Engl, Hanke and Neugebauer: Regularization of Inverse Problems, 1996
 +   - Andreas Kirsch: Introduction into the mathematical theory of inverse problems, Springer 1996
 +
regularization/tikhonov_regularization.txt ยท Last modified: 2012/12/27 14:40 by potthast