Numerical Solution of Ill-Posed Problems

In ill-posed problems, small changes in the data can cause arbitrarily large changes in the results. Although it would be nice to avoid such problems, they have important applications in medicine (computerized tomography), remote sensing (determining whether a nuclear reactor has a crack), and astronomy (image processing).

Two projects were performed jointly with workers at the National Institute of Standards and Technology (formerly the National Bureau of Standards). In the first, data related to non-destructive detection of crack formation was analyzed by filtering techniques using a small problem-dependent set of basis functions [J11]. This was the first hybrid algorithm for ill-posed problems: the given problem was projected to a subspace and regularization was applied there.

In the second project, confidence intervals for spectroscopy data were computed using a nonnegativity constraint on the solution [J21] [J38].

Other work focused on the choice of optimization criteria for ill-posed problems [J37], studying the characteristics of solutions produced by various regularization methods, including truncated least squares, regularized least squares, regularized total least squares, and truncated total least squares [C11] [C15] [J43] [J51]. Efficient numerical algorithms for computing these solutions in image processing applications were also proposed [J50] [J45] [C16] [J61].

The regularization of discretized problems by iterative methods was studied with Misha Kilmer [J57], demonstrating that regularization could be provided easily if the iterative method was viewed as projecting the continuous problem into a lower dimensional subspace. Choice of the regularization parameter was considered in [J59], [J85], and [J87].

New alternatives to standard regularization methods were explored in [J102] and [J103].

[C11]

Richardo Fierro, Gene H. Golub, Per Christian Hansen, and Dianne P. O'Leary, ``Regularization by Truncated Total Least Squares,'' Proceedings of the Fifth SIAM Conference on Applied Linear Algebra, J.G. Lewis, ed., SIAM Press, Philadelphia, 1994, 250-254.

[C15]

Dianne P. O'Leary, ``The SVD in Image Restoration," SVD and Signal Processing III: Algorithms, Architectures, and Applications, Marc Moonen and Bart DeMoor, eds., Elsevier, New York, 1995, 315-322.

[C16]

James G. Nagy and Dianne P. O'Leary, "Fast Iterative Image Restoration with a Spatially Varying PSF," in Advanced Signal Processing Algorithms, Architectures, and Implementations VII F. T. Luk, ed., SPIE, 1997, 388-399.

[J11]

Dianne P. O'Leary and John A. Simmons, ``A bidiagonalization- regularization procedure for large scale discretizations of ill-posed problems,'' SIAM J. on Scientific and Statistical Computing 2 (1981) 474-489.

[J21]

Dianne P. O'Leary and B. W. Rust, ``Confidence intervals for inequality-constrained least squares problems, with applications to ill-posed problems,'' SIAM Journal on Scientific and Statistical Computing 7 (1986) 473-489.

[J37]

Per Christian Hansen and Dianne P. O'Leary, ``The use of the L-curve in the regularization of discrete ill-posed problems,'' SIAM Journal on Scientific Computing 14 (1993) 1487-1503.

[J38]

Bert W. Rust and Dianne P. O'Leary, ``Confidence intervals for discrete approximations to ill-posed problems ,'' The Journal of Computational and Graphical Statistics, 3 (1994) 67-96.

[J43]

Ricardo D. Fierro, Gene H. Golub, and Per Christian Hansen, and Dianne P. O'Leary, ``Regularization by Truncated Total Least Squares," SIAM Journal on Scientific Computing, 18 (1997) 1223-1241.

[J45]

James G. Nagy and Dianne P. O'Leary, ``Restoring Images Degraded by Spatially-Variant Blur," SIAM Journal on Scientific Computing, 19 (1998), 1063-1082.

[J50]

Misha E. Kilmer and Dianne P. O'Leary, ``Pivoted Cauchy-Like Preconditioners for Regularized Solution of Ill-Posed Problems," SIAM Journal on Scientific Computations, 21 (1999) 88-110.

[J51]

Gene H. Golub, Per Christian Hansen, Dianne P. O'Leary, ``Tikhonov Regularization and Total Least Squares," SIAM Journal on Matrix Analysis and Applications, 21 (1999) 185-194.

[J57]

Misha E. Kilmer and Dianne P. O'Leary, ``Choosing Regularization Parameters in Iterative Methods for Ill-Posed Problems," SIAM J. on Matrix Analysis and Applications, 22 (2001) 1204-1221.

[J59]

Dianne P. O'Leary, ``Near-Optimal Parameters for Tikhonov and Other Regularization Methods," SIAM J. on Scientific Computing, 23 (2001) 1161-1171.

[J61]

James G. Nagy and Dianne P. O'Leary, ``Image Restoration through Subimages and Confidence Images," Electronic Transactions on Numerical Analysis, 13 (2002) 22-37.

[J85]

Julianne Chung, James G. Nagy, and Dianne P. O'Leary, ``A Weighted GCV Method for Lanczos Hybrid Regularization," Electronic Transactions on Numerical Analysis, 28 (2008) 149-167.

[J87]

Bert W. Rust and Dianne P. O'Leary, ``Residual Periodograms for Choosing Regularization Parameters for Ill-Posed Problems", Inverse Problems, (invited paper) 24 (2008) 034005 (30 pages). DOI:10.1088/0266-5611/24/3/034005

[J102]

Julianne M. Chung, Matthias Chung, and Dianne P. O'Leary, ``Designing Optimal Spectral Filters for Inverse Problems" SIAM Journal on Scientific Computing, 33(6) (2011) DOI: 10.1137/100812938

[J103]

Julianne M. Chung, Glenn R. Easley, and Dianne P. O'Leary, Windowed Spectral Regularization of Inverse Problems," SIAM Journal on Scientific Computing, 33(6) (2011) DOI: 10.1137/100809787