Lecture
Notes on Large-scale Unconstrained and Constrained Optimization

- Introduction and Basics of Unconstrained Optimization (basics, characterization of solutions, overview of algorithms)
- Line Search Methods (step length conditions and algorithms, global convergence conditions, and rate of convergence)
- Trust Region Methods (basic trust regions ideas, Cauchy point and minimum progress, dogleg and twodimensional subspace minimization, global convergence, local convergence)
- Linear and Nonlinear Conjugate Gradients (linear cg, rate of convergence, nonlinear cg, Fletcher-Reeves, Polak-Ribiere variant, global convergence FR CG and FR-PR CG)
- Quasi-Newton Methods, Nonlinear Least Squares, and Background of Constrained Optimization
- Interior Point Methods for Linear and Nonlinear Programming

Lecture Notes on Nonlinear Systems of Equations

- Introduction
(discussion of basic issues, convergence of Newton, linesearch,
stopping criteria)
- Newton
with Gaussian elimination (exact Newton, finite difference Jacobian, exploiting sparsity)
- Newton-Krylov
methods (soon)
- Broyden's method

Lecture Notes on Large, Sparse Eigenvalue Problems

- Introduction
(basics - chapter 1 sections 1 and 2 and chapter 4 section 1)
- Perturbation
Theory (chapter 1 section 3 and chapter 4 section 2)
- Overview
of Methods (for this course)
- Krylov
subspaces and Rayleigh-Ritz approximation (chapter 4 sections 3 and 4)
- Krylov
sequence methods, Arnoldi and Lanczos methods and variants (implicitly restarted Arnoldi/Lanczos)
- Newton-based
methods, Jacobi-Davidson method, Davidson's method

Useful material on Numerical
Linear Algebra, based on David Watkins, *Fundamentals of Matrix Computations*
(2nd ed.), Wiley:

- Intro
- Overview
- Chapters
1 and 2 - Solving Linear Systems of Equations
- Chapter
3 - Overdetermined Systems
- Chapter
4 - Singular Value Decomposition
- Chapter
5 - Eigenvalue Problems Part I
- Chapter
5 - Eigenvalue Problems Part II
- (more fun
stuff to be added)

Large, sparse eigenvalue problems and sensitivity/accuracy of eigenvalue problems (from Summer School organized by the Materials Computation Center):

- Derivation
of Methods
- Sensitivity
and Accuracy