Updating conjugate directions by the BFGS formula with variable storage
Date
1992
Authors
Lee, Ann
Journal Title
Journal ISSN
Volume Title
Publisher
Abstract
In solving unconstrained nonlinear optimization problems, the quasi-Newton (QN) methods and the conjugate gradient (CG) methods are widely used. These two methods can be regarded as representatives of two quite different approaches: the QN methods usually have very good convergence properties, but require O(n²) memory space; on the contrary, the CG methods require only 3n or 4n locations of memory space, but usually have a slower convergence rate than the QN methods. Some algorithms have been developed trying to achieve some kind of trade-off between the QN and CG methods in terms of memory requirements and convergence speed. These include the Variable Storage Conjugate Gradient method (VSCG) which runs the QN algorithm for a number of iterations (depending on the available memory space), then switches to the preconditioned CG method until termination.
Powell proposed a method to use a matrix Z instead of the quasi-Newton matrix H, where Z can be updated in a similar way to updating H by the BFGS formula. The advantage of using Z is that ZZᵀ remains positive definite despite accumulated roundoff errors during the iterations, which is important to keep good accuracy for those problems that are very sensitive to roundoff errors. The disadvantage of Powell's method is that Z is not symmetric, so this method requires twice as much storage as the QN method.
In this thesis, we develop a so-called VS-ZZᵀ algorithm by applying the strategy of the VSCG method to Powell's method, so that we can largely maintain the good properties of Powell's method, but with a variable storage requirement. Instead of storing the whole matrix Z, the VS-ZZᵀ algorithm consists of two parts: the QN-part runs a modified Powell's updating formula based on the information saved at each iteration; the CG-part runs the preconditioned CG algorithm, with a fixed preconditioner Hₘ = ZₘZᵀₘ obtained from the QN-part. Numerical experiments have been made to verify that our algorithm does keep the main advantages of Powell's method, but the storage requirement can be adjusted by the user depending on the available memory space.
Description
Keywords
UN SDG 17: Partnerships