# Optimization Methods for Tensor Toolbox

Most MATLAB optimization methods have different interfaces or none at all. In Tensor Toolbox, we are adopting wrappers for moderate consistency, and these will be called from within other Tensor Toolbox functions. Here we outline the choices, their installation instructions, and then the user-tunable parameters.

A few general notes:

• These methods all require explicit initial guesses as well as handles for the function and gradient calculations. These are handled by the calling routines.
• The parameters marked with asterisks should generally not be modified by the user because they will be set by the calling routine.

For more information on the details of these methods, see Developer Information for Optimization Methods in Tensor Toolbox.

## lbfgsb: Limited-Memory Quasi-Newton with Bound Constraints

In most methods, setting 'opt' to 'lbfgsb' will enable this method, and then any of the settings in the table below can be modified by passing these as additional options to the method.

 Name Description Default lower Lower bounds, can be vector or scalar -Inf upper Upper bounds, can be vector or scalar +Inf maxiters Max outer iterations 1000 printitn Printing frequency by iteration (0=no output) 1 m Limited-memory parameter 5 subiters Controls maximum calls to function-gradient evalations 10 ftol Stopping condition based on relative function change 1e-10 gtol Stopping condition based on gradient norm 1e-5 mdesc (*) Method description printed out before run 'L-BFGS-B Optimization' xdesc (*) Variable description auto-generated

Installation Instructions. Download and install L-BFGS-B by Stephen Becker. Please see that web page for full details on references, installation, etc. Here we provide cursory instructions for installation:

2. Unzip and goto the Matlab/ subdirectoy with MATLAB
3. Type compile_mex

Detailed notes. The wrapper for this method is tt_lbfgsb in the Tensor Toolbox. Notes regarding mappings to the parameters of Becker's L-BFGS-B code:

• maxIts maps to maxiters and the default is increased from 100 to 1000
• printEvery maps to printitn
• maxTotalIts is set to maxiters*subiters and this effectively changes the default from 5000 to 10000
• factr is set to ftol / eps and this effectively changes the default from 1e7 to 4.5e5
• pgtol is set to gtol

## lbfgs: Limited-Memory Quasi-Newton

In most methods, setting 'opt' to 'lbfgs' will enable this method, and then any of the settings in the table below can be modified by passing these as additional options to the method.

 Name Description Default maxiters Max outer iterations 1000 printitn Printing frequency by iteration (0=no output) 1 m Limited-memory parameter 5 subiters Controls maximum calls to function-gradient evalations 10 ftol Stopping condition based on relative function change 1e-10 gtol Stopping condition based on gradient norm 1e-5 mdesc (*) Method description printed out before run 'Poblano L-BFGS Optimization' xdesc (*) Variable description auto-generated

Installation Instructions. Download and install Poblano Toolbox, v1.2. Please see that web page for full details on references, installation, etc. Here we provide cursory instructions for installation:

2. Unzip and goto the poblano_toolbox-1.2/ subdirectoy within MATLAB
3. Type install_poblano to save this directory to your path

Detailed notes. The wrapper for this method is tt_lbfgs in the Tensor Toolbox. Notes regarding mappings to the parameters of Poblano's L-BFGS code:

• MaxIters maps to maxiters
• Display maps to printitn
• MaxFuncEvals is set to maxiters*subiters
• RelFuncTol is set to ftol
• StopTol is set to gtol

## fminunc: Optimizaton Toolbox Quasi-Newton Method

In most methods, setting 'opt' to 'fminunc' will enable this method, and then any of the settings in the table below can be modified by passing these as additional options to the method. This requires the MATLAB Optimization Toolbox.

 Name Description Default maxiters Max outer iterations 1000 printitn Display (0=no output) 1 subiters Controls maximum calls to function-gradient evalations 10 gtol Stopping condition based on gradient norm 1e-5 mdesc (*) Method description printed out before run 'Quasi-Newton Optimization (via Optimization Toolbox)' xdesc (*) Variable description auto-generated