The {ao} package implements alternating optimization (AO) in
R.
AO is an iterative process that optimizes a function by alternately performing restricted optimization over parameter subsets. Instead of solving one joint optimization problem, AO breaks it into smaller sub-problems. This can make optimization feasible when joint optimization is too difficult.
The AO process implemented in {ao} can be
-
viewed as a generalization of joint optimization,
-
used for minimization and maximization problems with custom parameter partitions,
-
randomized by changing the parameter partition randomly after each iteration,
-
run in multiple parallel processes for different initial values, parameter partitions, and/or base optimizers.
See the package vignette for more details.
You can install the released package version from CRAN with:
install.packages("ao")Then load the package with library("ao"). Here is a simple example of
alternating minimization of the Rosenbrock
function:
rosenbrock <- function(x) (1 - x[1])^2 + (x[2] - x[1]^2)^2The resulting optimization path is shown below.
It is obtained with:
ao(f = rosenbrock, initial = c(2, 2))
#> $estimate
#> [1] 1.000895 1.001791
#>
#> $value
#> [1] 8.016137e-07
#>
#> $details
#> iteration value p1 p2 b1 b2 seconds
#> 1 0 5.000000e+00 2.000000 2.000000 0 0 0.000000000
#> 2 1 1.519238e-01 1.366025 2.000000 1 0 0.010935068
#> 3 1 1.339744e-01 1.366025 1.866024 0 1 0.004087925
#> 4 2 1.176778e-01 1.320824 1.866024 1 0 0.006880045
#> 5 2 1.029278e-01 1.320824 1.744575 0 1 0.003880978
#> 6 3 8.966402e-02 1.278883 1.744575 1 0 0.008837938
#> 7 3 7.777546e-02 1.278883 1.635540 0 1 0.005197048
#> 8 4 6.719114e-02 1.240415 1.635540 1 0 0.009642124
#> 9 4 5.779955e-02 1.240415 1.538630 0 1 0.004590034
#> 10 5 4.952339e-02 1.205560 1.538630 1 0 0.008391857
#> 11 5 4.225482e-02 1.205560 1.453374 0 1 0.004535198
#> 12 6 3.591491e-02 1.174366 1.453374 1 0 0.007378817
#> 13 6 3.040344e-02 1.174366 1.379135 0 1 0.005630970
#> 14 7 2.564430e-02 1.146792 1.379135 1 0 0.007120132
#> 15 7 2.154801e-02 1.146792 1.315133 0 1 0.007462025
#> 16 8 1.804492e-02 1.122712 1.315133 1 0 0.008414984
#> 17 8 1.505832e-02 1.122712 1.260483 0 1 0.005286932
#> 18 9 1.252724e-02 1.101923 1.260483 1 0 0.010216951
#> 19 9 1.038836e-02 1.101923 1.214235 0 1 0.004881144
#> 20 10 8.590837e-03 1.084167 1.214235 1 0 0.023866892
#> 21 10 7.084101e-03 1.084167 1.175418 0 1 0.005457878
#> 22 11 5.827377e-03 1.069149 1.175418 1 0 0.023604870
#> 23 11 4.781578e-03 1.069149 1.143079 0 1 0.005419970
#> 24 12 3.915156e-03 1.056558 1.143079 1 0 0.021442890
#> 25 12 3.198754e-03 1.056558 1.116314 0 1 0.005636930
#> 26 13 2.608707e-03 1.046082 1.116314 1 0 0.052335024
#> 27 13 2.123531e-03 1.046082 1.094287 0 1 0.004125834
#> 28 14 1.725945e-03 1.037424 1.094287 1 0 0.006824970
#> 29 14 1.400576e-03 1.037424 1.076249 0 1 0.004114151
#> 30 15 1.135093e-03 1.030310 1.076249 1 0 0.005856991
#> 31 15 9.187038e-04 1.030310 1.061539 0 1 0.003890991
#> 32 16 7.427825e-04 1.024492 1.061539 1 0 0.005117893
#> 33 16 5.998755e-04 1.024492 1.049585 0 1 0.004287958
#> 34 17 4.840462e-04 1.019754 1.049585 1 0 0.004914045
#> 35 17 3.902161e-04 1.019754 1.039898 0 1 0.005736113
#> 36 18 3.143566e-04 1.015907 1.039898 1 0 0.004766941
#> 37 18 2.530454e-04 1.015907 1.032068 0 1 0.005411863
#> 38 19 2.035803e-04 1.012794 1.032068 1 0 0.004764080
#> 39 19 1.636760e-04 1.012794 1.025751 0 1 0.003930807
#> 40 20 1.315375e-04 1.010279 1.025751 1 0 0.004863024
#> 41 20 1.056496e-04 1.010279 1.020663 0 1 0.003813028
#> 42 21 8.482978e-05 1.008251 1.020663 1 0 0.006778002
#> 43 21 6.807922e-05 1.008251 1.016570 0 1 0.003966808
#> 44 22 5.462405e-05 1.006619 1.016570 1 0 0.007411003
#> 45 22 4.380882e-05 1.006619 1.013281 0 1 0.003860950
#> 46 23 3.513011e-05 1.005307 1.013281 1 0 0.006740093
#> 47 23 2.815916e-05 1.005307 1.010641 0 1 0.004112959
#> 48 24 2.257018e-05 1.004252 1.010641 1 0 0.009145975
#> 49 24 1.808332e-05 1.004252 1.008523 0 1 0.004053831
#> 50 25 1.448872e-05 1.003406 1.008523 1 0 0.007412910
#> 51 25 1.160399e-05 1.003406 1.006825 0 1 0.003869057
#> 52 26 9.294548e-06 1.002728 1.006825 1 0 0.004619837
#> 53 26 7.441548e-06 1.002728 1.005463 0 1 0.004005909
#> 54 27 5.959072e-06 1.002184 1.005463 1 0 0.004581928
#> 55 27 4.769667e-06 1.002184 1.004373 0 1 0.006098986
#> 56 28 3.818729e-06 1.001748 1.004373 1 0 0.004686117
#> 57 28 3.055717e-06 1.001748 1.003499 0 1 0.006733894
#> 58 29 2.446111e-06 1.001399 1.003499 1 0 0.004671812
#> 59 29 1.956863e-06 1.001399 1.002800 0 1 0.003676176
#> 60 30 1.566279e-06 1.001119 1.002800 1 0 0.004958153
#> 61 30 1.252688e-06 1.001119 1.002240 0 1 0.004378080
#> 62 31 1.002554e-06 1.000895 1.002240 1 0 0.006283998
#> 63 31 8.016137e-07 1.000895 1.001791 0 1 0.002936840
#>
#> $seconds
#> [1] 0.4485366
#>
#> $stopping_reason
#> [1] "change in function value between 1 iteration is < 1e-06"If you have questions, find a bug, or need a feature, file an issue on GitHub.