I'm using optimx and I find that if I run a loop using the methods Nelder-Mead and BFGS while passing the fitted parameters from one method to the other the objective function value converges to a better solution than if I simply run optimx on either method by itself. Why would this be the case? I already checked the tolerance and iteration limit and I can't seem to find a setting that will give me as good of a result as when I run a loop mixing both methods. The loop takes a long time to run so I would like to avoid this. I'm looking for a suggestion as to what parameters to check to make sure the optimizer gives better parameters than looping through both methods.
Here are other methods I've tried that couldn't give better results than the Nelder-Mead and BFGS mix:
- L-BFGS-B
- CG
- Rcgmin
- Rvmmin
Methods that didn't even converge:
- spg
- nmkb
- nlm
My objective function measures the spread between two time series of continuous values. I can't give any more details for the objective function. This probably limits my answers to general ideas that may not be applicable and I understand if that's the case. Just want to hear thoughts on the issue.
Here is the code that mixes the methods:
myOptimizer<-function(guess, inputmodel, period, data, controlList=NULL){
eps<-0.0001
improvement<-Inf
iteration<-1
while(improvement >= eps){
# Here I make sure I run Nelder-Mead first since it usually yields better values
if(iteration>1){
oldVal <- r$fnValue
# optimizerRun is defined below.
r <- optimizerRun(r$paramList, inputmodel, period, data, controlList, NM_first = FALSE)
improvement<-(oldVal-r$fnValue)/oldVal
}else{
r <- optimizerRun(guess, inputmodel, period, data, controlList, NM_first = TRUE)
}
iteration<-iteration+1
}
return(list(paramList=r$paramList, fnValue=r$fnValue, convergence=r$converge))
}
optimizerRun<-function(parameters, inputmodel, period, data, controlList, NM_first=FALSE){
optimxControl<-controlList$optimx
itnmax<-controlList$myMax
optimxControl$maxit<-itnmax
# Make sure the first method is Nelder-Mead
if(NM_first){
optimal <- optimx(par=parameters, fn = objectiveFun, inputmodel = inputmodel,
period = period, indata = data, method = "Nelder-Mead", itnmax = itnmax, control = optimxControl)
}else{
optimal <- optimx(par=parameters, fn = objectiveFun, inputmodel = inputmodel,
period = period, indata = data, method = c("BFGS", "Nelder-Mead"), itnmax = itnmax, control = optimxControl)
}
# You can ignore this part since this is just me formatting the parameters
###################################################################################
if(length(unique(optimal[,"value"])) > 1){
min_val<-min(optimal[,"value"])==optimal[,"value"]
index<-which(min_val)
}else if(length(unique(optimal[,"value"])) == 1){
index<-NROW(optimal)
}else{
stop("No minimum value found when evaluating optimizer")
}
v<-optimal[index, 1:(attributes(optimal)$npar+1)]
v_params<-as.numeric(v)[-(attributes(optimal)$npar+1)]
names(v_params)<-names(v)[-(attributes(optimal)$npar+1)]
v_val<-as.numeric(v)[attributes(optimal)$npar+1]
###################################################################################
return(list(paramList=v_params, fnValue=v_val, converge=optimal$convcode))
}