COMPGROUPS.NET | Search | Post Question | Groups | Stream | About | Register

### rescaling for fmincon

• Email
• Follow

```I am running a least squares minimization with fmincon to solve a least squares problem.  Essentially, a 5-tuple of parameters maps to an 8-tuple of percentages.  The parameters are of the form (mean, 1/variance1, 1/variance2, 1/variance3, constant scaled to mean).  Unfortunately, it seems that the sum of squares function is very flat around the minimum over a pretty big range of variance1,2,3 and constant.

I have rescaled the problem so fmincon inputs (mean, s.d.1, s.d.2, s.d.3, constant) with a neutral starting guess of [0,1,1,1,0].  I run fmincon, and then rescale by the last guess so that fmincon starts at [1,1,1,1,1] and minimizes new_guess.*(mean, s.d.1, s.d.2, s.d.3, constant).  Is this an good way to deal with flat functions?  Is there a better way?  I'm fairly new to modeling, so I apologize if this is trivial.
```
 0

See related articles to this posting

```Your procedure seems OK.

However, I wonder why you don't use one of the least squares solvers,
such as lsqnonlin, which can do a better job of overcoming the
well-known problem of flatness near the solution. Perhaps you need some
nonlinear constraints. But if you only need bounds, try lsqnonlin.
http://www.mathworks.com/access/helpdesk/help/toolbox/optim/ug/brhkghv-18.html#brhkghv-19

Alan Weiss
MATLAB mathematical toolbox documentation

On 7/27/2010 3:10 PM, Samuel Edwards wrote:
> I am running a least squares minimization with fmincon to solve a least
> squares problem. Essentially, a 5-tuple of parameters maps to an 8-tuple
> of percentages. The parameters are of the form (mean, 1/variance1,
> 1/variance2, 1/variance3, constant scaled to mean). Unfortunately, it
> seems that the sum of squares function is very flat around the minimum
> over a pretty big range of variance1,2,3 and constant.
>
> I have rescaled the problem so fmincon inputs (mean, s.d.1, s.d.2,
> s.d.3, constant) with a neutral starting guess of [0,1,1,1,0]. I run
> fmincon, and then rescale by the last guess so that fmincon starts at
> [1,1,1,1,1] and minimizes new_guess.*(mean, s.d.1, s.d.2, s.d.3,
> constant). Is this an good way to deal with flat functions? Is there a
> better way? I'm fairly new to modeling, so I apologize if this is trivial.

```
 0

```"Samuel Edwards" <DJeter1234@AOL.com> wrote in message <i2nauv\$bfo\$1@fred.mathworks.com>...

> I have rescaled the problem so fmincon inputs (mean, s.d.1, s.d.2, s.d.3, constant) with a neutral starting guess of [0,1,1,1,0].  I run fmincon, and then rescale by the last guess so that fmincon starts at [1,1,1,1,1] and minimizes new_guess.*(mean, s.d.1, s.d.2, s.d.3, constant).  Is this an good way to deal with flat functions?  Is there a better way?  I'm fairly new to modeling, so I apologize if this is trivial.
==============

The ideal way to correct for bad scaling is to supply a user-defined analytical Hessian to fmincon. Since this is only a 5-parameter problem, that doesn't seem to be a very difficult thing to do.

A cheaper knock-off would be to evaluate the Hessian at your neutral starting guess and use the  diagonal of its inverse as scale factors.
```
 0

```Thanks for the input.  I had tried running iterated lsqnonlin, but for some reason or another it seems to converge on a worse answer that the iterated fmincon, albeit more quickly.  The difference in sum squared is very small, but the difference in estimated parameters is large.  Neither seems to find and actual minimum, because increasing FunTol and increasing the iterations find better answers.

Alan Weiss <aweiss@mathworks.com> wrote in message <i2ndqb\$k7c\$1@fred.mathworks.com>...
>
> However, I wonder why you don't use one of the least squares solvers,
> such as lsqnonlin, which can do a better job of overcoming the
> well-known problem of flatness near the solution. Perhaps you need some
> nonlinear constraints. But if you only need bounds, try lsqnonlin.
> http://www.mathworks.com/access/helpdesk/help/toolbox/optim/ug/brhkghv-18.html#brhkghv-19
>
> Alan Weiss
> MATLAB mathematical toolbox documentation
>
> On 7/27/2010 3:10 PM, Samuel Edwards wrote:
> > I am running a least squares minimization with fmincon to solve a least
> > squares problem. Essentially, a 5-tuple of parameters maps to an 8-tuple
> > of percentages. The parameters are of the form (mean, 1/variance1,
> > 1/variance2, 1/variance3, constant scaled to mean). Unfortunately, it
> > seems that the sum of squares function is very flat around the minimum
> > over a pretty big range of variance1,2,3 and constant.
> >
> > I have rescaled the problem so fmincon inputs (mean, s.d.1, s.d.2,
> > s.d.3, constant) with a neutral starting guess of [0,1,1,1,0]. I run
> > fmincon, and then rescale by the last guess so that fmincon starts at
> > [1,1,1,1,1] and minimizes new_guess.*(mean, s.d.1, s.d.2, s.d.3,
> > constant). Is this an good way to deal with flat functions? Is there a
> > better way? I'm fairly new to modeling, so I apologize if this is trivial.
```
 0

```"Samuel Edwards" <DJeter1234@AOL.com> wrote in message <i2nauv\$bfo\$1@fred.mathworks.com>...
> I am running a least squares minimization with fmincon to solve a least squares problem.  Essentially, a 5-tuple of parameters maps to an 8-tuple of percentages.  The parameters are of the form (mean, 1/variance1, 1/variance2, 1/variance3, constant scaled to mean).  Unfortunately, it seems that the sum of squares function is very flat around the minimum over a pretty big range of variance1,2,3 and constant.
========

In addition to my last point, is there a reason that you are parametrizing using
1/variance instead of the variance, or better yet the standard deviation, itself? By taking 1/x, you are diminishing the sensitivity of the objective function to variations in x when x is large. This could easily contribute to scaling problems.
```
 0

```Sorry if that was unclear, the objective function m-file takes 1/variance as the input, but i define an anonymous function that takes s.d. as an input and feed this to the solver.  In retrospect this information was irrelevant.

"Matt J " <mattjacREMOVE@THISieee.spam> wrote in message <i2nk6k\$k6b\$1@fred.mathworks.com>...
> "Samuel Edwards" <DJeter1234@AOL.com> wrote in message <i2nauv\$bfo\$1@fred.mathworks.com>...
> > I am running a least squares minimization with fmincon to solve a least squares problem.  Essentially, a 5-tuple of parameters maps to an 8-tuple of percentages.  The parameters are of the form (mean, 1/variance1, 1/variance2, 1/variance3, constant scaled to mean).  Unfortunately, it seems that the sum of squares function is very flat around the minimum over a pretty big range of variance1,2,3 and constant.
> ========
>
> In addition to my last point, is there a reason that you are parametrizing using
> 1/variance instead of the variance, or better yet the standard deviation, itself? By taking 1/x, you are diminishing the sensitivity of the objective function to variations in x when x is large. This could easily contribute to scaling problems.
```
 0

```"Matt J " <mattjacREMOVE@THISieee.spam> wrote in message <i2nig9\$1qp\$1@fred.mathworks.com>...
> "Samuel Edwards" <DJeter1234@AOL.com> wrote in message <i2nauv\$bfo\$1@fred.mathworks.com>...
>
> > I have rescaled the problem so fmincon inputs (mean, s.d.1, s.d.2, s.d.3, constant) with a neutral starting guess of [0,1,1,1,0].  I run fmincon, and then rescale by the last guess so that fmincon starts at [1,1,1,1,1] and minimizes new_guess.*(mean, s.d.1, s.d.2, s.d.3, constant).  Is this an good way to deal with flat functions?  Is there a better way?  I'm fairly new to modeling, so I apologize if this is trivial.
> ==============
>
> The ideal way to correct for bad scaling is to supply a user-defined analytical Hessian to fmincon. Since this is only a 5-parameter problem, that doesn't seem to be a very difficult thing to do.
>
> A cheaper knock-off would be to evaluate the Hessian at your neutral starting guess and use the  diagonal of its inverse as scale factors.

Doesn't fmincon approximate the Hessian automatically?  How would I approximate it better?  I do not have the actual gradient to the problem, and it is impossible to solve analytically.
```
 0

```"Samuel Edwards" <DJeter1234@AOL.com> wrote in message <i2no52\$3oi\$1@fred.mathworks.com>...

> Doesn't fmincon approximate the Hessian automatically?  How would I approximate it better?  I do not have the actual gradient to the problem, and it is impossible to solve analytically.
==================

If your problem is poorly scaled, fmincon's numerical Hessian approximations might not be very accurate. If they were accurate, there shouldn't be much of a scaling problem, except in regions of the function where the Hessian is not strictly positive definite.

Why is the function so hard to differentiate?
```
 0

```"Matt J " <mattjacREMOVE@THISieee.spam> wrote in message <i2o5ds\$bd6\$1@fred.mathworks.com>...
> "Samuel Edwards" <DJeter1234@AOL.com> wrote in message <i2no52\$3oi\$1@fred.mathworks.com>...
>
> > Doesn't fmincon approximate the Hessian automatically?  How would I approximate it better?  I do not have the actual gradient to the problem, and it is impossible to solve analytically.
> ==================
>
> If your problem is poorly scaled, fmincon's numerical Hessian approximations might not be very accurate. If they were accurate, there shouldn't be much of a scaling problem, except in regions of the function where the Hessian is not strictly positive definite.
>
> Why is the function so hard to differentiate?

It's a numerical assessment of an 2-d integral defined over 2 regions, where the bounds of the second region are zeros of a function that 1) has a discontinuous derivative and 2) has has a standard normal cumulative density and probability density term involving the coefficients.
```
 0

```"Samuel Edwards" <DJeter1234@AOL.com> wrote in message <i2pu5h\$t78\$1@fred.mathworks.com>...

>
> It's a numerical assessment of an 2-d integral defined over 2 regions, where the bounds of the second region are zeros of a function that 1) has a discontinuous derivative and 2) has has a standard normal cumulative density and probability density term involving the coefficients.
===============

OK, that does sound pretty hard, but what about my other question? Why write the function in terms of expressions 1/variance? This will make the function insensitive to incremental changes in large variance values. Why not define alternative parameters

p1,2,3=1/variance1,2,3

and express the function in terms of p1,2,3
```
 0

```"Matt J " <mattjacREMOVE@THISieee.spam> wrote in message <i2q461\$76k\$1@fred.mathworks.com>...
> "Samuel Edwards" <DJeter1234@AOL.com> wrote in message <i2pu5h\$t78\$1@fred.mathworks.com>...
>
> >
> > It's a numerical assessment of an 2-d integral defined over 2 regions, where the bounds of the second region are zeros of a function that 1) has a discontinuous derivative and 2) has has a standard normal cumulative density and probability density term involving the coefficients.
> ===============
>
> OK, that does sound pretty hard, but what about my other question? Why write the function in terms of expressions 1/variance? This will make the function insensitive to incremental changes in large variance values. Why not define alternative parameters
>
> p1,2,3=1/variance1,2,3
>
> and express the function in terms of p1,2,3

Sorry if that was unclear, the objective function m-file takes 1/variance as the input, but i define an anonymous function that takes s.d. as an input and feed this to the solver.  So fmincon should be taking steps/gradients/hessians in terms of s.d.  There may be a small amount of noise in the function because the integral/bounds are only an approximation, although if so it's not visible when i plot the area around a "minimum."  However, I think it's probably poor scaling, but because I need this program to run automatically, I can't think of a better way to rescale than minimize sum(([1;1;1;1;1].*previous best guess-actual).^2).  I need it to run automatically because the actual data has a somewhat low n, and I'm attempting to see what the optimization recovers from other data-sets that could have been generated by the "true" coefficients.
```
 0

```"Samuel Edwards" <DJeter1234@AOL.com> wrote in message <i2q59p\$ki5\$1@fred.mathworks.com>...

> Sorry if that was unclear, the objective function m-file takes 1/variance as the input, but i define an anonymous function that takes s.d. as an input and feed this to the solver.  So fmincon should be taking steps/gradients/hessians in terms of s.d.  There may be a small amount of noise in the function because the integral/bounds are only an approximation, although if so it's not visible when i plot the area around a "minimum."  However, I think it's probably poor scaling, but because I need this program to run automatically, I can't think of a better way to rescale than minimize sum(([1;1;1;1;1].*previous best guess-actual).^2).  I need it to run automatically because the actual data has a somewhat low n, and I'm attempting to see what the optimization recovers from other data-sets that could have been generated by the "true" coefficients.
====================

My only other idea is that you create a simulated, but realistic,  version of your parameter estimation problem where you know the ground truth parameter values. You could then do a numerical computation of the Hessian at the known global minimum (i.e. at the groud truth parameters). You could then use this Hessian as the basis for your parameter scaling.
```
 0

11 Replies
456 Views

Similar Articles

12/6/2013 11:17:32 PM
page loaded in 2709 ms. (0)

Similar Artilces:

fmincon Prolem
Hello everybody, i'm writing my thesis at the moment and i have a problem with fmincon. I have nonlinear constraints ( state constraints and control constraints including a rather complicated dynamic equality constraint of a car) and i have to find the optimal solution for the state vector with a direct multiple shooting algorithm. (The state vector consists of: x1 = x-Positon x2 = y-Positon x3 = velocity of the car x4 = orientation angle in reference to the world coordinate system x5 = yaw angle x6 = yaw rate ) This algorithm uses fmincon. The resulting trajectory... objective and constraint functions are smooth analytical functions, but are way too small when these functions are the response of a simulation. The accuracy of your gradient approximations has an obvious impact on first-order optimality metrics. -James Magnus G wrote: > Hello everybody, > > i'm writing my thesis at the moment and i have a problem with fmincon. I > have nonlinear constraints ( state constraints and control constraints > including a rather complicated dynamic equality constraint of a car) and > i have to find the optimal solution

CONSTRAINTS FOR FMINCON
Using fmincon, the objective function (in my case -V'*x) can be inserted simply in the code: x=fmincon(@(x) -V'*x,zeros(5,1),A,b,Aeq,beq,lb,ub, NON LINEAR CONSTRAINTS?) but i didn't manage to do the same with the non linear constraints, i tried for example x=fmincon(@(x) -V'*x,zeros(5,1),A,b,Aeq,beq,lb,ub,@(x) [[],sum(x>0)-2]) but it does not work... can You help me? thanks "Isaac " <asimov4000@hotmail.com> wrote in message <hdrfij\$64j\$1@fred.mathworks.com>... > Using fmincon, the objective function (in my case -V'*x) can be inserted simply in the code: > > x=fmincon(@(x) -V'*x,zeros(5,1),A,b,Aeq,beq,lb,ub, NON LINEAR CONSTRAINTS?) > > but i didn't manage to do the same with the non linear constraints, i tried for example > > x=fmincon(@(x) -V'*x,zeros(5,1),A,b,Aeq,beq,lb,ub,@(x) [[],sum(x>0)-2]) > > but it does not work... > > can You help me? An anonymous function cannot return TWO output arguments. Use a regular m-file for that part. John "John D'Errico" <woodchips@rochester.rr.com> wrote in message <hdrgks\$bed\$1@fred.mathworks.com>... >

exemple Fmincon
Hi, I have a probleme to use "Fmincon". i want u to send me same exemple. and if u have same munite, tanks to explain me how to use it. tanks lot. Why do you not try 'doc fmincon' ? imane wrote: > Hi, > I have a probleme to use "Fmincon". > i want u to send me same exemple. > and if u have same munite, tanks to explain me how to use it. > > tanks lot. The Getting Started example is here: http://www.mathworks.com/access/helpdesk/help/toolbox/optim/ug/brg0p3g-1.html You will find many more worked examples here: http://www.mathworks.com/access/helpdesk/help/toolbox/optim/ug/brn4nh7.html Alan Weiss MATLAB mathematical toolbox documentation

fmincon and TolCon
Hello I am trying to use fmincon to solve an engineering problem, where the optimization variables are not explicitly written in any of the constraints nor the objective function. The problem is working well because I can introduce the optimization results into a simulator and double check that the system can actually operate in reality. Now, the problem that I am having is that before using fmincon to solve a part of the problem I was using lsqnonlin. lsqnonlin was working well, but with there was no way to guarantee that the residual is less than a required tolerance, so it will need several iterations before it converged to the value I needed. To speed the problem up I decided to switch to fmincon and use the same objective function that lsqnonlin uses, but twitching it a little to make it converge to the tolerance is less steps. All the changes that I was introducing to my code were working well, until today. The last change (that is very similar to another one that I had previously introduced) gave me this answer: Initial point is a local minimum that satisfies the constraints. Optimization completed because at the initial point, the objective function is non

avoiding NaN in fmincon?
Hi, I am running fmincon on a rather complicated function of matrix argument. I have to run it for different data sets. Now it happens that for some data sets and some region of the parameter space the objective can turn out to be NaN. The problem is that in this case fmincon basically gets stuck in that region and only terminates when it runs out of 'MaxFunEvals'. I want to be able to : a) detect when the objective becomes NaN b) get fmincon to terminate when it is NaN and c) exit with the last values of the parameters that gave that NaN so that I can better understand the problem. Any help on how to do this would be really appreciated. thanks, matt. Try this: data = [1 2 3 nan 5] isnan(data) If you want to replace the nan with any value use: data(isnan(data)) = 0 Hi Brian, It is not the data but the objective function which is being minimized by fmincon which can turn to NaN. How do I get fmincon to recognize it? Is there anything like if isnan(objective) then "exit fmincon and return to main program..." thanks, matt Brian Jensen wrote: > > > Try this: > > data = [1 2 3 nan 5] > isnan(data) > > If you want to replace the nan

Using fmincon to optimise vectors?
Hello, I have an optimization problem which contains two unknown variable vectors [x(1) and x(2)] I have tried using fmincon to find an optimum for these two vectors but the tool does not seem to be able to do it. My objective function would usually be a vector of the same size as the unknown variable vectors but to turn this into a single scalar I have taken sum(objective function) to be my new objective function f. The script runs but feasible solutions are not found and the optimiser routine exits either with: Optimization terminated: magnitude of search direction less than 2*options.TolX and maximum constraint violation is less than options.TolCon. Active inequalities (to within options.TolCon = 1e-006): lower upper ineqlin ineqnonlin 1 2 or max no. of iterations exceeded. Is there a way to use fmincon correctly to solve my problem or do I need to use a different approach. Thanks John In article <ef1c369.-1@webx.raydaftYaTP>, "John Chapman" <johncchapman@rokstudent.com> wrote: > I have an optimization problem which contains two unknown variable > vectors [x(1) and x(2)] I have tried using fmincon to find an optimum

setting bounds for fmincon
hi, I am new to MATLAB and I want to use the function fmincon to estimate 8 parameters, let us say for example: erp=[a1 a2 a3 a4 a5 a6 a7 a8] However i want the parameters to be bounded in such a way that a1>a2>a3>a4>a5>a6>a7>a8 Is there a way that I can do this? I would be really grateful. Thanks in advance. On 4 Apr., 09:12, "Nihal " <nihal.i...@gmail.com> wrote: > hi, > I am new to MATLAB and I want to use the function fmincon to estimate 8 parameters, let us say for example: > > erp=[a1 a2 a3 a4 a5 a6 a7 a8] > > However i want the parameters to be bounded in such a way that > a1>a2>a3>a4>a5>a6>a7>a8 > > Is there a way that I can do this? > > I would be really grateful. > Thanks in advance. a2-a1<=0 a3-a2<=0 a4-a3<=0 a5-a4<=0 a6-a5<=0 a7-a6<=0 a8-a7<=0 You can set these conditions as linear constraints (A*x <=b) by choosing A and b accordingly. Best wishes Torsten. you should put it in the format A*erp<=b so you can pass it that way (see help on fmincon) So: a1-a2<=0; a2-a3<=0; et cetera. So if these were your only equations (erp=[a1 a2 a3

Fmincon with system of nonlinear inequalities
Hi Everybody, I have an optimization problem and i use fmincon to solve it. I have three different nonlinear system of inequalities as my constraints. 1- f(x,y)>R1 2- g(x,y)>R2 3- x^2+y^2<=1 where f(x,y)>R1 and g(x,y)>R2 are two systems of nonlinear inequalities. x and y are unknown parameters and R1 and R2 are known. It is easy to write first and second constraints in fmincon by using nonlinear constraints option. However in my problem my constraints are complement of 1 and 2. So, constraint that i want to write are 1- (f(x,y)>R1)^c 2- (g(x,y)>R2)^c 3- x^2+y^2<=1 where A^c is the complement set of set A. I am a little confused about writing complement of a set in fmincon as nonlinear constraints. Any suggestion for this issue? Regards "kamuran turksoy" <kamuranturksoy@gmail.com> wrote in message <k1jc76\$eqf\$1@newscl01ah.mathworks.com>... > > I am a little confused about writing complement of a set in fmincon as nonlinear constraints. > ================ But isn't it true that {f(x,y)>R1}^c = {f(x,y)<=R1} And isn't f(x,y)<=R1 easy to express in FMINCON

fmincon with whole numbers
My question is simple. Is it possible to optimize with fmincon where the minimizer x is constrained to be a whole number. I can surely a put a round() in my objective function, but that will surely waste a lot of optimization iterations! Details: I am trying to do mutual information registration using fmincon where the transformation is limited to a translation. Currently my method works fine with fmincon, except that its quite slow. The reason for this sluggishness is that most of the time fmincon passes an x (the translation parameters) in fractions. These fractional translations, understandably, take a large of time in resampling/interpolation of pixel values. The reason why I am asking for an fmincon with whole numbers is that I want to run this algorithm in 2 steps. In the first step, the optimization should find the best registration with a pixel level accuracy - where I will be able to use a simple translation without any resampling. In the second step, the optimization should find translation to a sub-pixel accuracy - here I will use the slower bicubic resampling for translating the image. But note, since we had already found the best registration to a pixel level

The DerivativeCheck option used in fmincon
Hi, I'm using the fmincon command in my Matlab program to find some minimum of an nonlinear objective function. The constraints are all nonlinear equalities. Furthermore, I have the analytical gradients for both the objective function and all the constraints. To verify whether the gradients are correct, I turned on the DerivativeCheck option using the optimset command. When I run my program, however, I got the warning message that goes like the following: --------------------------------------------------------------------------- Function derivative Maximum discrepancy... function that are on the order of tens of millions. Still, the DerivativeCheck sends a warning saying this discrepancy exceeds the tolerance. I double-checked all the gradients carefully and didn't find anything wrong. But this warning message keeps popping up everytime I run the program. But, the result produced fmincon seems to be correct. My questions are the following: 1)Is it possible for fmincon to produce the correct local minimum and yet still there is some large discrepancy between the finite-difference values and the gradient values that exceeds the tolerance? 2)Which