Levenberg-Marquardt Algorithm #2

  • Permalink
  • submit to reddit
  • Email
  • Follow


I am trying to write a MATLAB code of the Levenberg-Marquardt Algorithm based on Section III in the paper "Training Feedforward networks with the Marquardt Algorithm" by Matin.T.Hagan. So considering I have a set of training inputs and a set of Target inputs, my question is 

  - How do I calculate the Jacobian Matrix ? I understand that it is a matrix of the partial derivatives of the vector being calculated. But am not sure how do I actually use the Jacobian command that is in MATLAB to compute the jacobian matrix ? .

   hoping somone can help me out. Thank you .
0
Reply manosamuel (35) 1/24/2010 5:01:02 PM

See related articles to this posting


On 24 Jan, 18:01, "Mano Samuel" <manosam...@rediffmail.com> wrote:
> I am trying to write a MATLAB code of the Levenberg-Marquardt Algorithm b=
ased on Section III in the paper "Training Feedforward networks with the Ma=
rquardt Algorithm" by Matin.T.Hagan. So considering I have a set of trainin=
g inputs and a set of Target inputs, my question is
>
> =A0 - How do I calculate the Jacobian Matrix ? I understand that it is a =
matrix of the partial derivatives of the vector being calculated. But am no=
t sure how do I actually use the Jacobian command that is in MATLAB to comp=
ute the jacobian matrix ? .

I would be very surprised if there is such a command.

The Jacobian is a matrix of partial derivatives of the
function you attempt to optimize. If you have an analytic
expression for the fucntion, compute the partial derivatives
analytically and arrange in a matrix.

If you can't find the analytic expressions for the partial
derivatives, you can come up with all kinds of numerical
estimates. But be aware that not all such estimates are
very good. In which case you might prefer to use a method
that does not require the Jacobian at all.

All this is standard material in an intro class on
numerical optimization. Find a textbook and read.

Rune
0
Reply allnor (8506) 1/24/2010 5:57:58 PM

MATLAB has two potentially relevant commands.  The jacobian function
in the symbolic computation toolbox computes a symbolic Jacobian.  The
numjac command uses finite difference approximations to compute the
jacobian of a function at a particular point.
0
Reply borchers.brian (28) 1/24/2010 6:02:52 PM

"Mano Samuel" <manosamuel@rediffmail.com> wrote in message <hjhuce$6fh$1@fred.mathworks.com>...

>   - How do I calculate the Jacobian Matrix ? I understand that it is a matrix of the partial derivatives of the vector being calculated. But am not sure how do I actually use the Jacobian command that is in MATLAB to compute the jacobian matrix ? .
================

jacobian() is a command in the Symbolic Math Toolbox, so first off, you have to have that. If you do have that, its usage is documented here, with an example: 


http://www.mathworks.com/access/helpdesk/help/toolbox/symbolic/jacobian.html
0
Reply mattjacREMOVE (3194) 1/24/2010 6:07:02 PM

"Matt J " <mattjacREMOVE@THISieee.spam> wrote in message <hji286$9u7$1@fred.mathworks.com>...
> "Mano Samuel" <manosamuel@rediffmail.com> wrote in message <hjhuce$6fh$1@fred.mathworks.com>...
> 
> >   - How do I calculate the Jacobian Matrix ? I understand that it is a matrix of the partial derivatives of the vector being calculated. But am not sure how do I actually use the Jacobian command that is in MATLAB to compute the jacobian matrix ? .
> ================
> 
> jacobian() is a command in the Symbolic Math Toolbox, so first off, you have to have that. If you do have that, its usage is documented here, with an example: 
> 
> 
> http://www.mathworks.com/access/helpdesk/help/toolbox/symbolic/jacobian.html

Thank you .. 
0
Reply manosamuel (35) 1/25/2010 9:45:04 AM

"Mano Samuel" wrote in message <hjhuce$6fh$1@fred.mathworks.com>...
> 
> I am trying to write a MATLAB code of the Levenberg-Marquardt Algorithm based on Section III in the paper "Training Feedforward networks with the Marquardt Algorithm" by Matin.T.Hagan. So considering I have a set of training inputs and a set of Target inputs, my question is 
> 
>   - How do I calculate the Jacobian Matrix ? I understand that it is a matrix of the partial derivatives of the vector being calculated. But am not sure how do I actually use the Jacobian command that is in MATLAB to compute the jacobian matrix ? .
> 
>    hoping somone can help me out. Thank you .

Hello Mr.Mano Samuel,

  I am also trying to write own code for   Levenberg-Marquardt Algorithm for feed forward neural network, i hope you would have programmed it so can you please help me out in programming the same. i am new in using matlab so can you please help me out to program for  Levenberg-Marquardt Algorithm

Thanking you

raj
0
Reply prk_mect (15) 4/5/2011 8:09:04 AM
comp.soft-sys.matlab 203856 articles. 540 followers. Post

5 Replies
314 Views

Similar Articles

[PageSpeed] 44


  • Permalink
  • submit to reddit
  • Email
  • Follow


Reply:

Similar Artilces:

Levenberg-Marquardt algorithm
In the Optimization toolbox help on the L-M method (http://www.mathworks.com/help/toolbox/optim/ug/brnoybu.html#f204) it mentions that the "ScaleProblem" option can be used to improve the performance of a poorly scaled non-linear least squares problem by multiplying the Hessian by the lambda parameter. Basically, I am wondering how this helps exactly. Is it a type of matrix normalization? Any reference material which described the LM method from an engineering stand point would be greatly appreciated. Thank you! "LU " <g21u5@unb.removethis.ca> wrote in mes...

Levenberg Marquardt algorithm
Hello, I am wroking on a problem that requires me to use the levenberg marquardt algorithm. I am not a physics major and have never herd of any curve minimization / levemberg in my life before this day. The probelem goes like this: I have a set of points. x - axis : Energy y - axis : log of sensitivity this gives me the initial curve and i make an initial guess. i have a function f Yt = ( ( sqr + 1) * ((1 + exp(alpha*(initialGuess - omega - (energy))))) ); f = log(ys*(1/Yt)); This gives me another set of points based on my initial guess. I have to minimize it. But I have no clue of...

bounds with levenberg-marquardt algorithm
I posted a similar question last week. I am trying to fit mass fractions using lsqcurvefit using the Levenberg-Marquardt algorithm. Since these are mass fractions, I need the values of these fi between 0 and 1, and the sum of the fi to equal 1. I have tried saying: if sum(f) < 1 fi = [f(1) f(2) f(3) f(4) 1-sum(f)] else fi = [f(1) f(2) f(3) f(4) 0]; end for n = 1:length(fi) if fi(n) < 0 fi(n) = 0 end end Often, fi(n) values turn out negative, specifically, fi(5), since sum(f) > 1. Is there a better approach to constrain these parameters (i.e., each individual mas...

Levenberg-Marquardt algorithm #3
Hi Does anyone know if the Levenberg-Marquardt Method uses any regularization process? "Mar" wrote in message <l6o19g$a63$1@newscl01ah.mathworks.com>... > Hi > > Does anyone know if the Levenberg-Marquardt Method uses any regularization process? i dont think so... usually it is mentioned when a solver uses scaling (i assume you mean that). but pls note that more than one procedure use levenberg, so maybe you can be more specific. in lsqnonlin and fsolce you can scale the problem with the jacobian (see the options ...

Modify Levenberg Marquardt algorithm
Dear All, Could anyone kindly let me know if it is possible to modify the levenbergmarquardt.m file so that the values of the parameters being optimized can be printed to the console and modified based on a given condition during optimization? I only tried endlessly to do so via the lsqnonlin routine but to no avail. Besides, I also wanted to find out if a complex model such as F(x,Theta)= theta3*exp(-theta1x2) + theta4*exp(-theta2*x3) would converge for all possible randomly chosen inital conditions? I also generated the values of the x matrix using random numbers. Any help will be gre...

SWEngr-AVP-Algorithm,2-5yrExp.in design & Development of Audio Algorithm #2
Greetings ! Dear Sir / Madam The following positions are needed at urgent for one of our reputed MNC client located at Bangalore . Order No.1 SWEngr-PDE Algorithms,2+Yr exp. inDesign&Dev.Video/Audio/ Image/Speech CODECS. PDE Algorithms Name of the Position: Software Engineer Number of Positions: 03 Qualification : ME/MTech/BE/BTech in Electrical & Electronics/Electronics & Communication/Computer Science & Engineering. Experience : 2 + Years of experience in design and development of Video/Audio/Image/Speech CODECS. Skills Required: Designed and developed...

Re: Levenberg
In my previous post, I did not take into account the problem of constraints on the parameters. The ORTHOREG procedure does not allow one to impose parameter constraints. However, the REG procedure allows one to RESTRICT the values of parameters. From the description provided to this point, I believe that the REG procedure with a RESTRICT statement is all that is necessary to fit the model specified. Dale --------------------------------------- Dale McLerran Fred Hutchinson Cancer Research Center mailto: dmclerra@NO_SPAMfhcrc.org Ph: (206) 667-2926 Fax: (206) 667-5977 ---------------------...

Fitting an equation to data using the Levenberg-Marquardt algorithm, with boundary constraints
Hi everyone. I am using lsqcurvefit to fit a signalequation to some data. I am using the= Levenberg-Marqaurdt algorithm but would really like to put some boundary c= onstraints on the output. The most important constraint is that all the out= put parameters are positive but I would like to constrain the values to cer= tain ranges.=20 lsqcurvefit does not accept boundary constraints for the Levenberg-Marquard= t algorithm, so do any of you have an idea how I can cheat lsqcurvefit to a= ccept boundary conditions indirectly somehow or (even better): do any of yo= u know of another i...

linspace(2,2,2) returns [2 2] ??
May be the help for linspace should change? It says: "LINSPACE(X1, X2, N) generates N points between X1 and X2." Notice, it says BETWEEN. But when I ask for 2 points, between 2 and 2: ---------------- EDU>> linspace(2,2,2) ans = 2 2 --------------- May be mathematically this is ok? i.e. there are infinite points between the point and itself? (on the real line), but from programming point of view, it is confusing. no? --Nasser Maybe it should say "generates N points between X1 and X2, inclusive." What did you expect to...

2 * 2 * 2 * 2 or pow(2, 4)?
Hi , I hope this isn't too trivial to ask, but I'm evaluating a polynomial that involve lots of terms like x^2y^2z^3, and although I can use the power function pow(a, b), I wonder if just writing x*x*y*y*z*z*z for example wouldn't be more efficient than pow(x, 2)*pow(y, 2)*pow(z,3). Or is it all the same? Thanks, - Olumide Olumide wrote: > I hope this isn't too trivial to ask, but I'm evaluating a polynomial > that involve lots of terms like x^2y^2z^3, and although I can use the > power function pow(a, b), I wonder if just writing x*x*y*y*z*z*z for > exam...

2**2**2**2**2 wrong? Bug?
In Python 2.5 on intel, the statement 2**2**2**2**2 evaluates to >>> 2**2**2**2**2 20035299304068464649790723515602557504478254755697514192650169737108940595563114 53089506130880933348101038234342907263181822949382118812668869506364761547029165 04187191635158796634721944293092798208430910485599057015931895963952486337236720 30029169695921561087649488892540908059114570376752085002066715637023661263597471 44807111774815880914135742720967190151836282560618091458852699826141425030123391 10827360384376787644904320596037912449090570756031403507616256247603186379312648 4703743782954975613770...

2^n algorithm for textures #2
For textures with a dimension X that is not a power-of-two, what's the best algorithm for calculating the smallest power-of-two greater than X? I've been using the following, but is there a simpler way? int PowerOfTwo(int x) { int w = 1; while (x != 0) { w <<= 1; x >>= 1; } return w; } ...

Algorithms for X^2 and G^2 Exact Tests?
Does anyone know where I can find the SPSS algorithms for calculating the exact Pearson X^2 and exact Loglikelihood Ratio G^2 tests? I can find algorithms for Fisher's, but not these exact tests? In addition, are there circumstances when these tests can not be relied upon (eg. very low expected frequencies)? Thanks. On 7 Feb 2006 19:27:16 -0800, "LogReg" <mezasocool@hotmail.com> wrote: > Does anyone know where I can find the SPSS algorithms for calculating > the exact Pearson X^2 and exact Loglikelihood Ratio G^2 tests? I can > find algorithms for F...

2.2.2 Annoyance
Caveat: Python 2.2.2 (#37, Oct 14 2002, 17:02:34) Type "help", "copyright", "credits" or "l >>> l = [1,2,3] >>> l[1] 2 >>> l.__getitem__(1) 2 >>> l.__getitem__(slice(1)) Traceback (most recent call last): File "<stdin>", line 1, in ? TypeError: an integer is required >>> According to the manuals of Python 2.2.2, passing a slice object to __getitem__ is all right and should work... Cost a few (grey) hairs of mine, while debugging a largish program, this is not so. Would someone care to pass on th...

VIBE 2.2 #2
About: Visual Information Broker Enterprise (VIBE) is an IDE that enables application developers to automate the design and development of infrastructure components for complex distributed cross-platform, cross-language systems applications. You can define messages and generate infrastructure support code for the IBM MQ information bus, Sybase, Microsoft SQL Server, Oracle, and other JDBC compliant databases, XML with transformations that use tools like XSL, and J2EE. It also includes a fully functional Business Rules engine, along with a graphical programming environment. Changes: This relea...

OTM 2.2 #2
Hi all Just installed OTM on a cust pc (Win XP) Got a problem that when using the system terminal to access Meridian Mail, I am unable to exit from mail as the Cntrl ] function is not recognised. Now is this a problem with OTM or with the customers PC? Thanks Solved it. The term type was set to VT100 and not VT220 ...

what is 4.2.2.2
Hi I came to found from many websites when i had a problem in Internet explorer when if i browse any websites, its showing me "Page cannot be displayed" and in bottom of the page it shows "Cannot find server or DNS Error" Internet Explorer that they advised me to put 4.2.2.2 in my Primary DNS Server. I am in DHCP enabled Network and using windows xp sp2. My question is what is this ip address mean (4.2.2.2) ? Pl explain in detail. Can we give it in our Primary DNS server ? Thanks in advance Chandra -- chandra Message posted via HWKB.com http:...

Printing 2^1, 2^2, 2^3, 2^4 and 2^5
Hi, When I use a while loop, the output is: 2 4 8 16 32, as desired: <?php $c=1; while($c<=5) { $d = newpow(2, $c); print "$d "; $c++; } function newpow($base, $power) { // statements... } ?> The But when I use a for loop, the output is: 64: <?php for($c=1; $c<=5; $c++); { $d = newpow(2, $c); print "$d "; } function newpow($base, $power) { // statements... } ?> Why is this?? Many thanks, Michael mejpark wrote: > When I use a while loop, the output is: 2 4 8 16 32, as desired: [...] > The But when I use a for loop, the output is: 64...

Does 4/2/2= (4/2)/2 =1 OR = 4/(2/2)=4
I don't have matlab and I am trying to translate some simple matlab code. Does 4/2/2 = (4/2)/2 = 1 or 4/2/2 = 4/(2/2) = 4 On 9/11/2012 9:43 AM, sarah.englander@gmail.com wrote: > I don't have matlab and I am trying to translate some simple matlab code. > > Does 4/2/2 = (4/2)/2 = 1 yes. At a given precedence level (and obviously two of the same operators are the same level) evaluation proceeds from LtoR. ....snip... <http://www.mathworks.com/help/techdoc/matlab_prog/f0-40063.html#f0-38155> -- dpb <none@non.net> wrote in message <...

how to do this? #2 #2
Hi, I am working on a program which can do numerical evaluations for integrations. The program I have is like this, double foo(p1, p2) double p1, p2; { double kk,step; int ii; step_sz=(p2-p1)/100.; kk=0.; for (ii=0;ii<100;ii++) kk+=sin(p1+ii*step_sz)*step_sz; return (kk); } This program used the basic Newton method. My question is here in my program foo, I can only calculate the integrations for a sine function. If I want to do an integration for a cosine function, I need to rewrite another function foo1 with cosine being inserted in. I am sure there is a neat way to do this wit...

Levenberg Marquardt
Can anyone recommend a good text that describes exactly and in detail the Levenberg Marquardt Algorithm? Thanks >Richard wrote: > > Can anyone recommend a good text that describes exactly and > in detail the Levenberg Marquardt Algorithm? > > Thanks How about Numerical Recipes? <http://www.library.cornell.edu/nr/bookcpdf/c15-5.pdf> Ken ...

2+2
4 -- ��Dreamweaver, being the #1 pro web design tool, is used by many pros � Different viruses are still different even if in the same "family" � OS X users are at far less risk of malware then are XP users _________________________________________ Usenet Zone Free Binaries Usenet Server More than 140,000 groups Unlimited download http://www.usenetzone.com to open account On Mon, 03 Jul 2006 23:35:36 -0700, Snit <SNIT@CABLE0NE.NET.INVALID> chose to bless us with the following wisdom: >4 Way I love the little girl, you'll never know Come on, baby don't you wan...

2 + 2 =
2 + 2 = 4. Period. ...

Are there algorithm differences between 4.2.4 and 4.2.5?
Are there any algorithm of other differences which might cause noticeable performance differences between 4.2.4 and 4.2.5? For example: - 4.2.5 being more reluctant to increasing the polling interval from 64s (with a mixture of local stratum-1 and Internet pool servers)? - 4.2.5 showing an increased variability of offset when viewed on a timescale of minutes? Of course, these may just be different manifestations of the same underlying difference. Thanks, David David J Taylor has hit on an apparent regression in timekeeping between 4.2.4 and 4.2.5. He's posted some graphs at: ...