Levenberg-Marquardt Algorithm #2

  • Permalink
  • submit to reddit
  • Email
  • Follow


I am trying to write a MATLAB code of the Levenberg-Marquardt Algorithm based on Section III in the paper "Training Feedforward networks with the Marquardt Algorithm" by Matin.T.Hagan. So considering I have a set of training inputs and a set of Target inputs, my question is 

  - How do I calculate the Jacobian Matrix ? I understand that it is a matrix of the partial derivatives of the vector being calculated. But am not sure how do I actually use the Jacobian command that is in MATLAB to compute the jacobian matrix ? .

   hoping somone can help me out. Thank you .
0
Reply manosamuel (35) 1/24/2010 5:01:02 PM

See related articles to this posting


On 24 Jan, 18:01, "Mano Samuel" <manosam...@rediffmail.com> wrote:
> I am trying to write a MATLAB code of the Levenberg-Marquardt Algorithm b=
ased on Section III in the paper "Training Feedforward networks with the Ma=
rquardt Algorithm" by Matin.T.Hagan. So considering I have a set of trainin=
g inputs and a set of Target inputs, my question is
>
> =A0 - How do I calculate the Jacobian Matrix ? I understand that it is a =
matrix of the partial derivatives of the vector being calculated. But am no=
t sure how do I actually use the Jacobian command that is in MATLAB to comp=
ute the jacobian matrix ? .

I would be very surprised if there is such a command.

The Jacobian is a matrix of partial derivatives of the
function you attempt to optimize. If you have an analytic
expression for the fucntion, compute the partial derivatives
analytically and arrange in a matrix.

If you can't find the analytic expressions for the partial
derivatives, you can come up with all kinds of numerical
estimates. But be aware that not all such estimates are
very good. In which case you might prefer to use a method
that does not require the Jacobian at all.

All this is standard material in an intro class on
numerical optimization. Find a textbook and read.

Rune
0
Reply allnor (8506) 1/24/2010 5:57:58 PM

MATLAB has two potentially relevant commands.  The jacobian function
in the symbolic computation toolbox computes a symbolic Jacobian.  The
numjac command uses finite difference approximations to compute the
jacobian of a function at a particular point.
0
Reply borchers.brian (28) 1/24/2010 6:02:52 PM

"Mano Samuel" <manosamuel@rediffmail.com> wrote in message <hjhuce$6fh$1@fred.mathworks.com>...

>   - How do I calculate the Jacobian Matrix ? I understand that it is a matrix of the partial derivatives of the vector being calculated. But am not sure how do I actually use the Jacobian command that is in MATLAB to compute the jacobian matrix ? .
================

jacobian() is a command in the Symbolic Math Toolbox, so first off, you have to have that. If you do have that, its usage is documented here, with an example: 


http://www.mathworks.com/access/helpdesk/help/toolbox/symbolic/jacobian.html
0
Reply mattjacREMOVE (3194) 1/24/2010 6:07:02 PM

"Matt J " <mattjacREMOVE@THISieee.spam> wrote in message <hji286$9u7$1@fred.mathworks.com>...
> "Mano Samuel" <manosamuel@rediffmail.com> wrote in message <hjhuce$6fh$1@fred.mathworks.com>...
> 
> >   - How do I calculate the Jacobian Matrix ? I understand that it is a matrix of the partial derivatives of the vector being calculated. But am not sure how do I actually use the Jacobian command that is in MATLAB to compute the jacobian matrix ? .
> ================
> 
> jacobian() is a command in the Symbolic Math Toolbox, so first off, you have to have that. If you do have that, its usage is documented here, with an example: 
> 
> 
> http://www.mathworks.com/access/helpdesk/help/toolbox/symbolic/jacobian.html

Thank you .. 
0
Reply manosamuel (35) 1/25/2010 9:45:04 AM

"Mano Samuel" wrote in message <hjhuce$6fh$1@fred.mathworks.com>...
> 
> I am trying to write a MATLAB code of the Levenberg-Marquardt Algorithm based on Section III in the paper "Training Feedforward networks with the Marquardt Algorithm" by Matin.T.Hagan. So considering I have a set of training inputs and a set of Target inputs, my question is 
> 
>   - How do I calculate the Jacobian Matrix ? I understand that it is a matrix of the partial derivatives of the vector being calculated. But am not sure how do I actually use the Jacobian command that is in MATLAB to compute the jacobian matrix ? .
> 
>    hoping somone can help me out. Thank you .

Hello Mr.Mano Samuel,

  I am also trying to write own code for   Levenberg-Marquardt Algorithm for feed forward neural network, i hope you would have programmed it so can you please help me out in programming the same. i am new in using matlab so can you please help me out to program for  Levenberg-Marquardt Algorithm

Thanking you

raj
0
Reply prk_mect (15) 4/5/2011 8:09:04 AM
comp.soft-sys.matlab 203507 articles. 529 followers. Post

5 Replies
285 Views

Similar Articles

[PageSpeed] 36


  • Permalink
  • submit to reddit
  • Email
  • Follow


Reply:

Similar Artilces:

Levenberg-Marquardt algorithm
In the Optimization toolbox help on the L-M method (http://www.mathworks.com/help/toolbox/optim/ug/brnoybu.html#f204) it mentions that the "ScaleProblem" option can be used to improve the performance of a poorly scaled non-linear least squares problem by multiplying the Hessian by the lambda parameter. Basically, I am wondering how this helps exactly. Is it a type of matrix normalization? Any reference material which described the LM method from an engineering stand point would be greatly appreciated. Thank you! "LU " <g21u5@unb.removethis.ca> wrote in mes...

Levenberg Marquardt algorithm
Hello, I am wroking on a problem that requires me to use the levenberg marquardt algorithm. I am not a physics major and have never herd of any curve minimization / levemberg in my life before this day. The probelem goes like this: I have a set of points. x - axis : Energy y - axis : log of sensitivity this gives me the initial curve and i make an initial guess. i have a function f Yt = ( ( sqr + 1) * ((1 + exp(alpha*(initialGuess - omega - (energy))))) ); f = log(ys*(1/Yt)); This gives me another set of points based on my initial guess. I have to minimize it. But I have no clue of...

bounds with levenberg-marquardt algorithm
I posted a similar question last week. I am trying to fit mass fractions using lsqcurvefit using the Levenberg-Marquardt algorithm. Since these are mass fractions, I need the values of these fi between 0 and 1, and the sum of the fi to equal 1. I have tried saying: if sum(f) < 1 fi = [f(1) f(2) f(3) f(4) 1-sum(f)] else fi = [f(1) f(2) f(3) f(4) 0]; end for n = 1:length(fi) if fi(n) < 0 fi(n) = 0 end end Often, fi(n) values turn out negative, specifically, fi(5), since sum(f) > 1. Is there a better approach to constrain these parameters (i.e., each individual mas...

Levenberg-Marquardt algorithm #3
Hi Does anyone know if the Levenberg-Marquardt Method uses any regularization process? "Mar" wrote in message <l6o19g$a63$1@newscl01ah.mathworks.com>... > Hi > > Does anyone know if the Levenberg-Marquardt Method uses any regularization process? i dont think so... usually it is mentioned when a solver uses scaling (i assume you mean that). but pls note that more than one procedure use levenberg, so maybe you can be more specific. in lsqnonlin and fsolce you can scale the problem with the jacobian (see the options ...

Modify Levenberg Marquardt algorithm
Dear All, Could anyone kindly let me know if it is possible to modify the levenbergmarquardt.m file so that the values of the parameters being optimized can be printed to the console and modified based on a given condition during optimization? I only tried endlessly to do so via the lsqnonlin routine but to no avail. Besides, I also wanted to find out if a complex model such as F(x,Theta)= theta3*exp(-theta1x2) + theta4*exp(-theta2*x3) would converge for all possible randomly chosen inital conditions? I also generated the values of the x matrix using random numbers. Any help will be gre...

Re: Levenberg
In my previous post, I did not take into account the problem of constraints on the parameters. The ORTHOREG procedure does not allow one to impose parameter constraints. However, the REG procedure allows one to RESTRICT the values of parameters. From the description provided to this point, I believe that the REG procedure with a RESTRICT statement is all that is necessary to fit the model specified. Dale --------------------------------------- Dale McLerran Fred Hutchinson Cancer Research Center mailto: dmclerra@NO_SPAMfhcrc.org Ph: (206) 667-2926 Fax: (206) 667-5977 ---------------------...

Fitting an equation to data using the Levenberg-Marquardt algorithm, with boundary constraints
Hi everyone. I am using lsqcurvefit to fit a signalequation to some data. I am using the= Levenberg-Marqaurdt algorithm but would really like to put some boundary c= onstraints on the output. The most important constraint is that all the out= put parameters are positive but I would like to constrain the values to cer= tain ranges.=20 lsqcurvefit does not accept boundary constraints for the Levenberg-Marquard= t algorithm, so do any of you have an idea how I can cheat lsqcurvefit to a= ccept boundary conditions indirectly somehow or (even better): do any of yo= u know of another i...

SWEngr-AVP-Algorithm,2-5yrExp.in design & Development of Audio Algorithm #2
Greetings ! Dear Sir / Madam The following positions are needed at urgent for one of our reputed MNC client located at Bangalore . Order No.1 SWEngr-PDE Algorithms,2+Yr exp. inDesign&Dev.Video/Audio/ Image/Speech CODECS. PDE Algorithms Name of the Position: Software Engineer Number of Positions: 03 Qualification : ME/MTech/BE/BTech in Electrical & Electronics/Electronics & Communication/Computer Science & Engineering. Experience : 2 + Years of experience in design and development of Video/Audio/Image/Speech CODECS. Skills Required: Designed and developed...

2^n algorithm for textures #2
For textures with a dimension X that is not a power-of-two, what's the best algorithm for calculating the smallest power-of-two greater than X? I've been using the following, but is there a simpler way? int PowerOfTwo(int x) { int w = 1; while (x != 0) { w <<= 1; x >>= 1; } return w; } ...

Algorithms for X^2 and G^2 Exact Tests?
Does anyone know where I can find the SPSS algorithms for calculating the exact Pearson X^2 and exact Loglikelihood Ratio G^2 tests? I can find algorithms for Fisher's, but not these exact tests? In addition, are there circumstances when these tests can not be relied upon (eg. very low expected frequencies)? Thanks. On 7 Feb 2006 19:27:16 -0800, "LogReg" <mezasocool@hotmail.com> wrote: > Does anyone know where I can find the SPSS algorithms for calculating > the exact Pearson X^2 and exact Loglikelihood Ratio G^2 tests? I can > find algorithms for F...

Levenberg Marquardt
Can anyone recommend a good text that describes exactly and in detail the Levenberg Marquardt Algorithm? Thanks >Richard wrote: > > Can anyone recommend a good text that describes exactly and > in detail the Levenberg Marquardt Algorithm? > > Thanks How about Numerical Recipes? <http://www.library.cornell.edu/nr/bookcpdf/c15-5.pdf> Ken ...

Are there algorithm differences between 4.2.4 and 4.2.5?
Are there any algorithm of other differences which might cause noticeable performance differences between 4.2.4 and 4.2.5? For example: - 4.2.5 being more reluctant to increasing the polling interval from 64s (with a mixture of local stratum-1 and Internet pool servers)? - 4.2.5 showing an increased variability of offset when viewed on a timescale of minutes? Of course, these may just be different manifestations of the same underlying difference. Thanks, David David J Taylor has hit on an apparent regression in timekeeping between 4.2.4 and 4.2.5. He's posted some graphs at: ...

algorithms #2
I Would appreciate it if anyone who is knowledgable about algorithm could point me to a good tutorial or two. Yes I could use google and come up with all kinds of stuff but I've found the best tutorials come from those who have experience. I want to write a little program to print all possible combinations of words or characters from a phone if you enter the numbers of your phone number. For this I could use a struct, memory allocation or arrays in C. I can study it out but I think I should be learning a few algorithms now. I hope I am clear. THanks, Bill I don't q...

algorithm #2
hello I have the contour of a polygon and i want to detect the coordinates of all "summit" (intersection of two sides). The contour is a set of point X, Y and a set of code of Freeman (0 to 7). How can I retrieve the coordinates (x,y) of each summit polygon. I haven't any idea. Thank you for your help. Philippe ...

ALGORITHM #2
hi all. ..i know how to calculate the sum of divisors of a number as well know how to calculate a^b mod n; but here question is to calculate S % mod n where S = is the sum of divisors of a ^ b. a and b are very large...but can fit in int range..but its multiplication can not fit... give me a nice algo..i m stuck frm past 6 hrs...!!! $u!fur wrote: > hi all. ..i know how to calculate the sum of divisors of a number as > well know how to calculate a^b mod n; > but here question is to calculate S % mod n where S = is the sum of > divisors of a ^ b. What's the meaning of ^ in yo...

ALGORITHM #2
// MATCHCASE // REPLACE1WITHLOREPLACE2WITH+ // REPLACE WITH2 REPLACE2P2WITH2LZ REPLACE2NP2WITH2L0 ...

levenberg Marquardt
Hi, &nbsp; I want to use the Levenberg-Marquardt algorithm to extract the best fit co-efficients for the following equation: &nbsp; Y&nbsp;= r*((L+dL)/(W+dW)) &nbsp; I'm getting confused looking at the the non-linear curve fit VI. I'm not sure what to connect to the input terminals. &nbsp; Input terminals: data, f(x,a), Y, X, Weight, initial coniditions, termination &nbsp; My co-efficients will be = (r, dL, dW) so my initial co-efficients will be something like (500, 0, 0). I have an array of Y values which depend on L and W so the Y input terminal is ok. Weigh...

SWEngr-AVP-Algorithm,2-5yrExp.in design & Development of Audio Algorithm
Greetings ! Dear Sir / Madam The following positions are needed at urgent for one of our reputed MNC client located at Bangalore . Order No.1 SWEngr-PDE Algorithms,2+Yr exp. inDesign&Dev.Video/Audio/ Image/Speech CODECS. PDE Algorithms Name of the Position: Software Engineer Number of Positions: 03 Qualification : ME/MTech/BE/BTech in Electrical & Electronics/Electronics & Communication/Computer Science & Engineering. Experience : 2 + Years of experience in design and development of Video/Audio/Image/Speech CODECS. Skills Required: Designed and developed...

In want know about one algorithm (k-node) and how to converted distributed algorithm for link failure?. #2
Hi to all, My name is R.Delhi Babu. Now I am studying M.Phil (Computer Science) in ANNA UNIVERSITY in INDIA (TAMIL NADU). I am dealing with RELIABILITY OF DISTRIBUTED SYSTEM. That area base on´┐Żdistributed algorithm (most failure concept), cryptography, neural network, automata. You can suggest to any one problem give to me. I can try to solve it. Thanking you. Bye Delhi babu.R ...

Regarding acoustic echo cancellation using frequency domain LMS algorithm and subband LMS algorithm #2
Dear all, I am working on the acoustic echo cancellation.I have implemented the time domain LMS algorithm for acoustic echo cancellation and it is working fine only for some audio files and not working for some other audio files which has the large eigen value spread.So to avoid this eigen value spread now I am working on the subband LMS algorithm in frequency domain for acoustic echo cancellation.So now I am searching for the documents of subband LMS algorithm and frequency domain LMS algorithm in which it is mentioned clearly which samples are to be considered and why.But I ...

SWEngr-PDE Algorithms,2+Yr exp. inDesign&Dev.Video/Audio/ Image/Speech CODECS. #2
Greetings ! Dear Sir / Madam The following positions are needed at urgent for one of our reputed MNC client located at Bangalore . Order No.1 SWEngr-PDE Algorithms,2+Yr exp. inDesign&Dev.Video/Audio/ Image/Speech CODECS. PDE Algorithms Name of the Position: Software Engineer Number of Positions: 03 Qualification : ME/MTech/BE/BTech in Electrical & Electronics/Electronics & Communication/Computer Science & Engineering. Experience : 2 + Years of experience in design and development of Video/Audio/Image/Speech CODECS. Skills Required: Designed and developed...

Data Structures Algorithm Analysis In CPP (3rdEd) - Weiss
Data Structures Algorithm Analysis In CPP (3rdEd) - Weiss - Solutions is available for purchase! Contact me at instructors.team[at]gmail.com ...

simplex algorithm #2
Hello all., Iam working on simplex algorithm for a CMOS inverter circuit.. it consists of 5 cmos inverters split in 2 paths. 3 at top and 2 at bottom connected to same supply.i have to vary all the parameters lik W/L or drain , source area to eqaulise the delays from both the paths at the end of both the paths.. how do i go about it? should i make delay times a function of some parameters i have to vary?? Plz if anyone has some idea as to how to start in matlab, den reply hsvjap . ...

ECC- Algorithm #2
I am currently working in a development of error detection and correction algorithm. I found a sample ECC algorithm provided by Samsung for NAND flash. Please checkup the algorithm following link. http://www.samsung.com/Products/Semiconductor/Memory/appnote/samsung_new_ecc_for_512byte_256word.txt The ECC algorithm detects all Single bit errors. It generates 3 ECC (Parity code) bytes for a given data of 512bytes We need to generate ECC codes for the original data and corrupted data. We can generate ECC bytes using the function make_ecc_512(). Later we need to compare them using the functio...