f



Cramer Rao Bound

Hi,

I guess this is more of a math question, but I thought someone here
could help me. I'm confused about the different forms of the vector
Cramer Rao Bound. Specifically, let x be the parameter vector, and
let L(x) be the log pdf. Then the Fisher matrix is

(1) E{dL/dx*(dL/dx)'}

or

(2) -E{d2L/dxdx'}

Now I understand the second form- it is a full rank matrix. But form
(1) is the product of two vectors- hence, it is rank deficient, and
cannot be inverted to obtain the minimum variance. So how are these
two forms equivalent? Am I reading the math wrong? Thanks!
0
no18 (4537)
2/9/2006 6:41:36 PM
comp.soft-sys.matlab 211266 articles. 22 followers. lunamoonmoon (257) is leader. Post Follow

2 Replies
317 Views

Similar Articles

[PageSpeed] 44

"Tom Georger" <no@spam.com> wrote in message 
news:ef2858e.-1@webx.raydaftYaTP...
> Hi,
>
> I guess this is more of a math question, but I thought someone here
> could help me. I'm confused about the different forms of the vector
> Cramer Rao Bound. Specifically, let x be the parameter vector, and
> let L(x) be the log pdf. Then the Fisher matrix is
>
> (1) E{dL/dx*(dL/dx)'}
>
> or
>
> (2) -E{d2L/dxdx'}
>
> Now I understand the second form- it is a full rank matrix. But form
> (1) is the product of two vectors- hence, it is rank deficient, and
> cannot be inverted to obtain the minimum variance. So how are these
> two forms equivalent? Am I reading the math wrong? Thanks!

It's not rank deficient when you take the expectation. 


0
2/9/2006 8:18:36 PM
Tom, this is more of a sci.stat.math question, but lemme take a shot at it:

Consider the two-param case, where the score vector is [dL(x)/da dL(x)/db] and 
the Hessian is

    [(dL(x)/da)^2  dL(x)/da * dL(x)/db;  dL(x)/da * dL(x)/db  (dL(x)/db)^2)]

I've suppressed dependence of L on the parameters (a,b).  These partials are all 
sort of abstractly a function of a random variable x.  If you were to plug any 
particular observed value x_i into those formulas, you'd get a Hessian that had 
rank 1, and it's easy to see that, because if you multiply the second row by

    dL(x_i)/da / dL(x_i)/db,

you get exactly the first row, and you've verified rank 1.

But don't think of sample values for x, think of these partials as random 
values.  To get the Fisher information I, you need the E[-H], and once you start 
thinking of a random H and expectations, you can no longer find a multiple of 
the second row that is equal to the first row because, for example, while

    (dL(x_i)/db)^2 * (dL(x_i)/da / dL(x_i)/db)
                            == dL(x_i)/da * dL(x_i)/db

for any fixed x_i, it's not true that

    E[(dL(x)/db)^2] * E[(dL(x)/da / dL(x)/db)]
                            == E[dL(x)/da * dL(x)/db]

Hope this helps.

- Peter Perkins
   The MathWorks, Inc.
0
2/9/2006 8:28:19 PM
Reply: