f

#### Cramer Rao Bound

```Hi,

I guess this is more of a math question, but I thought someone here
could help me. I'm confused about the different forms of the vector
Cramer Rao Bound. Specifically, let x be the parameter vector, and
let L(x) be the log pdf. Then the Fisher matrix is

(1) E{dL/dx*(dL/dx)'}

or

(2) -E{d2L/dxdx'}

Now I understand the second form- it is a full rank matrix. But form
(1) is the product of two vectors- hence, it is rank deficient, and
cannot be inverted to obtain the minimum variance. So how are these
two forms equivalent? Am I reading the math wrong? Thanks!
``` 0 2/9/2006 6:41:36 PM comp.soft-sys.matlab  211266 articles. 22 followers. 2 Replies 330 Views Similar Articles

[PageSpeed] 53

```"Tom Georger" <no@spam.com> wrote in message
news:ef2858e.-1@webx.raydaftYaTP...
> Hi,
>
> I guess this is more of a math question, but I thought someone here
> could help me. I'm confused about the different forms of the vector
> Cramer Rao Bound. Specifically, let x be the parameter vector, and
> let L(x) be the log pdf. Then the Fisher matrix is
>
> (1) E{dL/dx*(dL/dx)'}
>
> or
>
> (2) -E{d2L/dxdx'}
>
> Now I understand the second form- it is a full rank matrix. But form
> (1) is the product of two vectors- hence, it is rank deficient, and
> cannot be inverted to obtain the minimum variance. So how are these
> two forms equivalent? Am I reading the math wrong? Thanks!

It's not rank deficient when you take the expectation.

``` 0 2/9/2006 8:18:36 PM
```Tom, this is more of a sci.stat.math question, but lemme take a shot at it:

Consider the two-param case, where the score vector is [dL(x)/da dL(x)/db] and
the Hessian is

[(dL(x)/da)^2  dL(x)/da * dL(x)/db;  dL(x)/da * dL(x)/db  (dL(x)/db)^2)]

I've suppressed dependence of L on the parameters (a,b).  These partials are all
sort of abstractly a function of a random variable x.  If you were to plug any
particular observed value x_i into those formulas, you'd get a Hessian that had
rank 1, and it's easy to see that, because if you multiply the second row by

dL(x_i)/da / dL(x_i)/db,

you get exactly the first row, and you've verified rank 1.

But don't think of sample values for x, think of these partials as random
values.  To get the Fisher information I, you need the E[-H], and once you start
thinking of a random H and expectations, you can no longer find a multiple of
the second row that is equal to the first row because, for example, while

(dL(x_i)/db)^2 * (dL(x_i)/da / dL(x_i)/db)
== dL(x_i)/da * dL(x_i)/db

for any fixed x_i, it's not true that

E[(dL(x)/db)^2] * E[(dL(x)/da / dL(x)/db)]
== E[dL(x)/da * dL(x)/db]

Hope this helps.

- Peter Perkins
The MathWorks, Inc.
``` 0 2/9/2006 8:28:19 PM