I am using a MATLAB routine (NEWRB) for designing a particular type
of Radial Basis Function Neural Network (RBFNN) regression model.
With this type of model each of the O outputs is a linear combination
of H Gaussian functions with equal scalar covariance matrices (i.e., a
single variance times the unit matrix). The squared distance in each
Gaussian is that between the current input vector of dimension I and
I-dimensional Gaussian center.
A more general type of RBFNN would use a different variance for each
Gaussian. A typical general design would use N input/output-target
pairs to obtain Neq = N*O equations which would be solved by
minimizing the mean squared error (MSE) between the O design output
target values and the RBFNN output. The unknown parameters to be
H*I Gaussian center compponents
H*O Gaussian to output weights
O output bias weights
The total number is
Np = (I+1)*H + (H+1)*O in general
Np = 1 + I*H + (H+1)*O equal variances
The MSE is is obtained by dividing the sum squared error (SSE) by the
number of equations.
MSE = SSE/(N*O)
However, this error estimate is biased because it is obtained by using
the design data. One approach is to "adjust" the estimate by replacing
N by the N-Np the degrees of freedom available after N meaurements are
used to estimate Np parameters:
MSEa = SSE/(O*(N-Np))
The MATLAB design contains constraints whos effect I hope to quantify
by using Npeff, an "effective" number of parameters. Then the
estimation bias can be mitigated by using
MSEa = SSE/(O*(N-Npeff))
The MATLAB design:
a. Single variance
b. Maximum allowable number of Gaussians, Hmax
2. Starts with one Gaussian (H=1) center equal to the first
3. Sequentially chooses additional centers by choosing the
input vector yielding the highest output error (H = H+1).
4. Stops when either SSE <= SSEgoal and/or H = Hmax
1. How is the I*H term in Np modified when each center component is
not determined independently. Instead, it is one of the I components
of a chosen input vector?
2. Without changing the algorithm or using nondesign data, is there a
better way to estimate an unbiased MSE?
Thanks in advance.
||8/21/2011 5:54:24 AM