Hello guys,

I have a question that might seem simple but I don't seem to be able to find the answer. 

I have a very simple neural network which I train using the Neural Network Toolbox GUI. My performance criterion is the MSE.

I am wondering why each time I train the network I get different results for the performance. 

What can I report as the performance of the network. Should I run the algorithm for ten or more times and pick the best performing one?

Thank you,

George

"Georgios Mavromatidis" <mavromatidis.g@gmail.com> wrote in message <jschib$36f$1@newscl01ah.mathworks.com>...
> Hello guys,
> 
> I have a question that might seem simple but I don't seem to be able to find the answer. 
> 
> I have a very simple neural network which I train using the Neural Network Toolbox GUI. My performance criterion is the MSE.
> 
> I am wondering why each time I train the network I get different results for the performance.

The initial weights are chosen randomly. This results in different local minima solutions. 
 
> What can I report as the performance of the network. Should I run the algorithm for ten or more times and pick the best performing one?

I typically run the algorithm Ntrials = 10 or more times for each candidate value of the number of hidden nodes, H. Then, for each value of H I tabulate the summary statistics 
min, median, mean, std, max.

Be sure to specify the initial RNG seed or state so that you can replicate the run.

Typically, the net with the smallest H that meets the performance goal is chosen. I typically use MSEgoal = 0.01*mean(var(t')) to obtain R^2 <~ 0.99.

Sometimes a weighted average of the outputs from several good nets are combined  to form an ensemble.
 
Hope this helps.

Greg

Hello Greg,

Yes your message did help a lot actually! Thank you very much for your help and your quick reply!

Best,

George


"Greg Heath" <heath@alumni.brown.edu> wrote in message <jsckaf$g8s$1@newscl01ah.mathworks.com>...
> "Georgios Mavromatidis" <mavromatidis.g@gmail.com> wrote in message <jschib$36f$1@newscl01ah.mathworks.com>...
> > Hello guys,
> > 
> > I have a question that might seem simple but I don't seem to be able to find the answer. 
> > 
> > I have a very simple neural network which I train using the Neural Network Toolbox GUI. My performance criterion is the MSE.
> > 
> > I am wondering why each time I train the network I get different results for the performance.
> 
> The initial weights are chosen randomly. This results in different local minima solutions. 
>  
> > What can I report as the performance of the network. Should I run the algorithm for ten or more times and pick the best performing one?
> 
> I typically run the algorithm Ntrials = 10 or more times for each candidate value of the number of hidden nodes, H. Then, for each value of H I tabulate the summary statistics 
> min, median, mean, std, max.
> 
> Be sure to specify the initial RNG seed or state so that you can replicate the run.
> 
> Typically, the net with the smallest H that meets the performance goal is chosen. I typically use MSEgoal = 0.01*mean(var(t')) to obtain R^2 <~ 0.99.
> 
> Sometimes a weighted average of the outputs from several good nets are combined  to form an ensemble.
>  
> Hope this helps.
> 
> Greg