f



Threads in Neural Network in Train/Validation/Test

Dear All,
I am using MATLAB R2009a. I have a different answer for train/validation/test from two different method: 

1) I Use: 
[net tr] = train(net,trainV.P,trainV.T,[],[],valV,testV);

 to train the network,   and simulate the different train/validation/test result as: 

normTrainOutput=sim(net,trainV.P,[],[],trainV.T);
normValidateOutput=sim(net,valV.P,[],[],valV.T);
normTestOutput=sim(net,testV.P,[],[],testV.T);

and then i obtained MSE for training validation and test as:

MSETrain=tr.perf(end);
MSEValidate=tr.vperf(end);
MSETest=tr.tperf(end);

And from them finally R2 square value as: 

R2Train=1-NMSETrain
R2Validate=1-NMSEValidate
R2Test=1-NMSETest

And the result i obtained directly from MSEtrain1=mse(normTrainOutput - tn(:XX)), that starts from training interval period i defined. And so on validation and test. Why these two values MSETrain1 and MSETrain differ?

2) I have R2 Train = 0.7738, R2 Validate = 0.7934 and R2 Test = 0.7926. And from the linear regression plot i obtain R train = 0.89584, R validate = 0.81805 and R Test = 0.92432. Does it mean the R2 value of neural network is worst than linear regression model? OR the result i obtained during training = 0.89584 from regression is quite good.

3) Every times i simulate my network, my R2 values sometimes good and sometimes even worst  -ve. How to make it constant, if i assume i get 27 epochs,  hidden neurons  =18 the best R2 value?

Thank You.
0
2/21/2013 8:07:08 PM
comp.soft-sys.matlab 210195 articles. 11 followers. lunamoonmoon (258) is leader. Post Follow

1 Replies
359 Views

Similar Articles

[PageSpeed] 19

"Subodh Paudel" <subodhpaudel@gmail.com> wrote in message <kg5upc$pgj$1@newscl01ah.mathworks.com>...
> Dear All,
> I am using MATLAB R2009a. I have a different answer for train/validation/test from two different method: 
> 
> 1) I Use: 
> [net tr] = train(net,trainV.P,trainV.T,[],[],valV,testV);
> 
>  to train the network,   and simulate the different train/validation/test result as: 
> 
> normTrainOutput=sim(net,trainV.P,[],[],trainV.T);
> normValidateOutput=sim(net,valV.P,[],[],valV.T);
> normTestOutput=sim(net,testV.P,[],[],testV.T);

Using norm in the output names is confusing because norm has a special meaning
(help/doc norm) 
> 
> and then i obtained MSE for training validation and test as:
> 
> MSETrain=tr.perf(end);
> MSEValidate=tr.vperf(end);
> MSETest=tr.tperf(end);

I think if tr.stop indicates validation minimum stopping you should 
replace end with end- tr.max_fail or tr.best_epoch.

> And from them finally R2 square value as: 
> 
> R2Train=1-NMSETrain
> R2Validate=1-NMSEValidate
> R2Test=1-NMSETest
> 
> And the result i obtained directly from MSEtrain1=mse(normTrainOutput - tn(:XX)), that starts from training interval period i defined. And so on validation and test. Why these two values MSETrain1 and MSETrain differ?

If tr.stop indicates validation minimum stopping, then the last max_fail epochs should not be included. Find tr.best_epoch ,tr.best_perf,  etc
 
> 2) I have R2 Train = 0.7738, R2 Validate = 0.7934 and R2 Test = 0.7926. And from the linear regression plot i obtain R train = 0.89584, R validate = 0.81805 and R Test = 0.92432. Does it mean the R2 value of neural network is worst than linear regression model? OR the result i obtained during training = 0.89584 from regression is quite good.

If you had chosen the val minimum epoch, I would have expected

R = sqrt( R^2 )
> 
> 3) Every times i simulate my network, my R2 values sometimes good and sometimes even worst  -ve. How to make it constant, if i assume i get 27 epochs,  hidden neurons  =18 the best R2 value?

You get different values because of the random data division and random weight initialization. If you intialize the random number generator to the same state (e.g., 
rng(4151941) ) before data division and weight initialization, you will reproduce runs.

I usually use a double loop over numH candidate values for H and Ntrials weight initialization runs to get five Ntrials X numH sized matrices for numepochs, R2trn, 
R2trna, R2val and R2tst.

Search NEWSGROUP and ANSWERS for greg Ntrials (or other of my characteristic 
variable names MSE00, Neq, Ntrneq, Nw, Hub, R2, R2a,...)

Hope this helps.

Greg 
0
heath (3982)
2/22/2013 1:14:14 AM
Reply:

Similar Artilces:

How do I find out which dataset was used for train, validation and test in a trained neural network?
I'm trying to do a code to train as many networks as I want automaticaly. Nevertheless I'm finding a hard work trying to get the datasets which were used for train, validation and test. What I do, but I don't know if it's right is to use diverand function befor train function. Is it the righway? "Fernando Cunha" <cunha.fr@terra.com.br> wrote in message <hksfg2$rgg$1@fred.mathworks.com>... > I'm trying to do a code to train as many networks as I want automaticaly. Nevertheless I'm finding a hard work trying to get the datasets which were used f...

Neural network: Retrieve training, validation, testing dataset
Hi there, Two questions: 1. I'm using newff and train to create a neural network (LM algorithm). I need to retrieve the training, validation and testing dataset I'm using after the training (I'm using the default 60,20,20 division of the input dataset). I'could obtain those using dividerand in the input dataset (and get a train, val, test input dataset) and then using the indices with divideind to get the corresponding target datasets. But, then, how to use those datasets when calling "train"? Or when doing: [trainP2,valP2,testP2,trainInd2,valInd2,testInd2] ...

Training, testing and validating data set in Neural Network
Hi everybody, I'm working with Neural Network toolbox and I had a doubt about how to define the training, testing and validating data set. I mean, how to distribute the 100% of the data into these three categories. Which percentage for each one?? And also, how many tests should i consider for a neural network of 1 hidden layer, with 21 neurons in this layer, 10 in the input and just 1 in the output? is there any way to compute the total number of tests?? I tried everywhere, but i couldnt find any answer, so any advice or help is welcome. thanks in advance, Munoz. On Nov 7...

neural network training in matlab
hi,i am doing a project on speaker recognition using artificial neural network.could anyone share information regarding the training of a neural network and how to initialize the weights ...

Neural Network Architecture with MATLAB Neural Network Toolbox
Dear neural network experts, I would like to make a neural network model for a situation when some of the input variables can interact, while others don't have any interactions. This information is based on logical considerations and I have to include it into the architecture of the net. For example, I have 3 different groups of input variables. Within each groups the variables are disjunct (i.e. no interaction). Variables of different groups may interact. This would result in either - restrictions to some weights of a network consisting of completely inter-linked neurons (i.e. some weight...

Neural Networks Train/Test
Hi, what?s the command line for training, so that i have just train samples and test samples (not using validation) ? I tried changing this without success.. [net,tr] = train(net,trainSamples.P,trainSamples.T,[],[],validateSamples,testSamples); Regards, Andre On Nov 18, 7:40=A0pm, "Andre Santos" <and...@brturbo.com.br> wrote: > Hi, > > what?s the command line for training, so that i have just train samples a= nd test samples (not using validation) ? > > I tried changing this without success.. > > [net,tr] =3D train(net,trainSamples.P,trainSamples.T,[],...

data splitting: train&test or train/test/validate
Hi, I have some questions. Anyone willing to answer - big thanks in advance. a)Which one is the best way to split data?Advantages & disadvantages? 1.train & test 2.train/test/validate b)What is the different between test & validate? Aren't they supposed to be the data that different from train set but still fall within the same range? c)How we define the data set is small? eg: I have a set of data contains 6column (represent 5 input & 1 output) which every column has 31 data point. Is this consider small for MLP & RBF network? d)Data splitting come first o...

How to execute trained neural network in Matlab by myself?
Hi! I have a neural netwok which is already trained. It has 150 inputs, 30 outputs and 40 neurons in hidden layer. I have stored weights in IW, LW, b1 and b2 variables. Then, if I compute the expression purelin(LW*logsig(IW*Inputs(:,13) + b1)) + b2 I get totally different values than by sim(net,Inputs(:,13)) Why? What I am doing wrong? What extra parameters I should take from net object? Thanks. On Nov 25, 1:14=A0pm, Dims <dim...@hotmail.com> wrote: > Hi! > > I have aneuralnetwok which is already trained. It has 150 inputs, 30 > outputs and 40 neurons in hidden layer...

Testing Neural network matlab code
I have written this NN to classify different cases. Simulation is excellent but I could not get the required target T during the testing. Please let me know your suggestion. I am desperate to solve this problem due to submission of my postgraduate final year. P: input data; T: targets, P2a: testing input data; t2: testing targets Regards This is the Matlab code Inline Attachment Follows: matrixdatatraining2.m %% Neural Network Classification % training Matrix data load P0;load P1;load P2;load P3;load P4; % training data (Inputs) size(Pi=909x3) [r0 c0] = size(P0); [r1 c1] = si...

Neural network train problem with validation
i'm Working on modeling very high dimensional data with very few observation, and of course there's no way to obtain a good model in prediction, but i'd like to implement a new approach for assessing neural netowork but i really need a way not to divide the set of dat in training - validation -test, but only training and use the test set for detect the best network in prediction. how can i set the parameter in newff for exclude the validation set? i tried with: net.divideParam.trainRatio = 0.8; net.divideParam.testRatio = 0.2; but it doesn't work. I also split randomly my initial data set in 2 different sets of 80% of data for training and 20% for test as follows: Nexp = 96; Npos = 4 Domains=95 perccamp=20; Gen1 = ceil(rand(Nexp,Npos)*Domains); numtest =round((Nexp*perccamp)./100); datiinput=randperm(96); test= Gen1(datiinput(1:numtest),:); train = Gen1(datiinput(numtest+1:Nexp),:); trainp= train(:,1:Npos*Domains) traint=train(:,Npos*Domains+1) testp=test(:,1:Npos*Domains) testt=test(:,Npos*Domains+1) net= newff(p',t',nn); but i don't know how to train the net with only the testset Any suggestions? "Davide De March" <davidedemarch@unive.it> wrote in message <hs11hg$rfe$1@fred.mathworks.com>... > i'm Working on modeling very high dimensional data with very few observation, and of course there's no way to obtain a good model in prediction, but i...

trainbr (neural network training) in matlab R2008a
Hi, I have a problem when training a neural network in Matlab R2008a with TRAINBR. When I type the following code in Matlab R2008a versus in Matlab R2007b, the result is always much worse in R2008a than in R2007b: p=-1:0.01:1; s=sin(2*pi*p); t=s+0.5*randn(size(p)); net= newff(p,t,40,{'tansig','purelin'},'trainbr'); net.divideFcn=''; net.trainParam.epochs=2000; [net,tr]=train(net,p,t); y=sim(net,p); figure; plot(p,y,'r.',p,s,'b-',p,t,'go'); In R2007b a nice (or at least a reasonable) result is obtained, as well as in previous Matlab ver...

MSE compararison between training and testing for neural network
Helo,i have done neural network for training part. After i get the Mean Square Error(MSE) i want to compare it with MSE in testing part. As I know, 80% for training and 20% for testing, but my problem is I dont know how to start the coding for testing part to get the MSE so i can compare it with training part. Have an idea? Thanks. On Nov 10, 10:03=A0pm, "Reff " <rafi...@yahoo.com> wrote: > Helo,i have done neural network for training part. After i get the Mean S= quare Error(MSE) i want to compare it with MSE in testing part. As I know, = 80% for training and 20% for tes...

How to test Neural Network with real world data after training it?
Dear All, I am currently using nprtool for pattern recognition in Matlab version 7.2 (R2011b).I followed the steps in GUI and trained network and saved the network in Matlab work space as net.What is the procedure if I want to test real world data ? If anybody have answer please post it, it will really help me a lot. Thank you all. "Suresh " <dnbdsuresh@gmail.com> wrote in message news:kekt5s$qrb$1@newscl01ah.mathworks.com... > Dear All, > I am currently using nprtool for pattern recognition in Matlab version 7.2 > (R2011b).I followed the steps in GUI a...

exporting MatLab trained neural network to separate file/function
hello, I have trained a neural network on MatLab and I need to transfer it (the trained network) to a separate file (or generate a function) that I can then use somewhere else (on a robot running Red Hat) where I can use that file as a lookup database or (I will send a particular input pattern/vector and I need an output pattern/vector returned). any idea of how I can generate that file in MatLab? Thank you very much. r. robert <rob_xx17@yahoo.com> wrote in message <28674830.1211505346752.JavaMail.jakarta@nitrogen.mathforum.org>... > hello, > > I have trained a neural network on MatLab and I need to transfer it (the trained network) to a separate file (or generate a function) that I can then use somewhere else (on a robot running Red Hat) where I can use that file as a lookup database or (I will send a particular input pattern/vector and I need an output pattern/vector returned). any idea of how I can generate that file in MatLab? Thank you very much. Hello Robert: I did that some time ago, to use a trained ANN within FORTRAN. The solution was to write down a code which explicitly set the weights and performs the calculations, one at a time. Something like .... w(1)= 0.122334234 w(2)=-0.00569882 f=w(1)*x(1)+w(2)*x(2) .... I wrote a matlab script to create a source code like the one above, and then I have compiled it within FORTRAN. My script might be general enough... for the class of ANN I have dealt with! But on princip...

Neural Network Validation: Train vs Sim output puzzle
Here are the two functions of the puzzle. [net1,tr,Ytrain,E,Pf,Af] = train(net0,P,T,Pi,Ai,VV,TV) Ysim=sim(net1,P) In the context of regression/prediction. What are the differences between the Ytrain network output from TRAIN and the Output from SIM? In estimation the Ytrain network output is equal to the output from sim. From my understanding both represent the neural net regression fit. In the validation step the Ytrain network output is NOT equal to the output from sim. (even if they are somewhat close) This is my issue. Which one would represent the neural net regression fit (if one ...

HELP NEEDED--matlab train function in neural network toolbox
Hi, I have a problem while using the train function in the neural network toolbox with R2009a version. Each time I run the train function I got the following message: ==================================================== ??? Error using ==> mse Too many input arguments. Error in ==> calcperf2 at 219 perf = feval(performFcn,El(OC,:),Yl(OC,:),net,net.performParam); Error in ==> traingdx at 230 [perf,El,trainV.Y,Ac,N,Zb,Zi,Zl] = calcperf2(net,X,trainV.Pd,trainV.Tl,trainV.Ai,Q,TS); Error in ==> network.train at 217 [net,tr] = feval(net.trainFcn,net,tr,trainV,valV,testV); ======...

Can training set be divided further into subsets of training set for training neural networks?
I have read on some forum that there are training methods. one is to train = NN on whole training data at once, 2nd is to train NN in parts like first f= rom 1 to 1000 samples, then 1 to 2000 samples and so on. 3rd is to train NN= in parts like first from 1 to 1000 samples, then 1000 to 2000 samples and = so on. I want to ask that whether all these methods correct and which to us= e when?? On Fri, 4 Jul 2014 01:47:17 -0700 (PDT), Aamir Nawaz <engr.aamir09@gmail.com> wrote: >I have read on some forum that there are training methods. one is to train NN on whole training data...

Testing Tools Online Training | Online Testing Tools Training | Training Testing Tools Online
Biginfosys is the best leading Online Training Institute in the World we ar= e providing Testing Tools Online Training with efficient Testing Tools prof= essional trainers. We have more than 10 years experienced professional Test= ing Tools trainers and we are also providing complete material that covers = each and every topic related to Testing Tools.and We have completed more th= an 200 Testing Tools batches through Online Testing Tools Training program. Please call us for the Demo Classes we have regular batches and weekend bat= ches. Contact Number: USA: +1 303 495 3408, Ema...

From Matlab neural network toolbox to matlab code
Hi All, I have been working with neural network toolbox in matlab 7 for the calibration and validation optimization problems. but i wanna to do it with matlab code. is it possible to transform the things that happen in toolbox, training process and adapting outputs get via code.. is there anyone who transform the things happen in toolbox to codes. "murat " <muratt59@gmail.com> wrote in message <iabd8c$8pc$1@fred.mathworks.com>... > Hi All, > > I have been working with neural network toolbox in matlab 7 for the calibration and validation optimization...

Best QA Testing Online Training | Online QA Testing Training | Training QA Testing Online
QA Testing Online Training by SUN It Labs It is a one of the best course fo= r your better future. But all institutes are may not be provide best output= .. So you need to take good decision while choosing the best institute, Sun = IT labs Online Training Institute providing the best QA Testing Online Tra= ining when compare to the other Institutes. Because we have highly dedicate= d and experienced IT professionals as Trainers they gives a best output. We= have completed more than 200 QA Testing batches through Online QA Testing = Training program, Our QA Testing Classes covers all the real time scenarios= , and it's completely on Hands-on for each and every session. Some of QA Testing topics Covered by our professionals: Our course covers all major topics of testing concepts like * Manual Testing, * Automation Testing Tools, * Quick Test Professional - QTP online training, * Load Runner, * Quality Center and * Selenium Tools Our QA Testing Training Course Highlights: * Cloud test labs * Course material with latest trends in testing * Updated versions of curriculum * Real time scenario based live examples * Instructor led face to face training * Expertise faculty from testing industry * Interactive QA online training * Customized subject topics for each And many more sub topics are there, for more details go to through the our = Professional website Please call us for the Demo Classes we have regular batches and weekend bat= c...

hello, i ve been trying to use matlab neural network for predicting the fatigue properties of materials from the tensile test data...how to enter the data?
my network has five inputs and a single output..so what should my input matrix be like? wats the logic? On Mar 31, 3:35=A0am, "anoop asranna" <anoo...@gmail.com> wrote: > my network has five inputs and a single output..so what should my input m= atrix be like? wats the logic? size(p) =3D [I N] size(t) =3D [O N] help newff help newrb Hope this helps. Greg ...

How to train different narxnets with the same training, validation and test sets
Hi, I am studying narxnets and I have two questions : 1) I want to train different narxnets with the same training, validation and test sets. 2) I want to know how to calculate R2 in the training, validation and test sets in the case where the series is not normalized by zscore. See below for the discussion: 1) I want to train different narxnets with the same training, validation and test sets. In order to do that I use the function divideblock before the definition of the net: [trnind,valind,testind] = divideblock(N,0.7,0.15,0.15); Where N is the number of...

neural network: Recurrent Neural Networks
Hi Greg and Anders, thanks for your help of my backpropagation topic. Now I switch to recurrent neural networks. My question: 1. Does it the recurrent neural networks need to divide input data into training, validation and testing 3 sets? I tried to convert my previous brackpro code into recurrent but it prompt out the error that the "train" code section has too many input arguements, which include VV and TV... Felix Hi I tried to use Elam network. One strange result came to me. I used 10 hidden units it shown that the preformance goal is not met. and the mse is about 8.5 . Then ...

Neural Network Training
Does anybody know of a Neural Network solution in Smalltalk? I'm especially interested in the classification of objects by symbolic attribute vectors (learning by example). Andre Andre Schnoor wrote: > Does anybody know of a Neural Network solution in Smalltalk? I'm > especially interested in the classification of objects by symbolic > attribute vectors (learning by example). > > Andre > Andre - Here are some ideas for you: 1. Smalltalk/X comes with some NN code (I think just backprop though). 2. I had developed a backprop in smalltalk...

Web resources about - Threads in Neural Network in Train/Validation/Test - comp.soft-sys.matlab

Cross-validation (statistics) - Wikipedia, the free encyclopedia
is a model validation technique for assessing how the results of a statistical analysis will generalize to an independent data set. It is mainly ...

Serena Williams eyes history, Caroline Wozniacki craves validation in women's US Open final
As Serena Williams eyes another slice of tennis history, Caroline Wozniacki is craving validation with a watershed US Open final triumph on Sunday ...

Travers Duncan asks High Court to strike down ICAC validation laws
ICAC could be sent back to the High Court as a challenge to new laws validating corruption findings looms.

Adam Scott proclaims Texas win as validation of No.1 ranking
... No.1 golfer, beating American Jason Dufner in the third hole of a play-off at the Colonial event. "It's a good feeling and maybe some validation ...

Hollande to sign gay marriage bill after Constitutional Council's validation
Hollande to sign gay marriage bill after Constitutional Council's validation People's Daily Online ... 26, 27, 28, 29, 30, 31. Jan, Feb, Mar, ...

Marketers, Get Ready for the 'Validation Era'
The public brands included has been engaging in a "friends" arms race. But there are signs that we know we overdid it.

Validation and Uncertainty
What an extraordinary conversation I had recently on Twitter. It started with Neil Killick’s statement that we should not consider our stories ...

Nvidia Icera 410 LTE Modem Passes AT&T Network Validation
... Today's news is a small step and logical precursor toward both goals, Nvidia's Icera 410 LTE and HSPA+ baseband has passed AT&T's network validation ...

Ting’s Q1 Subscriber Growth Suffers from Sprint’s New Unlocking Validation Process
Back in February, Sprint updated the process it uses to validate unlocked devices that customers want to utilize on other carriers or one of ...

[Android M Feature Spotlight] Stricter APK Validation Now Actually Checks If All Of An APK's Files Are ...
While you might file this one under "really? We weren't doing this already?" if you're a security expert, Google has added stricter validation ...

Resources last updated: 3/10/2016 8:23:21 AM