Threads in Neural Network in Train/Validation/Test

Dear All,
I am using MATLAB R2009a. I have a different answer for train/validation/test from two different method: 

1) I Use: 
[net tr] = train(net,trainV.P,trainV.T,[],[],valV,testV);

 to train the network,   and simulate the different train/validation/test result as: 

normTrainOutput=sim(net,trainV.P,[],[],trainV.T);
normValidateOutput=sim(net,valV.P,[],[],valV.T);
normTestOutput=sim(net,testV.P,[],[],testV.T);

and then i obtained MSE for training validation and test as:

MSETrain=tr.perf(end);
MSEValidate=tr.vperf(end);
MSETest=tr.tperf(end);

And from them finally R2 square value as: 

R2Train=1-NMSETrain
R2Validate=1-NMSEValidate
R2Test=1-NMSETest

And the result i obtained directly from MSEtrain1=mse(normTrainOutput - tn(:XX)), that starts from training interval period i defined. And so on validation and test. Why these two values MSETrain1 and MSETrain differ?

2) I have R2 Train = 0.7738, R2 Validate = 0.7934 and R2 Test = 0.7926. And from the linear regression plot i obtain R train = 0.89584, R validate = 0.81805 and R Test = 0.92432. Does it mean the R2 value of neural network is worst than linear regression model? OR the result i obtained during training = 0.89584 from regression is quite good.

3) Every times i simulate my network, my R2 values sometimes good and sometimes even worst  -ve. How to make it constant, if i assume i get 27 epochs,  hidden neurons  =18 the best R2 value?

Thank You.
0
2/21/2013 8:07:08 PM
comp.soft-sys.matlab 206928 articles. 560 followers. lunamoonmoon (258) is leader. Post Follow

1 Replies
179 Views

Similar Articles

[PageSpeed] 25
"Subodh Paudel" <subodhpaudel@gmail.com> wrote in message <kg5upc$pgj$1@newscl01ah.mathworks.com>...
> Dear All,
> I am using MATLAB R2009a. I have a different answer for train/validation/test from two different method: 
> 
> 1) I Use: 
> [net tr] = train(net,trainV.P,trainV.T,[],[],valV,testV);
> 
>  to train the network,   and simulate the different train/validation/test result as: 
> 
> normTrainOutput=sim(net,trainV.P,[],[],trainV.T);
> normValidateOutput=sim(net,valV.P,[],[],valV.T);
> normTestOutput=sim(net,testV.P,[],[],testV.T);

Using norm in the output names is confusing because norm has a special meaning
(help/doc norm) 
> 
> and then i obtained MSE for training validation and test as:
> 
> MSETrain=tr.perf(end);
> MSEValidate=tr.vperf(end);
> MSETest=tr.tperf(end);

I think if tr.stop indicates validation minimum stopping you should 
replace end with end- tr.max_fail or tr.best_epoch.

> And from them finally R2 square value as: 
> 
> R2Train=1-NMSETrain
> R2Validate=1-NMSEValidate
> R2Test=1-NMSETest
> 
> And the result i obtained directly from MSEtrain1=mse(normTrainOutput - tn(:XX)), that starts from training interval period i defined. And so on validation and test. Why these two values MSETrain1 and MSETrain differ?

If tr.stop indicates validation minimum stopping, then the last max_fail epochs should not be included. Find tr.best_epoch ,tr.best_perf,  etc
 
> 2) I have R2 Train = 0.7738, R2 Validate = 0.7934 and R2 Test = 0.7926. And from the linear regression plot i obtain R train = 0.89584, R validate = 0.81805 and R Test = 0.92432. Does it mean the R2 value of neural network is worst than linear regression model? OR the result i obtained during training = 0.89584 from regression is quite good.

If you had chosen the val minimum epoch, I would have expected

R = sqrt( R^2 )
> 
> 3) Every times i simulate my network, my R2 values sometimes good and sometimes even worst  -ve. How to make it constant, if i assume i get 27 epochs,  hidden neurons  =18 the best R2 value?

You get different values because of the random data division and random weight initialization. If you intialize the random number generator to the same state (e.g., 
rng(4151941) ) before data division and weight initialization, you will reproduce runs.

I usually use a double loop over numH candidate values for H and Ntrials weight initialization runs to get five Ntrials X numH sized matrices for numepochs, R2trn, 
R2trna, R2val and R2tst.

Search NEWSGROUP and ANSWERS for greg Ntrials (or other of my characteristic 
variable names MSE00, Neq, Ntrneq, Nw, Hub, R2, R2a,...)

Hope this helps.

Greg 
0
heath (3982)
2/22/2013 1:14:14 AM
Reply:
Similar Artilces:

ThreadPoolExecutor
Hello, In the ThreadPoolExecutor (TPE), is the callback always guaranteed to run in the same thread as the submitted function? For example, I tested this with the following code. I ran it many times and it seemed like `func` and `callback` always ran in the same thread. import concurrent.futures import random import threading import time executor = concurrent.futures.ThreadPoolExecutor(max_workers=3) def func(x): time.sleep(random.random()) return threading.current_thread().name def callback(future): time.sleep(random...

asynchrous testing
Hi , &nbsp; I am building automated Burn-in test tool for product . I have created the work cell for each device . The work cell&nbsp;contains the information like start test,time stamp&nbsp; ,end time ..etc .&nbsp; Each work cell will have to under go different test steps (Test1 ,Test2.....etc). The&nbsp; requirement is ,test run should be asynchronous,meaning when test running for Work cell 1 ,the other two work cell&nbsp;testing &nbsp;need to be ideal. When ever the test completes for the work cell 1 , testing for the work cell2 should start and so on . As per th...

To thread or not to thread
My app needs to check a number of things on the web. This can be done in the background. I use idHTTP to pull data in. This process is broken down into small parts and the subroutinen are called in the OnIdle handler of the main form. I notice that sometimes idHTTP needs a considerable time to build the connection/fetch the data and the application doesn't repond to user actions during such periods. To cope with this I have now implemented the web data fetching in a thread with priority tpIdle. I think the problem of non-response has gone, ut it is difficult to test. Am...

Messaging between threads
Hi All, I have written an multithreaded application on Win32 Platform. The backend of the application consists of 3 threads and is coded with Win32 API , while my frontend or GUI is developed using MFC. The GUI is entirely a different thread or the main process thread. I have to send messages to my GUI thread from the backend threads. I used SendMessage Function for the same but using that caused my application to stall. Then I used postmessage function which solved my problem. But now I m having a problem that my application responds slowly to any event occurring. Please help me ou...

test
test On Sun, 21 Dec 2003 18:01:36 GMT, Philip <phsu@ix.netcom.com> wrote: >test You flunked - next time try alt.test Mornings: Evolution in action. Only the grumpy will survive. ----------------------------------------------------- Please note - Due to the intense volume of spam, we have installed site-wide spam filters at catherders.com. If email from you bounces, try non-HTML, non-encoded, non-attachments. ----== Posted via Newsfeed.Com - Unlimited-Uncensored-Secure Usenet News==---- http://www.newsfeed.com The #1 Newsgroup Service in the World! >100,0...

SAP HANA Online Training
FOCUSTRAININGS Provides the Best SAP HANA Online Training under the guidanc= e of well Experienced, Highly Qualified and Certified SAP HANA Professional= s. So, Our Professionals provides Real-time, Practical and Project based kn= owledge transfer on SAP HANA Activities to the students. Our combination of= infrastructural strength and proven teaching models on SAP HANA course has= helped many learners accomplish their career goals. Some of the SAP HANA Course topics that covered by our professionals: 1.Introduction to SAP HANA. 2.SAP In-Memory Strategy. 3.HANA compare to BWA. 4...

J2EE tool advice
I have a client who wishes to find a tool to help them work out what tests they have to re-run when they make a change to a component within a Java Enterprise Edition (JEE) application. They would like to know what web user interface components (JSPs) would have to be retested and what (Actuate) reports would have to be checked after, for example, changing a value object, or a session bean. There are several Message Driven Beans involved in the application as well. I have seen the demonstration of Rational Purify Plus adding code coverage information to gcal and suggesting what h...

Release: vizann-2.0 Tkinter Neural Net Demo Program
Release Name: vizann-2.0 This freeware program may be downloaded from http://sourceforge.net/projects/annevolve. *Notes:* This is a program to graphically demonstrate the operational details of two types of ANN (Artificial Neural Network) when used to implement the XOR function. The program is 100% GUI, meaning that there is no line-by-line input nor output. The program is written in Python. (a free download from http://www.python.org) The .zip file contains only these five files: VizANN.py - The program source code VizANN.txt - documentation xormlp.gif - an image file which VizANN.py w...

listener-thread err -930
Hi Folks: I've just got a listener-thread err -930 on a new instance I put up last Friday. This box is in my test bed and is not critical but I would like to know why when I load data from my production server on this test server then shutdown Informix it generates this -930 error when I try and start it up again. I've checked the sqlhosts file and the entry seems correct but there must be some configuration file out of sync. This is on an older AIX box and the database was created using the ISA. ...

wx threads and blocking IO broblem
Hi. My program deals with the blocking IO, and to achieve the efficiency, I am using threading. The technique is rather standard: I am creating the reader thread, which blocks in a read() syscall, and as soon as it has data, it sends the events to the main thread. This works rather well till the moment when I need to stop the reader thread. Naturally, the sequence would be the following: 1. notify the reader that it have to terminate 2. interrupt the blocking syscall 3. wait for termination 4. delete the thread object 3 and 4 are obvious. It is unclear how to achieve 2. Does...

Pipeline thread model
I want to build the C++ class for pipeline thread model that each thread will be assigned a fixed tasks, it process the input and send output to the next stage. Because i'm inexperienced with thread programming so I need some help for my starting point. - First of all, do I need separated input and output buffer for each threads on the pipeline. Can I shared the buffer between the output of one stage to the input of the next stage. What is the better option. - Second, how many thread I need for one pipeline stage. I can see one option that require 3 threads for one pipeline stage: ...

Best Oracle 10G Online Training in India | UK | USA | Australia | Singapore | Canada
Oracle 10G Online Training by Biginfosys, We are one of the finest online t= raining institutes for Oracle 10G course. We are one of the most trusted on= line training institutes among the world. We hire trainers who are most eff= icient in their related field having real time experience of more than 8 ye= ars. Our Oracle 10G trainer's explanation will make the subject easy with t= heir real time experience. We are dedicated in providing Oracle 10G online = training by maintaining Global Standards and according to the trainee requi= rements. Some of SAP Basis topics Covered by ou...

Suddenly Network Places stops responding when trying to connect to another computer on LAN
I have Windows XP home edition on this computer and Windows XP Professional on the other computer in my house. Up until 2 days ago, the networking was fine. I tried to browse files on the Pro computer and Network Places stops responding. From the Pro computer I can still browse folders on this one - no problem. Any suggestions? Thanks, Peg peg@pcsflint.com (Peg) wrote in message news:<ae66a30a.0401010945.e8a232@posting.google.com>... > I have Windows XP home edition on this computer and Windows XP > Professional on the other computer in my house. Up until 2 days ago,...

Terminating a Worker Thread
Consider this example program... The main thread starts, creates a worker thread, waits for some signal, then terminates the worker thread before rejoining it and exiting. In my "keep it simple" mindset, I figure that if I just pass the thread a boolean flag and set the flag in the main thread when the worker needs to exit, the worker can be pretty simple: void * worker (void * data) { bool * done = (bool *)data; while (! *done) { // ... } return NULL; } As long as it can periodically check the flag I should be set. I do have ...

thread::pool : job whon't start after a idle thread has exited.
tcl 8.4 thread 2.5.2 sol 8 Command to create a thread pool package require Thread set tpId [tpool::create -maxworkers 4 -idletime 10 -exitcmd "puts end"] Command to submit job: tpool::post $tpId {puts allo} The first time that i submit the job i get allo if i wait 11 seconds i get end, no allo my job doesn't get run, why? If i submit an other i get allo. Is there a bug when a thread gets idle and exit? David David@capitale.qc.ca (David Paquin) wrote in message news:<c72a3ff5.0308081127.2a5d5ff8@posting.google.com>... > tcl 8.4 > thread 2.5.2 > sol...

US-TX-Ausitn: Sr SoC Validation Enginee, BS in CS, EE or ECE (waived with approp (45358332407)
US-TX-Ausitn: Sr SoC Validation Enginee, BS in CS, EE or ECE (waived with approp (45358332407) ============================================================================================== Position: Sr SoC Validation Enginee Reference: MLS00026 Location: Ausitn TX Duration: Perm Skills: BS in CS, EE or ECE (waived with appropriate experience). 8 to 12 years of experience in embedded software development. 3 plus years of experience with development on FPGA emulation platforms. Expert C language software developer. ...

Help about Graphy Theory and Communication Networks C /C++ plz..
Hi everyone.. I need help about graph theory ( network problems) .. I have below problem . Does anyone know any existence algorithms or solutions to this problem ?? The Problems is: " We can use graphs to represent a communication network. In such graphs, the vertices represent communication stations and the edges represent communication links. Critical points are the vertices whose failure will result in the network becoming disconnected. Similarly, critical links are the edges whose failure will result a loss of communication. Sub components are the graphs which don't contain...

Availables : More that 3500 Solutions manuals and Test Banks (Part 1) #2
List of Solutions Manuals _________________________ contact me to : mattosbw1@gmail.com mattosbw1(at)gmail.com NOTE : "THIS SERVICE IS NOT AVAILABLE FOR : CHINA, INDIA, PAKISTAN, IRAQ, IRAN, PHILIPPINES, NORTH KOREA, NEPAL, BANGLADESH, SRI LANKA, MALDIVES & BHUTAN". If your wanted solutions manual is not in this list, also can ask me if is available (They are some only). Then if you need solutions manual only contact me by email. This same "list of titles" of texts with availables solutions manual is for download from : http://www.megaupl...

Multi Class Label Neural Network Implementation
Hi, Did anyone come across or use a Neural Network Implementation the support Multi Class Label?. I would appreciate if anyone pointed me to it. Regards, -- Motaz K. Saad On Oct 17, 4:09=A0pm, "Motaz K. Saad" <motaz.s...@gmail.com> wrote: > Hi, > > Did anyone come across or use a Neural Network Implementation the > support Multi Class Label?. I would appreciate if anyone pointed me to > it. Any implementation that supports multiple outputs will suffice. Just use {0,1} targets for each output. Hope this helps. Greg On Oct 20, 3:32=A0am, Greg Heath <he......

Low tech mains-free network
Three (house) boats... low tech, urban nomadic, low cost... want to share ADSL. They are within 120 metres of each other (not in one line and could choose for wireless - but it's more expensive and depends on power for wifi - or cables, but these have to be water and weather resistant. Which is best/easiest/cheapest Are there (inexpensive) cables that can be laid under water or should we be looking at wifi? Rgds Martin ...

thread communication
hello, ' i would like to developp an application with a master and ten&nbsp; threads&nbsp; which are in a listening state of datas. To realize this,&nbsp; each thread must control the presence of&nbsp;datas in it own pipe and can write datas in it pipe too. &nbsp;i have already realize this with fifos, but&nbsp; i don't know if must use two fifos for the communication between the master and one thread or only one, because when the&nbsp; master has written a data in the pipe of the thread, he takes a state of "wait the acknoledge". &nbsp; &...

OT: Networking with 3Com 3500...
Well, I have been tearing down my microchannel network lately, as the contract at work that it was supporting ended a few weeks ago. Just a single 9595 left now, but I think I will keep it at work for some tinkering in the future. I used a 3Com 3500 unit with FDDI and 10/100 blades for nearly a year, enjoying the higher speed I saw in moving data sets across FDDI than 10baseT. What caught me by surprise last Friday was when I learned the unit supports ATM blades as well. Hmm, 10/100 + FDDI + ATM all in a single switch... that would be interesting to try. exwisdem > We...

AOA(AON) network representation
In network planning techniques, such as PERT,CPM, There are two possible ways of representing each project: Activity-On-Arc(AOA), Activity-On-Node(AON). we know that some project management computer packages use both the network representations. is there real need for using the AoA representation (Because of minimum dummy arc problem)? If the AoN representation is only used in project scheduling, what is the nature of the mistake made here? I couldn't find any clear-cut and admissible explanation. Thanks to everybody for their contributions. Mustafa Ege ...

Threads + Ncurses
Hi, I'm fairly new to ruby and wanted to write some example programs in ruby using ncurses-wrapper-library. That is, in fact, really simple and works well. But now I have a problem: I want ncurses to draw some things on the screen and update it every second - until a key is pressed. I'm trying to do it as follows (short example): stop = nil Thread.new do until stop Ncurses.printw("foo") # just an example... Ncurses.refresh sleep 1 end Ncurses.endwin end Ncurses.getch stop = 1 But I suppose Ncurses.getch blocks any other ncurses-functions from writing any...

How to test spanning tree protocol
Hi, I want to know how to test the spanning tree protocol with the bridges and switches. Please help me in this. Regards, F Joe Gladston gunajoe@gmail.com wrote: > Hi, > > I want to know how to test the spanning tree protocol with the bridges > and switches. > Please help me in this. > Depens on the manafacturuer i did this last year for cisco kit in the local cisco academy you can show the state of each port on a switch as the network stabalises I also did a test network of our teams SEM 4 Design and tested it for real (this was with full m...