f



Backpropagation for input=[0 0;0 1; 1000 0; 1 1],t=[0;1000;1;0]

I'm working backpropagation for xor training. the following code works for xor function but it cannot work  input=[0 0;0 1; 1000 0; 1 1],t=[0;1000;1;0]
Here is my code:
 clear all;
clc;
tic,

%2-in-and
%x=[0 0 1 1; 0 1 0 1];
%t=[0 0 0 1];

%2-in-or
%x=[0 0 1 1; 0 1 0 1];
%t=[0 1 1 1];

%f=A+BC
%x = [0 0 0 0 1 1 1 1; 0 0 1 1 0 0 1 1;0 1 0 1 0 1 0 1];  %input
%t = [0 0 0 1 1 1 1 1];           %target

%f=A(B+C)
%x = [0 0 0 0 1 1 1 1; 0 0 1 1 0 0 1 1;0 1 0 1 0 1 0 1];  %input
%t = [0 0 0 0 0 1 1 1];           %target

%3-in-XOR(sum of 3-in 1 bit adder)
% x = [0 0 0 0 1 1 1 1; 0 0 1 1 0 0 1 1;0 1 0 1 0 1 0 1];  %input
% t = [0 1 1 0 1 0 0 1];           %target

%2-in-XOR
% x=[0 0 1 1; 0 1 0 1];
% t=[0 1 1 0];

x=[0 0 1000 1; 0 1 0 1];
t=[0 1000 1 0];

% [x,meanx,stdx] = prestd(x1);
% [t,meant,stdt] = prestd(t1);
% x=x';
% t=t';


[ni N] = size(x);
[no N] = size(t);

%nh = 2;  % no of hidden unit 
errlimit=0.05;      %maximum error limit
Ntrials = 10;       % no. of trials for choosing the best network
out=zeros(no,N); % output after the backpropagation
for j=1:10
    nh=j;               %no.of hidden unit
    for trial=1:Ntrials
    %initialization
    wih =randn(nh,ni+1);
    who =randn(no,nh+1);
    neth=zeros(no,nh);
    outh=zeros(no,nh);
    delh=zeros(no,nh);
    netk=0;
    outk=0;
    delk=0;
%out=zeros(no,N);

    iteration = 0; %c=number of iteration

     while(iteration < 1000)
        iteration = iteration+1;
        for i = 1:N    
            for j = 1:nh
                neth(j) = wih(j,1:end-1)*x(:,i)+wih(j,end);
                %outh(j) = 1./(1+exp(-neth(j)));
                outh(j) = tansig(neth(j));
            end
        % hidden to output layer
            for k = 1:no            
                netk(k) = who(k,1:end-1)*outh' + who(k,end);
                outk(k) = 1./(1+exp(-netk(k))); 
                delk(k) = outk(k)*(1-outk(k))*(t(k,i)-outk(k)); 
            end
         % back propagation 
         
            for j = 1:nh
                s=0; 
                 for k = 1:no 
                     s = s + who(k,j)*delk(k); 
                end
                delh(j) = outh(j)*(1-outh(j))*s; %for each hidden unit 
            end 
           
            for k = 1:no
                for l = 1:nh
                    who(k,l) = who(k,l)+.5*delk(k)*outh(l);
                end
                who(k,l+1) = who(k,l+1)+1*delk(k)*1;
            end  
            for j = 1:nh
                for p = 1:ni
                    wih(j,p) = wih(j,p)+.5*delh(j)*x(p,i);
                end
                    wih(j,p+1) = wih(j,p+1)+1*delh(j)*1;
            end    
        end   
     end
h = tansig(wih*[x;ones(1,N)]);
%h=1./(1+exp(wih*[x;ones(1,N)]))
y = logsig(who*[h;ones(1,N)]);
error=abs(t-y);
errmax=max(max(error));
        if errmax<errlimit
            out=y;
        end  
    end
end
out=out
%out=(out)
%a= poststd(out,meant,stdt)
toc
 I try to make the input to zero mean and unit std. But its not working. Whats the problem?
Can anyone help me??


 
0
files1 (71)
10/15/2012 10:15:08 AM
comp.soft-sys.matlab 211264 articles. 26 followers. lunamoonmoon (257) is leader. Post Follow

1 Replies
2208 Views

Similar Articles

[PageSpeed] 39

"shamima" wrote in message <k5gnnc$qp7$1@newscl01ah.mathworks.com>...
> I'm working backpropagation for xor training. the following code works for xor function but it cannot work  input=[0 0;0 1; 1000 0; 1 1],t=[0;1000;1;0]
> Here is my code:

C1: After writing a long code with loops use
      SELECT ALL
      SMART INDENT

>  clear all;
> clc;
> tic,
> 
> %2-in-and
> %x=[0 0 1 1; 0 1 0 1];
> %t=[0 0 0 1];
> 
> %2-in-or
> %x=[0 0 1 1; 0 1 0 1];
> %t=[0 1 1 1];
> 
> %f=A+BC
> %x = [0 0 0 0 1 1 1 1; 0 0 1 1 0 0 1 1;0 1 0 1 0 1 0 1];  %input
> %t = [0 0 0 1 1 1 1 1];           %target
> 
> %f=A(B+C)
> %x = [0 0 0 0 1 1 1 1; 0 0 1 1 0 0 1 1;0 1 0 1 0 1 0 1];  %input
> %t = [0 0 0 0 0 1 1 1];           %target
> 
> %3-in-XOR(sum of 3-in 1 bit adder)
> % x = [0 0 0 0 1 1 1 1; 0 0 1 1 0 0 1 1;0 1 0 1 0 1 0 1];  %input
> % t = [0 1 1 0 1 0 0 1];           %target
> 
> %2-in-XOR
> % x=[0 0 1 1; 0 1 0 1];
> % t=[0 1 1 0];
> 
> x=[0 0 1000 1; 0 1 0 1];
> t=[0 1000 1 0];
> 
> % [x,meanx,stdx] = prestd(x1);
> % [t,meant,stdt] = prestd(t1);
> % x=x';
> % t=t';
> 
> [ni N] = size(x);
> [no N] = size(t);
> 
> %nh = 2;  % no of hidden unit 
> errlimit=0.05;      %maximum error limit
> Ntrials = 10;       % no. of trials for choosing the best network
> out=zeros(no,N); % output after the backpropagation
> for j=1:10
>     nh=j;               %no.of hidden unit
>     for trial=1:Ntrials
>     %initialization
>     wih =randn(nh,ni+1);
>     who =randn(no,nh+1);

C2. Initial weights should be smaller to prevent sigmoid saturation

>     neth=zeros(no,nh);
>     outh=zeros(no,nh);
>     delh=zeros(no,nh);
>     netk=0;
>     outk=0;
>     delk=0;
> %out=zeros(no,N);
> 
>     iteration = 0; %c=number of iteration
> 
>      while(iteration < 1000)
>         iteration = iteration+1;

C3: Remove semicolon to help keep track. Also add

       elapsed time = toc

>         for i = 1:N    
>             for j = 1:nh

C4: UHOH! You are already in a "j" loop

>                 neth(j) = wih(j,1:end-1)*x(:,i)+wih(j,end);
>                 %outh(j) = 1./(1+exp(-neth(j)));
>                 outh(j) = tansig(neth(j));
>             end
>         % hidden to output layer
>             for k = 1:no            
>                 netk(k) = who(k,1:end-1)*outh' + who(k,end);
>                 outk(k) = 1./(1+exp(-netk(k))); 
>                 delk(k) = outk(k)*(1-outk(k))*(t(k,i)-outk(k)); 
>             end
>          % back propagation 
>          
>             for j = 1:nh

See C4

>                 s=0; 
>                  for k = 1:no 
>                      s = s + who(k,j)*delk(k); 
>                 end
>                 delh(j) = outh(j)*(1-outh(j))*s; %for each hidden unit 
>             end 
>            
>             for k = 1:no
>                 for l = 1:nh
>                     who(k,l) = who(k,l)+.5*delk(k)*outh(l);
>                 end
>                 who(k,l+1) = who(k,l+1)+1*delk(k)*1;
>             end  
>             for j = 1:nh

See C4

>                 for p = 1:ni
>                     wih(j,p) = wih(j,p)+.5*delh(j)*x(p,i);
>                 end
>                     wih(j,p+1) = wih(j,p+1)+1*delh(j)*1;
>             end    
>         end   
>      end

C5: Learning Rates are too high. Will saturate the sigmoids.

> h = tansig(wih*[x;ones(1,N)]);
> %h=1./(1+exp(wih*[x;ones(1,N)]))
> y = logsig(who*[h;ones(1,N)]);
> error=abs(t-y);
> errmax=max(max(error));
>         if errmax<errlimit
>             out=y;
>         end  
>     end
> end
> out=out

C6: You don't keep the best of the 10 trials 

> %out=(out)
> %a= poststd(out,meant,stdt)
> toc
>  I try to make the input to zero mean and unit std. But its not working. Whats the problem?
> Can anyone help me??

I only ran your 2-input XOR and made the 6 above comments. 

Search the archives for old posts of mine re the probability of convergence for the XOR design. It is surprisingly low. Therefore, I suggest you run at least 10 weight initialization trials for each value of nhi. 

As for your initial problem with 1000. The solution is obvious:

When you are dealing with sigmoids,

1. Normalize your inputs (help/doc zscore)
2. Use nonsaturating initial weights
Search comp.ai.neural-nets using the keyword nonsaturating

Hope this helps.

Greg

 
0
heath (3983)
10/15/2012 4:57:08 PM
Reply: