f

#### Backpropagation for input=[0 0;0 1; 1000 0; 1 1],t=[0;1000;1;0]

```I'm working backpropagation for xor training. the following code works for xor function but it cannot work  input=[0 0;0 1; 1000 0; 1 1],t=[0;1000;1;0]
Here is my code:
clear all;
clc;
tic,

%2-in-and
%x=[0 0 1 1; 0 1 0 1];
%t=[0 0 0 1];

%2-in-or
%x=[0 0 1 1; 0 1 0 1];
%t=[0 1 1 1];

%f=A+BC
%x = [0 0 0 0 1 1 1 1; 0 0 1 1 0 0 1 1;0 1 0 1 0 1 0 1];  %input
%t = [0 0 0 1 1 1 1 1];           %target

%f=A(B+C)
%x = [0 0 0 0 1 1 1 1; 0 0 1 1 0 0 1 1;0 1 0 1 0 1 0 1];  %input
%t = [0 0 0 0 0 1 1 1];           %target

%3-in-XOR(sum of 3-in 1 bit adder)
% x = [0 0 0 0 1 1 1 1; 0 0 1 1 0 0 1 1;0 1 0 1 0 1 0 1];  %input
% t = [0 1 1 0 1 0 0 1];           %target

%2-in-XOR
% x=[0 0 1 1; 0 1 0 1];
% t=[0 1 1 0];

x=[0 0 1000 1; 0 1 0 1];
t=[0 1000 1 0];

% [x,meanx,stdx] = prestd(x1);
% [t,meant,stdt] = prestd(t1);
% x=x';
% t=t';

[ni N] = size(x);
[no N] = size(t);

%nh = 2;  % no of hidden unit
errlimit=0.05;      %maximum error limit
Ntrials = 10;       % no. of trials for choosing the best network
out=zeros(no,N); % output after the backpropagation
for j=1:10
nh=j;               %no.of hidden unit
for trial=1:Ntrials
%initialization
wih =randn(nh,ni+1);
who =randn(no,nh+1);
neth=zeros(no,nh);
outh=zeros(no,nh);
delh=zeros(no,nh);
netk=0;
outk=0;
delk=0;
%out=zeros(no,N);

iteration = 0; %c=number of iteration

while(iteration < 1000)
iteration = iteration+1;
for i = 1:N
for j = 1:nh
neth(j) = wih(j,1:end-1)*x(:,i)+wih(j,end);
%outh(j) = 1./(1+exp(-neth(j)));
outh(j) = tansig(neth(j));
end
% hidden to output layer
for k = 1:no
netk(k) = who(k,1:end-1)*outh' + who(k,end);
outk(k) = 1./(1+exp(-netk(k)));
delk(k) = outk(k)*(1-outk(k))*(t(k,i)-outk(k));
end
% back propagation

for j = 1:nh
s=0;
for k = 1:no
s = s + who(k,j)*delk(k);
end
delh(j) = outh(j)*(1-outh(j))*s; %for each hidden unit
end

for k = 1:no
for l = 1:nh
who(k,l) = who(k,l)+.5*delk(k)*outh(l);
end
who(k,l+1) = who(k,l+1)+1*delk(k)*1;
end
for j = 1:nh
for p = 1:ni
wih(j,p) = wih(j,p)+.5*delh(j)*x(p,i);
end
wih(j,p+1) = wih(j,p+1)+1*delh(j)*1;
end
end
end
h = tansig(wih*[x;ones(1,N)]);
%h=1./(1+exp(wih*[x;ones(1,N)]))
y = logsig(who*[h;ones(1,N)]);
error=abs(t-y);
errmax=max(max(error));
if errmax<errlimit
out=y;
end
end
end
out=out
%out=(out)
%a= poststd(out,meant,stdt)
toc
I try to make the input to zero mean and unit std. But its not working. Whats the problem?
Can anyone help me??

``` 0 10/15/2012 10:15:08 AM comp.soft-sys.matlab  211264 articles. 26 followers. 1 Replies 2208 Views Similar Articles

[PageSpeed] 39

```"shamima" wrote in message <k5gnnc\$qp7\$1@newscl01ah.mathworks.com>...
> I'm working backpropagation for xor training. the following code works for xor function but it cannot work  input=[0 0;0 1; 1000 0; 1 1],t=[0;1000;1;0]
> Here is my code:

C1: After writing a long code with loops use
SELECT ALL
SMART INDENT

>  clear all;
> clc;
> tic,
>
> %2-in-and
> %x=[0 0 1 1; 0 1 0 1];
> %t=[0 0 0 1];
>
> %2-in-or
> %x=[0 0 1 1; 0 1 0 1];
> %t=[0 1 1 1];
>
> %f=A+BC
> %x = [0 0 0 0 1 1 1 1; 0 0 1 1 0 0 1 1;0 1 0 1 0 1 0 1];  %input
> %t = [0 0 0 1 1 1 1 1];           %target
>
> %f=A(B+C)
> %x = [0 0 0 0 1 1 1 1; 0 0 1 1 0 0 1 1;0 1 0 1 0 1 0 1];  %input
> %t = [0 0 0 0 0 1 1 1];           %target
>
> %3-in-XOR(sum of 3-in 1 bit adder)
> % x = [0 0 0 0 1 1 1 1; 0 0 1 1 0 0 1 1;0 1 0 1 0 1 0 1];  %input
> % t = [0 1 1 0 1 0 0 1];           %target
>
> %2-in-XOR
> % x=[0 0 1 1; 0 1 0 1];
> % t=[0 1 1 0];
>
> x=[0 0 1000 1; 0 1 0 1];
> t=[0 1000 1 0];
>
> % [x,meanx,stdx] = prestd(x1);
> % [t,meant,stdt] = prestd(t1);
> % x=x';
> % t=t';
>
> [ni N] = size(x);
> [no N] = size(t);
>
> %nh = 2;  % no of hidden unit
> errlimit=0.05;      %maximum error limit
> Ntrials = 10;       % no. of trials for choosing the best network
> out=zeros(no,N); % output after the backpropagation
> for j=1:10
>     nh=j;               %no.of hidden unit
>     for trial=1:Ntrials
>     %initialization
>     wih =randn(nh,ni+1);
>     who =randn(no,nh+1);

C2. Initial weights should be smaller to prevent sigmoid saturation

>     neth=zeros(no,nh);
>     outh=zeros(no,nh);
>     delh=zeros(no,nh);
>     netk=0;
>     outk=0;
>     delk=0;
> %out=zeros(no,N);
>
>     iteration = 0; %c=number of iteration
>
>      while(iteration < 1000)
>         iteration = iteration+1;

C3: Remove semicolon to help keep track. Also add

elapsed time = toc

>         for i = 1:N
>             for j = 1:nh

C4: UHOH! You are already in a "j" loop

>                 neth(j) = wih(j,1:end-1)*x(:,i)+wih(j,end);
>                 %outh(j) = 1./(1+exp(-neth(j)));
>                 outh(j) = tansig(neth(j));
>             end
>         % hidden to output layer
>             for k = 1:no
>                 netk(k) = who(k,1:end-1)*outh' + who(k,end);
>                 outk(k) = 1./(1+exp(-netk(k)));
>                 delk(k) = outk(k)*(1-outk(k))*(t(k,i)-outk(k));
>             end
>          % back propagation
>
>             for j = 1:nh

See C4

>                 s=0;
>                  for k = 1:no
>                      s = s + who(k,j)*delk(k);
>                 end
>                 delh(j) = outh(j)*(1-outh(j))*s; %for each hidden unit
>             end
>
>             for k = 1:no
>                 for l = 1:nh
>                     who(k,l) = who(k,l)+.5*delk(k)*outh(l);
>                 end
>                 who(k,l+1) = who(k,l+1)+1*delk(k)*1;
>             end
>             for j = 1:nh

See C4

>                 for p = 1:ni
>                     wih(j,p) = wih(j,p)+.5*delh(j)*x(p,i);
>                 end
>                     wih(j,p+1) = wih(j,p+1)+1*delh(j)*1;
>             end
>         end
>      end

C5: Learning Rates are too high. Will saturate the sigmoids.

> h = tansig(wih*[x;ones(1,N)]);
> %h=1./(1+exp(wih*[x;ones(1,N)]))
> y = logsig(who*[h;ones(1,N)]);
> error=abs(t-y);
> errmax=max(max(error));
>         if errmax<errlimit
>             out=y;
>         end
>     end
> end
> out=out

C6: You don't keep the best of the 10 trials

> %out=(out)
> %a= poststd(out,meant,stdt)
> toc
>  I try to make the input to zero mean and unit std. But its not working. Whats the problem?
> Can anyone help me??

I only ran your 2-input XOR and made the 6 above comments.

Search the archives for old posts of mine re the probability of convergence for the XOR design. It is surprisingly low. Therefore, I suggest you run at least 10 weight initialization trials for each value of nhi.

As for your initial problem with 1000. The solution is obvious:

When you are dealing with sigmoids,

1. Normalize your inputs (help/doc zscore)
2. Use nonsaturating initial weights
Search comp.ai.neural-nets using the keyword nonsaturating

Hope this helps.

Greg

``` 0 10/15/2012 4:57:08 PM
 Reply: