Hi All

I present the method I use for weight and variance determination in NNs.

I developed a direct search method in the early 1990s which is a updated
and improved version of the orginal Westinghouse Electric Pattern Search
of Hooke and Jeeves.  It is called "Vector-Step Pattern Search" and it
uses "vector steps" of 100%, 10% and 1% for a step.  Until down to the
1% level is exhausted is the step size finally reduced. In this way
difficult hyperspace paths may be transversed although at times it might
appear slow.  However, it is tenacious!

Vector-Step Pattern Search has been tested up to 1000 variables and
seems to perform flawlessly and very accurately.  However, it is not a 
"Global Method" and all that implies.

Paul Birke, Electrical Engineer
Guelph, ON
CANADA

**************************************************************
here is a version of vector-step in QuickBasic:==> (enjoy!!)
**************************************************************


10 PRINT
"******************************************************************"
11 PRINT
"*                                                                *"
12 PRINT "*       NONLINEAR PROGRAMMING VIA VECTORED EXPLORATORY
STEP      *"
14 PRINT "*       PAUL VICTOR BIRKE, P.
ENG.                               *"
17 PRINT "*       COPYRIGHT SEPTEMBER 25th,
1993                           *"
18 PRINT
"*                                                                *"
20 PRINT
"******************************************************************"
99  DEFDBL A-H, O-Z
100 DIM X(20), XL(20), XU(20)
101 DIM BE(20), BR(20), BP(20)
999 GOTO 4000
1000 REM SUBROUTINE OBJECTIVE FUNCTION
1010 KF = KF + 1
1020 REM OBJECTIVE FUNCTION TO BE MINIMIZED UNDER THE FOLLOWING
CONSTRAINTS
1021 TAU = 1000000!: F = X(1) + TAU * (X(1) ^ 2 + X(2) ^ 2 - 1) ^ 2
1900 REM PRINT F, X(1), X(2)
1999 RETURN
2000 REM SUBROUTINE EXPLORATORY
2001 SE = 0: REM EXPLORATORY MOVE SUCCESSFUL WHEN S=1
2002 KE = KE + 1: REM COUNT EXPLORATORY MOVES
2010 FOR I = 1 TO N: BE(I) = X(I): BR(I) = BE(I): NEXT I
2020 GOSUB 1000: FE = F: FR = FE
2025 FOR NR = 1 TO 3
2030 FOR I = 1 TO N
2031   DD = D * (ABS(BR(I)) + D)
2032   IF NR = 2 THEN DD = .1 * DD
2033   IF NR = 3 THEN DD = .01 * DD
2047   X(I) = BR(I) + DD
2048   IF X(I) <= XU(I) THEN GOSUB 1000: FU = F: XU = X(I)
2049   IF X(I) > XU(I) THEN X(I) = BR(I): GOSUB 1000: FU = F: XU = X(I)
2050   X(I) = BR(I) - DD
2051   IF X(I) >= XL(I) THEN GOSUB 1000: FL = F: XL = X(I)
2052   IF X(I) < XL(I) THEN X(I) = BR(I): GOSUB 1000: FL = F: XL = X(I)
2060   IF FU >= FR AND FL >= FR THEN X(I) = BR(I): GOTO 2200
2061   IF FU < FR THEN FR = FU: X(I) = XU: BR(I) = X(I)
2062   IF FL < FR THEN FR = FL: X(I) = XL: BR(I) = X(I)
2200 NEXT I
2500 IF FR < FE THEN FE = FR: SE = 1: FOR I = 1 TO N: BE(I) = BR(I):
NEXT
2599 IF SE = 1 THEN ES = ES + 1: SE = 0
2600 NEXT NR
2999 RETURN
3000 REM SUBROUTINE PATTERN WITH ACCELERATION
3010 FOR I = 1 TO N: BP(I) = X(I): NEXT I
3020 GOSUB 1000: FP = F
3050 KP = KP + 1
3060 GOSUB 2000
3070 FOR I = 1 TO N: IF ABS(BE(I) - BP(I)) > EX GOTO 3090
3071 NEXT I
3080 D = RD * D: PRINT KF; F; X(1); X(2); D: IF D >= EX GOTO 3050 ELSE
RETURN
3090 IF FE > FP THEN FOR I = 1 TO N: X(I) = BP(I): NEXT I: GOTO 3050
ELSE FP = FE
3099 NP = 0: NPMAX = 0
3100 FOR I = 1 TO N
3110   XX = X(I)
3120   X(I) = 2 * XX - BP(I)
3130   IF X(I) > XU(I) OR X(I) < XL(I) THEN X(I) = XX
3140   BP(I) = XX
3160 NEXT I
3200 GOSUB 1000: IF F >= FP THEN FOR I = 1 TO N: X(I) = BP(I): NEXT I:
GOTO 3050
3210 IF F < FP THEN FP = F: PS = PS + 1: NP = NP + 1: IF NP < NPMAX THEN
3100 ELSE 3050
4000 REM MAIN PROGRAM
4020 N = 2
4029 PI = 3.141596
4030 DS = PI
4031 D = DS
4032 RD = .1
4033 EX = 1E-10
4040 FOR I = 1 TO N: READ XL(I): NEXT I
4041 DATA -1E10,-1E10
4050 FOR I = 1 TO N: READ XU(I): NEXT I
4051 DATA 1E10,1E10
4060 FOR I = 1 TO N: READ X(I): NEXT I
4061 DATA 1.1,.1
4110 R = 1
4111 KF = 0: REM KF IS THE NUMBER OF FUNCTION EVALUATIONS
4112 KE = 0: REM KE IS THE NUMBER OF EXPLORATORY MOVES
4113 ES = 0: REM ES IS THE NUMBER OF SUCCESSFUL EXPLORATORY MOVES
4114 KP = 0: REM KP IS THE NUMBER OF PATTERN MOVES
4115 PS = 0: REM PS IS THE NUMBER OF SUCCESSFUL PATTERN MOVES
4120 GOSUB 3000
4121 PRINT "KE="; KE, "ES="; ES, "KP="; KP, "PS="; PS
4130 FOR I = 1 TO N
4132   PRINT "X("; I; ")="; X(I)
4134 NEXT I
4140 PRINT "R="; R
4150 PRINT "MIN F(Xi)="; F
4160 PRINT "FUNCTION EVALUATIONS ="; KF
4161 PRINT : PRINT : PRINT : PRINT
4170 R = R + 1
4300 PRINT "FINAL SOLUTION OBTAINED"
4999 END

Paul Victor Birke <nonlinear@rogers.com> wrote in message 
news:<jsfjc.5470$Qcs.3916@news04.bloor.is.net.cable.rogers.com>...

> I present the method I use for weight and variance determination in NNs.
> 
> I developed a direct search method in the early 1990s which is a updated
> and improved version of the orginal Westinghouse Electric Pattern Search
> of Hooke and Jeeves.  It is called "Vector-Step Pattern Search" and it
> uses "vector steps" of 100%, 10% and 1% for a step.  Until down to the
> 1% level is exhausted is the step size finally reduced. In this way
> difficult hyperspace paths may be transversed although at times it might
> appear slow.  However, it is tenacious!
> 
> Vector-Step Pattern Search has been tested up to 1000 variables and
> seems to perform flawlessly and very accurately.  However, it is not a 
> "Global Method" and all that implies.

-----SNIP

Thanks Paul.

If anyone is going to translate the code into MATLAB, I would
appreciate a copy.

Hope this helps.

Greg