f



Clustering with Growing Neural Gas

Hello!=20

I need to use a Growing Neural Gas network for clustering.=20

I know how the algorithm works, but I'm not sure how it determines the clus=
ters. So I don't know how to interpret the result. Based on images (e.g. pa=
ge 11 in www.booru.net/download/MasterThesisProj.pdf) I guess that it sees =
all the neurons that are connected by edges as one cluster. So that you mig=
ht have two clusters for two groups of neurons each all connected. But is t=
hat really it? What if the clusters are too close?=20

I'm also unsure about the number of iterations. Let's say I have 500 data p=
oints to cluster. Once I put them all in, do I remove them and add them aga=
in to adapt the existing network? And how often do I do that?

I mean.. I have to re-assign them at some point. When adding a new neuron r=
 between two old neurons u and v with large errors then some data points fo=
rmerly belonging to u should now belong to r because it's closer. But the a=
lgorithm does not contain changing the assignment of these data points. But=
 even if I remove them after one iteration and add them all again, then the=
 false assignment of the points for the rest of that first iteration change=
s the processing of the network doesn't it?

I hope you can help me with understanding this!=20

Thanks,=20
Jessy

0
Jessica
12/15/2016 10:05:13 AM
comp.ai.neural-nets 5773 articles. 2 followers. tomhoo (63) is leader. Post Follow

2 Replies
518 Views

Similar Articles

[PageSpeed] 46

On Thu, 15 Dec 2016 02:05:13 -0800 (PST), Jessica P
<jessica.petrasch1@googlemail.com> wrote:

>Hello! 
>
>I need to use a Growing Neural Gas network for clustering. 
>
>I know how the algorithm works, but I'm not sure how it determines the clusters. So I don't know how to interpret the result. Based on images (e.g. page 11 in www.booru.net/download/MasterThesisProj.pdf) I guess that it sees all the neurons that are connected by edges as one cluster. So that you might have two clusters for two groups of neurons each all connected. But is that really it? What if the clusters are too close? 
>
>I'm also unsure about the number of iterations. Let's say I have 500 data points to cluster. Once I put them all in, do I remove them and add them again to adapt the existing network? And how often do I do that?
>
>I mean.. I have to re-assign them at some point. When adding a new neuron r between two old neurons u and v with large errors then some data points formerly belonging to u should now belong to r because it's closer. But the algorithm does not contain changing the assignment of these data points. But even if I remove them after one iteration and add them all again, then the false assignment of the points for the rest of that first iteration changes the processing of the network doesn't it?
>
>I hope you can help me with understanding this! 
>
>Thanks, 
>Jessy

I've never used Neural Gas but increasing nodes while data partitions
are being used for learning seems to me to be much like cloning.
Cloned nodes often appear in clusters around existing nodes. Clusters
of clones around previous clones rarely happens but it can. My
understanding could be way off. 

Here is a clip from my help file to describe cloning.

Cloning Nodes >>

Cloning can help when there are problems finding a neural network
configuration that will learn a complex dataset. Neural networks can
be built with cloning enabled or cloning can be enabled while the
network is learning. Hidden nodes are cloned during the learning
process when the average error has stopped reducing. The hidden node
with the greatest net input is frozen to determine if it is
contributing to the learning process. If the node is contributing to
the learning process it is cloned. Nodes that are not contributing are
skipped and the node with the next greatest net input is tested. If no
suitable hidden node can be found a new node is created and fully
connected with random low weights. It is then cloned producing an
exact copy. The node that has been cloned is frozen again and its
clone takes over in the learning process. The freeze level of the node
that has been cloned is reduced until both the node and its clone are
being used in the learning process. The learning process does not need
to restart when a node is cloned. 
 
About Clones & Cloning
 
1.     Nodes will not be cloned if the neural network is learning and
the average error is decreasing. 

2.     Clones may be produced during learning if cloning is enabled
and the average error is not decreasing.

3.     Nodes that are cloned are initially frozen.

4.     The freeze level may reduce to zero very quickly and never be
seen on the network display. 
5.     Clones are first produced in hidden layer 1, then in hidden
layer 2 and then in hidden layer 3.

6.     Clones produced in one hidden layer may move to other hidden
layers.
 
7.     Networks that have been created with no hidden nodes or
connections can use cloning.  
 
8.     Clones can be cloned. 
 
-- 
Neural Network Software for Windows                 http://www.npsnn.com

0
Stephen
12/15/2016 2:24:35 PM
Hello Jessy

why do you "need" to to use GNG? I this some sort of assignment? Just curio=
us.

Anyway, GNG can be seen as method to topologically represent finite point s=
ets or continuous probability distributions by using nodes and edges. When =
the data falls into non-overlapping clusters with an inter-cluster distance=
 much larger than the intra-cluster point distance, one can expect the netw=
ork topology to well reflect the clusters after the network has reached a c=
ertain number of nodes.

1D example A:=20
____x_xx_x_x______________________xx_xxx_x_____________________x_x_x___

In example A you can expect GNG to form 3 disctinct clusters at a network s=
ize of 6 or 7 because the edges between the clusters will get old and get d=
eleted quite soon.


1D example B:
_____xxxx_xxx_xxx____x_______x___________x_______xxxxx_xxxxxxxx_xxxxx__

In example B GNG might never form two stable disconnected structures since =
there are some data points ("noise") in between the two visible clusters. W=
henever one of these data points is chosen as an input signal an edge is in=
serted which connects the two structures (if they actually were separated).

This is not to say that GNG can not not be useful here, but instead of rely=
ing on the network structure alone to identify the clusters, one would have=
 to find another way to determine clusters (of course there might be severa=
l good clustering solutions on different spatial scales, e.g. 3 large clust=
ers vs 14 smaller ones). How to modify/enhance GNG to find "good" clusterin=
gs in the presence of noise is left as an excercise to the reader. :-)

Regarding the presentation of your say 500 data points you could either sim=
ply always choose a random point from your set (without removing it) or - t=
o really guarantee equal presentation frequency - make a random order and p=
resent the 500 points one by one. After 500 signals you make a different ra=
ndom order of your data and repeat this process until youre done (I guess y=
ou know that you need to define a stopping criterion which my influence how=
 many clusters you find. =20

Hope this makes it a bit clearer to you.

Regards
Bernd

PS: For some more intuition on related methods check out http://www.demogng=
..de
0
Bernd
12/16/2016 11:24:58 PM
Reply: