f



Reservoirs as information consentrators

I happened to notice a coment in a thesis about reservoir computing the idea that a resevoir could consentrate information from input data with sparse statistics.
That would be amazing if it were true?  Especially if the effect were significant.  

http://www.ai.rug.nl/~mwiering/Thesis_Adrian_Millea.pdf

I'll take that with a grain of salt for the moment.

A fast way to do a random projection (RP) is to do random sign flipping of the input data and then feed that through a Walsh Hadamard transform.
If you feed the output of the RP back into its input you have a stable reservoir that you can add information into.

https://drive.google.com/open?id=0BwsgMLjV0BnhWWZCR3kyVmlVNnc
0
sean
12/4/2016 3:18:30 AM
comp.ai.neural-nets 5773 articles. 2 followers. tomhoo (63) is leader. Post Follow

4 Replies
419 Views

Similar Articles

[PageSpeed] 32

Here's the reference Adrian Millea gave:

http://minds.jacobs-university.de/sites/default/files/uploads/papers/STMEch=
oStatesTechRep.pdf

It is interesting as well that reservoirs are also connected to thermodynam=
ics.
If you have one reservoir with information circulating in it and then break=
 some of the output to input connections, linking those broken outputs and =
input to a empty second reservoir.  The information would equilibrate betwe=
en the two. As if you had one chamber containing a gas and one in a vacuum =
state and you connected the two. 
0
sean
12/4/2016 4:10:54 AM
There are actually a lot of similarities to physical systems.  You can regard such an oscillating random projection system as a complicated resonator.  I suppose if you feed in sparse data you will get standing waves.  

Generally there are quite complicated things going on because as you keep adding information eventually there is no more state space to store that and older information is dissipated. I'm sure the maths of that is pretty difficult. 
0
Sean
12/6/2016 1:16:33 AM
On Monday, December 5, 2016 at 5:16:36 PM UTC-8, Sean O'Connor wrote:
> There are actually a lot of similarities to physical systems.  You can regard such an oscillating random projection system as a complicated resonator.  I suppose if you feed in sparse data you will get standing waves.  
> 
> Generally there are quite complicated things going on because as you keep adding information eventually there is no more state space to store that and older information is dissipated. I'm sure the maths of that is pretty difficult.

 I could see oscillation in flat square space with a detector NN looking at it.
 If this flat space has truly randomness and is a lot smaller
then a image that it going to be tied to later than it could be a hash tag
generator.  The hash image is the input to a trained decode NN which outputs a image.


 Hash image is inputted to a decoder NN and trained to generate a image of a
cat.
 With a detector nn that activates when it sees a cat in the real world. Look
 at this encoder NN while training. And stop when the Cat nn activates.


  This way a hash image could be a bots internal language or description
of the world.

 If a very large area flat space does stabilize into many stable
oscillations. A oscillation could use drive a decoder nn to play video or
 the motion of a robot.
0
keghn
12/15/2016 12:16:48 AM
The song of the resonator.  
It would be another thing helping the brain detect patterns in the environment.  I guess anything that helps, helps.
0
sean
12/21/2016 2:44:12 PM
Reply: