How to reproduce key aspects of Figure 3 with Auryn v0.8-beta3

Questions concerning the use of Auryn classes. If you want to contribute a howto for a given example code please consider getting write access for the Dokuwiki.
Post Reply
Becky
Posts: 3
Joined: Sun Sep 04, 2016 12:10 pm

How to reproduce key aspects of Figure 3 with Auryn v0.8-beta3

Post by Becky »

Moderator comment: This topic was split off from here: https://www.fzenke.net/auryn/forum/view ... p?f=3&t=42

Thanks for the very detailed description.
It seems I meet some problems to reproduce the experiment result in your paper like picture 3d.
:(

I use the first three patterns and here is the outcome. (attachment)

Do you preserve the old version which can reproduce the data ?

Thanks! ;)
Attachments
rf2.png
rf2.png (7.54 KiB) Viewed 10420 times
User avatar
zenke
Site Admin
Posts: 156
Joined: Tue Oct 14, 2014 11:34 am
Location: Basel, CH
Contact:

Re: How to divide the neurons into groups in paper "Diverse synaptic plasticity..."

Post by zenke »

Hi,
that looks indeed a bit different, but it seems as if there is delay activity after all. That's already already a good thing :-) Are you currently using both the most recent version of Auryn and code from the orchestrated STDP repository? How do you call the code? Just with "run_orchestrated_stdp.sh"? And what do you mean by only using the first 3 patterns? If you give me this info, I will try to reproduce your setup here and see if I am getting the same problem here.
And to answer your question: Yes, the original code is stored in the github repository https://github.com/fzenke/pub2015orches ... /orch-v1.0 and a detailed discussion of howto make it work is here http://www.fzenke.net/auryn/forum/viewt ... 9&start=10
Becky
Posts: 3
Joined: Sun Sep 04, 2016 12:10 pm

Re: How to divide the neurons into groups in paper "Diverse synaptic plasticity..."

Post by Becky »

Yes. This problem have already raised when I tried to use four pattern. For four pattern, sometimes, two kinds of pattern will mixed like the green and red in the pictures above.
I try to find the reason behind that because the file you email to me doesn't have this phenomenon. I got one assumption; When compiling the "pub2015..." files, maybe you can have your path to auryn checked to make sure the path are connected to the right file.

The branch I use is "git checkout -b last-release v0.8.0-beta3".
Thanks! :geek:
User avatar
zenke
Site Admin
Posts: 156
Joined: Tue Oct 14, 2014 11:34 am
Location: Basel, CH
Contact:

Re: How to divide the neurons into groups in paper "Diverse synaptic plasticity..."

Post by zenke »

It can happen sometimes that the patterns that are learned have some overlap. That's what you would describe as "mixing". The reason is simply that the input patterns have up to 80% overlap (eg. circle and square). However, some overlap in the cell assemblies is usually not a problem as long as you can still differentiate these patterns. Some overlap is in fact even desirable, because it allows you to store more patterns ... as in the Hopfield network.
Now, which assemblies the network forms and how the patterns overlap can depend on a variety of parameters and most of all on the order of stimuli during the initial minutes of stimulation. Either way our results should at least be qualitatively the same.

So let's see what I am getting. Here the code that I am using right now:

Code: Select all

Auryn v0.8.0-beta3-10-g177ca65 
https://github.com/fzenke/auryn
commit 177ca659ffd78e9daa673e7605e84c4de8f3a2c4
Date:   Fri Sep 2 00:27:39 2016 -0700

Simulation code 
https://github.com/fzenke/pub2015orchestrated
commit 994949442b4068388f2a306f403cc0c8d8a5f29d
Date:   Thu Sep 1 19:02:04 2016 -0700
Then running "./run_orchestrated_stdp" ...
This first runs phase rf1 once and generates a rf2
So things seem to look good during rf1.
rf1.png
rf1.png (145.07 KiB) Viewed 10415 times
It seems as if there is some degree of delay activity for all four patterns. However, there is some overlap between Pattern 2 and Pattern 4 which is fine.

But you are right, something happens during phase rf2:
rf2.png
rf2.png (93.16 KiB) Viewed 10415 times
This seems fine too, but it is different from the paper insofar as Pattern 4 is somehow struggling to keep its activity up. Otherwise there is selective delay activity with some, but not much overlap. And the activity of Pattern 4 might be linked to this overlap.

In the latest commit on the master branch of pub2015orchestrated I played with the depression time constant of STP. You can set it in the file "globalvars.sh". I set it to TAUD="0.15" (150ms) because learning was faster. Synapses can produce more recurrent drive because they recover faster, but it is possible that this change also causes the later change in rf2.

In the paper we used tau_d=200ms and the only reason why I explored changing it was because there was a bug in the STP implementation for sims running in parallel (https://www.fzenke.net/auryn/forum/view ... p=155#p155) which was fixed in Auryn v0.7. This change and complete rewrite of the random number generation somewhat affected the behavior of the original simulation and although all the parameter sets in the newer backports of pub2015orch.. work and produce qualitatively comparable results, the details might be different. If you insist on non-overlapping representations, I'd recommend playing with tau_d=200ms or maybe tau_d=180ms or try a different master random seed which will change the order of stimulus presentations. I don't have much time to work on this now, but I will try tau_d=180ms out of interest now. Lemme know what you find.

Cheers,
F
User avatar
zenke
Site Admin
Posts: 156
Joined: Tue Oct 14, 2014 11:34 am
Location: Basel, CH
Contact:

Re: How to divide the neurons into groups in paper "Diverse synaptic plasticity..."

Post by zenke »

Here is the result for tau_d=180ms.
This seems to resolve the overlap "problem" if you want to avoid that, but
it does not seem to change much to the different firing rate of the patterns.
rf2b.png
rf2b.png (79.67 KiB) Viewed 10404 times
This phenomenon is likely linked to an unequal allocation of cells into the assemblies. From a quick analysis of the cells counted in rf1.pat I get

Code: Select all

# Pattern 0 sparseness=0.195068
# Pattern 1 sparseness=0.196045
# Pattern 2 sparseness=0.173828
# Pattern 3 sparseness=0.242432
showing that pattern four is also the "largest". My best guess is that this differences is due to a different random seed in stimulus group, but other differences in the Auryn v0.8 beta version could be the possible cause of this discrepancy.
Post Reply