What do people mean when they say “computational x”?

Inman started off a discussion in the CCNR mailing list after Emails announcing two new Journals: Computational Intelligence and Neuroscience, and Cognitive Computation. He wonders what is meant by computation in those titles, and provides four possible options:

(A) That subset of intelligence we display when we are doing computations, in Turing’s sense of the word. I.e. when we are following explicit algorithms, such as doing long division. I.e. ‘computational’ limits the domain of intelligence being studied.

(B) Studying intelligence (not so limited) with the working assumption that the brain is some form of computer. I.e. using a computer and computations as a model for the brain.

(C) Studying intelligence (not so limited) in ways that make the use of computers a practical tool, allowing types of studies unavailable (or more difficult) without their assistance. I.e. using computers as tools.

or indeed (D) any further variation you might propose.

At least two people in the post have suggested most people mean the same thing.

With his permission, I have converted Inman’s Email into a poll. I have made three different ones. I think the term that follows “computation” might change the results.

The idea of the poll is, of course, not to sort out the mess by popular vote. I’d like to see if – as some suggest – one of the options is indeed the most common answer throughout (but also because I really wanted to test the poll tool).



Thanks for voting! And importantly, if you voted for the option “other”, then please tell us what you mean.


Continuous-time recurrent ‘neural’ network as a model of intracellular signalling

This is a bit of a long post (took me the better part of today). It shall serve as my initial train of thought on something that I would like to refine, extend, and publish somewhere more seriously. So, if you find the post interesting, and/or you think you can help me shape it such that it becomes so, then don’t hesitate to contact me.

Continue reading “Continuous-time recurrent ‘neural’ network as a model of intracellular signalling”

A second level of ‘interestingness’ and a more minimal non-monotonic transfer function

The following is a follow-up to my transfer function post a couple of days ago. Two persons have suggested using a simpler non-monotonic function such as the Gaussian. I will incorporate results with that transfer function. Also, I will add one more category to my previous minimalistic categorizationof asymptotic dynamics: point attractors versus other dynamics. I am now including periodic orbits. Still very minimalistic.

Continue reading “A second level of ‘interestingness’ and a more minimal non-monotonic transfer function”

self-determined, environmentally-determined behaviors and more interestingly those changing over time

I’m writing this on the train, taking ‘advantage’ of the fact that I took the wrong one today – this one is taking me to Eastbourne. Luckily I’ve got my mp3 player filled to the brim with great tunes and the notebook to write this down on.

First. Regarding my previous post and the set of experiments that I have been thinking about. It is not purely non-environmentally determined behaviour that concerns me with these experiments. Well it is, but I can be more specific than that:

On the one extreme one has reactive systems. A good example of a system on this end of the spectrum is a leaf floating downstream in a river. If we choose the boundaries of our system of interest as ‘the leaf’ then we can say that its future behaviour is purely determined by external variables, its environment (in this case the water in the river). Ok, so not purely, the leaf does have its own small mass and thus inertia, but you get the idea.

On the other end of the spectrum we have purely internally determined systems. Their environment does not determine the future behaviour of these systems; in fact they are not even influenced by it. These are solely self-governing systems. Although autonomous, in the traditional use of the term (cf. Maggie Boden), these systems are not of interest because they cannot readily adapt to their changing environments. A good example of one such system is the classical humanoid robot, ASIMOV, whose each step is determined by a large amount of internal calculations based on the positions of its limbs and the distribution of the weight of its body as internally modelled. This, of course, is just as undesirable as our previous leaf system. And not for too different a reason: lack of adaptive power.

We are interested in neither ends of the spectrum. In fact, the interesting systems will be neither systems whose ‘reactivism/internalism’ stays constant over time, even if it has some in-between mix along the spectrum just described. Our hypothesis is that the cognitively rich systems will be those whose level of involvement with the environment (in the case of reactive systems) and/or de-coupling from the environment (in the case of internally-driven systems) itself changed over time to adapt to the changing demands of their survival. A good example of such a system is a group of humans in a raft navigating in white waters (btw it’s a lot of fun). Our system of interest here is the humans+raft+paddles. At some points this system is fighting the current, attempting move left, right or even to stop going forward. At other points the system is letting itself go, letting the current take it where it wants. It is this interaction, letting go and taking back, that is of interest.

And to summarise it all two figures taken from some slides I did for a Life and Mind seminar a while back:


To come back: all of this can be applied to the example experiment described in the previous post. The environment sometimes drives the system: this shows because it sometimes orients to the regions with hotter temperature. But the system is also internally driven: this shows in that it can change directions even on nearly identical runs – with no environmental difference to drive it.

An interesting result would be to (a) calculate a measure of ‘reactivity’ (i.e. how environmentally driven or internally driven a situated system is) and (2) plot it over time for one of the evolved agents. The hypothesis would be that the line would go up and down over time as the system follows the environment to go towards the hot regions but sometimes abandons the signal altogether to follow its internal drive to do something different every-time. In other words, sometimes internally driven, sometimes environmentally driven, most of the times some combination of both, but most importantly actively changing how much so over time.

The last time I mentioned this to Inman he mentioned the work of Andy Wuensche using cellular automata. I looked at it at some point but didn’t quite get it – I think it may have to do with me not liking CAs that much – their discreteness makes me clinch sometimes.

Second. There a number of methods to measure how chaotic a system is from one recording of the systems behaviour (e.g. time-embedding). The people studying complex systems have had to do this because of several reasons (e.g. limited data, recordings from the brain, etc). One question that I have is, does it make sense to study my evolved agents (which are simple 3D dynamical systems and whose full behaviour I can have recordings of) with such tools? Even though I could easily do something different and perhaps more direct? The issue that worries me is that they are non-autonomous dynamical systems, embodied, situated and so on. Suggestions welcome.

Third. Ideally I would like to use a CTRNN for this task and not have to introduce alpha-CTRNNs. In the unfortunate case of not being able to obtain small (ie ideally 3, but maybe 4-node) circuits for the task, I will (of course) need to include as fair as possible of a comparison between the evolvability of CTRNNs against alpha-CTRNNs. But otherwise there is no reason to introduce alpha CTRNNs to the world.

This brings up an old worry of mine: am I falling in the same steps of those who start adding bells and whistles to CTRNNs to add ‘extra’ functionalities to a system that never lacked them? I’d like to think that this is not the case. Although I would rather not introduce yet-another-CTRNN-variation, I feel the motivation for the alpha version is more to do with concentrating on finding regions of ‘rich’ dynamics, then that of adding any particular behavioral functionality to the system. Despite losing biological reality to real neuron’s properties, as Randy pointed out to me,  nonmonotonic activation functions may expand the regions of interesting dynamics. I feel this is related to, for example, Randy’s motivation when looking at centre-crossing CTRNNs, and more recently at Rnn regions of parameter space. At least it is closer to that then, say, adding weight-changing rules in the hopes of obtaining learning behaviour; or adding connection-growing algorithms in the hopes of obtaining developmental agents.

Update: got back to the lab and so last night’s experiments using CTRNNs are in. Although this is all very preliminary here is a comparison of monotonic [i.e. f=tanh(x)] versus nonmonotonic [i.e. f=alpha*tanh(x) + (1-alpha)*sin(x)] activation functions in CTRNNs for non-environmentally determined behaviours using 3-node circuits:

The solid lines depict the average over the best fitness on 20 evolutionary runs for 500 generations for monotonic (red) and nonmonotonic (blue) activation functions. The triangles at the end show the standard deviation at the end of the evolutionary runs for each experiment colored respectively. Too early to say something concrete, I will leave a proper discussion for later. Also I’m running exact same experiments using the more studied and traditionally used logistic sigmoid.

The plan now is to re-do these experiments using the logistic sigmoid, the tanh and the tanh+sin function for comparison using 2, 3 and 4 node CTRNNs, for a little longer (1000 generations) and with the same number of repetitions (20 evolutionary runs each) to start with (I may do 20 more for each if needed); for a total of 180 evolutionary runs. It takes between 100 and 150 hours to run 9 evolutionary runs on one computer, so if I use 20 it should be done in about a week. I’ll start them up tomorrow.

By the way, very nice landscapes along the way: smooth hills lightly covered with snow – beautiful. Will tell Lilia that we should travel this way soon. Also, it is sort of nice that I got to ride here today. Lilia and I are thinking of moving to London soon (towards the summer). I will have to do the commute, for a change. And so I’m actually looking forward to the commuting – as I see it an opportunity to get a lot of work done without as many of the usual distractions (ie. the internet).

ps. you know you have been in Britain for too long when you are calling security about an abandoned backpack in one of the carriages, only to have to wait for the guy to return from the men’s room a couple of minutes later.

Non environmentally determined decision making and chaotic CTRNNs

A good proportion of the tasks that our neuron-inspired dynamical system controllers are evolved for have a very strong environmentally determined component. In other words, what they do next is (by far) predictable more by their current environment then by their internal dynamics. Yet the motivation for most of us (at least those interested in cognition, however minimal) using such internally rich systems is to get coupled-with-the-environment but at the same time interestingly autonomous behaviors.

So I have been thinking about a smallish project (perhaps in the context of the upcoming ECAL? keeping in mind that I will be doing so while at the same time writing my thesis). The motivation would be to evolve networks to do a simple task in a simple environment in several different ways. The hypothesis is that the systems will evolve some form of chaotic internal dynamics to generate the variety of behaviors required. This is interestingly related to the random number generator in Ashby’s Homeostat, but more generally chaotic behavior has been observed in the nervous system and the contributions of chaotic dynamics to cognitive processes has been discussed by for example, Freeman and Skarda. In fact, for Walter Freeman chaotic dynamics in the internal dynamics of the living organism are at the very root of autonomy, learning, memory, and even creativity.

There would be two main objectives in such work. The first would be to measure the reactivity in evolved situated agents for different tasks. Reactivity being how much the behavior of a situated agent is determined by the environment as opposed to its internal dynamics. The second idea would be to compare CTRNNs with alpha-CTRNNs. I’m calling alpha-CTRNNs the brother/cousin of the CTRNN that I discussed from Ollie’s work in a previous post. Alpha is a parameter for each neuron that determines the monoticity of its activation function. The question that we would ask is, do nonmonotonic transfer functions facilitate chaotic behavior? I have found already a couple of papers (here and here) that suggest that nonmonotonic transfer functions may lead to macroscopic chaos in attractor neural networks. I still have to understand them better.

So what is the simplest version of an experiment that can shed some light on these questions?

I have been inspired by the simplicity of the models that physicists / dynamicists use. In particular, I have become interested in the forced double-well oscillator studied by Francis Moon and colleagues at Cornell. I think this is also known as Duffing’s mechanical oscillator (see here, here). As Strogatz points out, as soon as nonautonomous systems (in the DS sense) are considered strange attractors start turning up everywhere. I won’t go into any detail about the model, if you want to learn more about it you can start off with Strogatz’ ND&C p.441-446. The important points are that the model is extremely simple and that it produces chaotic behavior.

So what is the ‘alify’ version of that simple model? An agent in a 1D world with a temperature gradient that follows the following form:


The agent senses the local temperature and can move left and right.

The first task could be to simply go to the places of highest temperature – highly reactive systems would be expected. The second task would be to do the same, but in a variety of different ways – highly non-reactive / ‘internally-driven’ systems would be expected for this one.

I made some code for this yesterday. Is all only about 350 lines of C code (including the CTRNN and the GA) and ran a first batch of experiments overnight. The summary of the fitness is to maximize both time spent on hottest regions as well as diversity of behavior. I am using 3-node CTRNNs with a slightly larger family of transfer functions (each node with an evolvable alpha between 0.5-1.0) for 25000 (equivalent to 500 generations) with a population of 50 individuals. The overnight results is that 5/20 of the evolutionary runs were quite succesful.

In the next figure I depict two trajectories/behavior (red and blue) of the best evolved network. The x axis is time. The y axis is the position along the 1 dimensional physical space. The underlying shades of gray represent the temperature of the regions (white = hot, black = cold). The simulation is entirely deterministic – there is no noise. The starting states of the two trajectories are nearly identical, the difference is only in the exact starting place off by only an infinitesimal distance (0.005 to be exact).


The red and blue lines represent the behavior of two nearly indentical runs from one evolved agent. The point is that what determines which of the two peaks to head for in any given case is more up to the internal dynamics than to the environment – as the latter is nearly the same for both cases. Perhaps more interesting still is to analyse the actual internal dynamics. Is it chaotic? quasi-periodic? typical boring attractos? Although this is only preliminary, I think it might be still interesting. What do you think?

Of course, many things to do still. First, I shall investigate measures of how chaotic a non-autonomous system can be and use it to measure the best evolved agents. Second, compare evolving CTRNNs with different transfer functions (i.e. monotonic versus non). Any other ideas?

Generative music, ‘creative’ dynamical systems and CTRNN activation functions

I had the pleasure to meet with Ollie Bown and Alice Eldridge yesterday. Basically, they are working on a project in biologically inspired interactive music systems associated with the Live Algorithms for Music Group, based in Goldsmiths. They are doing a bunch of interesting stuff – but here is what I liked the most about it: they have this cool software based on Max/MSP stuff (which you can ask them about) and they used CTRNNs (among other stuff) to make music. Ok, not just to make music but a system to interact with musicians.

You may be asking, so what is the difficulty in this? Getting a dynamical system to be ‘interesting’, that’s the hard part. Not too boring, but not too chaotic either. A lot of people have talked about this in various different ways. Probably discussing the ways in which people have talked about this would be useful, I can mention off the top of my head Stuart Kauffmann with his complexity and order stuff, Takashi Ikegami with his search for balance between homeostasis and chaos, and a number of people at Sussex (in particular people like Ezequiel) in the search for ‘autonomous’ agents, systems that can perhaps perform a task but be ‘idiosynchratic’ in the way they go about it. I would dare to say that this is perhaps the most interesting problem in artificial life / adaptive behaviour, because it relates to synthetising systems that can at least appear to have ‘a mind of their own’ and therefore appear to be ‘creative’ in some way or another.

This is a obviously a really big topic and it has become particularly relevant lately in our discussions in the life and mind reading group and the CCNR in general. The post here is about something that I think has potential to help out in that search. Before I continue, however, I must be careful to acknoledge that most of the ideas in here come from what Ollie showed me yesterday. I will just be exploring some of a few aspects of those things in slightly more depth here. With the hopes to generate some arts/science collaboration going.

As you know CTRNNs are the coolest dynamical systems out there 🙂 So it makes sense to start here. More seriously, there are a bunch of reasons to use CTRNNs but I’m not going to go into that here. If you are interested in nonlinear continuous-time dynamical systems and are looking for an interesting model then go to Randall Beer’s site and have a look at any of his papers.

Assuming you know about CTRNNs, then you know about their activation/transfer functions. This is usually a sigmoidy looking one. In his recent paper ‘CTRNN Parameter Space’, Randy talks about a more general class of activation functions parameterized as:


The class contains several activation functions, including both the one commonly used in the context of CTRNNs (1,0,1) and the hyperbolic tangent function (2,-1,2) (alpha, beta and mu respectively).

Also in that paper he talks about Rnm regions of parameter space. This regions correspond to sets of parameters that when instantiated into the system generate dynamics where n out of the m neurons are active (non-saturated). This allows one to do a very important calculation which is, if you know the range of values from which you are picking (say at random) the parameters for your new dynamical system, then you can calculate the proportion of the volume of that space that will correspond to the case where, for example, all of the nodes are active. Calculating this is not easy, but in that paper he gives us some insightful estimations. For what I want to say here, all that matters is that the more ‘interesting’ (i.e. Rnn) regions of dynamics are rather small (between 0.01% and 3%). – and they can get smaller as the parameter range increases or as the number of nodes increases.

Knowing this, I think, is crucial if you are interested in evolving CTRNNs. One could (and certainly should) perform a number of experiments and tests to see how one can make the parameter range and other things appropriate for interesting behaviours to arise with ease. I think there is a whole PhD’s worth (or even a couple) there. Now, that is not what I will talk about at this point.

Back to the story (and it should become clearer why what I just mentioned is important later on). As soon as we met they started showing me their dynamical systems in action. Some artificially evolved – only slightly – others purely random ones. I was surprised. The behavior was very interesting with external perturbations (e.g. beats, tunes) and without. After seeing three or four of them I asked them to see the code. I was puzzled. It turns out they are using a number of variations on the transfer function – among other things. But that is what we focused on mostly.

All of you that have picked CTRNNs at random know that it is rather common for them to reach their equilibrium not too long (i.e. more or less as long as the largest time-constant) after the rather trivial transients have passed (you will see what I mean by trivial transients bellow). What follows are examples of the activations of 10-node CTRNNs integrated for 100 units of time. with parameters chosen from: weights [-10,10], biases [-10,10], time-constants [e0,e5] using different transfer functions.

First, using the common logistic function:


This is the type of behaviors that are common to see (and I’ll show the first 4 random ones that I obtained):





The system reaches its rather simple equilibrium point not long after initialisation. The transients are short and non-complicated. We could visualize this as a ball rolling down a valley.

Here is the type of behaviors that you get if you change from the logistic to the hyperbolic tangent activation function (which is almost the same as the logistic one but goes from -1 to 1 instead of from 0 to 1):





Perhaps a little bit more interesting? or perhaps just a tiny bit luckier? In any case, it is not a big difference. Particularly with the absurdly small sample that I am showing here. But I reckon it is worh looking further into the parameter space of CTRNNs with hyperbolic tangents – particularly in relation to the already studied parameter space of CTRNNs with the logistic function.

That’s not all however. They went on to show me their sine + tanh transfer function.


I’m not going to go into full detail about this. But there are two flavours here to keep in mind. One is that every node in the system has the same transfer function and the other is that each node has any of the transfer functions morphing between the sinouide and the hyperbolic tangent.

So what are the types of behavior that you get when you chose 10-node CTRNNs with all nodes using the middle gray transfer function – half way between sin and tanh?? Keep in mind that, except for the transfer functio, all of the parameters are chosen from the exact same range as before.


What is this? I’m blown away by it (and if you have played with CTRNNs then you should be too). Is this some really complicated transient towards the equilibrium? Is it a chaotic equilibrium?


Again a crazy transient, but this time it does reach a very traditional looking attractor.



I’m not kidding! this is the first 4 that came at random. You can now begin to guess why I became suspicious of such CTRNN dynamics that I was hearing when they displayed their CTRNN music.

Now, the interesting bit is that I think this changes (quite dramatically) with the mix of sin and tanh. Let’s call this the A parameter. Where F=A*tanh + (1-A)*sin, thus A=1 is the simple tanh, A=0 is the sine wave, A=0.5 is what we just showed. As soon as we get closer to the tanh (A=0.75) the system begins to get more ‘classical’ looking.




This is the type of stuff that really excited me. Some really chaotic behavior which ended up switching after a while into a completely different pattern. Really exciting stuff! and keep in mind that this is merely random.


In any case, as I hinted before – yes, they had one more card hidden – each node in his system could choose its own A. That is, its own mix of transfer function (in fact, this is simply the way they had it – I generated the in-between step where all of the nodes have the same – but mixed – activation function to ease the transition). Here I’ll show you some 4 behaviors where the A for each node is also chosen at random between [0,1].





You get the rough idea. To put it in terms of the Rnn regions of parameter space that I mentioned at the start and the likely (or unlikely) hood of stumbling by chance on some ‘interesting’ dynamics, it seems like having these other activation functions leaves the space pregnant with ‘interestingness’.

Now, I am very much aware that this is the furthest thing away from a serious paremeter space study of the dynamics. In other words: caution to those CTRNN evolvers out there, this is only very very preliminary stuff. Do not attempt this at home unless you are willing to study the parameter space in more detail. Those of you who know me, will now that I am not a fan of adding bits and pieces to CTRNNs  (e.g. multiplication in the weights, hebbian learning, homeostatic adaptation mechanisms). Particularly because people then end up claiming that it is because they have added those extra bits that their system evolved. Or even worst, that it is the extra bits that are ‘causing’ some functionality at the behavioural level. This is nothing like that. From the examples that I have seen, there is a pressing need to study this further. This shall follow. Also using some of those to evolve common minimally cognitive behaviors would be appropriate/interesting.

Footnote: Although I refer to ‘they’, Ollie is the main person behind what I have been talking about. So he wanted me to make clear that he has not been doing any dedicated research into CTRNNs and that he is really not at all familiar with the literature. His PhD research is focused on something different and you can read more about it in his website. He said to me, “I have just been exploring this aspects for my own purposes in the musical domain“. Classical artist-informs-scientist cliché.