activation function comparison & behaviour spectrum

Here is the continuation of the preliminary comparison in evolutionary performance between the three transfer functions in 3 node circuit CTRNNs over 20 evolutionary runs each. The green trajectory is the average over the best fitness for the logistic sigmoid activation function; the red corresponds to the f=tanh(x); and the blue corresponds to the f=(alpha)*tanh(x) + (1-alpha)*sin(x) activation function (where each node has its own evolved alpha parameter). The triangles at the end of the evolutionary runs correspond to the standard deviation of the 20 best fitness, colored respectively.

comparison.png

Going back to analysing further the behaviour of the best 3 node circuit (whose two infinitesimally-close-starting-positions I showed two posts ago).
exampletrajectory.png

If instead of looking at the full detail of the trajectories we concentrate on the broad areas in the 1D space the agent finds itself in over time, then we can have a higher-level view of its behavior: concentrating not in particular trajectories but in the overall performance. Imagine we color code each region like follows:
environment1.png
One peak is blue, the other red, the in-between region is neutral-white and the outer regions are darker blue and red accordingly. We can start the agent in 400 different positions uniformly between the range [-10,10], all with the same internal starting state, in the entirely deterministic simulation (no noise except for the inevitable computer rounding off errors) with a time-step of integration of two orders of magnitude smaller than the smallest time-constant to avoid absurd time-integration errors (timestep=0.01, smallest time-constant parameter=1, the time-step actually used for evolution is 0.1), for the regular 500 units of time. The trajectory of the individual is color-coded according to the previous figure.
spacestatespace318br.png

During the start the agent is not very chaotic, similar environmental situations cause similar behaviors, but as time passes the agent’s trajectory becomes more and more unpredictable.

Although harder to read, we can also see the full spectrum of the trajectory in the next figure, with an even higher resolution of 2000 uniformly distributed starting positions in [-10,10]:
spacestatespace318.png

What does this mean exactly? I’m still figuring this out in the broader picture of what I would like to say about this task and agent. In the more detailed and technical sense, however, I definitely feel that this is proof that the system is sensitive to initial configurations. Also, I think it suggests that the decision of what the system does next (at most? times) depends on its internal state more than its environment. What I would like to find out is how to tell when (or even whether) it is also being influenced by the environment. My intuition is that this is, of course, happening when, for example, the agent has to find the actual ‘hot’ regions. Ideally, I have to figure out how to present a map that can tell me when it is following the environment and when it is following its internal dynamics (but I’ve said many times before already).

Advertisements

3 thoughts on “activation function comparison & behaviour spectrum

  1. A few weeks ago I programmed the wheeled robot cart you built at IU-Bloomington. I swapped the front bump sensor for an ultrasonic sensor. The hope was that it would look more interesting if it didn’t have to run into a wall in order to detect it! At one stage I had the cart alternate which way it turned when the ultrasonic sensor was triggered. I thought this would add a little variety to the behavior patterns. Unfortunately, since the cart turned about 90 degrees it could get itself into corner, then alternately face each of the walls. Over, and over, and over. (Needless to say that was a beta version of the controller.)
    This problem would have been nicely solved if the reaction to detecting the wall were a little more chaotic!

    Bye,
    C.

  2. Great. I’m glad you are getting that robot to do something. In general, it seemed to me like that set was a good platform for trying evolutionary robotics experiments.

    Your point about reacting in a chaotic way reminds me of a point that I have wanted to discuss for some time. I can see at least two very different and interesting uses of chaos in the internal dynamics of an agent: at fast and slow timescales.

    At fast timescales, chaotic dynamics can solve problems like the one you mention. Doing behavior A slightly different every time. That’s pretty good but I think even more interesting could be the case of chaos at very slow dynamics. When some of the internal dynamics of an agent are sufficiently slow in relation to the fast-time behavior generating dynamics, then these act as parameters to the fast ones, changing their phase portraits entirely. Having strange attractors available in the slow timescale dynamics could produce completely novel dynamics for the fast time neurons, thus novel behavior. What one could see is changes in behaviors altogether: sometimes doing A, other times doing B, and if encountered with something entirely different then maybe behavior Y and Z as well.

    Thanks for your comment. All the best.

  3. Yes, it makes sense that chaotic dynamics on a range of time scales would be behaviorially useful. To an observer with not enough time observing the behaviors or not enough memory, the behaviors might appear non-deterministic.

    “Jump to conclusions”: Maybe this is where we get the impression of our own free-will? Our self-inspection abilities aren’t as capacious as our behavioral abilities. When we watch ourselves we feel as though we could have decided to do one of multiple things at any given time, but in actuality we just can’t figure out our own behavioral cause and effects.

    Glad this post is “uncategorized”!

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s