Autonomous spider learns to walk forward by reinforcement learning

[killer2611]

so beautiful watching it take it's first baby steps *sniff* i think i'm gonna cry :')

[StormWolf01]

Computer science is beautiful.

[intellisaforiginal]

OMG! awesome! ufff! love to see the last part when it perfectly learns to walk!

[Gigadrax]

LOL I was thinking, this is kind of less efficient than Animal walking because we're born with some instincts to walk and then i realized that this ROBOT leaned to walk with no prior knowledge in just 4 minutes.

[Aoitetsugakusha]

Is this robot programmed with the included software, or is it done in another language like C?

I've been thinking about getting the bioloid starter kit to experiment a little on my own time with neural network algorithms, but the major drawback for bioloids is that they don't seem as well supported by standard programming languages, where as others like NXT and VEX can sync with RobotC out of the box.

[micheltokic]

Dear Aoitetsugakusha,

the robot is programmed in C on the little PC-Board on top of the robot on which we run Linux. We use a very small (USB) serial-to-bioloid board for sending positioning commands to the bioloid actuators. So you see we are very flexible in the choice of our programming language :-)

Cheers Michel

[amightyo]

Really cool. I hope I would be able to implement this too for Robocup :-) Thanks for sharing and the link too :-)

[2bornot2b1984]

Does the robot have instructions to try to move forward?

[micheltokic]

The instruction of "what should the robot learn" is encoded as the reward signal. In this case the signal indicates how much the robot moved forward or backward.

[Ryuuken24]

How can I get this code in C, I wanna use it with the mindstorm NXT, thanks ! Cool robot !

[micheltokic]

We used the Dyna-Q algorithm as proposed by: Sutton, R. (1990). Integrated Architectures for Learning, Planning, and Reacting Based on Approximating Dynamic Programming. In Proceedings of the Seventh International Conference on Machine Learning, 216-224.

[Ryuuken24]

Thanks for the reply ! So I guess what you're trying to say is that, its a hell of a extended code, were C isn't as good as C++, I get it. Its still fun how the robot figures it out and remembers each sequence. I wonder if you made it possible for a biped robot, maybe climb high objects, or sensing edges. Anyway, thanks !

[sosata866]

what's the reward/punishment dude?

[ponocrates2]

Very impressive learning performance. I wonder how it works.

I d like to do it at home.

[MrBrander]

Aww... Looks so hopeless in the beginning. :) Like a little baby is trying to learn how to walk.

[csotanyag75]

Nice. I'm actually surprised that the learning was so fast. Usually Q learning requires a huge amount of trials.

[micheltokic]

The speed is due to a very simplified state space with only three states for each dimension (a bioloid actuator).