Evolving Artificial Neural Network

[rebirth3X]

Sorry, but what's it learning, what's going on here?

[vladimirzakharov]

I like the upper one at 5:10 (yep, I watched that long)

[yellowvsgreen]

this is beautiful!

[silentrageze]

Is it possible for the AI to just re-draw the xwing at the target as soon as the run starts, or is there a constraint on how far it's allowed to re-draw?

[Etaanaru]

@silentrageze Haha! That would be the best solution wouldn't it? It's limited to vehicular movement, it can rotate and move forward (and at one point they could move backwards).

[mdowse20]

@Etaanaru - if you don't mind me asking, what inputs and outputs did you give and get from your neural network?

[megothier]

4:18 keeps pausing here and I can't get past it. Otherwise I think this is very interesting. I myself work with Artificial Neural Networks, mainly MLMVN and am currently working on coding a MLMVN for pattern recognition.

good work mate

[Photon98]

Hey, please tell me What Programming language you are using and also how you did it, thanks.

[Etaanaru]

@Photon98 C# with the XNA framework. I think I've explained just about all I can remember about my project in my other replies to other comments, check them out and thanks!

[Photon98]

GOOD, Although it took me to see entire video, i can say that these things (WHATEVER are they) really simulate an Idiot HOUSE FLIES!!

[spiderx]

Looks like they just hug the wall since that's the easiest way to the target. Might be interesting to add friction to the walls, stop the simulation after a certain amount reach the target, or a drop dead time limit.

[michaeljolley337]

looks more like bird-of-prey ships than x-wings

[SwiftAdz]

Cool video, but how is it ur evolving the neural networks? i hav a similar program but i cant seem to perfect the way they evolve. are you crossbreeding the networks and mutating them etc?

[Skyferia]

I wouldn't call it boring, but what's the purpose of the video and how does it work and why such?

[TheDamped]

Love the disclaimer haha

[Oshyoga]

Each unit has it own "history" isolated or you put some kind of "exchange of experience" ?

[Etaanaru]

@Oshyoga There isn't an explicitly stored history, those who do well survive to the next generation, as well as randomly sharing bits of it's neural network with the rest of the agents in hopes to make them better as well.

[drite94]

well i mean how did u do that? i want to do something similar

[drite94]

what did you use to actually program the animations?

[Etaanaru]

@drite94 This doesn't really have animations. The neural networks updated each agent's position and orientation, and then on the next frame it's drawn to screen. The X-wing models are static.

[axeld93]

needs some techno.

[ZequeZ1]

Great! I like it =DI liked it, it's pretty amazing ^^

[cominup85]

i'm interested in neural networks from a statistical point of view. i enjoyed the video.

[PacSgt]

I've always wondered, how do I make it act upon special situations?

Always have I thought of it as a static sequence, but that won't work when if it is to appear in a different situation but the same concept, let's say always turn when wall is X units away with a static sequence that just doesn't work.

I'm a retard, so feel free to not answer my question if it's too retarded :V

[Etaanaru]

@PacSgt By not using an artificial neural network :) That's the thing with them, you can't guarantee any sort of behavior. You could try detecting desired behavior and rewarding them by giving them higher probability of survival, but it's a really tough thing to balance. I'd go for a different type of AI for special situations, check out state machines, hierarchical state machines, and force based systems like BOIDs.

[ecreif]

@Etaanaru i stick with hierarchical neuronal networks that are able to reuse simpler neuronal networks deeper in the tree..

[WSebastiaanZ]

@Etaanaru by not trying to get it to learn your solution. Our own natural neural networks are pretty capable of learning to deal with the unexpectable. Maybe the power of neural networks to be able to react and learn in special situations, more than the old algorithm AI approach in which the program has a fixed number of preprogrammed tools to manage with situations. In ANN-AI you can get it to "invent" a new tool... (and sometimes we never discover how the network works! --> see eg. NETTalk)

[f14maverick100]

@PacSgt You could also have that behavior hooked up to an output on the Neural Network. Every time that output neuron fires, it could trigger your specially programmed action. That's how most Neural Net-based game AIs these days work.

[TheRealFallenDemon]

What kind of penalties did you use to reinforce the behavior? Was it just a simple "if you don't get to the goal, -1000 points" or something?

[Etaanaru]

@TheRealFallenDemon Performance was measured as the inverse of the distance from the target point, performance = (1 / distance) where higher performance is better.

[digitlecat]

it looks like one will learn the rout and the others will fallow

[virgildog1950]

Hi, I was wondering if you can answer some of my questions:

1. How hard is it to program this?

2. What software/Programming Language are you using?

3. Have you ever tried making the X-wings have to try to survive? Like, there are spikes in strategic points on the map, and also having plants that only grow if there are no X-Wings nearby, that the X-Wing's can fly into to restore their energy, and the X-Wings die if they either fly into a spike or run out of energy?

I really like your video!

[Etaanaru]

@virgildog1950 It's not too hard, look up Artificial Neural Networks and Evolutionary Algorithms. I used C# in XNA. I never tried anything with this project beyond what you see here (other than making it into a game where you had to avoid them), but those would all be interesting experiments.

[VCat2006]

It goes well with Virgill - Artcity.

[BobbynEdward]

All life on Earth collaborated to form higher orders of intelligence; an unbroken line to our primordial beginnings. "Life cannot be contained, it must burst forth". Consciousness is the same. It cannot be contained (even within a box of human understanding). We look at the wondrous plants and animals of this planet, and know them as myriad expressions of an elemental happening. Maybe consciousness is also a singular event; one that human being, "enhanced", planetary, and cosmic are all joined.

[Etaanaru]

@BobbynEdward how appropriate :) I am currently researching swarm intelligence, the very essence being that information gained by an individual is shared and enhances the entire swarm. We all have to stand on each others' shoulders. Thanks for commenting.

[FlashonVideos]

Only 3 hidden layer nodes? What are the four inputs?

[Etaanaru]

@FlashonVideos Yes. The four inputs are the distance of the three "sensor" vectors (one straight out, one 45 degrees left, one 45 degrees right) which would collide with walls (so the numbers would get smaller as they approached a wall) and a vector to the position of the waypoint.

[Sez0144]

What determined the firing of the input nodes? I'm assuming they were connected to some sort of virtual sensors, but what exactly did each of the 4 sensors detect (what decided the activation value of each of the input nodes?). Also - the output nodes would correspond to the firing of steering thrusters - so that would be left and right (as you said it can be narrowed to 2), what were the remaining 2 responsible for? Thanks for reading. I'm a Cognitive Science student really interested in NNs.

[Etaanaru]

@Sez0144 If I remember correctly, the input nodes always "fired" their value into the next layer of nodes, or the threshold was part of the randomness in their genetics. I described the inputs somewhere in the comments...they are 3 distance vectors (straight ahead, left  and right 45 degrees) to detect collisions with walls, and a relative angle to the waypoint. The outputs were left, right, forward, backward.

[Sez0144]

@Etaanaru Thanks. Ok, I found that comment. I didn't see any of them going backwards, but the forward speed does vary - which I didn't notice at first. So, you're not using a standard back-propagation approach for NN training. More like a genetic algorithms approach for NN, with the "gene"-coded traits being the connection weights (?). How did you handle the "breeding". Was it a random mix of weights form the pool of the "fitest" individuals?

[Etaanaru]

@Sez0144 Unfortunately this was about two years ago, and I lost the source code, I may have forgotten specifics. I'm pretty sure I used back-propagation, but I may be confused with just learning about it or using it another time. I also detail the breeding in another comment :D but yes, a mixing of the weights. The very best one continues to the next generation, as well as "breeding" with all/some of the best to create offspring, as well as one or two new random individuals.

[Sez0144]

@Etaanaru Thanks for the info. Now that I think about it it might have been difficult to use back-propagation in this case since it requires that the output for each input be checked and then the weights readjusted compared to a desired output. So in a real-time system it means doing every frame and knowing what's desirable in each frame. (I guess it is used mainly when NNs are used for classification tasks - simple 1 wave of input data 1 output situations.) One more question: Java? C++? C#?

[Etaanaru]

@Sez0144 Yeah, that makes sense. I think I wrote stuff for back-propogation and didn't use it. This was in C# using XNA.

[Yumminess0]

What were the inputs?

[Etaanaru]

@Yumminess0 Check the other comments.

[Yumminess0]

@Etaanaru "and a vector to the position of the waypoint."

I've never understood vectors. Could you clear it up for me? I know it's a set of numbers, but whenever I ask what the values are they just say they point to something. Are the two values angle and distance? And, if so, is the angle relative to the current angle of the ship? Thanks.

[Etaanaru]

@Yumminess0 Think of a vector as an arrow. It represents a direction and a magnitude (length). From this, we can find the angle and distance to something. So the agents would get data about which direction the waypoint is. I may have unitized the vector (make it a length of one) which would make the vector simply a direction. Check out "Euclidean vector" on Wikipedia.

[Aurelovanovalopta]

(continued)

And also - how do you proceed to the next generation? In my case, I run 50 subjects and select 10 or 5 closest to the target. Then I take their ANNs, make small random changes and put them into the next generation...

Any help? (cause it is not working very well...)

[Etaanaru]

I think I said this in another comment, but I take the agent that did the best, then breed (mix weights) it with either all or a top percentage of all of the agents, and then add in a couple completely random new agents. There is definitely a sweet spot to hit, simplify your input/output layers if you can, and just experiment. I was amazed when this finally worked.

[Aurelovanovalopta]

Hm...

I am interested in ANNs for a while now... Can I ask you one thing? How exactly your ANN works? I mean, I've programmed an ANN class, with layers, neurons (with dendrites having a value from 0 to 1 as a weight) - and the value of a neuron is computed by adding together values of previous neurons times their dendrite weights with the current neuron... (I've also tried to select only the one with highest weight, but no change)

Do you have it programmed like that? (cont.)

[Etaanaru]

If I remember correctly, I had it so that the thing changing among neurons were their activation thresholds. And if it met the threshold it would "fire" and affect the next layer of nodes. How much they affected each neuron in the next layer was also a variable among agents.

[Aurelovanovalopta]

Um... How were those encouraged to get to the "parent" one?

I mean - how did they know they *should* get to that one?

[Etaanaru]

I placed a waypoint with a key, I actually controlled that X-Wing. I know it's confusing but I was short on assets. As far as how they "know," they didn't. I just made it so that the ones that made it closest "lived" and the ones that were far away "died" -- i.e. did not contribute to the next generation as much. Survival of the fittest.

[AbsurdistLover]

On the difficult run it seems most of them resorted to the left-side rule (i.e. following the wall to their left).

[DKM101]

what info is contained in the four inputs? And four outputs? All i see is turn left and turn right :S

Still, really cool stuff :D

[Etaanaru]

If I remember correctly...the four inputs are the distance of the three "sensor" vectors (one straight out, one 45 degrees left, one 45 degrees right) which would collide with walls (so the numbers would get smaller as they approached a wall) and a vector to the position of the waypoint.

The four outputs I'm pretty sure were forward/backward, left/right. That's why I said I could simplify it down into two outputs probably since they directly oppose each other.

[pudumula]

hi there

i am an undergraduate student

i would like to know how the above simulation was done

[Etaanaru]

I encourage you to read the video description, and all the comments. After that, formulate a specific question and I'll try to answer it (if I can remember).

[shneebleu]

Can each member "communicate" and help each other find the way or are they all individual?

[Etaanaru]

Individual, dog eat dog simulation.

[shneebleu]

Awesome. Well I suppose if they could communicate then they would all do exactly the same thing. What language is this written in, and what graphics library did you use?

[Etaanaru]

I used XNA so it was C# and whatever their graphical library is considered :p

[shneebleu]

Ah right I see :). So obviously the main focus was the Neural Network rather than display etc. I might try something like this in C++ with SDL if I have the time :).

[ludsoe]

whats wrong with the ones who crashed into the wall thats just sad

[Etaanaru]

Not everyone is a winner. :D

[SlykeThePhoxenix]

Wow, that's amazing. At first I thought "Some of them are retards", but then I realised that they needed to spin fast =).

[samisupidupi]

nice video! how do you do the selection and weight adaptation? crossover and mutation of the weights?

[Etaanaru]

I don't remember exactly! Hmm...I knew as far as selection, I would choose the best (the one that got closest) and keep him in the next generation, as well as breed him with at least the top half (or maybe all of them) and maybe randomly generate a few completely random new ones.

As far as what my "breed" function did...don't exactly remember. I THINK I randomly picked weights from "father" and "mother" so that each one contributed equally.

[samisupidupi]

thanks for the answer. I finished my own little ANN project now, see the video answer - although it doesn't look as impressive as your video...

[3zander]

This thing is fantastic! I tried to get a neural network like this working and got nowhere. The agents just spun around on the spot. :)

Also this video isn't boring, it's awesome watching the little things slowly become better at reaching the target.

[Etaanaru]

Thanks! I had that problem too, and you'll see a few of them still do it. It used to be worse though and I think I fixed that by reducing the complexity of the outputs.

Previously I had one output node for left and one output node for right, but I merged that into one node that could be positive or negative.

It can be very difficult to pick out the right number of nodes and hidden layers, start small and work your way up if you haven't tried! Good luck and thanks again.

[asterwilliger]

Cool video! Not boring. I watched the whole thing. I wanted more.

ANN's are amazing.

[dubdubhate]

So those little moving things have a mind of they're own and they can think and solve problems within time. Does ANN exactly mean that?

[Etaanaru]

Kinda...but not exactly. Artificial Neural Network just means that it has nodes/neurons connected to other nodes/neurons that fire to control behavior. In this program I built a fitness algorithm saying "If you get to the waypoint, good, if you're far away, bad" and it prunes the population to encourage the good behavior (basically selective "breeding" and "natural selection")

[macca1088]

So... What is the goal of the AI? To navigate and reach the uncontrolled X-wing?

[Etaanaru]

Basically. I actually control that one X-Wing, and what I do is place a waypoint there, and stay there so it's easy to remember where it is.

[Kintetsugakusha]

JohnnyK: "People say 'arn't you playing God by using technology to make people live longer?' The obvious answer to that is, if thats the case, so was inventing the wheel. We are people who have the ability to improve our world if there's anything we don't like about it, God made us that way." - Aubrey de Grey

That in mind, he gave us the ability to think, reason, and create. Creating intelligence is merely another thing he gave us the ability to do.

N3CR1S: We are no where near making a brain.

[N3CR1S]

I know we are far from making a brain. i never said that we should try. i was however saying that we are close to understand how the brain actually goes about solving problems and how it is functioning. How the nerves lay i layers, each layer sending signals to the one above it. a nerve has several conections from the layer below, thus being able to find "patterns" in the incomming signals. this being done in large scale, the brain can predict events, and understand patterns.

[N3CR1S]

watch Geoffrey Hinton and Jeff hawkins here onyoutube for starters :)

[johnnyk427]

This project is an abomination of nature. Only God can make something think.

[Etaanaru]

You're an abomination of nature. If God can only make something think, he must have forgot to do it when he made you.

[johnnyk427]

Does this 'thing' have a soul? No, clearly not. Man should learn from the bible, god flooded the world for a reason do you want that to happen again?? Better stop what youre doing then!

[N3CR1S]

@johnnyk427

Are you serious? if we are supposed to learn from the bible, we would all be uneducated and we would rape and stone people and still have slaves!!! i for one dont want to love in a society like that! go read "the brick bible" on the internet, it highlights all the parts that you so called "christians" want to hide about your horrid book.

continue your work etaanaru, it amazes me that we are getting close to how the brain actualy solves poblems :D

[N3CR1S]

meant "live" not "love".... the "o" and the "i" are so close together :P

[feodor24]

well, listen, i believe in God too but I don't think that it's all about soul. He created alot of amazing things in this world and I'm pretty sure that it's ok to learn them. One day science will defeat cancer, AIDS and so on. And it's ok coz Jesus himself was a person who healed alot of people during his life.

[Rybot9000]

You don't have a soul either, at least... not one that you have presented for observation. If it is clear from the absence of observing an X-Wing soul that it must not have a soul, then we should conclude none of us have souls either. It might be useful to define what a soul is. God may have flooded the world if by flood we mean melting of ice sheets following a change in global climate after the last ice age. In which areas of land flooded all over the world and drastically changed the land.

[Rybot9000]

And if by God we mean the entirety of reality which performs in accordance with it's own organizational structure. That which determines the fate of the quark and the jaguar. Such an entity may or may not be conscious but as with humans that fact would be irrelevant to its determination to do what it does. Neither such an entity nor us could act contrary to how we act and thus could not be justified nor blameworthy for its wrath. Saying it flooded the earth for a reason assumes something....

[Rybot9000]

caused the behaviour of the entity to occur. Something which antecedes in space-time the decision of the entity. This is contradictory with the entities presumed omniscience. Nothing could preceed or cause a decision of an entity which knows everything past and future. The decision is destined to occur. In addition any attempt at explaining otherwise is futile. To say you understand God is to say you know his reasons which is to say God is determined which means God is not God.

[colonel720]

Hey, cool! Is this a backprop network or an unsupervised algorithm?

Well actually, do you have a paper on this project?

[Etaanaru]

Hm, I had a paper, I don't think I still have it.

It's been a while, so I forgot the exact implementation. I'm pretty sure it was unsupervised. Random neural networks were generated, and ones that did well "bred" with the others, but also a few new random networks were always in the mix.

To determine how well they did, it was a measurement of how close they got to the waypoint at the END of the 5 or 10 second generation. Closer was better of course.

[N3CR1S]

to me, that sounds more like genetic algorithms then actual neural nets.... i mean, nural nets LEARN whilst genetic algorithms ADAPT. im pretty new to all this but atleast thats how i see it soo far. have just finished watching the Jeff hawkins videos here on youtube and im heading over to his page to download his version of the nural nets :D :D . if you havent seen his presentation then go and do it. im sure you will be intruiged :)

[Etaanaru]

Well it's both. The neural net is just a bunch of nodes with weights and activation thresholds, it can't really learn on it's own. The genetic algorithm is just making sure the best ones survive and doesn't really care how the agents "think."

I'm no professional on AI either yet, I've only taken the one class this project was for and am in process of taking another more general AI class.

[Rybot9000]

It is my understanding that limited supply of neural growth factor selects synapses and cell depolarization influences the process. That could be the "genetic algorithm" eerily similar to natural selection. Synaptogenesis lays the groundwork, the algorithm defines the structure and processes like LTP/STP maintain the plasticity of the brain.

[Monstrobishi]

This video isnt that boring, good job, I hope you passed your project thing.

[geodesicks]

what was being fed into the input neurons?

and what was being controlled by the output neurons?

biological brains evolve over generational timescales and over lifetime scales. so there's a little more to it.

[Etaanaru]

Input neurons I believe were the angle to the waypoint and 3 distance sensors which were straight ahead and 45 degrees to the right and left.

Output neurons were forward, backward, left, and right, though I know that I could simplify that to two outputs of amount foward/backward and amount left/right.

Biological? That's why it says "artificial" in the title ;p This is generations and lifetimes, just that lifetimes are about 5 seconds and every 5 seconds is a new generation.

[geodesicks]

pretty cool! can you have each critter try to "eat" other critters? that way, i bet, you could have some dim form of intelligence arise in a few generations.

[RFS671]

very nice job. its actually quite an interesting video. im doing a final project for my freshmen cogsci class involving computation and the mind. And I was looking up stuff about ANN and this came up. Was definitely interesting to see how the X-Wings learned. Good job. hoped your TA liked it.

[Etaanaru]

Thanks! Glad you found it interesting. Sometimes I wonder if maybe our own brains could be represented this way, we just don't know how many nodes and the weights ;)

[lobban2]

Probably many more nodes than you would like to make. :)

Our world is a little more complicated than the X-wings 2D world.

I would love to play a game, fps or other, where all the entities in the world used neural networks to learn how to survive. Maybe they would be get so fit in that world you couldn't compete against it. (i.e. it's so evolved to survive in it's own world it would be the supreme being in it's world)

[Etaanaru]

You should definitely check out NERO (Google "NERO game" so that you don't get the disc burning application)

It's a real time strategy game in which you can train units with a user-defined fitness algorithm.