 It's a brilliant, brilliant cat, so I hope you talk about that. I'm going to talk about that, yeah. Thanks Miguel. So when I was looking at the schedule, I was like, yeah, I've got loads to talk about. I want an hour, give me an hour. It's going to be great. We're going to talk about loads of stuff. And then I saw the schedule and all the other talks, these, you know, squeezed in around the place. So I apologize for kind of stealing all the time. I've been, I've been working on this mono in second life for about five years now. So, you know, given that the star date is now 2010, and we've been boldly going for five years, there's quite a lot of stuff in the captain's log these days. So I'm Jim Perberick. I'm going to be talking about second life. And more particularly, I'm going to be talking about scripting in second life. So second life is now six. Second life is a, is a online user generated virtual world. It was launched in 2003. It's now six. It's going to have its seventh birthday this summer. And it's a user generated virtual world. So you have, you have people in the virtual world. They're embodied as avatars. Their avatars are built by themselves or their friends. You can buy and sell clothing and items in second life. So you can go in second life and go right, I want to be an elephant. So you can go and buy an elephant avatar and be an elephant and walk around or you can go and make one. And being a user generated virtual world, this is a user generated birthday cake at the, at the sixth birthday party. So when second life launched in 2003, the world looked like this. So these, there's about 50 regions on this map here. Each of these regions is 256 meters on a side. Each of the regions is being simulated on a core of a server. Originally they were all in San Francisco. We're currently just escaping from San Francisco. We have colos all around the place. But and the reason, one of the reasons that the system scales is that if you have this region here, which is being simulated on a single process, it's only ever talking to the, the regions north, south, east and west of it. So, so you can, as more people turn up to second life and more people want land to build castles or shops or dance clubs or whatever it is that they're building, we buy some more racks. We built them to the floor. We plugged them in. They downloaded the software. They say, which regions do you want me to run? And they start running a region. So unlike most virtual worlds, most virtual worlds like things like World of Warcraft are like theme parks. You basically work out, I've got 200 content creators and they're going to fill the world with stuff for three years. And they're going to make about this much stuff. And that's going to make this much size. And we want to make it so there's only 10 minutes walk between each interesting thing. So this world can fill, you know, you can have maybe 5,000 people in this world at any one time before it feels so full. And the queues in the theme park are so long that it gets boring. So what you do, because this is computers and not a real theme park, is you pour people in until it feels nice and full and there's lots of people to meet. But not so full that the lines are too long. And then you make another copy of the world. So when you play World of Warcraft, if you go and get a user account and play World of Warcraft, you'll choose a server. And that will define the people that you get to meet in your World of Warcraft existence. So if someone else is on a different server, it's just kind of tough. You can't hang out, you can't meet with each other. Whereas in Second Life, what happens is the world constantly gets bigger around the edges. So as more and more people turn up, they want more and more stuff, they want more room to do things, we bolt more servers to the ground and this map gets bigger. And that means that if you create a shop in Second Life and you start selling stuff, that means your marketplace is everybody in Second Life. Everybody can come to your shop. Everyone can come to your nightclub. Your market is everybody in Second Life. It's not a subset. So this is what it looked like in 2003. And, oh, I've got a point at the computer and not the screen. This is what it looks like today. So remember those squares on the first map. Each one of these dots on this map is one of those squares. So we now have 25,000 calls running simulators in Second Life, plus loads of other servers running other stuff. There's 600 square miles of land. So that's about the same size as Hong Kong, and it's about as densely packed as Manhattan, right? There aren't many rolling fields full of wheat because you don't need wheat because you don't eat things. But you do need nightclubs. So you squash loads and loads of nightclubs together. And so you end up with something that's the size of Hong Kong but as densely populated as Manhattan. So there are 150 people, 150 countries represented in Second Life. There's 40 million user hours per month. There are about, you know, nearly 800,000 repeat logins a month. So that's kind of the population. Now, this is interesting. The peak concurrency of Second Life is 87,000. That's the biggest single virtual world that the world has ever seen. But there are 80 million running scripts in Second Life. So a script is something in Second Life which is in an object and makes it interactive, right? So in Second Life, you want to make something, you point at the floor and say make a cube and it pops up. And then you say add a script and now you've got a box with the brain. And so you can poke it and the box can go ow when you poke it. So that's a script in Second Life and they're used for everything from like making it so that you sit nicely on a chair, making it so that you can open a door, making it so you can dance. But there are 80 million scripts running in Second Life. There's roughly a thousand scripts for every person. So you can look at Second Life as this huge simulated environment which humans occasionally visit. But most of the stuff that's going on is scripts. So other numbers, there are 800 gigabytes of new content created a day. There's 390 terabytes of content that we store on our servers and our switches transmit 12 gigabits a second of data. So Second Life's pretty big. You can also make money in Second Life. So last year there were 47 million US dollars transferred between residents every month and 10 million US dollars transferred across the boundary between Second Life and real life every month. So you go to Second Life with 10 dollars, you convert it into Linden dollars and then you can go and buy shoes and hats and cars and whatever it is that you want to buy in Second Life. So that's Second Life. So today I'm going to be talking about the pixie dust, the secret scripted pixie dust that you sprinkle over stuff in Second Life to make it do things. And this is what turns Second Life from a kind of 3D environment with people that can kind of walk around and say hi to each other to an environment where you can go skydiving or skiing or sailing or dancing or whatever it is that you want to do. So as I said before, you use scripts in Second Life to animate your avatar. So you get this kind of like standard duck walk when you log into Second Life. If you want to have a humanoid that animates differently, you're going to use scripts to do that. If you want to make a tiny avatar, this is ingenious. We have this range of mostly the normal Second Life avatars are within a sort of reasonable range. They tend to be big because most people won't be tall when they join. So they tend to be big humanoids. But some people have worked out that if you use special animations, you can kind of make an avatar that kind of crushes his arms into his body and then folds his legs up so he's kind of sitting in a tiny ball and then has his head sticking out the top. And then you can like plug bits on so you can make it have like a mouse face. And then you can have a tiny cute little mouse avatar running around. But the way that works is you have to have an animation. You have to have scripts that control your animations because your avatar is no longer walking like this. Your avatar is kind of stretched up into a ball and kind of flapping his hands and feet around. So you have to use scripts for that. And also if you want to make, if you want to be a dragon or you want to be a non-humanoid, you have to use scripts for animations. Scripts are used for dancing, which is a huge thing in Second Life. They're used for vehicles, seats, games and guns, doors, elevators, teleporters. As I said, non-human avatars. Chickens was a really big thing in Second Life. So I talked to Miguel about the chickens in Second Life. You can go and buy chickens and you can breed them in each other and they have genes. And they kind of pass on traits to each other with generics. With kind of, what's the word? Inheritance. So they inherit genes from each other. And so, collision of namespaces there. And so you can go and buy chickens and you can breed them and you can say, I want to get a chicken that's red or green or purple or has tiger stripes or whatever and you can breed them together. And that was a big thing. But after the chickens were turtles, and now the big thing in Second Life is bunnies, but none of this wildlife would exist in Second Life without scripts. You also use scripts for selling stuff to each other. So you have, instead of being in Second Life to sell stuff, you'll make a vendor like a vending machine, except instead of selling cans of coke, it's selling cars or, you know, teleporting objects or whatever. So the way scripts work in Second Life is, as I said, you put scripts in prims. You make a box, you put a script in it, you link them together to make objects. So cars are like many boxes and cylinders stuck together. You have sensors via events. So you say, I'm going to create an event handler for this thing being touched and then you have effectors via library calls. So if you want to make a box that when you touch it jumps up in the air, you would have an event handler for the touch event and then you'd have a library called so move me up in the air. You can communicate between scripts. You can send instant messages to avatars or other scripts. You can send email to the outside world. You can receive email. You can use XMLRPC to communicate. You can use HTTP to communicate. So you can wire Second Life applications up to the web and people have done cool mashups with that. In terms of the platform, all this is untrusted code. So we're running it on our servers. So we really don't want you to be able to wander around our memory and do stuff and access our file system. Another interesting thing is that there are 2,000 scripts per process. So in order to get 80 million running scripts in Second Life, you have to have many, many scripts in every region. And so every door, every seat, every haircut, every pair of shoes we'll have scripts in and that adds up to many, many scripts. And as Alan was saying, the correct number of threads is one. We use a single thread and we run all these scripts on a single thread, but this is untrusted code and we allow people to have four loops. So how do we do that? We'll talk about that later. And the other thing is you need to be able to migrate these scripts between processes. You need to be able to put your foot down and drive a car across a boundary and all the scripts in your car need to move from one process to another process. So that's kind of some interesting things going on there. So the language we use currently in Second Life is LSL, which is the language I'm wearing at the moment. It's a sea-like language. It has vectors and rotations. So it's kind of a DSL. It has like language features for 3D built into it. So that's quite exciting, kind of. It's statically typed and there are six of them. You can't make any more. There are six static types. You better like them. The only collection is heterogeneous lists. It has states, which is kind of interesting. It's borrowed from various other game scripting engines. It means that you can do stuff like if you click on a door and it opens, it can switch into an I'm open state and you click on door again and it closes. So you can do that kind of thing. So the way LSL is implemented originally on the original VM, it's a textbook 16K bytecode VM. So you have 16 kilobyte block and it'll have a bunch of registers at the top and then it'll have a stack and a heap and they grow up and down and they meet each other in the middle and have a collision and explode and you run out of memory. So it's a textbook implementation, which means it's really easy to migrate. You just take these 16K blocks and you throw them down a network connection and they appear at the other end and they've got their registers and their memory and everything's done. It's kind of relatively easy to schedule them because the way the original virtual machine works is it goes through every opcode and checks the time between each one. So it goes, you know, opcode, get time of day, opcode, get time of day, opcode, get time of day. Oh, I've run out of time. I'm going to stop executing. And it's easy sandboxing because although it has references within itself to variables, basically you just need to check that all the pointers don't go outside that 16K box, which hopefully is okay. We still occasionally have bugs with this virtual machine, so we don't know how safe this is. It was built, we run it, we don't really know how safe it is. It wasn't designed like verification in the CLR, for example, by people who actually thought about it for more than 10 minutes. So it's really, really, really slow. It's interpreted, it's not doing any optimizations and it's very, very broken. So for example, we accidentally got the recursion in the grammar round the wrong way. So if you make an if else statement, if else chain that's too long and you kind of have to because you don't have switch statements, then the parser can't recurs. So it just gets bigger and bigger and bigger and then your compiler crashes. That's kind of one of the ways it's broken. Another interesting way we found when we were implementing this stuff on mono is that returns are optional. So you can have a function which has two control flows, one of them returns a value and the other one doesn't and that's okay. That will compile and run in LSL. So this is, there's some quite scary features in the language. So the thing I've been doing for ages and the reason I'm here and the reason I know you all and the reason I've had lots of fun over the last few years is that we decided it would be a great idea to run LSL and mono, right? Because mono was coming out. It wasn't Java, which no one was a fan of at Lyndon lab and it had a JIT and it was designed by a bunch of people who kind of knew what they were doing more than we did and had more time than we did. So we're like, okay, let's see if we can run LSL and mono. So this slide here is the talk I gave in Madrid and it's available online. I also gave it a Langnet. So if you want to go and see that talk, go and find it, go and find the video. I'm going to quickly run through it because we go to talk about this stuff, but there is another talk that you can go and get more detail on this stuff. So the big problem with running LSL on mono is that, well, so a lot of the things about the environment in Second Life aren't easy to do when you're running a script on mono. For example, it's not easy to take a running program which is in the middle of a for loop and move it from one mono virtual machine to another virtual machine, which is what we need to do. It's also not particularly easy to have 2000 threads running at the same time on mono in a single process. So the big thing that we did is that we rewrite assemblies for concurrency and mobility. So in Miguel's talk earlier on, he said that you've got these two things you can either make a function and then when you look at it, it's just a blob of code or you can make an expression tree and then you can look at the expression tree. And that is totally awesome, but it's not completely true that when you end up with a block of CIL code, that's the end because you can get the block of CIL code and then you can just pull it apart again and you can shove new bits into it and you can make it do things and you can kind of figure out what it's doing. So what we did is we induced the stack state, which is actually the same process that I should have moved all these bullets to the top, sorry about this. So it's the same process you use when you verify an assembly. You start off at this top of the method and you go, there's nothing on the stack and then you get like a push integer, push integer and you go okay between these two op codes there are now two integers on the stack and you can keep doing this through all the control flows through the entire method. And so what that means is once you've done that process, you know all the types that are on the stack at every point in the function and it means that if you decide that you want to pause execution at that point in the function, you know what's on top of the stack, you can make a heap object and you can make a heap object and copy those values from the stack into the heap object and then you can make a chain of these heap objects. So what effectively you're doing is making a chain of stack frames on the heap that contain all the information that was on the stack in the actual machine. So you basically, and in order to make sure that none of these things get caught in a tight loop, you make sure that you do that test at every backwards jump or every function call. So you avoid infinite looping and infinite recursion by going okay there's a backwards jump here. So before I do this backwards jump see if I've run out of time or see if I need to migrate to a different process. If I do then instead of doing the backwards jump do a jump to a post fix which we've injected which goes right call a constructor for a heap object consume all the things off the stack store them in a heap object and then return. And then the calling function will go is there any other any of these heap stack frames on the heap stack and if there are okay I need to stop as well. And so it will return and it will unwind the whole call stack and it will effectively turn the the call stack in mono into a into a chain of heap objects and then you can go okay I've got the script which is a serializable class and I've got these heap objects which are a chain of serializable objects so now my entire state the stack and the heap they're all in serializable object form and then I can shove them down a network connection and move into a different machine or I can schedule another script to run so that's basically what we did that was the big thing and it wasn't you know it wasn't it wasn't actually new it was kind of borrowed so this is the same thing that happens in java gox and breaks on java so several years ago when when mobile agents were a were a big thing people did this kind of stuff to build mobile agent systems that could wander around the internet but we did it in dotnet which no one else had done so so the other thing we did the other two things that were difficult is that we have a server process and when you're you're creating these objects on the ground and you're adding scripts to them and then you're deleting the object well you've created a you've created a an assembly and you've loaded it into mono and then you deleted the box you want to unload it but you can't so what we did there is that we we load two application domains and we keep track of how many of these junk assemblies have been loaded into each and when it hits a level we go okay there's a hundred junk assemblies in this application domain then we will we'll move using the same migration thing we'll move all the running scripts that still exist in that application domain over to the the other one in our process and then we've got an application domain that just contains junk code and then we can delete that application domain create a new one and then start the process again so that's how we do code unloading and that allows us to run simulators indefinitely without them filling up with piles of junk assemblies and the other thing we did is we subverted the profiler to do resource limitations so we want to make sure that you can't just allocate two gigabytes of memory in our in our simulator from a single script we want to ration it between the different scripts and different people that own land in that region so we subverted the profiler and every time we have a heap profiler that isn't actually doing profiling it's actually tracking allocations and then when an allocation reaches a limit we use Miguel's code to walk the objectory figure out how big the object actually is because the allocation size may be way bigger than the the actual resident size and then we'll reset the memory down to the resident size and then it will grow again and we'll measure its size and we'll reset it or grow again we'll measure its size and eventually if we measure its resident size and it's above its limit we'll go sorry you've had too much memory and we we stop the execution so if you want more on that go and see the other talk which is available online so I'm going to talk about a few details which which kind of came out in the sort of last year or so before we ship this which we haven't talked about to the mono community so one of the big problems one of the big differences between the the clr and the java virtual machine is that clr has managed pointers so managed pointers are when you pass an object by value it will copy the object so you'll end up with a whole object a complex object on the stack but then if you want to call a method on that object you'll end up with you'll end up creating a managed pointer which is a pointer on the stack which points to an object on the stack all right so with java you can go through you can go through the whole stack and you can go okay these are objects so when I'm serializing this object to this other process I need to remember to take all these objects as well in mono you can end up with a pointer which points to points to a value type that's on the stack so what do you do about that you can't you can't persist a managed pointer between the function invocation because when you return from the function the stack will will will go go back to where it was and the the pointer will be invalid so you can't do that so what we did here is that we is that we step backwards so we've done this stack we've induced the stack and we've gone okay I don't know in in in okay managed pointer right we want to yield at this point where there's a managed pointer on the stack well that managed pointer must have been generated somewhere in the code so we actually step backwards through the c il until we get to a point where there's no managed pointers in the um in the uh type stack and we go right okay so we actually in order to re-synthesize that managed pointer we need to have these objects and we're going to apply this set of opcodes to these objects in order to generate the managed pointer so instead we're going to copy these objects with this script and we're also going to copy this small slice of c il and then when we get it over to the other side we're going to take these two objects we're going to apply those il opcodes to the objects to re-synthesize our managed pointer on the other side so this is like the big thing that we've added to the whole the whole kind of stack kind of storage and restoring stuff um so yeah i talked a bit about the memory allocation monitoring um that's um that's one of the kind of big things we've we've um been working on recently but i've mostly touched on it so i'm going to keep going because i want to want to talk about new things um so with the scheduling thing this is one of the things we found in Madrid so in Madrid the monosame in Madrid was totally awesome because we turned up and um and uh and we left like three or four days later with our code running 10 times faster and this is why because um as i said before the lsl interpreter would check the time between um each instruction and it turns out it's so slow nobody really cares but uh the um what we were doing with the um the mono stuff is that we were checking the time every um backwards jump or every function call when we were doing this do i need to yield and it turned out that because we were running jitted code this thing was so quick that our performance was very quickly gated by get time of day system calls um and so what we did to um what we did at the mono summit to to fix this after lots of talking about the stuff is that we we allow every script to adapt to have an adaptive timer check skip value so it starts off with zero so your script will start off calling get time of day um after every backwards jump or function call and then if it turns out that we're doing we're nearly always um calling that um function before we've run out of time we'll go okay let's skip the next one so then we'll we'll skip the next check and if we're still missing we'll start skipping more checks and that can go get bigger and bigger and bigger and so this makes our code go much faster we left the mono summit in Madrid with our code going 10 times faster we were very happy um but then when we released this stuff we realized that actually people were figuring out what was going on they reverse engineered our scheduler in this algorithm basically and went right okay that means i can sit in a tight loop building up an enormous timerskip value by basically sitting in a tight loop that does nothing in between it build up a time skip for like 200 timer check skips and then launch into this massive thing where i'm sorting lists and calling these expensive operations and they get to skip over 200 of them before the schedule will check time again and so we're kind of like ah yes very clever we need to figure out what we're going to do about that and so probably what we're what we're going to do about that is we're going to um we're going to make sure we check the time after every expensive operation or every um uh or when we've run out of timer checks so this means that you'll still be able to get the speed up when you're doing um math and calculations where you want to stay in the the jitted code and not call backwards out into into system calls but if you start doing things like sorting lists and so on then we're going to say right we're checking the time every time so that was kind of interesting this was a fun one so um about two weeks before we shipped mono in second life one of our engineers came to us and said huh i just had a crash and when i had a crash the simulator didn't do its emergency sim state save so it turns out that what we were doing before is that when our simulators would crash they'd go ah try not to lose any data so they would they would try to basically spool all of the content of the simulator out to a disk in an emergency sim state file before then crashing properly right now this is arguably not a good thing to do after you've had a seg v um you can kind of get away with it if your scripts are 16k blocks but what was happening is um well so first of all we were like okay what's happened to our signal handler and it turned out that when you embed mono it steals a ton of your signal handlers uh so your you have a seg v and mono catches it and goes yeah this is not a null reference exception so i'll just do nothing um so this was a couple weeks before we wanted to ship so we were like oh shit um we looked at the code and we ended up implementing signal chaining in mono so um so what would happen we we made it so that mono would go when when it when you embedded it it would go are there any signal handlers already installed if there are save them away and then if a signal happens and we determine that it's not a managed it's not a mono exception it's not a null reference exception in the mono runtime it must be you know some exception in this huge part of horrible um unmanaged code over here that's embedding me then it will go right what was the signal handler that it had installed and it would call that one so so we made it we made it do signal chaining so if if you have a null reference exception when you're running um .net code then mono will handle it generated null reference exception if the um if the uh instruction pointer is outside managed code and it just goes okay this is some unmanaged stuff that's going wrong here then it will do signal chaining and it will call the original signal handler um so we did that then we realized that well things are still a bit different because it used to be that all of our scripts were just 16k blocks of memory so when we're doing this emergency save we would go take a 16k block and and save it's disk and this one and this one and this one this one and this one now we have mono scripts running there so we've had a seg v it's gone through mono signal handlers and gone right this is not a managed problem so we're going to call the unmanaged signal handler and the unmanaged signal handler is going right okay I need to save this mono script so I need to go and I need to go and serialize it right so we've had a seg v we've gone through the mono signal handler we've gone in back into the unmanaged signal handler now we need to call back into mono after a seg wür to say all right we need to um we need to serialize this object so we can save it to disk, turns out mono really doesn't like that. I mean we're like totally operating past our mono warranty now. We've avoided our warranty, there's already been a SEGV and we're trying to continue to use mono. So it turns out the the garbage collector really doesn't like running after a SEGV. It's like what the hell are you doing? You're busted, this should be game over, you should already be dead. But we're crashing so we kind of don't care about garbage collection because we're gonna crash in a minute so it turns out that we were able to just turn off the garbage collector, turn off lots of bits of mono and just say please try and do it, just serialize this, just call this serialization function and then we'll die. So that's kind of funny and this is like this is one of the great things about open source code, right? We were able to, two weeks before we were gonna ship, we were able to go, ahhh, mono doesn't do what we wanted, we should have realized this earlier but we didn't but it's just C code and it's open source and so we were able to implement signal chaining. The mono code is really nice, it was a pleasure to work with and then we submit the patch which lived in the mono bugzilla for about 18 months and it got to the point where people were patching our patch in bugzilla because it hadn't been submitted to mono. But now, so it was like we had this little mini fork living in bugzilla for about 18 months of our signal chaining code but I'm pleased to say it's now finally a mono and so so we were able to upgrade mono again because you know we had this fork of mono for 18 months and now now our code's in, we can go and get a new version. So that was kind of fun, this is where we got to so I'm gonna pause for a minute because this song is really awesome. The right hand one is one of the mono's. Mono is much much faster. Hooray! Victory! And also it's so much faster but what used to be a kind of back mode job, which is like a generated factor, can run so fast on mono that it becomes an interactive experience, right? On the left hand side I'm just watching this factor being generated and on the right hand side I'm able to explore it. I'm going to click on it and it can re-drill the fractal so quickly that I can just zoom in on it and so on and so on. We've increased the gamut of things that can be implemented in the second line and you know it runs much faster and it goes happy. And we can make a video with Jonathan Colton singing about Man of Rock, so that can be helpful. So, no this is Jonathan Colton, he's the guy who did Rea Brains. He's like a geek rock guy, it's really funny but yeah. This song's really funny, I recommend you go and listen to it. So yeah and what's actually happening here is I said, you know, everything in Second Life is a box that you put scripts on but each of these pixels is a box in Second Life that's been made really tiny so they've made a kind of pixelated raster screen by just putting boxes together in Second Life and then changing the color of it. So it's completely ridiculous but it makes for a good demo. So yeah, I'll skip forward, go and watch the video, go and listen to the song, it's loads of fun. So that could have been the end right? Victory! We've shipped it, we've got Mono running in Second Life after doing all this crazy stuff for years and years and years. Oh my god thank goodness, we've done it, right? We can go home. Scripts are on faster in Second Life, the scripters are happy, we mostly fix all the bugs, it's mostly a good thing. And when I said there's 80 million scripts running in Second Life earlier, 40 million of them are now running on Mono. So hooray, people like it. So that's good news. So that could have been the end, we could have gone right, okay job done, let's kind of go and do something else. And that's what we did for about 12 to 18 months. There was like Second Life was growing, we had scalability issues, we went and did some web services for a while. It's kind of nice break, you know, from all this kind of fiddling about with virtual machines. But the thing is we've now got Second Life with this crafty scripting language, see like scripting language that's really broken and doesn't have many features. And we've got it running on Mono. So it goes faster on Mono but you know Mono is a lot more than a lot more than a Jir, it's a lot more than an efficient virtual machine. So what we really really wanted to do was to add some of the features that Mono allows to our scripting environment to take take advantage of Mono. So after things with scalability stuff and some web service stuff for a while, we went right okay we're going to fix this, we're going to make our scripting environment better, we're going to exploit Mono, this is going to be great. So the first thing we're tackling is the memory model. So remember I said earlier, other cell scripts are this 16k block of memory and they're 16k and that's the end of it. They're 16k, no, can they be bigger, no, can they, no, can we, no, we can't do anything with other cell. 16k block it will always be a 16k block. And it turns out that's fine when you're writing a script which is like I want to make it so that when people sit on this chair they sit in the right place or you know make sure this door opens and closes it's really great for that kind of stuff. But we've added features like email and HTTP interaction and it turns out 16k is a real pain in the arse when a web service wants to give you a pile of XML. It also turns out that having a heterogeneous list as your only way of storing dynamic amounts of data is also a really big pain if someone gives you a huge pile of XML. So when you're creating scripts that interact with second life, LSL and 16k and DSLs are all right, but when you want to open second life up to the outside world, 16k is a real pain. And it turns out that what happens is lots of people go, lots of people build a script in second life that acts on second life, then they'll build a web server whose sole purpose is to proxy requests from that script, get a huge slab of XML in from the internet, turn it into a comma separated value list so that it can be sent back to the script and kind of squeeze functionality out of this tiny little amount of memory that we have. So it's really not good. So especially for breaking down walls between second life and the web, 16k is a real pain. So we wanted to get away from that. There's no reason that mono scripts need to be 16k, right? They're not a 16k block. They're a graph of object references that graph is not like within reason. So one of the projects we're working on at the moment is the Script Limits project. And what this is doing is it's saying for every square meter of land or for every avatar or for every whatever you want to think about it, we're going to give you x kilobytes of memory. And that's going to be a pool that your script can can draw from. And what that means is that all that means is your script can say I don't want to be 16k, I want to be 128k or I want to be 4k or I want to be want to be four megabytes. And depending on the amount of resources you bought in second life or the amount of resources you have available on your avatar, your script will be able to say I want to be bigger or I want to be smaller. And that's going to be really great. It's going to mean that lots of these proxies are going to be able to go away. You're going to be able to go, I want to reach out to the web, get a huge pile of XML, I don't care that it's a megabyte, I'm going to work my way through it, find the data I need. And I can just then build an application in second life and talk to the web without having to be a web admin, without having to run a server, without having to run all this external stuff, which would be great. So that's what the Script Limits resource pools and big scripts is about. Efficient scripts is kind of just an optimization in a model which was that previously, if you wanted to make an object which would resize itself, you have to put script in every single part of that object. And then you have to send messages between this kind of graph of scripts to say, OK, everyone make yourself 80% smaller and kind of reset yourself. So efficient scripts is just saying, look, we're going to make it possible for a single script in an object to operate on all of the linked objects in that object in second life. So the first thing we're going to do is we're going to sort the memory model out. This is going to be a huge, huge win. One of the problems at the moment is, although every script is limited to 16k, the amount of memory that scripts in as a whole can use in the simulator is unlimited. So what happens is that people go, oh, I've run out of memory in my 16k script, so I'm going to make 400,000 of them. And then you end up with a simulator just kind of bogging down and using, you know, going into virtual memory and swapping, and it's all just totally horrible. So this is going to set a hard limit on the amount of script memory in the entire simulator. But within that, within that block of memory, scripts will be free to say, I want, you know, half a megabyte or a megabyte or whatever it is that they need. So that's the thing we're working on at the moment. The next thing we're going to work on is C-Sharp. Hooray! Finally, a proper, a proper programming language in second life. So what's C-Sharp going to give us that LSL doesn't have? Well, for a start it's going to give us arrays. We haven't even got arrays in LSL. It's going to give us primitive types. So having a byte array when you want to be able to do like cryptography or talk to the web is quite handy. Enumerations, user-defined types, exceptions, all these things that you take for granted that people in LSL don't have and they have to kind of scratch their head and kind of jump through hoops to make work. And also there's a big ecosystem, right? There's a room full of people here at an open source conference talking about C-Sharp. At, you know, Microsoft conferences there'll be tons of people talking about C-Sharp. There isn't an LSL developer room at FOSTEM. There aren't huge amounts of LSL code out on the web. I did the, you've probably seen the kind of programming language popularity thing, which goes, you know, basically do a search for programming web pages and then figure out which is the most popular language. And, you know, C-Sharp is like has 5% of the hits on the web, which is pretty good. It's the same as Python, not as many as Java, not as many as a couple of other languages C and stuff. If you do the same job with LSL, you find that LSL is around about the 100th most popular language in the world. Now, that's pretty good because, you know, second life's been around five or six years, whatever. There's not that many people who program LSL. Having the 100th most popular language in the world is pretty good, but it means that you can't just do the kind of code by Googling that we're all really used to, right? You want to be able to go, I've got this job, I want to do, I'm going to Google for the, all right, there's a block of code, I'm going to base my code on that, or if it's open source, I'm going to use that code. That's great. You can't easily do that with LSL. So the ecosystem is a big thing. We did an Amazon search, there's like 800 books on C-Sharp on Amazon, and there's 28 books on LSL on Amazon, and most of those are kind of general purpose guides to second life. So this is going to be a big deal. The ecosystem is a big deal. So what do we need to do to make C-Sharp work? Well, you'll notice in this part of my talk, I have lots of question marks because I don't completely know, and one of the reasons I'm here is to find out. So if you know, if you can get rid of a question mark for me, then that'll be great. So first of all, we need to complete our U-thread injector. So at the moment, we've implemented this, the tool which takes an assembly and rewrites it to put these yield points in and these post fixes that go and save code. We've implemented enough of that to support LSL. If we're going to support C-Sharp, we pretty much need to support the whole of the CIL instruction set. So we need to complete the U-thread injector. So first of all, are there, you know, is that, how big of a job is that going to be? We're going to hit more managed pointer horrors, you know, are there going to be, it is like opcodes which go and execute delegates, are there going to be a pain to kind of deal with? I don't know, hopefully not. The next big question is, are we going to continue using the abandonware academic framework rail, which we currently build our U-thread injector on, or are we going to use Cecil? Which in the time we've been doing this has come on from kind of, you know, a nice idea to something that's being used for lots of stuff in mono. So are we going to move to Cecil? How big of a deal is that going to be? Miguel, please ask somebody to write some documentation for Cecil because it's, I've done some googling, I couldn't find, you know, I was like, how do we, how do we, how do we, how do I use Cecil? That would be good. I would like to know, I like to know how I use Cecil and how much, you know, how much different it is from rail so we can decide whether we're going to move. And one of the big deals there is, are we going to support generics or not? So rail doesn't, rail will rewrite version 1.net assemblies, but it won't write, rewrite assemblies that have generics. And so if we don't move to Cecil, we're going to be supporting C-shop 1, which means we won't get all the, a lot of the cool, shiny stuff that you, that Miguel talked about earlier. We'll get lots of benefits over LSA, but we won't get, we won't get the whole, the whole shooting match. So the next thing we need to do is we need to build a sandbox. So one of the reasons we're doing this now is that finally Mono has a, a, a bike code verifier and, and a, and a Core CLR security sandbox, right? It's built for moonlight, but, but we're a lot like moonlight in the sense that we are, we're a, we're a, we're embedding Mono in a different process and we are running untrusted code. So we need to do a lot of the same things that moonlight needs to do. So how, how is that done? How do we, how do we hook into it? How do we, we embed Mono at the moment. I have C-shop assemblies that make internal calls. Mono's totally happy with that. I need to be able to flick a switch somehow to say, don't be okay with, with transparent code that makes internal calls. And I expect my simulator to not start up, right? I expected to get to the point where it loads this assembly that has internal calls and to go, bah, this is not, this is not marked as security critical, so I'm going to explode. How do I do that? So is Core CLR security going to be enough? How do I do the assembly markup? Right? How do I, I need to effectively define another profile which is like the moonlight, the silverlight profile which goes, these are the things that you can use. These are the assemblies that it's okay for scripts to access. These are the things it's not okay for scripts to access. I want to be able to take the whole of the .NET framework, put it through a big sausage factory and plop out the other end, this tiny little thing that goes, you're allowed to, you know, you have these four assemblies, these ones are, these ones are security critical, these ones are transparent, whatever. So how do I do that? I need to find out. And some of this I can read docs on, but you know, if you tell me it's easier because just, just tell me he'll be easier. So another thing we need to do is figure out what our C-Shop API is going to look like. So for start we're going to have all of the methods available to scripts that currently are available to LSL scripts. And in fact a lot of this already exists, right? A lot of this API already exists. We compile LSL scripts down to iL, but then we have a bunch of C-Shop API that those scripts call to call back into the simulator, right? So we already have our library of LL foo methods like LL say, LL, you know, set texture or whatever. They already exist. It would be great to give people system.collections, I don't see why not. Hashtables would be great, you know, ArrayList would be great to people who've only had heterogeneous lists for a while. This is going to be awesome. Systems.security.cryptography is a really good one. I'll show you, hopefully, if I have time I'll show you a demo later of using that. It means that you can just go, you know, you've got byte arrays. Systems.security.cryptography, you can go, a few calls. You can then go and talk to, you know, secure websites using OAuth and it's like ten lines of code. So we'd like to have that. It would also be nice to have some XML stuff. So with XML things should be make available to scripts. It'd be nice to make some more web functionality available to scripts. Which ones should we do there? Do we want to, you know, we probably don't want to have remoting but, you know, anything, anything which you can go and talk to the web would be great. So I talked about the dotnet silverlight machinery. How do we, how does mono turn the dotnet framework into the silverlight framework and how do we author the stuff we need to author to make our own our own profile effectively. So types are kind of interesting because, as I said earlier, LSLs are DSLs. So you can do, in LSL, a list is one of the six magical types. There will only be six. So you can go, I want a list L equals foo 42, right? I'll create you a heterogeneous list. It's pretty nice syntax. It's a lot nicer than what we currently have here which is, I want an array list L which is a new array list which is, which is made from a new object array which contains foo in 42. That's pretty horrible. So maybe we want to do some of the kind of funky API, you know, chainable API calls so you can say, okay, I want a new list out of foo in out of 42. This stuff gets interesting when you get to, we have lots of method calls which take heterogeneous lists as parameters which is really horrible. I don't like it but LSL is so limited that sometimes you have to. For example, in LSL, when you want to make an HTTP request, you go LHTP request, you give it a URL, then you give it a list of optional stuff like HTTP method equals post. Well, like this turns, this gets really horrible in the current C sharp version because you, same problem here. You go, okay, I can give it URL but then I need to give it a new array list of a new object array over HTTP method post. So, but we can potentially do better than that by making kind of specialized builders for particular functions, right? So we can make a, we can make an HTTP builder class in C sharp where we'll go, okay, I want a new HTTP request dot URL, here is a URL, that thing can throw an exception if the URL is wrong or malformed, dot method post, this thing can throw an exception if post is not a valid method, dot make the request. So at the moment what happens is you give a, you give a list of objects to a library call and if you get it wrong most of the time it'll just silently fail, whereas if we use builders we can say these are the parameters, these are the types of the parameters, we can strongly type each parameter so that you don't for example pass, you know, this is a string when actually it's a constant and we can make it throw exceptions at the right time. So we kind of, we need to just decide about that kind of stuff. C sharp event handling, this is what Miguel designed in Madrid, which we currently still have and it's working quite nicely. So one of the ways things work in, in Second Life is that the set of events you handle control the UI to the scripted object, right? So if you add a touch handler then if you hover over the object your mouse will turn into a finger because you can touch it. And the way this is handled in LSI is you do these state changes, so you, you can go from a state which doesn't handle an event to a state which does handle an event, it's, but it's a bit, it's a bit icky. So the way we're currently planning on doing this in C sharp is that we'll go right, I've got script, it, it can either override or it can use delegates for event handlers. So I can override the on res event which is called when my object gets resed, which is the term from Tron out of my inventory. So that's just a straight override. But then I can say touch which is a delegate in the, in the parent class. I can say touch is a, is a new script touch target which points to the touch method and then the touch method can just go, hey I'm turning myself off and then it can say set the touch delegate in the parent to null. So then you cannot, you can have like straight override event handlers which is what you expect, plus you can use delegates to do the kind of dynamic event handlers which you need to control the UI. So is that the right model? I'd like to know. Should these, instead of these being delegates, should they be events? Should we be going, you know, plus equals some event handler? What's the overhead of events rather than delegates? This is the kind of thing we're currently thinking about. So C sharp libraries, at the moment in LSO you don't have libraries. So if you want to add some functionality someone's existed to an LSO script you have to copy and paste it into your script and compile it, which is just horrible. It would be much nicer to be able to, to be able to distribute inside Second Life C sharp libraries which I can just import and then just call. But if I'm going to do that, what's the scope? How does the, how do I build the command line in Second Life when I don't have a command line interface? How does naming work? If I create a, if I create a class called foo and then I create a different class called foo or somebody else creates class like called foo, how do we, how do we use namespaces to, to tell a difference? And also how do I version these things and choose which version I'm going to compile against? So lots of questions there, no answers. So that's the stuff we're working on at the moment. We'd like answers to those questions then in the even more glorious future maybe we can get to F sharp Boo and Haskell on the static side. It would be great to get to Python and Iron Ruby, but at the moment we only can inject our Euthred code into, into assemblies that have been generated from static languages. So the languages like Iron Python and Iron Ruby generate code at runtime and we need some, some hook to be able to go okay we're going to rewrite that code as well. IDE integration would be great. So we currently have like a little text box. It would be great to have mono developer or visual studio be able to be able to use these second life scripts and create second life scripts and potentially debug them as well because I'm not sure we're going to get to the point where we can debug scripts live running in second life but if you could create a script in visual studio and have a bunch of mock frameworks that can mock this stuff up then you could kind of you know unit test and debug your script in visual studio and then press a button and have it uploaded to second life which would be great. So how can you help? We're going to publish the C sharp API soon. So I'd love you to have a look at that C sharp API, have C sharp programmers and just go meh that smells a bit that's not good that should be something else. All some of the questions I've asked here have a look at that tell us if we're going to do something wrong. We're hopefully going to have a public beta this summer if we get all these questions answered. If you would like to as an open source developer if you'd like to hack on second life more extensively we have a snow globe open source viewer that's been open source for the last year or so and that is our open source fork of the viewer which people can add features to and we kind of have very quick releases of snow globe and then those features eventually migrate into the mainline viewer but it's kind of a it's a workplace for people to do experimental features in so please play around with download snow globe build it play around with it and hopefully we can talk now or later because I'm running out of time but hopefully we can get some questions so thank you very much I'm Jim Perberick in second life I'm Babbage Linden I blog at lindonlab.com I also blog at home we've got a couple of minutes have we got time for a demo I'm looking for Rubin now yeah well we're running 10 minutes behind the schedule we're running 10 minutes left until 10 past what time is it it's half past yeah so 10 minutes okay so let me do a quick let me do a quick demo show you where we are because it wouldn't be a proper foster and talk without a demo I did try this earlier and it mostly worked so let's give it a go so if you try doing this at Microsoft it doesn't work because they have firewalls but this is kind of free and open source and everyone's got their got their networks hanging out so so I tried it earlier and it did work um so this is the second life viewer I'm going to connect to the region that was last in and hopefully this will work it's looking promising so I was telling Miguel about this group I created yesterday I have a mono equals monkey group in second life which I have above my avatar's head because if you talk to an American there's a disease so I'm doing my bit to try and uh to try and uh tell people what mono actually means so this is second life this is my avatar um so this is a this is a slot machine in second life um I've got a couple of scripts on it let's have a quick look at them so the first one is a twitter oauth client so this is kind of how you do libraries this is LSL this is how you do libraries um in second life currently um and so you basically have to have you define a protocol so you define a bunch of um integers and then you know I've got some helper functions there um which are going to build my message and then so when I when my object gets created or read into the world I call the init function which sets up the um the oauth consumer keys and um and such and then and then when you touch the um object it's going to send a link message which is kind of like an internal message bus within the object so it's going to say tell this primitive the method message 9 9 9 0 3 which I to find at the top and I'm going to give it a string which is the message I want it to send to twitter and I'm going to give it a toucher which is the avatar so let's see if this works if you have twitter available follow s l tester right so so that's all you're doing that I'm going to close this so that's the lsl script and it's sending effectively a message to the other script in the object now the other script in the object is a c sharp script hey so and so this is the nice thing so at the moment we've got no sandboxing this is an internal prototype so I can do things like use whatever I like so I'm using system I'm using system block collections hooray I'm using system security dot cryptography hooray I'm defining my own types for tokens and caches of tokens this is all awesome that you what you just I mean you know if you've ever scripted anything in second life you'll be quite excited at this point so so I've got a load of code now there's one other bit of this code that I want to show you which is as you can see this is our wonderful IDE please replace it with something else so this is the other bit of code that I want to show you so this is just you know this is this is how to do h mac which you need to talk off in dot net you can just go you know I want a system text asking coding I want some bytes from my string using that encoding I want to get a signature bytes which is my message in the same way then I want just an h mac char one object which is defined by the dot net framework and then I want a byte array which is the hash of that message and I want you to compute the hash on those on this message using this key right so this is five lines of code and I borrowed it from the internet doing that in lsl took the whole lsl community about two months so I started trying to build this in lsl at Christmas and I spent three days on it and it turns for whatever reason it turns out you can't use our library functions for char one it there's no byte array objects you have to store things in lists of integers you have to convert between strings and lists of integers and back again and and base 64 strings and so on I spent three days on it and they went right okay this is my Christmas holiday I am I am going to have a have a glass of wine and do something else so then I told the lsl community you know the lsl scripters when I got back to work and said look I've tried to do this this is the code I've got to the whole lsl community has been hacking on it for the last two months and last week they got a version of it that worked right so so that's the difference here okay you get a load more functionality in c-sharp but you can go I want to build another library you can go find the code and you can write five lines of c-sharp instead of having to reimplement char one in a language which is relatively slow and doesn't have byte arrays and then implement hmac on top of char one in a in a language that doesn't have doesn't have byte arrays and doesn't have any way of converting you know between strings and binary values it's absolutely it's just it's unbelievable that people have made the incredible things that they've made in second life with these tools and I'm really glad that we're going to give them c-sharp so anyway that's the code let's show you it working so hopefully this will work so I'm going to talk lsl tester doesn't exist in here huh lsl tester lsl tester yeah all lowercase lsl tester t that is a t sl tester so so I'm going to click on the I'm going to click on the box the lsl script is going to get the touch event it's going to send a message to the c-sharp script uh the c-sharp script is going to go and talk to twitter and it's kind of interesting because this box is acting as a http client because it's talking to twitter web services an http server because it's registering a url that's been called back by twitter and an html browser because it's going to open up a page to say you know is this okay so it's quite a funky script anyway but it's much much easier in c-sharp so I click on the box it says load this web page to authorize twitter access so I open the web page and you know you will know about oauth the important thing here is that I'm talking talking to twitter.com I'm not talking to my lsl script so I'm avoiding the password anti-pattern I'm not giving my twitter username and password to anybody apart from twitter right so that's the whole point people have built stuff that talks to twitter from second life before but um they've used the password anti-pattern which is not a good idea so there's my username there's my password so I can choose to deny or allow this thing so I'm going to allow it then twitter is going to call back my script redirecting me back to the application this lovely web page ta-da was produced by my lsl script and then it's asking me do you want to show the status update so if I say go to page barrage linden gave a demo at foster I'm in second life here hooray we've talked to twitter so then so the other nice thing about this is if I click on this link then it's going to go to slurl.com slurl.com is a service we provide that gives you teleport locations into second life on the web so if I click on this link hopefully so anyway well that doesn't seem to be working so if you go to twitter if you click on that link then it will give you a map of second life pointing back to the location I'm in second life which means you can come and join me right so twitter is real time so I can be doing some cool stuff in second life the thing in second life can go hey your friend did something cool in second life and tell your friends on twitter then you can click on the link and you can come back in second life so this is quite fun anyway but it's also a good demo of yc sharp in second life is a good thing right now so hooray the demo worked any questions we have three minutes I think monosecurity naval course okay please send me an email hooray that's one question mark done any more question marks we can get rid of does this api look reasonable or does it look terrible is there anything that's as a c-sharp developer is there anything that makes you go ah it's horrible please fix it it was designed by Miguel so it may be terrible the audience in shunt 3 it automatically initializes any objects that can be commenced innumerable and open as an example so you can just say you are in this parenthesis and you have been embracing your data and it will just add to this so that won't be as obvious so if we can support c-sharp 3 if we can use sessil well I'd like to but we need to use sessil so then we need to and I'll make sure that we document sessil yeah that would be cool somebody's gonna be very happy next week but or you can use that to be a reflection yeah this is mad enough already yeah we'll document monosets we'll come up with that yeah that would be cool because I mean sessil looks totally awesome and all the conversations I have with jb in really good so I mean well the excuse of you know the excuse of developers get usually is well the api is still changing it will be documented so why is not everything running on mono yet? like you mentioned that most of the stuff is now running on the mono vm but some stuff still use your old legacy interpretive vm yeah I mean so the way we implemented it is we didn't just to be conservative we didn't automatically convert old scripts over to the new vm okay so if a script is running on the old vm and you wanted it to run faster or you were doing a new version you could open the source code and you could say I want to compile this to mono but we didn't automatically convert things running in second life already so there's lots of there are lots of scripts in second life which are running perfectly fine on the old vm and you know the people who are using them don't want to you know maybe can't edit them and so you know those those will continue running on that so we specifically made the choice that only people who could open look at the source code and we're developing the script could convert things to mono because otherwise you'd like you know you especially when we when we still have bugs in the mono implementation you might have a car in second life it works fine you bought it for like you know five hundred million dollars you say all right convert it to mono I've heard that's a good idea you convert it to mono and it explodes and then your car's broken and then you go back to the person who sold it and they say why the hell did you convert it to mono and so so we encourage people to create new versions of their objects running on mono and sell the new versions because it's not efficient but you know that hasn't happened everywhere and even where it has happened there are people who are happily using their other self-scripts from from from before anyway so those are all like three year old scripts well there are some people who are using still using some some other self-scripts so if you because of the memory model there are some things where if you're using a heterogeneous list it uses up less memory if you do if you run it on alice up then if you run it on mono because mono's using 32 bit pointers and it's in 64 bit pointers whereas the internal pointers are 16 bit there are some there are pooling cases where LSE does more efficient but generally mono is more efficient and and that's generally because that each LSE script is this 16k block so even if it's using 2k of memory it's allocating 16k of memory from the similarity and it's just up to speed 4 from 0 in the middle whereas if you run a mono script that just uses 2k of memory it will just allocate 2k of memory are you that is you're I mean trust me you don't see so are you going to allow those bits that are uh are you using your place within the script I would like to allow people to use as much as possible obviously sure and a lot of what and dotnet and a lot of what we're doing now is going to be defining what can we possibly have to use in the in the environment for instance now I'm going to take the way that you're doing that look pretty well now and how you're moving one thread so there will be there may be some things that we can't allow because the way we rewrite segments yes so yeah you're probably not going to be able to go you know have this to the work of all because instantly you've just created a load of work somewhere that we can work track it so yeah I think some things have got it right and sorry can I have the stuff you have no thank you for coming thanks for having me thank you