 Hello everyone, and welcome to the 1230 breakout session of the Open Simulator Community Conference 2013. As a reminder to our in-world and web audiences, you can view the full conference schedule on our website, conference.opensimulator.org, and you can post your questions in local chat, on your stream chat, or tweet your comments using the hashtag OSCC13. This hour, we are happy to introduce you Tony Alotalo, who will be presenting RealExtend and Open Simulator Development. Is there a connection? Tony Alotalo is a RealExtend developer and game programmer at PlaySign. He has been involved with RealExtend since almost the beginning, from late 2007, and was responsible first for the scripting API. In the next generation, Nali Tundra effort, and later at one point for the whole Open Source project. He now acts as a development coordinator within the RealExtend Association. Welcome, Tony Alotalo. Thank you, and thank you everyone for coming, and it's been really a pleasure to get to experience all this. I haven't had much time to be in world recently, not nearly as much as I hoped for, and even here I'm using just the default avatar that just somehow happened to come to this presence. I had three children and running a small company, and we are just finishing our game, and then there's all these RealExtend things going on as well, so living a bit busy life, but it has been really nice to be here now and focus on this. It was also really quite inspiring keynote I found by a creative push, but I figure here we sort of come down to more mundane, mundane matters of Open Source project and software development, but okay, so this talk is about RealExtend and Open Simulator, and basically the question is whether this talk is on topic at all, that does this have anything to do with Open Simulator? But to get a little bit of history, old news for I know many of them, but perhaps an interesting recap anyway. So what is RealExtend? There's actually quite often misunderstandings about it. Sometimes people think it's a particular piece of technology. Sometimes people think that it's a company, but actually it's an organization, an Open Source organization, and a really loose one. It's basically like a mailing list, so similar to Open Simulator and the idea of the organization is simply to do Open Source Virtual Worlds platform development and to allow anyone to set up their own virtual worlds and develop whatever applications on top of that. So in that sense, it's basically the same that the Overture Foundation and the OpenSim communities is doing. And that's of course also why it's really interesting to be here and seeing this in action. So this history, we have a lot of history with Open Simulator. The first RealExtend prototype was published in 2007 and it was a modified version of both the Linden Labs View and the Open Simulator server. And I wasn't much involved in the first phase, but the other guys hacked in quickly, a lot of stuff in a few months. Notably Mesh support in 2007 and the whole ogre graphics system to the Linden View that gave us, we could put our shaders there or whatever. And then there was also this sort of global avatar system which was not the same as HyperCrit, but similar idea that you can have your single identity and use that to visit any world. And there was quite a bit of enthusiasm then in the OpenSim community that there was all this development. We'll talk more later about what happened then, but sort of fast forward to today. We still exist. The nowadays is this RealExtend Foundation and we have the RealExtend Association. The divide there is that the foundation is sort of a secure thing that exists to protect the name and sort of protect the brand, but the foundation doesn't really do anything. Juhar Hulko, whose avatar is shown in the picture and who is the originator of the whole thing, is the chairman of the foundation and then the city of Oulu here in Northern Finland is also supporting the foundation a little bit. And then there's the association which is open for anybody to join in and which is in practice is largely the mailing list discussions and IRC talks. And there we have most actively the few like four or five companies that have been developing RealExtend during the past years and some other people and companies. What else? And the basic idea in this is that we are quite business oriented still, I would say. This Juhar Hulko founder is a businessman who started a company in the 80s, Electrobit, and they do sort of embedded software and electronics and he became financially independent through the 90s as the company succeeded. So in 2005 also this business person was just having free time and looking around that what's interesting and if encountered second life and really like the idea of virtual worlds, he saw so many applications exactly this kind of conferences and meetings to cut down travel needs but then also simulations and planning and everything and actually the motto for RealExtend by this person is to save the planet. He really wishes that that we can use virtual worlds to save the humanity. So we also have a connection to the keynote here. But anyhow, the idea is that the whole the basic platform is open source and free to use. Anybody can set up their own server like with Apache for websites or open sim for this kind of words. But the reason is that there would be business in the applications that companies can host and host and develop for other companies and so forth. And I actually don't have like an awful lot of material but I'm hoping for discussion here. So please just intervene at any point if something is unclear or if you want to ask anything. But continue from here anyway. So okay about the specific history that we have with open sim hopefully we can learn something from there. So we had this initial release with the prototype and and there was many open sim developers were developed enthusiastic and there was quite a lot of open sim developers at that time and many companies just kind of the boom and there was a plan to to merge for example the mesh support to the core and so forth. Make long story short that didn't really go that way. I actually don't know even all the reasons but eventually we got a quite nice solution that the real extent extensions were implemented in this Modrex an open sim module which you could install and then you would be able to use the enhanced view to get all the custom scripting stuff and the and the machines at the time and so forth. So at this point it became so that the when the companies here were trying to do business with this incarnation of real extent the problem point really became the second life viewer base. As it was discussed here in the in the viewer panel some hours ago it's quite spaghetti code it's difficult to to modify it's it's not modular and when and when some company wants to make a service or an application for some company they really have to to be able to make it the way the customer wants and needs it they it really needs to have exactly the kind of user interface and the user experience that is needed and and it's not a good excuse there to tell that we can't really change that it's it's hard coded in the in the platform. So after some studies and and I think quite a lot of discussion and so forth we eventually decided that okay. We need to write a new viewer application from scratch and and that work began began in the beginning of 2009 and I was quite quite much involved then and and the new code base there's two sort of key things one is that it's modular that it has this plugin plugin architecture for like physics and and rendering and and most of the things are implemented in plugins some of them you can actually enable and disable and and some not really but but for example scripting is a plugin and so forth. And the other thing is that that for the scene data so what in in second life and opensim you would call prim parameters or something like that that what what what kind of objects you can have in the scene and what what parameters to those objects have. We made that using a popular is entity component idea which has took that from the game engine literature where during the past five years then also it had become popular as a way to do sort of nicely modular and extensible environments. The idea is that instead of subclassing you you have these scene entities which are not typed so anything in the scene is just an entity and then you have these components which implement some functionality for the and for the entity so for example any entity that has a position has a placeable component where you have the 3d position. So that makes it simple in the way that that for example if you have like in second life you have avatars and primes those are two different kinds of objects and and if you then for example make a game just often quite deep type hierarchy that you have physical objects and non physical objects and and you get this tree of classes and subclasses with the entity components it does not work like that for example for normal objects and avatars they both have the placeable component exactly the same way so so when you for example want to move an object it doesn't matter whether it's an avatar or or some other object because the placeable component is the same for all objects so it's in this way it simplifies programming and the other idea is that that you can add your own components so whatever you come up with you you can implement it in a component and then just start adding those components to your to your objects and this is the same design that for example unity uses so quite many unity 3d plug-in some modules they they are these components that you can use okay so so we were working for on that for the view side and but then there was the question of the server and we wanted to continue using an open sim but the open sim scene was not made like like that it wasn't really extensible the prince prince have the certain set of parameters and that's that and so so there was this we called for people and we had some money for this work so we called for proposals from open sim developers to refactor the the open sim core scene to be extensible and eventually Adam frisbee's proposal got selected and he started working on it and the and the core developers agreed all that this is a good idea but unfortunately it happened during that year that that Adam drifted to other things I think some commercial MMOs and so forth so he wasn't involved with open sim anymore and and actually didn't finish this work he was really sorry and and didn't want the money of course but but it was also bad for us because we didn't get the the kind of server that we would have wanted so sort of the last step here is that that sort of as an emergency measure because we wanted also really not capable of doing this open sim refactoring ourselves and and this funding was ending by the end of 2010 as a sort of emergency backup plan we started using the new view code base as the server as well because there we already had the the extensible scene structure and and the modular architecture and everything so so so that's what we have nowadays this we call it tundra the this kind of SDK which is used both as the as the server and the client this is from Adam's proposal for for the for the open sim scene which he posted to the to the open sim developers mailing list in December 2009 and I was googling for this and actually also found it that it was in in just in predictions for open sim for yet 2010 that that this would happen so if you if you can see in the slide here the left top left part here is the what I believe is still the current open sim skin scene structure that we have seen object groups and those consist of scene object parts basically the prims I think and those have the fixed set of parameters and that's that the idea was to first make this hierarchical so that there wouldn't be the separation of groups and parts but it would be like a normal hierarchy where all the nodes in the hierarchy are the same and then in the in the last step which is at the bottom it would have been refactored to this to this new structure where add ons can add sort of new well let's say components to the to the scene objects so for example machines could have been added as an add-on and anyone anybody could add anything anything they want to the scene data from an add-on and the thing is that that then and like we have in Tundra now is that when you implement this own components everything works for those automatically so you just define the data that you want to have in your application and it all gets synchronized over the network to all the participants automatically you don't have to implement your own network messages or anything like that and also when you save and load the scenes they also your data gets saved so that's sort of the idea that to make it easy to make applications and I was just correct there's nowadays this the there is an extensibility mechanism to to open seem now but the idea here would have been to sort of refactor the core so that also the core features would have been sort of add-ons but okay that was that was that part of the story so now we kind of come come to where we are now so looking from our perspective from the real extent micro universe it seems that OpenSim and Second Life have developed nicely actually there's also these meshes here nowadays materials we have had wonderful stability for example in this conference it's mature in the sense that people know these tools very well and and the tooling with all the exports and imports nowadays hours and years nice the second life renderer is actually quite amazing does a good job rendering user generated content which is not easy at all and it's cool still looking from our point of view it's still the same and and the reasons why we had to stop using it are still there LLUDP protocol is the same as it was and it in effect kind of hardcodes this world model and and the scene is also kind of hardcoded than what what do you have this was also discussed in the in the viewer panel the client side scripting and custom user interfaces and controls are not really there yet except for this was it octopus from one of them your projects sounded interesting so on the real extent side our our little tundra SDK is stable it's not alpha or beta it's it's used in in business and there we have all the it's easy to do custom applications you can you can do any kind of user interface you want and and when when a user comes to your server using the client they then download the user interface assets and and and you can have most of the applications are developing in JavaScript so it so this client side JavaScript is then executed in the client and you can do whatever custom functionality you want for example, our task are not a part of the core but it's actually a script that you can put on your server if you want to have our task on your server and so forth so so it's kind of good we also have worked at least on all kinds of little experiments with web GL and web sockets client through the years and now it is sort of maturing there's an alpha client that you can you can test on the on the meshmon hosting service and the whole meshmon thing is quite key part of the real-extend economy so meshmon is by one of the companies Adminotech who decided to to focus on hosting and and they have this it's similar to quietly I think in the sense that it's dynamic so currently they have about 850 scenes or worlds in their service and it's free to create a test account there if you want to test creating your own space and and it works so that when somebody looks in it's then starts up a cloud instance and so yeah that's why it can be quite cheap so so this is how I see the the open sim community and the real-extended sort of much smaller but still surviving community and and my question here is basically whether whether whether we are relevant to each other I'm certain that it's a good idea to at least for me to come here to see all the exciting applications that people are doing and to sort of learn that what sort of usages for virtual worlds are interesting and what works and what is perhaps not so interesting but I'm also curious whether whether there would be actually something that that we could do together whether it would make sense to to perhaps share some technologies or whether yeah whether the real-extended technologies could be somehow relevant to the to the open sim community we did make them because we wanted to address basically this customization problem that that we we encountered with open sim or did we just then diverge into sort of separate parts and and and there really is no much much of a connection anymore so so that's that's why I came here yeah to answer Robert's question the it's been it's in a bit of a progress thing the there is a there is a completely open source web tundra implementation but that's using a really naive simple like just an uncompressed chase on messaging because there's also one other implementation by by admin or take which is an optimized binary protocol but that one is not open source yet they are in the progress of of open sourcing it but but actually the protocol and the message is used over web sockets is the same that we use in the native tundra which is open source and actually now it's also even documented just this week and the protocol is really small because we actually all we do in the network protocol is to synchronize this entity component model so so the messages are basically create entity remove entity like add component remove component and change attribute and something else it's a really small small protocol but I can give the pointer to the to the protocol documentation after the talk for example yeah about the we are about and about the JavaScript that we use in the native in the native client one clarification that that doesn't have anything to do with the web let's see what do I have as a next slide did my extra slide come here now sorry to go back yeah try to add one slide but I think I failed so the JavaScript that we have in the native client is QT script so so for example for the UI you have the QT API which I think is the same as the linden view uses I don't know I don't think they use QT for the for the for the witches but they do use QT something in in this client as well but anyhow that's so so the native tundra JavaScript is not a web browser JavaScript it's just it's just a JavaScript API to the to the tundra API where you can command the ogre renderer and the bullet physics and and and then QT for for the UI stuff yeah okay so some mark mark says comment on the this kind of leads me to the next next slide where I put down some some ideas and not necessarily good ones but anyhow some ideas about how we could try try doing something together again so if you if you find this our client interesting it would be again possible to to use it to connect to OpenSim if you if you prefer to to continue using OpenSim as the server so there's basically two options what we did originally in the gnarly view which was the name we used when it was a client only and then when we all when we added the server component we sort of renamed tundra so so gnarly had a module that implemented LLUDP so we could connect to even second life actually and and to unmodified OpenSim servers so but I think might be more interesting to to do the reverse this time to to actually implement a module tundra to to OpenSim because it's a really simple protocol and and then it gives this extensibility and LLUDP protocol is a bit mad there's a lot of messages and I think most of those have to do with selling parcels and that sort of stuff and and I think it's a nice idea to have this low level simple wire protocol which doesn't define the the application level stuff so so yeah that's sort of to comment to Markus about about this client and and I think it is a possibility for for the OpenSim community to to use this now mature code base to to innovate new platform features okay I read more of the questions yeah yeah justin makes a great point about unity and OpenSim and real extent and and it's true that it's can be confusing innovate tundra SDK is really similar to unity we have the same entity component model and and in the same way we want to want it to be like a generic engine on which you can do anything and but there's two differences one is that tundra is open source and the and the other is that it's inherently inherently networked so it's similar to OpenSimulator in that sense that it sort of works on the assumption that it's a virtual world that that anything you have is something also that that the others see and you don't have to do anything extra to have it a multi-user setup so but of course unity has upsides as well they have a great renderer and it runs fine on iOS and Android and so forth but but the but the open source part and the and the inherent inherently networked part are sort of the upsides of tundra yeah so idea three that's something that I I also think is sort of most interesting because the there also the the OpenSim community would get really something sort of yeah perhaps really useful and it's also something where where we are going to work on anyway we have now this quite substantial funding for working on the web client I'll talk about that more in just a second but when I'm going now back to Justin's comment about but his first point well one upside for real extent is that that I think it's no secret that that that we don't have a like much of a community there's we have a nice developers community but there's no there's nothing like the OpenSim users community we do survive because the companies do business but but we also I don't want to forget the sort of original dream of the of the 3d internet and and the and these and the and virtual worlds and and I for example wish that I would I could use these environments more for work for these kind of meetings and everything in the future and and the hypercrit for example is really really exciting so so one really motivation for us is of course that that if if for example some OpenSim people would sometimes use use the real extent client to for something yeah it would be just really really cool and interesting and and I'm I'm also think that really beneficial that we would get more feedback from from enthusiastic virtual worlds people and and and real extent would sort of serve its purpose that it allow people to to sort of do what they want with with their worlds and we can return to that later but but yeah I can talk a little bit about that EU project it's quite new yeah I think the final contracts are signed now like last week or something like that it's an ongoing ongoing EU project that I didn't even know about earlier you can see even in the bottom left corner there's this future internet PPP and and this specific project is freeware like if I were and that's the future internet software platform so so what's the deal here is that I didn't even know that this whole future internet thing existed earlier but it's been going on for I don't know two years or something like that and and the background is that the European Commission and some big European IT companies sort of got together and and they saw that Europe is losing and lagging behind in on the internet basically and I think it's obviously true that that Facebook and Google and Amazon and Twitter and all those are from from the States so so the EU big folks wanted to have some initiative that that would sort of yeah help European companies catch up and also make innovative nice internet services and so forth I have to say that I'm usually quite skeptical about this kind of like huge top-down development efforts like in the States there was the internet to but I don't know how much came out of the internet to project but I guess something did come out but anyhow there's this huge project it's 40 million dollars or 40 million euros the ongoing thing that started in a year or two ago and ends in next April and and it was really it was already an ongoing project and they had this open call last fall where they were asking for somebody to implement a browser-based virtual worlds system basically and and it was easy sort of for us to to respond to that call because we kind of had it already so this in the end then selected us to do it and now we are working on it so yeah so our part is this 2.7 million which is like a huge sum but there's so many other things this like augmented reality thing and display as service and GIS and point of interest things and all the other stuff that we don't have that much for the for the actual sort of the basic 3d platform development but we have some 300,000 also anyways for the for the sort of the bottom part bottom part here and so the plan in this project is to keep using the tundra server and the entity component model yeah and this is all open source the whole this whole effort is is targeted for open source development and actually we can use we will use some of this money to sort of open source some things that proprietary right now that have been developing some some real extent projects but yeah this is open source so we are kind of happy that now that the big EU guys they like the entity component model they they have adopted it already to some of their own server codes and so we will probably keep using that protocol in the networking and on the server and we are going to use proper web sockets we've been also experimenting with web RTC which I think is interesting because it's UDP and but then what happens on the on the client side is now sort of up for debate because it's a requirement from this sort of from the customers side that we need to implement this and declare declarative 3D stuff if you happen to know about it so we'll see how how exactly we will have the the scene in the in the client's memory but in any case the idea is to integrate to the DOM to the browser document object model so for example when you look into a server with the browser client you can use the browser debugger to see sort of the primitive as I guess you would say and you can even then use the for example the Chrome debugger to modify something in the scene and it reacts so it's going to have this kind of a tight browser integration and and like a high-level declarative way to to define scenes so that's sort of the new new stuff yeah but terrible this comment that's that's an interesting point we're actually doing a performance tests right now and the and sort of the other guys who are participating is from the DFKI and they have this library called xml3d.js there they already have this declarative 3d stuff and we actually now testing like which performs better so Robert knows xml3d already yeah so we're working together with those guys now oh and I think they actually work with you at Intel too because they were going to implement some distributed scene graph stuff to the server too so yeah that's sort of the big thing here now that we from this remote northern Finland we are sort of now getting to the talk with this sort of big EU guys and and and they liked real extent enough so that it's sort of now being adopted in this in this project there's about five minutes left also so I'll just show some pretty pictures and you can ask more questions there's some scenes that have been made there's a nice circus with three minigames by Ludocraft and an airplane flying game and and and this is a really nice church the one on the left from the 1800s that's really nicely modeled and all these are sort of available in the mesh 1 hosting system so you can if you install the client you can visit this and this is the one with the old fire truck is from the Berlin sort of fire museum and that has been actually what has been using the what has been driving the web client development that it was the first scene that was made accessible with the web client and works well looks the same basically with the with three it's a small simple scene this is something where I made the particles so I like this this is connected to a real-world private power plant which is used to produce electricity and heat for the buildings for this is actually Juha's house the guy who started real extent has his own power plant as well and he's using real extent to sort of monitor and control the the energy flows in his home and most use here in always and also elsewhere in Finland is actually in this sort of city planning and we are making a quite huge detail see model from the city of Oulu but but then then also this bottom left thing is from the Helsinki the Aldo University they are doing this sort of area scanning based model from whole Finland so you're gonna have some scalability issues and this shows something of the customizability so for example this application is for teaching how to operate this kind of agriculture device I don't know what it is called in English but you use it to collect the harvester weed so there's a little game where you have to learn how to maintain and the device so it's one example of a custom user interface yeah some kind of harvester this is a bit old from the school of the future project where we also made it sort of custom UIs and functionality that you could carry these objects around and you could easily just track and drop images and so forth all made with scripting tools and this is sort of and this is my last slide also sort of my favorite point point that for me the definition of virtual worlds is that they can be anything that that yeah it's often useful to have avatars and sometimes nice to have terrain and sometimes nice to have gravity but I really think that for me virtual world means really that that anything is possible so so I think it's nice to think of virtual worlds that we can define how gravity works or whether there is gravity and and so this is one example of a different kind of world world where the whole world is just bomb so this is basically like a game and if I if I want to place make a place for bomb doesn't necessarily need need avatars for example and but regarding avatars as a last point I would also like to stress that that the fact that we don't have anything about avatars in the core doesn't mean that we wouldn't think that avatars are important I think it's quite the opposite that when the whole when the whole for example avatar system is a script and and you are able to do any kind of features there without touching the wire protocol I think that is really can be really powerful that because anybody can then enhance the avatar system and come up with new additions and new sort of innovate innovate with the concept of the avatar and so forth so I think this is this is sort of the interesting interesting point about the extensibility and modularity but that was my talk and I think I used to time as well I don't know where what would be the best place to continue discussions I think for for now we can talk here I'm fine also with using the the open simulator mailing lists for example for discussions if you don't consider that's off topic you're also always welcome to the to post to the real extend mailing lists we have these Google groups for users and developers mailing at least a real extend mailing lists don't have much traffic so so at least there we there we can talk but but yeah I'm fine with using the open sim lists as well yeah and I see I'm on I'm on the open sim dev and we also have real extend dev so yeah and many of you are hanging out there still which is really nice yeah thanks again for for having me and and for the whole event has been really really great thank you Tony a lot hello for terrific presentation as a reminder to our audience you can see what's coming up on the conference schedule at conference that open simulator org this room the next session will be open simulator configuration file world tour with Justin Clark Casey thank you again to our speaker and the audience will be back shortly with the next session thanks