 Hello, everyone, and welcome to the 10 a.m. session in the developers and open source track as a reminder to our audience you can view the full conference schedule at conference dot open simulator dot org and tweet your questions or comments to at open sim CC the hashtag OSCC14 This hour we are happy to introduce a terrific session called virtual worlds on a web browser Our speaker today is Tony a la tallow Tony is a real extend developer and a game programmer at play sign He has been involved with real extend since 2007 and now acts as a development coordinator within the real extend association He is responsible for technical aspects of the University of Olu's Olu 3d infra project, which is building an extensive detailed model and services of the city of Olu Welcome all welcome Tony and let's begin the session Okay, so hello everyone and Great to be here again I have to say I really enjoyed last year's experience and I have been Telling about it to everybody that that this way of organizing conferences actually works A couple of words My slides are quite technical full of text Only the last one has screenshots So if you are more interested in in testing the technology in in action and so forth I'll just type the Address in the in the local chat That's one way How you can test this technology in in this mesh moona hosting service And the web web client there is called a web rocket and that's the That's the real extend based on technology that I'm talking about So My my topic to Today's is totally to focus on these capabilities of web browsers and and how virtual worlds Run on top of them and this is something we have been actually working on for for many years I could say a few words about real extend as a background Last year my my talk here was only about that about the relationship of Real extend and an open simulator, so I'm I'm not going to repeat all that here now But in the keynotes It was really and also the previous talk here By vitally it has been really interesting to hear this I'd interest in In customizable Viewers Because that's exactly what we did starting 2008 or so We first made this Nali viewer from the scratch as an open sim client with the idea that the That the user interface and the whole user experience is totally Defined by the application or the or the scene And that's what we have continued and and that's how we are doing business actually So it is it has been really really critical for us to be able to Sort of exactly to define the user experience in the applications for the customers often really simple Applications and sometimes with avatar and sometimes without Now we have some kind of math based environments kind of like a two and a half D experience and and so forth so So we are kind of happy happy there and happy with the technology that we have with real extender We can deliver to customers But the only thing that kind of Makes me sad is that we haven't been really able to serve the the open sim community Basically because we are not compatible with the With the existing content in in these worlds. We don't have primes and also We don't have the same kind of creation tools, but But I think that's something that I think we can I'll be thinking during the conference and we will discuss With the with the other developers here that perhaps we can find ways to collaborate in the future but here the focus is just like generic web browser stuff and and not about Real extender that was last year So here we go What virtual worlds mean to me is that this is really like a collection of all technologies that to create for example like this Experience that we have here at the conference. We need Inactive 3d graphics We also want 3d special audio, but But we also want real-time networking with with UDP or TCP sockets and and we also want speech and So that we can talk So it's really like a collection of basic kind in a way like all technologies that that computers can Can handle So So when second life started, I checked that it was kind of in in 2001 and And then when when we begin begun the the real extent work based on that in 2007 It was kind of simple in the sense that there was no choice Web browsers couldn't do any of that so So it was obvious that that you had to have a client a native application That people have to install like a game or so and and that's it There was no big dilemma because that was the only only choice But Since then since then since 2009 nine or so Mostly Google, but also other browser developers have been really pushing the the browser stack We'll see that details about that later But Okay, first this basic question that that why why would somebody be interested in the web And Yeah Because well, we already have this native clients like the what we are using here for the conference So why do we care about the web? so it really For us is simply this This basic point that easy access for the masses because We tried to do normal business like kind of like any web shop. So It's really like a no-brainer that when we talk with our customers that Who perhaps want some 3d world that that okay? It has to be easy for anybody to to access and and So there's no way that that they would use anything that requires installing customs of Preferably not even even plugins like unity, but But they really just want something that works works easily for the masses and on the desktop and and with laptops That basically means websites. So so this really is about Providing something for the masses So if you have a case where you're dealing with a limited group of people professionals for doing Some work or or your friends or whatever if you're dealing with people who can install Special virtual worlds client then you don't necessarily need the web for for anything But also I want to point out that will be sort of the last point in my talk that That is still not the whole story because even for expert users and the web technology is interesting because it's really flexible and as in Very freely programmable. So so I think that That's an interesting aspect of it for for everybody So So but the actual Technologies, I hope this slide is visible enough. It's a little bit smaller text So in the first Earlier I mentioned that okay, we need all this technology. We need graphics. We need audio We need real-time networking and we want to have voice chat so what has happened in the in the recent years is that the Well basically Google and Apple and and Mozilla have been Developing all these new new technologies Under this this HTML5 umbrella So so we basically basically have all this now for graphics GL which means That you can use OpenGL ES from From JavaScript And and the status of that that is pretty good It has been there for long in the in desktop chrome and firefox Safari also supports it but it doesn't work as well yet and and Microsoft I see perhaps no Microsoft was opposing the whole idea early on but But since then they have jumped aboard and actually During last summer like in June July and and I guess all the time now they have been improving the WebGL support in in Internet Explorer 11 especially and It's actually So for audio I Actually don't know that to well myself yet It's actually not to do list to to add 3d audio to to our browser client But my understanding is that with the web audio API it basically works. Okay Then for network synchronization We have had web sockets for Couple of years now as kind of the frozen standard and I think it's Well supported in in all process so So we are kind of fine there and Then then for voice and actually also for video and chat This is the newest Technology now this web RTC for real-time communications, which is actually a UDP based Which is really what you want for for something like an audio stream or video stream So that's like unreliable networking where you don't get extra latency Unlike with TCP so that's more recent so for long web RTC was Was only in in Chrome but But nowadays it is in in Firefox as well, but I haven't actually even tested so So I don't know what the what the status of web RTC in Firefox is but But in Chrome it has been working great for us Actually in that much more hosting service for for real extent worlds You can have voice and video chat as well in the in the browser client And and again if anybody has questions or anything just shoot We can discuss it at any point Mm-hmm Okay, so always when we are talking about WebGL and 3d graphics and especially open some worlds That's the question of performance and Yeah, I would say that it's not it's not too great Although it's very impressive I mean if you read for example 3js the rendering code There is still quite a lot that the That the JavaScript code has to do for every frame Like hundreds of lines of code That the browser runs in JavaScript to push the new data to the To the graphics card every frame so So yeah, simple scenes on my two-year-old laptop with with just this Intel graphics card They do reach 60 frames per second Which is to me quite impressive with considering the amount of JavaScript that is executed for every frame and That's possibly possible of course Because of the just-in-time compiling that cheats GITs that they have in the in the process But typically when we have a more complex scene At least my laptop goes down to 20 or 30 frames per second but but that's still like okay, it's still usable and And it's actually similar to what I get with usually with this with the second life year as well But but the content is usually simpler so so how does this work out? If thinking of open-scene content and just naively putting Huge open-scene scenes there. I think it would just die but the way we are able to do business Company and with our partners is that we simply make light enough scenes So So of course there's ways to optimize but also in many applications You don't need millions of millions of polygons, but but you can just make the designs so that so that they are not heavy Because we don't really have an option The web browsers are the only way that we can we can use in our business so so we kind of add up to that and and then Create the content accordingly Okay, but a note about complex scenes Like if we were to render for example something like this conference venue or or more complex Open-scene scenes I think even if you would I don't know like convert all that to Colorado and just push it to the To 3js for example, it would go down to I don't know one or two frames per second. I guess But that that does not mean That it would be totally impossible to to render complex scenes, but you would have to to make a nice system for that For example, I Read this one article some years ago How the how the IBM folks made a unity-based open-scene viewer And they were using this very clever idea that you had kind of layers that for the nearby objects they were kind of like one layer or one scene and and that was updating with good frames per second and then Then there was like three or four different zones that the more far away objects and especially small objects Were rendered with less frames per second, so so this this way you could have a like really a complex scene Still it would be like running fine when you control it And this kind of stuff might be actually quite simple to do. I'm actually tempted to to try it because with with a 3js library that we use it's actually really simple to have many scenes and and and it's just JavaScript to to control the rendering so you can actually Render multiple scenes to to compose the final image So So we haven't done anything with this yet, but we might actually try it's Might work And then for for complex scenes and especially for like Poor devices like mobile phones and tablets But also in general for sort of easy access for people to just see a scene It's also possible to switch to to server side rendering Which means that that you're actually Running the 3d on the server side On some beefy computer with with graphics hardware, and then you just send either video stream or Even just still images to to the to the client. Oh, yeah, and that's just the notes in the In the in the chat also second life has been doing this Yeah, and as Odin said Odin notes Dealing with LODs like level of detail for the meshes and sort of perhaps a parts of the scene and so forth Can also work so So there are ways to deal with this complexity, but At least we have not done such such work yet Okay, so Not about the mobile devices so So in a way webgl already works on mobiles Android Chrome Got it enabled by default. I think in April and now with the iOS 8 update that came like a month ago Also iPhones and iPads Run webgl by default But it's it's kind of slow so You can go through the examples and Well, some of most of them are like few frames per second and and only very simple ones run Like 50 frames per second and so so So that means that you can still do simple things and Again dealing with this Level of details and so forth I'm sure that it's possible to do nice Mobile applications also for the With webgl, but actually our conclusion about this was also that That within real extent we still also continue work with with native rendering In C++ so our current this sort of mature Client tundra, which is actually the same code that we use as the server as well That uses the ogre 3d engine, which we have been using since the beginning basically But actually now this mobile client We are going with a new 3d engine called our old ho 3d Which is actually Developed by by Lasse Ernie. Who is the guy who who has done the ogre work for for us Ogre is kind of old and slow So all the boys like a modern modern modern faster render So that's how How the guys are now implementing also this kind of really simple light-vegged small install client targeting Android tablets basically first part but So that's that's going on right now And it's quite interesting for us because the old tundra client is kind of big Well, also like the second left here with QT and so forth But with this we are gonna get a small native client more similar to the to the unity player, but But open source so so my Point here is two-fold that okay, we can already use webgl on mobiles to do simple stuff but but JavaScript is kind of slow there so so we are Gonna use the simpler C++ client on mobiles as well Okay Yeah, I see the There's comments about M script in the in the chat. So this is my my slide about that as well so So my view is that both a JavaScript code and native code have their uses so JavaScript is a good way to deploy to the masses on desktop PCs and laptops but but native is you easily need that with mobile devices and so now we have So If you do that, then then you end up having a like we currently have two Different code bases for the different clients. So so you get this kind of like duplicate work So there is this one one Possibility to to solve that problem and to only have one unified code base and that's Using C++ to develop the code and then use the M script and compiler To to compile that to JavaScript to run in the browsers and we actually made a plan With that idea already and in 2012 when M script and was kind of Started to seem a proven thing but it would have been quite a lot of work for us to to create a New client from scratch Because the old tundra with Qt and ogre that wouldn't really work You cannot compile Qt with M script and then expect it to to work in a browser And and also because we got funding for the web client From EU and that was specifically to to focus on the web stuff So so it was simpler for us to use this existing 3js WebGL renderer and then just at the networking and and the other stuff We need on top in this web tundra project but But in the meantime The unreal people at epic and that the unity folks They have actually followed this plan so so the webgl exports that you can do from from unity are Compiled with M script and and actually now that we are making this kind of small minimal client Using the old hope 3d engine our plan is also to try Compiling that with M script and to to get the same Client code to the to the web as well, but I think we will get to that in I Don't know in two or three months when when the code for that client is Is sort of when it has to enough of the basics so that it makes sense So So that's interesting, but but I'm also a little bit cautious of this of the M script and wait because how how M script and works is that okay you develop your application in C++ and or you use Unity as you do normally and and then okay you compile and you get this kind of big blob that you can then Just put to the web So one one problem there I think is that that the That compared to to us to a small minimal handwritten 3js Javascript application the downloads for this kind of big compiled engines can be larger and and then a Bigger point for me is that that okay if you just make a game like you usually make with unity or unreal You are like in a box and and you don't care and it's fine. You just want your game to work work on the web, but for me actually Because we We don't only work with games, but we we work with education and and And communications and all kinds of these kind of information systems It's really nice with this web tundra 3js way that That we can use all the other web stuff as well So we can use all the existing web UI libraries and and if somebody is using like a content management system We can just call those functions as well So so you can do something like like if you click an object in the In in world you can actually have that call some function Which is actually provided by some other Javascript library to like to publish a blog post over there And I'm afraid that there's no way to do that if you're developing in unity You cannot call other web libraries from your unity C sharp code. So So that's kind of a down side as far as I know of this M-script and C++ compiling way. So But perhaps there is some nice solution for that, but at least in the future, but I just don't want us to throw the baby with the bath water With this otherwise nice idea of having a single code base Because well, I like many web things. So Okay Well, I could comment also Alan Is asking on the chat whether we have had a chance to look at the virtual world framework Yes, actually, I was really interested when I when I noticed that I think that's the one from the Like connected to the US military Perhaps the same people who were here last year talking about the Moses project and and In this VWF They they have also written a Browser based and client from from scratch and seems like that with nice ideas I don't know it too well, but but I have had a look a couple of times and we should check Check those Yeah, I could comment on the other questions now as well. Thanks Galen. So This is different. So, okay, so there's a question that why is this different from web meetings and Google Hangout stuff today for the audio It's actually not different. So so web RTC Which was this technology for for video and audio chat has actually been pushed a lot by by Google Because They want to because I think currently still or at least a while ago Google Hangout Requires a plug-in the Google talk plug-in So and and Google doesn't like like plug-ins. So So That's why they have been pushing this web RTC standard to have The audio and video codecs and the and the UDP networking in the process so so we are really piggybacking that And just basically doing the same that Google is doing but but just Integrated to to 3d worlds because that's something that they don't have in in the hangouts yet Okay, then there's another question about building and an inventory so we actually have an editor Like a simple scene editor made So it works in the browser and also the 3js itself has an editor as well and if you want to have an inventory then Basically, you can develop that using the normal web web technologies But we don't really have like a reusable inventory system as an open source so Because our applications usually don't need that About standards, there's a question that what kind of standard should we develop? Well, I'm personally interested actually in the networking So I think it would be interesting if we had like a standard protocol for virtual world networking And and we have proposed our entity component attribute system for that. So Because in the process we already have Like the low level standards but but but it for virtual words Interesting to have this kind of higher level standards for for virtual world applications also a big topic is the Is the content formats like many in the game business are using fbx Which is really a pain because it's a closed and secret Format, so we have been testing this gltf format, which is like an optimized format for collada files so that's so like asset standards is something that would be good to have and And there's also a question about Babylon JS, which is a nice kind of like a 3d JavaScript game engine from from Microsoft Yeah, it looks nice, but 3js is more popular and kind of small and simple So we have been sticking to 3js and what's actually interesting is that that mic Microsoft itself Also has been using 3js recently because the photosynth to uses Uses 3js as well Okay Yeah Okay, I think I returned the questions soon, but I show my Final point quickly so I don't lose that altogether So So this is that one one point that I want to make that okay a big point point about the web is that This ease of access that okay, we can deliver stuff to the masses, but I think the For me a big point is also that the web is I think unique in in how programmable it is Like really like the process learn is an empty canvas and then then when you when you go to a website You download code and and it can do anything and and provide you any kind of user experience and I really love that and and and I don't think that really exists in anything anything else and how they have made it secure and so forth And of course there's like so many many libraries that you can mix and match and so forth so so how I see that what it means to the To the virtual worlds is that that it when we are running virtual worlds in a web browser and when we are using a Chava script Chava script libraries like 3js to do it We have all this liberty when creating the virtual well scenes so So you can you could even customize the rendering for your scene or or whatever in the in the user experience And you can use custom shaders for your objects or whatever so I think that's That's for me really interesting because because I wish that we would get Really rich different kinds many kinds of Beautiful artistic and also customized to use and all kinds of virtual worlds So so I think the web platform has a has great great potential for that and I also and this I mentioned also earlier that that Thinking of education or something where you have this Like Moodle and or something and or or blogging or whatever where you have content management systems The ability to to have both the virtual world code and then all this other web code in the same Like as a part of the same case in the same Chava script engine where you can just call functions from Like Yeah, any function from anywhere, so That's that's really interesting for for me So so that's what I'm a bit afraid that That we might miss if we only stuck to the to this M script and or unity way That's it. That was my my stuff There's some screenshots of our things, but I a check question is no no again Okay, so Oren was asking about web viewers So and yeah, you cannot just use UDP so so basically Yeah, what we are using is web sockets with TCP. So so yeah, if you would When I do that would need to add Like a web socket server Support to the to the open simulator and I think never done Did that already to some extent? so So that's the question whether it would be difficult to adopt the and the real extent API for that and I think that It would be actually really simple to to implement the the tundra protocol In open simulator because it's really simple We are using this entity component model and and the network protocol Reflects that so we only have like like six messages or something like that that create entity delete entity create component Delete component and modify attribute and then some like a login message So so it's a really simple protocol and really simple model But you can actually use it to to synchronize any data so for example to to add prim support you just would add prim components to the to the client side code and and then you would make the Open steamer socket code to to sort of put the prim To the to the prim component, but you wouldn't actually need any any new network messages when you added prim support because the generic component attribute synchronization would Give the and the client the prim data so so I think this way it would be possible to To map all the open sim data prims and avatars and whatever you want parcels and so forth By just using this generic protocol Yeah Austin has a Question that sounds a bit philosophical to me. Is it better to try and push the web into a viewer or viewer into the web? I Think it can be interesting to do it both ways that that we can push Like a viewer to the web so that anybody who just visits a webpage can see and participate in a world but but also like in Krista's effort and also what we saw in the previous session here to make the native viewers customizable It can be interesting to actually support Like having custom web views With HTML and JavaScript in in the native viewers so that so that also with this powerful C++ applications the The user interface can be or the user experience can be customized the same way So so I think we can go both ways Okay, I check if there's some other questions, but I don't think you have much much time though Yeah, or in days Talking about the Google Google and loading and so forth and that's something I would like to know that Google has been now integrating Google Earth to to Google Maps Using WebGL and And it's pretty cool like it's quite fast and and they they deal nicely with the level of detail so So that's also like a cool demo of how how you can render big worlds with with WebGL so Because earlier Google Earth required installing a C++ written plugin or or a separate Google Earth application But but now thanks to WebGL. They are sort of integrating it to the to the maps Okay, I misunderstood audience point but but anyway Anyway, that's that's what Google is is doing. Yeah Yeah, I don't know if I have a much wrap up I can say that But I'm happy that I have a working technology stack with the real extent stuff so so we can use the Tundra server and we can use to web Tundra client to To to create virtual worlds and make applications, but I said it might be interesting to to try adding a Support to opensim as well perhaps with the same protocol or or just doing an OpenGL or WebGL viewer otherwise Yeah If anybody if anybody's interested in working there we are certainly certainly interested in participating and and And we are also funded to do this work We have a two-year continuation for the EU funding that we have used for for one year now Okay, Tony. I think I think we're gonna have to wrap up the session, but thank you so very much As a reminder to our audience you can see what's coming up on the conference schedule at conference.opensimulator.org Following this session at 11 a.m We have a break in the schedule for lunch or dinner wherever you may be in the physical world We also encourage you to visit the story wheel exhibit in the education to region to view a tool created in the 16th century Called the books wheel which can be thought of as a precursor of the modern website In addition, if you are a crowd funder at the exclusive access level or above You are invited to a VIP Q&A session with today's keynote speakers in the staff zone auditorium at 11 a.m Finally we'll return after the lunch break in the keynote regions for an exciting keynote address from Philip Rosendale of High Fidelity who will attempt to answer the question. What is the metaverse? Thank you again to our speaker and to you the audience will be back after lunch. Have a terrific break