 Hello, everyone, and welcome to the 9am session in the Developer and Open Source Track. As a reminder to our in-world and web audience, you can view the full conference at the conference.opensimulator.org and tweet your questions and comments to atopensimcc with the hashtag OSCC14. This hour we are happy to introduce a terrific session called Dynamic Virtual Environments, Possible New OpenSim-Based Approaches. Our speaker today is Vitaly Chashan, the Technical Director of INTECH IND-LTD, which is a leading software developer of Open Source Community Triality. He received his undergraduate degree in physics from Moscow State University and the University of Liverpool, where he received his MSC in IT with a master's thesis on the use of virtual agents in the OpenSim environment. Welcome all and let's begin the session. Good morning, and thank you for your attention to my presentation. First of all, a few words about myself. I am working with OpenSim since 2008, and as Sun said, now I am a Technical Director of INTECH Limited. The main activity of this company is OpenSim-based 3D solutions. Also, I am a leading software developer of Triality Community, which is focused on Open Source OpenSim and OpenSim client development. This presentation is a result of our work during the last three years. Let's start from some theory. It's very simple, but it gives an idea what this is all about. If we look closer at OpenSim ecosystem, we'll see that there are two points of view on the whole program complex. Let's call it developer and scriptor points of view, and they are different. The developer knows how the client and server works in details. He could imagine what is OMV protocol. He knows about various parts of client and server and their interactions. But the virtual environment itself is a bit abstract for him. He thinks in the categories of protocol, client, server, and their models, and knows how do they form their picture on the screen. But that's why the picture, the scriptor sees all in a different way. For him, the virtual world is the main part. He sees objects on the scene and the interaction, and thinks in categories of scene and objects. He may know nothing about the protocol, and the client for him is a kind of black box because he can't do much about it, well, because parts of scene too, actually. If we'll try to talk together, we'll see a few classes or categories of developers working with OpenSim. First of all, core developers, who works on case services, scene code, OMV protocol, etc. I would put in this category of developers, client developers too. They also know very good how everything works. Then we have server extensions developers. They make in region models additional services, etc. And they also know the architecture well. And then we have the scriptors. Those two categories have a developer point of view. They think in terms of protocol. They think in terms of scene, but the possibilities are limited. It seems to me that there is a missing link or level, you name it, between these developer categories. This is a developer who can create full-scale applications in the frame of OpenSim, but at the same time, he could think in terms of scene. In this presentation, I would like to discuss the possibility of creation of this thing. Let's see what we may need for this. Let's start from the modular structure of OpenSim, which allows to extend scriptor possibilities. There are a number of ways to do this. Region modulus, direct API extension, for example, OIS-CC-AP, OIS-SLAP. Using a new .NET then, which is extension allows us to do this modern work, but all of this is not a scripted work. Usually the region modulus or core developer does this work. And as a rule, by very specific scripted requests, that's why this process is actually endless. The OIS-SLAP functions could be added and added and added. I suggest to look at the situation from the other side. When do we need to stop? Which set of possibilities is enough? To answer that question, I propose our definition of virtual application and dynamic virtual environment. So virtual application is a program unit of virtual environment fully independent from client serverization and network protocol application, which has a full access to programming language standard library and ability to use dependencies and full access to client and server methods. And a good analogy of virtual application is a program in a modern multitasking operating system. It could use all possibilities of the programming language fully self-contained and depends only on realization of standard libraries of programming language in the operating system. So we gave a definition. Let's see what's necessary for realization of such conception. First of all, we need a truly modular client in other world client-side plugins. Second, we need a possibility to use a full-scale .NET language in server scripts. Third, for many reasons, we need a possibility to use pre-compiled assemblies as server scripts. Most of this is required to create a complex project with various dependencies. And at last, it would be good to reuse as much code as possible from all sources. Client, server, and Libo MV, just not to reinvent the wheel. And now, I'm trying to tell what we have done already, but please remember that all of this is only a very low prototype, which requires a lot of additional work, testing and discussion. Many of our solutions may be far from the best. Start from the server part. What we see on the screen is a classic LSEL script. It has good sites and bad sites. Good state mission and overall simplicity. Bad sites are the absence of classes and entertaining class about arrays and more. And the main problem is that in some degree of complexity, due to global variables and many other things, our script became a crystal-pure example of spaghetti code, which is very difficult to read, test, and de-factor. So let us try something more interesting. And this is the same script in C-sharp. It looks quite similar, but a bit longer. This is not an open C-sharp. We have the class and the namespace here. Actually, this is open-sync script on some stage of code generation, which is hidden from the script. But the difference between LSEL and C-sharp scripts seems not very large at this moment, and it's unclear what it gives for us. Let's go further. This is more interesting. It's a work with class in script. What we did? We defined a very simple class. And wrote a couple of methods for it. These methods we could use in main script and by passing to a constructor of our class main script instance, we could use standard LSEL methods in our class. There could be a question about the state serialization of such script, and I'll say a few words about it later. But we have a new problem. It's very inconvenient to work with such code in a default client script editor. That's why we propose an integration with Xamarin Studio. Xamarin opens our script as external editor and provides a code highlighting and auto-complete. How it works? We have a template of a solution and necessary dependencies on client's side. When we want to open our code, the client creates a temporary Xamarin project. The next idea. Why we could not have a standard library of such classes for simplification and standardization. Here we see the cs.host class, which is actually a wrapper on a little set print something. This class is not serializable because it requires a full initialized LSEL API. That's why we implemented a veritable method initglobals, which works on script deserialization. We could init such classes in this method. And to finish the word Xamarin, there are two new asset types. First is a generic file. Actually, it's a blob without any properties. It could be used for inverts of file exchange because its only function in inventory is to download it. But it's mostly template, which we will need further. And the second type of assets is a pre-compiled script. Let's place it to the print inventory. It starts like a usual script. Also, it could be used as a dependency for other scripts. We haven't completed it yet, but first tests were successful. That's all about server side. Let's move on and let's talk about client. We'll start from one thing, which is a bit of a topic to virtual application theme, but gives some new possibilities. This is shared media controllers. What is it? Imagine that HTML, which sources on the left, displayed on a print side. We'll see two buttons. Up and down. JavaScript code, which runs by click on them, could send a message to script in this print. It's on the right side. The next is of those. By pressing these buttons, the print moves. We also added an opposite message. A script could tell something to HTML page. It could work not only for HTML in a print, but for HTML in user interface too. We also added a basic synchronization for media plugins, which allows some very basic co-browsing. This extends the script possibilities, but as I told, a bit of topic to virtual application. This could be used in LSO and does not require plugins on client side. So let's move on. For plugins on client side, we decided to embed monorun time. Minimal size of runtime is about 10 megabytes, and it's quite satisfactory. We made one main assembly, which allows this interaction with client functions. Unmanaged code. Also, it defines the plugin interface. A few simple methods. And on a client start, it's search for DLLs. It's realizing such interface in the plugins folder, trying to load them and register in the system. LibreMv code is actively used in the process. We'll see you this later. But first, let's discuss the main problem of a plugin, which is what is necessary to realize. First, we have to bind plugin with the main message protocol. We need to track in and out packets and react on them and have an ability to send such packet from plugin. Second, we need to provide an interaction with client interface, user interface, client UI, user interface. This includes an ability to read and set user interface values of user interface controls to track messages from them and to add new elements from plugin. Third, we need to provide some connection with client internals. We could not do everything just by sending a packet in frame of protocol. Let's see what we have done already. So, message system. If you are familiar with GridProxy from LibreMv, it will be clear for you. There is no actual proxy in our realization. Plugin passes the message name to client and the client subscribes it to this message. When the client has this message, it passes the message to the plugin as a row by array. Next is simple. We are using LibreMv to deserialize the packet and to work with it. We have not realized yet packet modification like in GridProxy, but it will be done soon. The sample is on the screen, the sample code for the plugin. Packet injector works in the same way. We create packet, serialize it into byte array with LibreMv and pass it to client. So, that's all with messages and next is client interface. If you want to work with a client control from plugin, you have to give it a unique grid in interface description. By change of such control, message on control will be sent to plugin and you can get a set value of control by this grid. It may be a bit cumbersome, but I think that it could be extended and simplified by a new project type for Visual Studio or Xamarin. We could create a graphical user interface editor and with some code generation, we could simplify the process to inform variants. So, you draw a button, double click it, and you got your handler encode. Also, we added an ability to create and register a new floater and to register an XUE command handler. Well, the code shows how it works and it's simple enough. And the last point is client internals. Of course, we just have no time to create monobudnance to all client functions. But as it seems to me, it's not necessary from architecture point of view. Most of things we need are in a few single tones in new view project such as G-agent, G-avatar, and so on. They are single tones and because of that, we could easily bind the methods via static wrappers, an example is on the screen. It would be very good to define what functions we need to bind in such way, but I believe that it's possible only on open development and testing stage. So, we just created a few bindings to test the conception. Client. And now let's see what we have in F-hole. We have two types of projects. Client plugin and server assembly. Both have really wide possibilities, but they are two different projects. And it's not a virtual application because we have to think in terms of message exchange between them, in terms of protocol. So, what we can do about this? A lot. Let's start from the realization of the interface to server assemblies in our main script. This gives us a possibility to find the client by UED, UID, unique identifier. And to call some client-side methods from server script, the sample is on the screen. Then, let's define a total specific server script from client plugin. To do this, we'll implement the methods, which allows to register a script as interacting with plugin. Let the script be registered with exact name and unique ID. Then the client enters to the scene. It receives a list of such scripts. Then, we can find the necessary script in plugin and pass the direct message to it. Now, we are at the process of realization of the opposite operation. Message from server script to exact client plugin. This is a bit better than separate plugin on server script. But we still have message exchange here, and we want to hide it from the scripter. There, we can use this problem, and the next slide is some ideas about it. We need a code generation here, and that's why a project, a new project type for Xamarin or Visual Studio. It would be good to have three project types, a client plugin, server assembly, and the application version, which is both of them together. It would be good to have a user interface designer, as I have told you before. Maybe it would be possible to create some testing environment for debugging of such projects. Then, we could use a code generation, for example, by attributes. We could mark some methods in server script and in a client plugin by a specific attribute and generate a code, which allows us to code directly from the opposite side, like VCF or other web services, you know. And combine that with file as a type, and you have the ability to send a plugin installer to the client from server script on some event. And this all is very close to our definition of virtual application. And conclusions. Of course, such project has a lot of problems, even if we don't take an account that the current realizations are prototype. Code is dirty, not covered by tests, and so on. I'll try to name some of them. First of all, it's a security problem. Client plugins are dangerous, because the source could be risk-exclosed, the source is closed. But server assemblies are much more dangerous, even if they're running a sandbox, because they could crash the server. The aspects of security are very serious, and I think it's require an open discussion. Second is performance. Monocles and marshalling could be a problem here. We need tests and measurements. Third is hypergrid. I have no experience with hypergrid code, so I could not imagine the problems here, and I suggest this as a point for open discussion too. So, conclusions. We may have a full-scale model, we could even move some parts of a standard user interface to managed code. Full-scale server-side programming language, an ability to create a standard library to unite them. Visual Studio Oxymarine could be used for projects and development. In future, we may have a virtual application by its definition. Also, it should be noticed that plugins and server assemblies are not under LGPAD-BSD license, like the core code. They could be distributed on commercial basis. For example, I think it's possible to create some kind of plugin market. This was not the goal of our development, but as it seems to me, it could help for the globalization of OpenSim in general. And finally, I would like to say a few words about the history of this project. In 2009, we started work on the project for aviation personal English training in AiraFloat Russian Airlines. This was the continuation of implementing distance learning in AiraFloat. It was a leading developer. Project assumes wide using of agents or bots in a training process. And the aspects of the projects made it really difficult to realize by OpenSim standard tools. Bots were used for role-playing games and demonstration of a typical situation. And the bot scenario was really hard to submit. We need some kind of visual editor for such scenarios bound to OpenSim and an ability to start this scenario for demonstration and an ability to change one or many bots to students. And also the ability for a teacher to get the control of each bot at each scenario step. Well, the system should be tightly integrated with model, airbus, LMS, and internal ERP services of AiraFloat. Some of them are really old legacy. And bots needs voices which was realized on text-to-speech and there was many other difficulties in this project. Well, in AiraFloat, all of this could be done by a standard OpenSim approach. I will sell server models, but it was really hard. That's why we started to extend the client abilities, the client and server abilities and in time they understood that it could form some concept and it was a concept of virtual application. The project for AiraFloat was successfully realized and now it's widely used in AiraFloat. It's used in AiraFloat and in current moment it's and its extension is assumed. So, that's what I made this slide. If you have any questions, then please email me to WCHF and Yandex.ru. And there is no open repositories for our code for some reasons because the code is a prototype and as it seems to me, this code could be dangerous in Griffer hands. It's it would be really simple to write some Griffer plugin, but if you are a developer and you are interested in this code, then just please mail me and I'll give you the link or send you the code in some other way. We also set up a send box where you can test and try the they can try all of the abilities of our client and server. There will be a link on a trial community site which is on the screen by trial, by triality room. I think the send box will be complete on Monday. So if we have some time, if we have some time, I can try to show a video of our Aeroflot project. One moment. If I can find it. Yes, this should be a video. So you can click to play it. And this is a game with bots for Aeroflot Russian Airlines. Someone wants to ask something. I don't know if you saw, we had one question in local chat Vitaly asked, why is C-Sharp bad for scripting? I don't think that C-Sharp is bad for scripting. I think quite the opposite. It's much better than I ever saw. Can you also provide the link for the video that was up during case folks weren't able to get it to play in world? Yes, one moment please. Thank you Vitaly. Is that the expo virtual link that you gave before the conference? I can drop it in. Yes, yes. The question is, what operating system did you embed? Mono in the client's site. Currently, our client is Windows 64. We have a Linux variant too, but because actually two and a half developers in our community and I'm not familiar with Linux and Linux developer is a half of developer. Does Mono facilitate making a secure sandbox for client's site scripting? Fine. I think you can rule out file system access and perhaps open arbitrary network connections. Yes, as far as I know, the Mono sandboxing is quite the same as .NET, so it could be controlled. Would a more traditional Lua or Squero make more sense? We tried Lua, so we tried scripting on Lua. We also tried a JavaScript version of the V8 engine. And yes, it works for scripts, but I think that we need an ability to distribute a plugin in a computer like this. So did you actively active use the same C-Sharp code in shared URLs on both client and server sites? Yes, it's one of main features and one of main ideas of virtual application to use the same code that's called standard library on a client and server site. Do we have any more questions from the audience? This compares with using something like JavaScript client site. First of all, how do you think this compares with using something like JavaScript client site? The question is, well, first of all, we can use JavaScript on our client site via shared media controllers. We could use an HTML as parts of our client interface and it will interact with things. And it's a second one from our experience with Google V8 engine with embedding it. I would say it's more difficult to bind JavaScript than to bind Mona. But it could be done and I had an idea to use the JavaScript for user interface only and Mona for other parts of plugins but it's still not realized. Yes, of course. How do you think this plays with what Krista was talking about about this morning? Is there a room for cooperation? Do you think? Yes, I hope that there is a room for cooperation and I really would like to finish this project and to discuss what can be done about it and to contribute it. It's called to OpenSim. Can you elaborate on your statement that this paradigm might provide an opportunity for commercial ecosystem for plugins to OpenSim? Yes, because plugins are loaded dynamically. They are not under lgba license of our client. So the plugins itself and server-side assemblies could be closed source. It's not a violation of a license, I believe. Okay, any more questions? Looks like there's no more questions. So thank you for your attention and please feel free to mail me if you have any questions and visit our sandbox which will be ready in one day and you can try our client server. Thank you. Thank you Vitaly for a terrific presentation. As a reminder to our audience you can see what's coming up on the conference schedule at conference.opensimulator.org. In this room the next session will be virtual worlds on a web browser with Tony Alatalo at 10 a.m. Thank you again to our speaker and to the audience and we'll be back shortly with the next session. Yes, one moment please. It should be... Oh yes, this one. Thank you.