 For joining this presentation and I will try to share a concept about what I call this web of twins So it's more an experimentation than a Established project So first who am I so I'm a software engineers and I have been a long time a software Free open source software user and I started to contribute Like 20 years ago now and I was I had the opportunity to join some industry and open source project Like Tyson I was doing some contribution for Intel at the time and then I moved to Samsung to work more on IOT and Currently I'm open for any kind of cooperation of sponsorship or I have to think about what can I am to do in the future? but anyway first let me just Introduce the concept of what is a digital twin so in a year 2000 and when I was a student I hacked something which was very close to this Concept, but I didn't know it so I made us some kind of Puppets you are you have in 3d in a VRM early in the browser and I was working with people Puppeters we have some kind of sensor and we are trying to animate the the puppet over the web. So it was a Nice demonstration there was no real market for this But it was possible at that time But it was not using any open source thing. So now is the thing that change and I will explain why so the concept itself is has been well promoted by the Researcher from Florida Institute of Technology and he described as a digital replica of physical entity and The specific is the specific surface at that both are somehow connected So if you move the 3d model for instance, it will have an effect of the rear World object and the same in the opposite way so from this you can have some kind of New service to to design application so you can do some simulation, of course You can use this as a design stage of the product of the device You are trying to consider and you can try to think about what kind of IPI is this thing can be can share to the others and trying to Establish new services, even if you don't have the product So it's a part of the product life management and it's quite interesting because it can When sometimes you have some delay due to production of sourcing you can work on let's say the poor version of the device of the thing you want to work with and It's quite popular now in the industry when people want to do some simulation of an production line and so on Instead of building the whole thing you can try to get some simulation and see how it behaves and if you are configuring or altering a bit You can get some early feedback without any cost and then cities also is a in the domain that can be useful. Yeah so before going on I want to and find some ethical consideration because Large-scale deployment of Digital twins can be problematic. Yeah, we knew we have now a lot of many issue with social media private information Private communication and so on so if we do if we start try to do to duplicate the world in the digital world Maybe we'll try to have some ethical Consideration from the beginning though. So don't wait for a problem to find solution. Maybe you just need to design your system and trying to avoid problems. So one thing interesting recently is that European regulation as recommendation a strong recommendation for system designer to handle privacy by design and by default So I believe that's it's an opportunity not to miss from open source communities because Privacy is part of our DNA. We are somehow users. So we know that Everything should remain private other when it's possible. So we I think it's something where we can Give us answers to this kind of future problem So now let's talk about The web it's a design for enter our ability that's for sure and it's a it's was it's a huge Achievement because today we are still able to browse these websites that been Designed maybe 20 or 25 years ago and it's from a solar you can connect everything to the web. So that's The base to do something it has now a really good 3d support It was not the case when I made this puppet we are you need to add some kind of plugins and so on it was a bit Boiler plate. Now you have a web GL inside the browser. This means you have a good performance because it's working directly on the GPU so making in games in a browser is totally frisable and Yeah privacy is one of the Feature of the browser. So you have some kind of process in isolation in the browser So that's something interesting for the previous slide and next people in a IOT and specifically the W3C. We are specifying specifying protocol and so on they propose some There's some recommendation to design IOT system without reinventing the wheel but by adjusting existing standard So we have like HTTP G-Zone, G-Zone LD and WebSocket and this kind of Technology is still applied to IOT and Modzilla made an implementation of this web of things concept and the selling Point of this project is making a home platform a smart home platform To give users the control of their device. So they are handling privacy from the beginning of the project so device and Get away are working together into user Network so there is no leakage possible no Data logging and so on so the Software itself is providing some UI to control all those things to connect them together to establish some behavior and Also, you have extra features that you can connect to your home gateway remotely Share some resource one per one if you select to share Let's say a door to some members of your family you can do this And it has been designed to be flexible and to be Support new plugins and new features. So that's the platform I've been working on so But web things now are dumb device that's connecting to the gateway, but they are using a basic HTTP Protocol so it's based on rest and they are describing all the resource using a Modzilla schema. It's Simplified version of what the W3C is doing But basically same it's a set of properties to describe the thing is that the home button I can turn in on and off some level if I want to adjust the volume of the lamp of whatever and some action if you want to give instruction to your device like okay faders the brightness of your lamp slowly and then turn it off for instance and also receive some events from device so But if you want to implement simple things you can just focus on properties So security is under advice a gateway. So this mean if you want to share the outside this gateway is Ex giving some rest API with tokens so you don't have direct access to the device you need to get identified somehow and Yeah, that's it So for instance, I made a very minimalist example I call it a smart home example, but actually it has only one thing inside This is just a solar panel on the top. So it's just a simulator so you can try it it's open source and What you see on this Description you have some ID to identify and the properties as I just mentioned and a link for each properties And then you can put some you can get the value or put the value using Get or post request it was straightforward So now if we want to To address the problem of a visa is anything the things so I just mentioned about the WebGL for G for the Accelerated graphics and now lately we have a new specification called a web XR Which is an evolution of web the how we used to have a couple of Years ago. I think this was he was showing just before and now it's more Unified and it's support in augmented reality use case and To do Some application you can use a frame framework, which is very very simple So 3d is not something it used to be something really complicated But now you can do this as easy as writing an HTML page in JavaScript. So there is a basic one you just import the library and Describe a structure of different element and you can bind with this. So here is a sphere on the on Inside it so So then something interesting is a GTLF format So it's a format made for delivering 3d content to the web It has been proposed by the Cronus group who was in charge of open GL and also 3d 3d formats and It's basically it's just a structure in G zone describing the same graph of each element of the Of the content and it's already supported in a blender 3d So that's really interesting because if you have to work with designer probably they will not Describe all their world by writing HTML 5 HTML a frame tags that I will use a tool so here is an example with the This GTLF Outputs so what you see in the background, maybe it's a bit dark, but there is a blinking piece on the on the roof on the right which is simulating the value of the my solar panel, so if there is a Lot of Sun you get higher value and you can simulate this and What I'm showing in this example is that you have a direct bindings between the description from the web thing so the physical thing that really is a the actual device and The same description is aligned to the GTLF format. So if I'm here what I'm doing in this I'm just updating the value So if the value move physically like from a sensor it will Access to the DOM tree of the web of the web page and change some attributes Now if you want to do something more physical so something real I made a DIY Smart home of this size so it's like a toy and I really I made a my solar panel with a With a you know the garden nymph you can find for super cheap in the outside You can you just plug this directly to a microcontroller or board and you get some Basics of what a smart home with a solar panel will work and it's running a JavaScript on the microcontroller so you can then expose all the resource using the web thing API for IOTJS which is a tiny JavaScript runtime made for low-end the GPT microcontroller and also I made also a module for dealing with sensors So here is demonstration, but I can try to make a live demo so Just reloading Connectivity is not that fast. I'm not sure it will work, but Okay, so what you see on the corner here is is a camera I have on top of my desk and there is a small robot here So I can control different units. So here is a light for instance, so I can change the value and The robot here is described from so it's just a set of motor and you cannot adjust different angles So let me show you It's running in the right time. So so what you see in In the top is the actual thing and the BOLO is a Virtual one made with a frame. It's not a one-to-one model It's it's not the same because I made it by my own and I found that It can be a bit challenging to have exactly the same system because if there is some mechanical instruction Oh here. I'm showing some the rule system when you can Change the attribute of the physical things depending on some context and so so There is the same rule in the opposite direction So when I'm pressing on the on-button, it will move forward and if I'm turning it off It's with good backward. So as you can see it's updated in the same time. It's not there is no smooth animation Oh, yeah, I put a sensor on the head of my Of my robot and I'm measuring The color of what I see in front of me. So if I change the color of the light, yeah, it's updated in a sensor So this is a one use case also I made a 3d version of my dashboard So this is not the actual thing. This is just abstract widgets in 3d and below it can interact exactly the same so you can Be in some kind of immersive world and interact with it and it has the same effect in reality So maybe it can be useless for many Any cases, but if you are considering you are you have a robot somewhere on some planet in a space And you want to control it. It's more maybe from an Environment which is immersive is better, but more versitically when you have very small mechanical parts that fit in a very very restricted Space having having the ability to scale it to larger level and to see and look around how does it behave can be also something interesting Yeah I think I have another demonstration here Oh, yeah, this one is about using sensor Oh, but maybe I handle more questions later So yeah, so what I have on the screen is a it's a It's a 3d scene which is just a light bulb and it's updating accordingly to what the sensor is showing So it's working the browser and browsers now are working in a vr ad set So you can you have done you don't need to create a specific application. You can Have something so sorry for the quality of this video, but what I wanted to show is that Uh, I can make also standalone application here. I made a link and if you look at what I'm clicking This is the address of my web thing. So I can get direct access to the To the sensor and then it will update in my 3d view. So it's not it's vr now I think yeah, we made a version with a here at the same time So yeah, the application is portable. You don't need to deploy it to a store You don't need to to under a specific SDK. That's the premise of the web. Well, but that's the theory because in practice most of Vr vendors I'm using extension and some So yeah, here I'm controlling Some led and all the widget I'm having action on now updated in a In a vr world also. So what am I doing here? is okay turning on Uh A fan. Yeah, it's off And I think yeah, I also a smart outlet Um Here is a switch so I can So it's the same switch we have in uh dashboard can be also integrated for the 3d world and it's having effect in the same time So yeah So the beauty of this demonstration is that I use a modular Um schemas and it's generic enough to describe all kind of things You don't need to have to do something specific if you have to deal with billion numbers and so on It will work for a wide wide range of devices So yeah, the same in 3d here So yeah, if there is any question, I would be happy to give any hints So I'm sure there will be questions because not everything was Self-evident, especially when you discover all the possibilities. So let's start with the first question rather By the way, you forgot your laptop. Yeah, it's okay. You get to I think yeah Oh, this is another demonstration. I can show it's only a minute So yeah, I made this robot and uh, so first time made a web thing of just the motor A server motor where I can adjust the angles and it I was updating a airframe So as soon as I had this Working it was straightforward to To animate a tree a real robot with four motors. It works for one. It works for four So here is a 3d version of it where you can look around it as I said previously So anyway, it's a proof of concept so far I made it I'm very modular. So if you want to build things you can just pick some specific components and you can build something else And it always opens also. I need to share you the links Okay, I I was just wondering Which part was the the most challenging one the most challenging part of the project Okay, thanks. So you said what was the most challenging part of this? Yes. Okay I would say that Uh to identify the wrong person to to introduce me to this kind of technology So this story started actually one year ago in this room when we met together And we had some kind of a different approach of something common and we met again In my home city, right in France and we started to act together something in and in less than a day We we we we established the both way direction from sensor to really to To 3d and back and forth. So that's we got pretty fast results. That's for sure And to answer the question No major difficulties, but when you are dealing with iot device, it's not very reliable for instance So this robot is made of form. It's like a toy. Actually you can buy it for A couple of euros and it's sometimes it gets stuck and so on, but Pop-up, but I think it's not a problem. It's not something you're running in production So if you want to build something real like physical device Yeah, reliability of the device is uh, it's just it's a full full-time job. Yeah Here it's a research and development project. Yeah Do you have any plans to use? Other robots that are low cost robots, but A little bit, you know better than yeah Yeah, so the question is If I have any plans to use a different range of robots I'm not specifically a geek of robots. I wanted to make athletes wants, but I figure it out that The robotic community now in open source is quite interesting now. They are making a lot of open hardware device you can get but Yeah, does it work the investment for me? Not specifically, but if there is Sus project if they are looking a Software stack to use that's probably yeah, I will be more than an empty for them to use this But again, if you go into robotics Probably JavaScript is not the best option. Yeah Because you have a lot of hard real-time constraints So I'm not sure it's part of the standard yet. So The most important platform for instance is Rob's the robotic operating system But for me, it's not very it's not critical because it's just Unharmed is not going to to to break anything. But if you are making like a RC car or something like this. Yeah, if you break too late, you can destroy it Okay, can you Oh Okay, so Let's start for the lower level. So first we need a hardware platform. Oh, okay. So question is What is Where are the tools are interacting each between each other and What is specific to modular and other components? So once you have this hardware Uh Set up you need to implement the red this modular rest api. So it's uh, it's not a software It's mostly a protocol. So you can do in any tools you want for me, I made a I ported a Node.js library to iot.js, which is a A javascript runtime from samsung So yeah I'm in this case. I am in both world And then once you have this What rest api from the thing? Yeah, you need to connect to the gateway for sure And then if you want to get access to the thing directly if you want to do it properly you need To share the resource through the gateway to another device that will understand the the rest api so This api can be implemented in any any different languages So I made my javascript code. This is just a client and then from the rendering I reuse another project which was initiated by modzilla, but it's not It's not it's not living by its own which is as a frame framework And this framework is based on the standard I mentioned about web xr Which is standardized by another group. So the full chain is a different pieces of open source each is smoothly working together because They are aligned to a fixed specification. So Doing some quick and dirty acts is something interesting, but To achieve some cooperation and You are if you don't want to lose time to do some bridge from one technology to another It's better to to use the same all the longer all the chain Nobody is speaking after so I can stay here the one one year No, no, that's our schools Or what are the what's the ideal contribution you would get on those repositories? Something unexpected Good because we have a creative crowd Now something I think that could be interesting is to To geolocalize different objects and having them working together. So maybe either at a human level or maybe something bigger like a city, let's say We are Realizing traffic jams and so on in real time. That's something that can be can be nice to see But yeah, no, I have no specific ideas I just wanted to to validate that this can be done. So then we can apply to the many use cases Yeah, if you have some Some hints about how do you Create a web framework that interact with in real time with a lot of different sources That can be interesting because if you here it's easy because I have only one device and maybe a couple of dozen But if I want to scale to maybe a thousand Maybe it's not the best way to do it. So I have to maybe Use a different protocol. Yeah, I know that there is some streamer streamable Like time series and so on. So maybe it's something I should look look at