 Okay, so hello, I'm Aurelius Bonales, software developer from the Twainia company called NFQ and today I'm going to talk about webcam-based games. There will be three parts. First part will be from the user perspective. What is like to play webcam-based games? What's the main difference compared to another? Second part will be technical parts for developers. What do you need to know to make one game and how to debug it? And the last part will be from the business perspective or just some practical tips where webcam-based games could be used the most or what added value it can bring them. So let's start from the first part. What webcam-based games are? Basically, if we compare webcam-based games to other types of games, the main difference I see is that instead of using traditional input devices such as keyboard or mouse, we use movement and enough of here. Let's go to the demo. So one of the simplest example of a game is just that you can shoot the bubbles and it should work. That's all the demos. Okay, so I will just let you also play because it will be not fun. Okay, so yeah, just okay. So basically, yeah. Yeah. No, the person in front should not click let's have us to also to play it. So basically, as you can see, if it will be a traditional game, I will have to give everybody a keyboard or a mouse or some mobile phone. So those two play, but now I can just use one single webcam and all audience, well, at least not the widescreen webcam, but most of the audience can play the game. So that's the main difference what they see about webcam-based games. And here is one of the example how you can input data. It's like you see moving couches and you just try to try to hit them. Another way to do is have some here. And yeah. Well, you know, it's a demo. Okay, so I will just switch to the there will be a slide about the precision and all the stuff. Okay, so the bugging on the stage as usual. Yeah. Yeah. Basically, can you turn on the light? Yeah. Well, basically, what is this example? So yeah, well, basically, I will show what went wrong in the next slide. But basically, what you can do is not only get the movie and culture, but to have some controllers, fixed controllers and to on and have them like virtual buttons. So one, one can, for example, show that you are in this building and one that you have a good party at Friday. And basically, I made this kind application, not not like a game, but more like a VJ tool. VJing is like DJing, but with visual graphics. And so one of the most common thing in VJ tools is that you can have this four square effect. So basically, this is the second demo I wanted to show and the third demo to show the what are capabilities of basic games is to include some 3D graphics and some more complex, some more complex input methods like this. And what I wanted to do is you see the FOSDOM signed. And if you like the FOSDOM, the idea is you need to move it in direction. So it could be funny for them. I'm not there unhappy because you can see that, well, because of the issues with the first demo, basically, there is a sad face when you rotate the 3D model. So basically, just to sum up what I showed or wanted to show during the demos, there was three ways how you can input the data into game. First was just moving objects and you tried to get them. Another was an example when there were some virtual buttons, statically attached to some positions. And the third was when you have something like screw boss on the scene and you can just have different direction to the left or the right. And those input methods made three different interactions. One was you could just draw things on the view from the webcam. Secondly, you can manipulate the view or add some effects like the quadra view or quadra representation. And the third was just you can separate the controlling webcam and the output. And the last one, for example, can be used in advertising when there is a wall and people just come through and the paper is flipping inside the screen. So you don't need to see the controlling or the webcam is used to just make, concentrate the people to the advertising, not how it is implemented. So to sum up the first part, webcam based games are those who use motion, not the traditional input devices. Okay, so next part is for developers. And this game will be not possible without the open source computer vision library called OpenCV. And it gives two main things. One is that it gives you access to webcam devices. You don't need to worry about hardware, about operating system. And another thing is it gives a lot of utilities to make the work with visual graphics very easily. For example, one liner to just fetch the frame from the webcam, a lot of filters. And you can see the code here. Example is basically except from the real script. And it shows how the first example of bubble shooting was implemented, just comparing pixel by pixels to frames. But just reading the code is not fun. And in practice, if you're building something related to a webcam based games, you will enjoy the debugging when you have a webcam view enabled. So the last demo hopefully it will work. So you can see the input, the traditional what you can get from the webcam. But open computer vision is popular for many filters. And one of the filters is edge detection or camera filter. So it is mostly used to better recognize different elements and work with low light conditions. But the most important filter, what I want to show is using differences. Basically, if I'm not moving, you see the black screen. It means that there are low values. And when I move something, there are difference between previous frame and current frame. So I can know which, when I try to hit the bubble and when I try to, where I have some motion and interact with that. On top of it, I can put different more complex input logic. For example, that logic where I try to move the object to the left or to the right. Basically, the idea is that you need to distinguish the direction if I'm moving it to the left or moving to the right. Because it is not enough to just know the moving if it is pixels changed or not. So what I do in this example, you see the white color shows how intensive the differences are between the frames. And the blue ones shows the historical data of that differences. Basically, to save resources, I don't store all 20 or 30 frames. I just subtract the value. And over time, those blue values just decreases, decreases. And I can, for example, if I move from right to left, I can compare the right pixel with the left pixel and see which is higher and know in which direction was the movement. So basically, that is how we can debug computer vision. So just to sum up what I showed, you can analyze current frame. You can analyze two frames. And you can store historical data and make decisions based on that. But as you saw from the first and the second demo, there was some problems with recognition. And the first thing you will hit when you are doing something with computer vision or webcam is the noise and precision. So basically, what happened when I was doing like, if you see here, you can see that I was standing in front of the screen. And the screen was just showing what I was doing. And I was moving and the screen was moving also. So the background was, differences in the background was causing the error input. And the same goes if we have clouds, some mirrors, people coming from the street or somebody turns on the light or just there are cheap cameras that automatically adjust the white balance and the value is just differences. So it is one of the drawbacks of computer vision. So you need to have it always in your mind. So basically, it is the second part for the developers. And to sum up that, basically, the suggestion is to use OpenCV Library and just compare the differences between frames and between values and current and the older frames. Okay, so for those who think, so yeah, we think webcam based games is fun, but can it give some added value or even how can I sell it, etc. So basically, when I started learning computer vision and webcam based stuff and creating some games, I saw that, well, it's not used, it's not mainstream and there's some problem about it. And compared to other competitors such as PlayStation Move or V or Xbox Kinect, those are very similar in technology, but those are more popular than more popular than webcam based games. And the main reason as you saw it during the presentation during the demo is the precision and all the dependence of the environment. So you can't just give anybody a webcam and hope that he will not stand in front of projector and other environment you will never imagine or there will be no lights to mount. So there is a problem why webcam based games are not mainstream, but I see one area where those can have good to use cases and those are advertising and in particular advertising in IT conferences or other related resources. But for example, when I was trying to present my own project, I faced the issue that when people are coming from one auditorium never, they basically don't look at anything except the shortest path to the next room and the old ontogenetic modules such as paper or people just standing so they don't attack. Well, except to enforce the Microsoft freebie, but that's the exception. And I see more and more games in the conferences that can attract people. So if you have two or three people standing near you, so there are more people can come by. And the second thing is you can solve a problem I had during this presentation is the environment. In the conferences or other static or other control environments, you can put your webcam only in places where you know it will work. You can test it before so the accuracy could be much better. And what's important for open source that can base games, you can basically release them openly because you will be happy if competitors will use your brand and advertise your company in other conferences. So I see many advantages of using webcam based games particularly in advertising and conferences such as Faust and even. Okay, so as first presenter today talked about it's not easy to make games. So I realized it by trial and error. And one suggestion is even if you are making webcam based game, you still need to make it a game meaning that you need some sort of challenge even at least a score level or whatever. Also, there's webcam based games are quite unique and uncommon thing. So if you just put webcam and say, hey, it's a game nobody would understand. So just making some intro videos or some explanations is very valuable. At least when I tested it. So that was the response. And the last thing I want to suggest from personal experience is that you still need to keep the balance between all the new technologies, all the pretty graphics and the usability because if it's not responsive and if you can hit the bubble, even it is shiny, glorious, but it's not a game if it's if you can play it. So basically it sum up to the third part. And my suggestion is to if you can think of a better way to use webcam based games, it's in conferences or advertising. And that is my last slide. So I invite you to try this technology or even for my project, I showed the demo and use it in your own community. So thank you. Questions, thoughts? Yeah. Yes, we mentioned that there are many issues with the live variation. You want to get very into that, that you use pixel based methods. Can you try the feature based method or in the motion or network? Well, basically the question was I used I used to motion based detection if it is as a device, but there are many other ways. So I tried comparing well, first of all, this was part of learning open computer vision. So I wanted to make a game that works, not to be a PhD science in computer vision. So well, I didn't have enough time to compare to learn all the model tracking and all those algorithms such as well, all that detection and model comparison. But well, I tried color based. I tried well, basically, you can use some models that can recognize the face or recognize the arm, but still you will have the issue of blue or you still have the issue of wrong light, noise, etc. So it's just from my personal experience that comparing two frames was the most efficient and I haven't tried the more advanced ones. Other questions? Yeah. Is there an embedded software version of OpenCV? Yeah. The question is, is there an embedded version of OpenCV? Basically, I saw that there's version for Android. I don't know if there are a version for other devices, but probably there you just need to go to OpenCV.org and check it out. So it should be some version because it's quite tall project and quite famous. Well, basically OpenCV is used not only for games but to make self-driving cars. So it's very big, very impressive library framework. So using a webcam-based game such as very small parts, what you can do with OpenCV. Questions? Other? Okay, so thank you.