 Okay Is it fine? Can you hear me? So nice to see you Okay, before I forget this is like my company request. Let me make a photo with you. You can smile, but it's not obligatory Best audience today It's really nice to be here. I arrived late 4 a.m rise from Romanian testing conference after two days of workshops and then my flight today is cancelled and No, that's me. It's like speaker somewhere. It's like red dots during FedEx Visual on okay, I Have long-hand I can I can catch it. Yeah, so once again, it's really great to be here Survived still alive. My life will be happy. I Will talk about IOT today like Funny thing is that I do not work with IOT anymore I spend like five years testing IOT devices, but I still do it on my own just for fun So that's one of the most interesting topics for me. Well, that's the thing I really like to do I really like to talk to people and If I will find like two or three people who really want to check on your own not even at the company after my speech That's perfect. That's that's that's my success my personal success. I hope and hopefully later You will find it's it's real fun because it's fun and IOT Is just growing and growing so yeah before we start Click click as I said, I am David before I started speaking quite frequently I was David. Now I can be less international. So it's David. I came from Krakow in Poland, which is pretty far. So my travel here was 18 hours And I was barely entered the country because I the funny reason I had no printed Virtual visa So that's the purpose of virtual visa. You have to print unless you won't be allowed to enter the country Okay, and but I'm here. So what I do at my free time. Actually, we have two initiatives it's Krakow a and before I spent sometime working in Ukraine and we and set up something called Ukraine QA and probably it would be different city. That would be something QA and Then my free time. I just travel like here and I like food, which is pretty dangerous to be here Like free Okay. Anyway, let's talk about food later Back to the topic. That's actually the story when I started. So when I started was 2012 and probably When you would ask someone what IOT is at the time Maybe 5% of people would say, okay, I can more or less say what it is Now if I would ask the same question, who of you can explain what IOT is still 5% Which is cool because it's still growing, but it's still unknown and it's not surprising for me That's still so little of you are Aware of that. So just just just just before I was in Chicago during the conference and because it's like Market for most technological companies. There were more people aware of what IOT is But still it was much much below 30% but it's really not a lot. Which is not a lot. Which is surprising If you know how many devices we have on the market, but about it in a minute Now IOT, that's the definition from Wikipedia And I would I like to explain in just one sentence Internet of things is exactly the same as we define internet, but for real devices So if you want to talk to your friend You have communication. You send message and maybe later if there is proxy in the middle It's transformed. It makes some action and so on exactly the same with IOT devices instead of humans You have hardware devices in the middle. So we can have bulbs in the cellar You can have lightning environment and those devices will communicate between each other if you have smart home and you have Temperature regulator and then you have blinders. So you can you can make it lighter and darker You can turn on both. You can you can dim it so much make it Lighter and darker Everything can be connected and talking to each other. So that's exactly how we communicate how we use internet And that's the same for hardware devices everything depends on implementation, but simply Explaining that's IOT. So we have devices communicating between each other's Sending messages and then making some actions based on that message. And that's how I see it because IOT Because it's internet still internet is not only about hardware communicating between between each other But communicating with normal internet as well. So very often you have IOT devices connected with social medias There are many funny items. I worked for the company doing cameras for pets So now just before the speech and before I split my coffee I played with my cat, you know, because I'm here. So my cat is alone and and sad So I just turned on camera and then play with laser just to let him run Which is funny and this IOT Actually the market internet of pets is super funny, but this is different in different topic So that's how it's connected you you can have many challenges. There is Mechanism implemented. It's called gamification. So you can have the same devices I don't know smart month and who of you Just made more steps during the day. Yeah, or you can make the challenge Which is quite risky if you play with women Who lose more weight, right? And that's pretty common as well because you have smart way This is data from 2016 and there was 6.4 billion and it's pretty reliable source because it's Gartner Just one later. It was 8.4 It was 2016 Sorry 17 last year it was over 11 and this year is not ended, of course But it's predicted to be like up to 15 in it till 2055 Should be at least 25 billion devices so or more and more devices are connected And what is the reason well actually because small small companies are doing prototypes Defining technology and so on and then if it's successful big companies are buying those small companies and producing own devices. There is nice implementation. It's got is done by Philips, which is called now Philips Hue before it was just device now is separate company and then they they install 50,000 of devices and stadium arena It's pretty a lot Now what is that that the General picture is very simple. So we have and user Very often like 80% using mobile device because we want to be mobile We do not want to open a web page every time we have to just open application On the other side, we have hardware whatever it is like sensor bulbs cameras for pets And some somewhere between you have back-end processing data Sending data gathering because because of so many devices. There is a lot of data So big data here is nice topic to to think about as well And you have different ways of communications So it can be without any limit like Wi-Fi or limited by Distance like Bluetooth or ZigBee and so on Which is better? Actually, there is no good answer because it depends on the purpose. It depends on the context Bluetooth for example is Great if you do we are talking about smartphone so we can control only if you are inside You don't want to allow someone from outside you want to keep security If it if it relates to camera security camera, then probably Bluetooth is not the best option Yeah, being in range of Bluetooth, which is like 50 meters to a preview camera Well, I'm like now seven thousands to probably it's not for Bluetooth So that's simple picture. That's how I see it and that's how actually 100 of devices works If you if you think about single device now Why there are so many challenges and why it can be so interesting in terms of of testing Do you know the the the the actually idea of iceberg? So there is usually it's like, you know coaching You have one that 10% above which is success and below you have sacrificed hard work And so so at the top here We have something that the customer see and usually it's mobile application and some functionalities to Set up Actually your environment your home environment or external environment so very simple UX is the clue here to make it as simple as possible Ideal IoT device is the one that you can give to your grandma and you do not have to explain how to do it But below you have hardware. So all that stuff that must be Design developed and tested and then you have firmware. So who knows what firmware is Okay, more people than about IoT. So firmware is basically software on hardware Which means that we test the same thing software but in terms of integration with as it with hardware, so we have two levels like firmware itself and firmware of hardware now that's What I what I said before that small companies are trying to get to the market very very very fast very quickly And because of that, it's pretty painful because time to market is the clue It's not about perfect quality. It's about quality good enough for the customer or good enough for your investor But it's about functionalities. So that's slightly different approach slightly different different world And because of that, there are many challenges that some major companies Don't really have At first it's lack of time because you have always deadline. Yeah, you have meeting you with your investor or you have It's Yeah, about it later to a lot of time that causes you are missing many of basic Information and many of basic approaches that should be done Actually documentation that's something I have seen eight years ago last time and Till that time I worked for three different startups and there was no documentation So in that case for example, we need to find a way to make documentation as a side effect Of something there is no time for for documentation the same with tests You need to focus on one area to speed it up or use the tool that should allow you to speed it up And not enough testers actually when I joined there was always one person and about hundred developers And then at first there was always my Like really I was really proud of that person Wow, just revived with all those people and just just doing testing now because no one wanted to do it and it's like, you know Changing mind with 100 people who have never tested there's one person if one person can click it Why do I need someone else and it's it's it's never ending fight Now I'm fighting with that again. So we have two people Then even driven development. I Don't know if there is such name, but I call it even driven Which means you provide it you want to provide some functionalities to write to the event See, yes, it's it's I think the more the biggest one technological one in us Or to investor because in their stories waiting if he or she or the company wants to spend money And the last thing very very very chaotic environment and frequently changing environment because Customers the king in that case without customer, which usually is investor We can do nothing because we have no money to survive We have no business model. We are not the company who can earn money Like big enough or even at all to cover our employees So that's how we have to work those are rules we need to follow now I I You have a pretty small but Basic picture how the industry really works and I think even if the if in IOT will be more popular And instead of like 5% there will be 50% who can explain the approach to work at that companies won't change and For now, let's focus on one simple scenario because I'm actually does a use case or case study We need to study something Let's imagine scenario and we have two bulbs one switch and the scenario is like that We create network with them and at first we want to use switch and then mobile app already servers Yeah, it's mobile app. So I click and my bulb is turned on or off and then I do the same using switch Because from one side we have customer and hardware endpoint And then we have hardware hardware, which is like the real IOT communicating devices communicating between each other The scenario flow So at the top you have all Steps presented before closer activities. I mean for example, we talk about Selenium We talk about opium is closing drivers is clean environment is Disabling camera, which I will show later because we need to capture it means somehow and so on before we have flashing and flashing means The same as up installation just for hardware. So we need to somehow deliver the software Then we have a configuration and finally we can execute our test suite in that case is just one Once with one test case and from that flow still it's pretty fine. It's pretty simple We have three steps and then we have some pre activities Then that might be simple. Well, when you start thinking about the whole concept What is below it became more and more Challenging and that's my story from the very beginning when I started Now the only difference is I realized that's challenging And still a lot of problems and I know I won't solve them very quick At that time. I didn't even know that So what are the challenges just just the main ones to not get too deep into details at first? We need to somehow deliver that So that software to hardware. So something we called firmware for application We have mobile app, right and we inject that application. That's that's fine for For web we just deploy right if something went wrong in a couple of seconds We can river even for desktop application is fine Because we just install and if something goes wrong in worst case We ask the customer to uninstall and install again And that's my case from like two weeks ago with another IOT company for hardware if something goes wrong. Well That's pain because you cannot do anything Because you have no access you want we want to make our devices wireless So how to connect to them then how to test in that case The second thing how we want to how we can control our buttons So the scenario is using mobile application as a one Action, but we have switch which is the second action For mobile app probably a lot of you would solve the problem, but like we just trade test automation We can use APM or any other tool just to communicate But later we do the same just using hardware how to navigate How to check if there if our bulk really changed the color again if we have mobile app we can Find the element and check the status if it's on if it's off if the color is proper but the status in mobile app doesn't have to be the same as the real behavior and How to combine all those results and much more into a single one how to process them and how to create our Single point single logic to aggregate and then say if it's working or not and Of course much more. I just put four to not Discarge you to try later because that's not the purpose of this speech And when I think about areas that should be covered Like probably half of them are known for you But the second half is something you would have to invest you'd have to learn on your own mostly connected with electronics and I Like my knowledge about electronics was I think more or less the same like yours about IOT now So not very high. I was just ambitious. Maybe too much So later why and why why why that happened? But really if you will talk about automation and just before and there was a speech about coach potato So there was about Raspberry Pi and that's perfect to Control because we have physical input and outputs not just software but otherwise we do exactly the same Okay, so my impression was that must be easy at first. There is no water away This is what I as a customer and guy joining the company see just both and mobile app click turned on click turned off Okay, that's fine, but then start it like breaking down into smaller pieces. Well, it's not really like that However, the most important thing which I learned and now doing Fourth time in a row is to start with the good approach because you will have 100 people who have never tested or Were forced to invest into quality There is no chance you can do it on your own because you do not have to enough capacity. So at first Try to find the common methodology Well, usually it's one product But different areas can be mobile app. It can be hardware team firmware team. It can be Front and back end But probably all of them follow different Strategies and then for us for you as a testers or DevOps It's very hard to capture. It's very hard to catch catch with them and be on the same page Then facilitate process. So that sounds like definition of scrambler there, but it's not it's more about I would say a guy leadership Which is not really about IOTR, but unfortunately, that's the usual small company If not small, then surely dynamic one. So you need to invest your time in facilitating processes Then automation tools freedom I suppose there is never Good enough automation At the beginning even if there is there are many many other things to start and it shouldn't be you shouldn't be forced to do it You should be able to investigate to find the time to make mistakes And just learn on them. Of course, if you do the same mistake twice or three times then me as a CEO with fire you but If you have just one mistake That's perfect. So Personally, I always I always encourage my team members to make mistakes because then we have quick feedback that Okay, this approach won't work. Let's try another one. We can quickly react Then comes consider architecture which is like obvious for developers Or at least it should be but very often it's not brainstorming session as a Approach as a tool to generate ideas Because my assumption is that most of the people have know A lot of experience working because it's pretty new Pretty New area and team autonomy Just to feel free to make those mistakes and then learn from them Now that's a big picture of automation. So we have something I got called master scheduler. It can be whatever and What you should remember from that As you see, there is a cloud at the top and on the left. So that's how we want to follow Put into the cloud everything because it simplifies it's cheaper and it's easier to maintain The problem with IOT is that you cannot put Hardware device into cloud. I mean you can buy that space Rocket, but it's not that cloud We want to send our software or hardware So it must be stored at the company somewhere. We must have something like laboratory and then if our communication protocol Is not wi-fi only we need to have mobile devices at our laboratory as well So farm devices of farm of mobile devices Well, not really if you want to use bluetooth and our hardware is at the company Then our mobile phone have to be at the company as well So those two areas unfortunately and Whatever Will be done like the world Evolving like more intelligence more artificial intelligence machine learning and so on You just cannot send hardware to To amazon and say that say them those guys Did you turn on the device and another mobile phone to be in range of bluetooth? Well, that's that's not the best approach So that's really work to remember any IOT devices Must be stored locally and that's how pipeline Simplified pipeline can work so We have two places when it can be triggered. So the earliest one Is something I call I called core build core is just It's a firmware core is the basic logic on our On our hardware why it's separate? Because then we have functional logic. So we can say to our led dial. Hey, just turn on And that's part of core, but then we have when it has to be turned on Well, for example, when the connection is lost with the internet The led should be red And so on so we have functions and we have core one is very low level the second one is On a much higher level of abstractions and it shouldn't be combined. It's like, I don't know if you are For lower or lower of that I'm talking about full stack approach It's really nice, but then you are missing the expert. So it's good to have someone that's connecting everything But for me having only full stack developers of the company is very risky And the same it's it's here. Then we have hardware devices connected somewhere at our company and it Triggered either by core or functional Pull request to our repository and then we run test and to run the build But at the same time we have mobile application if there is something to change in mobile application the same we're going to install app and execute not just mobile app Testing but the most important is integration with our hardware Because they communicate exactly. They must communicate at the same level That's why I said about internet of things are to Many hardware devices communicating between each others If one is changed the second must be able to understood it And the same if we change in hardware and then finally We can have some data parsing. We can have higher level of testing We can have integration or end to end or both of them. You just cannot skip Combining hardware with mobile app and it depends on time. It depends on the context But at least one of them integration that not end to end must be executed And then we have like normal process. We have some data that should be parsed We can come in we can connect with Our reporting system It can be kibana it can be uh Grafana, it's just the tool and connect it with our project management tool And as far as I know like at least 80 percent of uh companies are using Jira For some reason so it's it's global Now the question that uh was on our side when starting was if it's worth to automate well Considering everything that was before Is definitely worth I would say even it's must Because still um, I don't know if if if you have heard something like that But in Poland we say that every good specialist can be exchanged by finite number of students uh That's not the best approach and For us it's super funny, right because you can just bring students but It won't work. You can you don't want to make repetitive work like just turning on bottom. It's not the way automation At least for the major scenarios here is is the is the key um Now what are the requirements for our environment? Well at first must be cheap because we are start-up We are technological companies and we are waiting for looking for investors Um, then simplicity to do it fast. We have to do it simple So follow keys. Do you know keys rule from? Yeah from from from development side. It's exactly the same Then modularity. So we have three steps Let's try to Separate them into different areas and about it in a minute Scalability because there is a like we have 25 millions in five years So we need to scale it performance is a very important issue here and reliability We don't want to Have flight test like anywhere Now we have quick introduction done. Let's try to prototype. Let's try to write something and let's try to automate our test Um, let's model flashing and we have again raspberry pi as during the last speech and the clue here is We have one part at the in the corner Which is everything connected. We have debugger just to flash to send our firmware and we have our um Instance Somehow communicating with our hardware Uh, probably with cloud as well to pull the firmware to store it somewhere and then just push Firmware to the to the real device um One note here because I want to present you some thickened trips as well If you have device connected to 230 or 110 Let's remember as physics you don't want to lose your notebook our Fingers in worst case So try to isolate it and it can be very costful. That's the the lesson from from my history Remember about it now Switch control I said it's easy, but I was too ambitious That's something I did after two weeks Without any knowledge about electronics. They're actually learning everything from the scratch actually it worked and it was Real fun to do it. Uh, what was wrong? Well, then I discovered I can order something like that from ali express paying like three dollars Exactly the same more consistent more reliable because no wires So that's that's that's my uh another tip If you want to solve something just try probably there is something similar on the market and you can customize to your purpose Uh regarding software We have inheritance of the the following class. That's the best approach and that's for most of the hardware devices So we have bits and bytes Setting some some outputs and that for belt. For example, it can be turn off and turn on it's binary one Binary zero then We have releasing time so we can deem it can be lighter or darker And that's another layer of of abstraction and then having that we can easily control our logic For example, if we want to make factory reset There is some special sequence to not allow the customer to do it by accident and that's another layer What's important because it seems obvious. Well, we still operate on hardware device And at first we have gpio which are physical outputs. So we need to remember that we are somehow bounded by by physics And the third thing which is vision system. We want to capture the real behavior So we have both we want to know if it's red or green and so on and we have mobile app to turn on Turn off actually changing the state of our hardware device That's dirty cheap approach And Do you know what is gtpr? Is general protection? Yeah, exactly Unfortunately, we have it. So I'm not allowed to probably personal data So this is gtpr improved Now for us the main goal was to make it simple to implement and again make it scalable Make scalability very easy to maintain So the idea was to prepare matrix with devices inside and then just split it into Many parts This is another Cheap approach. I think there is logitech Internet camera somewhere because we didn't need really Something better Then hood and then we have the image Actually, why there is the black hood because we want to have unchanged environment if there would be like sunny day Or clouded day. We would have very flaky tests. We need to isolate our environment. That's just like it's like docker for software We want to make our isolated environment And That's another Trap I made that was my expectation So I trigger signal I gather matrix value of matrix for example If I have nine of them and then I get an output Well simple as that. Well in reality that was more like that It's image processing. So have you ever worked with image processing? any of you Yeah, so it can be painful like really painful And that was very helpful Surprisingly for me later because one of the the products except for like laser for cuts That was treat launcher. So you had like pet cookies and it was thrown into the air like to let your pet be really happy about being fit And we put just camera And then checking if trees were really launched. What is the vector? What is the speed and so on and it was still for testing So that knowledge Well, for some reason Really surprisingly was Useful for me later. And that's why I mentioned here because you never know one one tip here just be careful about parameters for the camera what I found out if you have physical device if you want to As we do have separate the environment Like docker for example container when you create the new one everything is reset it somehow or you can just Disconnect and connect again your camera everything will be changed again So just be careful. It's very often the reason of flakiness Now we have implemented our free items and somewhere in parallel, of course, we have mobile up testing Which I didn't mention here because it's probably the Something that most of you know But we cannot forget about it about the customer point of view So it's brainstorming session is the output from brainstorming session Now what is important? Well by thinking about this simple scenario just turn off and turn on We were able to by using those small components like vision system like Switch controller and so on created much more complex use cases like really from the customer perspective Okay, I want to turn on but then my wife came and said I want to have it pink because that's my Internal feng shui and then someone came and said I want to sleep and just turn it off Like scenario turn it on using switch and turn it off using mobile device. It's not real scenario. It's just to check Simple functionality, but again having them it's much easier to create another And that's Road view of the implementation. So we have four small blocks just to Like lego bricks to join them together Three of them I mentioned and mobile app with some capabilities That you know probably what are covered capabilities or you surely know you are at apium conference, right? Now let's create scenarios now. Who of you is familiar with behavior driven development approach? Okay, it's given when then right Well, I was never the lover of that For very long now it changed. So like two days ago in inclusion in romania I did bgd workshop because like I changed my mind slightly but here The very good approach if you make it smart is that using Just more tdd approach You can easily write it in a way that are that is easy understandable to understand what's going on here in test Well, we just make factor reset to make everything clear like clear environment clear setup Wake up our switch then add device and check if it's a bit Simple, right? I joined the company. I'm the new employee I have now one who can explain me what's going on here But I understand because I can easily read it The same here we can reset bluetooth model Then we can wake up our switch then add device find it And make factor reset again And the last one is just sharing our setup using Gmail or google drive and here is everything clear now. What happens if we have more something like that? Well, probably you have to spend some time reading it and there is No 100% of certainty that you will understand it and here What can help you? Is bdd now when you read it, I will give you like 30 seconds. It's much more understandable. What's going on here? We have some associations We have some channels on the switch and we have some real behavior of our bulb and reading that that is really simple What is what is nice in terms of coding moving from the previous step Like having this to this approach, which is much more readable Is not it is not very costful You have all the steps prepared because you made steps here You just add another layer of abstraction. So I suggest if you have something complex It doesn't have to be for everything but for things that are hardly understandable You can use another approach bdd is not the only one, but that's what I found out very helpful Now Let's set up our environment set up the final one So in the middle, we have something combining all those results like input from Bob's Controlling switch and so on then we have mobile apps here. We have cloud Sharing both firmware and mobile app and then we have Like further steps with our pipeline like continuous delivery approach So communicating with another environment like builders deploying Maybe maybe sending some data to JIRA and so on but Under the line with Tesla result is the physical environment. So we have phones. We have raspberry pies. We have switches And that's the output that's kind of dashboard to make it more colorful because Jenkins is pretty rush rough and like very hard to Encourage product owners or product people to look at it If you have more colors With your office in the background then okay, I can look at it and all it is red and then finally make some reaction So we have that automation we have one scenario that we started Thinking about at the beginning and as a result we get the environment that can be easily scaled too much More complicated from one side and much closer to the real user behavior Well take a ways I have four minutes for take a ways. That's great. The first one think at but think big But at small and that's how we started. So we have Just iot environment some lighting industry, but then when you start breaking it down You think in smaller parts How should I implement my vision environment? How should I control my switch? How should I handle installing firmware? And so on so that's what I what what I mean by think small By think smear by think big but act small now One of the first sentences I said today Encourage your people to make mistakes Without that you are not able to deliver solution quickly just Remember to not repeat it again the same the same thing especially in terms of 230 volts repeating it is like 5k dollars Approximately from my experience When everything explodes But it's different story. It's more fun What it depends who are you talking to if it's your boss or cco? And it's not fun when you talk about it like during the conference. This is more fun Unleashed in creativity Brings on obsession for example, this is just one of the mechanism. You can find the answers There are like hundreds of ideas find two or three of them useful. That's enough. You can speed up Focus on interoperability Now we have android and is like almost 5050 We have mac and we have windows Maybe not 5050 but still we have quite Equally distributed think about it in terms of Implementation of test environment. We are not focused on For example, google chrome only modularity You have your lego bricks Think about scenario and just join them build your tower Oh, actually tower is not really stable. So I don't know build your tank has better protection Risk-based testing who's familiar with that? Okay. I really encourage you to Get deeper into that because what's important and it's not for iot only is for my current company as well if you have hundreds of scenarios You want to somehow choose what is worth to automate because actually you can automate everything But is it worth? Well, not really. It's good to find what is the the balance between the risk and the value we can achieve For iot is even more important because we have not three levels of testing But five or six of them So it's not just for iot and I encourage you to check it Maybe it will be useful for you and if not then you have knowledge during Maybe it will be useful doing another interviews. Who knows physics iot is about physics. You just need to remember. Oh, who likes physics? Wow, that's good. So that's for you Now I like before I I didn't really Just have it keeping in mind Then iot is hardware. We're just physics as well. So there are slightly other risk. There are slightly other factors uh, you just need to know And the most important thing Security and I said nothing about it. Why because it's the topic for another couple of hours I have no time to give some examples unfortunately, but I encourage you to just google security iot and then you will get a lot of nice examples what went wrong And that's it. Thank you very much. If you have any questions, just reach me here because I have applied in like four hours So, yeah better catch me online. Thanks We would not have time for questions. We'll have to