 Hello, hello and welcome to this talk. It's called the open source automation struggle between comfort, openness and usability. I will have a first, I will have a few question on you. Does any of you have been in my talk last day of going first? Nobody? Amazing. Do you, are you familiar with Home Assistant? What's up, what's up? Amazing. Have you ever heard about ESPIO? Have you ever heard about ESP Home? Amazing, amazing. So let's start. Thank you very much for coming. So who am I? My name is Tomasz. I'm the bold bearded but barbarian from the east. I came from Slovakia and I moved to Czech Republic about 15 years ago. I considered myself a software engineering grease monkey. What does it mean? I worked as a software developer for about 15 years. Now I do operations, system administrations and software development on the side. And my motto, somebody gave me a sticker on these years for them. And the sticker said, I can ride my own zero days. Thank you very much. So this became my motto. I live by it. I can ride my own zero days. And I did it on numerous occasions. So what's the motivation doing this talk and what's the motivation even like doing home automation and smart homes the open source way? So the biggest is privacy. We all know, as I talked about the struggle, we all would like to have those devices that you just connect to the home and they work and they are safe. They do not steal your data. They are not spying at you. And the software engineers did a good job. I'm not sure how many of you are software engineers. I am and I don't trust software engineers because I make mistakes. Everybody makes mistakes. So I pinpoint one thing. This is a picture coming from a relatively recent paper which is from German and Netherlands universities and the cooperation, those two universities, Max Planck Institute and some other from Holland. I'm sorry, I don't remember the name. So what they did is the day tried by bunch of those smart connected devices and connected into the network with IPv6. You know that amazing new technology. It was invented in the year 1998, right? And nowadays, still only 30% of requests to Google are over IPv6. This is a great example. They took bunch of smart TVs created in the years 2019 till 2022. Most of those devices have implemented the standard from the year 1998, meaning they were able, they created something which is called Extended Unify, a unique identifier, which means that in the first implementation of IPv6, your MAC address was part of that address. So you buy your TV from a renowned company and you can use your internet and then you realize there is your MAC address. There is your other space. Huh, what happens? You can be tracked. You can be tracked. Amazing new technology. Nobody did anything wrong. They just forgot to read the documentation, which was released in year 2000. Just to check, it's a typo. If you're tracking ID, it doesn't match the phone call of the laptop. So, you mean here? Yes, sir, the laptop. Yes, because the laptop uses privacy extensions for IPv6. So it has randomized the part of the MAC address, but you still have on your network that one device which is identifiable. So, hey, it's the same network. It's the same guy. This TV, his computer, and here goes the advertisement, right? So this is just one of those examples. It's a really nice one because it's a relatively new one. The others, you may have read the news. Amazon got fined $30 million for privacy violations of ring cameras. And there is much more. Every year, and we will see much more every year. So that's why. I started this project about five years ago when I bought my house and decided to make it connected to everything. So this is how I started. They started with simple home assistant installation with the PostgreSQL as database and ESP home as building block for all those sensors and cameras and all connected stuff that I have. This is where I have moved. This is three years later. So it's about 30 environmental sensors, about 20% bunch of cameras. And I have added it. I have removed some stuff from the stack. I no longer use Postgres and I have added a new framework which is called Tosmota. I'm not sure how familiar are you with it. I will get back to why a little bit later. And this is basically where I got or regarding the connected devices that I have at my home. All of them except six are handmade by me, others I bought and flushed with open source software. So how did I get here? So from one Raspberry Pi, which I experimented with, to cosplaying as a sysadmin. So now I'm running two firewalls, two storage nodes, the virtualization nodes too. Everything on the double because I want to be sure that my house is always connected. And I can rely on me flipping the switch and the pump going on. Or the most important, more important thing when I flip the switch, it goes off. So I moved slowly but surely from Raspberry Pi. This is the Raspberry Pi 4, but I started with two, which was 32-bit ARM. I'm not sure how familiar are you, but 32-bit ARM was a mess. So I tried a number of boards. The performance was not there. It's fine if you are experimenting with it when you have small networks with dozen of devices. It works perfectly, nothing to fear of. But the moment I started to grow, I realized that, okay, what do I do? Let's try a new hardware. So I tried one of the first 64-bit ARM boards, which is NVIDIA Jetson TX-1, which was widely available. And around the home assistant, that suffices for some time, but then you hit the bottleneck as with the most ARM board. At that time, which was storage. It only had one set-up board, which is slow. So that's six-bit, it's very slow storage nowadays. So I moved to standard hardware, like PCs, X8664 machines, nothing fancy, just the consumer grade hardware, which I bought from the shop, put it together, deployed stuff. And over the time, my wife started complaining because you can see there are three nodes. There was another three nodes just lying on the ground around the house. So I decided to optimize and start playing as a seasoned man, so I bought my first drag, built everything in it, and it didn't work. So after a bunch of debugging and redeployment and understanding all the underlying technologies, I choose to use Trunas as my storage server, which is a really nice open source project. The scale offering from them is based on Linux, so no BSD needed. It does the replications, it does all the magic for you, and you don't have to do a thing. Podman, for running some of the infrastructure related things like databases, they do influx DB now, so running influx, running some pipeline for image recognition, that's where you can see a bunch of graphics cards there. To be honest, I have to admit, the picture is from the times when I was mining, but I still have the hardware, so why not to run something useful on it, right? So I started to play with different things. So let's get to what I started using compared to my previous solutions. In the first talk at DevConf like three years ago, I presented different set of sensors that I was using, but over the time, somebody pointed out some other sensors like the MH1, which is CO2 sensor, which is basically a replacement unit for some AC something, but you can buy it relatively cheaply, it's a relatively good CO2 sensor. Then there are a couple of Bosch sensors, which is environmental sensor and sensor for illumination, so I can tell if there is enough light in my rooms, and if I need to turn the lights automatically on, and the presence detector, which is basically a microwave radar. It uses reflection as a means of identifying movement in its surroundings. I am really surprised how precise this sensor can be. If you have the right board behind it and the right piece of software, you can make it tens of millimeters precise, which is nice, and that will be it regarding sensors. The other thing that you need, if you are, I need it, I want it to work with our displays. Sometimes you just need some information. You need to quickly see, for example, the pump is half of its cycle, and I don't use smartphones, so I don't carry phone around my home. I don't always have a notebook with me, so having nice randomly placed displays that will show you some information can be really useful. I'll just pinpoint a couple of them which are really nicely supported in the ESP home platform, and those are the two small OLED displays. They are amazing, colorful, really nice, and really easy to work with. Then there is a two of TFT displays, which are LCDs, which are LCDs, one of them you can buy as a separate module, or maybe both of them, but the seven segment one is used in some devices such as this one, which is power metering device, and they are nice. They are colorful, but they are small, and they don't have really high DPI. This is 240 on 180, I think, or something like that. So a really small number of pixels. And then I saw on the internet there's this nice IPS display that you can buy. Seven inches cost about 35 euros, and it can do high DPI, so I bought it. And then I realized you need Windows to write the UI. And I was surprised here. My experiments with the Mexican displays end up because I don't run Windows. I don't run close source applications. I have no way how to put the UI on there, but it's running ARM M3 Cortex, so there is something over there. I was able to get a serial console on it, but I wasn't able to flush it. My embedded knowledge is really limited. Even when I'm standing here talking to you, I do not know that much about embedded systems, but I know something. So don't be afraid of MCUs. This, I choose this platform as something to show you. I haven't had to do a bit of soldering to build these two things. Don't be afraid of it. You can just buy it. This is a platform from Chinese company that you can stack together in standardized connectors. You can just put it all together, write your own code, flush it on there, and play with it. It doesn't hurt, and it's really simple. The ESP home that I was mentioning is basically declarative framework, which allows you to write code without writing code. So you need your YAML files, and they will generate the code for you and create all of those bindings that you need to the libraries that you are using, or load all the drivers that you require. So they don't bite. And another thing that I have added to my smart home or connected home is a power metering. So these are those products that I don't build because working with mains power and switching mains power can be really dangerous. You can see, nice example of the danger here. It's not my picture. It just took from the internet, but I used the same box. This was error of the user. He connected wrong inputs, but I have seen similarly burnout box, one of mines, when I used this device, which is the dual R3 from company called Sonoff, and they guarantee you 2,000 watts of resistive load. So I connected 2,000 watts of induction load to see what happens. Who would expect that it burned, burned, really nicely. So this is really interesting bunch of products from the company called Sonoff, as you can see here. So what do they do? This is a line they call, do it yourself. So there is a serial console. Not sure if you can see it, probably not. There is always a serial console somewhere there. All of those boards have it. It's under the hood. Either they have just soldering pads, or they directly give you the serial console. And as we know, where is the serial console by the way? So you can basically flash whatever you want there. I will mention all of those devices. So these two up here, basically the one on the right is an older revision that the company already stopped selling. I believe it's because of the circuitry they used, and basically it wasn't very well built for the purpose that it was supposed to do. They, it's about five months. They created absolutely new version, which is called Power Free, Power Free, which is the biggest box here. And this is really interesting because they listen to us, they listen to people who use their products, and they created box which is able to switch five kilowatts of induction load, which is amazing. So what I did, that's right. And it works. I connected my three and a half kilowatt induction load on it, and now it's switching, I think it's 3,500 cycle. I just automatically switch it every day a couple of times to see how long it will survive. I will definitely update my page once it dies, because I'm looking forward to see if they have learned something. Because one of those, the Power Air 2, used a really cheap relays. And after I did this test with them with just about a couple of hundred watts, after three, five thousand cycles, somewhere around there, most of the relays get stuck. And that's not something you want, especially if you are switching some load that can be destructive. So pumps and things like that, you want to have reliable relays and reliable switches. So these parametering devices, the newest one is probably the best design also because it uses CT clamp. So you can see that the measuring interface is basically outside of that circuit, which is not true in all of the other solutions by them. In all other solutions, it's directly on the board. There is some resistor, which is used for measurement of the voltage and measurement of the current, but that can lead to fires. Doing it this way, doesn't matter what you will put through there. If the cable does not melt, you should be okay. And I just mentioned three of the sensors that are used there for those of you who are interested. They are basically one-purpose things to measure current differences and voltage differences. Go figure how and why they use those three. The last one is probably the most precise one. Those two are probably the cheapest ones. So that's about the parametering. So let's talk about how we connect those devices. So I never had any reason to do anything else, the 2.4 gigahertz wifi, because it was too expensive. I didn't see any reason for it. But a couple of years ago, there's this shop called Lidl here in Europe, where you could get one of those boxes, which is basically a base station for Zigbee with some RTL, some Realtek MIPS SOC, and some Zigbee SOC. And so, me being me, bought it for, I think, 15 euros or something like that. I opened it, which you don't find. There's a serial console in there. So I tried, I played with it a little bit, poked around it, but as I mentioned, I'm not that good with embedded systems. And there was already somebody smarter than me, named Paul Banks, I think, who already did flesh it. So I looked at his work, basically followed it, copied it, changed the configurations a little bit, sued my needs, and fleshed it back with Alpine Linux, running my software, not contacting Chinese government, data centers, and just talking to my stuff. But this device, and I bet it on the SOC, is not reliable. The network I created using this base station was clustering all the time if I connected more than 15, 20 devices. So they decided to switch, something different, so that this USB dongle, just again, from the same company as the parametering devices, and again, they put serial console over there, so you can flesh it with whatever you want. And it's Texas Instruments Radio, based on similar chips, but it's stable. Again, the price is around 20, 25 euros. I have deployed three of those devices, and network of 65 ZigBee clients, but I wasn't unable to make it work for longer than 20 minutes. Every time I try to create bigger ZigBee networks at my home, they get clustered. They always get clustered, and I don't know what to do. I don't really use ZigBee that much, I just wanted to test it, so I sold most of the stuff, and left just a couple of light bulbs, and that's it. For the other radios that I use, it's Bluetooth, so I'm not sure if you're familiar, this is PineWatch, this is BengalJS, those are open hardware smartwatches that you can buy. You can flesh whatever you want on them. They have Bluetooth LE, so they transmit a lot of information over the Bluetooth LE. You can get through it. I use it for presence detection. So if I want to know where the people are in the house, actually, exactly me and my wife, so where do we are, where do lights need to be on, when they can be turned off, but. There is no direct integration for Home Assistant, so I had to use some other means. One of them was Room Assistant. This was a really nice project I found out a couple of years ago. It's basically really, really, really, really, really simplified Home Assistant, which should be working on the level of rooms, but it has moved to be an abandoned where. What it does, it's basically, for me, it was Bluetooth to MQTT gateway, which basically just transmitted the messages from the watchies to the Home Assistant, and then I could kick in the automations and do the Home Assistant stuff there. There is a huge amount of implementation of Bluetooth to MQTT. Why? Most of them are just basically shell script, Python script around Bluetooth CTL. We just paste the commands and work with that. The difference is probably the open MQTT gateway, which is a project that has a number of integrations and it does support a number of protocols. So you can use Bluetooth, Lora, whatever you want. I have more or less dumped all of those solutions and I basically use the watchies Home Assistant with its beacons and that's it. Another part of the setup are new cameras. So probably the most prevalent that I have at home is ESP-CAM. It's basically ESP32 chip with camera on it. I built a couple of housings for them. You can see there is an AC power. There is the camera. I prefer when I put some mains powered things, I prefer to use these kind of boxes. This is electrical installation box. Why? I prefer to use them. They do not burn. So even if you try to put them on fire, they will not burn, which is amazing. But soon I realized that ESP-CAMs have huge disadvantage and that is the CCD chip that's on there. It is two, three, five megapixels, depends on which version you have, but for my use cases, which I will show you in a moment, it was not enough. So I played with Raspberry Pi, that's better, but still, if you want to take a picture of something which is more than two or three meters away, the basic Raspberry Pi camera is not really useful because you know there is a person, but that's it, nothing more. The great thing about it, about using Raspberry Pi, is the platform. You get Raspberry Pi, this is the mini, or how is it called, which is really nice, small single core with 500 megabytes of RAM. You can put Linux on there. It is nice. There is another thing that happened some time ago, again from the Pine64 company, which is the PineCube. This is basically Linux-based camera. So what they try to do is that they created a board. Problem for me, again, is the CCD. It might be enough for indoor usage, but for me, it wasn't good enough. And then we get the last thing, which is just something called IP webcam. And this is the logo of the application. You can find it on fdroid. If you don't mind your privacy, you can find it on Google Play as well. And it's basically an application that gives you beautiful interface to a camera system. You can have multiple renders. You can hook it into whatever you want. You can hook it into the automations in Home Assistant. You can hook it into different NVRs. You can do whatever you want with it, and you can reuse any old phone, which is nice. And phones have really nice cameras. So here you can see the difference. Well, this is why I mentioned that ESP cameras are not enough. The one with the blue car, it's made by smartphone. The one with the red car, it's made just by the ESP cam. And you can see that the picture is not so much clear. Maybe you have noticed there's a bunch of squares there. So there's a yellow square. I'm not sure if you can read it, but it says car there and then a percentage. How sure it is that it is a car? How sure it is that it is a car? And usually it's more than 60, so it works. As it goes for the software, I used Motion Project. It's a demon for Linux that allows you to use any camera input and detect a movement on it and take the still pictures, create a video. There is another project called Motion Eye Project, which basically encapsulates the demon into the web UI. For a while, there were releasing Motion iOS, which was basically an image that you could flash on a number of ARM-based boards and it contained minimal Linux system and the UI. So you could just then hook it into the NVR that you already have or use it as an NVR. And those boxes that you can see are created by this amazing project called Dudes 2. I forget what the acronym stands for, but it's the second version. There was one couple of years ago. It's a really nice open source project because it provides you a REST API for object identification. So you just post a picture there and you will get, you can run it locally and you post a picture there and get the result back. So I'm running it on a bunch of Nvidia cards or I was running it on a bunch of Nvidia cards. But then you realize that mining Ethereum is one thing. But when you want to process the images, you don't necessarily need a kilowatt of power to consume just to process a bunch of images, right? So I moved from experiments with graphics cards to just plain CPU work, which costs less. So the other important thing about having connected things is the way it notifies you. So this is a bunch of connected light bulbs. As I mentioned, they have a bunch of the ZigBee ones. When you ping me on IRC in our federal channels, this is what happens at my house. So I start getting notifications because I work from home. Similarly, something similar starts happening in my office. If our composites fails consequently in two days, it will start really, really blinking. And the other thing that I built, it's the wall thingy. I call it the wall thingy. It's basically the e-vase they found around the house and put it all together on one piece of board. It's a bunch of displays. One of them is currently off. It's a bunch of displays. You see, this is the Home Assistant dashboard that I use. This is my driveway so you can see when the check post arrives because they don't wait. And that's it. And that's basically it about the talk. So do you have any questions? Why do I need 15 cameras? So you can't see it, but there is a road here. And I want to have them on the photos. So that's why. I basically monitor only outside of my house. No cameras are pointing inward because I don't trust software engineers, so I don't trust myself. And only the cameras are only on outside. I want to keep track, but keep track about the life that's there because I live at the end of the village. So I track the birds that are flying there. I track the animals that are there and things like that. And those funny photos of your cats and dogs and stuff that you can get of the PanCube. Not really. I always just powered it from a really cheap power supply, something like I have here on the camera. This is really small, 700 milliampers, I think, or something like that, and it booted. So I'm not sure about the exact power consumption. You have to ask my wife. She is the marketing department. Are there any other questions? But BT Home? Sorry? Are you aware of BT Home, the new standard for low-energy messaging and like charging presence and all that? No, not really. Not really. Can you please repeat? BT Home. BT Home? I'll look into it, thank you. Any other notes? I'm open to anything. If I was wrong, I'm not correct, so just tell me. If that's it, thank you very much, yeah. Do you use some kind of scenarios in your home, like when I enter to a light on, then coming out from home to a light on, or don't? So not really regarding the house itself, but for example, my office, where the wall thing is. So now it's turned off, because I'm not at home. So yes, then some small portions of it, like for example, especially my office is probably the most automated part of the house, and then the garden, because there's a lot of work there. So yeah, if I walked out of the office, and it's after 8 PM, means I'm not longer at work, everything shuts down, and then in the morning, usually I wake up, the home assistant realized that my watches have moved to the kitchen, so I'm making a coffee and everything turns on, computer boots. The question is, how far, how to support with scenarios? If you decide to use this kind of do-it-yourself hardware, like those switches I have noticed, things like this, it's extremely easy. It's extremely easy, because you just need to identify, let's say, person, you, then you need those relays to switch the things. There you are. And home assistant is declarative, so is YAML. So we will declare in the YAML file how it should act, similarly to automation and Ansible, and it will do the things. OK, thank you very much.