 OK, I'd like to introduce Durgall Matthews to you. Durgall is currently one of our PyCarn UK guys. In two months' time, that may be different because he comes from Scotland. And in contrast to Germany, which has unified, we seem to be possibly deconstructing in the UK. So after that, we may call it the disunited kingdom. Anyway, carry on. Thanks, John. Cool. Yeah, so I'm a PyCarn Easter and I'm a skier. That's like two of my favourite things. And as John said, I'm Durgall. I live in Glasgow. I am a developer at Red Hat, where I work on OpenStack sort of day-to-day. And that's my Twitter. It's Durgall with a zero rather than an O. But yeah, and that's my GitHub and various other things as well. But this project is nothing to do with work. This is obviously like a side project of mine, something I've been working on for around six months on and off and that kind of thing. So just to talk about home automation, I'll first define what it is about home automation that interests me, the things that I'm aiming to do. And I'd like to split it into three different categories. So the first one's monitoring. It's like what is happening now in my house? What is the temperature? How much electricity am I using right now? The next one is historical data. So this is sort of collecting this over time and hopefully identifying trends and learning something from it and then adapting. This one will take a lot longer to pay off. And I probably don't have enough data yet because seasons change things so much and how you sort of behave. And then the last one's pretty obvious, it's controlling. So it's like I want to turn the lights on, I want to turn something off. I want to sort of turn off power sockets overnight because I'm not going to be using the TV like between 2 a.m. and 6 a.m. or something ever. And you can just stop things using a bit of power on standby. So that's the kind of the things I've been doing. So yeah, then for my setup, the sort of the brains of operation at the moment is a Raspberry Pi. I've actually bought a big old own Black that I'm going to move to just for a bit more speed essentially. And then the sort of the core communication of everything is using a device from a company called RFX-COM which is a TRX RFX radio transceiver. So I use that for sending or receiving and sort of communicating between all these devices. It's a radio device and it works on 4 through 3 megahertz. And one of the things that surprised me about getting into this is how many companies make devices which work on this frequency. And that's been quite nice because I kind of figured I'd have to support a bunch of other things but so far I've managed to avoid it. I might look into Bluetooth low-energy because it's quite neat but for now it's not really on my radar fully. So just to sort of quickly go through the devices and let you see what they are. This is what the actual radio thing looks like. It's not particularly attractive but it works well. And it's a serial device that works over USB. So you can just see the micro USB input there. And yeah, it works well but the only thing I have issues is I live in like a really old house from like 1900 or sorry 1920s I should say. And some of the walls really block the signals but otherwise it's really good. Then for electricity I use, there's a company called Al and they make this device which is the LCM 160 I think. And basically there's a little clip on the left that you can attach to the cable going to your electricity meter. And then every minute it sends out a packet which has got like the current watts and the total watts. And then you can just pick this up and I'll show you that in a minute. And you can get a display with it. It's kind of it's more expensive you buy it with the display side. If you're actually quite serious about doing some of this stuff I'd recommend not getting it. I got it originally because I wasn't sure if I'd get bored of this project. So I wanted to at least be able to see the data somewhere because otherwise it's useful just having radio waves. Then temperature humidity is again pretty boring in terms of looking at the device. And it updates every minute again sort of roughly and it just spits out the temperature and the humidity of wherever you have left it. I've got these kind of scattered around my house so you can kind of start to see what's going on in different rooms. And then sockets and light switches from a company called Lightwave RF. These are just remotely controllable so you can turn them on and off. The one thing I should have made a bit clearer. So the electricity readings and the temperature and humidity they are transmit only. They basically send a message and assume somebody is going to receive it. With the switches and sockets they are received only. So you kind of have to send something and assume they have had you. And the way it works is you put them into a learning mode and you send a command with an ID. They remember that ID and then if you send a game with that ID they will listen to you. So to kind of give you an idea of how the data looks like this is an example packet coming from one of the electricity devices. I'm kind of going through this stuff fairly quickly because it's I don't know it's not that interesting but I didn't want anyone to have like not really understand what I was using at all or like how it worked underneath. So that's the kind of takeaway from this bit. So this is it's an 18-byte packet and there's sort of a similar standard for all of them but they vary after the first four bytes. So the first byte the X11 tells you that it's a 17-byte packet so it's the length of the packet but excluding the first byte. And then 5a is the so it's the hex for the sort of the family of devices so I know it's an electricity monitor and then 01 tells me that it's that specific L and then after that it can vary depending on the first three sort of things. So you've got to really look into the spec and that's quite hard to get your hands on. The guy who makes the RFX column you can get the sort of the SDK but you've got to sign an NDA for it so you can't actually give out the SDK to anyone else. His reason being is he doesn't want anyone to ask them questions it isn't a real programmer. He just does not want to support this thing basically to anyone else. Yeah and that at the bottom there that then just shows the actual extracted data from that packet so there's the ID, the total watts and the current watts. That's basically all you get out from that. There's other things like battery level but yeah I mean that's the interesting data. And with a system like this we basically the whole thing is like an event stream there's just stuff happening all the time and you just have to sort of react based on what's going on. So some of these are like incoming and some are outgoing so the lights is like an outgoing thing. I'm sending that message to say I want to turn on a light then I get readings from a room sort of a month in one room and so on and then an electricity reading and it just continues over and over and over. So that's kind of what drew me to something like async.io. Well actually I kind of wanted an event loop. I wanted something where I could react on events and it seemed like quite an obvious match to me but then I wanted to play with new stuff so that's what actually brought me to async.io and now we'll sort of move on to that part of the talk. This is how the Plec and Documentation describes async.io. I personally don't find that a very useful description. It's kind of unless you are already familiar with a lot of the internal terminology it's really it's kind of confusing. Daniel poked on a talk earlier this week and he went into a lot of more detail in sort of event loops. It was actually the title is a G event talk but he went into a lot more detail about event loops and how sort of async.io works. So if you want to really understand that in a lot more depth you should watch that one. Check out there's some good talks by Guido on async.io as well. It's fairly new so there's not a huge amount out there but there are some good videos. But basically I sort of the really dumbed down version of what an event loop is is that it dispatches, sorry, it waits for and dispatches events and then calls an appropriate handler for that event. Oh yeah and just sort of before I move on to this because one of the things about async.io is it's Python 3. It can have a little show of hands of who's using Python 3 at the moment. Sort of in work or anything like that. Okay, it's a small number and keep your hand up if you have used async.io. Alright, so that's probably about half the Python 3 people so you're all in at least 3.3 then so that's good. Right, yeah so let's look on some code. This first example here is kind of a dumbed down example. It's just Python, right? There's nothing non-standard here. This isn't async.io yet at all. One of the things that Guido is sort of pushing for is that the async.io code should just read like normal Python when you strip everything out. So I kind of proved this fact by going to the documentation, taking one of the async.io examples, removed all the async.io special bits and now just works like a normal code. So basically what we're doing here is we've got a function which calls out to an expensive compute function and then we would get a result from that. So just to turn that into like a non-blocking operation, you just need to do a few minor adjustments. So you'll see here that we put on a decorator on each function and then when calling the expensive, sorry, when calling compute and when calling the sleep, we use the yield from. So basically when that happens, we suspend the code routine. So we'll go into print, we'll get to the results and when it yields to compute, that will then be suspended and then we go into the compute function and then it is suspended when it gets to sleep and then it kind of moves up and down the stack like that. So I'll just quickly skip back again so you can kind of see the difference because it's really, it's fairly small there. They're really quite similar apart from the decorator, the yield from and then there's a little bit down at the bottom which kicks off the event loop, which is kind of ugly, I think, but you only really do it typically once per app. You can use multiple event loops, but that's a fairly advanced feature. So to move on to something that's a little bit more relevant to what I've been doing, I never actually mentioned this device because it's really quite simple stuff, but I've got like a webcam which you can just access over the network so you can just make a request to a URL, grab the image and then store it. And what I was trying to do is I wanted a recent image that I would just store and have it in a convenient place so I update every five seconds and I just have one. So when I want to view the live stream, that's fine, but I don't want to load that on every page. I want a nice recent image just to show in someone browsing the website which I'll show later as well. Yeah, so basically I'm using here, I'm using this async.io. There's another library called AIO-http which is like async.io requests clone essentially, and then there's an async.io redis library, which again is just like another library written for async.io to interact with redis. And this is actually one of the sort of the pain points that I find when working with async.io is you suddenly have to start looking for libraries which support the protocols and the way that it's working. So you've got to kind of rethink things and it's like you're suddenly back at a really small standard library, or sorry, not standard library, a really small ecosystem of things you can use that will be non-blocking. But again, if you read this function, you will find that it's really fairly straightforward to read. So if you ignore the coroutine decorator and the yield from, it's a simple procedural code which would otherwise work as normal. But instead it's got, it's yielding to creating the connection, it's yielding to downloading the image and then yielding to the redis so it's all non-blocking. And then it just repeats every five seconds, so I'm just pulling down an image every five seconds that I've got stored in redis, which is convenient. Although I actually stopped using this approach, this is something I was just messing around with, but I thought it demonstrated quite well there's a bunch of different libraries that are useful. The main reason I was trying to do it is I didn't want to write to an SD card on a rush-by-py every five seconds and eventually burn the thing out. Okay, so Python RFH com is an ASIN-Coreo-based library for working with the radio device, so this is something that I've been writing. There are other Python libraries for working with the RFH com, but there are no other ASIN-Coreo libraries. There's one that uses Twisted, and they've actually done a reasonable abstraction for the transport protocol, so I did think about just writing a different sort of transport for it, but some of the code choices I kind of objected to, so I decided to write my own and sort of a reach-quit when it started suppressing errors. But since I wrote this myself, it's fairly new, and it's currently limited to devices I support, but it is working fairly well, and I'm adding things when people are sort of helping me out with testing. So the problem I found is when I got into this, I was new to ASIN-Coreo, and I walked straight away into the callback trap. It's kind of like the very simple model, and this gets kind of ugly easily. In this particular app, it's really fairly simple, so I'm basically setting up the transport, I'm passing it the serial device I want to read, and the event loop I'm running on, and then the callback I want to be called. It works fine, but it just doesn't really feel very elegant, and there's sort of a bunch of pitfalls with callbacks, and the biggest one really is it breaks out of your standard Python development flow. I'm still working on a better solution for this, and if there's anyone that's like familiar with ASIN-Coreo, then I would love some help. The problem is some of the APIs are restricted to callbacks only, so I use the low-level function for calling the file descriptors, and that expects a callback, so I need to somehow go from... I want this callback when I can read the serial device, but then I want to move to this of the generators and futures, and I found that a bit tricky so far. So this is showing another example. This is something that I've actually found quite useful with the yield from, is you can be more explicit about how you... When you've yielded to the event loop, it allows you to do a combination of blocking and non-blocking operations. So this is actually kind of pseudo-Python, which is just copied and then reduced down a bit from the code, because when I'm doing the setup of the radio device, I need to reset it, and then I've got to wait a period of time and then send a status packet to check that it's ready. And if you get the timing of this wrong, it just doesn't work right, so I wanted to be more blocking in the way I was doing it, but then afterwards I wanted to not be blocking, so it kind of switches to then using the yield from where I set the mode, which is how I enable and disable different protocols. And I'm not sure how you do something like this in CG event when you want to have a mixture of things when you've got the monkey patching happening. So for the... That's kind of like the data collection and how I've been working with that kind of thing and with the actual physical devices and then presenting the results on the dashboard that I've written. So it's a fairly simple web app, and it's kind of a side to the bits I find interesting in the talk, so I'm kind of skip over this fairly quickly. It uses Flask Redis, SQL, Combo, Bootstrap, and basically it's just a way of viewing this data, sort of graphing it, so I've got a sort of a simple time series, which I implemented, although that turned out to be a big mistake, I have to say, to write your own. And all the code for this is on my GitHub, so if anyone actually wants to dive in or try and use it, they can do the code. It's horrific. This has been kind of like the dashboard. It's kind of been like my play area, and I'm slowly spinning stuff off to actually make more reusable stuff which is tested and not just terrible. Yep. Would you like to say that Flask Web App is not using Asyncio? No, yeah, that's kind of why I'm glazing over it. It's just, yeah, it doesn't, just for the mic, it doesn't use Flask at all, which is the... Sorry, it doesn't use Asyncio in the Flask Web App. Yeah, so this is kind of the results of what I'm seeing, so this is just like the default view which shows you the 24-hour period of what's going on in my house. I kind of cut it out to remove some of the more sensitive information, but you can get the idea. So that's like the electricity uses aggregated over the day and at the top, and then that's the current readings at the bottom, so you can kind of see things happening like when an oven's been turned or you're boiling a kettle, you get huge spikes. And the one that I try and get down as low as possible is at the beginning, there's this sort of idle period where we're sleeping or we're not in the house, and that's really when you're wasting a lot of energy, a lot of the time, just because there's stuff in you, sort of the things are happening and you're not there to take advantage of it. And then just to show another graph, we've got the... This is humidity and temperature. It was in my lounge, it was like when we had a really hot period, we do have them occasionally in Glasgow. And as the afternoon comes round, the sun goes right into the living room and it just looks like you see the temperature rock up. Okay, so to sort of move into some of the lessons I've been learning, how we're doing this project, I'm finding ACNCO is pretty nice to work with, but it does feel quite at a low level. This is one of the actual points of it. The idea is to have further abstractions added on top. So I'm hoping that, say, Twisted is probably going to support it and that might aid their port to Python 3. I don't have a clue if they're actually doing that, but I'd be surprised if it's not on the cards at some point. The ecosystem is young, so at the moment it's quite tricky doing things. There's not a huge amount of examples or documentation out there. So when you're learning about ACNCO, it's kind of tricky. And most people that are building services or they're doing, I guess, network traffic and things, there's not really anyone else that I've found doing serial devices or messing around with these particular APIs. So if you go looking, for example, some GitHub, I can find how to scrape websites, that's fine. But yeah, doing something like this doesn't really that popular with Python 3.4, right? Yeah, and the other sort of the key thing is I should have used Graphite for my data collection. And writing your own Redis time series is actually fairly easy, but it's just a bit of a pain in the ass if you want to maintain it. And yeah, the reason I didn't, I was just being really lazy about actually getting set up and I just wanted something running. So I started just inserting data into Postgres and it's kind of slowly moved. But Redis on a Raspberry Pi is actually still amazingly slow. Okay. I think I've kind of touched on some of the other things, but the co-routines of futures is something that... When you're working with them, they feel a lot nicer to work with. So there were some in my examples, I probably didn't call them out that well, the difference between the two. But the problem with, as I was saying with the APIs, is it's hard to know how you can use callbacks in futures when the documentation is explicitly just defining only callbacks. So the kind of confusion between the two is a bit tricky. And that really is about everything for me. So I went through that a little quick. But yeah, we'll have you take some questions if anyone has some. Have we any questions? Okay. Just a quick question. What sort of money is involved there in these sensors? Oh, it's an expensive hobby. 100,000... I've spent a few hundred, I guess. I could actually go and find... I can give you links to all the devices you want to find them there. They're pretty easy to buy on Amazon. It's tricky knowing what is compatible, which is partly why I wanted to list everything at the beginning. Yeah, it does get quite expensive, because you're kind of like, oh, I could just buy another one of these. But yeah, it's an expensive hobby. So I'm definitely not saving money by cutting down my electricity usage yet. Okay, I have a question as well. Are there any sit-will devices for monitoring gas consumption? So this is kind of odd, because there's an area blocked out on the speck for it, and it's actually to find how it will work, but I can't find anyone selling them. So I've been trying to think of creative ways to do this. The only option I've thought of is a webcam and OCR-ing the number, but that seems a bit crazy. Hi, Diggle. So, I'm here. So you've wired up your house, your plugs, that you can ping, and... What about the security of all this? So I could drive up by your house and switch everything off, perhaps? In theory, you could do some of that. So the range isn't that great on the devices. Yeah, so if you sit outside and you listen to the packets I was sending, and then you just need to take the IDs, and then you can turn things on and off, you wouldn't know what you were turning on and off, necessarily. Well, that'd be kind of fun to watch the lights switch on and off, wouldn't it? One of my friends is doing a similar thing, but he's been doing it with Node.js and stuff, and I was considering going out and trying to troll him, just like setting up a Raspberry Pi with a battery pack and things, take the car out next to his house and see. Okay, and given that this talk is being live-streamed on the internet, it will be available on YouTube. You don't have any concerns about the... I don't think I included my address, but... Good point, good point. Thank you very much. What about higher-level web frameworks? And I think, do you know if anybody's doing any work on doing that sort of thing in Python at the moment, and if there's anything that's particularly difficult about that? I don't really know, to be honest. I did spot, I think there's somebody who's done a port of Flask, but it just seemed like I already feel like I've been learning a bunch. There's a web developer, all these kind of bikes and stuff is a bit odd to me, so I feel like I've been learning a lot, and then I started looking at Async.io web framework options, and I was like, I don't have time for this, so I just went with the Flask approach, which seemed simple enough. But yeah, I've actually been kind of regretting that in a way, because I then ended up writing my own REST framework on top of Flask, right? So ideally what I want is an Async.io REST framework, because I can just do the UI all in JavaScript, that would be much nicer. Hey, you said something about a gas meter reading just now, about an idea of looking at it with a webcam. I don't know if your gas meter is also from the 20s, then you're probably out of luck, but usually these meters have an infrared beep for every cubic meter, or so you can look into that. Cool, I'll take a look at it. The one extra ratio I have is my gas meter is outside, so it's like weather considerations, but... RF, right? Yeah, well, I'll have a look. Thanks. Do you have any support for motion sensors? Yeah, so the company that makes the switches and the sockets, they've also got... Actually, loads of people make them, but I've got some from the same company, the PIR sensors. I'm not doing a huge amount with them yet, so that's still kind of work in progress, but... Yeah, so the one thing I've got is in the hallway, I'm turning on the light when it's dark, and so if you're going to the lewitt night time, the lights turn on for you, that's my girlfriend's my favourite feature, I think. Thank you. Anyone else? Oh, well, thank you to all that's been interesting. Can you tell me if we've got another one? Sorry. Hi, let's not make a question. If you would have chose Python 2.7, which technologies would you have choice, for example, for the library? Yeah, so if I'd done it in 2.7, I would probably have been twisted, simply because they have got serial support included in the twisted standard library, I don't know what they call it, so that would have been a lot easier. I ended up having to kind of figure out a bunch of stuff there, which I wasn't too familiar with. So, yeah, twisted would have been a good choice, I think. Well, you've certainly provoked some interest there. I expect all the euro and pythonesis will have their houses automated next year. If anyone else wants to try my code out, as I say, it's all on GitHub, so I'd be happy to help anyone, because it is a good one. Thank you, and 12.30 at 7.30.