 Thank you all for coming This is Sauron do-it-yourself home security using a Ruby. My name is Jonan I go by lead bot on the internet on Twitter if you aren't able to read that Twitter handle Then maybe you should look up on Wikipedia lead speak. It's an old old people joke. That's not very funny to begin with but I do just can't think of me as the guy who's really hard to find on Twitter You have to be really dedicated to want to follow me. It's just a bunch of numbers. I'm from Portland It is a beautiful city. You should come visit in the spring anytime in the spring or summer great times to visit Portland They call me webmaster. I am the webmaster for the Cedar Hills Kindergarten and preschool It's a co-op where my son works. It's pretty important gig mostly I manage mailing lists adding people to mailing lists taking people off mailings if you have any mailing lists related questions You can reach me at webmaster at Cedar Hills KP.org. I have another side gig At this company called New Relic where I work on a project called New Relic RPM Which is the Ruby agent the gem that you all installed to get New Relic instrumentation for your applications So if it's broken call someone else if you want to compliment the code come and talk to me If you're ever in Portland come and visit I would love to take you out to lunch at the carts across the street and we'll go sit up there on the 29th floor of our fancy building and look out at the skyline It's beautiful if you were up in Portland for Rails comp You probably saw our office. We had a party up there Couple years ago, but if you're in Portland, I'm gonna be upset if you don't come calling you'll have to find me on Twitter first If you are at all interested in becoming one of us and drinking the New Relic Foulade You can go to bit.ly nerd life or you can just talk to me This is a list of jobs that were open at one point and many still are but there are also other jobs I don't know. There are links on the site. I'm sure you'll find your way around But if you have any questions feel free to ask me and in particular we are looking for a Ruby agent engineer Someone to work on our team with us to Metaprogram every Rails app in the world in really important ways. So if you are interested in that kind of thing Come and talk to me Would be nice to meet you So I want to talk to you all about my knife set to start off here This is a talk about kitchen knives as it turns out. This is my kitchen knife set. It's really nice actually I'm pretty pleased with it. I have a lot of Ruby friends who are big into cooking Ryan Davis for one has a nice set of shun knives And he once bragged to me that they were he took him to the manufacturer to sharpen And they were so sharp that it took all the hair off his arm when he dragged it along his arm and several layers of skin But I prefer my kitchen knives in like a not accidentally amputate anything sort of way So I like these ones. These are good knives They've made me very happy And they're especially cheap Which is the nicest thing about these kitchen knives because my children literally eat hundred dollar bills for breakfast This is a picture of my children eating hundred dollar bills Everyone thinks I'm joking when I say that they're little monsters, right? So I love my knives. They're cheap, but I take good care of them I've got some sharpening stones my friend rain gave me one and I got another one so I can sharpen my knives and keep them in fine order and They're kind of important to me It is a nice kitchen possession to have a good set of knives and occasionally we end up in a situation like this Where the knives are left in the sink to rust in the water sometimes with filth on them And I know that I am not the one doing it right because I don't do wrong things Generally speaking and my wife of course insists that it is not her So there's only really one reasonable conclusion to be drawn and I'm sure we're all thinking the same thing It's Sauron right Sauron likes knives and home invasion So clearly Sauron is coming into my home to spread evil in the world by dirtying my knives and leaving them in the sink And I had to solve this problem, right? So I came up with the solution and I've trademarked that I'm still talking to group on about it. Their lawyers are after me, but I think it's gonna work out just fine So I'm gonna talk to you today about the solution and kind of tell the story of how I built this monster and What all went into it and show you a little bit of the architecture But let's start with the name right I decided to name it Sauron It's gonna be a gem the first first bit that lives on The device that's taking pictures is gonna be a gem called Sauron And then I went after starting my project and writing a fair bit of code I went to Ruby gems and found this squatter on my name for some definition of squatter where it means legitimate useful valuable project and They stole my name and it was really important for my do-it-yourself home security system But I got over it. I got a new name. I decided to call it all saying I Which is actually more to the point. I think right. That's what I'm building is an all-seeing eye And then I after writing some more code you think I would have learned my lesson, but no I went to Ruby gems later and someone is squatting again with a useful project that does things and Finally I settled on the perfect name which I should have thought of in the first place Which is all seeing pie duh because it's a raspberry pie I have a raspberry pie with a little camera on it and a gem that runs on the pie and takes pictures And so it's the all-seeing pie so this is Yes, slow clap for the ultimate This is a raspberry pie if anyone hasn't heard of them They're little itty-bitty computers about the size of a credit card And they're fantastic and they're super fun to hook up to all sorts of things This is actually a very simple use of a raspberry pie There are a lot more interesting things you can do with them than this But you can buy this little camera for them this is a Raspberry pie cam it plugs right in behind the ethernet port on the on the pie and then it takes pictures There's a little command line utility you can use called raspice still takes a picture and stores it to the hard drive little jpegs You can do whatever you want with it So in my case because sour on is wily and probably spot the camera right away I want to immediately get the images to the clouds, right? So when the camera takes a picture I push it into the clouds of Mordor Service provider and store store my images up in the cloud, right? And then I have another piece that's going to consume that information But the the point there is that I want the pie to do as little as possible because they're tiny computers and they aren't very fast So you want to offload as much as you can obviously in the beginning So I have this other app called Palin tier And the Palin tier app is to do all of the other things that need to happen So the pie only exists to take pictures and the all saying pie gem is to run on the pie In a simple script and just take pictures and upload them to the clouds, right? So naturally Palin tier is written in rails because we all remember the Lord of the Rings train From the Lord of the Rings movies. You guys remember this train. I'm sure Trains were a big theme in Lord of the Rings This one has an eye of Sauron on the front and a laser. I think coming out of the eye of Sauron So I had to build it in rails, right? I was pretty pretty set on rails And then I have the all saying pie, right? It's my job So I want to talk a little bit about how the all-seeing pie is put together first So it's it's pretty simple. It takes a picture from the camera again using a command line utility Stores it locally on the hard drive and then pushes it to the cloud and then deletes the local one Because I don't want to just fill up the pie. I mean, there's a reasonable amount of space You've got an SD card on there You can get a big SD card and store a bunch of images But better to just put them up in the clouds and then forget about the local versions And so I delete them as I go and I upload To the cloud and then the idea was that I was going to have this Palantir app and it was going to pull I guess the Storage the cloud storage and wait for new images, right? And pulling is kind of terrible You don't want to do that if you don't have to usually because it was a lot of resources and there are some timing issues that Can come along and like maybe I'm getting more images or images more often than I'm pulling and it became Something of a question for me in my initial architecture. So I decided That what I would do to kick off this collection of images Was that I would have the the pie the raspberry pie push the thing into a queue Right and we could have this queue here and we can push a message onto the queue and then the the app on the other end will pick up the image and Do what it needs to do right so Palantir will carry on it's life based on this kind of little eventing system That uses a queue But everyone knows the queue doesn't belong in this universe actually the Lord of the Rings application does not need any queues So I decided against the queue because it's basically kind of a bad form of a simple API that I could just build instead and it took me a while to get to that conclusion, but I decided that what I would do instead is I would I would take the picture with the camera And then I would have a little API hit that just sent the URL over to Palantir and their Palantir would know that It had received a new image And it would take over from there and my my thought was I want to keep this very simple, right? I want the the bits of Ruby that are running on the pie to do almost nothing right they upload a picture and then they send a little post with an image create over to the Palantir application so Interestingly those URLs can be kind of handy because they include the directory name Where I can go and look for the image from the other app and they include the key name for the image itself So I could from the URL store all of that information on my My Palantir side by extracting it from the URL and then using it to do the rest of the calculations that I have But again, I'm still with this this situation where Palantir is gonna have to download the images because we're gonna want to compare these images somehow, right? so Palantir is gonna need to compare local copies of those images to hash them and In order to do that. I'm gonna have to be pulling them down whether I'm gonna say be high API hit or not So the keep it simple plan fell apart and I just got it keep it everything planned instead Where I just send everything over because this is all the information that we really need and if I send it all over to the Palantir application, then I don't have to actually go and download the images at all, which is fantastic, right? so in here we have EvilCloud.co and then the image name is where the images stored and then the name is the key in the bucket or directory online Then we have this P hash thing that I'm gonna explain here and the directory itself is called doom, right? So the P hash it's a perceptual hash As if people heard of perceptual hashes by show of hands. This is the thing people know of good I'm actually really glad that I get to explain this a Perceptual hash is defined here is a a fingerprint of a multimedia file derived from various features of its content Which is correct and also more complicated than maybe I would say I would go with a number a pH is a number So It looks like this and it is a number as you can see and there's a way that we get these numbers when we take an image We turn them into numbers Using this magical P hashing algorithm and there are actually a lot of different ways to do this and do it faster or slower But this is one way that it is done. So first you take the eye of Sauron and with no regard for anything You just smash it right we smash it down to 8 pixels by 8 pixels and then we have 64 little pixels of color And we're going to use this to build up that fingerprint that number we're just talking about So we take those those 64 pixels and we turn them gray So now we only have 64 color values instead of thrice that many right and so we we end up with This grayscale image where we have 64 different shades of gray all the way across and we yes That's not the book. I wasn't referencing the book I'm told that's like the programmer version of 50 64 shades of gray Okay so This is the the image on the left the grayscale image and on the right what they've done is you take a mean value for all of These gray pixels and then if it's bigger you turn it black and if it's lower you turn it white And so you end up with this binary distribution of black and white And it still kind of looks like an eye if you look at it right that looks sort of like the eye of Sauron that we've smashed down And so we can take that then and go through the image Starting at the top left corner and just give bit values to each of those right and as we go across the image Assigning these bit values we end up with a 64-bit number Generated from this image initially and it's pretty fast right we smashed it to eight pixels We turned it grayscale and then we changed them all by that mean number that we got right This is a quick thing you can do it fairly fast And if actually if you look at this kind of squinty you can see the shape of the eye still in the binary on the right Right so this binary if we interpret it as an integer then we to I to it right here in Ruby and we That's how Ruby works. I don't know you use an orange arrow and a to I to and then you get this integer On the right side, and that's our hamming distance, or I'm sorry, and that's our perceptual hash I just I spoiled the next part So we're going to use those perceptual hashes to compare the images and decide how An image has changed we're going to detect changes in the images, right? Again, this is your last warning I'm about to turn on a device that may take pictures of you and put them on the internet So if you're on this side of the room and would like to not be on the internet move to this side of the room You have three seconds Okay, nothing exciting is happening yet. Maybe it's happening. This is I mean It's really a house of cards built out of garbage and chewing gum if this works at all It'll be a miracle So we're going to try and detect changes between these p hashes right and a p-hash or a Hamming distance the the thing we're going to use to detect these changes is an information theory the hamming distance between two strings of equal Length is the number of positions at which the corresponding symbols are different from Wikipedia again Accurate, but so is different enos. This is the different enos of the two things, right? If you guys need any other definitions, just look me up. I'm on Twitter I'm happy to help We're just talking about the different enos between these two strings, right? So we have this little 8-bit number and then this other one the difference is one, right? So many transformations we have to wait make on the top number to get to the bottom number, right? So the same is true here then if we take the top and we compare to the bottom We'd have to change three different things to get to the bottom. That's the hamming distance These two numbers have a hamming distance of three, okay now interestingly if you're looking at an image like this It takes rather a lot of change for ten of these to To flip to the other side, right? Because we take the mean and we're comparing the colors in each of those cells over the top rather a lot has to change in an Image to get to a hamming distance of ten. So if we cut off at ten, we know that we're we're likely looking at something else and Now the perceptual hashes that should be noted are not unique, right? This perceptual hash could describe lots of different things at full-scale it could be a donut or it could be an eye of Sauron, right? They may have similar but I but if you take a donut and you suddenly turn it into a duck This is going to notice we're going to get the duck donut conversion, right? All right, so Let's talk about the responsibilities of the all-seeing pie here So the all-seeing pie again is the jam running on the raspberry pie It takes the pictures from the camera and it uploads them to the cloud and then it notifies Palantir It hits it over the API and says hey Palantir a thing happened, right? It creates an image basically. It's a create end point It just creates an image in Palantir. So That's all of the responsibilities for the pie and these are the responsibilities for Palantir now So Palantir has got the UI in there so we can have a look at what's going on in the app And we're constantly refreshing from a little API endpoint to get the latest image Pulling in a little bit of JavaScript so we can show on the page the latest image that pops up In addition to that Palantir is responsible for comparing the P hashes it has to take those P hash values It doesn't have to have the images locally Which is nice because it got the P hashes from the all-seeing pie and it just compares them as they come in and if the New one that just came in is too different than the last one that it has then it knows to raise an alert and When it raises an alert it detects that something is too different It's going to go up to the cloud and it's going to create its own alert bucket So up in the cloud we have one pile of images that is all of the images for the Palantir and it's being depleted They go through and they it deletes images as it goes so the bucket doesn't grow unbounded, right? But then off to the side we have this event pile of images and once an event happens We take all the images that are over here There's ten of them they get rotated and we move them over to the event So all right well now an event has happened We've got these ten images over here any time a new image comes in from the pie Then we put it in the event bucket up to a hundred we assume that most events will finish in a hundred Images and these are all configurable course so then the the final thing that the the Palantir does is it notifies me It sends a tweet to me And lets me know that Sauron is in the house and he's after the knives, right? So oh actually once we're done it has to go and clean up those images that Galician that I was talking about that's Palantir's Responsibility so a pie running unchecked. We'll just fill a bucket to infinity and I'll get a $16,000 S3 bill Maybe that's a design flaw. I don't know That's on how you're looking at it And then we have the responsibilities of the cloud. What does the cloud do in our app nothing? It's a cloud sits there and the Amazon cloud says I can do stuff and I say no right Because the Amazon cloud has these like messaging services and it could have solved my little queue problem It would have been great, right, but then you're kind of locked to that provider And that's also a way I stopped using this gem this AWS SDK gem Well part of the reason so I started out using this to you deal with the S3 I was creating the buckets and uploading the images And it was quite handy in the beginning and I was recording my requests with VCR I'm sure people have used VCR a fantastic tool for that kind of thing if you have to record a question put replay them in your tests You can do that But I would much rather not if I have the option and it turns out that you do because there's this gem called fog Which is terribly difficult for me to understand when I try to read the code But I don't care because I don't read the code very often I just get to use the code in this case actually it's it's not that bad on the inside the The last time I looked at it was a couple years ago And so when I went back now, it's it's changed significantly But fog is fantastic and the nice thing the really nice thing about fog is I don't have to use VCR I can use fog mocks So you turn on fog mock and then you get your own little pretend s3 And you could just talk to your pretend s3 in your tests You can check the things actually got stored there And it's much easier than having to replay cassettes and glee cassettes and track these giant yaml files So everything that happened again. I'm not knocking VCR VCR is fantastic. If you have to do that thing then you should definitely use VCR And so when I was making this conversion I went into my image service I was very clever and I encapsulated this in an image service inside my app And I deleted the AWS gem and I put the fog gem in there And I just had to change the client and like a couple methods where I was talking to it Through its public API and then I was done it works great, right? but then I switched all of the VCR tests over to using mocks and while I was doing that I noticed that I've been using these kind of like Portland artisanal objects throughout my tests, right? And I wanted to fabricate them instead So I put the fabrication gem in there and I started fabricating objects so I could create them in kind of a reliably random way And in order to do that I've got faker in there And so it's now creating my new objects and suddenly I've lost in a very deep dark wood because I've just changed Many things about my application and how it functions And I spent two very sad days with just broken up and read tests and everything was terrible and it would never be okay again And I want to just stop us in the side here real quick to point out that you should only ever do one thing ever in your applications I feel like I've been doing this a relatively long time and I have to relearn this lesson with every day probably click Oh, yeah, I was just gonna do the one thing But then I was fixing and then the other thing right and you think oh I can just do these two things. No You cannot do two things Do not do two things, right? You're like they come in twos. No, they do not do one thing All right, and by doing one thing you will avoid spending two days in the sad forest of sad It was terrible But I fixed it and then the forest was beautiful and I came out into the sunshine And my application was working again, and I had green tests and I got to add some more things So one of those things I added was the API I talked about I have to have an API I decided to use grape to implement the API Maybe not the choice that anyone here would have made or not I don't know grapes actually kind of fun to use I enjoy it and I had to learn it for work because we were adding Instrumentation for grapes so two birds one stone so I have this API and The browser is able to hit this API to get the latest image right this one function there and then the pie is going to Create the images it's going to send us a post to create an image in Palatira when we get a new image from the pie So This is basically our diagram our architecture diagram at this point is pie sends a post and then Palantir to everything okay, and I'm going to describe everything here for you Because keep it simple is really pretty important to me when I try to build things right like I try to keep the pie very simple So naturally when you're keeping it simple, I'm sure We're all thinking the same thing you want to use rails callbacks as much as possible right you want to just stack these bad boys in there just Callbacks on callbacks on callbacks. You can see where this is going. I still can't believe that this train is a real thing. It's so awesome Somebody please get a Lord of the Rings train. Give me a ride So the new image comes in right we send this post over here and Palantir has to create this event right sometimes We don't always create the event, but if the different genus is too much then we create an event So we've got this little callback when those those images come in we create the event Conditionally right and then we also have to copy new images to an event So a new image comes in we have to create the event if we don't already have one if we do everyone then we copy the image That was just delivered to us over to the event. It's ongoing right assuming that event is still open. So It's pretty reasonable. Okay images also will delete the oldest images So we get that rotation. I was talking about what we go through and drop the last image those ten images so We've got these callbacks on an image right Three not terrible. Let's talk about the event here. We create an event An event needs a name right the event has to have a name that's unique across all of s3 So it gets its own little bucket name We generate a directory name for it when we're created and then we actually have to create that directory as well Right, so we we go up there to s3 and we hit the the cloud and we create that directory And when we create that directory, we've got the callback here. We've generated the directory name We've got a callback to create the directory Then we've got a copy the existing images those ten images over to the created directory right in case there was something there We want to keep track of so now we've got three on the image and three on the event We're still going and now we assign the last image that came in to the event so we can keep track of the image That started the event So finally we send a notification off to me on the internet and we have five callbacks For the event and we're keeping it simple right? I mean all that happens is this Right here when the event comes in you hit the API and then these eight things go so If I were to do this again, I would probably choose anything else ever It was really terrible And maybe it's just because I'm bad at rails I would make like I mostly live in the Ruby world and I work with Ruby all day And I don't get to play with rails very often, but man, this was terrible So if you have recommendations of alternative ways to do that thing that'd be great. Why don't we talk later? I I surely could come up with some manager object in there or something basically like I consider Using callbacks in that way that I'm like hiding the existence of some business object that I really need Or maybe an event system anyway So I want to tell you how you can get a sauron if anyone is interested in having a sauron of their very own Has anyone noticed if the little red light is blinking up here. It's blinking at you sauron is winking. He loves you, okay? So if you want a sauron of your own, they're actually really pretty easy to get you can pay $40 to these nice people Who make this awesome hardware and you should pay them $40 for no reason, but if you happen to they will give you this thing Which is awesome? And this actually pictured is not the correct one, but they'll give you an a and a camera for 40 bucks It's a pretty good deal and then you can go and you can get the web app Palin tier feel free to mock my code privately keep your thoughts to yourself. It's terrible and I feel bad But no actually if you want to make port requests, that'd be great constructive criticism I also have another app out there another repo This is the the Palin tier repo. I'm leap out on github as well. Good luck And this is all saying I that's the repo that is all saying pie, but didn't get renamed when I changed in the name over I'll rename it later today when I get back So with that I think we should probably start a live demo of course using these Two application components and the cloud and the Wi-Fi and everything will go fine. I'm sure So if we look over here, let's see if I can escape out of this And now I need to un-mirror. How do I un-mirror? Someone knows this it's command-f1 is a thing Oh, it is. Did I do it first try? God, I'm so smart. Okay. I Look, it's you on the internet Hooray, okay, so we've been taking pictures of you here Over in and Eric's moving into the cloud and I hope it didn't catch you because it may have triggered an event so What the image is doing now it's taking pictures of you over here in this side of the crowd And it's comparing them one after another and dumping them into a new event. There's Eric just jumped in there And those that new event that gets triggered happens by too much change in the image So I'm going to give it another couple images here to get Eric out of there And we're going to look at like normal image right now. Okay. This is our training now in order to do this It's right now. It has an event already. That's open and it's saving images to it So I need to go and close that event So I'm also just going to hop in here to this console on heroku, you know Live because I have to and there's my event and it is not closed. So I'm going to Close it I'm going to set this last event to closed So remember that an event is created when two images are too different, right? And so if I were to have just turned this thing on it would have taken the image from the hallway where I was Testing it before my presentation and compared it to the image of you right now and immediately fired an event Which is obviously not what I want it to happen What I want to happen is I want to save this event And close it and now the new images are coming in and this should assume that everything is fine for now It's not going to detect that there's been too big a change in the image because it's just looking at pretty much The same thing when it smashes it down that little 8 by 8 pixel image, right? So Now if you all would please Stand up and give more of a Friday hug gesture than you are giving now And we will try to change the information significantly enough and see if a picture loads. We're waiting There's a picture of you still not moving Waiting on our API probably a couple images have been taken out and this looks pretty different to me This room is not at all committed over here. This side is like you guys are nuts I am not going to be involved in your tomfoolery and it looks like we don't have our new image Okay, did it blink did the red light blink at you? All right. Oh Well, well, I'm gonna let you guys sit down then I Tried but I'll show you what would have happened in an event normally. How's that? so we have these images coming in and Down at the bottom of our application here. We have some events that I took earlier Like let's see. Here's one and when the events are created they are populated with all the images copied over as they come in and You can see that they Line out all of the images as they pop up and I'm really sad that we didn't get it to change Why won't you change? I'm just gonna put a thing in front of it here and see if it'll ever take a picture again If it does it'll get a picture of my badge and then it will tweet a thing So the idea is it's taking pictures of you now sitting back down again. It just didn't like the different genus Okay, well, let's go back to the talk here. So that's the app That's the app the Palantir where the images have been uploaded and we have pictures coming in from the hallway It'll be out there again later today if you guys want to come by and play with it and this is The future of this app step one make it work. Oh Yeah, I think that's someone is Tweeting at me on the internet and I had the Twitter open I should go and turn off Twitter right now actually because you guys are a bunch of jerks and I know better Then to do that. Yeah Yeah, I know right Okay so The The bit where it didn't work just now I think I have to figure out first of all But it does notice when things have changed in the image and maybe if we changed too many things I don't know but that's the first step and also we're going to have a faster setup process in the future so getting this thing rolling is actually quite difficult you have to deploy the Palantir app to Someplace online and then you have to get an API key out of it and put it on to the Raspberry Pi And you're like SSHing into the pie and manually adding king fake files And I think I could simplify that whole process quite significantly the other bit of course Is that we have a lot of data now about images and hamming distances and we could Store all of that stuff and probably learn from it So you could learn times of day when things were an anomaly or not or you could learn What particular things in the image happened to look like like maybe you're triggering your hamming distance on your cat But the cat always triggers the hamming distance in the same way because it walks along the floor on the bottom And so you could learn to ignore those types of things And finally as I mentioned I would like to fix everything In the app there. This is also the camera bit that I'm using is particularly slow So the raspi still utility that comes on the pie takes pictures about a second apart It takes a long time But then I have to push it up over the API and there's a faster way to do it There's an app called or a utility called raspi fast chem D Which is a demonized version of the camera that you just send a user signal to And it'll take a picture and it can take pictures like sub 10 milliseconds So I can take many many pictures very fast and upload them as quickly as I take them And they do end up being quite small plus I get to have a demon inside of my Lord of the Rings application here So thank you all very much for coming. It was a pleasure. This is Eric earlier on the camera when it was emitting events And I will leave you staring at my Palantir app here or not because I can't get it up I guess I have rather a lot of time and I'm mistaken. Did I go really quick? Anyone had any questions about how this thing went together now would be a good time. Yes, sir I'm gonna call it done. Thank you all so much for coming