 So good afternoon, my name is Dave McAllister and bear with me. It's right now. It's about 7 o'clock a.m. My normal time So I will try my best to stay awake and answer all the questions without demanding a cup of coffee during this talk I work for a company called solace and solace is in the the business of moving data from one place to another We power a lot of financial communities. We power Singapore smart city. We're connected car initiatives We're in transportation and what I'm going to do is talk a little bit about sort of the Experiment that I did in trying to work from an IOT sensor environment Into cloud foundry and in this case I will admit I'm using pivotal cloud foundry simply because we had such an environment set up So it actually all started real simply and I'll try to stand in front of the microphone But I'm really bad at standing behind podiums here Actually, let me see if I can pull this off It all started actually really simply nobody in my neighborhood has the faintest clue as to what I do for a living And so I decided to build a fairly simple little Structure that said I'm going to build something IOT that I can show people and then explain from that What I'm doing and so I wanted to use a message to move the data around here I wanted to build a simple set of services so I could could mimic the Necessity of a microservices architecture at the same time And I wanted to learn as much new things as I could at the same point the three Surprised-looking creatures behind me are my three cats And this is the kind of look that I usually got from my neighbors when I tried to talk about data movement and messaging in the first place So it was pretty much and so it was a very simple aspect I wanted to take a simple little particle sensor and this is a Sanjay PBT 42 and it measures to either 10 micron particles or 25 micron particles And I wanted to be able to build this into something that would report Something very simple and of course that requires me to have the rest of the parts that were there And I'll get a little bit more into the part list here This was based originally off a project called dust dweeno and my thanks to mineral munitions and Matt shroyer This project itself is open source So if you go out and switch word dust dweeno you'll find the source codes available for this and this was my starting point I'm a firm believer of minimum I was I was involved in open source before it was called open source I was at the meeting when Larry Einstein called it open source would have got named by his person But nonetheless, I'd like to build on top of things rather than trying to recreate something from the beginning The dust we know is very simple It has to come up in this orientation Because there's a little resistor down here at the bottom that heats the air so the air goes up across the sensor field And that's it. This is all it does just measures the dust particles that are inside of that So very simple architecture this sensor Connected in to show some display stuff and what you're actually looking at is for my My neck of the woods is data that came out of just being recorded and then dropped into a Excel spreadsheet to graft out for five days in August And as you see the numbers vary and there's a reason for the numbers varying and we'll get into that As I go into sort of the the next steps inside of here But because I wanted to treat this as a message rather than just treating this as an environment You know just passing the data. I stuck in the middle this virtual message router Constructing virtual message router community edition free for use to have fun do whatever you want to you can find it out there But I did this by passing a message that meant that I had to look at data protocols. I'm no longer just passing a text string I'm actually passing a message header a message body and going into this environment again pretty straightforward Pretty simple inside of that but then naturally I showed this off to a couple of people and Somebody promptly says hey, I work for this local area park called Edgewood Park and Edgewood Park is a very unique place It was actually saved because of a butterfly. It was about to be made into a golf course It was saved because of a butterfly called the Bay Checker spot and the Bay checker spot was an endangered species and one of the last two locations It was found was in Edgewood Park. And so Edgewood Park was saved. But if you notice There's a major road that runs right through Edgewood Park And what happens is that we start seeing all sorts of invasive weeds show up Because of the road coming through there's all sorts of stuff that comes from the park And somebody said wouldn't it be quite a cool if we could figure out what particles are coming over and what happens when the wind Direction changes and all these different things and they said, oh Dave you did it for one. You can just do it for the park No problem turns out that IOT Scale matters when you build the architecture very honestly and I'm not even talking about a huge architecture I'm gonna talk about now some of the scale aspects that happen in IOT as it comes up But even going from one simple little thing to 20, which is our original goal became more of a challenge And so what happens is now my sensor is feeding MQTT data message queuing tracking telemetry data into the VMR And I wanted to feed that MQT data via the cell as VMR Into cloud foundry and I wanted to feed in road data coming in and I wanted to feed in weather data coming in and then I wanted To put something out so I could show it wherever be on my phone be it on my my Computer browser all these different places So MQTT data on one side rest data going in and rest data coming out I'm sure that with you with most of y'all if you're familiar with cloud foundry at all You know that the access of going into a public cloud and into cloud foundry has some challenges if it's not HTTP fortunately, there are some solutions around that and One of them that I'll talk about I did use is inside of here So this is what it actually looks like all the different pieces here And there are actually two solutions here as we've started rolling this out I'm getting other people to help pay for this it works out this way This thing here is a juice solar battery So it charges itself during the day and it can support the entire needs for the Arduino There or there using an Arduino Uno Inside of it So it's perfectly capable of doing that and the battery itself without any charge will run that environment for about four days So that's that became very straightforward. It's also waterproof and it makes a nice protective top for the environment that we're going inside of here the original thing Used I think it's this one. Yeah, it's this one here, which is an ethernet wireless Environment, so Arduino has an ethernet shield that needed to be connected to it And the one below it is actually the ethernet wired one which I happen to hide up lying around It was my first test case was to use that one. So now it's a wireless one Now I'm actually gone to using this Arduino, which has wireless built into it So the entire environment is this Arduino this sensor and this battery and that's it And now I'm tapping the wireless is that already exist around the park, but that becomes the concept The goal is to ring the park and right now I've got probably Five or six sensors that are out here I do not have a dashboard and as I say this is a very bad mock-up as you can tell from my PowerPoint slides. I am a lousy design person So somebody else is gonna have to work on the design for me going forward here But as you can tell we do tie things such as wind direction We tie a little bit of the traffic activity and we tie in particular the particulate count that goes on and right now I'm not dropping it continuously. I am dropping it on a thinker. What did I pick up up here do? Basically a daily basis inside of here But the goal is to be able to tie all these things together into a visual representation So you can see where the traffic is busy and you can see what direction the wind is blowing from the wind has a major impact on How much particulate data these sensors pick up as well as does the elevation above sea level for that environment? So in the center of this park there is one very tall hill and the particulate matter up there is Not affected by the wind direction as much as you would expect it to be and that's because being in California We spend eight months of the year in a completely dry state and the dust gets picked up And so as it goes up the hill the dust seems to go up that hill and it's a pretty consistent Measurement not always you'll also can tell I should I should have put it on here But you can also tell that there is one Distinct dip. I think it's the last day. No, it's the state Number four here this distinct dip is because it's a Sunday morning And so the traffic on the road is not as busy and so there is definitely a distinct dip that happens inside of that So why did this all become kind of an interesting thing? Well, you know as it turns out IOT is really all about the data We can talk about the sensors. We're gonna talk about the environments We can talk about the gateways, but very honestly IOT is all about the data It's big data and we do mean really big data We're building connected car initiatives here But the average connected car will produce 25 gigabytes of data per day To our per hour to the cloud and this is after it's already sorted out all those different things and figured out what to do But what it's going to and this is Hitachi report So if we all have to look for a connected car 25 gigs Hitachi, you'll find it I should have put a pointer inside of here The other one is actually the Google connected a lot of autonomous car and it produces 750 megabytes per second Which a completely different model and by the way, that's not the the data that the car is producing That's the data the sensors around the car are producing and right now. They do record all that they do put it up there But in a truly IOT type environment most of that data would get thrown away at the car level inside of that But just to give you an impact 25 gigabytes of hour per hour for is 6,875 gigabytes or 6.875 terabytes per year For the US marketplace alone if every connected car if every connected car Sorry if every connected car in the US was running this for the length of time that they think about running Which is 240 hours a year for here. It's us 1740 exabytes of data and if you take it worldwide to that it's 14 set of bytes per year IOT is a big data problem, and I can tell you just moving from 1 to 20 was a headache So scale matters inside of this So I would show off this demo, but this is a demo of the FAA Ground control network and it's something called swim Which is the system-wide information management? It's now being looked at in your control It's being looked at over in Japan in Australia New Zealand because it turns out that being able to know where the flights are at all times Actually improves the rate at which planes arrive and depart on time So swim data becomes real interesting real interesting is very widespread the swim Data feeds can be accessed. You can go off and sign up for them. You can get the data access anywhere you want to This is ground control. These are the train planes that are moving around on the ground and Unfortunately for me every time I have tried to do this talk in the last six months The FAA has managed to turn off my data feed every single freaking time for this But nonetheless you can go off and play with this There's a little pointer on the slide So it comes back up you'll be able to go take a look at it and see what it looks like yourself. So What's the real problem when I started looking at IOT inside this environment? I had two sets of problems that I'm going on here I have a Device problem at the end and I have a cloud problem at the top and I needed to cross the streams between those two and In my case, I made it a lot simpler all my sensors are identical So I didn't have to worry about a lot of things It's very simple data set that I'm passing in a triplet form which is pretty much what sensor are you? What time is it? What's the data? It's pretty much what happens inside of a sensor environment. You get three things that happens Those things talk up. I can record the data I can add more sensors to it which is what drove me crazy in the first place was adding more sensors to it I then can expand to store the data I then could if I was in an industrial IOT environment or in a really sensitive environment I add firewalls and load balancers inside of here and then I want to do something with the data So I've got to start analyzing the data. So it's got to communicate back and forth between these tracks and Then somebody can come along and say change out the sensors That's already been brought up as well. This is a Texas Instruments sensor tag And I'm going to make the strong recommendation because occasionally I get asked the question If you want to start playing with anything in IOT Get yourself a sensor tag a very clean API set very simple to work with and has Somewhere between 14 and 16 sensors built into it and Dirt cheap and by the way so far the battery in this one has lasted slightly over two years So I'm not I'm not too unhappy with it the sensor tag model sensor tag model will tell me things like movement So if I brought up the dashboard you'd see able to see the shake you can tell when it's light or dark inside of here You can tell acceleration you can tell humidity you can tell temperature And so this is something that people are now saying can we attach one of these to go along with this? So we know all the environmental details at the same time that we're knowing all of the Capabilities inside of that the short answer is yeah, right as soon as I retire and have lots of time to play with this but secondary issue So In an IOT environment fortunately not in mind Sooner or later you're going to want to talk back to the devices the sensors and Excuse me Continue forward. There are lots of other sensors right now I'm trying to bring in three different sense of sensor data the weather data the road data as well as the particulate data I could be bringing in another 14. I hopefully not anytime soon But you're going to be a talking to them in the same way you got to worry about security in this environment my environment Not so much. If somebody goes in this boost my sensors. I really don't care, but why security is important Last October You may have remembered or is it November there was a little thing called the din DNS Shutdown there was a denial of service attack against the major DNS server for the US East Coast and Everything basically stopped Netflix stopped Amazon stopped lots of things stopped inside of here that attack was generated by a number of video cameras IOT video cameras reporting data So security matters inside of here my other one is a much more famous favorite example is that the one of the universities in Paris demonstrated a successful hack on the Phillips Hue light bulbs and what they did was they flew us a drone past the building and the flight Balls started blinking SOS The nice thing about this from a viewpoint is that the light bulbs could then infect each other and In a city the size of Paris the estimate is that 15,000 light balls will become a self-sustaining infection And that the entire city of Paris would be infected within 24 hours if there have been 15,000 light bulbs It gave you an impact of what's going on in there Fortunately most of that has been closed off But that just means that somebody will think about something new inside of here so think about security on Scale when you're thinking about security for instance with MQTT You want to make use of something that actually does authentication checking against to make sure that the device You're talking to is the device you think you're talking to you also want to be able to do not just publication Subscription one of the standards message exchange patterns that's reports data or sends data out But you also want to do requests reply back to the device so that you can say hey, I've got an update for you Can you tell me can you take it now or even just simply ask them what version of a system? It's running we find that particularly in consumer electronics Nobody ever bothers to update their bloody video cameras that are ringing their houses You know nobody bothers to update their their their Nest thermostats if it doesn't happen automatically it doesn't happen However be fair the video camera meltdown Was because 80% of the people had never bothered to change their password And they were all default passwords the biggest headache inside of security is always going to be people Guaranteed 100% so we can ever get people out of the loop in this and the internet of things will probably be pretty secure So Giving that to a little larger example here This is the Singapore land transit authorities model for traffic tracking inside of Singapore if you're going to drive a car in Singapore your Singapore car will have this thing called an OBU Which stands for on-board unit? That's pretty straightforward and it reports MQTT and so MQTT data goes up goes into a collection aspect Then goes into a secondary pass-out aspect and then comes out of the other end for instance as JMS and the device is actually smart enough and one of the reasons I love them is that they can actually change me from MQTT to JMS and I don't have to write any code to do that I don't have to worry about it. It'll just change the the protocol format for me automatically Number one complaint I've always heard when showing this maybe not so much over in Europe But in the US is go I'm not going to spend to have an on-board unit put inside my car Well, the average Ford Mustang inside of Singapore cost $200,000 US They don't want you driving in Singapore. So cars are really expensive adding a $25 OBU doesn't matter to these guys however Because this thing is constantly in communication and constantly knowing what's going on and rolling data up here if you park some Place for six minutes you get charged for six minutes of parking if you park for two days You'll get charged for two days if you park illegally you will get a ticket guaranteed All those different cases in Singapore Even though they try to control traffic. It's Unbelievably busy and so you'll be going through the Central Expressway And it will just be backed up and they charge you like three Singapore dollars for every kilometer I'm probably making you know I probably got the numbers reversed or something here, but nonetheless. It's pretty expensive This capability because it knows where your car is what your car is can come back to you and say hey If you're swinging over to the southwest freeway instead of the central freeway You'll get there faster and give you a time estimate and our reducer costs from three to one dollars for every kilometer You run and so it becomes a social mechanism that then transports back Now this is done because of the fact that the thing is in constant communication These are always on environments even if you go through a tunnel and it disconnects it reconnects The protocol has to be capable of sustaining Disconnection in an IOT environment MQTT isn't always on protocol But being able to cash those environments means that you get back whenever you come back into connection and that becomes a really unique The the person the architect who built that this talks about for instance He lives a little bit out of town. So his first 10 kilometers of driving And he usually does it at four in the morning. There's not a lot of cars And so he tends to slightly exceed the speed limit I think it's the way he phrases it by slightly exceed the speed limit is something like he does about two and a half times The speed limit for that section. I think it's a 30 kilometer or 35 kilometer section He does like 85 Besides this and once a month he gets a letter from the police department saying we know you're doing this If you don't cut it out, we're going to going to do something to you No, they don't have a law that lets them do that, but they could certainly say we know you're gonna do this at 4 33 every morning And we're gonna have a car sitting there waiting for you So there are good sides and bad sides inside of here, but this is fairly simple Now I wanted to talk about IOT protocols Mainly because what's wrong one? Manly cuz there are a lot of them MQTT we've just talked about message queuing to let telemetry tracking became out of the IBM MQ environments MQTT SIN for sensor network is a version that actually allows Disconnected and reconnected sensors environments co-app constrained application protocol and you can go down the list DDS AMQP advanced message and queuing protocol. It's actually turns out to be very popular for IOT over here in Europe It's about 11% usage and that sort of leads me to the next thing Which is okay, so I got a lot of days, but you really only have to worry about two or three HTTP and the rest protocol is still the number one leader Fort whips, okay Back yes, HTTP rest protocols are still the number one leader for protocols in IOT space MQTT and co-app are the next two that follow This is data from the eclipse foundation. I put two years on there. Okay, so you can see that MQTT and co-app are the fastest growers in MQ in IOT protocols however, anybody who comes back and says None Please don't talk to me because if you're not using a protocol for IOT traffic you got more problems to worry about then I'm going to solve So HTTP and MQTT first to co-app special case So a little bit of consideration. I've covered some of this. You have to worry about connections subscriptions and cues That's how these devices talk to the back ends You have to work worry about the message change patterns inside of here device initiated pub sub sensor talking to the back end and Request reply cloud initiated request fly is the other one that you have to worry about Keep in mind with cloud foundry Pivotal cloud foundry and so forth. These are not as easy as they look Thankfully, there was this thing called TCP routing or TCP roots, which GE predicts. I first think they first talked about it in cloud foundry in 2015 Which allows me to inevitably handle MQTT into this environment So I mentioned that I that there was a workaround for MQTT going into cloud foundry. I didn't have to go HTTP That's how TCP roots. I think it's TPC roots are the way that you can manage to pull this off very slick piece of code The headache there is making sure that it scales appropriately to your environment Come on So let's talk a little bit of quality of service and very quickly For this QoS is zero is I'm gonna send it if it gets there good if it doesn't don't worry about it QoS one I'm gonna send it and I'm gonna make sure that it gets there But you can duplicate the result and this is where time stamping can be your best friend in times inside of a sensor environment You can easily identify duplicate environments here and then QoS two which is once in exactly once Each of these require more traffic. So when you're looking at this from a bandwidth viewpoint It becomes very important to realize that your bandwidth goes up as you the amount of traffic goes up There is a stateless challenge if you're doing multiple tiers like this when the device at the bottom tries to talk to its back end piece It's load balanced into an environment because that's not a single computer in the middle here for that And then it then talks to another computer which talks into its application space If that connection goes away, there is no guarantee that the load balance server restored to the correct point The point that it last was and so riding through a QoS to environment within a multiple environment Firewall in space is non-trivial. It can be done, but it is non trivial When you go into a cloud environment where you then are crossing additional boundaries for networking It's even less non-trivial or more non-trivial whichever way that works Protocols I'm gonna scream through this section because I can talk about this for about four hours, but nonetheless AMQP Great brand new fully standardized It supports TCP UDP High processing power you usually see this in the back end environment, but I am beginning to see it show up in in IOT environments Particularly again here in Europe about 11% of the marketplace is using AMQP as a sensor level environment to talk to the back end here It is bandwidth intensive header fields are big for that JMS Everybody's used JMS JMS owns the data center very honestly inside of here It's dependent on devices and there's no interoperability between stacks So while JMS defined a wire connection. It did not define an API environment And so there is no guarantee that if you change JMS providers that you will be able to adapter code without change inside of here All classes of service, but there's no interoperability becomes a bit of a headache inside of here Rest of course, everybody knows rest rest lives everywhere. Don't worry about it. No QoS. You're on your own No message exchange patterns. You're on your own and rest is a blocking protocol So no rest will respond breasts is going to wait until it gets a response And so in an IOT environment where you are passing messages around to say break because there's somebody inside of the the crosswalk It may not be your best idea So you may do it may need to be able to say I didn't get a response slam the brakes on anyway Reliability is application dependent. So if it doesn't work, you're on your own MQTT Personal bias. I like MQTT. It's simple. It's lightweight. It's low power But it doesn't support header fields So I don't know what's in the message until I open the message unlike all those others and So that means that adds another layer of complexity so that I can no longer do really good routing I have to open the packet to see what's in the packet before I can take the packet to the right place I can do header routing, but I can't do content routing and MQTT be kind of nice to have those things It also doesn't directly support queuing capabilities And so there's some headaches inside of that it does have an S can end version and then constrained application protocol if you remember the list HTTP MQTT co-app co-app is a Request reply model pub sub is coming. It's been coming for about a couple of years now. It's here This runs below TCP. It runs on the UDP level And so if you're going to cross bridges and go beyond that two at 127 devices, you're on your own There are some great bridging tools. If you want more information, I'm running out of time Stop by my booth upstairs. I'll discuss those with you So quick summary so I can have two minutes of questions here IOT has been an interesting light Writing IOT without dealing with the cloud is fairly straightforward problem. It is still a scale problem adding the cloud into this IOT space and They're not using any of the current IOT platforms that exist in public clouds As a as an observation. I haven't tried this yet is non-trivial and it's all about how the data Communicates and how the data moves in and out inside if you're so the one thing I can say is if you can Stay with the standard wire protocols Do not get trapped into a proprietary Structure for your data inside of here and be as flexible as possible in movement and with that Then information is me you can find me on Let's see Twitter Facebook or Skype on those on LinkedIn or GitHub and DW McAllister LinkedIn I'm Dave Mack Here emails here if you want to play with this yourself There are two things the VMR that I mentioned here is free complete usage You can find it on our devs portal and source code is available in GitHub for that and while y'all are reading This is my favorite connected car Autonomous car carting of all times If I ever own if I ever rich enough to own you know like a Tesla I'm never going to leave it running inside of that with that questions shock If not, yes So the question is why did not use MQTT over web sockets and I could but the headache becomes when you start trying to look at That as a scale problem. So if you're setting up 10,000 web sockets at the same time you're setting up 10,000 separate MQTT cues It becomes a challenging problem to be able to manage both of those environments separately I now need to be able to relate every single cue to every single time think of it in a sense that I'm going to be Setting up a virtual environment for every single connection point Whereas MQT already has that capability built into it to recognize each device bias message header feels and so it simplifies the problems to simply use MQTT as a native basis So yes, that would work in my my in more park example But it would not work for a connected car company who's trying to run 8 million cars If not, I will be in our booth if you've got questions or just want to tell me what you're doing in IoT Because I love to hear stories inside of that. Thanks for your time and attention and enjoy the rest of the show