 All right. Okay, so yeah, thanks for coming to my talk. This is my first PioHio talk, actually second time attending, so pretty excited to be here. Excited that I could wear the shirt somewhere and have people get it, and instead of looking, you know, like I'm making some kind of political statement or something. So I'm excited for that too. A little bit about me. I've been in the software business for a long time now, mainly doing CC++, mainly doing CC++ development, but a lot of incidental Python. So I'm definitely not a FNista by any stretch, so any any misstatements I make are out of ignorance, not malice. And what, so this thing that I'm going to talk about today, Google Protobuf, is something that I've used in, it's popped up time and time again as a good tool to fix problems that I've encountered. So let's get on with it. What are protocol buffers? They're designed by, or developed by Google. It was developed as one of their, what they call, I don't know, 10% projects. They're extra-curricular projects, so, but it's been widespread adopted throughout Google. They use it in, I don't want to say all of their projects, but many of them. It's an extensible language, platform, neutral, efficient way to encode data. So, yeah. So, at its core, what it does is it takes, you know, various arbitrary data, makes it as small as possible, not using compression, but just using clever encoding. And as I said, it's widely used in Google. I think they're, I read on their web, in the GitHub page that somebody did a search of their repo, and there's over 40,000.proto files, which are kind of the source code of that. So, it's, and it's open source and pat-free. So, that's all good. The three benefits that you get from Protobuf that gets me coming back with it time and time again is the backward compatibility is easy to get. It's, they designed it for that, for that purpose, so they don't have to, you know, upgrade all their servers at the same time to talk the same language. The encoding is very efficient, and the code to parse these buffers is automatically generated. So, there's not a, not a bite of handwritten code where you have to, you know, oh, this is just kind of a field I need to convert that to, you know, an int or whatever. It's all built in. This was largely stolen from their Protobuf GitHub as well. Basically, they made a comparison to XML. The XML is great for a lot of things. I use it a lot. It's very, very readable. It's very wordy, though. If you're doing something at scale, so either, you know, something of the scale of Google where you have, I don't know how many, you know, tens of thousands, hundreds of thousands, whatever, however many servers they have, they need something that scales. And, you know, when they're, I don't want to see what their bandwidth bill is, but shaving one byte off of every pack that they send to their server, I'm sure saves them, you know, a lot of money. So, compared to XML, it's three to ten times smaller, twenty to a hundred times faster for parsing. The data accessors are automatically generated. So, there's a simplicity there that you may not get with XML, at least not that I'm aware of. It's really what, in addition, so what attracted me to it for this talk is that it's, in addition to using it at large-scale deployments all over the world, you can also use it for very limited bandwidth or limited RAM, limited storage, so forth. So, like, little devices. So, that's what I show later. I've got some kind of antiquated little devices, but that's what I had. So, that's what I went with. Did anybody go to Greg Svota's talk yesterday? No? Yeah. Okay. He showed off this little $5 monochip, the ESP8622, or 8266, yeah. And it was, he threw an Arduino into the audience that here you can have this. You don't need to use these anymore, because that thing is pretty awesome. So, I don't have those. So, I have something maybe hopefully a little more familiar to people who aren't into, like, small computing devices. So, and I should add that all of this, everything, the slides and my code and everything will be, it's on GitHub, so at the end there'll be a, you know, links page, and so you can pull all this stuff down and look at my terrible code. So, that's kind of the reasons to use Protobuf. Some of the reasons that you might not use it is if you have really big data sets. The Google Protobuf page says, if you're sending, like, more than a megabyte in a chunk, maybe think of something else. That said, I read in a forum recently that somebody was complaining about, eh, we're not so happy with Protobufs, because we're throwing around 64 megabyte packets and it's just not holding up. It's like, well, it wasn't designed for that. And as you asked, it's a binary format. They're not interested in making this human readable or editable. This is something that you put in the pipe, it shrinks it down, sends it over the pipe, or maybe you're saving data, you know, to hard disk or whatever, but the idea is it's not meant to be something, you can just open up a browser and say, oh, well, let's look at all the fields in this structure. That's not gonna happen. So, because of that, you can't do things like, it should leave other, like, if you're, like, XML is really easy to stuff some JavaScript or, you know, HTML or something like that in there, but this, you're not gonna do that. Okay, so, like, stop me if you've heard of this one, being Python developers, but they've kind of had a schism in their world, too, between Proto2 and Proto3. It's not fair to say it's a schism. It's, Proto2 came out 10 years ago. A lot of people use it. Last year, in July 2016, they released stable Proto3. It's recommended for all new code. It is, it is, but in terms of making your message definitions and stuff, because the part of Proto3 is they really made the language less wordy and, and just, it's better to use it this way. And it's, it's basically an iteration on Proto2. And, and yeah, they, they kept backwards compatibility as much as possible. But generally speaking, you wouldn't be mixing the two anyway. It's more, the backwards compatibility is more within when you set up, initially you set up your system and you have your packets and whatever that you want to send. And then later as the system grows and evolves, you're like, oh, we got to send new data. We don't need that data anymore and so forth. That's the backwards compatibility that I'm talking more about. But the only reason I really mentioned Proto2 is because a lot of the docs on, on the GitHub for ProtoBuff, all their examples are still in Proto2. So it's gonna just avoid, you know, prepare you for the possibility of confusion. There's gonna be, you'll look at the examples, you'll look at the, the language reference and, and they don't match up. Well, I shouldn't say that. All the examples will be in Proto2, but you want to do it in Proto3 and it's not hard to map one to the other. So how do you use it just from a, a workflow standpoint? So the protocol mess, ProtoBuff messages are defined in a plain text file, usually with .proto as the, as the file name extension. And those are then compiled down to the, the generated, your target language that you want to use with ProtoBuff with, use this Proto compiler to generate the data access classes in the target language that you're interested in. So Python 2 and 3, Java, C++. There's all the, yeah. And there's also a lot of additional third-party plugins. So these are the ones that they support just kind of out of the box. You can get additional third-party, you know, open-source plugins that people have written to support other languages, you know. So it's definitely a vibrant community. The, a lot of activity on this, this project's Proto, this project's GitHub is, you know, even now, even after 10 years. So it's definitely a thriving and broadly used technology. And you can get pre-built binaries, you don't have to deal with any of the problems of, you know, sometimes you've got to build the packages from source and all that. You can do that if you want, but especially with Python, you know, they've got the Python package index, I'm pretty sure. I know they have it for Python 3, probably have it for Python 2 as well. Just, you know, you can just do a hip install, ProtoBot. And you're done. So, so the, so the basic flow goes like this. You, you start with your, your Proto file. You run it through the Proto compiler. In this case, for our purposes, you know, it generates this foo underscore pb2.py. This naming isn't so flexible. They always append underscore pb2 even if you're in Proto 3, which is kind of, yeah, there's, there's a, there's some, some rough edges that I don't know why they haven't sanded them down. But just be aware of that, like you, because it's in the, it's in the Proto file that you specify which version of the language you're using. And it just, it generates the right thing, but it doesn't reflect the name. And then, and then you just, you import it into your, your app, your application, just like you would any other Python module. Okay. So, as you might expect, any kind of data encryption language is going to have support for, you know, a lot of numeric types, so double float. They've got a bunch of different 32 and 64 potato types, which I'll talk about a little bit more later because they're, they're important distinctions about those. String support, most languages, UTF-8 only, Python, there's some kind of unicorn support that I haven't fully grokked, but you know, if that's important to you, it's definitely an option. Boole, we all know what that is. Bites, though, if, if their string type doesn't cut it for you, or, you know, you need some other bag of bytes, bytes is the type for it. Enums, if you're familiar with C and C++, they have the concept of an enum. It's basically just a bunch of named constants that are in a group. So those, those allow you to have kind of a set of predefined values that you can give meaningful names to instead of, oh, that's type one. You can say, oh, that's a, you know, that's a corvette. And then finally, there's this any type, which is, it's like a bag of bytes, but you can also give it a URL to say, this is how you interpret what this bag of bytes is. So I haven't used that before. It sounds pretty cool, but just haven't used it so I could, couldn't say too much about that. Okay. So yeah, I mentioned before that there's different ints for different uses. Most of them have a variable length encoding and that's, that's where protobuf picked up a lot of its compression, or I don't want to call it compression, encoding efficiency. But some don't. So it's important to know that the type of int that you're using matches what the values you expect to go into it. So for instance, if you have to have an int 32, which sounds like, oh, you can put any kind of integer in there. Well, no, if you're expecting data that's mostly positive integers, then int 32 is, is okay to use because if there's a possibility it might be, might go negative, then int 32 is probably okay. But if it's often that you're going to have signed values that go negative, then you would use the S int 32. And similarly, if you use, if you know they're always going to be a positive, then you'd use the U int 32. And the reason being that they've got, well, I'll get into that in a bit, I talk more about encoding. And then finally, doubles and floats are, they don't encode very well because, you know, generally speaking, there's, you know, you're going to, if you take the value of one over three and put that in a float, it's, it's, you'd probably don't care about the 19th digit of precision, but the protobuf is no way of knowing that. So if at all possible, scale your, scale your floats and doubles into int values that you can get a more compact range from and you'll get better at coding. Okay, some other notable features. Messages can be nested. So you can have, it's kind of like, you can think of it as C++ classes or, or the equivalent in Python of, basically, you can define a class within another class, such that only, you know, you have some, some scoping is effectively, so you can allow, you know, a little bit of a, oh, oh, I guess, I mean, this isn't meant to be like a full programming languages, but it does have kind of an object oriented way of specifying how your data is structured. Likewise, you can use messages as fields within. So really, if you, again, if you know a C or C++, this is, you're going to see, I'm going to show a proto, a simple proto in a minute, and you'll see that it's basically looks like a C struct. And that's a way to, a good way to think of it if you're familiar with those. But also the, the language has associated maps so you can, you know, it's like a dicton in Python. They've also in, in proto three added a, in addition to the binary encoding, you can do a JSON encoding and decoding. So, you know, give you a little more flexibility with maybe existing deployments and such. And then you can also import definitions from other proto. So it's, again, using the C metaphor, it's, it's just like a header file you can bring in for, to build up more complicated messages because you don't generally want to try and stuff all of your message types into one proto if you don't need to. Okay. So here's a simple proto file. As you can see, it's, like I said, it looks a lot like a C struct if you're familiar with that. There's some keywords like, so message is a keyword. So there's data types that I talked about before. There's enum keyword. And, and that's the, a pretty good example of what a very simple proto buff message might look like. So here's, I named my message type sensor data. So it's got a temperature or humidity and an enum that I use so I can say which kind of units I specify the temperature in. And so now here diverges a bit from C. They have these, these tag identifiers. And this is kind of key to how they achieve their, their encoding efficiency. Because these, these tags specify basically the, the order in which messages are kind of serialized and serialized. And also they're, they're used to handle cases where, because all of these, I guess I should mention that that all of these fields are optional. So I could send an empty, well, not an empty sensor data, but I could send a packet that's got a sensor data message in it that I didn't change anything. And, and proto buff will just supply default values to all those things. And in that case, it will send it very efficiently because, because these are all considered optional, the, the sender when they encode this message, they will, it actually shouldn't encode very many bytes at all because it's not going to send any of the values that I didn't set. It's just going to basically send a token saying this value should take its default. Right. So the question is, yeah, do you have to specify the tag idea? Yes. That's kind of an irritation. But it's, it actually turns out it's, it, I, I used to think, well, why can't they just do that automatically for me? But in a minute, I'll get to like, some of the reasons why they can't. But yeah, you have to specify them. Okay, so this is, again, I mentioned they, we use the proto, protoc compiler is what generates this code that does the parsing and encoding of the data that's specified in the proto files. That command line there is what you would typically use to generate a Python output. You can generate all the, all the languages that supports, you can generate them all in the same location of Botox. So if you, you know, if you've got this cross language system that you're developing, and you, you know, you want to generate Ruby and Java and Python and all that, you can specify them all in the same command line and have it all generated at once. And like any good utility, you can invoke it with help and it will help. And then the output is a, in, in the case of Python, it's a, it's going to be a static descriptor of each message type, which along with a meta class you can use in your, your Python program to access the class at runtime. So, here's an example of what they serializing that previous message would look like. You create, create an instance of one of these sensor data objects and some, you know, the name comes, the, the, the module name comes from the, you know, the file name that Protoc generated. So, simple in this case, PB2. And then you set your valid fields to whatever you want them to be. And like I said before, if you leave one out, it'll just get the default value and so I didn't have to set all these and I'd probably demonstrate that, probably shouldn't have set them all, but anyway. And then you call serialized string and that does just what you think it would, which is, takes that sensor data message, turns it into a little, little string bytes and then you're ready to do whatever you want to do, send it over the wire, whatever. On the, on the other end, the receiver of this data, they, they get this serialized data chunk. Again, they create a simple sensor data object called parse from string and then they can access the fields to do whatever they want. So, and again, if, if the sender hadn't sent a, say, a temperature, when, when I read this temperature here, I would just get the default value, which is typically for, for numeric values of zero. And, in fact, if you look back at this, one reason, one reason, see there's a, this enum here has got a, a value of zero, that's, you have to start your enums with a value zero because that's what Pro-Toc will understand to be the default for that enum type. You don't have to have them in order after that, but you do have to have a zero. Okay, let's do a little, little command line thing here. I'm just going to show a simple example of where do I want to do this, let me know. The projector's on, I think I can see the light in there. Well, this is unexpected bonus. Let's see if I can figure it out here. What's that? Yeah. Yeah, but this has never happened. It always works. Okay, so I'm just going to show you a little example of that. So here I've got that simple proto again. So this is what I showed before. In addition, I'll just point out here that there's that up at the top here, I specify the syntax of proto three. If it's not there, it assumes proto two. So again, this is just what you're looking at before. And now the, that example that I showed of, well, first of all, I'll show you. So pro talk, I can type Python. Okay, so that's, that's how you're on the compiler. Pretty neat, huh? And then we'll run a simple thing. So let me just scroll that up a bit here so everyone can see. So in that, so you can see the payload length is nine bytes. So this is, it's sending basically 12 bytes of data because we've got a, there was a float, an SN32 or a U32 and an enum. So mapping that again to like a C world, that would be, if you were just sending a raw struct of the wire in C, you might not even just dump the struct and that would cost you to call bytes. This is only nine. And it's also got the structure encoded in it. So you can see already, it's a reasonably, you know, decent encoding savings over just putting a raw, raw bytes on the wire without even getting the structure that you need. So I'm probably saying this poorly, but this is less, less bytes to send just the raw amount of data that you would otherwise. But you're also getting the structure of the data in that encoding as well. So is that clear if it makes sense? My job is done. Okay, so let's go back to this. You can see I'm an expert at giving presentations. Very smooth. Yeah, so there we go. All right. So let's talk a little bit about accurate compatibility. So pro two to pro three is not what I'm talking about here. It's this is more along the lines. If you've got, you've got a server or device deployed in the field, and it's using like the first the early version of your protocol definition. And let's say now you want to add some fields or change how it's set, you know, structure, but or whatever. What about stop that built in? And they just have a few rules to guarantee backward compatibility. And that is one of them is that those numeric tags that I mentioned earlier, you can't reuse those. Once you once you've deployed it, let's say I want to get rid of the temperature field. So that was tagged to once I've deployed that I can't reuse that if I even if I want to stop using the temperature field, it's, that's just part of the that allows the old deployed devices to keep functioning when they get a new message that doesn't have that field. So new fields are ignored by old clients. And likewise, if an old client sends a field that's been deleted, new clients or new users of the protocol will just ignore it as well. So that's kind of the beauty of it is you don't have to care what you don't have there's none of this. I mean, you can still do it, but you don't have to do an explicit check when you get a packet. Oh, is this a version nine of the packet, then do this and this. Instead, you just say, Oh, I'm going to read this packet. And if temperatures are great, if not, deal with that. But if you know upfront that you have to deal with the possibility of getting default or missing data, then, then versioning, you know, becomes not such an explicit problem, you just deal with it implicitly when you're deciding how to handle your packets from start. Okay, so and then the encoding efficiency. In that previous example, I showed there was, you know, nine bytes to send 12 plus the structure was in there too. The way they do that is by looking at the properties of the data and, for instance, like when you're sending integers, a lot of the times the integers you're sending are not two to the 23. They're, you know, 12. Yeah. So they, the guy Google, you know, figured, you know, that that's the common use case. So they use a couple of approaches. One is called bar ins, which is basically, if you're only, if you only need, say, a byte to encode a value, then just use a byte. And they do that by basically using the most significant bit of each byte to signify whether or not the value continues. So and that and zigzag encoding. So that's for when you have a situation where your values, excuse me, then jump around zero. So negative and positive. Zigzag encoding is another way. And you can read the details and get help about how it works. But basically, the idea is you keep all those small magnitude numbers should have small encoding. And as I mentioned before, the selection of the data type makes a big difference in the how efficiency, how efficient the encoding is. So if you use an int 32 or 64 for negative values, the protobuf is going to encode those using 10 bytes each for the 64 value in 64 case. And maybe even an int 32 case. So definitely you need to make some, you know, intelligent decisions about your type of data, your representation of your data. And then also the fields that you use that you expect to appear frequently, those should have lower tag numbers than those that don't. So like the first 15 tags should are going to be your most frequent tag fields. Those will get encoded using one byte for the tag information. And then the next step up is to 2047. So your first 20,000 or 2047 field values should be your most common ones. And then again, that, you know, it does matter. Okay, let's do another command line thing. See if I can use the slag in this time. Correct. Yeah, yeah, they don't have to be sequential. They just they just have the the those few restrictions about I think you have to have a one one that's got tagged one. And then but after that, you can go nuts with ordering. Yeah. Okay, so another example, showing the efficiency again. So let's see here. This one's a little more, it's not going to be. I'll just cat this here. It's a bunch of different ways of encoding that same message. What it boils down to is, I've got a proto here where I want to show how choice of data type can affect the encoding efficiency. So what I've done is I've taken that same sensor data guy again, and I've introduced this new sensor data history message type. So basically, I can send a bunch of a bunch of sensor data messages all in one group. And in this case, I've written a little, you know, a little Python script to just bundle up a package of 20 of them. And just to show the efficiency, how the efficiency can vary based on the encoding that you choose for the values. So in every case, the it's sending 20 of those sensor data messages. And again, in a C++ world, that would be the equivalent of 12 bytes of data. So you've got your temperature is four bytes, the humidity is four bytes, and then the four bytes for the unit type. So in the first case, it's, I've used, I used a float to encode them. And also I varied the, the kind of range of values just to show you how the encoding can vary based on the values that you're putting in those fields. So here in this case, I've got some negative, you know, a lot of my all my numbers are negative. And so you can see sending this floats is still 220 bytes for 20 were 20 of those sensor data. So it's 11 bytes per per packet per per struct, which is still less than the total size on on on encoded or uncompressed. And it's got the the structure in there as well. But then you see if if I use asset 32 instead of a float, now I'm down to nine bytes per per sensor data message. And but now if I go down to now my all my values are positive. So now you can see the difference between using an int. So up here, I was using an 32. And I was putting all negative values in there. So I was ending up with 17 bytes per message. Now, using the right data type, I'm down to 180 bytes, so, you know, nine bytes per per sensor data message. So the point is data data type matters. And then get when you get down to really big numbers, you don't, you don't see a lot of say, yes. Okay, yes. So the question is, what's the difference between essence and int? Is that Oh, I was saying the int in 32 versus this in 32. Yeah, so yeah, sorry. Thanks for a quick thank you. Yeah. So, yeah, up here, all my all my values that I'm putting into my temperature are are negative. And I'm putting it into an in 32. So that's, I'm it's it that results in this really efficient encoding. But down here, my values are all positive, because they're right around 50. And so I, the the using in 32 doesn't penalize you there because none of the values have gone negative. All right. Oh, so not yet. Not questions yet. So now I wanted to give a more in depth worked example of how this all kind of rolls together. So I've got over here, I've got the raspberry pi and an Arduino mega, although I got working on a Uno as well. The point being, this protobuf is running on both of these. And you know, it's, it's this is a mega. So it's got 32 megs of 32 K, no 32 K of storage. This protobuf plus the whole ethernet stack, takes about 10% of the storage on on that Arduino. And then on the Raspberry Pi, it's, you know, we're not so concerned about storage and such. What what it's, it's, it depends on the language. So like, oh, well, so for C in particular, they have there's a there's a third party package called nano PB. And that's what I'm running on the Arduino. That's, I think they said three K, this is the, you know, the size of the library on the on the embedded device. So this, yeah, so that Arduino is running C, C plus plus, you know, it's the it's the Arduino language, you know, which we all know is C. And that's what's running on there. So all right, yeah, so what I've done here is I've got basically an extension of that sensor data class. So let's say I've got I've come up with a simple, I got my little network of Arduino is deployed all over my farm to give me temperature and humidity readings throughout the day. So and let me just show you what that looks like. How's my time doing? Okay. Okay, so here's, I won't go into all the particulars. But basically, this is the the message system that I've set up basically set up. This is kind of like the top level message that is used to communicate between between my deployed Arduino's and my controller guy that's, you know, sitting on my laptop and or my desktop or whatever at home. And this is like just a simple protocol that I've set up so that I can when when the devices in the field want to connect to the controller, they send one of these connect messages. When the controller wants to tell them to do something, the controller can send them a command message. And right now, the only command type I have is report. So that's what that third report is. That would be like the the deployed deployed nodes would be using report messages to report their data back to the controller. And here's the report looks should look familiar, at least for the temperature and humidity part of it. There's our friends answer data. So basically, here's an example of nesting. So I've got my report. And within report, I've nested the sensor data class or message type. And then down here is an actual instance of the sensor data field. So this report message has two fields. There's a device identification field, which is defined up here, which is an ID and a name string. And then that's used for a lot of the so for like connect message. Basically, I can connect to the controller and that allows me to identify it, who I am so that can control and keep track of who's who. So that's, okay, that's all well and good. But so now, I've written a little controller class that basically sits there and wait in a thread for network connections. Now, because I'm a coward, I didn't do this with like wireless devices. So it's all, it's all hard wired ethernet. But you know, don't let that distract you because it's, it works just as well. In fact, even better over low bandwidth connections. But this is just so I had one last potential disaster for this talk. I get a Greg's photo, you know, he did all his wireless and it was like, real time debugging. And yeah, I don't have the heart for that. So so I just did it all wired. But you know, don't don't hate me for that. Okay, so anyway, I got my my controller is running here by laptop and it's waiting for UP connections from from anything that speaks that knows it's there. So let's see, I'm going to oh yeah, so I'm going to go ahead and so I've got, sorry, we know it's just set up. It just the first thing it does is when it powers up as it just tries to make a connection. So I'm going to just reset it now. And now, in a few seconds, we should see, you know, response coming to the controller that says, Yeah, yeah, all right, I got a connection from that IP address. And so now the controller is going to periodically send messages to the Arduino saying every second or I think every five seconds, give me a report of your current data. So you can see it's reporting temperature and humidity. I don't another power leading that I didn't have temperature sensors or anything like that. These are just random numbers. They're randoms. But they have variations. So it looks like it's realistic, but it's not. It's really not 27. Well, it might be 26 degrees Celsius. What's that? Okay, so yeah, that's neat. So now we got this this guy running and let's say now it comes along. And I've got I've got my raspberry pie here and I decide, you know what, that thing's got a lot of power. I can send a lot more information. So I'll send a little bit more. And that is not what this is. So this is fancy sensor. So if you go to my GitHub repo, you'll see all the code laid out. And you know, I think I called the Arduino one is called puny sensor. And this is fancy sensor and fancy sensor. So much more than puny sensor. So let's do the profile here. Cheat. No, it's called sensor version two. So now readable. Yeah, okay. I've added a few things to this protocol. So it now I've added this history. So I've got to denoted my changes for version two of the protocol with this comment here. Protobuf doesn't care about comments. So I've just put it there for illustration purposes. So inside of my device identification message, I've also added this new field keeps history so that the controller can act accordingly. Also, down, I've also added a new commands. So in addition to reporting data, now I can also tell the remote guy to report data history and clear your data history. So and then some adjustments here to the the sensor, the report message kind of scroll off there. But the important thing is now I've added this repeated sensor data data history field, which this repeated keyword tells protobuf, you can have zero or more of these whereas normally, like the field right above it data, you can only have zero or one. This repeated says zero or more. And then the top level message thing still the same. So, so like right now, if I run, if I run my fancy sensor, and yeah. So I'll pop over to this window. So you see it's it's recognized this new connection right there. So there's device at the meat. And the device name is a luchador. But you can see it's still only sending old, old data. So for here from 1337 or not old data, but it's it's only the controller doesn't know that this is a fancy sensor yet. So because I haven't upgraded the controller yet to speak the new version of protocol. So the new fancy sensor is already sending or potentially sending this new version of the protocol that the controller isn't asking for it yet. So now let's go ahead and kill that controller and run controller version two. So now let me reset this to get them get the old Arduino. You know, continue to send out its old boring data. But now okay, look at this, I disconnected my Arduino connected. So it's still using the old protocol. But it's the controller is reporting Oh, hey, look, it doesn't keep history. So now I know, when I talk to the old to the Arduino, I know not to send it things that are germane to keeping the history like these new commands like send your history and clear your history. It could send it. It's just going to get ignored. So it's not like you have to carefully check for this. It's just more for keep things same on the controller side. But so now if I go to fancy sensor, and run version two, I should mention here that so that fancy sensor was actually running the old protocol as well. But now it's running the new protocol. So I guess I lied there before, but now I should see here. So now it's, it's, you can see the the old Arduino is still using old protocol. So the controllers telling it just send me your one data chunk. But now the the the Raspberry Pi is using the new protocol, which keeps history. And now it's every on the Raspberry Pi end of things. It's, it's every five seconds, it's taking a new data reading. So it's taking temperature humidity, rainfall, so cumulative rainfall and barometric pressure. So what's what's going to happen here is it's gonna, you'll see it start to building up and then it gets once the controller gets to a certain number of iterations through this loop, it's going to send a clear your data history message. And so you'll see, you'll see all those values go away and it'll get reset too. So yeah, so there it happened there. So get a report and sorry, catching up again. But so here's where it in the time that it sent the last report and they got the clear data report. The Raspberry Pi had already taken two readings. That's what it's to there. But really trust me, it works. Okay. Excuse me. Yeah, so that's okay. So that's, that's, that's what I got. Any questions? What? Oh, yeah, I guess they ran right up to it. Sorry. But we have a break after this. So if there's a lot of questions, go ahead. Well, a little bit. So it basically it breaks down to there's an initial bite and that's, that's got like the specifier of what the first field that's in this message is. And then if there's, if that message is there, then that message is data, or that's sorry, fields data is there. And then and then it has the next bytes or bits really that tell it, okay, is field two there? Okay, if field two is there, then here's field two's data and and so forth. So that's kind of a big note that means but back there. Well, so, so the question is, how do you know when you've hit the end of your message, right? Yeah. Yeah, so this doesn't solve that problem. It's so like for my examples, what I did was, I took that encoded payload, and I send an extra byte as the first byte of the packet, it's the size of the encoded payload. But I know it's so if it's a TCP packet, it gets busted up or whatever. If I read, oh, they're supposed to be 19 bytes, and I only got four, I'll wait to get the rest of them. Yeah. Yeah, that's, it's, it's not going to matter. It'll, yeah, it takes care of all the network and coding. Yeah. I just repeat the question, does ending this matter? No, it doesn't. Sorry. Oh, so you want to reverse engineer the, the, whether, yeah, you, I think there's enough information on the protobuf GitHub to, you know, that describes the encoding that you could figure it out. It might be a little tricky, though, with, like, fields that are omitted, you know, so because everything is optional. So if they only send the, you know, check for valid license key, you know, message every 20 minutes, you may not catch that. And, and you might not get the full protocol, but yeah, sure. I think you could do that. Yeah, so the, you're saying this is Warchar module for that. I think you have to have the proto file for that, though. I may be wrong, but I think that's what I remember seeing about that. But anyway, it should be possible, you know, you can, you know, give it enough time and enthusiasm. Yeah. So the question is, can you control the defaults, not the proto three? So that's one of the changes that I made from proto two to proto three. Proto two, you can specify an arbitrary default, which was a nice feature, but it made language generation a lot harder, at least for some languages. So now they have these rules like, oh, it's a, most numeric types of the default zero. The enum default is the tag zero default, is the default, and, and like that. So in my mind, that was kind of a sad thing to lose for, you know, between going from proto two to proto three, because I use proto two a lot in past stuff. But yeah, you get what you get. Well, just going on the proto buff GitHub, their comparison to like the equivalent XML parsing was that it was 10 times faster. So it's because it is so less wordy, you just have that many few bytes to chew through. And it's not doing any kind of fancy compression, you know, it's not, you know, applying some kind of, you know, convoluted algorithm. It's a pretty simple algorithm, so it's pretty fast. Right. Thank you for mentioning that, because yes, and, and so the benefit over that is you don't have to now keep, yeah, you have automatic parsing and, and you don't have to worry about, oh crap, I, we changed byte three to, you know, now it's two nipples. You don't have to do that. It's, it's all taken care of for you. And yeah, if you're an electrical engineer, yeah. I'm an electrical engineer, so I can, it's okay. Somebody who's raising their hand back there? Yes, no. Okay. Well, I'm going to be here a little longer heading back to Detroit soon. But if you've got any more questions, feel free to grab me. And like is it all the, I think? Yeah. Thank you. That's my, you can find all my stuff there. I don't, I don't do the Twitter, I don't do the Instagram, none of that stuff. So, you know, look me up on GeoCities though.