 Of course, as soon as I give the signal to begin recording, we get some people coming in. Well, you've ruined my nice, quiet room that I picked, so I would have no audience to interrupt my flow. So this is an evolution of a talk that I gave at Picon Australia earlier in the year. Did anyone see that talk at Picon Australia? Okay, that's good or bad, I don't know, we'll find out. So I have a much bigger time slot than I did in Australia. So the target of the talk is for people who haven't done a lot of computer science engineering degrees. So it's trying to educate people in state machines who don't really know what a state machine is. The drama that I've had is that now that I've got all this time, I have to go and do add on some of the more advanced topics. So the target is still very much people who don't know what a state machine is, and by the end of it, you might not really know what Python is, but I'm still very hopeful that you'll get something out of it. So a few things have changed since that conference. The company I was working for has changed names for reasons of transparency. I've decided that the talk, I should focus the talk contents a little bit, so I've added 20 slides. And I've got a few more graphics, and there's actually a little bit of color in this one. I have done my best to make my slides nice and easy to read, regardless of how uncaffeinated the projector is feeling today, which is fairly. So I need to be really clear upfront that the library I'll be talking about, Automat, is not my library. These slides and examples, they're definitely my fault. So what is a state machine? At a very abstract level, a state machine is anything, we'll deal with programs that has a variable that changes values. So you've got a program with foo, and it starts off being bar and changes to bats. That is a state machine. Not a very useful one, but it is one. So any program that's got variables that change is a state machine. So all of your programs are state machines, and the code that you are writing to modify the values of those variables, it is handcrafted, it is a handcrafted state machine. There is a subset of state machines that can be modeled in a structured, formal way. And the reason you wanna do that is because if you've got something that you can model in a nice structured way, it means that it's repeatable. And it means that the process of modeling, coding, and testing that can be done in the same way over and over again. So instead of coming up with the spoke code for your state machines, you can let the library do most of the work, and you can worry about the interesting stuff. Often what happens when we're coding something is that we're solving a problem, and we are writing a state machine, we don't really think of it in terms of a state machine, and it means that some of the problems of state machines aren't apparent to us. So if you've got one variable and it's a boolean, your state machine can only be in one of two states, or if it's Python, three states, because it could be nothing. If you add in a second variable, then you've got three by three, so you've got nine states. How many variables do your programs normally have? A little bit more than two. You get this wonderful combinatoric explosion where if you had to map out, I've realized now that I'm annoying the video person, if you had to map out all the possible states of your program and had to try and test each one of those states, you'd go nuts because it's just not possible. So part of the reason of doing a structured state machine is that you don't have to test all those possible states because you know that there's only a subset of those states that your machine can ever get into. There are state machines that model real-world things really well, and there's a lot of examples of real-world things that engineers have modeled as state machines, and instead of the... Essentially, you've designed something to fix a problem, and then you've designed a tool, and then that tool ends up designing the answer to your future problems. So things like traffic lights are really good examples of state machines. There's red, green, orange flashing maintenance mode. So they've got a certain number of states. Things like train guidance systems, for example. The way that most train systems are modeled is that you break all of your tracks up into segments. Over in Queensland, it's five kilometres segments, and in those segments, you either have no train or one train. That's it. You can never have two trains in the one segment, and it means that when you model it like that, it's really obvious when you get outside of your state machine and something's gone wrong. It's a really good way of being able to describe your problem domain at times. You can go to your customer, describe their problem in terms of a state machine, do it in a nice, pretty diagram, and then code it. And it means that you've got nice traceability from your pretty diagram that you've done at your requirements analysis all the way through to your implementation, all the way through to your tests that are generated at the end. From a coding perspective, what it means is that... The biggest advantage is that you don't have to write a lot of state testing code, and if you add in new states later on, Queensland probably will end up adding like a fourth colour to our traffic lights, for example. You don't have to go and modify all of your previous code. You only have to add in the new transitions from your old states to new states. So there are roughly 1,000 libraries on PyPy for implementing and helping implement state machines in Python. Automat came to me to be quite interesting because it solves a lot of problems in a particular way. It doesn't try to force state machines down your users' throats. When you use Automat to write a state machine, the users of your library class object, they don't see it as a state machine. They see it as a plain old Python object that's got some functions that you call on it that do things. And if you call those functions in the wrong order, you get an error back. So it really does that whole object-oriented thing of encapsulating the solution to the problem and hiding it within your object. It's by the author of Twisted. Twisted is a network protocol library for Python. Twisted historically, not so much these days, but historically is used lots and lots of callback codes. Twisted is an asynchronous networking stack. So it means that you send some bytes down the socket and then you wait for the kernel to tell you that they've been sent. Okay, I can send some more. And the way that you do that, at least in slightly older names, is that you pass a function that will get called on you. And funnily enough, when sending a HTTP request requires you to send a function, get a callback that you've connected, send another function saying, I can write the headers. Okay, I can write the headers. Send me another function callback when I can send the body of the request. Send me another function when the server has sent me back the response. It actually is kind of easy to get yourself lost in all that stuff. And the author of Twisted designed this state machine library to get themselves out of trouble. One of the interesting ones is TLS or SSL. One of the interesting protocol requirements that TLS has is that at any point in time, either peer, because it's not quite server, it's very much peer-peer, either peer can ask for a renegotiation of the underlying encryption keys. So you are reading from the server and the server says, no, I need a new encryption key. So you go, oh, okay, I was writing. So I'll keep what I was writing to you at the side. And then I'll start this new state machine over here to send, do the three-way handshake with the encryption keys. And then I'll send the data that I previously had. And you can very much get into a state where like one of the error codes for SSL write is SSL write once read. So your program is all set up to do a write and then the server said, no, you gotta do a read now. And the same thing with read once write. So it's really, really easy in seemingly simple applications to get into states that are very confusing. And instead of writing more and more code to handle all these states as they pop up, the methodical state machine way is to explicitly list those states and explicitly list those transitions between those states and get the code generated for it. So the example that I'm going to use throughout the rest of the talk is a kettle. All state machines have a very special state, the starting state. I'm going to model three states for a kettle. So it's empty, it's full of cold water and then it's full of hot water. Now, there could be lots and lots of states in there like it's full of medium hot water or it's half full of cold water. That's fine, and that's an ongoing thing. For the purposes of this talk, these three states provide enough interesting things to look at. But the important thing there is that we are, even when we're doing methodical state machines, we are modeling the real world. It's never going to be a perfect one-to-one match with the real world. So we have our transitions from empty to full back to empty. And if you think of the kettle as one big state machine, so draw a big circle around the whole of it, those inputs to the state machines fill, boil and pour. So those, if you've got a kettle object, you can think of those fill, boil and pour as methods on your kettle object. You fill it with water, you press the boil button and then you pour it. So that's the underlying state machine we're going to be using as the main example for the rest of the talk. I didn't say before, we've got plenty of time, so please ask any questions and make sure I remember to save the question back to you for the recording. So, it's a kettle, it's got three states, it's got three transitions. How hard could it be to write some code to handle that? Now, I apologize a little bit because I've had to get rid of all the white space to make all of this fit into one slide. But this is a, it's not a terrible first attempt at writing a by hand state machine. So if you see our init up there, I've decided to use two variables to hold our state. Whether or not, and they're Booleans, so they're nice and simple. Whether or not we're full and whether or not we're hot. You can see I've got the fill, boil and pour methods, so the three state transitions and they've been turned into methods. And I've got some code there that's like, you can only boil a kettle if there's water in it and if it's cold, you can't re-boil the hot kettle, all that sort of jazz. There are a number of errors with that. Example up there, can anyone spot any of them? Yep. The other one here is a pour after I pour out the hot water and give you back the hot water. I don't actually reset it to full, so I could just keep pouring out the kettle and it apparently would keep pouring out hot water for you. So it's those particular tests, so it's particularly the tests to make sure that you're in the right state before you accept a transition. Automat will handle all of that and it will also handle changes to new transitions for you so that you can't mark that up. So now I'm going to go through the example and how you would do it with Automat. So the class that you need is the methodical machine. We have a kettle and it's under machine equals methodical machine. So the under is definitely important. Any of the variables that we have with an underscore start with an underscore, it means that they're private. We don't expect public users of our class to interact with that. So it gets back to that whole idea of encapsulation where we are solving a problem with the state machine and that's internal private details that our customers don't need to know. We declare the three states that we've got. We don't actually have any code in these states. They are just functions there as reference points for the library. So we've got empty, full of cold water and full of hot water. And we use the machine decorator to declare them as states and the only interesting one is the initial equals true. As I said at the start with our first diagram, all of our state machines, they have to have a starting state. If you try to instantiate a kettle, if you try to instantiate any of the Automat state machines that don't have a start state, it won't let you. So you can have only one, you must have one and only one start state. And this is the code for the input functions. So the methods on our state machine. Again, these don't have any code associated with them. They are reference points that Automat will hang things off. So we haven't really done much so far. Like we've declared our functions, sorry, we've declared our states, we've declared our transitions. There's no real code hanging anywhere. This is how we wire up our states and our transitions. So the general pattern is at the top there. From our start state, upon receiving a certain input, we transition or enter into a new state. So from the empty state, upon receiving a fill input event, we go into the full cold state. If we're in the full cold state and we receive a boil input, we enter the full hot state. From the full hot state, if we receive a pour, we go into the empty state. So that's like the first real code that we've written. And that's the stuff that actually wires up our transitions in our states and links them together. So the only transitions that our state machine can go under are those three transitions there. If the user tries to do any other state transitions, they'll get an error. We've got an example of that coming up. Any questions? I had to fly here from Brisbane. I'm being a pain in the neck, yes. So in the normal case, you would use this kettle and I want to point out here that it looks like a popo, a plain old Python object. You instantiate your kettle, you fill it, you boil it, and then you pour it. The result of that is absolutely nada. There's no errors, there's no output, nothing happens. If you try to do something out of order, so we instantiate our kettle and then we boil it without filling it up. Now I've slightly simplified this particular traceback. I've gotten rid of any magical Python stuff in there. But you get back an error from the Automat library saying from the empty state using input boil, there's no transition. So we have not written any code in boil to say we've got to be full of cold water and otherwise error out. Automat, so we've declared the three transitions that our state machine can have and Automat won't let anything else happen. So all of that code that we had, like are we full, are we cold, excellent. Do all that, you don't have to do that. Where things start getting a little bit uglier is the methods that we have on our object. So we want to fill the kettle with a certain amount of water, for example. So what we need to do there is slightly modify our fill input. So we've got an amount argument that we've passed along there. Now we get onto outputs. So we modify the transition that we've added before. So when we're in empty and we receive a fill, we enter the cold state and one of the outputs that we have is save water. And all save water does is on our kettle saves underwater to whatever the amount that was passed in. So inputs and outputs are chunks of code that get run for us. So basically when somebody does a fill on an empty kettle and it's in the right state, all of the functions in output save will get run. And in this case, it'll be save water. So if we've managed to get water into the kettle, we also want to get the whole water out of the kettle. So here we modify a pour transition. We add the return water pour. To outputs and all return water does is returns self.underwater, which we saved in fill. So that part of it is quite reasonable. It gets a little bit ugly further on. So the drama here is that outputs is a list of functions. So for each of these state transitions, you might want to do multiple things. So the way that Automat does this is that it gives you a list of functions to run on successful state transitions. So we have initialized a kettle. We've filled it with half a liter of water. If only there was a nice library for writing units. Are the output functions run in the order you put them? Yes. So we filled our kettle with half a liter of water. We've boiled it and then we've poured it, but we've also printed out the results of that. And that results in a list with half a liter of water. So it sort of makes sense. We've put half a liter of water in and we've poured it, we've boiled it, we've poured it and we get half a liter of water back. But because outputs is a list, we get a list of results. So that's kind of where the Python Australia talk finished. So we're into overtime now. So if you finished your kettle implementation at this point in time, it'd be a little bit funny because when people pour the kettle, instead of getting back half a liter of water, they get back a list of half a liter of water. So what's going on with that? That might be okay in your application. If you're writing an engineering application and you've got three or four or five state machines all talking together, you might be totally okay with getting a list of results back. That might make sense for you. If you just want to apply in Python object that returns a number, this is the sort of mess that you've got to start getting into. And this is definitely where Automat stops being as Pythonic as it could be. So we haven't changed the return water function at all there. I'm just putting that there because we're using it. I'm adding a whistle to our kettle that goes off when the kettle is boiled. And I'm specifically doing that so that our outputs list has two values in it. The new thing is the collector. So what happens here is that when Automat receives an input, sorry, receives a poor event, is that a, sorry, is that a question? Nope. When, ah, sorry. When we're in the full hot state and we receive a poor event, we'll shift to the empty state. But just before that, we will run blow whistle and return water. And we'll take the results that we get from that and pass it along to collector. So collector as results will receive blow whistle, blow whistle, the result of blow whistle, and the result of return water. Now, as a function, as a plain Python object, it only makes sense for one of those things to have a result. So it's up to you to design your output functions to only have one of them return a result. Otherwise, if you need to return two or three things, then your collector function might return a tuple or it might return a dictionary if you're returning multiple things or it might return like a named tuple or something like that. In this case, we've decided that the whistle won't return anything and the only thing we're caring about is the amount of boiled water. So we've slightly generalized it, but basically we get a list of return results. Blow whistle will be a none and return water will be half a liter of water, for example, and we will filter that list, get rid of all the nuns, and just return the first element of that list. So this is our noisy kettle. We fill it with one and a half liters of water. We boil it, then we print the result. So we have a print tutut in the blow whistle. So, whoops. So we know that the whistle has been called. Obviously, if it was a real kettle, we'd actually hear the whistle, but I'm not gonna do that on the slides. I'm not very keen on presentations that have a kettle whistle. And we get a poor result back, which is just one and a half liters. It's not a list of one and a half liters, it's just 1.5. One of the other things that Automat has recently sort of grown some nicer tools for is the ability to visualize the state machine. And it's kind of an interesting thing because the way that you mostly do these things is you actually start off drawing your diagram like I did at the start and drawing arrows, then transferring them to code. This is kind of the other way around where you've got your code and you're printing out your state diagram machine from that. That doesn't look too good. Apologies, I haven't actually looked at that on that projector before. And I should have turned on the black and white mode. So I'm not a very visual person. You'll see the border around empty is slightly thicker. That's indicating that it's the special start state rather than just saying, you know, start with an arrow next to it. We've got our two other states, the full hot and the full cold. We've got our transitions, boil, fill and pour and we've got the outputs associated with them. So it's one of those things like if you're not entirely sure that you've coded up your state machine in the right way, the visualization might be able to help you figure out that you haven't done something wrong. But it's the sort of thing where you can sit down and go through those state transitions and make sure that you haven't done obvious things like if there was a state sitting off at the side that didn't have any transitions going into it or it only had transitions going into it, that you couldn't get out of it, for example, that could indicate a bug depending on what you're modeling. There are other things that you can do with Automat, so it will, so if you're doing an online thing, an online thing, excellent. I started doing computers before there was this whole networking thing, I'm sorry. It's just big bad fad that I'm waiting to go away. If you're doing an online sales system, a person logs into a website, they have a login session, they've got a cart associated with their login session, they're adding a couple of products to that. Maybe they get called away from the computer for whatever reason and that cart times out, they haven't touched their computer for 20 seconds. You've got a state machine there, so you've got the not logged in state, you've got the logged in state, you've got the cart, you've got each of the products being added to the cart. If you're using an Automat state machine there, if that person times out, you can serialize that state machine into a database record, write it out to the database and the next day when that person logs in, you can deserialize that entire state machine back up and it can have all of their peculiar shopping habits in the cart. So all of the things like discount vouchers, whether or not the person has gone to other sites before they've come to your site, whether or not they've followed a referral link, all of that stuff can be modeled as a state machine and can be stored in the database and brought back when you need it. That is the Automat GitHub URL. They do have some quite reasonable docs. I was more going for docs for people who've not used state machines at all, so I hope that was helpful. And I probably have lots of time for questions, I think. So the visualized graph and what formats it can go in, you can definitely output that in a much more, I'll just repeat the questions, it's cool. It definitely has a lot of output options. So there's SVG and JSON, so JSON might be the interesting one if you were going to do some particular things on it, like if you were going to actually try and parse it for having data and stuff like that for sure. Oh, yeah. So can you nest state machines? You might want to nest, so I'll try and describe nesting state machines. Without trying to cause too much of an international incident, I suppose. When engineers are coding things, it's very common for engineers, and I'm meaning electrical engineers here, it's very common for them to use state machines to describe everything because it's a known system. It's a process that you can follow and you can take your analysis and derive a correct system at the end of the day. One of the things that you might want to do with that is that if you've been in the industry long enough and you've seen a particular tool or a particular widget, you might have a state machine just for that particular widget. And when you're solving a larger problem, you might want to reuse some of those widgets and you might want to reuse some of those state machines and draw them together. There's no explicit support for nesting state machines in Automat, but that's sort of okay because if each of those state machines is just a plain old Python object, then they can call any Python objects they want. So there's nothing explicit in there, but you can do it yourself. Sorry, a whole bunch of hands went up prior. So that will, so how do you get the results? So when you run Kettle.poor, the result that it returns is the list of results. So in this case, when the transition from full hop to empty has happened, it will, Automat will run blow whistle and return water, and it will then pass those along to filter out none and then it will return those results. So from a user's point of view, when you run Kettle.poor, you will get back 1.5 as the result. If you don't have a collector, you'll get back a list of results that you've got in outputs. So you don't, so, no, no, that's fine. So one part of that question is, yes, I'm trying to. No, no, no, it's not a bad question all. So 1.5, the question is on validating inputs that you're parsing. So like if it's a two liter jug and we accept three liters of water, how does Automat help with that? Short answer, it doesn't. So you still have to do some checking there. The other way of dealing with it though is that you could, it sort of depends. Like if you're modeling a industrial kettle that self fills, it will have a sensor and it will model that sensor as part of your state machine. And part of the state transitions that you've got is that you can only keep the tap on while the sensor's off. So it sort of depends. So in this simple example, there's nothing stopped like if the kettle has a five liter capacity and we put six liters of water in it, there's nothing in there that will cover that. For your input validation, yes, because it is just, at the end of the day, it is just a Python object. If you want to validate your inputs and raise an exception, that is totally a thing you can do. So I've not used it directly. I've mostly used it through Twisted. So I do use Twisted for work problems and that's basically where I've seen debugging a Twisted problem when they come across this state machine thing and then you've got to work out the state machine thing to figure out what's going on. So I've not used it directly for my own problems. No, I've only used it through Twisted. And when they, so essentially Twisted is one of the older Python projects and they are at a point now where they've got some really gnarly bugs and they're trying to fix these bugs and one of the things that they're doing is rewriting a lot of their code in terms of state machine so that they don't have if statements at the start of their functions, making sure they're in the right state, making sure the parameters are correct. They make the state machine do all that and they have found a number of bugs by doing that. Automat does not have an end state. No, there's nothing like a particular end state that they have. There's nothing, I mean, it sort of depends on the way that you want to do it but you could certainly have the, like the easiest thing would be just have an internal variable and set it and then check it later. I have all the time in the world. Are there any automatic test generators? So they have some built-in code that helps generate tests but it's not quite there yet. But the thing is that it means that because you're using a library, what it basically means is that when you've divided your system up into this way, you can target your testing at the particular states. Instead of having a plate of spaghetti code and having to work out all the different ways of testing that, you know that the states that you have to test, you've got all your three states and you've got all the transitions coming out to and into those states. So they do have some support for generating like a list of functions that you'll need to fill in. So it's like, you know, these are the states and these are the transitions that you'll need to test but there's nothing there that automatically does that. They have some support but not full support. Can the state itself return the outputs? So one of the interesting aspects, so the way that Automat is designed is that the resulting object that you have is meant to look like a plain old Python object and the user is not meant to know that there's a state machine underneath it. So there is no simple way of asking an Automat state machine what state it's in. They've deliberately made that as hard as possible. So there's no simple function that you can call on Automat to say what state are you in? Are you full, empty, or whatever. And they've done that specifically so that you don't end up producing a kettle that you have to drive like a state machine. They want your kettle objects to be driven like a plain Python object. So there's no real easy way to get the current state of a state machine. I've sort of avoided your question there. I mean, you can make the output functions return anything you like but they make it very hard to work out what the current state of the state machine is. If you add a particular function that only gets called for particular transitions, that's probably the easiest way of doing that. So there is, there's some stuff that I haven't looked at yet which is some tracing code that they've got so that if you're trying to debug a problem and you've got a state machine, it'll actually show you what state it's in, what inputs it received to change to another state. So perhaps the tracing code is the right place to look at for that. So can you also look at, so can you also look at the valid transitions out of a state? Yes. So you can output that visualization in any of a way of a number of things and you will be able to parse that for sure, definitely. Yeah. So I've not looked at the JSON form but I presume that it's mostly going to be through verse. So it's almost certainly going to be source destination. So you're probably going to have to read it and do a reverse index on it. Yeah, yeah. Oh, sorry. I was just thinking about what you said about the use of visualization where you might want to see a state that may be one that does in fact have a form of static analysis which allows you to say which highlights. It does have some error checking like that. So if you try to instantiate a kettle that doesn't have any transitions or if you try to add in like a repeated transition, like if you've got two states and you add in two transitions that look exactly the same, it will error out on those things. So that's runtime but that's at instantiation time that it will do that. So there is some form of static or early runtime checking on it but there's no tool that you can say here's Python run my py over it and work it out. There's nothing like that I don't think. What more do you need than that? How do I document the kettle? So because it is a plain Python object you can put all of the documentation code in fill in that doc string there. So the API for your kettle is fill empty in boil and you can add those doc strings as you see fit. So you can document it there. Okay, thanks very much.