 Awesome. Hello everyone. Welcome to the last talk of the second day of Kiwi Picon. Are you all still awake? Excellent. I'd like you please to join me in giving a super enthusiastic welcome to Ben Shaw, who's going to talk to us about microservices. We're good, Sandman. Sweet. So, Picon, I love coming to Picon because I get to think up crazy ideas and then investigate them and then hopefully do an entertaining talk for you guys. So, this year microservices, as it says, is HTTP the only way. This talk's kind of long, so hopefully I won't go over time, but if I do, you guys can just suck it up because you got nowhere better to go now. So, what's a microservice? I'm sure there's an official definition for microservices out there, but for my purposes, I'm going to say that microservice is an application that reads some data, processes it, and then outputs the result. Now, I'm not defining where the results get output to. Maybe they go to a file, maybe to a database, maybe emailed somewhere. It doesn't matter. The applications should operate in only one domain. And what I mean by that is in terms of the users, this microservice should handle user authentication or this microservice should handle message storage or something like that. And the applications behavior should be simple and describable in just one sentence. So, I don't want people to try and get around that by saying, having a stupid long run-on sentence like this, a user can sign up and log in and post a message and follow other users and sure it's posted, no, let's not do that. So what is a microservice not? I'm not including definitions that say a microservice needs to use only one database table. If necessary, you can have as many database tables as you want. I don't think that a microservice necessarily has to run on a different server. I don't think a microservice needs to be written in a different language. You can have your main service or a bunch of microservices all written in the same language. It could be easier for your DF team that way. And the last point, which is the whole point in this talk, I don't think it's necessary for a microservice to communicate over HTTP. In this talk, I'm not going to cover the pros and cons of using microservices. This is probably one of the consequences you have. GitHub gets paid for more private repos and finer replaces harder, but it needs to be up to you as to whether microservices are what you're going to need for your application. So to illustrate all this stuff, I needed an example application. And I've come up with something with these features. So it's just going to be a web app where users can publicly post messages. And it needs an authentication system so users can sign up and then log in. Users will be able to follow other users and receive notifications when that user posts. And the maximum message length is 141 characters. So this probably sounds quite similar to something you might have heard of with one important difference. So my service is called Uwta or Twitter plus one. And just to start off, I'll give you a couple of screenshots. So welcome to Uwta, post a message. Hello, Kiwi Picon. After I posted, well, you can see a list of messages. And then after I post, I get a message saying you eated. So if anyone here works with Twitter, please don't sue me in case I accidentally say Twitter or tweet in this talk, you know what I mean. So we start off with a monolithic application, which will then break down into microservices. Starting off, it's really simple. That's the architecture of it. We have a Django server, Django front end, sorry, and a database, a single database to store all the stuff. And there's three main tables that we're interested in in the monolith. There's a UI table, which will store the messages. There's a user profile table, which stores extra information about a user. In this case, all we're storing are the follower relationships, which user is following which other users. And finally, there's a user table, which will store user ID, username and a hash of the password. And the number of the URLs that we have and the views that we have are also quite simple. And we can break them up into three sort of categories. The first few there are all about viewing Uweets. We can view Uweets and post a new Uweet. You can search for a Uweet, which I chosen a different name because it's really hard to say. You can view all the Uweets for a username. And you can go to this user Uweets URL, which will show all the Uweets of the current user that's logged in. The next two URLs are for following and unfollowing. So if you post to follow username, you'll add that username as a follower of the logged in user. And likewise, if you hit unfollow, you will remove it. And then there's a couple of accounts, a couple of URLs dealing with accounts. So there's a register one and a login URL. So next we've got to take this monolith and somehow turn it into a set of microservices. So using those three sort of URL schemes is how we're going to look at, or the three ways the URLs are separated is how we're going to look at the monolith partitioning the monolith. So we'll start off by moving all the Uweeting stuff into its own service. So the Uweeting microservice, it's one sentence definition is that we need to store, retrieve, all or by user and search for Uweets. So that's pretty simple. We have a single database and a single message table in which we're storing the ID of the message, the user ID of whoever's posted that message, the message content and the date it's been posted. This table is pretty much the same as it was in the monolith. We've sort of just taken that table and moved it into its own separate database. Then to actually retrieve stuff from this database, I've created a single file with all the database API stuff in it. So this is just called db.py. At the top of it, it's just got a bunch of SQL, can we db connection stuff and ORM definition for the message. And then I expose four functions or four methods for retrieving messages. So the first one is actually for posting message. The second is for searching messages. It takes a search string and returns a list of messages. We can then list messages, which will retrieve all the messages in the system, or we can just list user messages, which we list messages for a specific user. So our architecture did look like this before and what we're going to do is separate it out into the message service, which is on the right, which is going to be, we're going to talk to over HTTP and use Flask for serving. So the HTTP API for that is fairly simple as well. It's a single file app, which is called app.py. It doesn't bother with authentication. It's rest-ish. I'm not going to get into semantics with that. And it talks JSON. And this is basically the URLs that we're exposing for the app.py. If you hit messages slash an integer, it'll get the user messages for that user ID. Messages search will hit that search messages function. Just going to slash messages will list all the messages. And if you post a message slash posts, it will post a message. So we take a look at the Django view that we actually have for getting the messages and rendering them. Then moving to a microservice because of the way this Django view is written, I haven't actually had to change this at all. I used a message facade, like a message service facade. And I can retrieve that just by calling this get message service function. And that just looks into the Django config. And depending on if you say use local messages or use a microservice, it'll pull back a different class. Then we just get all the Uweets with the all messages function. And just send them off to the Uweet list.html file to be rendered. So looking at this message facade a little bit more. The Django RRM1, which is the original is at the top. And we just get all the messages using the Django RRM. Getting the messages over HTTP is easy. I use the request library. We get that flask URL in JSON format and pull the messages out and then send it through to this transform raw dict function. So due to laziness, due to the demo simplicity, the transform raw dict function adds properties that the microservice return values are missing. So yeah, for simplicity, the microservice is just returning, for example, the user ID of the user that posted the message, but no other user information. And so this transform raw dict function is going out and retrieving the user from a local database. So in reality, what you'd probably do is have your message server return all the information that you need rather than just the minimal amount. And it also looks at a date string and converts it to a datetime object. So the advantages of talking to your microservice over HTTP is that it's well supported. There's so many libraries out there now and make it really easy just to create an HTTP interface for your app. I mean, my flask file is only a few dozen lines maybe. And it is the defective solution. So I was talking to a colleague of mine and I said, I want to do a talk at PyCon about microservices. And he said, what, do you use flasks to write your microservices? And I thought, well, it's interesting because people always just assume that you say microservice, you mean HTTP. And there's a reason for that. It's like defective solution. And you get security and authentication built in. So to secure a web app, you can chuck an SSL certificate on top of your web server and it's got SSL pretty much. You're probably not going to have any networking issues. So your firewall is probably open at your organization just because HTTP is so common. And if you need to do load balancing or proxying, that's a really well supported solution as well. And it's synchronous. And I view this as an advantage because you've got a synchronous request basically coming into your front end server. And then you're making another request, a synchronous request out again. So you've sort of got this whole thing that fits together really well. You're not having to make asynchronous calls synchronously and that kind of thing. There's some disadvantages of HTTP, though. There's overhead with sending headers. May or may not be an issue though. Load balancing is not magic. I mean, where there's if you have a load balance, so you need to tell it what servers it should be load balancing for. And it needs to have some way of knowing which those servers is busy and how it should be receiving messages. How it should be routing messages to those servers. That's all configuration. It's not magic in terms of picking the server that's the best for handling the resource at request at that time. And I think being synchronous can also be a disadvantage. And especially if you if you have a message that you can handle in an asynchronous way and you're forced to wait for a response from a server. So that's what I want to talk about next. So we go back to the partitioning. The next thing going to look at is partitioning this notification service. The one sentence definition for this is that when you use a post that you eat, we need to notify all their followers. I'm going to do this in this case by using a message queue. So message queue is a real simple. You put stuff on a queue, and then you have a number of workers who read messages from the queue. In this use case, the queue is a message sync. So a message is going into the queue and disappearing and then the front end can forget about it. We're not trying to read stuff back off the queue again. And I decided to use kombu as a message transport. You probably heard of celery and underneath celery. And it's also part of the celery project. But underneath celery, kombu is the message transport there. And it doesn't really matter. But in this case, I was using redis as the message broker. And I decided to use kombu instead of celery. It's because in theory, I haven't tested this, but in theory, this should allow us to be cross-platform. So you can have your microservice written in a different language to be able to read these messages. So this is how architecture is now. And this is what's going to be. So the way that the message sync works is that Django will generate a notification, send it off to the message queue. Message queue, the Notifier service will then read that notification from the message queue and then send it out somewhere. That notification is not being sent back up to the Django end. So using producing messages with kombu is pretty easy. I just created a producer base class, which will take a message that you give it and publish it onto a queue. And then I have subclass that just adds this Notifier post message, which is just a sort of helper function. But all it's doing is building a message from the poster follower and content arguments, and then calling put message. So this is on the front end on the Django side. So after a message gets posted in Django, we create a Notifier, which is an instance of the Notifier producer, and we tell it which connection and queue it should be using. And then for every follower in the profile, we just notify that follower that someone's posted. Then to actually do something with this message, this is on the in the microservice side. I've written this Notifier service file. And we can just import this consumer mix in from kombu. And we can just create a message consumer subclassing that. And we just need to implement this on message method. And every time a message gets read off the queue, this will do something with the message. In this case, we're printing I received a message. In reality, you'd probably want to email the user or something. And then you just need to make sure that you acknowledge that message at the end. Otherwise, the message will get back onto the queue and get used again. Second part of this file is to actually run that service. So I create a connection, create an exchange and a queue from that exchange. And then I just create an instance of that notify consumer and run it. And so every time I run one of those, it'll just sit there waiting for a message to come onto the queue. As soon as it's got a message, it'll pull it off, do whatever with the message, and then go back to waiting. If there's already another message on the queue, it'll just read it off straight away. So the advantages of a message sync, so not a back and forth message queue, but somewhere where we're putting a message in and it disappears from us, is it's basically said and forget. You can take your message, put it into the queue and just forget about it. And scaling up is kind of magic. By that, I mean, if you need more notifiers, you can just create more notifier processes. And when they're not busy, they'll be reading off the queue. And they won't be sort of forced to try and handle a message if they're not ready. And it's asynchronous, as I discussed before, you don't have to wait for a response to come back or anything, just fire it off. But there are some disadvantages to using a message sync. You need to have a message broker that you can communicate with. So if you've got firewalls in place or whatever, your consumer and your notifier both need to be able to talk to that message broker and it's not as ubiquitous as HTTP, so there might be some configuration issues there. And having this intermediary service, the broker, it does add a level of complexity for debugging. So you need some way of inspecting the broker and finding out what's going on in there if things aren't working as you expected. And a side effect of this is that it gives you a false sense of security because your application, it's your front end application, will receive confirmation that a message has gone into a queue, but there's no guarantee that there's actually a consumer on the other end reading off that queue. So what can we do about this last point? What can we do to make sure that we know that a message has been read and has been dealt with? So we'll talk about the third partition here, which is the authentication and how we're going to use a message queue but be able to go back and forth with messages. So the definition of the authentication microservices, microservice, is that it should store new user registrations and then authenticate them with a username and password. Again, we have a database with a single table that stores a user's ID, username and a password hash. And similarly to the messages service, we have a db.py file with the SQL economy stuff and then three methods as it exposes. A create user method, which will obviously create the user. We can authenticate the user with a username and password, which will return a user object. And we can get a user by ID, which again will return the user object. So our architecture looks like this. When we add the all service into its own microservice, it'll end up looking like this. That's kind of getting ugly. So what I didn't tell you that I did was that I also rewrote the messages service to be its own... Sorry. I rewrote the messages service to communicate with a message queue as well. It was using Flask. Now it's using the sorry, architecture looks like this. And it's quite nice because you sort of have... You've got Django there and then a message queue and then a bunch of different consumers running off it. So it's real nice. You sort of have one place we got to send off any messages you might have or any kind of communication is all done this one way. So just to talk about how the steps I needed to go through to get two-way communication working using this message queue. So normally we have a producer and a consumer on a queue. And the producer will say I'm putting messages on the queue and the consumer will say I'm reading messages from the queue. So if the consumer wants to send a message back to the producer it'll go something like this. Here's a response producer. Thanks for the message because it's just reading any message on the queue. So to get around that we need to have two queues like this. A queue for sending from the producer to the consumer and one for sending back from the consumer to the producer. That's not quite right because we have a producer and consumer at each end. So this is actually like Django to a service and then service back to Django. Right so time for some hacks. So making a message queue synchronous. As I said we need two queues. One for requests and one for responses. So there's an auth request queue and auth response queue with a producer and consumer at each end. So if we're going from the web front end across to a service this is easy. Putting into a queue a number of services reading it off. Going from a number of services back to a single web process is actually kind of difficult. We need some way of identifying which returning message belongs to which web request that's coming. That's easy. I've written a new method called message request which will just add a message ID to any message it's putting onto the queue. So it gets a request dictionary and then it just generates a UID for the message ID sticks it in the request dictionary before it puts it on the queue and then it will return back the message ID. So we have a message ID so we know which message we're waiting to get a response for but now we need some way of waiting for the response that we actually expect. So here's some more hacks. We're going to use some shared variables. So these are kind of like singletons I guess in a way. We have a received event that we use for letting any consumers know or any requests know that a new response has come in and then we have a single consumer that's going to consume the response and just hold on to them until they're ready for the request to pull it off. We then start the response consumer and run it in a different thread. So what this response consumer actually does is store new messages that come in. So when it gets a message it looks at the message ID and stores that in a received messages dictionary and then it sets and clears an event so that anything that's listening for that event will know to check and see if the message has arrived. So when a web request needs to wait for a message it uses a new synchronized message call function. So it'll go off and call whatever message function it's passed in which will give it back a message ID. Then while true you know that's going to be good. If the message ID is in the list of received messages or in the dictionary of received messages we'll get the response we'll pop it off that dict and then return that response. Otherwise we'll just sit there and wait for that event to be set and as I said it's this shared received event that we created before. So hopefully this should be sitting off some alarm bells because we've got global variables which are maybe singletons. This will only work if you have a single Python interpreter. It has cross thread data access so luckily we have the GIL. Everyone's been trashing it today but finally it comes through and there's probably a race condition in there as well. So yeah it turns out they're going from one service so what I've tested going from multiple services back to one web process that works okay. Going from multiple services back to multiple web processes that's probably going to be a terrible mess because we rely on a on a singleton being there to read the message. Unfortunately all of these are running would be reading from the same queue so you don't know if your web process is going to your web message is going to be consumed by the right web process that the request is at. So how could we make this work? Maybe a return queue per process maybe dynamically creating a return queue for each request maybe put the results in a database and read from them or maybe just use salary because this is what it's designed for but that is provided that you have Python code running at the other end. So advantages of message queue you get magical scaling in one direction but for it to work you need terrible code that works only with a single Python interpreter and you have to handle your own security you can just use salary. So another transport we could look at using is just raw sockets. So go back to our authentication architecture it looks like this we use again I wrote a flask front-end that I didn't tell you about but there's a flask front-end for authentication which we talked to over HTTP so all we're going to change in this example is that instead of talking to it over HTTP we're going to use a use the TCP socket. So writing a socket server is pretty easy this is basically the example code from the Python docs you create a socket you bind it to a port and you have it listen and again this is example Python code you create a socket you tell it to connect to a port you send it a message you receive a message back and you can close the socket. So all I'm doing here is sending JSON over TCP so if you've done any sort of raw TCP programming before you know that you need to have some kind of format to tell it how it should marshal your messages so all I'm doing is sending four bytes at the start which is the length of the message in ASCII so hopefully the message is less than nine nine nine characters long and in real life you'd come up with something better than this and then to receive that JSON just read in the message length again by reading in four bytes turning it back into an integer and then until you have that number of message and until you have that number of bytes just keep reading then to code the JSON and return your your message so raw TCP is actually has a really is actually really easy to use it has a low overhead like I said it's easy to code the python client if you want to do load balancing and you can use something like haproxy and it's synchronous so again it kind of fits into that HTTP synchronous workflow type thing there's some disadvantages though again firewall issues unless you're talking of a port 80 may have to open up firewalls and you've got a diy a lot of stuff there is a socket server library which does make it easier but you do have to sort of handle your own security and threading and it is synchronous which could be a disadvantage so I like to so that's TCP sockets um what about using UDP so if we look at the notification microservice again where they when notifying users when someone they're following has you eat it maybe that's like less critical so that the notifications that we're sending out they don't need a response they're just one way message sync type thing and if some users sometimes don't get notified of some posts maybe maybe that's not so bad so what about using UDP so before each um using Django to talk HTTP to flask let's change it Django send UDP uh packets to a service so writing the UDP listener is easy um I don't call it a UDP server because it's not really a server going back and forth it's just listening for stuff coming in uh the run server is slightly different we're creating a sock underscore D gram um server TCP socket so this is a UDP socket we bind it to a port as with the TCP one and just start the message loop so well to read a message from the socket and print it out and the received message received jcel messages um it's pretty simple just reads in a one or two full bytes again the first four is the length of the message uh it passes the rest of it as json and retransit the UDP sender is quite easy as well uh it's pretty much the same except that we're not binding to a socket we're we're just creating a UDP socket there um message sending is the same except we're not sending to a bound connection we're just firing out a UDP packet to a machine on a port so advantages of the UDP socket is that there's a little bit less over here than TCP um again it's it's easy to code that is pretty much python example code as well it's faster than UDP faster than TCP uh and it is asynchronous so when you want the asynchronous stuff good you can just dump stuff in there and forget about it some disadvantages of course as I mentioned UDP is unreliable and again you've got a DIY your uh your stuff socket server again can make it easier uh you need to enroll your own security but perhaps you could sign the message and then inspect that at the other end and make sure it was what you're expecting maybe that would be okay uh and you have to handle your own threading so just some other thoughts that I um things that I noticed while I was working on this project you sort of naturally tend towards using the facade pattern so I had a bunch of set methods for db access and I just switched out the transports for sort of interacting with that so my db.py file didn't have to change I just created for example an app.py file or a uh or a service file or whatever so that was cool and I noticed that I did a lot of code sharing or duplication so despite everything being the same git repo I treated all these services and stuff as separate projects and therefore I didn't want to have any cross project imports so the easiest thing for me to do was to copy and paste some code so for example the db creation code is is duplicated um the message sending and reading code is is duplicated and the socket uh json socket writing stuff is duplicated but in real life hopefully you would as I did notice that this was happening and split your base classes and libraries into separate projects as well and manage them separately with their own repos and and have proper libraries and stuff so to conclude is HTTP the only way no but it's definitely the easiest that's the end there's github link if you're interested in looking at this all my crazy flash gaps and stuff um yeah thanks so any questions we have time for a few excellent questions I don't want to put you all off but it better be a good question anyone okay it can be just an okay question up the back