 Helo. So, I'm Matt. I am unashamedly a geek, as are I think just about everyone here. I have many levels of geekiness. I've always liked model. We're always, at the moment, I do a lot around observability and monitoring and things like that, so drawing pretty pictures and interacting with APIs and pulling all the data off. This project goes back a good few years. It moved. Excellent. That means it's working. Keep an eye on the train, by the way. It will move back and forth, hopefully, if everything's working throughout the presentation. If it starts to look like it's going to fall off either end of the track, please shout. Cos the one thing I haven't put on here yet is a buffer stop. So, anyway, I've been mucking around with model roles for years and about nine or 10 years ago Network Rail opened source of the load of their data and I built a website for train spotters. The idea was that on your phone you could look up what your nearest train stations were. It would tell you how many services were going through them during a certain time of the day. You could then go off and you could collect the locomotive numbers or the wagon numbers or whatever it was you were interested in. I called it lunchtime trains and I didn't think anyone was using it. So I went to shut it down and loads of people went, whoa, no, don't do that. But unfortunately I couldn't afford to keep it going. So in the back of my head I've always had a dream and the dream is this, a completely autonomous model railway using these network rail feeds to move trains around the network in real time. Okay. Stations are big. My house is not. Therefore, we have a small test track. There's actually an even smaller test track that's in Engage, which is behind it. But a lot of my stuff is in storage at the moment for reasons I won't bore you with and therefore I couldn't actually find my Engage locomotive to demonstrate that side of it. So what do we need in order to get the stuff from Network Rail controlling the train? Like you have already seen, I'm really glad that worked at the beginning because I'm not convinced that it's working at it. First of all, we need the data feeds. Then we need something that will convert that into something useful. Then we need some kind of commander control software. Then we need a hardware interface to take the stuff from the commander control software and send it to the track. And then finally we need the track itself. So in my particular setup I'm using Network Rail for the data feeds. The data converter is some custom code I've written that is available on GitHub and I'll be sharing the link at the end of that if anyone's interested in that. The commander control software is something called JMRI. It's American so it's the Java model rail road interface instead of the model rail way interface. And then finally we've got a hardware interface from an organisation called Merg. I should have asked this at the beginning about how many people in here are interested in the model railway side of this. Quick show of hands. Okay that's a significant proportion. How many of you are interested in the network rail side of things? Okay there's definitely an overlap in that group. But to prove the point how many of you are interested in both? Yeah that's like half the room. Awesome. I knew this was the place if I was going to give this to you I knew that this was the place if I was going to give this talk anywhere I knew that this was the place to do it. How many of you know Merg? The model electronics railway group. One, two, three, four. So maybe five or six of you. For those of you that don't Merg is a community that is global. I believe it started here in the UK. I've not been a member for very long. But they do all kinds of kits that you buy as PCBs and components. Most of them are through-hole but we are seeing some SMD stuff creeping in now which is a shame for me because I can't stand doing SMD soldering and I missed the workshop on that one. But they do kits that will do pretty much anything from driving the locomotives like we've got here to controlling servos and solenoids for point motors and everything like that. If you couple it with JMRI you can do a computer controlled model railway with an app called engine driver so you can drive the trains from your behave. The one signal I have got is that because this has got lights on it the red light is now this end so I know it's going to move that way next. Merg, you can do a computer controlled layout for about 75 quid for all the electronics and stuff. It costs £16 a year to be a member and as part of that you get access to the forum and the forum is full of people who know far more about electronics, never mind trains than I dare say I ever will and they are incredibly helpful as well. We've got Merg that's doing the circuit boards that are underneath this and I'll be stood out there somewhere after the talk if people want to come and see the circuit boards properly I've got some photos of a couple of them later on in the talk but Merg are a really cheap way of getting a computer control. As an example, if you wanted to go out and buy a commercial offering such as the e-cost system or even the Hornby e-link or whatever it's called you're probably talking a start of about £250 I think the e-cost platform starts at about £500 so Merg is a really cost effective way of doing this and you get to solder loads of stuff together at the same time and then finally there's the model railway someone did ask for the geek specs given how many of you put your hands up and said yes model railways, the locomotive is a backman class 66 it's got a lock sound ESU sound decoder in it which is what's generating the sounds that we can't actually hear yet again I'll play with that at the end if I've got time and the track is pico code 75 Okay so network rail data feeds this site the wiki.openraildata.com it's probably the best place to go to get started there's loads of examples on there and it gives you all the information about all of the data because as it says at the bottom data comes in multiple formats and not all formats contain all of the data there's one particular data set where if you download the Jason version of that data set there's one particular data set Jason version of that data set it's missing three fields that are in the tab delineated version of the data set but it's the same data set that took me about three and a half hours to debug because it was data I wanted to look at they've got static data sets that are updated once every six months or so that have got things like every single possible stopping or passing point on the entire rail network and that includes London Overground and things like that so all of the signals, all of the level crossings all the signal boxes where they still exist the junctions, the points the stations, the platforms the through lines, the up all of it and it will tell you whether a line is an upline or a downline or whether it's supposed to be that depending on the running phenomenal amount of information I'm getting jittery now about that and so and then they've got other dynamic sets of data so on a Monday you can download a file that will give you a list of every single service that is meant to run passenger and freight for the coming seven days and then every morning you download a much smaller file that gives you an update on what's changed on that and then what they give you and it's what I'm using for this demonstration is they give you a fire hose of data and that comes in over a protocol called STOMP which I'll talk about in a moment and in that fire hose it's a Jason blob I'm assuming everyone knows what I mean when I say Jason yeah cool great lots of people nodding if you don't it's a really nice way of structuring data in a way that's almost human readable but is a lot more machine readable than XML and so you get this blob of this Jason and you get one of those through about every 30 seconds or so and it can contain anywhere up to 6000 data points thank you you can contain anywhere up to 6000 data points every 30 seconds and it is every single movement that has happened on the network every single service freight or passenger since it last sent a message that's quite a lot of data to be decoding so I'm deliberately looking for key things as I run through it and again I'll come on to that in a moment so STOMP then it's a bit of a weird one it's a message queue protocol that's been around for decades it's not a particularly nice format to deal with but I'd rather be dealing with what most things at EMF camp seem to run on just about everything else I've used recently does as well which is MQTT I assume people are familiar with MQTT yeah cool so preferred way of doing this stuff take a Jason blob dump it onto MQTT pick up the Jason at the other end absolutely awesome don't have to worry about any delineations or anything like that STOMP is almost a I have to get this right it's not quite its own binary blob it's sort of a packed message format so find someone else that's written a library to deal with that for you that's what I do and don't worry about the format but what we effectively do is we connect to the server we set up a callback function and the callback function gets triggered whenever we get a message come in and you can see it on the screen here thisner is a generic callback thing we've got the connection details in there we're connecting to data feeds and every time a message comes in we're going to look through it and we're going to say ok if in the body of the message we've got a field that's called lock stanox stanox is one of the multitude of descriptor types you can get for a particular location on the railway you have tip locks stanox I can't remember what the other ones are so the vast majority of locations actually have three different references and when you get on to platforms in specific stations you've actually got references for both the station and the platform and it all turns up either in jason or tab delineated text it's horrible but it works so what we're saying here is if the location stanox matches 77301 which happens to be Cardiff Central Station all lines um then print a message saying this particular service has just arrived at Cardiff Central on platform whatever and in that jason blob that comes through and this is coming through on the fire hose by the way this is this fire hose message that I was talking about in there for each service you get this little block and it's got all the information that I need in order to print this message out so we're connecting to stomp through now we need to pass it to something else so we're going to pass it to jmri and as I say that is the brains of the operation should have kept this closer hang on that's better okay so jmri it's java based it works it's been around for a long time actually it's been around for a good 10 12 years I think maybe longer than that the nice thing about it being java based is that it will run on windows it will run on mac and it will run on linux so my laptop's running a bun too and it runs really nicely on there you can run it on a raspberry pi as well so if you want to make a raspberry pi the brain of your railway then that's great if like me you're having trouble sourcing raspberry pi's at the moment and I think that's probably everyone I've actually found out that the old del wise thing client terminals come with really good resources in them and you can pick them up for about 20 quid on ebay for 4 gig of ram and about 32 gig of flash with a dual core processor in there and a nice graphics card so I'm starting to move towards using those as well they do 8 gig versions and 16 gig versions of the ram of those as well it's got multiple applications on what you want jmri to do this can get quite confusing because you have to launch them individually so panel pro is what I'm running at the moment in the background here to do the actual control of the railway and you can map out your railway in that and you can put in all your points and when you click on the point it moves it on the layout and all that kind of stuff if you want to work with the decoder that's inside the locomotive you close down panel pro and you open up decoder pro and you then use that and there's a couple of other ones in there as well so that side of it can get a little bit complicated panel pro because the decoder works so I don't need to worry about that it uses jison I'm a python developer anyone here heard of jison before? yeah okay I thought it would probably be a low number jison allows you to run python inside a java environment okay this is cool jison is really old jison is so old that actually it doesn't support python 3 when did python 2 go end of life? was it 2000 was it last year completely end of life but it was un-maintained for a good 5 or 6 years security updates only isn't it and things like that it doesn't look like they're going to do a version of jison that runs python 3 so this causes me a problem however jomri supports mqtt but that requires even more stuff to be set up because now you need to add an mqtt server into it and actually it only supports mqtt for controlling signals and points and a few other things but it doesn't allow you to use mqtt to control trains and move them around it's got a TCP based API called ythrottle or wethrottle for control via mobile devices as I mentioned earlier on however the API for that is unique and we'll have a look at that now that is what a message to the way ythrottle API looks like so it uses these brackets to differentiate between the locomotives that are stored in the system and then it uses those ones to show the values for the settings the settings are always in the same order but never key value pairs and then that's your delimiter between commands so it's not comma separated it's not tab separated or semicolon it's semicolon bracket separated so it's an interesting little thing to work with and it sends data back to you terminated by new lines but it will send you multiple new lines but it won't tell you which of those new lines is the final new line it's going to send you so here's an example this is how we set a function so we have a throttle throttle zero we're sending an action we're making sure we're using DCC short mode for those of you that don't know and I didn't until I started doing all this stuff DCC for controlling which is digital control for model railways as opposed to just varying the voltage that runs through the track has two modes it has short mode which is zero to nine and it has long mode which is zero to 999 so depending on how many locomotives you've got depends on whether you want to use short mode or long mode we're using short mode channel three that's what the decoder is set to in fact if you buy a new train that's got a DCC chip on it it will be channel three so if you already have multiple locomotives and you put them all on the network at the same time and you hit go they all move and because they're not set to the same speed levels they all move at different speeds so the first thing you do when you get a new locomotive is you put it onto the track and you change what the idea is on the chip so that they don't start crashing into each other if you're really lucky they will all at least move in the same direction sometimes they don't we've then got that weird delimiter thing in the middle that says okay we know what we're talking to now let's work out what the command is and so we're setting a function then we're saying whether we're going to turn that function on or off and then we send it anywhere from zero zero to ninety nine as the function now I've picked ten arbitrarily here one tends to be the lights zero tends to be the lights on and off and then everything after that is sound or internal lights or whatever it is so how do we move a locomotive if that's setting the functions on and off how do we move a locomotive and then we send it another command and that looks like this so you can see we're sending the action we're using the same throttle we're setting the speed which is v I'm guessing that stands for velocity I don't know it doesn't actually say in the documentation and then we're giving it a value of the speed between zero and one hundred and twenty six so I'm either overrunning by twenty six percent when I'm up to maximum I'm missing two bytes somewhere and I haven't worked out which it is so that's the control and then if we want to change direction then we send it this so r is the direction and zero is reverse and one is forward so even that's not particularly intuitive because to my mind r would be reverse so you'd say r to one if you wanted it to go backwards and r to zero if you wanted it to go forwards so I write this and I don't have the knowledge to write something like this so therefore I'm working with what I've got so I decided to make it a bit easier for myself and I wrote myself a python library and this makes things a lot easier this is the code that's on github by the way and it's under this model railways as code thing which I've started as a thing I would love that to be just a dumping bucket for people's experiments on how to generate model railways and automate them and all of that kind of thing so if anyone's got any projects ping me a message more than happy to look at including it as you can see here we register the loco we control the functions we change the direction we set the speed so if I was to run this it would create the new locomotive on channel one it would set the lights off on this one so locomotive one function one value whether it's on or off it would change the direction so it's running in reverse now we set the speed to 10 we sleep for 5 seconds we set the speed to zero and then we disconnect from the server so all of a sudden I don't need to remember all of those delineated commands so 23 minutes in I'm going to rock it through this next bit but like I say I'll be outside so if anyone has any questions these are what the circuit boards look like from Merg this is one of their servo controllers so you have a little 9 gram servo underneath your train track and when you send it the right signal it moves the arm the arm moves the points to either open or close them so that the train ends up going in the right direction I've gone through most of this oh I've set on here block detection that was the bit I was missing block detection the network is divided the national rail network is divided into loads of blocks trains enter the blocks and then they leave the blocks and all of that is monitored because that's how you know how close the trains are together that's very simplistic I know we have some signal engineering engineers in here hi at the back there forgive me if that's a little bit too simplistic but that's effectively what we're doing you can do exactly the same thing on your model railway and it basically means that you can then start doing automated signalling so you can say there's five trains on the layout I've got 18 different blocks if there's a train in block A then there can't be a train in block B it has to be in block C before the train in block A can be released into block B all of that kind of stuff JMRI actually allows you to script all of that and will deal with all of it as well so that's an awesome protocol running on it as I say cheap to join, kits aren't expensive get full computer control at your layout so if we put all of this together he says rapidly trying to come to the end of his waffling this is what the new function call looks like so this is using my function this is using my library that I've written so when the message comes in it gets sent to this on message event and there's the code from before if the stanox is this Cardiff Central then if it's passed through just print pass through to the screen don't do anything because I'm only actually interested in stuff that goes back and forwards if it's coming in then move the train that was perfect timing this is live by the way every time this moves that means something it's gone through Cardiff Station it's pulled in at a platform it's pulled away from a platform at Cardiff so this is real time and then once you've done that because it's only short section of track change direction so the next time a train comes in you move the other way now if you were running this on a much larger scale you can actually allocate this to platforms so you can say when a train leaves platform 6 at Cardiff you move the train that's in platform 6 on my layout out of platform 6 and off onto the network onto the more railway network I have actually slightly modified this and that's why the microphone's there for those of you who were wondering in theory at least if a train goes steaming through Cardiff and there aren't many that do if a train goes steaming through Cardiff the horn should sound on that locomotive using one of the function calls but that's what that puts and you can see where we're calling the functions and you can see service blah has just arrived at Cardiff central on platform 0 or on platform 6 these are screenshots from the other day when I was putting this together so on my screen I get a read out of what's happening so that's the system what's next I want to build a proper layout rather than this that means finding a suitable location the St Ives branch line is probably going to be my first target there's a number of reasons for that how many of you know the St Ives branch line okay a handful of you cool okay so first of all it is absolutely beautiful it is a stunning bit of scenery running from Carbys bay and in fact before that Leyland all the way along the Cornish coast into St Ives and if you're really lucky you can look out the window and you can actually see the seals some bathing on the rocks and stuff like that at the right time of year as you're travelling down that time that thing perhaps more importantly than that it's a single line track that makes it a lot less complicated because trains either run to St Ives or from St Ives but never at the same time it's also quite quiet so I don't need to worry about having to deal with 400 trains an hour or whatever it is that might go through somewhere like Reading or Clapham Junction or something like that so it's quite a nice small compact line that I can use to expand on this and actually get it to the point of okay that train should actually be at that signal have I got the speed settings right on my code so that the train on the layout is doing the same scale speed as the train that's on the national network obviously you need to work out the tip-lock the stand-ox and all the rest of it for each section of the code for that and then write the code based on we-throttle to talk to JMRI to drive the merg boards to drive the train but that should be fairly straightforward given that this bit is working so that's my plan eventually I would love to get a field and build Reading or Cardiff or something like that but the storage alone the storage tracks alone for something that was that big would probably take up at least this tent if not more I also want to continue to update PiWi-throttle to add more functionality at the moment it just drives trains I want to get it doing the turnouts the signaling and everything else if you know Python, if you want to get involved in this jump on GitHub, come and have a look I'd love contributors it's under the MIT license as well so anyone can contribute to it and then finally create more projects under the Model Railways I've already started on one that lets you describe Model Railways as YAML and then you can run a decoder and it should in theory output a track plan that you can import into JMRI so that it draws all your panels on the thing that's not going as well as I'd hoped but it's going so there we are and that's it from me so thank you very much if you've got any questions I will be outside the tent at the end of this github, Model Railways is code github proffulcon or proffulcon on Twitter thank you for your time, I hope that's been useful