 Same day patches, am I right? And games full of bugs. Two things I've discovered are not entirely the fault of the programmer. Game engines are complicated as all get out. And until you actually start using a AAA grade one, you have no idea just how big they are. But that's an aside. So, lovely. Empty slide content. Well, at this point, I think we're just going to go straight to demoing this thing, because I've got code and a working thing. So, skipping some of the backstory while the robot just sits there and dances. Unreal Engine is, if you haven't heard of it, the most successful game engine in the world, according to Guinness Book of World Records anyway. It's shipped over 140 titles over 20 years. It's spent more than 22 years in active development, shepherded by the same lead engineer. It's continuing to improve at the moment into AR and VR experiences on every platform. It's basically the best thing out there in terms of available to use and just getting started. It's completely free to just download and look at the source code. It isn't, however, technically open source, because if you do wish to use it in a commercial product, you will need to commit to paying Unreal, sorry, Epic, a licensing fee for whatever you may make, only if what you're doing actually matches those things, because if you aren't actually using the game engine as your product, they don't care. So, if you were just doing visual arts, they're quite happy to let you use that for free, which is why it's actually starting to see a large amount of usage inside of post-production houses for early-stage visual effects visualizations. Now, on that point, that's actually where this Python plug-in has got the most legs and the most paid development. The original author is Ronaldo. I'm going to butcher his last name. And he is actually the original author of Ubizgi, which I'm sure most of you have heard of. So, he's someone who knows a thing or two about embedding Python inside of other programs. And he's done a tremendous job getting Python inside of Unreal Engine. It's quite a testament that he's got such an extensive access to Unreal Engine's internal APIs. From very little Python code, you can reach out to literally hundreds of C++-based classes that make up the entire engine from its physics through to its graphics and AI and behavior. It's truly staggering once you start looking under the hood just how far it can go. However, it is not yet very convenient for most of those things because while you're actually able to use it in the game, the primary push in terms of people paying Ronaldo to actually push on it has been arts, studios, and designers because if you didn't already know, Python is the dominant tooling language for 3D modeling and 3D effects. It's powering the software, making up about 75 to 90, depending on how you cut that pie, percent of the software in that industry, everyone from Pixar to small shops down the street, just doing TV news bumps. It's all Python except for a few rare outliers and they tend to be worked around by people rather than ignored. And as a result, these people are very excited to actually have an ability to go from their existing Python tool chain directly into their environment. So to use a hypothetical going off our little robot here, if you'd just finished modeling this up in something like 3D Max or AutoCAD and you wanted to push it through for pre-visualization to show someone how it looks moving around the environment, a little bit of Python scripting can glue it straight into one real engine, create automatic pivots, set it up, create a rigging and animation cycle off whatever periods you've scripted it from, and just drop it straight into render. Now, that's quite impressive, but it's not what I wanted to talk about today. I just wanted to make sure that if anyone does go and have a look at it, they're not confused by how much it's talking about running the editor and not the game. So this one here is actually automated a little unusually because you'd think if you didn't, once you see under the hood anyway, you see it's not actually the robot that's automated. The robot is a stock asset. You can just go and bite yourself off Unreal's marketplace. It comes with a weld point spawner, and I've scripted that because this is just one of those examples where the system is more flexible than you think even on the first go. I could have scripted the robot to do all these moves, but it was much easier for me to augment the existing code that someone else had written, but just a little bit of Python to get this effect with very little code. In fact, it's literally this much. Actually making it a bit bigger. There's so little of it. All it's doing is it's just keeping an eye on the last time. It did something, and then calling a function because building this extra loop into the game's tick system inside the complex blueprint would be quite cumbersome. This, I can just bolt this on as a component, and it just works like magic. And that actually works the same way for a lot of these things. For instance, in that environment here, where you can see all the sun and everything else, I've actually automated the setting of the sun. That right there has been set automatically based on the current time of day here in New Zealand, and if I'd gone further, I probably would have been able to get it to do the weather and the access, but I didn't manage to get that to work over here in New Zealand. I got the weather working over in Perth before I had a new API. And here's another example. This is a pretty complicated one. This one here is actually what this presentation started as. It's a good example of how you'd actually script up the intro to a game. You've got a bunch of things inside the engine, and you want to have them do something before you actually kick off an unsupervised experience. You've got all these things that you want to effectively make them dance. And what I was doing was I was actually going from where my camera is and attaching a browser, which explains probably why my slides are all blank. The internet's probably down on my laptop. But all of these are just chasing through. It's finding the browser it wants to move, it's grabbing that browser, and then it's pushing it directly in front of the camera, regardless of where I've left it. So I could just have a stack of browsers just sitting wherever I left them inside the game environment, and they would just turn up wherever I wanted them to following my camera, which can make a lot of workflows when you're just trying to keep track of lots of assets in a game environment. A lot easier. Because the bigger your world gets as you spread things out, the more annoying it gets moving around it. Here's a bit more of a game example. I'm sure that you're all probably noticing the keys straight away there. And it's this little one here. He is not moving fast enough. There we go. So I've bound the movement keys to the model in the world. A little slower than I'd like, but you guess these things. And these can be combined. There it is going diagonally. It's all powering that physics engine because it's all callback-based. I've bound these functions. I'm going to use a generic here because Python gives me this power. For me to do this, I'd have had to go all the way to C++ because the Blueprint system, while it's powerful, is not as flexible as Python because it has definitely been designed from the get-go by Epic as a way for them to bring C++ light. And in fact, they've really delivered on this in the last two versions because the visual scripting language, Blueprints, actually compiles down into C++ code that is embedded into the environment. So they've got a very concise and tight runtime, which is why Python augmenting it is actually quite powerful because it gives us this flexibility and freedom to just do quick little things like this when we want to start prototyping our gameplay long before we actually need to do the harder work of doing this kind of generic behavior in C++ that confines the game engine itself. Because while some of these things are very easy and just regular C++, Unreal Engine has a complete reflection system built inside of it in order to provide dynamic lookups across parts of the code base so that things can work interactively across the environment of the game itself. And that's actually how the entire plugin has been built up. It's using the reflection to look up for an object and then binding off that object. And it's quite excellent because I've attached this and the Python plugin knows that the uObject is whatever it was bound to, and I can just call these functions. How are we going for time? Well, given that those slides have failed to load, I might just show you the key one, and the browser's closed too. Let's see if they're refreshed. Yeah, Cactus, which is quite surprising, but, oh, well, Internet. I should plan that one better. Let's just go for questions. I've spent literally two months with my head completely buried in this, so I'm sure that I can answer a lot more questions than I can stare at blank slides, and it'll be more fun. In fact, I could probably even just start live coding if anyone was curious to see how it actually works in practice. That sounds exciting. That sounds exciting too, so if you'd rather do that. Well, I mean, a lot of people are thinking of questions. You should be able to answer this pretty easily, probably, so I didn't fully understand how you're getting that reference to the uObject inside of your class. I don't see, like, where it's imported. It's a built-in. The actual environment running the code has that as part of its instantiation. Got it, so it gets injected for you, and you can just rely on it being there. It's coming straight from the Python environment itself and it's constructed by the plugin. It runs the interpreter and puts it all in there. Excellent. So you mentioned earlier that it's easy to prototype your gameplay in Python. Before you start coding it in C++, is anyone actually also maybe seriously considering writing the actual game in Python and leaving it in Python, or is it just not suitable? Well, that's actually a really interesting question that if my slides weren't broken in the first half, I would have actually elaborated on. The primary author of this library started out with that dream. He still kind of has it, but he very quickly realized that was not going to be practical for quite some time because there is just so much to get through in terms of API coverage and loose corners and things that could trip you up deep into a process that it's not really feasible yet to rely entirely on Python to create a game with this. However, it can get you a lot of the basics and certainly that first third where you're still working out, is this a game or is this a bad idea? More than likely could be done entirely in Python before needing to bring in the more advanced functionality like dynamic textures and real-time changing environment geometry, things like that, that will definitely want to be pushed down into C++ for speed reasons. But before I forget it, the original author actually has an excellent one-hour presentation on how he started this and he goes through the process of it and he actually has a tremendous demo that I was tempted to copy but didn't want to because he does such a good job of it where he builds up using Python to grab portions of the screen from inside of Unreal Engine, pass those in as bitmaps into TensorFlow and then does image recognition real-time inside of Unreal Engine in order to have Unreal Engine display what the person is looking at through machine learning, even though you already know what's in the world, but it's just a fantastic example of closing this loop in a way that is unusual and interesting. And from there, there's actually been a number of people who have experimented with it for training drone software because you can prototype an environment just to complete minefield for these things to fly through. It doesn't cost you a dime because it's all virtual and then you can set your algorithms at this task and have them work in this completely unique environment that you've created much more freely than just trying to bolt it into existing programs like you're seeing a lot of now with the OpenAI sandbox sort of work. Can you also interact with the 2D UI in Python or is it only for like... When you say the 2D UI, which bit do you mean? There's a lot of UIs here. The overlay that you'd have on the game with like... So like the menus and the screens like that? Yeah, the menus and the stuff. Absolutely. That's all. As I said, there's a lot of rough patches and you can get to almost everything if you go the very hard complicated way. I don't know how many of you have actually ever used a sort of the proxy pattern in reverse in Python. It's not a common one. We have subclasses. They're great. But there is almost complete access through the proxy pattern if you're determined enough to make the right calls and chain things the right way and sprinkle a little bit of C-types in if you need them. But you can definitely get to that UI layer and just completely play with that. It's worth noting on that point, if you are prototyping something like that and things are simpler in something like blueprints, one thing to remember is if you do start tinkering, Python can call the blueprints as well as the blueprints calling Python. It's very interchangeable. In that demo where I showed the code, the browser was moving around to, that was Python code calling into blueprint code that was calling into C code that was calling back into blueprint code that was being checked by my Python code before making a decision and then moving again. And that kind of complete intermixing is you can just do it. It doesn't actually get in the way other than where the Python might be a performance bottleneck. You may have answered a question or may not be a sensible question in the first place, but... It's looking like it wasn't a sensible presentation. So, please. These questions are good. Just wondering, could you use this to provide a real-time interface to maybe to an unreal game so you could create your own bots to play the game whilst you're having cup of tea or something like that? I'd say you definitely could based on what I see with some of that drone stuff. Whether it would be terribly useful by the time you've got enough compute power to get sensible behavior out of them compared to just the decision trees and behaviors stuff that's built into the game. I guess you'd have to try it and find out. But I know some people are definitely experimenting with that kind of thing out there if you start looking on GitHub and people's blogs. I'll find a screenshot in here because I do believe he has a screenshot of the... No, wrong screenshot. There is a screenshot in here of him doing the... Facial recognition. Here we go. Well, he didn't translate that blog post very well. Leading on from that, do the existing games like, do they have a Python interface or is that something developer adds? They don't. Python has kind of fallen out of favor with game designers or developers more so. The designers still love it because all their software uses it. But the developers have sort of shifted away from it and starting about ten years ago was pretty much done moving away from it. There was a brief period where you did see a number of AAA titles ship powered significantly by Python because at the time it was more than adequate for what they wanted. Battlefield 2, Battlefield 2142, the map engine in Civilization 4, the sequel... Vampire the Masquerade bloodlines, not the first one, the sequel bloodlines was scripted in Python as well. I don't know why they jumped ship on that one, but they did, and that was scripted with Python, but it hasn't continued that way because with Blueprints, I'll just pull up a Blueprints screenshot for anyone who hasn't actually seen it because it is quite... Has everyone actually seen a visual scripting language before? No? Okay. Actually, better than that, I've got the editor here. Why don't I just start showing you live stuff because that's way more interesting. So this is a Blueprint. It's a visual scripting environment where you can chain together a sequence of events and the processes that come out of them. So in this case, I'm getting the world of Robot, I'm getting the state of it, and then I'm doing a conditional check to make sure it's not doing something already. If it fails, I'm printing something. If it isn't failing, I'm printing something else just to keep track. And then there's another call to actually create a point in the environment to trigger the other Robot to do the world because that other Robot is programmed by default to seek for any unfinished work and do it. It's a very simple loop that the author built for that Robot, and so being able to script it this way was much cleaner and able to create that little effect very simply. And the Blueprints make it very easy for me to augment this with Python because this entire function here is what's actually called by my Python code where I put that in there as spawn weld point because that Blueprint that you see here is spawn weld point, and Python is called into this to keep this process off. And with Epic making these compile down to C++ and into machine code, it may look a little childish if you feel as though it's... Some people have the opinion that it's not really a programming language if you don't have an editor, but Epic have made absolutely clear that it has just as much power as C++ does. It just isn't as flexible as raw C++ is. And they just recommend that if something needs to be done, you just do the small piece in C++. There's a large number of people who make Blueprint libraries, so they'll make lots of little functions in C++ and then just keep doing the rest of it in the Blueprint system because it's that flexible. And Python is a natural extension on top of that because as Python programmers who know what we... We can think that process through simply and easily because it's what we're used to thinking. We're not used to thinking back to this kind of system of either C++ or especially the visual scripting given how it's not very common in other fields. A significant majority of people programming with Unreal Engine do work with Blueprints. The core of most major titles is still C++ because the people who do that work have been doing C++ for a decade, two decades, in the case of people like Tim Sweeney who actually works on Unreal Engine three or four decades. And these are the people who are behind things like Gears of War and the peak of the triple-A titles. And they've been living in C++ so long it would be weird for them not to use it. But indie developers picking up games and using Unreal Engine for the first time, many of them making quite successful careers out of it now, they do just use Blueprints because they can get so much done and it's very easy for them to bring on new people to assist them more flexibly because the barrier to picking up a Blueprint and working out what it's doing is a lot lower than looking at a random part of the C++ code base and trying to work out what all these macros that someone's left all over the place are doing. And the browser embedded in this is the Chromium embedded framework. If that gives anyone in the audience a clue just how nasty the macros I had to work with to get this to actually not work as the case may be was. Did that kind of cover that? I feel like I may have steered away a bit. So we've got about five more minutes. I'd be kind of curious to hear what you actually used this for. Well, this was actually a learning exercise to begin with. And then as I got more of it to work how I wanted to, it started to look a lot like some of the presentations software I've used in the past, particularly it reminded me of Prezi's early days before I got a bit overblown and particularly Impress.js when I first saw it with its sort of 3D CSS effects kicking around. And the idea of being able to add a lot more than just the web into this kind of thing seemed very interesting and worth exploring. One of the examples they have, and I don't know if they'll be a screenshot, but one of the examples is a, what do they call it, it is a graph plotting. Ah, we have screenshots. Previous would be brilliant. There we go. So he's actually added in Matplotlib. And Matplotlib is creating that chart there as a 2D bitmap which he's transferring over into an Unreal Engine texture and then applying that to a surface in the game world based on code that is written to set up the logic of what creates the data for Matplotlib. And he's got a couple of blocks off to the side there and whenever you go over and hit one of them it shuffles things around. From memory I think it's whichever block you hit most recently or they just add the blocks up that you've hit and just updates the chart like that. And being able to go beyond the traditional kind of presentations seemed very compelling in some cases and as a learning exercise it proved quite challenging. And I'd say over the last two months I've learned enough C++ that I can actually fix things that are non-trivial. I wouldn't go and say I could write them from scratch but I was patching other people's forks of Chrome to get this to work which is something I would never have thought I would do just three months ago because once you walk far enough down it's nice to make sense when you've got a narrow view of these kind of things in the sense that hey I've got brilliant documentation by Epic who have put together comprehensive labels on all of their types, the macros are well documented it starts to actually make sense in a way that a lot of times it hasn't in the past I'd say. Anybody else? Anyone want to see live coding if there's time? Or is there only two minutes I think? I'll see what we can get out of that then. I'll just do a quick one to start with. I'll just make this robot faster because we all know that a welding robot is going to be a lot faster if it's easy one weld every three seconds. So let's actually get this guy going nice and fast. So if it's anything over a second you'll get back to work. Back in and this time we'll just run it in the simulation which is another cool little feature they have of the editor where it actually just runs things in line and you can jump to the various pieces so if I see something weird on the back of the model I can jump in and have a look at why that texture is strange without actually having to jump in and out of a full game session. Or I can just go to the part of the level that seems broken and work there. So we have our much faster robot. Let's use what I didn't use. I was going to have this throw everything on the map entirely to the edge of the world but it's a tornado. And if I felt like it I could bolt on some Python and steer it around with Python. And that would just be as easy as going in and actually adding a Python component which I'll need to stop the engine for. So pop my tornado back in. There we go. Add a component. There's our Python. And then we will just add... I'll just use the Python editor in here actually. What just happened? Chrome crashed it with memory. It's very not optimised. The other nice thing about this is if you are just thinking of trying to get started with programming again having Python and the niceties of this system where you don't have to learn C-sharp like you would for Unity or use a less well supported language like... I think they're called Boo or something. I don't know if it's still supported but they had that for some time. Whereas in Unreal it is full C++. The visual scripting language is very easy to learn and then you can add all of your existing Python knowledge on top of that and combine it with a wide variety of assets available from the store that you can just buy and include in your game. There's a great little robot there. It had a sale as a lot of the American companies do in November and one of the things I grabbed was... There we go. Raptors. For anyone who hasn't had enough Jurassic Park in their life you can make your own full of your own dinosaurs and make them do whatever you like. There we go. That's what I was animating. So we have 144 different animated raptors modelled to the skeleton, adjustable. You want to rip one of them arms off, you just make a modified version of the skeleton and cleave that off and tell it to ignore the animation nodes. Now you've got an injured raptor to mix into the bunch and it makes it very easy to go from, I have nothing but an idea to, okay, I'll buy a few pieces and start playing and you can just start building up as simply as that. And some of them are quite impressive examples. There's complete implementations of pathfinding systems as well, actually. If I can find it in the list. Actually, here's a great one to finish off on. This is the one I had working with Perth Weather but I could not get a matching API in time to try and translate it over to here is a full dynamic weather simulation, water simulation complete with a physical buoyancy model attached to the actors in real time. So if you feel like letting the air out of one of those little floating beach balls, you are more than welcome to in the game engine, we'll just update to accommodate. And that's just dropped right in there after being purchased. It's really easy to just pick these things up and begin experimenting with them and particularly with Python because I could just bolt Python onto any one of those things and do whatever I like to it. So I could have the boat steer in a circle with the Python script by itself for the weight of one of these things, cycle up and down and whatever else combinations might actually produce something interesting. Thanks. Right of time. Oh, that's good.