 Yes, good morning. So my role at Moodle is I'm the development team manager but I spend a lot of my time doing development and a heavy focus at Moodle is on automated testing and the reason is we're making the core product that everybody else uses and we need to make sure that everything that goes into core is of the highest standard. So in order to ensure the quality of the Moodle core code we put in a number of automated testing systems. They test different parts of Moodle in different ways. So we have unit tests for functional code and things like that but we also have this very cool system for automated testing of actual browser pages which is what I'm talking about now. So a typical Moodle site looks something like this and what we're looking at is sort of a picture of the makeup of the code on anyone's particular Moodle site. So you would have a whole bunch of code which is just core Moodle but then you also have other things as well. So you might have plugins that you've installed from the plugins database. You might have local customizations people you might want a custom theme or some change in the behavior of the way that Moodle works out of the box that you're willing to maintain and then you also might have local bug fixes. So if you find a problem with Moodle you might fix it locally and send the patch back to us hopefully but you'll still sort of apply that bug fix to your site before you wait for us to integrate it and test it on our side which happens sometimes. So all of these extra bits of code every time you change something in Moodle a change could be a customization a bug fix a new plugin it could even just be a admin setting change perhaps relating to some other system or something like that but every time you change something in Moodle you take a small set of customizations and then you add one small change and you get a much more complicated bigger set of changes. So that's because all the changes interact with each other and it becomes sort of exponentially more complicated the more changes you have. So the way to manage that is through a change control process which might look something like this. So I've written some change control processes in the past and I've seen quite a few and they're usually very detailed and very complicated and this is not a real one but it's just to sort of demonstrate how pedantic the change control processes can be and that's for a good reason because you want to test everything and make sure you don't break Moodle when people are going to submit assignments on the last day and things like that. It's very important. So people's deployment environments usually have hopefully have at least three you might have two if you're saying you're testing everything and then deploying it really I think you need three so that you can actually test the deployment process for changes as well. So you would develop something on a testing or development server, get it ready, get it approved and then that would be moved to the staging server. Someone would test it again to make sure that it was moved correctly and it's still behaving in combination with any other changes that you might have made at the same time and then when that's approved you would send it on to the production server. So this sort of shows that process. The thing is that the code could fail at any point and then it would have to go back and you would restart the process from the beginning and really this is just all about ensuring that you're not going to be putting untested changes on your production server. So the first stage of testing is where somebody who's requested any kind of change needs to test that the change that you've made or a developer has made actually achieves the outcome that you were looking for. So it's not just sort of does this thing work, it's also does this thing work nicely, are we happy with the user experience and exploratory testing, different browsers, different combinations of settings and things like that to make sure that yes we definitely want this change on our production server. Now once you've deployed it onto your staging server someone then also has to go back and check that it's still behaving the way that you're expecting it to and when it's used in combination with all the other changes which might be included in an update that it's still doing what you want and so you might have this is sort of an example that I found on the internet of a manual test plan where it's very detailed sequence of steps that somebody has to go in and perform on multiple browsers and then at the end of the test they write some comments and they say whether this is working correctly or not. Now that's very time consuming. It's also a bit hit or miss because as clearly as you can write the testing instructions there's different people who can read and interpret those testing instructions in different ways. So you might end up writing a test and because it wasn't very specific two different testers might either pass or fail the exact same test on the exact same server which gives you it's not really what you want you want a hundred percent reliability in your testing in so you can say whether something is correct or it's not correct. Yeah I don't know that I need to explain this one. So now now we get into what Beehat is so perhaps I can have a show of hands of who's actually heard of Beehat in Moodle so it looks like about a third of you. So hopefully for the other two thirds of you I'm trying to convince you not to learn everything about Beehat now but I'm trying to convince you that you need to be using something like Beehat in your change control process and that you're going to go away and find out more about it. This is a Beehat test so the language of a test is English it's not code the tests are written in sentences so that a non-technical user could come in and read the text and understand what the test is trying to do and also once you're used to the syntax it means that non-technical people could write the tests which is very important so it doesn't have to be the developers that write the Beehat tests. In this particular example we have a feature so every feature is basically it's one thing that we're testing so in a change control process that could be an ID, include an ID number for a change request or something like that and the first section is just informative so the texting there could be anything you like but when you're looking at it later you'll appreciate it if you're sort of including enough detail to understand what the test is about. Not how it works just what the change is that you're testing. Then you have a background section which from a clean Moodle install this is how you create the courses and the users and the activities that you need in order to do the test that you're after. It's using a bit of a shortcut syntax there so it doesn't quite look like English but once you get used to Beehat you'll definitely want to use the shortcuts instead of writing them all out but then this scenario is an actual test so this is the bit that is very easy to understand. This will open up a new web browser, it'll follow links in the navigation, change fields in a form, click on buttons, it can interact with JavaScript. Anything that your actual browser will do you can click on an interactive in Beehat and it just looks like English so I should click on this button, I should not see this. This is what it actually looks like when somebody runs a Beehat test. It opens up the browser, does all the steps and then if anything fails, so if it's looking for some text in the page and it doesn't find it, it will fail immediately and it gives you a big red failure. That failure can be built into other systems so that you can get reports on running entire tests. This particular test goes for about five minutes, we wrote this when we wrote ATTO which is the text editor in Moodle so this automated test goes through and it tests every function in the ATTO, every single button to make sure that it works correctly and we can run that on every single browser that we support and just verify that nothing, any change we make to ATTO hasn't broken anything. We're looking at using it for performance testing a little bit now. There's other more complicated ways to do that so for example you can run a test and then you can record all of the interactions from that test in a file and replay it to test, do load testing and performance testing but I mean we haven't set that up, we have different tools for performance testing in Moodle at the moment. So on to Moodle, we use this a lot so in Moodle we have almost 1,000 scenarios and each line of those scenarios is called a step so we have 36,000 steps and that's growing almost every single day and it gets longer and longer. So we run these tests continuously, we have integration process which means that as soon as we make a change to Moodle if that fails a test we get notified about it. So now if you're using automated testing in your change control process, things behave a little bit differently. The first manual testing where somebody who's requested a change and they need to verify that the change does what they expect, that still needs to happen but the second part, once you've approved the change you can automatically have a robot come in and test that change on a changing server. But testing that change once is good but every single time you make any other change from that point in future because this test is added to your test suite it will get rerun on all of the browsers that you have set up in your testing environment. So that's just showing that. Now to find out more about how to set this up, if you're working with a Moodle partner you can ask them, they may have already got some automated testing happening in their change control process hopefully but if not you can ask them about it. There's lots of information. So at Moodle we have an integration server which is integration.moodle.org and it's public you don't have to log in. Anybody can go there and see the status of all of our automated tests at any time they want. And we have continuously running tests on that server. And so for example just in the Moodle master branch it takes four hours for us to run all the tests and that's running the tests in parallel so you can actually be running I think we run three parallel tests so if we just ran them one after the other it would take twelve hours to run all the tests in Moodle. And that's just for one branch so we have that exact same structure set up for at the moment we're running tests for two seven, two eight, two nine and master. Now the code that we use to manage all of our tests is a tool called Jenkins which continually executes all the tests and gives us reports on whether they're passing or failing. And all of the code that we're using to configure Jenkins is available on that GitHub account which is linked there. So that's information you can pass on to developers or partners to go in and see how we've done it and copy our examples. Okay so that's it I hope everybody's interested and they're going to go and find out more about VHEP.