 Welcome to the orientation session for quality assurance for the Drupal.org D7 upgrade. I'm Melissa Anderson So known I guess for lending a hand with the git migration Which is kind of how I got started on all of this QA work and Neil Drum is here with me as well Who's the tech lead and project manager for the second phase of the upgrade Neil? Did you want to introduce yourself? Sure So yeah, I've been object managing and also made a lot of the Sweet so I got involved in Drupal.org like I said with the git migration and During that process and getting ready for its deployment We did a lot of manual click testing to make sure that all of the functionality of just the git section of the site was Was working as expected and so it was out of that that I became passionate about automated user testing So we're using tools called be hat and mink to check the site and let us know how we're progressing we started the process by working with some Developers from Capgemini. We the company donated Three full-time developers for six months to work on a project and they ended up landing on this one so what we did was look at the way that the Drupal 6 site works and Try to describe what is supposed to happen with all of the features on Drupal.org which is a huge undertaking And it's not documented anywhere So that just discovering what the site does in all of its detail was a big part of the work that we did and Then taking the tests that we wrote and automated against that and moving them to D7 you know people are really excited to find out when the upgrade is going to launch and Neil can talk more about that later But my feeling is that it's going to be really soon. They're logistical and scheduling issues And so the point of this is to let you know why we think it's ready and What the process will be once we open it up for community testing? so the the tests that we wrote are organized into 15 major feature sets there might be 16 of them now I haven't counted lately and This is how I'm keeping track of progress for the upgrade. So anything here in the issue queue that is postponed Means that there are open issues for it that We're waiting for a fix from Neil or someone else who's working on it. So for example, if I click in here There's just three tests that are all failing because of a single issue and that's a open So that's very close And that's the case with most of the things in the postponed state that there's a couple of tests and a couple of issues out and some of them like with releases we're just looking at needing to rename a field So feel very close even on the ones with open issues When I put them in the needs work state Like project in its case that means that there's a test that the test is bad And I know it and it hasn't been updated to D7. So that's outstanding And that lets me know that they're not in sync yet as well as in this case a couple of issues with project But again, nothing hugely significant from the automated testing standpoint This big swath of green is the RTBC set of tests and that means that on a regular basis If there's no problem with network connectivity, which can interrupt tests sometimes these are passing So for example, you can see the planet issue. They pass regularly You didn't even get in here to the folder and take a look at what's happening. So here These are all collapsible The tests are written in human readable language So it's actually possible to follow the steps for the most part just from the interface Occasionally you need to look at the code that supports the automation, but this will tell you what's being tested automatically So all of these steps here The testing is taking place on get seven site So it's get seven site dot dev Drupal dot org And that's noted on the issue queue, which is the primary place for community QA to get involved all of these links are collected there so the htaccess is Drupal and Drupal and it says it on the issue queue and it says it on the little pop-up window and that's to keep robots out You can log in with your regular username and password It's all HTTPS and you can do the things that you would normally do on Drupal dot org And that's a question that people sometimes have like where should I start testing something as big as Drupal dot org And clicking around just to see what you can find is great, but it's also really important to have you get involved With the things that you care about That you would notice if they broke and that are important to you and making sure that they work the way that you actually use them So the hope is that having these automated tests will make the process Smoother for the community as they get involved in QA Hopefully we've been able to find a lot and resolve a lot of issues before before you even join us so The staging server gets rebuilt daily and I was going to let Neil talk a little bit about the schedule for that and The process of migrating the database yeah, so the site is in the red bar at the top is rebuilt every day and That's just so we get back to having no one set data for the site in fact You'll actually see new issues and form posts and everything coming in from production and Behind that the actual upgrade process is run Three times a week Tuesday Wednesday Friday It actually takes a little bit more than a day to run through the entire upgrade script and Make a database snapshot of that. So you'll see The data from production lags by three or four days all together Sometimes longer if that fails, of course, we have to You know start over and I'll take another two days So yeah, what you should see on the site is You know new data coming in and we're doing this so we test Everything a lot Yeah, that's really similar what we to what we did when we did to get migration as well trying to rebuild on a daily basis so that we know that the process for the day that the Deployment happens has been tested many many times and in general Everything that all issues that get fixed Should show up on the site fairly quickly Once we were something to be CR you might see us say that in the issue queue They'll show up on the site within about five minutes Yeah, of course, but sometimes automation doesn't pick up everything exactly right so Look good, you know, let's know in this UQ and also the exceptions that is anything changing a Migration that's already happened that of course will need a full upgrade But usually we add new update functions and those will go ahead and run on deploy here Also So you do deployments of code as you're working throughout the day so functionality made change About that at that level, but the database functionality doesn't change except for once a day Right. Yeah, so Neil just mentioned the issue queue and that's kind of a fun segue into the only major change in Functionality and I think it's I'm pretty excited about it is the way that the issue queue Works so it's becoming a little bit more like other bug tracking and issue tracking software and Solving some of the issues with how you change the status of the issue queue So there's a document here and it is linked to in the qaq as well This is this describes all of the ways that it's expected to work And I'll do a brief demonstration of that just so that you know how we think it should be happening So if I create a new issue on get seven site It looks pretty much like the old create issue and I can do a demonstration of the new issue functionality So we've got some selected fields. We'll call it code And once I click save that's where you'll start to see the big difference here so instead of having all of the Issue status fields attached to the comment like they are on the current site And you can see an example of the way the current site is now the comment isn't just a comment box It's changed the title of the whole issue and change all of the settings So if you come over here, it's now just a comment field and you can press save and It shows up below the issue So I left the body blank But if I wanted to update the issue, there's now a great big green button That makes it really clear that this is what's supposed to be happening So you can set the status here and make changes And one of the exciting pieces is that we will also be getting the opportunity to relate issues to each other So in the issue relations if there's a meta issue and you want a bunch of child issues We're gonna have functionality for that too and again the details are described in the document And then you can comment when you make the change in the status Why you're doing it? So we'll change it from bug report to task Save it and now we've got the updated Information over here And we still have a comment and we have the ability to look at the changes So what was different at the time so you can see when the change was done in the stream of comments But you keep the data with the note itself and Then one of the other features that and I think this might be the one I'm most excited about I don't I don't know for sure Is the upload new file functionality so it's right there with the issue and You can comment this So patches or screencasts or whatever you're gonna Screen captures whatever you're gonna use to Explain the comment that you make are being gathered at the top with the issue in order and They're linked to the comment down below And you can also find them linked in the comments So make it a little easier on those very long issues that are that are gonna be out there So as you're looking at the issue queue and you're curious if it's working right again that document Is pretty detailed about the issue queue changes and you get a chance to look at it and get comfortable with it So the exact time for opening up for community QA is really a balance Between moving things forward in a timely fashion as well as Getting as many issues that we already know about fixed as possible so that we don't get a lot of duplicates And it'll also be ultimately be Neil's call when we decide to open that Right now you can see the list of confirmed or known issues And this link will be available on the QA issue queue as well So there's a lot of green in here, but you can see there's still a handful of Issues to be addressed so it'll be his call on when he thinks he's ready to start getting community input Relative to fixing those it's good to be aware also of the deferred issues So this is functionality that isn't going to be implemented until after the big deployment so there are things that are Delayed and you can find them here tagged with Drupal org seven one So when you get ready you've clicked around you found an issue and you're getting set to report it It's really helpful Important to do your best to look and I have a link where you can search in both the known and the deferred issues To look for the issue Before you file the bug report so there will be lots of people looking at it And we want to keep the noise of duplicates to a minimum so we really appreciate that searching But when you found something after you search You can go to the project slash D7 QA. So this is the key where we're going to keep all of the community reported bugs You've done your searching you create a new report There's a little bit of templating done here for you. It's in the category of miscellaneous It's hard to really judge which of the components it is you don't need to worry too much about the component But getting the bug report is already prefilled Normal priority is fine if you don't know what to set it to and then what we're looking for is if you saw it on a page Do put the URL in like cut and paste where you saw it That is one of the most helpful things that you can do so for example I might want to open an issue about seeing the error message at the top of adding a new issue So I would go back for that See and just create that so I can recreate the Find the existing piece I don't see it now. So Neil, do you have any idea what is up with the error message that we saw when I was clicking through earlier? That one looks like it was part of the Notification code so that came up As the site was actually sending out the email notification Okay, and you can get those Issue notifications and any other email this site Sends out you go to your user account And fill in a real email address You know since this is real triple dot work data we sanitize out Everyone's email addresses and so we don't accidentally Send people emails so feel free to update existing issues on You know triple core whatever projects What the modules and games you want? And if you use your account on this site, it's updated during the address. We'll get that And you know, of course that will be reset overnight or whenever that time is In your time zone And yeah, I saw that that issue and that's something having to do with authentication going out So yeah copy and paste the error messages. That's also really helpful Yeah, that's what I was going to go demonstrate actually But it is just as good a point that I'm now not sure how to reproduce the error message that I saw And unless I can reproduce it, I'm not going to open an issue about it Um Reproducibility is really important So as you're working through things pay attention to the steps that you're taking Because it's we have to have a way to to see the problem in order to fix the problem So if I were opening that issue I would paste it in the full link to where I saw it It is possible that the data because it gets reset every day that this link might not be good tomorrow Because it's test data that isn't getting imported from drupal.org So then explaining the problem So this is where you paste the actual text of the error message if you saw it Cut and paste it in so that it's in the issue It's really helpful for developers and then the steps to reproduce. So It's helpful to know what user account you're logged in as because sometimes they're related to permission. So if you just say I test with some fake users, but if I was logged in as myself When I go to and I try to paste URLs whenever possible So that it's easy and people don't have to who are trying to reproduce the bug or the developers. They don't actually have to go Click through so we might have seen it on one page, but when I go here, that actually I would have been to node Add so anyway put in that part Um, and I create a new message create That's a new issue I get the text So a specific issue can be getting people to be able to reproduce the issue And then when you're all done save it and I like to keep the titles in a subject verb kind of A short concise way of explaining in regular language what the problem is something short like that With the rest of it in the body so that it's easier to scan through the issues But everybody has their own style for that for sure. So anyway after you create it and you save it That issue is out there for other people to look in the qaq to see if it's an existing issue as well If you need to you can annotate the screenshot if you're pasting in the error message It's not likely that you need to do that, but it's very very helpful The circles and arrows of where to pay attention to a problem if it's subtle So once you get that open Then there's also A second step so in order to keep the developers as focused on possible as possible on fixing the issues So that we can keep things moving along If you're not sure what to test but you want to lend a hand you can verify the reports So a lot of communities do this when someone reports a bug There's either somebody with the organization or a community member who'll come in and reproduce the bug and say Yeah, this is worth spending developer time on we're going to have them take a look So you don't even have to necessarily open issues of your own You can go to issues and see if you're able to reproduce the bug that was reported And if you are able to reproduce it and you change the issue status From active to needs review The reviewed issues are the ones that will get the developer's attention So If you were able to reproduce that the bug report was clear great If you feel like you can add more information from your experience feel free to edit the issue If you can't reproduce the issue It's helpful to comment on it and explain where you got lost so that the person who originally opened it Has an opportunity to see or to explain and clarify so that we can do more with the issue itself And then finally when it's in that needs review state then it will go into the Into the queue for the developers and when the fix comes out We'll be able to verify those and get them marked as closed. And so that'll be the basic process of working through Qa as we get ready to deploy Neil did you have anything that you wanted to add to that? I can't think of anything right now Can you talk a little bit about the considerations that affect choosing the actual deployment date? uh, yeah for deployment, uh you know the We have a couple of big things left as far as performance testing and performance improvement And So I don't think we'll be able to fit the limit It'd be nice to but you just you know, you don't want to rush through something Before you get on an airplane uh, so We're going to look at scheduling the deployment uh sometime after to the contract Not hopefully within a short like within a month after that uh And yeah, of course, I'll have a lot of Factors to go into Claiming that you know trying not to do it during any big triple cancer Of course, the three itself will come a bit of downtime It takes about a day to work that great On our staging site, uh the production server will be quite a quite a bit faster, but I don't know. Yes still gonna be a kid downtime Yeah, and I have one thing in my notes that I forgot to do which makes me Sad because I'm super excited to thank the folks at QED 42 They've uh taken one of the folks who work for them. Her online handle is rucky Um, and you'll find her in the issue Q four hours a day Um, they've donated while we're doing the community QA process She's been helping a lot with the automated testing already going through and making sure that the tests are really doing what they say They're doing and she will be verifying bug reports as well. So as the community stuff comes in She's going to go through and look for steps to reproduce So that we can move things along to developers and I think that's going to be a huge bonus And I cannot thank them enough for the time that they've been helped given to us for keeping that the testing process going So Yeah, big shout out to QED 42 The last thing that I wanted to mention Where should people look for the opening meal as far as having the community come in and start doing QA Do you think you're ready for that this week or next week? Uh I think probably next week, uh, we'll get that open soon as possible Yeah, this week, uh, it's I guess 29th that's being recorded. Um There is too much going on this week. So Yeah, we'll talk about it on Monday and should be open sometime next week. I think And where should people look for the announcement? Uh, the association blog, uh, Tatiana has been doing A general update posts there Okay, and I'll make sure that we get a posting out to the dribble dot org improvements group as well Yep So that people will have a clear idea and I'm sure it will be tweeted about And I want to also give a huge shout out to everybody who's watching this thinking about Uh, giving a hand with making sure that we have a quality deployment It's so important to have people looking for things that the automated tests can't catch And they definitely cannot catch all of the subtle issues that experienced Drupal dot org users will see so thank you all very much And if you're interested in the process of QA, I just want to do a little plug for the core conversation If you're going to be in prog and you want to talk about How to take care of this place that we call home From a very logistical level Consider checking that out