 Good morning, everyone. This is Dan Gordon. This is a special enablement training for November 30th. We also have ring one on Thursday. This is the Tuesday one. This is two-parter with some focus on Microsoft's Azure DevOps, formerly known as VSTS or Visual Studio Team Services. I'm going to go in and show you what we have collected information about that. This first enablement, we're going to focus on this, making sure everyone's got a baseline, a familiarity of what comprises Azure DevOps. A little bit more information to some of the capabilities around that at a high level, and I'll show you where all that information is, and then some kind of overall ideas about some strengths and weaknesses. Then on the next enablement, there's people who go deeper into positioning and how we can approach, and what we need to do in a situation with Azure DevOps. You'll notice that I am running off of local. I've had some pipeline issues as I was trying to get the screenshots in, and I'm running everything off of local right now. But this is effectively our website, as you guys probably recognize, under DevOps tools. This is where we are collecting all the comparison information about different tools in the ecosystem. I'm going to go back and also mention that you've seen this graphic, which is continuously evolving. Also underneath that is the button to get you to that same screen, which is DevOps tools DevOps landscape. So this is the path to get to the Azure DevOps information. So one of our competitors and the one we're talking about today is Azure DevOps. You'll notice they do appear in multiple columns under plan and create down here, and under verify and under release, and you'll see why in just a minute. So if you click on that, that'll take you to the page that covers our general tools page and comparison page for Azure DevOps. There's a bunch of information on here, including the summary information about just what it is. We do point out most of the information we have, which is that this used to be called Visual Studio Team Services. Also in a way, it was also TFS or Team Foundation Server, and recently on September 10th, Microsoft renamed the whole suite to Azure DevOps. Now what they're doing, and it's important to know is what used to be Visual Studio Team Services, which was their SaaS developer tool offering, is now called Azure DevOps, just plain old Azure DevOps. The TFS, which was the on-prem version of their developer tools, their developer suite, is being renamed Azure DevOps Server. What I've got from their project management when they didn't know I was GitLab, and talking to them was that they're releasing on a three to four month cycle, and that is specifically for HBSTS, and they expect that in three to four months, then those releases that are going to Azure DevOps will trickle down, and then be released in the TFS version of Azure DevOps Server. Azure DevOps Server is getting the backseat, and it's going to be behind an adoption of new features by about three to four months. Now, this is actually not private information or super secret information. This is actually available on the roadmap, which is linked to here. They've taken to doing what we've been doing, not necessarily because we've been doing it, but they also do have their Azure DevOps feature timeline up. You can see what they're focusing on in the quarters to come in the months to come, etc. This is what they plan for Q1, Q2, Q19, etc., similar to what we do, and you can also see their current features as they're adding them again similar to what we're doing. However, if you go, they take away the access to the issue definitions or description of the problem, and if you go to most of these, they are pretty much just empty. They have their idea up there. I don't know how solid this is. It is what they're presenting to the world as their roadmap. You'll notice also that this column here where it says server, that is for Azure DevOps server or formerly TFS. You can see the lag that they have that they expect with these things. As Q4 2018, they expect sometime in 2019 that these features will trickle down into TFS or Azure DevOps server. Going back to the main page. This is the page that we've had, which is a high-level overview. It's a place where you guys can collect information. It's a markdown page so it's easy to add to, where we collect resources like links to their main pages. Most importantly, where if you hear anecdotes or you hear complaints or people talking to you as you're talking to customers or even pricing information, this is where you can capture that. For example, this is the blog that the Azure DevOps product manager put out when they released it or announced it, and it has some useful information such as their plan about what's becoming what, and this is where I'm getting this information from. As well as information on Hacker News and the general feel and response about these changes. It also has pricing information, pointers to pricing like the older pricing, VSTS and TFS, and then the newer pricing, which is all the feature by feature low-level, bit by bit comparison of what they do versus what we do, and this goes on and on and on, lots of information here. But that's what we had. What I want to show you is the next level, which is the overview. This overview is meant to give you an overview of Azure DevOps. It's like us, it's a big product that covers a lot of pieces. I wanted to go over that piece of it, make sure that everyone had an understanding of what, when we say Azure DevOps as in the former VSTS, what we actually mean by that. This is what it consists of, and this is right now listed in their order, how they list it, not compared to how we order it. So we'll have to just kind of make the connection. So they list pipelines first. Pipelines is basically new, a new piece of their offering, and it has some of the call outs, has native container support similar to what we have. You can save containers to any registry. There's a lot of information on these pages, and I won't go through every single one, but I'll point out areas, you can get a general feel. The bold guys here, Linux, Mac OS, Windows host agents, deployment stage, release gates approvals, hundreds of third party integrations, and in this case, design pipelines by YAML or UI. These are bold because these are things where we are not, we're falling short in those areas compared to what they have. In particular, for example, we have Mac OS agents, but we don't have hosted Mac OS agents. We have the ability to do approval on merges, but we don't have release gates, where you can set different behaviors, having a release go out based on, for example, test results and approvals and other criteria. We have integrations, but not hundreds of integrations. This is a play you'll see a lot as we get into the release area. And we do design pipelines by YAML, but not with the UI. And this area is fairly new for them, and this is also the area where we have the most distance in what we have, in what they have or so what we have. There's some cost information here. The cost structure with Visual Studio, Azure DevOps is a little bit messy, but the gist of it is is that you get some free amount of capabilities. Pipelines is free for public projects, and this is how they're supporting open source software. And you get 10 free parallel jobs in unlimited time if you have a public project. So that's their nod to open source. If you have private projects, you get one free parallel job in 1800 minutes per month. So that's, I think it's 30 hours. One parallel job is like one job in one pipeline at a time. So that's not a lot, that's a very minimal amount. And if you want more than that, it's $40 per parallel addition. So each additional parallel job you want to have run. So like for example, when we look at GitLab in our auto DevOps, for example, we do tests and we blast out and we do, what is it? What is it? Seven security tests at once, I think, right? And we have them all running in parallel. Well, you would have to pay an extra $240 just to have that same level of capability with the Azure Pipelines. And then self-host it. So if you provide your own agents so that you can run the work on your systems and set up on their cloud systems, then you pay an extra $15 for each parallel job. So price-wise, it can get up there. Now that's just for the Pipelines. Another piece, a main piece of what they offer is boards. Similar to what we have, they had this in VSTS and TFS previously. So this is just a new UI and the older capability. It has things like work issue tracking, backlogs, Kanban boards. I'm gonna skip these two bold ones for a minute. Scrum boards, et cetera. Custom, they have customizable work item flows. What they do a little bit better than we do is they have team dashboards, like real dashboards with summary reports about the team and about the issue tracking and about the movement through BSM like something that we're working on but that they're doing better right now. Cost-wise, all of these are free for less than five users. But once you have more than that, then you start paying $30 a month per 10 users roughly. Is there cost? They also have a place in the calculator where you can do $6 per user, but they have some mixed pricing information. I think they're still kind of ironing things out there. However, it's basically $30 per month for 10 users if you want boards to have issue boards and whatnot. The artifacts, so they have an artifact repository, binary repository. It can support Maven like ours can and NPM and NuGet packages, which are all Windows kind of focused packages. This is something that's a little bit more than what we have at this point, although I believe we're headed that direction. They also have a caching proxy of external repos if I ask for something that's at an external repo, it will go get it, bring it local and then cache it so I don't have to keep going out and getting it. And if that repo shuts down or erases that file, I still have it. And that's most of the artifact repositories have that. And I believe that I'm pretty sure that's also an issue that we have that we're working on. And then they like us, they have artifacts integrate natively with the pipeline so you can easily reference them from the pipelines. So this is a new capability that was in TFS and VSTS. Again, free for lesson five users and each additional user you want is $4. So again, they're adding everything you add on compared to what we have out of the box, you're adding more and more costs here and there. And that's kind of the takeaway I want you guys to get here. They have repos. They, again, from VSTS and TFS, they've had these. A lot of functionality here, similar to what we have. The only one they have that we don't have is support for TFVC, which is team foundation version and control, I think it is. Who cares? Nobody cares. Only people who actually were using team foundation server before, it had get care and most people are moving to get, which they also offer. And as I mentioned in the tool comparison page, Microsoft themselves are pushing everyone to get. So this is a legacy item that isn't really important. So this is included with the $30 per month for 10 users that you get when you get team foundation, when you get to keep doing that, when you get Azure DevOps. Now here's something that we almost entirely don't have, which is test plans and execution. Again, this is something that they brought from previous VSTS and TFS. Test feedback, exploratory, basically being able to do exploratory testing and actually capturing pieces of the results and pulling those into issues or what they call work items. Test planning and tracking and execution of even running automation, test automation, load testing. These are all things that we don't have currently that they do offer that they've had for a while. The load testing is only if you're using their cloud services. They also have user acceptance testing and centralized reporting. We do have that in centralized reporting in the terms that everything comes back into the ML with the ability to get like browser performance results back and even just having review apps available to allow people to actually look at the code. Now this is a big one though. This is actually really a fully separate price, $52 per month per user. So again, if you're in a situation in a sales situation and they're saying, yeah, but what about tests? I have that, you guys don't have that. Well, that's actually a full separate product and price even though they grouped that in as Azure test. And then kind of a couple of important points to point out. So these are things that they do have that are separate but extra cost as well as the testing that they don't call part of the Azure DevOps suite. It is part of Azure in general but they have Azure monitoring, APM infrastructure data services. We have all of this. It's app centric. It's separate from Azure DevOps in particular. This is kind of the pointer that describes the kind of contort in this unit or sorry, that describes their tool. Sorry, that was a different one. It's paid by use. So this is just an extra you pay for use. Ours is again, part of our package. They have the full-blown IDE that they've always had which is free with your subscription. And they've also now got the Visual Studio Code which is a super light IDE similar to Adam and we're building out the web IDE. Which I think is even better because it has the integrated configuration right out of the bat. So we have something comparable. Not available as part of the Microsoft offering but we have, so this is something we wanna be conscious of and is valuable is security scanning built in. They don't have anything native that does code scanning for security situations. They do have security, operational security tools but not development security tools as part of this suite. Container registry they don't offer but they do integrate and allow you to push your built containers to any registry. They don't have a notion of review apps. They don't have built in deployment scenarios. So you can with their pipeline you can define behaviors and each stage is an environment. So you can actually, you can define it but it's custom done. So they don't have out of the box canary or incremental deploys for example. They don't have feature flags which we've just added and are continuing to, we'll continue to build out and they don't have chat off. General notes, all the paid plans include a stakeholder user. So this is something we've heard about about the request for like a guest user that can come in and see stats and get graphs and understand what's going on without actually having to log in. So they do have that. This person can contribute to work items on boards and view dashboard charts and pipelines. That they seem to be executing on pretty I'm not sure as far as how many features they drop off but they are keeping a pretty good pace as far as releases as a concern. So they have managed to work in a DevOps way and become more continuous in delivery. And that is for Azure DevOps or VSTS not as much for TFS. It looks like that's on a much slower clip. And then for TFS on-prem, the pricing is kind of messy. It's pretty clear that they're trying to push people off of the on-prem offering and onto the cloud offering. You basically buy a visual studio license and then you buy Azure DevOps users and they actually even state in this TFS pricing link here. That way it's easy to convert over when you're ready, right? $6 per month for every user plus a license. The license is either $45 or $250 depending on what level you get. That includes test manager and all these other a couple of those other items. So let's see what it do. Yeah. And so right here is where they actually say like, this is really great because it'll help make it easy for you to get over to Azure DevOps on the cloud. So it's pretty clear what their intent is here. Just real quick before I stop and open for questions. These are the graphics I didn't quite get in but these are just some screenshots. Let me blow this up a little bit here of what their tool looks like. It's also helpful to have a sense of that. These are their Azure boards. Again, similar tires for basically you can set your flow of your work items. They do have a nice feature in their boards where you can actually interact with the underlying tests of the work items and set those. Their repos, I can make this any bigger. This is their repo. Looks pretty much like ours. It's access to files. They have a built-in editor. They have, you can look at the commits. It's nice that they have the graph of how the commits have where they've come from and how they've come together. Something that we heard even from the analysts is important even for folks who are comfortable with doing YAML and being on the command line because it helps give a good overview of things. We have that visualness when it's running but not when we're creating it. Their data which comes into the pipeline. So we capture all our information around the MR. They capture it around the pipeline and they have pretty nice summary data about like tests for example within their pipeline output. Lots of integrations as you're building your pipeline, drag and drop and releases which is we call CD. That's kind of just what they call pipeline. They call it CI CD and it's all just pipeline but they have a nice display of the environments that releases are going to. Here's dev, prod and dev QA and prod and the ability to interact with it from there. Artifact repository allows you to store mixed Maven NPM and NuGet packages. This is their whole test plan tool that they brought into Azure DevOps where you can actually define your tests and the scripts and you can actually run them, execute them from here and pull back the results. Very manual but this also will do it automatically. Something nice to have again, they're pretty advanced with tests and test planning because they've been doing this and they've been up against HP and ALM and all that for quite some time with this and so they've got a lot of great graphs and built in dashboards about how the tests are coming out. So that's kind of the overview of Azure DevOps. I want to jump back and I'll point out strengths and weaknesses. We're gonna talk about this on Thursday with some added information. I want to stop now so that there's time for a few questions. So let me stop the recording.