 Thanks Mike. It's probably more useful to start telling me when it's like 15. What are we up to? No, it's 30 minutes. Cool. Whenever we have audio, we have audio. This is as appropriately titled. You got that? We're good. And I was supposed to talk for the gentleman, and do I want to keep talking? Can I start? We're good? Sweet. I said let's start. And because I care about time, mostly because I have another engagement. I don't mean to be rude, but I got to go intro a guy. And we wrote a very funny antler joke, and I don't want to miss it. So, pretty excited about the antler joke. So, come to the keynote, get your antler joke, you'll be fine. This, unfortunately, has no jokes. So, concourse. I'm going to start with, this is an intro to concourse. This is an intro to the tutorial that I started writing when I discovered concourse a year ago. Concourse was coming out of Pivotal. They had suffered under many, many CI systems. And there were some things they wanted. They wanted a dashboard that does the animation. You can sort of see work flowing through. Gray means the job's never been run before, but it looks more dramatic to go from gray to green. So, the idea here is that some inputs have changed. We've got some jobs running, which is the green with the yellow. And as the work succeeds, that value of what was being tested. So, let's say the controller, that's a resource, let's say it's a Git repo. What's being passed through is one of the hashes. A specific commit is sort of being passed through as what we're testing. And it only gets passed through if the job before it passed. And so with this, we get a pipeline. And we might be modifying that controller task. We might just be producing things. We have, a year later now, we have this wonderful ecosystem of what we call resources. Resources how we touch the external world. And so, I recently was forced into a situation where I had to go back to a client. The first client I ever forced this on them because you get very excited about things. It's no fun to play with stuff at home, so I took it to work. And we used it for everything. And we were having great success. But I heard that one of their employees was unhappy through a very roundabout public channel of Reddit. And so, I had to ask, are you okay? I'm sorry, because I take responsibility. I was very excited about concourse. But a year later, I'm still really excited about it. Like, of all the problems in my life, CI is not one of them, which cannot be spoken for most people who introduce CI into their lives. And so, I'm still really excited about this. And it turns out the client is still two a year later. So, I'm really excited to share this stuff. And I don't want, you know, getting started is, I think, the next step. So, when a year ago, there was no, it was like, there was this vagrant up, and then that was it for the tutorial. That's not quite chronic. So, I started evolving over time, and other people have contributed to it, and I thank all of them for it. So, a little bit of a demo. This is concourse's own tutorial. This is concourse's own concourse pipeline. This is the pipeline that builds concourse. And the yellow flashing means that job is currently running. The idea of this is that you might put it on a TV. And more dramatic is if you glance and you spot that, you know, one of them has failed. The idea is that would draw someone's attention. But you can also have resources that send messages to Slack, HuBot, or wherever it is that, you know, you'd send messages to bring attention to something not working. This is the one for one of our projects for Dingo Postgres. The idea is that we're building a Bosch release in the test flight job. We're testing it on a Bosch light. If it works, then we'll go and do a bigger test on a vSphere environment. And then whenever we want, we will run the ship at job, and that will cut a new final release. We then have another pipeline that takes that release and turns it into a pivotal tile. It's somewhat arbitrary to split it into two, but it's sort of like two teams. One team working on the release, and the other team working on the distribution to wherever. And so you get this nice petitioning of deliverables. This is me building something for your team as one pattern that we use. And so here's the tutorial that we'll see how far we get through in the 24 minutes that's remaining. I guess 24. Is it 24? 26. Bang. I'm going to use two minutes to tell a joke. No. There's no jokes in this. Let's get through. There's together, with 26, 25 and a half minutes, I will do it. And you will see how good I am at doing my own tutorial. The idea in the tutorial is you would boot up your own concourse using Vagrant, and that's also what you'll find at the bottom of the homepage. They now have a nice, very beautiful, as going 1.0 they did a great job on a website. It's a very simple pipeline. That may make you think this is cool, but it may not teach you what's next. So what we'll do is we'll sort of start from scratch. And I sort of wanted to build... I wanted to bring the ideas together one by one. You know, you're trying to linearize all these complex ideas and build it up to where you feel, ah, I get it. So let's start with the smallest thing that we can play with, which is a task. Just imagine it's like a shell script. You want to do something. It's a bit of functionality. You want some inputs, you do something to it, and there may be some outputs. Happiness, sadness, which in Unix land is one or zero or one. That's... If Unix was recreated today, you'd probably pass emoji around, but we have zero and one. That would be awesome. Just piping emoji. Sorry, let's get started. I will run this command and then talk about it. And I will do it. So I have targeted a concourse, and it's actually the concourse that's ours running on a vSphere environment somewhere in Europe. When you do this, you'll have a vagrant machine. And the reason for doing it is the Docker images that I'm using are big, and it's a bit sucky to bring them down to my laptop on the same internet that you're using to do BitTorrent. And I don't want to be rude. So the idea here is amongst all of that, that was the output. So that's a little messy, perhaps, but verbose. When it comes to CI systems, you care about nothing except when an error happens. And at the time an error happens, you want to see all the verbose that you can get. So let's live with verbosity. All right, but the outcome of this. Now, why was that? Well, let's look at this task. A task is described as a YAML file. And the most important part in the context of where we're at right now as we're just getting started is this bit. This is us running a command. Where does this get run? I mean, I could run this command here. We go echo hello world. And there's nothing for you to understand why that that's not where it got run. It got run inside a container somewhere, wherever I've got concourse running, but inside of some image. And later on, when we start to build up actual functionality, you're going to say, well, I need Git or I need the CF command line tool or I need the Bosch tool. Where do those things come from? And they come from the base image resource. So when we use Cloud Foundry, there are 1 in 1,000 people might wonder about how do I change the base OS, which are called stacks in Cloud Foundry. In concourse, it's very common to build a bespoke image to use. It's kind of the one you're going to use. It's got your dependencies. Super important, I believe, that you own that. Don't go around borrowing some other teams image. Images are free, build your own. Not exactly what's in it. All right, so in this case, it didn't really matter, it's just some image. But what if we ran... It looks a little complex. Let's just go through this command. We're running the fly tool. Most things in concourse have an aviation naming system. The CLI is called fly. We explicitly target which concourse. Like many things in concourse, there are lessons hard learned from living with Bosch and Cloud Foundry. If you listen to any of my other talks, I'll tell you that one day, I ran Bosch delete deployment accidentally on the wrong Bosch. Because it's really easy to do that because you have shared state about what was the last one. And so when they did concourse, they said, soon we're not going to do that. You always have to specify explicitly which one you're talking about, even if it annoys you. So that's why we have target. And after a while, you start wishing that, like the CF tool explicitly... You cannot even explicitly target something with a CF tool. Or you have to do a login and then the command and hope that that's atomic enough that you don't have something parallel crossing over. So I like this. All right, we're running execute command, which is us playing with tasks. Typically, you don't play with tasks. Typically, we build pipelines, but we're building up. Let's run this task. This one is going to use a different base image, and it's going to run the uname minus a command. It downloads it for the first time. Actually, it downloads it a lot, but it's finished pulling that base image. You don't really need to see that. You don't need to know that the image got downloaded unless you do because it didn't work. I mean, we've all had build failures, not because our thing was broken, but because something we depended on upon didn't work. And Docker Hub is a thing you depend on in this case. So let's be explicit so that if that fails, we know that's the reason and we can just rerun the task. And here is our output of running. So running, there's the command we ran, and that was the result. And so we can see it's obviously depended upon. In fact, we were running in an Ubuntu image. But let's have a look at that task. So there was the command we ran. You might wonder why do we split the name and the arguments into two, and I can only guess this is because of Go. This is the way you run and execute and Go is that thing. And I just assumed that instead of fixing that, they just said let's just pass that onto the user. And it's annoying. But here is the base image we picked. And we can pick other ones and run other commands. So here we're just borrowing other images. You will, as you use concourse, learn how easy it is to build Docker images because we have a Docker image resource. It is so easy. And you'll make your own, and then you'll use them in your own images. So you get to build whatever dependencies you need, your team, your team alone. Actually, I'm going to say this as many times as like, there are people of Stalker Wayne who disagree with that and like to share them. And I think it's a terrible idea. Well, I'm not sharing yours. I have all my own. James. Because they're free. Now there are reasons, right? Are we all bumping them, or are we not bumping them? All right, I just wanted to... There are many people of Stalker Wayne who are wonderful at this stuff. And most of what I know is coming apart from what they've shared or the concourse team themselves who have their own Slack account. The people that make concourse super engaged in helping you be successful. So once you've learned a few things, they'll answer your questions. Did I just palm that off? I think I did. What did I just say? Don't ask me. All right. Next is inputs. So there's this idea I'll just quickly show. There's this idea that the last task we just ran had no inputs. Like, it brought up a container, it ran the command and we're done. But that was dull, right? The only thing we could use was whatever was on the image and that's relatively static. So let's play with inputs. Firstly, let's play with inputs in a way that's bad, or not bad, but you know, doesn't work. A nervous press of the caps lock. So if you're playing with the execute command you'll see that it complains. And similarly, in a pipeline environment where a pipeline is made up of running these tasks, if you forget to pass an input it will complain similarly. So, what? How do we fix this? Well, here we have the minus I or minus input flag. And then the question is, what's an input? Is it a file? Is it a folder? Do I just point to Dropbox? Where do I? What is it and how am I playing with it? We have a very simple API or experience which is it's a folder of things. It could be a folder of folders but it's a folder. And so let's play. So the name that it complained about was called some important input and by that we give it a directory to the things it will package that group of things up and upload it which is super convenient. You know, if we have enough internet. And again, we're just playing with this execute command to sort of understand. I mean, you can use execute. It means you can start to do your Java testing in a test environment rather than on your laptop. You don't need to have all that stuff. Super wonderful if you don't have a lot of bandwidth to bring down dependencies. Like Maven loves to download the other half of the internet. So like it's the happy counterpart to BitTorrent. One downloads movies the other one downloads all of Java. And you don't always have to provide. If the name of the folder you're in is the same then it's optional. All right. Input. So now we have a way to bring things into our container. We're still running a fixed command but we have variable content to work on. But what we can do now is we can run a variable script. Like let's let the inputs this is an arbitrary, this is what every pipeline does. We bring what commands to run, what script we're going to run is as variable as the things being pipeline. Like it's something, it's almost like you have to keep this idea in your head. There is the work we're passing through and then there's the code that we're running and they're passed through similarly. So sometimes you can get confused not only about what not confused not the right way. Somebody have to keep track of not only what's being worked on but what version of the script is being worked on at the same time. So we're going to pass in some commands and this is, we'll call them task scripts. All right, so task number three. There is no way I'm getting through all of this just to manage all expectations. But more importantly by the end you should go, right I know how to get through the first seven the next seven are really easy. All right, so we ran, it's the same idea we ran the uname command but instead of like, sorry instead of running it directly we ran a task sorry, we ran a shell script. This is the pattern we use a lot. A YAML file that describes what shell script to run. Now in order as I've led you through this path in order to run that shell script we need to bring it into the container somehow. We bring things into containers with inputs and then we want to run it so we have to reference and we said that that the inputs are just folders of things and so let's go back up to here when we ran that the command we ran in the previous one was the ls command the list, show me files so if the name of the input was some important input you will find in that container a directory called some important input. It's so easy and so if you want to run a script that's inside that input you just reference it relative to that that name I'm going to say that the input is called zero three task scripts why would I give it that ridiculous name because that's the name of the folder so either I have a clean name and I have to explicitly pass it in or I just give it the same name as that and I'm going to explain that better. But that's why so we have this shell script here is our shell script once we're inside the container we can find that shell script underneath the input name slash script name so when we ran it it's going to take the time I could also go explicitly pass that so explicitly provide it but because it's the same as the name I don't have to I could tasks show name this is a terrible input name let's call it scripts or CI that might be a good one oops if I change the name of the input therefore where I find it on that machine is going to change if I run my fly command my execute command we'll get the error that we don't I'm not providing the CI command I have two options either I could rename the folder that I'm in or I could explicitly match the two so we're just saying this is the folder so we're building up this is the idea of a task you'll see this so often that if you don't follow it right now you'll play with this so often that I just wanted to start with the smallest possible thing that's us so let's build a pipeline the basic pipeline the idea of a pipeline is the same idea so in a way we've done three steps prior to what's on the website if we look at the website you can see here that we have this run section we saw that before in each of the three steps we run something we specified the image resource I don't know if there's actually other ways other than docker image but conceptually there is and we said that there's Linux now this is where I do know there is a windows environment that you can run and a Mac one why or you might be building executables or final deliverables and you send those requests in parallel after those machines they'll send the results off to wherever so you can get the tasks running where you need them and that's why on a windows machine perhaps you're not using docker perhaps alright we're uploading a pipeline and it gives us the URL of where we'll find our pipeline let's go look this is so exciting this is a pipeline you must be proud at the top it's blue blue means paused by default a pipeline starts life not running paused why because it might be destructive like if once we learn about triggers and those sorts of things your pipeline might run straight away and you might not quite be ready for that so they just default to paused it's relatively simple this is all our pipelines of what we play with whereas my hello world down the bottom we click on the little arrow with our baby voice and now it's unpaused there is also fly whereas it's going to be unpaused job command you can run so you can script it in fact and it warns you there to remember to unpause now this pipeline doesn't have this pipeline has no inputs you'll see the inputs are the black things these represent some external thing in the world Git repo S3 object that's another good input what these are wonderful ideas Docker image whatever you want to process an input if you can make your own resources it's a solid line that means it's a trigger if concourse discovers a new version just get polling for new versions if it finds one it will run the next jobs if it's a dotted line then it's a dependency but not won't trigger compare that to our job it's gray because we've never run it before it has no black thingies coming in so no way to trigger this unless we manually give it a kick which involves clicking alright the little plus button is a way you can manually start the job and it finished very quickly let's make that go again so yellow means running and this one black line each black line represents a task little hard to imagine this when there's only one so let's quickly look at a more complicated task here is each of these little tasks running they're an arrow means that it fetched something from the internet the little carrot with an underscore symbolizing a prompt means a script was run, a task like we've been playing with and an up arrow means we push something out to the internet concourse has no state that you can borrow there is no build numbers no little handy things that you might have used from Jenkins or whatever anything that you want to know about you'll pass through, you have to externally manage and we have say you want to do a version number then that's called we have a sem ver a semantic versioning resource and you can manage that yourself now you own it, it's not hidden inside concourse it's yours so that would be a more advanced one I have this little diddy in my head from my grandfather alright let's look at our pipeline task extracted into resources let's have a look at our pipeline I sort of pointed out before that if we let's compare the pipeline to our original task our original task ran hello world you can see down here oh look we have that bit we have the image resource oh there that is and we specified that we wanted to run it on a worker a Linux platform and that's that there in fact this section here is exactly the same as this little snippet that we had and so you can put them exactly into the pipeline but your pipeline is going to very quickly become a very long yaml file and one thing that most of us do is we move it from flattened in the pipeline and putting it back into the file like we've been playing so let's go through the thinking of this we're going to keep this file this is our task file this is the file we want to run this is the file we want to run that describes how to run hello world but when our pipeline runs if you can just imagine where we're going the pipeline will run, it will run our task a container will come up busy box whatever how will it actually get hold of this yaml file with an input so we have the idea of inputs now on our laptop I use the fly execute command like I had the input on my laptop and I just pushed it up in a pipeline we need to get the input from somewhere and then pass it in these are called resources that's the little up and down arrows so in step 5 at this so this is the change from red to green like Bosh we're going to remove the task the task has a name and previously we explicitly described that config and now we want to push it back out into a file in order to do that this is that we use the file and that is you know that path that was going to be in the same way so let's talk about resources resources are things on the internet things we might get things we might put and the type of this one is a Git repo conveniently this very Git repo that I'm playing in it's not getting it from my laptop it's getting it from GitHub so concourse is running it's going to go and get it's going to download that Git repo it's going to give it the name resource tutorial and then my task all that yellow, green and blue maybe more confusing than okay so there is our two steps we went from one step to two first step get a resource second step use that resource to run a command remember we have two things flowing through a pipeline things that we want to work on and you know you might wish it was a little more explicit what was what but in a way it's nice that those two ideas have the same sort of system alright so we're going to find that under here and what will it look like will it look like this command that we've been playing with that's exactly the YAML file so instead of fly execute concourse is going to fetch that and run it alright now we still have no trigger so we still have to press plus to make that run alright so now we have two steps we have the fetching the down arrow the getting of a resource it's not explicit here that it's a Git resource that's back in the pipeline but this is sort of the Git information and then this step use that to run we go back to our pipeline view which is what might be on the dashboard we can now see our black box oh very exciting you're a patient crowd and I appreciate you for it dotted line means no trigger alright some of you are following along and open to interactivity I actually learned once that in a presentation you've got like 30 seconds to teach the audience whether that's going to be an interactive session or not we are well past 30 seconds so you're off the hook alright we'll skip that uh huh let's get to 10 and is that funny? there was no funny there's nothing funny there's nothing funny about this this is serious work alright it's very serious you know it's serious because I'm not telling any hilarious jokes I care a lot about this it's really interesting stuff so now we've passed through resources for the purposes of providing the scripts to run so now let's provide an input resource for the normal reason you'd want input switches things to work on so in this one we're going to work on a go application go language you don't care but if you use go gray if you don't we're going to bring down some source code and we're going to run its own tests now in this case it is a go application and I choose to use an image that already has go built into it so we're going to use this image um we're going to move to having you know there's so we now have two inputs one is the input that will have the commands to run which happens to be this tutorial and the other one will be the app with some code in it that we're going to run the test this is not common typically you would put these two things together in such a simple example the resource app resource would normally be the one that had the commands in it that's what that's the pattern you will see there is some sort of central repo that's that's being worked on and it's the most convenient to put all the pipelines and the shell scripts in our CI folder that's the common common place to put it when you start looking at other people's as we all do we copy and paste other people's work and you copy and paste that pattern we're now going to go down to SP standing for set pipeline UP which means unpause pipeline and the new pipeline name will be simple app so we're going to give up our um give up our hollow world pipeline and then we'll finish so now we have our two inputs one of them is a trigger the way triggers work or it sort of pulls every minute or so and then it will start the thing so now we have this yellow I didn't have to press the plus button let's have a look how it's going so we have our two inputs get the tutorial we got the app and then the next task downloaded that uh next task downloaded that go lang container image from from docker hub and we ran the command that uh that let's have a look what command we ran we ran the run task run SH and you can sort of see that was the command we ran there is um I'm going to wrap up there is uh that was step 10 on the just the read me of that tutorial and if we go to the top uh just in this read me which is like the minimum you once you've gone through these 14 steps um you now basically the basic understanding of how you might go about doing anything you'll understand the idea of resources we haven't yet done a push resource of modifying something which is obviously super important otherwise you're just testing things you know you can manufacture goods and services alright that's probably overstating it um this I call these things robots which in the context of of our keynote from Monday is both appropriate and hugely inappropriate um but as you can see a large portion of the you know what's happening at Stark and Wayne we have pipelines for both client things although most client things have their own pipeline a lot of our open source uh miscellaneous tasks and uh this is working out well for us uh to follow on in both in terms of that that concourse tutorial repo which you can follow along at home we will start to take some of the this training and start to put it on this uh videos site and kind of has I've always been envious not envious that's the wrong word um a huge fan of the rails cast from when I was doing rails um so with that sort of idea in mind we'd like to share both introductory things and little edge things I know we've always done blog posts uh I think videos are a nice way to sort of describe things and so over time you know hopefully they become you know something you can come in and use because we can't all come together like this as often as we'd all like except unboxing day in Australia concourse tutorial finishes at 11 sharp that's on the cricket um anyone would like to ask a question that's awesome I appreciate your all time sensitive and scared um yes Michael um no the excellent question um in that sense and and obviously when you make your own image you there's a base you might want to borrow so if if this was of some importance to me I might build an image where it's a you know from that that one I just used build my own but at least now I own that that tag right so it's you know otherwise it might just wander along at 1.6 something else and I break and I don't know why um and and you look at some of our pipelines often there is this one job that builds that image and its inputs might be the upstream image and some other things and so it might still be building that image full time but at least now we're tracking it and we can see if it's you know that's the root cause of things uh sorry his question was why I made the comment that I like to build on my own images why did I then only use other peoples to which I snarked at him and answered and we have another question the question is in the pipeline in the pipeline we say that the task has a name and the question is why why have a name and why is it arbitrary I'll show you where it goes and then you can say ah that's kind of useful um so let's have a look at our holo world actually it was not that one it was going to be where are we this one so the name of the task is in the black bar and so you can sort of if it fails you have ability to take that token back to the pipeline and go right what was that supposed to be doing um and it doesn't have to be you can put white space as well I don't see a lot of white space ones but you can put a white space string in there that's what that's for so you can go and find thank you very much for coming