 All right everybody. Good morning. Welcome to chat ops for distributed teams fun and profit My name is Michael Sharon. I'm a technical architect for Acquia Been a Drupalus for about six years. I was raised in small Drupal shops So I'm familiar with a lot of the problems that small shops have in terms of communication when they're working with Folks that are distributed around the globe or you know different states across the country. However, you are distributed Hi guys, my name is Chris urban. I'm a delivery manager at Acquia Michael and I worked on some projects together my responsibility is more in Overseeing let's say risk mitigation for larger projects in terms of what is being delivered by our team My background is project management more much more technical. I come from a marketing agency background a lot of experience in Digital strategy pre-press things like that old-school So why chat ops I think I found this week in talking to folks about my session that this is a term that not a lot of Folks are familiar with so chat ops is using tools like Slack or hip chat or even IRC in order to integrate your teams and get them to use these tools in order to Accelerate their workflow to facilitate communication With that communication right, so one problem we see very often with Large teams and on large projects meaning multiple developer teams in multiple time zones Just getting everybody on the same page as a priority One of the one of the first obstacles that we encounter is Having everyone understanding exactly what's happening on a project at any given time with the more people that are involved The more information that needs to be conveyed and communicated getting that into as much of a clear concise format as possible is Paramount Next is having all of that information in one place that everyone has very easy access to so it's quick to research quick to find That answer, you know, I remember somebody was saying they were having an issue with this module with this Ticket where is that information being stored? Last thing is any kind of ceremonies that you have if you're using agile Scrum or Kanban actually quick show of hands who here is using agile or Kanban right now in their development process Awesome. So if you're let's say using a two-week sprint or a continuous delivery model You often have Ceremonies and meetings you have to have all those in all those notes in one place So these are some of the challenges we face the larger the teams get right if small shop or medium-sized shop with five to ten developers On a project. It's easy to keep that communication streamlined Three or four teams in multiple time zones a development team in India development team in South America Development team in Europe a development team in Canada development team in the US design in the Canada Qa in the US all of these teams need to know what's going on Chats is also really great for team culture right when you have your developers and your Qa and your product owners And they're distributed all over the globe sometimes. It's hard to get those disparate groups to coalesce as a team right and Having a place where they can come together and they can share funny images They can you know create gifts you can integrate chatbots with whatever tool you're using in order to give karma So kind of like a you know like a reddit concept Where people can upvote each other inside a slack chat room and say hey, you know Mario's a great guy and he really helped me out with that project So I'm going to give him karma points and these points are totally arbitrary I mean they don't actually mean anything in the real world other than hey Mario's a great guy And he really helped me out and so now he has more points right Yeah, I'm sharing funny things and it really does help to you know bring your teams closer together Even though they don't share a physical space Another really awesome Thing that you can do with chat tools like slack is integrating your continuous Integration system how many folks in here use something like Jenkins or Travis or some other CI tool? Okay, awesome So sometimes those tools tend to get ran in isolation right you have someone who's acting as a deployment manager And they are managing your builds when those builds either complete or fail or if they trigger automated tests It's very useful to have the results of that work being put out into a place where the entire team can see it right often your You know day-to-day You know frontline developers don't have access to your continuous integration tool for security reasons But it's useful for them to know what that tool is doing right especially if your CI process is Sophisticated enough to where maybe it knows if a specific commit has failed You know that developer would then realize hey This is you know potentially my commit that has a problem and now I can respond to it without having to be told Hey, your commit has a problem, right? So Jenkins integration with slack and this is just one example of integrating You know a CI tool with a chat tool. It really is super simple I mean if you've ever added plugins and Jenkins this should be a very familiar process You know you just go in and pause that that's going real fast Whoops So right so you just you know you have the plug-in. It's super easy and then there's just some real basic configuration you just need to add there is a key that you get from Slack and you add it into Jenkins Right here. You get this integration token from logging into the slack website for your team domain You just give it the team domain your API key and what project channel you want this build to post into And you'll see there's several options for notifications at slack consent It can say it'll notify your team when the build is started if it's aborted if there's a failure If there's success if it's unstable meaning, you know, yeah, the build exited successfully But maybe your tests failed right that's that's a nice thing to know and it can also include test summaries Right, so if you have be hat wired up into your CI tool, which I think a lot of us probably do It can actually output the results of your be hat tests into a slack channel so that your whole team can see what's happening So there's also a really great community module called slack and it allows you to sent to have Drupal post Information into a slack channel It basically just exposes a very very simple function called slack send message You can define a default Channel that slack will send messages into But you can also per instance override that and have it send to really any channel in your team domain that you want Right, and there's some you know some pretty awesome use cases that you can have for this like let's say for example You use workbench moderation, right? You have a group of editors who they will get content That's assigned to them after a new note has been created, but it's pending approval This can then post into the editor's channel in your slack team and let them know Hey, this new note is now available for you to review. Go ahead and pop in there and give it a look over You can also do things like Commerce checkout, you know if a cart has been completed go ahead and notify some people You could have like a DevOps channel and you know let your team know when crown runs have completed that kind of stuff Again, this is super simple to set up. You just you log into the slack admin panel for your team domain and Just grab that webhook URL And you can customize the name you can set a special icon, you know So it looks cool and so your slack, you know, you can give it a funny robot icon or something. Whatever you want to do, right? And Then you just slap that bad boy in a Drupal give it whatever default channel that you want Slack to usually post into but then like I mentioned with that slack send message function You can override that and send it to any arbitrary channel that you want to You can define if you want it to have an emoji for an avatar I mean, there's all kinds of fun stuff that you can do right just to make it clever and cool for everybody and I will show an example of sending a message from Drupal into Slack They have a little test Dashboard there so you can just say hey, hello, New Orleans, and it will then post that over to my slack channel Thumbs up. Yay. Hello, New Orleans. I mean, it's really simple. It doesn't take a lot of time There's that slack send message Function that I mentioned You can give it you know, you don't have to even send what channel you want it to go to it'll use the default You can just send any message string and it'll just post it to your default channel, which I think is nice. I Also created a community module that right now is a sandbox We're waiting to get it promoted to a full module and this allows the opposite, right? The current slack module allows you to send Information from Drupal to Slack, but there wasn't anything to go in the other direction, right? And I noticed in the just in the slack world. There's tons of really fun bots, right? But they're all running in Heroku apps or Ruby apps or you know some some other Interface right and so why wouldn't it be why wouldn't we build chat ops with our chat bots with Drupal, right? so I created a community module called slack receive and it basically exposes a Menu hook there excuse me a menu callback that allows you to post information from Slack into Drupal And then Drupal will do whatever you want with it You can build Drupal powered slack bots. You can run Drush commands Which you know I imagine the use case where you needed to clear cache really quickly and you're not near a computer You don't want to have to SSH into the server and run a drush command you could actually do it from your phone using the slack app, right? You could check module statuses you could check or set variables in Drupal 7 right now There's only a Drupal 7 release once I get it promoted. I'm going to do a Drupal 8 port and Go from there one caveat though is that you're basically exposed You're basically making slack as an interface for the Drupal shell, right? And so you you really want to make sure that you're escaping any input that you're taking from slack before You're running it on a command line, right? And I'll show some examples of that So here's the sandbox page for it right now Super exciting. You don't know what sandbox pages look like again. You just use an outgoing web hooks integration in Slack You get a handful of these for free So this is you know and you could do all the stuff that I'm describing You don't have to pay for for on slack Which is really great the only thing that you need to pay for slack for is to have history that's recorded indefinitely Yeah, again, so you just need to grab the key which they generate there You need to put in the the menu hook URL there the menu callback URL there and What word you want your bot to trigger on so when you type in the slack drush bot and You pass it in the command that I'll show you how to set up later Then Drupal will respond and send in whatever you want it to. There's the key Super exciting Could set you a little icon there. You can you know upload whatever you want give it a cool name So I create a an admin panel to set the API key inside Drupal here is Slack receive like I said just create a menu callback there is user access control on it I'm actually changing this right now so that it will The the version that I have up right now doesn't tell you what the status code is so if it's a success It's not it's it's always a 200 even if sites offline. It's always 200. I'm changing that so that it'll actually give you meaningful status codes, but Whoops Sorry guys right, so here is an example of basically I create a web create a custom hook that you can implement in your own module and Then I also create an example module that shows some very simple implementations of slack receive It just gives two examples where you can you can list what modules you have currently installed and you can pass in a string And it'll it'll then grab the the modules list command from Drush and then return those modules and you can also clear cache Note This is important. So when you get down into this get PM. So this is the Sample command where it will list out the current installed modules with drush so notice the use of Escape the shell command and escape shell args. Basically that is taking the input that's coming over from the slack String command and escaping it so that you couldn't do things like you know RM-RF and completely delete your directory and other security problems So when you're implementing slack receive, I highly recommend using those commands to make sure that you're sanitizing user input Same deal with clearing cache Just escape the input that you're getting back Hey So now I'll show an example of actually sending the command. So we're going to do a cache clear for views So drush bot says yep views cache is cleared Then we'll do a PM list for the Twitter module And it tells you what version of the Twitter modules installed whether it's enabled etc, right? So just I mean these are just you know examples that show what's possible with it Beer bot is an actual a bot that you can download now It's not powered by Drupal But this is a good example of the kind of thing that you could do right you could set up a bot and pass in a string for the name Of a beer or a movie or something right and it would then return in this case beers You know and what the alcohol rating is and and some some user rating data, right? So you could you could do some pretty cool stuff with this right like you could have You know like a giffy type implementation where you have a bunch of you know gifts that are stored in Drupal And if you send in a certain key it'll then fetch that gif and display in your slack channel, or I don't you could do all kinds of fun stuff Okay, so We mentioned before now. We've got the tools with slack to get information back and forth to Drupal What about everybody who's working on this project? How do we keep them all on track and on point? So one of the key things in any agile workflow are the ceremonies and the most probably most important One of those is the daily scrum call. So first question here who uses a daily scrum call and Who here has a daily scrum call that is more than 10 minutes? More than 15 minutes More than 20 minutes you get my point. So scrum calls are meant to be quick updates Primarily focused on the issues that need resolution questions that need answering blockers that need unblocking and If you have a lot of developer teams on that phone call It will take a long time before you find out that Important information you do want to know what everybody's working on you want to know what they're planning to work on But you really want to know What do you really really want you really want? Sorry Second time today, he's made that joke It's this lobster claw in front of me. I'm sorry It's important to to be able to get that important information out So in order to get that phone call down to a reasonable amount of time and still be able to convey all the information That needs to be conveyed We use a slack bot called howdy Howdy is online at howdy.io And it's a plug-in another integration that you can connect with slack It's really really simple the original premise of the bot believe it or not Same kind of fun. We were having earlier was to collect lunch orders. What does everybody want to eat for lunch? Right, so they wrote a little tool in slack It pulls everyone Asynchronously and they realized this would be a great use case for a scrum call So what you do is are and you basically gather all the information from all your developers You can set up whatever kind of custom questions you want you can do traditional What was I working on what am I working on today? Do I have any blockers? Do I have any questions that need answering? And you specify that the duration that they have to answer But all that information is collected and then posted in a channel in slack So then if you do this, right, let's say your scrum call is at 10 a.m Set this to run at 9 30 you give everybody half an hour to respond Before you actually go into the scrum call it'll post the results in the slack channel Everybody sees what everybody's working on. There's no issues of communication. Hey that you know Skype call dropped out I didn't hear what you said. It's text-based. It's nice and short and sweet and Doing this we've cut the time on the scrum calls to half or even a third of that original 20 minutes All right, so just to show how this works so from with within Slack once you've had the howdy bot Set up right the same way you go into did I pause it? Yeah, the same way you go into the integrations Panel in your slack if you go to howdy once you're logged in to slack it'll just say allow to be allow Howdy to connect to your slack team Once it connects you just go in and say hey, I want to train howdy to run my scrum You give it a name you give it the questions you want it to ask so what I put in here What did I work on yesterday? What am I working on today? It's going a little fast let me back this up So What did I work on yesterday? What am I working on today? Do you have any blockers and that's it and you can tell it how long to wait 15 minutes an hour eight hours actually used to be eight hours and they upped it to 72 because some people don't do Daily check-ins they may do it every two or three days I don't want to wait for three days, but you get the idea so you're allowed then What will happen is howdy will come back and say Would you like to test it you can say yes or no it'll basically test it on you I know this will work. So I don't need to do it, but what it does tell you then is How to fire it right you tell it to point that command to somebody specifically or to somebody in a channel And then you can schedule it Do I want it to run automatically you could put in here to say run it at 8 a.m. Or whatever time you want So now whoops I went too fast. I'm quick on the trigger See if I had those lobster claw mitts on I wouldn't have this problem All right, so As soon as Okay How I get that thing disappeared as it go away, okay So now once I want this to run I just say howdy runs from general and you'll see up at the top the notification kicks in and it's polling everybody in the channel Hey, do you have time to talk? Yeah, okay. What did you work on yesterday? I worked on my triple cone presentation. What am I working on today? How do I get that to go away? And do you have any blockers and of course I say, yeah, I'm running out of beer Um, and then it says okay done And then you'll see a notification back in the general channel in the meantime Michael's answered his questions And it posts it all in the main chat. It also logs it online So all those notes are recorded And archived so you can go back to the scrum call from three months ago and find out what everybody was working on So if you need to have documentation Of traceability of what everybody was working on on some date now you have it automatically cataloged all right now There are some catches, right? There's always a there's always a hook or claw. Sorry. All right, so You can use Look at me like that You can use the stock questions, which is basically what I typed in or you can make them very very specific So do you have an issue for the legal team to review? Do you have items that the design team needs to review? Whatever those are you can set those up You're able to specify the timing. So maybe you have developer teams Let's say in israel that are not working on friday you turn off friday and put on sunday If you want it to fire it at a certain time you can go online and set it there Or you can do it all from the command line and slack You can split it up. You can give one team one set of questions and another team another set of questions Have them both post into it the same channel You know, that's kind of nice if you have segmented teams The most important factor here is that of all the other bots that we looked at This one allows for asynchronous reporting the other ones that we've run and tested They're basically with a for loop and if one developer doesn't answer The rest of the loop doesn't run. So you're stuck because developer number three of 12 hasn't answered We never find out about developers four five and six and so on In in howdy at the end of the time slot, you'll get a list that says You know developer a b and c did not respond So that automatically gives you a list to say hey, I need to go talk to those devs and find out what they're working on Why are you not responding? Are you dead? What's going on? You know, all right So a couple of things if you are on a large scale project where you're using enterprise slack Remember Michael mentioned it's free But if you want to keep a history you're going to pay for the enterprise model In that model you have permission levels on the users assigned in order to view this information on the web They have to have full access. So a restricted user cannot see this So if you have a partner scrum master that's on with restricted permissions They need to be promoted to full access. So that might be an issue for some clients And this I have to update this they just changed this it used to be only an eight hour update They've updated it like I said to 72 hours. So if your scrum calls are every third day that will work And that's it there was this was an issue and they've resolved that Some developers would get stuck answering a question. There was some latency issue They seem to have fixed it. So not a blocker All right so So because you don't need to take pictures we're going to post all this all the links to everything that we talked about here Are right here. So anything you want to look into further It's all here. We'll post the pdf on the On the session page. It's already there So thank you very much We have time for questions if anybody wants to ask questions sending messaging No, um, they're Actually, I know there's a couple of slack plugins that I haven't touched And I would be really interested to see if that works I would just be concerned about security right just making sure that somehow You're going to be able to correlate the user that's requesting That's making the request in slack with a user that actually has access in jankers And I could I could see that being kind of tricky But um, yeah, I wouldn't be surprised if somebody solved it So on the project that I'm on right now We have uh, in addition to using slack we're also We have the github plugin for jankins running So when a developer makes a pull request against the current active developer development branch when that pull request is merged Jenkins will automatically deploy it out to our dev qa and testing environments in aquea cloud And then that process all gets notified inside slack, which is nice, right? Very rarely do we have integration problems, but if we did we would know about it because of what's going into slack So that's that's a nice thing I think we started testing it. Yeah, we started testing it in october november And I think we put it into full gear probably about the Around christmas so since january Oh, no, so that's the thing the idea is that once you get that summary posted into slack Uh, here we go once you have that summary list posted that we set it up to be timed so that this is posted Five ten fifteen minutes before the actual call So then what we end up focusing on on the call are the real issues the blockers and we Phrase those in the question So first thing to do is to look through What I would do is look through the blockers section first Identify those as my priorities that I need to get resolved Once we go through those or other if there isn't any then you can now go to the next tier Which is what are we working on? What are we planning on working on? Is there any potential blockers? So I think it depends on the freezing of the question But if you time it that way then you have this posted into the main chat And then on the phone call you focus on what those questions or issues are If the summary is sufficient enough to say yep, I'm working on ticket, you know, one two three I don't have any blockers You don't need to talk right you actually don't even need to be on the call The possibility is I mean we still don't want to have everybody on the call because there's a possibility There may be something else that depends on that person You know, it's important to have participation But they don't have to say anything we can skip right over them If you're if your scrum calls are anything like ours, they used to be you know 15 minutes of people saying well yesterday I worked on project x and nobody cares and today I'm working on thing why and nobody still cares So just eliminating that section alone where it's literally just the only thing that we're actually spending time talking about Is problems that require collaboration and that is a game changer. I think yeah One of the thing, you know, the standard model is you know, what was I working on yesterday? What am I working on today? What am I planning on working out tomorrow? Do I have any blockers another one you can add is And after this was one thing we heard from a small boss section So it's a specific request to have a couple of minutes with a product owner or another developer Right on the call because we know everybody is on the call So you could put in another question to say do you need do you need a minute after the call? And since you're everybody's already there, this is a quick way to say yeah, I just have a quick question You know Give me a minute at the end of the call. So this way everybody can see what's going on Free It's open. It's a public beta So I suspect the typical model would be they would Give you what you're getting right now, which is a lot. I think it's a it's plenty of Plenty there to begin with but the premise was to have another enterprise model where maybe What they're keeping or the logging that they're keeping? Track of is going to be limited unless you pay a licensing fee in most projects You know, I really don't need to go back to a scrum call from three weeks ago. It's done I need the last week. That's it I can imagine that they would add some more filtering or other functionality as some sort of premium model But as of right now, there isn't one it's free So Uh, I don't know yet. I can I'll find out that's a good question Yep, they're really really responsive. Actually, one of the guys from there was supposed to be at triple con. I Just realized I'll show that as them Right Yeah, you were today not ours. Yeah So, yeah, well, but that that was the idea you could you can just you could run the same chat But in segments of teams you can manually go in and assign, you know Developer a b and c get this scrum set of questions But they're going to get it at 9 a.m. And then the us team will get it at 12 p.m. Or, you know, whatever the corresponding times are That's actually a little trick. Um, I recommend making sure the time zone settings in your slack are correct because You can change it and sometimes if you're working on a client, let's say who's in New york, but you're based in la you're thinking a lay time zone, of course Some just double check We've had it where it runs at three in the morning. Nobody's getting like the scrum questions But, you know, the idea is that you could run it in in concurrently in the same channel So, you know, these devs get the question in their time zone Typically for india teams, we would try to do it at the end of the day So that it'll be the morning of when we're doing the scrum call in the us So it is kind of scrum of scrums ish But not really But it's it's a good way of doing it Yep, so actually the beer by here, this is my hu bot and it's I'm hosting it So it's not um, we just modified but it's a real basic, you know, api call and you can customize it to no end Right, but yeah same principle Thanks Okay, so um, I'd answer that maybe on two levels because one is the fact that we're On slack to begin with and I think that by itself Having everybody on the common team and the flexibility for setting up Custom rooms or channels specific to the issues that the developers are working on makes it very easy Prior to slack. We were on Skype and it's it's a bit clunkier basically So that's one. I think just having that as a very easy Means of communication across the team by itself is already Let's say an order of magnitude more efficient or easier to communicate But with that there's a catch And this was always a challenge And it's not really specific to how many let's say but it's just a matter of how the projects are set up in general One of the things I always advocate for is making sure there's clear concise communication One of the other Pillars that go alongside it is documentation You can have all the communication in the world But if nobody's like logging that or concisely editing and posting that information in a place that everybody can get to It doesn't do us any good It'll spend an hour searching through slack for the information you're looking for so as long as you're Following good documentation Strategies making sure you know everything in your workflow is outlined making sure everyone understands their roles and responsibilities Then that dovetails with using something like slack or howdy So why i'm saying this in howdy we have this scrum report that's being generated We would still be logging. Let's say in a google doc or a confluence page those daily meeting notes Just to have it because there are other people that are not in that slack channel But want to see what the progress is on their ticket The next thing I think would be to find a way to hook this back into jira Which we're using for ticket tracking which you can do and we haven't gone to yet So any kind of specific reporting or questioning on a ticket Is it automatically associated with the tickets comments in jira? This is the big question right because you'll have a scrum call. Hey, i'm stuck on ticket 123 Uh, do you have time to talk? Well, I can't talk on the phone right now, but can you just ping me in slack in an hour? I'll be free. Okay. Now you have a conversation in slack with the product owner about the ticket It may be in the general channel, but you know an hour later. It's back scroll You don't know that it's there No, you don't remember that it's there getting that morsel of information back into the jira ticket That's the key thing So that would be the caveat. That was a really long answer. I'm sorry That's is that does that help? Didn't we have a number like how much we reduced the amount of time like 60 percent or something? Yeah, yeah This is a running joke with us And it's not it's not just Uh, quantitative it's qualitative The fact that you're spending half an hour on a droning scrum call Is not the way to spend your morning, right? So everyone comes Yes scrum rage exactly So it's like god damn it, you know and the first time we started doing this Let me tell you there was like that was the rage on the slacks for us like oh my god Scrum was less than 10 minutes today. You know, that's a mile. That's a huge milestone So that like emotional burden the fact that scrum shouldn't be a burden It should be a check-in and keeping it objective and not an emotional ceremony that that's something I don't think you can put a number on Okay, thanks everybody It's my menu hook