 a post and he will be talking about how he introduced GitLab in his DevSecOps journey. So, Tin, please take it away and share your presentation with us. Thank you, Isaac. Good morning, everyone. I'm Nitin Sharma. I work for Australia Post. Now, before we get started with the topic that I'm presenting today, I'm going to quickly touch on DevOps and then DevSecOps. So, DevOps is rapid code delivery for me. Yes, I'm going to repeat that again. It's rapid code delivery for me. Now, before you throw the DevOps handbook at me, I will acknowledge the cultural automation, lean measurement and sharing aspects of it. So, yes, I acknowledge all of that. Now, DevSecOps, that would be rapid and secure code delivery for me. So, I'm going to emphasize that this basically means that it's about bringing security into all different stages of your software development workflow. You're trying to deploy your code or deliver your code as fast as you can, but while making sure that it's being done in a secure fashion. Before progressing any further, I would like to acknowledge the traditional custodians of the land on which I present this today. I pay my respects to their elders, past, present and emerging. I extend that respect to all Aboriginal and Torres Strait Islander peoples. This would be the brief agenda that I have put together. I am probably going to go into a little bit of detail. Apart from talking about Australia Post, I will be giving you a brief intro into our organizational structure. In terms of the problem, I will give you a bit more historical background. While discussing the journey, I'll be talking about different options we evaluated. There's going to be a few more slides apart from what I'm showing you here. Australia Post, now this is the organization I work for. I'm very proud to be working for them, and there are numerous reasons for that, but I'm just going to give you the facts, first of all. It's a 200-year-old, more than 200-year-old, 280 years specifically, I believe. It's a self-funded government business enterprise, which is quite a mouthful. What it basically means is that we are an enterprise or a company, but it's completely owned by the government. I want to mention that Australia Post does not receive anything from government and tax funding, which usually is in news to people who are not much familiar with Australia Post. In terms of how we serve the Australian government, 1.5 billion has been paid in the past decade to the Australian government, just in dividends. We have a workforce of more than 75,000 people. I'm sure that number fluctuates, and it would have actually gone beyond that. It's a combination of permanent and employees which are contracting with us. Now, most of you may know us from letters and parcels, but we also do have a massive digital footprint, and we provide a range of services for both our customers, so people who are paying and consumers, people who are consuming our services such as people just tracking their parcels and letters through our websites. Now, this is what our organizational structure looks like. I will mention a few things here. I'm part of Platform Engineering Tribe that sits under the Product Innovation and Engineering umbrella. There are around 300 people in pie comprising of engineering, delivery, and business cohorts. In Enterprise Forward, we have centralized technology functions. It's similar to what you may have seen across all the other big enterprise. We have got one quick correction here. The auxiliary services here should be, that's actually enablement services in our area, which comprises of central DNS team, account provisioning team, and our cloud services team, which looks after standardization and governance around cloud accounts. Then, of course, security like no enterprise can operate without a strong central security function, so we have that in place. We also have strategy and architecture. The other thing I'll quickly touch on, so apart from the software engineering tribes that you are seeing at the top, which are part of pie, there are a plethora of other software engineering teams, which are part of different business units. But for the sake of keeping it all simple, I have pulled them all under the enterprise bucket. Now, this would be interesting to especially the tech people who have joined us today. So, plethora of technology or tools that are like that shows you what kind of ecosystem we have been operating in, so that's essentially the glimpse of our world. I'm not going to go into too much detail about this. However, I will share a few facts with you. So, in our business unit, 90% of the developers use this ecosystem to build, test, and deploy their applications in the cloud. And by cloud, we are quite focused on utilizing AWS in our business unit at the moment, but that's also changing. The other number that I'll provide you is that last time when we checked, we were performing using this old ecosystem. We were performing an average of about 37 non-production and seven production deployments every business day. So, that just gives you some baseline. Now, in terms of keeping it all simple and focused on to CI CD, because that's where the whole introducing GitLab, the why behind it comes into play, I'm just going to quickly move on to the next slide. The one quick thing here, you might be familiar with all the different keywords which are here, or you can always Google for that. But if you're looking at InfraViewer and wondering what exactly is that, so that's an in-house web UI component. Think of Asgard or Spinnaker. So, yeah, we are in the process of decommissioning that. So, with that, I'm going to move on to the next slide. Now, I'm going to pause here on the slide so that you can have a good look at this. So, this is basically us defining our problem. We have been using Atlassian Suite. So, a combination of Bamboo, Bitbucket, Crowd for almost five or six years now. But we came across a bunch of issues or, I would say, areas where we needed a better solution. So, highlighting some of these points. So, one of the challenges we have is that currently with Elastic Bamboo, you can only run all of your agents in one account. This has got direct implications for our setup, like the accounting for the cost is actually harder when you have multiple teams, but you are running all the different agents in one account. And then there are also security constraints, which essentially you are now having to run agents, which will need cross-account access to your other accounts, which increases your blast radius. Then, Pipeline has code. Yes, Bamboo has Java specs, but there are much easier way to provide that feature. In terms of support, without being too harsh, I believe Atlassian can actually do better with this support. User interface and developer experience has been average. Bamboo has had the similar interface, similar developer workflow for quite some time. So, yes, that definitely needs some improvement. Stats were also another major thing for us that every time we have to collect something, like how many bills in a month have been built successfully across the whole platform or across a given team. Finding that information out of the box is still quite painful. Then the other point I'll mention is that cloud native support for self-hosted version of Bamboo is still suboptimal. You cannot utilize EKS. You can do that, which we are doing that. However, there's a fair amount of work that has to be put into that. Then, Fargate being utilized for running the agents is also another thing that is not possible at the moment. One of the most important things for us is scalability challenges. So, you are running Bamboo, and Bamboo has got a Bamboo Master. You can only scale that vertically, which can become very costly, and you also do not get any redundancy benefits. In our setup, we are running it on EKS, which does provide us for tolerance. However, the cost can be significant, to say the least. Moving on to the next point. It's harder to support compliant workloads. So, we have got assets which are PCI and ISM compliant, so payment card industry and information security manual. Now, if we were to use Bamboo to deploy these assets, then because of the way Bamboo is set up, or because of the model essentially that you have to comply with while running your Bamboo platform, you will essentially have to bring the whole she-bang under the compliance scope, and which was not what we wanted. So, we had to look for alternatives. Third point that I'll mention here, Azure AD integration and other integrations are quite important to us. So, that was also a reason we had to kind of look for options. Now, some of you may be looking at this and going, but Nitin, there are third-party plugins like Mini Orange, which can accommodate that feature. Yes, for people who have been running Bamboo for a while, plugins, they will probably know that plugins can be harder to manage, especially as you upgrade your Bamboo installations. In terms of the journey, so this is where I will touch on what we actually did about the problem that I have just mentioned. Before that, a bit more context. So, reiterating that our tooling was put together five, six years back, it did an amazing job for what it was conceived. However, there was no active investment which was made to re-evaluate the needs of our developers and also to look at what's out there in the market. During this period, our constraints changed, so we had more people deploying more applications. Like when our business unit was formed, it was formed with like 50 people or even less than that, the number of applications were less. The other very significant change that happened is that our strategy changed. We started talking about multi-account and the AWS worse if I may use that, and then we also started talking about multi-cloud. So, we are looking at other vendors, GCP and Azure being the other two. The other important aspect was that we wanted the same developer experience. We wanted a developer who is working on non-compliant workloads and compliant workloads to be able to deploy their applications in a similar fashion. So, we wanted to be able to support both types of workloads using a single platform. Now, in terms of confirming the cause, there are a few ways how we could have gone about it, and I will touch on what we did. Initially, we did a few internal exercises to evaluate if we can actually find the root cause, if we can confirm that, and if we can look at some alternatives. We did come up with some data points, but this is where we needed some assistance, or we needed a fresh pair of eyes, and hence we decided to bring in a mental group to assist us with the discovery work and to come up with a recommendation and a plan. Now, some of you may be listening to this and wondering if we do not, do we feel that we couldn't have done this ourselves? I believe we could have, but I have got a few wise that I would like to share with you. So, when you are too close to the problem, you may make assumptions for what your customers require rather than seeing in detail as to what they require through an interview process. Then also, if you interview your customers, your customers, because they know you and they may be biased for the lack of a better word, you may not get all the data which a third party may receive. And last but not the least, if your stakeholders or customers, they may not necessarily view your recommendation as an unbiased recommendation, which will affect their adoption. So, if you have taken, if you've done all the right things for some reason, and you have taken them on the journey, and they're still, because you're sitting so close to the problem. So, for all these reasons and funding reasons as well, we decided to augment our capacity by bringing mental group in. The one other important point I will share here is that trying to get the funding, especially if you work for an enterprise, trying to get the funding is an important part of the job. I know there can be fair amount of bureaucracy, but trying to get funding without a plan is not a plan. So, my strong suggestion for those working in an enterprise, business cases are important and collaborate with your finance people to better translate the tech language to what they will require so that they can actually get you the required funding. So, getting the bind is what I've basically talked about. I cannot emphasize enough on the communication component. As soon as you start the discovery phase, you have to consistently communicate with your stakeholders. You have to take them on the journey. So, basically, we ran work group meetings, working group meetings. We ran leadership team updates. We wanted to ensure that people at all different levels actually understand and they feel that they are part of this journey. We provided iteration updates to discuss the progress, risks, and dependencies with all of our stakeholders. So, people need to feel that they are on this journey with you. Then I will quickly touch on the options which is, so during the discovery phase, which mental group assisted us, there were 30 options which were evaluated. There was a massive matrix in which all the different features that were required, they were evaluated. And out of that, we had 40 or 14 shortlisted options. Out of that, four or five options were quite closely evaluated. So, you may be wondering what about the SAS offerings. So, yes, SAS offerings such as AWS code build, code pipeline, along with other options which are partially SAS, for example, build guide, they were all compared. We quite quickly consolidated our view of what we required to be successful in this space. So, some of those points I mentioned here, so we needed a unified view of the pipelines. So, those who are familiar with code build and other similar SAS offerings, they know that if you code build and code pipeline, they're pretty good, but then you have to, you don't get a single pain, single unified view of your pipelines. Then integration was a very important component for us. So, zero AD checkmarks, all of those things that we quite, we use on, they are mandatory at this point essentially for us. Then hosting the code base within Australian boundaries was also, it's a legislative requirement for us. High availability and scalable solution, that's also, that goes without saying, like when, especially when you are working with roughly 200 to 300 people, every single day during which your platform is inaccessible, you can simply calculate the cost by putting, let's say, 1000 bucks a day for 200 developers, like you're losing a 200 grand a day. Now, relevant security and compliance controls. So, GitLab, certainly the solution does have a better security and compliance model, which is more compatible with our needs. This is also another, comparing the self-hosted versus SAS offering. So, this is where we evaluated how much of an effort would go into running this platform ourselves, versus what we will be getting if we were to get SAS, or what we'll be missing out on. So, this is just a quick overview of that. Now, in terms of the solution, so that's our present essentially. We decided to settle on self-managed version of GitLab as our solution. We did put in enough rigor. We spoke to GitLab beforehand. We spoke to different vendors to kind of ensure that we're actually going to get the necessary support, not just while we are, while we will be bringing in the solution, but throughout the life cycle of that. Now, this migration, these four points that I have mentioned here, those are there to share with you what we are doing now with GitLab. So, we are in the process of the platform got set up. We had to go through a bunch of exercises to get the evaluation done, to get compliance assessment done. After that, now we are in the process of getting our developers to move their pipelines from bamboo, their pipelines as well as their code from bamboo and BitPocket to GitLab. Our security posture for sure has improved. So, things such as code signing, integration with checkmarks, and the fact that tenant runners now have a limited radius. So, if something was to go wrong, the radius is only limited to the account itself. The other good thing that came out of this is that now we initially went for we initially started this journey with the thought process, we're going to solve this problem for our business unit. However, it seems like this has, this is being adopted as the enterprise platform. So, more and more teams outside our business unit are coming on board GitLab to essentially utilize the features and that we are providing. The fourth component that we are focusing on is essentially education and support. So, there are a lot of internal presentations being done in which we are talking about, of course, a journey and also what we, what needs to be done to move people away from their existing CI CD solutions onto this one enterprise platform. The other two points that I'll quickly touch on. So, and some of you may be looking at this and going, Nitin, why is there not a high level diagram or an architectural diagram that you could have shared? Well, I'm preserving that for the next presentation, hopefully. Say, I wanted to keep it quite high level. So, this one, moving back to my second point. So, the setup that we have got is essentially running on AWS EKS. So, elastic humidity service which allows us high availability configuration. Mostly, I will explain by the mostly component later. So, we're also utilizing EKS based runners in AWS and we have also experimented with Fargate and EC2 runners. Some of you who may be looking at me mentioning EKS twice here may be wondering that, Nitin, if you're going to deploy EKS in so many different accounts, your cost base is going to go shoot through the roof. You are right. So, at the moment, we are evaluating the best ways of making sure that we are getting that high availability and we are getting the speed at which our developers expect our rails to run. But at the same time, we also have to come up with the balance strategy so that we are not paying AWS too much for the EKS cost. In terms of the future now, so, GitLab runners, we, as I mentioned that we are going with the multi-account approach. So, there is an overhead of deploying runners in new accounts and then maintaining them. We are working towards reducing that maintenance overhead and also the AWS resource cost. Multi-Cloud, our business unit is primarily focused on utilizing AWS. However, our stakeholders, our customers rather, from the enterprise, they have started using GCP as an example. So, we are looking at options to ensure that our developers do not have to make too many changes to their pipelines if they were to move between different cloud providers. So, this is where we are still having internal discussion about how to kind of see through that we come up with a solution for that problem. Now, I mentioned us running GitLab in high availability configuration before. Mostly, the only component which is not being run in high availability configuration is GitLab, Gitly. So, it's basically, we are running it in a fault tolerant configuration, but this is one area which we would like to improve so that our whole setup is actually, we can say that it's actually being run in a high availability configuration. The other important aspect that we will be assessing, we are already discussing that, but we need to come up with a plan. We want to have CI CD for CI CD. Now, essentially what we want to do is, if we want to maintain our GitLab server, we want to patch or maintain or upgrade or even orchestrate from ground zero. Then in that case, we want to come up with a combination of, for example, and the solution could be very different, but a combination of some other service, it could be a combination of AWS code build, code commit, and code pipeline, which then can actually help us set up our GitLab platform. I'm going to speed things up because I'm aware that I'm just two minutes left. So, takeaways from all of this. So, what I would suggest is that you, engineers are passionate about solving problems. So, you're going to figure out ways of supporting them. You have to listen to where they, when they are telling you that this thing is not working or things are painful. And at the same token, you have to provide a top-down support. So, in our case, fortunately, that all happened. We had really passionate engineers amongst ourselves, and we also had the right leadership support, which actually got us this outcome. The second point I'll quickly touch on is perfection is equal to procrastination. Don't try to come up with the best possible solution. Think about running a tight scope, especially when you're doing the discovery work. Know your requirements, then reiterate based on the constraints. Communicate with your stakeholders. Having a strategy in place is going to help, but what I will suggest is that if you just make an assumption that they will come, that might not always fly. Context constraints and the right tool. Know your context. Know what kind of budget you're working with, what your legislative requirements are. Your requirements may be very different from us, and you may actually be better off using a SaaS service, just to be mindful of that. Also, think about the right tool for the job. If you do not have the same developer footprint as us, as us, then you need to evaluate your requirements from that perspective. Fourth is change management and support. This basically is about sustainable engineering. Take people on the journey instead of trying to rock the boat, if that makes sense. Ask for more. Listen to the requirements of your stakeholders. Collaborate with your window. Ask them for the features. We have got an ongoing conversation with GitLab about the features or bug fixes that we need. A lot of that information we have publicly made accessible, but there are a lot of private information that we would discuss on a reoccurring basis with our account manager and solutions architect. Last but not the least, I wanted to say quick thank you to all the teams that were involved at Australia Post, especially the GitLab project team. This would not have been possible without their dedication and effort. And also a shout out to Meta Group and GitLab, and of course, Amazon Web Services. With that, I have run out of time, so I'm going to give this back to Isaac.