 We're officially starting. Thank you for attending this session in the Cloud Foundry, in the government track. This is a panel discussion focused on saving millions and delivering in minutes. And our esteemed panelists here to help you kind of follow along who's talking, I went ahead and put all the pictures that I could find on the internet. And I couldn't find one for John so I'm gonna let you apply some deductive logic to figure out which one is John. Actually though the first question that I have will be to each of them to just briefly introduce yourself and tell us a bit about your role and your involvement with Cloud Foundry at the National Geospatial Intelligence Agency. We'll start with you, Eric. So my name is Eric Levine. I'm the team lead for the operations team that supports NGA's deployment of Pivotal Cloud Foundry. I started this process in August 2016 working as a member with some other people that rotated off the project. So we had Pivotal come in and for eight months they helped us prototype pretty a different type of environment that would fit the government. We needed to be able to put antivirus, a bunch of things that ops manager wouldn't do. So we went with a Pivotal bits and an open source might like design using Bosch straight and not using ops manager, which made a lot of challenges for us but we're able to overcome and actually I think in the end to get a better product out there. My name is Nitro Ditovichir and I am director of development at Grunch Data and my first experience with Pivotal was actually at the agency when a big agency program called Geo Services which was meant to modernize the entire agency brought us in and they basically had one requirement is to take our secure and compliant distribution of Postgres and natively integrate it with the Geo Services. Back then I had no idea what Pivotal was or what the platform was but within a year we started working side by side with Pivotal folks when we start understanding Bosch and we created the first native integration of a Postgres database into the platform and that journey led to actually partnership and now our service is available in Pivotal for other large enterprises. Hi my name is John Lee and I was going to say there's a reason why you can't find my picture but you know just it was a last minute change that happened there's long story but we'll go into that here. I've been with NGA for about 15 years started as on the development side delivered various capabilities image processing also did some visualization work and that'll come up a little bit later I think when we ask some of the questions but most recently I'm the platform integrator for NGA trying to help make the platform successful trying to understand customer requirements the kinds of things that they are looking for from a development perspective how we can make the platform better to suit their needs and excited to be here today. Hello my name is Dan Zach I also work on the platform operations team and my background is the Unix Systems Administration and I've spent the bulk of my career doing enterprise management systems and tools deploying them in government agencies scanning them and the credit bringing them online and such and then I joined the the platform engineering and ops team right like sort of in that second half of the push before we were like getting the systems live and going through the accreditation like that and so after pretty much like most of my career seeing the struggles and the pains of deploying software until like these like government agencies and systems it's been refreshing to see this the work that we've done with Bosch and Cloud Foundry like start to make that process less painful and quicker so as you can see this this is panel far more focused on the platform operation side compared to our last talk which the active side so Eric you kind of touched on this a little bit well you guys have done a slightly different implementation of Cloud Foundry leaving out the ops manager you know tell us about which pieces you're using you know how did you choose that approach what are some of the other elements that you know have been important and how you've set up the platform specifically connected and it's an AVS environment that's disconnected from the world on what we call the high side and sorry AWS that environment ops manager can't function on so we wanted to keep everything the same and then we had some security requirements that forced us to sort of not you know use the constraints that ops manager gives and be more open and then we found like going down that path it took a lot a little bit longer and we had to develop you know lots of like our own sort of way of doing things with spruce to take a ops manager derived manifest and transform it into what we can deploy in our environment which requires certain like certain requirements that just are no it's a no go without but they're not they just make things a little more complicated so that's why we sort of chose that and then now we're able to have really consistent environments that are functioned the same way and we're starting to integrate credit hub and concourse to automate those environments and make them really consistent and really cut down the differences which is what I think is really powerful about boundary I've been in the government for a long time doing infrastructure and getting multiple data centers multiple networks deployments exactly the same has been a challenge until today basically is what I would say okay so so maybe you can just expand on a little bit more in terms of what does it mean to now have it be consistent what does that translate to in terms of operability environments that people would write an app for and then they're like okay we're ready we're gonna go into production they drop into production and nothing's the same nothing works right you'll have two data centers you know failover data center and primary data center and you fail an app over and nothing would work right like I worked at army knowledge online early on and that was a huge application being done in a smart like a semi smart way but whenever they failed over nothing worked and if they had if that had been on a platform like cloud foundry they would have been able to seamlessly fail back over and forth and everything would have been the same and consistent to today with cloud foundry you're now able to failover and move from dev into production deployment is developed that we have a prototype environment that people can develop bosh releases they have full rights to bosh they have our exact copy of what we run in production they developed that he hands over the release to me and then I deploy it all over the place and that kind of success works with the apps and the bosh releases that we have we have multiple bosh releases that we use that we develop in house or with partners um so how has cloud foundry changed the security practices at at nga we heard a little bit earlier from the air force in terms of the controls be great to hear about what does it mean at nga on the okay on the platform side i touched on this a little bit because when eric was talking he was talking about how we made decisions the whole time we were like laying out how we're going to deploy the platform we were always thinking about security controls right because that's if you were in the last talk the actual process of getting things stood up and deployed is very it's very lengthy right and then there's a set of security controls that feed into the risk process and so like you basically have to say how all the all the apps like the methods in which they you know meet the various controls and that's how you can gauge the risk of the overall system so with now the way we're doing things now um we're it's always focused on inheritance right so we designed the the platform in a manner that it's well documented and we've we've we sat with our risk teams and went through like how a bosh deployed service will meet these controls right and and we've demonstrated that every time you deploy it with this source controlled manifest you get you know the same outcome and getting that like training and working through the buy-in like of those teams now when we go through the accreditation process it's a known entity if we've documented which controls that the if it's a service the service receives from the bosh deployment and the logging and everything associated with that on the app side there's a lot more because you're higher up the stack right so um there's a lot more controls that they inherit and by I know that they talked about this in the in the last presentation if you're here you can see it dramatically reduces that time because before you'd have to bring to bring a service online or new new system you would have to go through all that testing and it would take like sitting down and manually going through all this would you know it takes months plus so now the the way we do things with bosh and cloud foundry those times are reducing every iteration right and we have these really lofty goals of getting almost on the app side almost like automatic based on pipelines and tools and on the services side you know it's days in some cases because you're it's just a small subset of controls and I know the crunchy data was one of the first services through and I can either can speak to that a little bit more right so obviously the end goal for any service at the place that we're working is complete not just security but a complete compliance right we can't just think of hey I just do this this and that we'll also have to understand how all of this is mapped in the world of compliance the what platform enabled us to do is to scale that compliance security and compliance practices to the entire enterprise so what he was talking about is we were able to get our ATR authority to operate at the agency in about three days after we demonstrated a complete automation of all the controls so we extensively there was one of the reasons why crunchy was brought in in the first place we extensively work with this to create a stick specific for the postgres distribution but then we converted that stick into a set of automated test cases that were integrated with Bosch so anytime all throughout the agency any user who creates an instance of a postgres database there's an automatically built-in compliance job running and constantly checking things for compliance and if you well a you can't misconfigure the database yourself but god forbid there's an intrusion and database gets misconfigured we're going to pick up that misconfiguration a matter of minutes and notify appropriate personnel and I would say the one thing that was critical of the success was actually taking the time and changing the working to change the mindset of those risk people and test assessors to like actually make sure they understand this new way of doing this is a new it's a completely different mindset than what they were in and then work through the automation and they eventually get on board because as Eric was talking about you always heard that you know like we use all these automation to deploy software and it's the same but in practice it never was really close but now like with these we've demonstrated them on many occasions it's easy demonstrable to see like yes it's exactly the same if you deploy it in this fashion so then they get on board because now they can actually like you get reciprocity on like our disconnected environments because we can prove they're built the same way and then your your savings start to pile up because if you think about the you know just the scope of the the foundations that we have deployed I mean every one we bring out now is much quicker okay so you know can you talk about some of the you know how apps were brought apps or different services were kind of brought online for developers before cloud foundry and you know then just sort of maybe walk us through an example of how that looks today uh for for a developer to access a you know a new service so I can probably take a crack at that so back when I was doing some development work you know the development process was was pretty good because we were doing that back in our factory but then we'd always talk about how long ago it would take to actually get it into production because then you got to take it through the testing process the security process and whatnot and sometimes that took months of time and then other times when we're actually trying to get additional hardware provision on our operating servers again months at a time you got to put a ticket in you got to wait for people to do this and that funding chase it all around blah blah blah that kind of thing now you go on to the platform and I can spin things up in seconds literally and it's a much faster process and in particular also when I'm trying to deploy things into production now I don't have to go through these lengthy process where the guy's talking about this before about how we've been working with security to reduce the timeline that it takes to get an accreditation right that's key if you just bring a piece of technology into the agency that by itself is not enough you have to change your business processes to suit the technology as well because what we found we brought Cloud Foundry in it allowed developers to go really fast but our business process was still slow so we're having to work to kind of help the business process catch up with how fast the tools that you go and so we're seeing people being able to deploy capabilities much faster now sometimes down into you know weeks days you know much much more rapidly than before and which is really revolutionary at our agency what about maybe just also another example from a database service dimitri you've been working as you said on bringing the first natively deployed postgres to cloud foundry in the environment you know what would that have looked like for folks to to provision a database before so when I work at other agencies or even private companies it always starts with the ticket you send the ticket to a dba then you wait a week for him to acknowledge that he received the ticket then it would months and months go on and they provision something and then you get a bunch of emails saying hey here's your user and in another day I'll send you another email with the password and sometimes it took months and months and months right now you can go in the agency you can go to your services marketplace and literally provision a secure compliant database in a matter of minutes from but but that they're just part of the game you know people think oh we we're running a dev up speed and agile speed at the agency but it means nothing if you can deploy your application or get the application in a matter of months and then wait six months for your at or security approval um with us creating a common controls now your job is much more as a developer is much much easier because you pretty much don't have to do anything you can claim the inheritance that database brings to the table almost immediately in in your documentation and interestingly enough just before we started this talk there's a person in the audience whose application is going through accreditation process at the agency and he told me how great it is to be able to inherit all the controls from from a postgres database and also on the app on another thing that was a savings on the app side we use like the single sign on service so there's another huge like there's an onboarding process to deal with because there's enterprise authentication requirements right so and then they're associated you know security checks so like now like in the same way they provision crunch eating or they can bind with the single sign on and like go like instantly have that function working versus like having you know to spend all this time doing that and like we were when we were rolling out the system we looked at the pain points of the developers and that was a huge one right having to go do all these off requirements and then they get all those associated controls that they've you know bind with that service and then logging would be like another big one big win for them because that's a that was a tedious process for them as well and there's a huge like bureaucracy to deal with you know the what you log and the requirements for what you log and such so we can we offer that too right that's that's pretty much just built into the way the platform does log aggregation and as long as the developers write to the you know the right places that I think and to add to what you're saying another big win was consistency of encryption algorithms because that's the encrypting data at rest is one of the biggest selling point of standardizing this type of approach and Bosch actually abstracted all the encryption at rest and all we have to do is tie into those apis and now every database that you provision encrypting data the same way in the fifth compliant matter all across the agency yeah we spoiled the developer so much that I see comments like that I deployed crunching it took 20 minutes is something wrong to kind of fall through a little bit I was talking to developer last week and she said little things that she said I can't imagine how it would be like would be like without developing on the platform she just can't you know imagine a situation like that so her eyes would bleed right now exactly I know it's like life before snapchat how did we get by I don't know so you know John you mentioned how the the business processes you know actually had been lagging what the platform has been able to deliver if from some of our kind of prior discussions how has the collaboration and sharing between groups changed and what's been kind of driving that and are there any practices or tactics that have made that possible that you know you could share with folks that might be applicable in their agencies sure actually it's actually on his shirt right here G1 services great to advertising G1 services is an initiative at NGA that we're trying to kind of do things a little bit differently try to embrace what's good about where we're at technology now as opposed to the old practice so we're trying to embrace open source we're trying to embrace you know DevOps and agile and all these kind of things and not just in the sense of buzzwords but I really like what the previous presenter said about a learning organization right you've got to continue to learn and you just can't you can't say stagnant say well I've got to figure it out I've got to continue to learn like Kubernetes is coming up we've got to figure that out too so things like that but there's also a push there so I'm assuming most of you are familiar with how the government works there's a lot of different contracts a lot of different fiefdoms and agendas if you will and we're really working hard in G1 services to try to work across that trying to make sort of a bad just not your government your contract you know to get away from that we try to really work collectively together as a team you're on this contract you're on that contract doesn't matter that's all try to work together to achieve the same goal so we're trying to make it a focus I won't say we're perfect we certainly have areas to learn and grow in but we're trying to really work together and we see a lot of good collaboration that's occurred to try to help us pull these things together and again you know part of the whole doing things with a platform that's new to a lot of folks it's different they have to kind of think about the world a little bit differently and say hey maybe I can really push off all this lower level stuff this undifferentiated heavy lifting and just focus on the business value that I bring at the top level the stack let somebody else take care of the other stuff and so we're starting to see that being embraced and the people come coming along with us for the ride maybe you could tell us a little bit more about the the gio and gio int gio and services services initiative you know is that something we're kind of creating an initiative like that is you know a way to kind of then have those conversations and prompt change it is a way basically with gio and services the way it's working is we're trying to show that it can be done differently even in a large government agency like ours that you can do to get a different approach and think a little bit outside the box and so you kind of build a coalition of the willing initially they need to have some champions to help you take that message across the organization and really get more buy-in and you know you have you have to celebrate the wins and show that yes I can go fast and sometimes you go so fast people can't believe like how could you possibly go that fast and get things out the door that quickly but you really can and so you know just you got to continue to work with folks and sometimes they don't get at the first time you got to go back and talk to them again and just keep trying to get that message out so in terms of building that initial coalition of the willing as you described it you know how did that come together initially well I think there were some key folks that kind of had a vision uh Ben Tuttle who's not here I think a lot of that was you know his kind of brainchild but there was a set of folks that initially wanted to really try to do things differently they see how fast things can happen outside the government embracing those technologies they wanted to bring that in and try to make really transform the government from the inside out and so you just have some visionaries that really wanted to change the way we do business and make us so we don't become relevant we stay relevant we're able to deliver value and they just get focusing on that and it's it's been challenges along the way and there's been some naysayers and people that like they like the old status quo but we can't we can't afford to keep the status quo the way it is we need to reinvent ourselves and get better so that's what we're trying to do I would say also just from a like a technical perspective on top of that like some of these automations and the way things work and like I was saying you can get around some of that like multi-contractor dependency bureaucracy by once like it's the systems are in place it may have taken a little bit of time to get it there but there's it's very hard there's no like way to open it take right like all those manual processes that usually slow things down now it's done just by automated provisioning right so it let like it helps kind of force a little bit of that mindset change where people can like try to slow things down and like make the you know nine different contracts to do one service you know like oh you could it just smooths that out in a like a less painful way in a way that's harder to break once it's all in place I guess okay I've got a few more questions I wanted to also just check to see if there were any questions from the audience I'm happy to kind of run out there put a microphone up okay one sec I was just curious if you can talk through some of the pain points again as a vendor on these projects I mean seems like a lot of the projects come with the architecture that's based on something from last century or last decade and try to adapt it to this platform model and then a lot of vendors try to build things 12 factor and you know from the get-go and I just wonder what you know what you guys experience has been trying to adapt these older you know older type systems onto this platform model and what you know I'm just curious what you have to say since I get in the room we tend to do more of the operations and we just only hear about what they're doing but um just getting the tools out to them and getting showing them how to that they can really reduce their amount of lines of code by using services like single sign on and stuff like that that and they start to see that they could deploy things fast go through accreditation faster and so they have the you know the incentive to to do some refactoring to make sure that it runs on the platform properly and I can I can add a little bit these pains of learning the new ways of deploying and new ways of creating applications are not just applicable to the old you know monolith applications the folks who are doing even the modern microservices still of their new to cloud foundry uh there there's some steep learning curve how to redesign your app to be able to run it on the platform so like in my experience there was this app I mean it was written recently called teggala and the the approach that says hey we have this great code but how do actually deploy it to cloud foundry so our directive was uh you you're not here to work on tests you here to empower other people so I spend numerous weekends working with those guys to actually show them how manifestory written show them how to transform their code into something that you can do cf push basically do things that normally dbh don't do you know that if I can add one thing as well I mean our agency is really wrestling with the fact that we've got a lot of legacy systems that were not built under 12 factor principles and you know we had started down this path of trying to migrate to the cloud where we're going to do a lift and shift approach and that hasn't worked out so well for us and so we're really trying to encourage folks to embrace doing things in cloud native way but there could be a significant cost of refactoring replatforming doing all that kind of work to get it into that so we're still wrestling with that when they still have a good answer but you know in some cases the the simpler kinds of things we're able to move those over and we're teaching people how to make those changes but sometimes they're just going to have in some cases we're just going to have to suck it up and do the work and and get it moved over like the guy that was at t-mobile is talking about some of the legacy applications that they've had to really work at to get onto the platform we're just going to have to make that investment and with the pks coming on now that gives us another opportunity maybe if we could go to a little bit lower level of abstraction maybe that work is not so hard to make that change how you doing my name is Scott I'm a recent Air Force retiree who's been working in the cyber security side for you know the last 15 years or so dealing with policy directives guidance you know from do d level have you guys ran into any type of issues along the lines of policy and how have you adapted to that those requirements in regards to making that change to where things are written in black and white in the government and we have to be able to be adaptive and and agile we've been we've been through our conversations and what a privileged user is and if there are privileged users on our system or not and you know so there it's just we just keep fighting the education and just keep you know trying to get them to understand that like these developers can do stuff but they can only do stuff in a really secure manner and they can't really change things right the level of change that they can inflict upon their own application is very small um and so that reduces a lot of the sorry that reduces a lot of the scope of the of the work that has to be done for accreditation and I mean it it's worth like the the persistence and also though like as you see they're pushing more like risk management more right so that lets you change the dialogue a little bit more from the black and white because really at the end of the day what are you doing you're trying to assess the actual risk right it's not about just you know meet the goals that have a secure system and so as that dialogue is changing and they're they're the conversations you're having with the security people it it becomes easier to make this point because doing it this way is much more secure than what they have now right so I think that's helped and that's probably been that's been valuable to some of our successes and getting that mindset changed in some of those entrant with some of those groups that have old entrenched views of how to do things so okay I think this will be our last question unless it's a one word answer um DevOps when we for production support after you've implemented implemented everything we were told oh the developers are going to now be the ones that monitor the system or take care of it if there's any problems with it but we didn't have any developers to do that because once they finish the code they move on to something else so I was trying to understand how you adopted your production support approach after you actually implemented your your your applications so maybe just so we're using DevOps in the sense that we're using CI CD pipelines to push so you can't actually do a cf push into production you actually have to use the Jenkins pipelines to push your code out but as part of g1 services what's being stood up is an operations group that is going to basically take care of and maintain those capabilities once they're pushed to production obviously they have to do that in partnership with the developers but we're trying a new construct where we actually set up a separate operations team to kind of take care of the the maintenance of those capabilities yeah and with the platform we've implemented tools like the open source log search release and Prometheus to like sort of check the platform make sure it's up and running and then that app usually stays up and running it's just the nature of the way Cloud Foundry creates that if it runs once it'll keep running and so there's a lot of reduced amount of of that monitoring that needs to happen okay thank you I want to thank our panelists I want to thank the folks in the audience who asked some questions great questions thank you