 From around the globe, it's theCUBE with digital coverage of AWS Public Sector Partner Awards brought to you by Amazon Web Services. Hi, and welcome back. I'm Stu Miniman and we're here at the AWS Public Sector. They're partner awards, really enjoying this. We get to talk to some of the diverse ecosystem as well as they've all brought on their customers some really phenomenal case studies. Happy to welcome to the program two first-time guests. First of all, we have Jared Bell. He's the chief engineer of self-response operational readiness at T-Rex Solutions and T-Rex is the award winner for the most customer-obsessed mission base in FedSiv. So Jared, congratulations to you and the T-Rex team and also joining him, his customer, Michael Thiem. He's the assistant director for the decennial census program systems and contracts for the US Census Bureau. Thank you so much both for joining us. Good to be here. All right, Jared, if we could start with you. As I said, you're an award winner sitting in the FedSiv space. You've brought us the Census Bureau which most people understand the importance of that government program. Coming up on that, every 10 year I've been hearing TV and radio ads talking about it. But Jared, if you could just give us a thumbnail of T-Rex and what you do in the AWS ecosystem. Yeah, so yeah, again, my name's Jared Bell and I work for T-Rex Solutions. T-Rex is a mid-tier IT federal contract and company in Southern Maryland. I recently graduated from Hubzone status. And so T-Rex really focuses on four key areas, infrastructure and cloud modernization, cybersecurity and active cyber defense, big data management analytics and then overall enterprise system integration. And so we've been AWS partner for quite some time now and with the decennial, we got to really exercise a lot of the bells and whistles that are out there and really put it all to the test. All right, well, Michael, you know, so many people in IT, we talk about the peaks and valleys that we have. Not too many companies and organizations say, well, we know exactly, you know, that 10 year spike of activity that we're gonna have. I know there's lots of work that goes on beyond that but it tells a little bit your role inside the Census Bureau and what's under your purview. Yes, the Census Bureau is actually does hundreds of surveys every year, but the decennial census is sort of our main flagship activity and I am the assistant director under our associate director for the IT and for the contracts for the decennial census. Wonderful, and if you could tell us a little bit that the project that you're working on that eventually pulled T-Rex in. Sure, this is the 2020 census and the challenge of the 2020 census is we've done the census since 1790 in the United States. It's a pillar, a foundation of our democracy and this was the most technologically advanced census we've ever done actually up until 2020. We have done our censuses mostly by paper and pencil and this is a census where we opened up the internet for people to respond from home. We can have people respond on the phone. People can respond with an iPhone or an Android device. We try to make it as easy as possible and as secure as possible for people to respond to the census where they were and we wanted to meet the respondent where they were. All right, so Jared, I'd love you to chime in here because I'm here and talking about the technology adoption. How much was already in plans there? Where did T-Rex intersect with the census activity? Since this deserves a lot of credit for their innovative approach with this technical integrator contract on which T-Rex was fortunate enough to win. When we came in, we were just ramping up the 2018 test. We really only had 18 months to go from start to a live operational test to prepare for 2020 and it was really exciting to be brought in on such a large mission critical project. I think this is one of the largest federal IT products in the cloud to date. And so when we came in, we had to really bring together a whole lot of solutions. I mean, the internet self-response, which is what we're here to talk about today, was one of the major components but we really had a lot of other activities that we had to engage in. We had to design and prepare an IT solution to support 260 field offices, 16,000 field staff, 400,000 mobile devices and users that were gonna go out and knock on doors for enumeration. So it was really a big effort that we were honored to be a part of. And on top of that, T-Rex really brought to the table a lot of its past experience with cybersecurity and active cyber defense. Because of the importance of all this data, we had to roll on security all throughout and I think T-Rex was prepared for that and then did a great job. And then, you know, overall, I think not necessarily directly to your question, but I think one of the things that we were able to do to make ourselves successful and to really engage with the Census Bureau and be effective with our stakeholders was that we really built a culture of decennial within the technical integrator. We had brown bags and working sessions to really teach the team the importance of the decennial, not just as a career move, but also as an important activity for our country. And so I think that that really helped the team internalize that mission and really drove kind of our dedication to the Census mission and really made us effective. And going, a lot of the T-Rex leadership had a lot of experience there from past decennials and so they really brought that mindset to the team. I think it really paid off. Michael, if you could bring us inside a little bit of the project. 18 months, obviously you have a specific deadline you need to hit for that. Help us understand kind of the architectural considerations that you had there, any concerns that you had. And I have to imagine that just the global activities, the impacts of COVID-19 has impacted some of the end stage, if you will, activities here in 2020. Absolutely. Yeah, the decennial census is I believe a very unique IT problem. We have essentially 10 months out of the decade that we have to scale up to gigantic and then scale back down to run the rest of the Census Bureau's activities. But our project, you know, every year ending in zero, April 1st is Census Day. Now, April 1st continued to be Census Day in 2020, but we also had COVID essentially taking over virtually everything in this country and in fact in the world. So the way that we set up to do the census with the cloud and with the IT approach and modernization that we took, actually, frankly, very luckily enabled us to kind of get through this whole thing. Now, we haven't had, Jared discussed a little bit the fact that we're here to talk about our internet self-response. We haven't had one second of downtime for our response. We've taken 77 million, I think even more than 78 million responses from households. We have out of the 140 million households in the United States, we've gotten 77 million people to respond on our internet site without one second of downtime, a good user experience, a good supportability. But the project has always been the same. It's just this time we're actually doing it with much more technology and hopefully the way that the cloud has supported us will prove to be really effective for the COVID-19 situation because we've had changes in our plans, difference in time frames. We are actually not even going into the field or we're just starting to go into the field these next few weeks where we would have almost been coming out of the field at this time. So that flexibility, that expandability, that elasticity that being in the cloud gives all of our IT capabilities was really valuable this time. Well, Jared, I'm wondering if you can comment on that. All of the things that Michael just said seem like just the spotlight pieces that I looked at cloud for, being able to scale on demand, being able to use what I need when I need it and then dial things down when I don't. And especially, I don't want to have to, I want to limit how much people actually need to get involved. So help us understand a little bit what AWS services underneath, we're supporting this and anything else around the cloud deployment. Sure, yeah, yeah, Michael's spot on. I mean, the cloud is tailor made for our operation and activity here. I think all told, we use over 30 of the AWS FedRAMP solutions in standing up our environment across all this 52 system of systems that we were working with. Just to name a few, I mean, internet self-responsal loan relied heavily on auto scaling groups, elastic load balancers. We relied a lot on Lambda functions, DynamoDB. You're one of the first adopters of the DynamoDB global tables which we use for session persistence across regions. And then on top of that, the data was all flowing down into RDS databases and then from there to the census data lake which was built on EMR and elastic search capabilities. And that's just to name a couple. I mean, we had, we ran the gamut of the AWS services to make all this work and they really helped us accelerate. And as Michael said, we stood this up expecting to be working together in a war room, washing everything hand in hand. And because of the way we were able to architect it in partnership with AWS, we all had to go out and stay at home. The infrastructure remained rock solid. We didn't have to worry about being hands on the equipment. And again, the ability to automate and integrate with those solutions, cloud formation and things like that really let us keep a small agile team of DevSecOps there to handle the deployments. And we were doing full scale deployments with one or two people in the middle of the night without any problems. So it really streamlined things for us and helped us keep a tight and agile for sure. Michael, I'm curious about what kind of training your team need to go through to take advantage of this solution. So from bringing it up to the ripple effect, as you said, you're only now starting to look at who would go into the field, who uses devices and the like. So help us understand really the human aspect of undergoing this technology. Sure. Now, the census always has to ramp up this sort of immediate workforce. We hire, we actually process over 3 million people, through I think 3.9 million people applied to work for the Census Bureau. And each decade, we have to come up with a training program and actually training sites all over the country and the IT to support those. Now, again, modernization for the 2020 census didn't only involve the things like our internet self-response, it also involves our training. We have all online training now. We used to have what we call verbatim training, where we had individual teachers all over the country in places like libraries, essentially reading text exactly the same way to exactly over and over again to the people that we train. But now it's all electronic. It allows us to, and this goes to the COVID situation as well, it allows us to bring only three people in at a time to do training, essentially get them started with our device that we have them use when they're knocking on doors and then go home and do the training and then come back to work with us all with a minimal contact, human contact sort of model. And even though we designed it differently, the way that we set the technology up this time allowed us to change that design very quickly, get people trained, not essentially stop the census. We essentially had to slow it down because we weren't sure exactly when it was gonna be safe to go knocking on door to door, but we were able to do the training and all of that worked and continues to work phenomenally. Wonderful. Jared, I wonder if you've got any lessons learned from working with the census group that might be applicable to kind of the broader customers out there. Sure, well, working with the census, it was really a great group to work with. I mean, one of the few groups I work with who have such a clear vision and understanding of what they want their final outcome to be, I think, again, for us, the internalization of the decennial mission rate, it's so big and so important. I think that because we adopted it early on and we felt that we were true partners with the census, we had a lot of credibility with our counterparts and I think that they understood that we were in it with them together and that was really important. I would also say that because we're talking about the GoCloud solutions that we worked, we also engage heavily with the AWS engineering group and in partnership with them. We relied on the infrastructure that management services they offer and was able to give us a lot of great insight into our architecture and our systems and monitoring to really make us feel like we were ready for the big show when the time came. So I think for me, another lesson learned there is that the cloud providers like AWS, they're not just a vendor, they're a partner and I think that going forward, we'll continue to engage with those partners early and often. Michael, the question I have for you is, what would you say to your peers? What lessons did you have learned and how much of what you've done for the census? Do you think it'll be applicable to all those other surveys that you do in between the big 10-year surveys? All right, I think we have actually set a good milestone for the rest of the Census Bureau. The modernization that the 2020 census has allowed since it is our flagship really is something that we hope we can continue through the decade and into the next census as a matter of fact. But I think one of the big lessons learned I wanted to talk about was we have always struggled with disaster recovery and one of the things that having the cloud and our partners in the cloud has helped us do is essentially take advantage of the resilience of the cloud. So there are data centers all over the country. If we had ever had a downtime somewhere, we knew that we were gonna be able to stay up. For the decennial census, we've never had the budget to pay for persistent disaster recovery and the cloud essentially gives us that kind of capability. Jared talked a lot about security. I think we have taken our security posture to a whole different level, something that allowed us to essentially, as I said before, keep our internet self-response free of hacks and breaches through this whole process and through a much longer process than we even intended to keep it open. So there's a lot here that I think we wanna bring into the next decade, a lot that we wanna continue and we want the census to essentially stay as modern as it has become for 2020. Well, I will tell you personally, Michael, I did take the census online. It was really easy to do. And I definitely recommend, if they haven't already, everybody listening out there, so important that you participate in the census so that they have complete data. So Michael, Jared, thank you so much. Jared, congratulations to your team for winning the award and such a great customer. Michael, thank you so much for what you and your team are doing. Appreciate all that's being done, especially in these challenging times. Thank you and thanks for doing the census. All right, and stay tuned for more coverage of the AWS Public Sector Partner Awards. I'm Stu Miniman and thank you for watching theCUBE.