 Good afternoon everybody everyone excited to be here My name is Jennifer Johnson. I'm the associate director of the quality assurance for the AT&T integrated cloud I have been with AT&T for 11 years. I've held numerous positions within IT operations from engineering lab management to quality assurance for the past four years I've worked with worked in AIC which is AT&T's globally distributed network cloud that has more than 80 zones deployed My team consists of architects as well as quality engineers around the globe I'll be moderating today's session with a panel of women talking about their experience working with open stack To deliver AT&T's integrated cloud This is my third year at the open stack summit and I've have seen tremendous growth within women Who've been here, so it's been really exciting to see the progression The women of open-stack foundation Have done a wonderful job with the encouraging women to be part of the growing community But let me say there's a lot more work to be done So I want to share with you a quote from our SVP of HR that really resonated with me cry like a baby Fight like a girl and change the world like a woman The mantra reminded me that it's okay to cry, but let me say not in front of everyone but fight like a girl stand up for what you believe in and Change the world like a woman be the leader and not be afraid to succeed Our panelists up here today know all too well What it means to overcome? obstacles and surmount challenges that they faced The work that they have done has been essential To AT&T's cloud transformation With that said let me introduce you to the panel who are key contributors from development Test and community teams within ASA Hi, my name is Janet Morris, and I'm a director of technology at AT&T responsible for the overall quality assurance of AT&T integrated cloud I Have 120 employees and consultants who work together and define the strategy for Testing the automation framework and tools we use the test plans and actually execute the tests We are certifying all new features that are being delivered as well as every deployment in the field Jen our moderator works for me, and she is responsible for the system test team I Get the privilege of being the most senior in years on this panel I've spent all my years at AT&T and have held a variety of jobs from software development system engineering and of course quality Assurance I've been working on cloud computing for about seven years starting with VMware and the open stack the last four years And I've seen the tremendous change within AT&T. It used to be open source was a bad word It was gonna, you know, it was gonna hurt our security It was gonna infringe upon our proprietary and you just couldn't use it and now we're fully embracing it So it's been amazing transformation On the personal side. I'm married and I have three children who I've raised during my career They're all fun their own being independent and I'm now enjoying the empty life emptiness syndrome not empty life AT&T make sure of that Pass it on to Anjali Yeah, my name is Anjali Bansal. I'm a quality architect working for AT&T integrated cloud project I'm an Accenture employee working for Accenture from past seven years and an AIC project I'm working from past two years We work on automating and execution different different type of test cases like API, CLI and GUI Which we will discuss in detail as the presentation goes on So I'll pass to Kayla. Yeah, hi. Kayla from me. I started at AT&T out of college in 2007, so I'm coming up on 10 years. I was a Java developer for about the first seven when I joined AIC at the very beginning when we were Had two production sites and I've been you know there from the beginning growing from two production sites to 80 around the world. I manage Directly manage the open stack development scrums Managed some data center large deployments and now I'm managing control delivery Hi, my name is Darla Ailert. I apologize for my voice. I've been struggling with it My story is a little bit different than these ladies because I'm pretty new To this world. I graduated two years ago today. Thanks to Facebook for reminding me of that this morning as I was growing through my news feed. I Graduated with my master's in computer science and I was hired into AT&T as part of the college hire program I started working for AIC shortly after I was hired into AI or AT&T I'm currently working as an upstream developer strictly solely in the community I don't really actually code anything in AIC all the work that I do is in the open stack community Which is really cool to be a part of And my main focus right now is open stack helm, which is going to be a huge part of Deploying our containerized control plane Thank you So now that you know our panelists Let's take a look back at AT&T's AIC's journey and the recent history with open stack and how we plan to continuously improve in 2017 in 2015 we successfully Executed and deployed Significantly scaled out AIC in a very short amount of time We deployed 54 zones in three months and fully tested each zone Through the effort we won AT&T won the super user award Back last year in Austin Very proud moment for all of us that worked on the stage and very AT&T The women here on this stage were instrumental to that and Enabling that so in 2016 we continued our zone deployments getting through 80 plus zones Which enabled us to virtualize 34% of our network? AT&T focused on contributing back to the community with that we improved our ranking to top 20 Now taking us into 2017 we are contributing to focus on network virtualization to support the network traffic growth that is going expecting to grow 250,000 percent by 2020 Our target is 75% virtualization of AT&T's network by them Part of supporting that we're augmenting our footprint and to increase our capacity and build seamless upgrade Processes to enable features So Janet can you share with us the audience and how we're going to accomplish our goals to transfer? sure In our leadership team and AT&T we're focusing on Touching on three areas that we're focusing on to bring our organization there First and foremost is to make sure that our workforce has the right skills that we need for dev ops environment So AT&T launched a corporate program called capabilities evolution that has built curriculums that deal with technology transformation Cloud computing big data and other software Areas where people can take the classes and earn badges and mature their career We also have an annual program focused on the soft skills where our leadership gets out there and Teaches us on how we can work with our teams or within our teams and embrace change take risks and become energized to To change and pivot of course. We're a very diverse Employee base and who we work with so diversity is important to us. I'm proud to say AT&T It was ranked number three in diversity Incorporated and has many programs Employee research groups like moment of AT&T They have mentoring circles and we outreach communities to encourage STEM programs throughout We are always looking to improve our processes and a common theme for us is automate You hear that a lot you'll hear today automate our deployments automate our testing and the other thing I want to touch about is our need to collaborate We're really encouraging collaboration if you walk into an AT&T building now You'll see walls being turned down in very open space and really encouraging us to work together since we are global We are doing leveraging all types of technology to communicate again with each other and tearing down those walls within our organizations and Also, it's not just to learn about the technology, but we must use it So tools have been an important part of our automation and as I said before now We look to open source as a first place We want to use things everything from it, but we also want to give anything we build back into community Which is a huge transformation for us Thanks, damn it. That's great to hear. So with all this talk around automation process tools Can you talk to us a little bit about development? Sure. So like Janet said automation automation automation. That's our mantra and development we've you know come a long way from the beginning days of automating a site of the prerequisites that come that go into building a site and including you know taking a 22 tab spreadsheet and turning it into an application that feeds input into a deployment tool we've In addition to deploying sites a lot of effort has to go into managing those sites we've Transformed our day to operations once a site is deployed. We must try to keep it consistent and not try But we must have it to be consistent to ease our operations and also ease future upgrades and future deliveries So we've provided tooling to Do day to operations including what we call open-stack resource manager or M It Deploys and creates flavors and tenant images and many sites across the world And that allows those flavors and images to remain consistent We have automation that audits sites and exposes variables across the site so that you can see the different package versions That you know shouldn't be there In addition to automating the deployments of our sites and reducing the time it takes to deliver a site we You know a lot of effort has in development has gone into enabling the network transformation Which Jen described we have contrail at the center of our platform and we've put a lot of effort into Automating from beginning to end of contrail so taking packages frequently and into development and delivering them rapidly Also upgrading existing production zones with workloads on them and upgrading contrail and existing sites So there's a lot of work with the automation and the processes and the tooling and I'll let Darla talk a little bit about the people sure So whenever I first started back two years ago roughly We had pretty much zero contributions to the community there were some but they were far in between Actually, one of the contributors is sitting in the back of the room. So thank you for actually being a contributor before we started our team Myself and a few others were tasked with Starting to grow a team that was solely focused on contributing back to the community along with that we wanted to Really be able to shift the mindset of people in our company and our leadership To realize that not only is using and adopting open source a good thing But contributing back to it is even better, you know, the the open stat community has given us so much It allows us to run our cloud. Why don't we give back to that community? So We started building up a team In Barcelona, I gave a presentation or I was part of a presentation that talked about that journey About where we were when we started which was essentially nothing to where we had grown up to the time And we continue to grow our contributions keep getting larger and larger, which is exactly what we want Along with that we've kind of shifted our mindset just a little bit more To be a community first development We want to go out to the community with our ideas about new features or new projects and Collaborate with them on on those new features or projects first and that way we can make sure that we're helping build something That's going to be beneficial to the entire community And once you know once we have it out in the community We can backport it back into a IC as as needed So I think that that's a really big part of what we're doing now I think a really good example of something is OpenStack Helm We have recently started an official OpenStack project called OpenStack Helm which Uses Helm charts to deploy OpenStack on top of Kubernetes It's going to be a huge part of our October goal of having a containerized control plane And that's going to allow us to have you know the the seamless upgrades that I think Jen mentioned a little bit ago So that was kind of where we've come from or where we've gotten to I should say Thanks Darla. I think that's a lot of great information now. We're moving along right we need to test that so Janet Why don't you talk through where we were in our journey for testing? I Love to talk about the our ship to the left because it's a real success story for us When we started testing AIC we focused on creating our test plans and our test cases for all the new features and we executed them Manually at the end of the test cycle then we would look at it and say okay Which of these tests we want to automate and put into a regression suite and that we would use to validate our deployments and we would do post Validation of future releases to make sure we're backward compatible So we we were proud of our automation there. We saw how good it was going and about a little over a year ago We questioned and said wait a second. Why are we waiting till after our testing before we automate? Why don't we start automating before so we worked with Accenture who is our partner and said hey We want to start automating faster and sooner so we looked at our staffing profile We said we need to have DevOps skill We need our testers to know how to develop and how to write scripts and and do that from the beginning And we implemented an agile scrum team just focused on delivering the automated test scripts So now we have 80% of our functional tests automated at the start of our system test cycle we're able to execute about over five times the number of test cases in a set cycle because of that and We're finding defects much earlier in the process So of course we're not satisfied with shifting to the left. We are continuing the future to do more We're working closely with development to make sure that we are partnering and moving tests even earlier in the development And they're sharing their testing from unit test with us. So we want to continue on that journey Also, we're really proud of our test framework that we have and we're looking within AT&T as we partner with Our e-comp team and own app team within the company of extending that so we have one single framework that works across and Now Anjali is going to talk about that framework Yeah, sure Janet so as you have mentioned that we have shift left automations So that is simultaneous development and automation which will help help us to achieve 80% of automation Prior to ISD code drop so shift automation help us to execute 3500 test cases in first three days of Execution and which will directly help us to achieve 80% of execution in first 24 hours So we have different type of test cases as I have mentioned before that we have API CLI and GUI Which we have different type of frameworks So for API whether it's OpenStack API or non OpenStack API we use framework called as Tempest So for Tempest whether it's a so what is Tempest? Tempest is a set of integration test which we run against OpenStack live cluster Tempest we mostly do the black box testing To run Tempest we have configured a lot of parameters like tenant, flavor, network, etc So as of now we have configured around 200 options in our config file To run if we have to run small set of test cases what we call as sanity or smoke We use the ToxE smoke if we have to run some bigger set of test cases What we call as regression test cases we use the ToxE full But if we have to move to non OpenStack API such as provided with Contrail or VMware We use the Tempest plugins. So what are Tempest plugins? So Tempest plugin help us to integrate external test suite to a part of Tempest run We can either run Separate plugin or we can integrate all the plugin at the same time and we can run whole Now second type of test cases what we call as CLI test cases So for running non OpenStack functionality with the CLI we use the framework called as TestInfra framework So TestInfra framework is a Pytest implementation of Tempest is the same Pytest implementation of ServerSpec framework ServerSpec framework is based on Ruby language But as of our OpenStack guys love Python the most so what we have decided we have decided to move the same ServerSpec framework to a To a Python language and that we called as TestInfra framework In TestInfra framework We execute our test cases and we have achieved our complex configuration like SRIOV and DPDK etc Now our third and last kind of test cases that is called as Selenium We run our GUI or UI test cases with Selenium headless framework and the object Which we have the model which we have used is called as page object model So whether we have to run horizon test cases or we have to run the non OpenStack UI test cases We use the framework called as Selenium headless framework Oh No, wow, that's a lot to take in right I think from you can see it's very complex What we're working on and how we're moving the needle forward? So You know, I'm gonna start this off asking some questions to the panelists You know, I'm gonna move this back a little bit and in a direction of talking about women in technology Janet, can you can you talk to us about your own personal experiences and challenges that you have personally faced being a woman in IT? And the biggest challenge it's not just for women, but I think we maybe take sometimes still hard faster is balancing that work between family and Work For me when I had my children as I said I've worked right here the whole time I wanted to be a part of I don't mom, but I also did not want to stop my My work I thought if I took some years off with the way technology changed I would become irrelevant So back in most days, there was really no telecommunity went to the work every day and long days So I chose to take a path of working part-time Which I did for about eight years until my young children were full-time in school So that was great, but it was also tough because I still wanted to work on the the best projects the top projects I wanted to be the leader and not take a second, you know, see behind someone else So I ended up working real smart I guess very focused most people didn't even know I was part-time and you know because I Was trying to do everything in that period of time I don't regret it I think it was the right choice for me at that point in time and going back now Technologies change so much and it's given us a lot more flexibility, which is great when my kids did get sick I could I did not take a vacation day anymore. I could work at home and things like that But even you know at this point in my life It's those boundaries between life and work and the hours are always there and it takes a hard discipline To to separate those two, and I think that's a lesson for all of us It can become very encompassing and that's the hardest thing is to balance it I can definitely relate right work life balance is is quite hard Especially when you have young children at home or you have older kids that you're trying to transport and you're on calls and Working all types of hours being in IT Kayla, how would you what would you add to that being a woman in IT and and going through? You know the career and some challenges that you have faced Yeah, just add on to what both of you said it really helps when you have co-workers and bosses and Peers who support your work life balance as well, you know if we've all called each other on the weekends or at nights and You know when you work together to achieve a common goal You know, I don't care if your kids in the background screaming We get through it and move on and then we solve a problem and go back to work the next day and continue Yeah, so Darla You've recently graduated two years ago. I'm curious. What was it like in the classroom versus the workforce? In the community Yes, so whenever I was in school Undergrad and graduate actually I was it was normal for me to be the only girl in my class I did pursue a computer science degree, and it's not very common for women to go into a degree like that You know there might be two girls in the class with me And that was like a really good time because it was the only time I got to actually talk to other females No comment so anyway As I moved on into AT&T, you know, I've only been out in the the corporate world for a couple years But there are so many more women than I expected They may not be developers. They might may not have gone into, you know a STEM degree But they have the technical knowledge that they need and just working with a lot of women in the technology area and Seeing, you know seeing them flourish is it's a really cool thing Being here today. We had our women of open-stack lunch this it was in this room The room was almost full and I thought that was awesome because I've been to three summits now and I feel like each time the number of women that attend that lunch is growing And I think it's a really good thing to see is, you know, promoting more women in this type of workforce Thanks So, Anjali, my next question is for you Technical difficulties So where did the skills that are required for you to be successful in the environment like as large as ours? So I can list on any number of reason of what are required for a QA to be successful in an environment like us So first of almost which I want to say we should have holistic view of our platform For a developer to be successful They just need to know one functionality for which they have to code but for a tester to be successful They have to know all all about the platform how they are connected between the platform How the data is flowing how APIs are flowing so they should have know each and everything about their platform To work on an open stack project like us. It's a very big project So we should have a strong networking environment. We should have strong networking knowledge To code the functionality. We should we should be good in programming a skill We should be good in automation a skill. We should be good in development skills And also the person should able to try as the code which is developed by others and able to fix that code as well The other thing that Testers should have good analytic knowledge because they have to write the test cases or the use cases prior to the zone Handed over to us so they should able to think what is the functionality Expected and as we have now moved to the shift left now We have started the automation prior to the zone handed over to us. So we should have good analytic knowledge Thanks a lot of great skills. I think we need right to be successful So Kayla You know around automation It's essential, right? So why don't you talk to us especially in this scale? Why don't you give us some tidbits around? Some challenges that you face and some tidbits that you give back to the audience Sure, it's hard to think of just one challenge that we've had to work through. I think the biggest thing that has made us so successful is the Complete automation in day two operations. So, you know Automating pieces of a platform is a little simpler than automating from end-to-end in the last You know half of your year. We've gone from You know automation several tooling to a complete full continuous deployment cycle leveraging a Jenkins pipeline that deploys a Cloud site with zero touch Wow, that's pretty impressive. I would say So At this point we're going to open it up for any questions that the audience may have for us Please use the microphones if you do No questions guys Thanks Given your comment about how few women were in your classrooms. What do you suggest? University should do to better attract women We get people confuse us all the time all the time Okay, I'm Darla Having women pursue STEM degrees is a huge For me personally I try to always gear towards the younger girls I'm very active in getting younger girls involved in STEM I Participate as much as I can in the STEM related Activities that AT&T provides being part of the college hire program gives me the opportunity to go out and volunteer for events like that I think that it's really important for Younger girls to understand that they are capable of getting a STEM degree You don't have to be a teacher You don't have to be a stay-at-home mom if you want to go out there and get a math degree or a chemistry degree or anything Of that's like anything of that sort you can do it and I personally really enjoy talking to young girls about it because I Struggle with myself. I'm definitely have imposter syndrome. I don't know anybody in the room that probably doesn't but Whenever I was going or whenever I was in high school I was trying to decide, you know, what degree should I go towards? It wasn't if it wasn't for a high school of female high school teacher that I had telling me Hey, you need to go for this because I know you'd be awesome at it. I would have never gone there I would have never even thought that that was a degree that I could pursue and I think that it's really important for us to Start getting in the minds of younger women and letting them know like you're capable of doing anything that you would really want to In one comment I have because we have this conversation is you know graduated, you know more than 30 years ago and computer science degree and Have a similar experience. So I was kind of shocked that 30 years later. They're not more penetrated. So it's pretty sad I don't know what we can do to encourage it And for me it's since med darl. It's been inspiring seeing her do those things and I've tried to take opportunities to do the same AT&T offers number of programs right to give back and get involved. So we try to do that as much as possible Any other questions So I have a question for the whole panel considering that Technologies a traditionally male dominated field what experiences have you guys had that has helped Shaped who you are today. It has enabled you to succeed in that field. I can start So I grew up the oldest of four kids and have the 49 first cousins and there were a lot of boys around my age And I grew up playing sports Outside and also on a mostly boy boy. I'm CA soccer team and went to college and played soccer But I think that made me very competitive and also confident in what I'm good at and You know, I've always been told you know the most at what you are doing and that and continue doing that and Improve on your areas of weakness So I grew up My father was a I grew up basically in a data center. Let's let's put it that way When when computers were around so I used to take the tapes out and put them back in and that was my job, right? And I swore back then I wasn't going in when going to be in technology, but interesting enough I followed in at footstep and you know here today. I Love what we're doing. I feel like I'm challenged I think that if you are analytical and you want to contribute, I think this is the field for women I think this is something that you should take part in and I encourage young girls to to do that, right? so when I recently had the opportunity to speak to some some boys and girls well back eighth grade class of my sons and It was interesting to hear a lot of the girls did not want Did not want to go into technology. They're all about fashion marketing right all the all the nice little trends of you know looks great on paper, but Once I got to you know talk to them a little bit about the technology and and talking to them about Facebook they kind of Whether the social media sites where they can relate and how the data is stored They were really got they were really interested and they started to ask some questions like oh, so you know Where do I have to go to school to do that? You know so it was interesting so you know bringing that back I think it's important especially as us as women is bringing it back to to the younger youth to say This is a field you want to be in so I'll just say my comments slightly different I'm like was for my old female family and you know your first generation to go to college So I didn't have anything. I just love solving problems and math was great cuz You could reverse engineer everything and know you were right So I just went there but I started working and I was not used to being in a very male classes with the same way And I think AT&T helped me a lot because they were very open to women even you know those years ago and gave us Opportunities and learnings and you know they helped me really grow and flourish and a lot of times it was uncomfortable for me I had to go out of my comfort zone to do things differently, but I really attribute AT&T to Making me where I am in my career Yeah, so as I mean I'm from India I'm from a small town so 10 or 15 years back girls education was not preferred that much in India so To continue my bachelor's outside of my hometown I fought with my family that I want to continue and then after getting campus lecture I have to join Ex-Ansure that time also I fought with my family that I have to continue and I have to work in IT Then from India coming to US that time also I fought with my family that I want to continue and grow my career in IT So I would say I have faced a lot of struggle But I continue what I want to continue and I think I think along with all those things Being in a male dominated field having male advocates is amazing There of course there Some people that aren't necessarily advocates But the ones that are I think make a bigger difference and you know most of my college professors were male But I had so many of them give me such good advice and push me harder than I would have ever thought and You know having even just anybody regardless of gender. They are pushing you. I think that's an extremely important thing Absolutely challenges Any other questions Since some of you worked in prior domains with an IT Before moving into the cloud space and open source and open stack, how would you compare? Diversity in terms of you know how you've Approached your career the last couple years and compare that to Yeah, I most of my years prior was traditional software development Applications and I do think on the application side You do have you had more women than I saw when I started moved to hosting and working more Hardware and infrastructure so networking when I'm working with network engineers. That's where it's a difference in lack So I do I did see more experience and more development More in the software side now everything is going software So I think maybe that will increase but I think that's a big difference when I started in the latter part I started seeing the old trends come back because I was moving more into an infrastructure networking world Yeah, and for me I started off writing requirements engineering and E-commerce right so there was a lot of women and believe it or not. There was pretty I would say we had Had a team of ten it was about five. So it was about 50 percent You know then as I shifted from into lab management, right? definitely the It was male dominant right for sure It was probably I think I was the only female at one point So it's encouraging starting to see now from a platform from a quality assurance team I have a very large team that has a Significant amount of women and I try to make sure that we do have diversity in the team because it's important Right to show that and to have different perspectives Any other questions best bosses I ever had her name was Lynn Bickley. She used to work for AT&T and Maybe taught me the most of my career So I I'm not sure she wasn't the best but but there are other bosses that might hear this So I have to be careful Are there any special challenges you mentioned diversity in your group so that's got to include men Have you had special challenges with men on your teams? or How do you approach it? Is there any? Thing different I Can take it Most of my team I like I say I've only been in the in the workforce for a couple years But I've pretty much only worked with men And this started like I said prior whenever I was in my classes Prior to college I was used to working with my girlfriends You know if they say your teacher says group project y'all like look at each other like yeah, you and me you got this we got this And then I would I went to college and I was with all men They don't want to pair with the girl in the room I Don't know why I don't know if they're just like nervous around girls, which is true Or if you know, maybe they I don't know if they think I'm less competent or maybe they think I'm a genius And they're scared to work with me. I don't know but you know that that is something that I struggled with Even now We have a lot of male advocates. I feel like on our team, which is awesome But I have worked with other males that are Not as willing to work with me and I it could just be random But I don't know my gut tells me it's because I'm a female. So, you know, it goes both ways There have definitely been challenges, but I think for the most part we I We just get through it Right. I think it's having the common goal, right? You have something you're going to work on you focus on that and you need the diversity of different perspectives, right? Everyone you may have Person who's very analytical right who is a male female male doesn't make a difference, right? They have a different view set than what you may have So it's hearing listening and hearing what they're saying is really important but making sure that you're building the team to have equal rights, right equal perspectives and and If you have that then you'll have a team that's successful that can collaborate work together You know synchronize together and be in play, right? Then that's teamwork, right in collaboration. So Before I joined ASC I was a software developer and then I'm very small team and I came into a IC and I thought wow this is the most diverse team and organization I have ever seen and it was Exciting to work with people from all different kinds of backgrounds because you learned where they came from and I learned so much outside of Just the technology Okay, any questions? All right, we got a wrap up So I'm just going to wrap up with one slide so We here at AT&T are partnering with other large enterprise companies to focus on contributing operators needs so if you're interested in Working, you know long along side us and contributing back We're partnering with several different Companies large companies We have multiple sessions that are being that are going on So we would love for you to join us one's on road map working session Also, another is on contributing back as far as the large opens back contributors operators Thank you. Thank you