 Hello, everyone. Can you all hear me? How is everyone doing today? Thank you so much for being here. I appreciate you taking time to share these ideas with me and I look forward to an interactive session. So I welcome you to share your questions or points as and when you see them. So the title of my talk today is time for disrupting the status quo case for diversity in open source. So we will talk about diversity in open source. And we will talk about, you know, what is going on and what is the need of disrupting things, what we want to do as we go forward. So my name is Wanda Singh. I am an associate professor at the high school at University of Tennessee. I have about 20 years of research experience on open source software I've looked at variety of settings variety of issues and open source software. My dissertation research from University of Illinois at Urbana champion was also on open source on users of open source. Recently I have been working on diversity in open source software I've been specifically focused on experiences of women in open source software. And so I would share some of those things with you if you would like to know a little bit more about me then here is the link to my website and also my email is here if someone would like to get in touch with me or ask questions or collaborate or you know whatever. You might want to share with me and then my email is right there. Before I go deep into my talk, I would like to address and acknowledge that we all are going through, you know, the global pandemic COVID and everybody is dealing with a lot of different things. And so I would share with you that I am originally from India and, you know, right now India is going through a horrific second wave. And so we are all dealing with a variety of, you know, issues so please be a cognizant empathetic towards people who you might come across who are either from India or from anywhere else this is something that's happening everywhere. And I would just request you to be compassionate and show a little bit of empathy to your colleagues to your coworkers to your mentees to people who you're working with. Because this is, you know, personally just all seeing for myself, I have in the last week heard of so many deaths and COVID positive cases and lack of resources. So just wanted to put that out there and just share this space in a little bit of issues that I just wanted to acknowledge that and not, you know, not think of our existence as being in vacuum of those issues. Oh, someone said, I don't know how that might have happened but you can unmute if you have something to say. And or if you would like to share something with us or you can type it in the chat as well. So I would like to go ahead and talk a little bit about you know what the outline for the talk today is the first thing I would like to say is that I would really encourage you all to participate in it I would like it to be an interactive session. If there are things that I have to share then I will also ask for your input so please do participate in the input and and you know, share your thoughts on it. I'm really pleased to see the increasing number of participants I think that's a very good sign that you know this is something that people consider important so. So the way I want to talk today about this topic is you'll get started with diversity in OSS. Why bother. Why are we here why why should we think about it why are we talking about it. Why is it important so we'll go get started with you know some definitions and stuff but then we will talk more in depth about. Why is it important, why is it important that we care about diversity equity inclusion in open source software. As I said I have about 20 years of experience of doing research in open source software, I will share some of my recent work in women in open source experiences I'll share some of my. You know, key takeaways from a few of my papers, but more specifically I will present some actionable results from that research. You know what are the things that I found and what are the things that we can take and take actions on. And then we'll also look at what is going on around us you know what are the. Current diversity initiatives that exist in open source software and some of you might know more than me on that because you might be involved in that in those issues in those initiatives so I will welcome your input in that aspect also. And then we will look at you know these initiatives exist. What are the challenges, what are the challenges that people face, or we face as a community when we talk about issues. And then, what can we do about it, what is the way forward, we have, you know, we know diversity is important by the point we get to this thing we're talking about what are some of the actions we can take. What are the further things that we can take and I will count on you I will ask you I will put you on spot to make commitments of what will you do to make a difference so we will, we will talk about what can be do as individuals what can each one of us do. And then we will also think about what can communities and community managers do what can be done in order to. To make a difference to move the needle. So that's the plan for today and like I said feel free to ask questions or was them in the Q&A channel or if you have a quick thing that I could answer. Also we will have these breaks so at any time you know feel free to communicate. So I would like to start with some definitions. We're talking about diversity, equity and inclusion. What, what does it mean, how do we define how is diversity defined. And this is, you know, little text heavy so please bear with me. Diversity is essentially used to describe differences, it is used to describe individual differences and it is used to describe group and social differences. Individual differences can be about your life experiences what you have seen around you how you learn, what do you like to learn, you know, are you a visual learner, are you a auditor or learner. What are your working styles, are you an extrovert, are you an introvert, and then also the broader concepts of group differences or social differences so you know the race the socio economic status the class gender sexual orientation abilities country of origin intellectual traditions, cultural and religious. So essentially in a nutshell diversity is when we, when we are talking when we are using the term diversity we are talking about differences. Differences among people they could be at individual level and they could be at group level they could be at country levels they could be at racial levels. So we are talking about diversity we're talking about differences. So it just establishes that there are differences. And we have to talk about inclusion so knowing that there are differences is good. It is important to acknowledge and appreciate and, and, you know, to say that yes it exists these type of differences exist among us we are not all the same. The inclusion term comes in where we are making an intention. So that is going to be one of the things at the end of the session I will ask you to do is make an intention make a commitment to engage an active intentional engagement with diversity with these differences that we see so the differences, and then we commit intentionally to engage to acknowledge to appreciate these differences. How do we do that how do we engage with these differences. And when we talk about that how do we want to engage so okay we want to provide resources that's where the concept of difference between equality and equity becomes very important. So here this sometimes you know some of you might be very clear in what the differences are and some of you might not be so just for the sake of clarifying the definitions I wanted to present this image and this definition of equality versus equity, equality talks about giving everybody the same resources and equity talks about giving everybody the resources that they need so that they have a fair chance at a fair outcome. This is a distribution of resources in in the perspective of what is needed so you can look at this picture right you see the little baby in the purple shirt. If we are doing something that is equality wise then this is what we're doing we're giving them all the same treatment we're giving them all the same resources. The baby doesn't have a chance to look at at the at what's going on in the game at all if we do that. So equity is giving them support that they need so that they have a chance at least have a chance at a fair outcome of whatever is going on there is you know fairness in that. So that's one way of looking at these definitions. There's also this really fun example that is from the University of Michigan diversity center that I really like so diversity is where everyone is invited to the party. So everyone can come we're inviting you you're here and we acknowledge the differences equity means that everyone gets to contribute to the playlist. So everyone can say what is going to be the music in the party so you're all like a part like everyone can bring different things. So even means that everybody has the opportunity to dance. So there is this is further engagement this is intentional further commitment engagement. Yes, so, you know, thank you Kyle for sharing that. So do you know this is the opportunity that is given to everybody so that everybody has the same chance at a fair outcome. So this is the base level you know this is the base level understanding of what diversity equity and inclusion stands for. With that. So, why is it important. Why is that diversity inclusion and equity important. Why should we bother about it, and then specifically in open source software. Okay, why is that important. So I would like you to think about it for a minute and we're going to brainstorm so we're going to use a Jamboard, and the link is right here. We share the link to the Jamboard, then I'm going to pull it up. So we'll take about let's say about five minutes for everybody to contribute to this Jamboard. I would like to see what you think. Yes, Neil. Yes, more outcomes that are more effective for everyone. Absolutely. So, if you could go ahead and use these sticky notes like this, and then just post what you want to say so here's the sticky notes you can use those and Yes, yes Bianca, a diverse set of contributors can bring more productivity to the team. Absolutely. Very good point. Several approaches. Yes. More empathy. Yes, that is correct to get a variety of thought processes. Yes. How does that help us having a variety of thought processes. What does it get us software is more secure since neurodiversity yields better results exactly. Very good. Yes. If we are not diverse we are missing enormous swaps of the population as potential contributors employees even users right like, you know, if you're not more inclusion. Different backgrounds can get different approaches in software diverse opinions can better target a wider net of users and invite them exactly invite them to use your products. Diversity means outcomes are more effective for everyone exactly, exactly. Diversity improves the working experience even for the existing community members. Yes. Different backgrounds enable us to tap into different approaches and to ensure that open source is open to all yes very important it's our philosophy it's the philosophy of open source movement so why would we have to block it for certain groups right to ensure impossible for design for what you cannot understand impossible to design for what you cannot understand yes. More diversity means better cross disciplinary integration design better products designing a product in US for people in Africa doesn't work without participation from users real users absolutely that's why so we see so many products that we don't know about the end users. More useful software for everyone yes diversity means we are closer to what the real world looks like yes, exactly yes open source is a driver in science and diversity of thought is critical to benefiting all people in cultures. Yes. Yes, that's very good unbiased software theater experiences better opportunities. Yes, that's how open source work it should be open. So collaboration means we grow as human beings we are challenged. Yes. Can help to improve the accessibility of the project to different audiences that's a very very important point the point about accessibility because that is really going forward is is has to be more of a priority than it has been so far is to make accessible make products that are accessible so that people neurodivergent people and people with all kinds of accessibility requirements are able to use tools. There were set of contributors can bring more productivity to the team yes diversity increases productivity like the anchor has just shared a link there's also a link that I'll share in a few minutes about from Harvard Business Review which did a study on different groups and found that if the groups are diverse than the productivity increases. The groups are contributor base and our user base by bringing a larger pool of people into the process yes complete and better thought out solutions. Yes. This is open source for everyone no matter who are you and where are you from it is for anyone and anyone should be able to work together on the project that's excellent that's a very very good point attain and retain talent to contribute. Thank you so much for contributing I'm really pleased to see all this and we'll be making you know screenshots of these and then. Making these available as part of the slide deck. This is wonderful group think rarely leads to the best solution yes. That's exactly what it avoids is the best solution diverse perspectives lead to improve solutions everyone wins and wins more together. Yes. Diversity means less people will feel lonely and more people will feel represented and representation matters it is very important to see people like yourself. It's very important for all underrepresented minorities to see someone like themselves being represented in the environment in which they are working. It's the right thing to do yes it is the right thing to do we are running out of room okay so this is perfect thank you so much thank you for participating this is wonderful. Yes, and if I missed reading any please go ahead and read your. What is the sticky note did I miss anything, otherwise we'll get back to the presentation. Okay. All right, perfect. So, thank you, and I'm going to just close this window so that I can be back on the presentation. And then of course I'll come back to this to this again also. So that's we've seen so many examples right we've seen so many things of why it is important it improves productivity these are all excellent points that we just saw so you guys are already very you all are already very aware of why it is important. I'll put a screenshot of the Jamboard right here later on, but just some quick statistics about you know kind of getting deeper into open source software. So overall, women make about 51% of the US workforce, more than 20% of the tech workforce is women which is not very high you know we have to we are working on it we are working on it for decades to improve it but we're not going beyond the 22 30% but the less than two to 3% of the OSS communities women that's that's really very, very, very low and also very shocking to a lot of people. But that's the reality and for decades these numbers have been stagnant there have been opportunities there have been things there have been initiatives for about 10 years. But seems like we are really not moving forward, we're not getting more women participating in it so two to 3% is kind of the average of all the different statistics that I found I found something as extreme as 1% and then some things which are five to 6% but nothing, you know, nobody says there's more than you know 6% 5% women in open source software. So two to 3% of women, why there's 50% women in the workforce. If we say it's just about technical skills there's 2526% in in the technical workforce. So what is going on with the open source environment. What is going on why are our numbers not very strong. Let's look at some more reasons for why diversity in open source software. Like I said, you know it doesn't look very good. I don't know why we have to think more about it we have to find out why these numbers are where they are. They look pretty horrible. And the experiences that we hear about the anecdotal experiences, even empirical research what we hear about is that people's experiences are mostly negative, we hear a lot about that they get a lot of press. The bottom line is hurting, right. There are more and more jobs in it, but it seems like we don't have enough qualified people. And then we have qualified people but for some reason they're not participating. So we do need to fix that right. And like you all said, diversity is great for innovation, multiple perspectives people with different perspectives when they're there. It improves productivity, it improves innovation, supporting diversity is in favor of creating innovative software and who doesn't want to create innovative software right if you want to create innovative software then you should have diversity in your projects. And if it's not there then you need to come into it and you need to do something to fix that. I think that point was also mentioned somewhere there that women use the software, right. I'm reading Jim's comment women are but one group underrepresented African Americans indigenous Latinx another also not at representation levels that would be appropriate that's exactly correct. So, you know, I will talk about some of the statistics in terms of underrepresented minorities which includes a lot of, you know, these groups which absolutely are not at any, you know, levels of acceptable numbers. But you know some of the work that I have done has been focusing on women but as soon as I entered that area of research I immediately realized what you're saying Jim. It, you know, I am broadening my scope for understanding, there's been, you know, there's been very interesting differences when we look at just the women versus underrepresented minorities as a group which of course it's not a homogenous group neither is women a homogenous group, but still looking at these groups there are differences and we have to acknowledge them and we have to work for improving diversity from all perspectives not just just from the women perspective. And that's why defining diversity in all those differences that we saw is important. So, you know, the last point is if women are going to use it and that can be said for you know like it was said for people from Africa people from India or people from anywhere are going to use it. It would be good if they had an equal say in creating it so that the products can be successful. So again, this that I'm saying can be said for anyone who is underrepresented in open source environment, why would underrepresented minorities or why would women care. Why. Okay, you know a lot of people say well they don't like it they don't like the technology or they don't want to be working in these environments. Well, that's, that's not true, first of all, from the research that we have done either on education or in these communities that's not true. But also, there is this inequality that happens when we have less women when we have less minorities. And that is that all the benefits that other people who are participating in these communities the benefits of participating in open source software, they are, you know, devoid of those they don't get those benefits. So we are creating rather perpetuating a social inequality that exists. People who are in these communities are missing out on these benefits and opportunities of participating in OSS software. You know in my early days when I in early 2000, when I was doing research and open source software the whole big question was, why does anybody do it who are these people who are contributing to you know what are the motivations of developers was a big question. One of the big things that all the studies on that found is that the skills that they learn the, you know, programming skills the coding skills, management skills, the networks and connections that people make the job opportunities that they avail because of being in open source environment. They have a documented experience so we all are aware of these stories of people who did really well who created a very good open source software and were hired by a big company for big parts right we have all heard stories about open source and people who keep developing new things so there's a whole lot of benefits that come from participation in open source software, and by this under representation we are excluding a whole group of people from these benefits. So that's why they care, that's why they would want to be in there or that's why they should be in there. So learning right learning is very important in these communities, a lot of people and join these communities just for the purpose of learning. So, so in, you know, why, why diversity in open source software. In addition to the things that we have talked about in the previous slides and on the Jamboard. I was saying that open source culture offers the best work life balance for women and those are the types of things that you know we will talk more about is that those are the types of things that we need to make people aware of we need to make people aware of what are the benefits. And also how we deal with the bad reputation you know there's all these stories of hostility all these stories of, you know, anecdotal evidence of, of harassment of misogyny of discrimination that people feel are experienced. How do we, how do we talk about that. These are shows hostile discriminatory predatory experiences of women and underrepresented minorities so we have to, we have to do something about that right. I'm going to read Bianca's comment, women's contribution or so, or even more accepted than women's women do not join or leave us as because besides learning they are highly motivated by yes. Social factors for the reasons for which they want to create. Yes. Yes, it is a combination of anecdotal and also empirical research I will just Gina in a minute share my own research where I have documented this so I completely agree with you and one of the reasons why I went into this research is because there was a lot of discussion of this just being anecdotal evidence. So I wanted to document it in empirical high level journals of software engineering that this happens and that we need to address it so I'm right there with you. You must be aware of the privilege of free time though many historical and representative groups are disproportionately in charge of care work and other responsibilities that take away from free time for contributing. Absolutely and that's why we have to find solutions like sure I was saying earlier we have to find solutions that work for people that work for different groups of people and the benefits that they get from them have to be different. They go back going back to, you know that slide about equity, right, how do we make it so that everybody has a fair chance to gain gain those skills. So I'm going to now talk about some of my research that I have done on this. I will share a few of my, you know, papers and some actionable results for them. I do have a list of all the published articles on on my website, and also current ongoing projects that are there. So if you would like to, you know, the articles of course have a lot more details and right now I'm just presenting, just presenting the key takeaways and what the study was about. So that we can talk about more things than just my research but how to move forward from what I have been doing when I tried to understand the experiences of women in open source software. So one of the first papers I wrote about this. The title of the paper paper is motivated and capable but no space for error women's experiences of contributing to open source software. So the methodology was surveys and interviews. I conducted surveys from, you know, with women who are active in open source software so one of the big things with this study is this is of the people of the women who are currently active in open source and not the ones who had quit because that's another group that I was very interested in studying is you know people who leave so this is this is from the people who are successful in open source software. Yes, I am not seeing the name but I see the comment that you're very interested in seeing the research that looks across various types of communities to see if there are stark differences in female representation absolutely. My personal experiences show very few women in networking projects but reasonable representation in cloud native projects okay. Part of it is surely self fulfilling in that more women lead to more inclusively to more women joining exactly yes. But how to get it moving towards a critical mass in the other ones in the ones that are more known for more exclusion absolutely is important. Yes. So your your first line and this first line on this slide is the same thing that you know in this in the surveys and interviews so I had 5657 survey and 11 interviews that I did and all of the people all of the women who I talked to talked about this idea this concept of under representation of women that and the feeling of isolation, and that works two ways so there is. This is this is the in this survey it is self identified, you know it was I put it out as experience of women and there were men also who participated in it. But the results that I'm presenting are self identified a few women in this so I did not sort them through a program this is people who said, you know that that I am a woman and I am an open source and I would like to participate in your site. So the thing about isolation, the feeling of isolation is not just about that I'm the only one but it's also one of the interview is talked about that when I am in a room and I'm the only one, then I am asked to be representative of all women everywhere and you know, this is one woman's perspective that we're going to use to use so there is more need for representation that then a single person then on whose shoulder it comes to represent every woman. So I mean girl communities women need to learn to support each other and recognize there is room for all of us absolutely. And also, along with that Kathy thank you for sharing that along with that I think we also have to very be very clear that isolation is a big word women is a big word there are it's not a homogeneous group right there are nuances differences individual differences societal differences all the diversity that we see at you know general population level is also then can be seen in in these smaller groups. So that has to be a very important point. Yes, and if you're lonely then you struggle and then you feel isolation exactly. Yes, Rupa that's a very good point that you know so the third point here is it talks about that that coding is not the only thing that you can contribute you can contribute in many different ways like documentation ui ux and all those things. Yes, thank you Bianca. Yes, Adam so you know the I, I don't think I have anybody in this project but I have looked at I will share some more projects that have looked at a broader community sense so this is just like I said this was the starting of my looking into the diversity project because the two to 3% really bothered me and then this survey and interviews I did with the very open mind of like, okay what are the experiences how do women themselves see their contributions. What do they think newcomers should be doing what do they think the community should be doing so I'll quickly go through this, you know, a slide and talk about you know one of the things that they talked about was the first experience. So that means that when they were introduced to open source if there was a person, a welcome person or a mentor, who helped them get into it it could be a person they met at a hackathon it could be somebody who organized a conference session on that. So mentors and safe spaces are very important piece somebody to help you with that, but also in safe spaces so safe spaces can be these college based or conference based hackathons or get together of open source communities, and then in online environments safe spaces play a big role. So it was very important that the first experience was critical and of course, like I said these are the people women who are right now in there. If somebody had a broad bad, you know, when finished when I finished this analysis I knew if somebody had a bad experience I didn't talk to them because they're not there anymore so you know that's why going forward I'm looking at that. Another thing that was very important that I honestly did not know before this. Thank you Rupa. Yes, I missed I think she was comments let me just scroll back up a little bit. Yes sure exactly. Exactly. And so one of the things in this research that they talked about you and that's what you know was no space for error but also the extra work that women have to do to justify exactly what you just now said that you have to prove so the example a woman shared was that if I write a patch and I write a whole last list of this is what I know this is why I did it this is why I think work versus if a man post city just says, here's the patch and it will work. So there's this additional work that women have to do and they're motivated to do it and they do it like sure is just now sharing, but there's also, you know this extra work that they have to do. So again, going back to the another key thing for me was before I did this study, I did not think anything about codes of conduct, you know, I didn't that was not on my radar. And in the study repeatedly I had women saying that they look at codes of conduct that they care about what codes of conduct say, and they look for does it talk about gender diversity, does it have ways in which problems occur. How will they be addressed. So that becomes very important, right. Well then there is a question in the question and answer box about with respect to welcoming contributors. Did you ask about whether the experience was better in a face to face interaction or an online interaction. Okay, there was no difference in the group that I studied, I had examples of people who were who met face to face, and who were introduced to communities face to face that this was someone from. I'm not going to say the country but it's someone from a country in Europe, and then there were also online mentors and in this case they said if if they were helped to get into the community if they were introduced to the community. It was good. So a good mentor was good irrespective of where they were. Okay. Thank you. So then when I, you know when I understood this idea of safe spaces, then I thought well this is really important, we really need to know and create and support safe spaces online safe spaces is what I focused on in this research. And so what what my first question was, what exists, right, what is out there that can be saying called like a safe space or women only spaces. Now I will, I will put the disclaimer out there that all these spaces that I studied, which were women focused spaces so they're not women only spaces but they're I started to looking at women only spaces, but all these spaces had very clearly mentioned that anybody who identifies as women or who sees themselves as underrepresented, or even doesn't see them as under represented are welcome to come and share in these spaces so they were not exclusionary in any way, but they were, you know, they were there for underrepresented minorities to discuss things and more often than not there were women discussing things. There were, you know, several examples of other type of things too but the first question that I had was how many of these online safe spaces do exist there and what if they do. What are the types and how many are there and what goes on in these spaces right like if we want to create more safe spaces we need to understand which one works and which one doesn't. So what I did was I did website content analysis of about 355 open source websites. And what I found, essentially was that there is no standardized. And so that's what I was just saying that they were not exclusionary. They were open. So they were not women only they were women focused. Yeah. So, so the first thing was just to get the lay of the land you know what, so there was no standardized space there were some lists serves there were ircs there were blogs. There were pages of women related things there were forums discussion forums forums have been around for a long time so that did catch my attention at this time and I was just doing this lay of the land. What were the types of activities that were happening in these communities in these spaces online spaces. It's a long comment so I'm going to take a minute to finish the slide and then I look at that. One of the things that I found happening in these spaces was things like increasing the visibility of active women. So if there are women then how do we showcase successful women. So that's very important for people to see like, you know we talked about in the diversity slide representation matters if you see somebody who's like you who's there then it is important. So many mentors, again, another very important aspect opportunities for collaboration, I'm working on this why don't you, you know, join me on this or I'm creating this event why don't you join me in this or I am teaching this class why don't you take this class with me. So many events, any events were being organized and they were there educational opportunities, and then, most importantly, for me at least when I was reading them, they became these spaces for social support, right. So social support was what they were sharing with each other. So then I started I wanted to look at it more in depth. So I wanted to look at what kinds of support they're offering each other. How are they helping how are women supporting and helping each other. So, what I did was I did forums content analysis so you know forums going back to about almost 1517 18 years old, and I looked at all the messages in the women focused forums. I looked across five forums and again the details of these studies in terms of methodology and data collection and all that is is in the complete articles. So five forums 10,698 messages from 100 1344 participants. That's what the data set was and I'm looking at it still looking at it from some other perspectives already looked at it in a bigger form of you know okay what are these safe spaces doing. What are the types of activity there how quick, you know which topics generate most activity, which topics are what type of support are women providing each other. These, these forums do work as safe spaces. In these spaces women receive and provide social support, emotional support, informational support and technical support types of support that I identified within the messages. So, what are the other two succeed in the hostile OSS environment by sharing experiences expertise and opportunities. So if they are facing hostility they are able to come here and talk and talk about how you know they can deal with it or even their frustration that this is too much for me and I'm going to quit. I'm not going to continue in this, or I'm going to leave this group and I'm going to go to another group, all those types of discussions were there. In this study, rather than focusing on those experiences I focused on you know how this helped them, you know how the support was being provided. And so in that way I see these women focused spaces as safe spaces which foster gender diversity because women are able to support each other in the toughest of things which could be you know these harassment issues, but also in technical and you know in basic learning of how to do and what to do. Now. Yes, yes, yes, Bianca yes so they tend to hide their identities and gender, and I will share something with you window I did the first study, you know, I'm, I was really that was such a catalyst moment for me when I did that first study just to see what's going on because I was bothered by the two to 3% when I sent out the survey I got a response from a woman. She emailed me and the IRB institutional review board for ethical research, and she said in your survey you're asking what product I contribute to, and which country I'm from, and by putting those two together, anyone can identify me because I'm the only one in my country who's in that product who's also a woman. And the fact that she noticed that realize that emailed me and was worried about backlash just broke my heart. What is her fault, what is she doing she is contributing her intellectual input to this environment. She is giving her time, but she's scared to talk about it she's scared to be identified that that just was, it was too much for me that. And of course I immediately revised my survey and I reached out to her and I said I'm going to change this I'm going to always present my data in, you know, in, in categories and non no identify information will be present. But that's a trigger for me I will always remember that and that drives my research in this area because why should she feel like that she's contributing it, you know, why should she feel that she will face backlash if she talks about her experience of this community. So I'm committed to that person and I'm committed in future to keep doing this to help others see what is going on and we will talk about some of these challenges of you know, they in way forward. So yes, the identity hiding, and another big problem with that identity hiding that happens is that they're not able to claim that expertise, right they can't then because it's not in their own name. So even if they have this expertise that they're learning within their then they, and I'm sorry I missed Jim's comment but it's called too fast on my screen, I'll get back to it when we have a little bit of a break. So there are a couple of questions one the now in the question and answer. The first one is why women workforce face these challenges in the OSS IT industry in the work community. Do you know the reason behind it. Why they face it. Yes, we will look at what are some of the ways in which we can fix it but also they're you know the the challenges that women face or any underrepresented minorities face these type of challenges are based in, you know, oftentimes they are not about you know one of the first things we'll talk about a little bit later is in challenges is, is lack of awareness you know people don't know that other people are being harassed around them, or they don't perceive it as harassment or they're not tuned into it. So they're not helping out in these situations. And so if there is one bully and they're bullying different people in different places that they're able to do that and they're able to really mess up experience for a lot of people. Also, they approve to others that this can keep happening. So you know there are things that need to happen that are not happening to stop this type of things. And so these type of discriminations keep happening because there is no, you know, I'm not saying they're not stopped at all, but we can do better. And that's why these type of harassment, these type of things happen. These are, you know, sometimes these communities are not moderated very well. Sometimes there is just a very heavy presence of a particular group, you know it could be men, it could be men from a particular place, and there is no place for in their minds there is no need for others. So one of the other findings from that first paper was that when you know the capable word in the title came from capable and motivated the capable word came from reporting that I am a programmer but when we go to an event they want me to set up the table. So there is this, you know, subtle gender discrimination thing that is maybe unconscious. But you know there that they're saying well I'll do the demo but why don't you go set up the table and women deal with that. So why is that person doing that. And if they're really doing it to harm the other person but it could be unconscious bias it could be what they're predisposed to again within diversity we also have to understand that people have different perceptions people have different perspectives. And so therefore awareness is a key thing awareness of this is going on awareness of, and I will talk about that a little bit later again come back to it, but you know just the two to 3% number I don't know if it shocked anyone of you but a lot of people are shocked when I say in my presentations that there's only two to 3% women because they say oh no that that that's not true there's there's enough. So it's, I think what we can do about this is create awareness and make people know that this is going on and then hopefully this will happen less. And I'm going to read Jim's comment. The visible enforcement of any yes you see is critical the open source software foundation is going through this right now yes with our mess exactly. People are waiting to see what comes off it. We are yes, and the chat threads of the community range from not a problem to a big problem without actions as you see just becomes a worthless piece of paper. And who will act on violations there's a key indicator to reinforce, we mean it we value you we do not tell it XYZ and I will talk about that in this article that I know what I talked about, you have to say you have to spell spell it out. Gender diversity whatever types of your diversity you are valuing you have to spell it out so that you can say that. The table set of thing yes, so you have to say we support these minorities and this is the type of actions we will take. So, you know, like I said the course of conduct thing was quite new to me but then I started looking at it and in this study which is just a pages copied here the title is quotes of conduct in open source software for warm and fuzzy feelings or equality in the community so is there any teeth in the COC is essentially what I'm saying, and what I found out was not just about the enforcement which is the critical aspect but also the fact that there's not, you know there's not enough. There's not enough community or not all open source communities even have a codes of conduct, and then those of whom have it a lot of them have copy paste so it's not customized, or they are called, you know, like 11 I think in the 28 codes out of open source communities, you found 28 had some kinds of guidelines, and they had 11 different names by which they were called. So some kind of standardization in that is needed and the next slide has, you know, one of the organizations that helping open source community come up with standardized codes of conducts that are customized so customer important they're important but customized are better. You should spell it out your support five minorities. The enforcement like Jim is saying is critical. You have to talk about conflict of interest because in the forum analysis that I did so I combined the forum analysis with the codes of conduct analysis. And the forum analysis I picked up examples of where women were saying that cannot be enforced because somebody in the Community Council is doing this, you know is violating the code of conduct. So when you think about conflict of interest, we have to make the enforcement visible, and we have to include women or underrepresented minorities in the governance, and I should be going faster than this because I'm just only at my study right now. There is the one study which specifically focuses on discrimination and hostility, and it's not published yet, but it has a lot of quotes and examples and in the damage that this kind of misogyny that they face this kind of a vetic versus the other genders this expectation creates. Yes. Yes, so we're going to talk about allies, just after this so I will quickly go through this and go get to where just a few action items just based on the research that I have done. One is create and support safe spaces for women we need more of those spaces and we need them to be better than just the discussion forums and so I myself am engaged in research in where we want to create much better much fun much engaged spaces with much more opportunities to do things. We have created and enforced customized codes of conducts highlighting profiles of successful women and underrepresented minorities facilitating mentoring and networking opportunities, engaging women in OSS governments, those are five action items from different studies that I tried to come up with that I think are, you know, actionable things that people can do. So I'm going to now look at some examples and I'll go quickly through these. These are some examples of current initiatives in OSS space so what are the DI initiatives that are out there and you might be familiar with some of that you might hope you're hopefully also, you know participating in some of that but I'm just showing that to show that you know all that is happening but that doesn't mean nobody is thinking about it right. Yes. Yes. Thank you for sharing that Gina. So there's the Linux Foundation, more will be, you know coming up on that but this is the software developer diversity and inclusion initiative. There's of course on GitHub they have the diversity inclusion and belonging at GitHub and one of the good things that they do is they consistently publish their report on the diversity in GitHub so that's a very good thing something to learn from for them as well. There is the open source diversity which has projects and spaces and programs different things. So if you are not aware of these then here is just quick listing I'll quickly go through them you can look at this one hyper ledger the diversity and inclusion group is, you know, engaging and participating with HBCUs with girls who code black girls who code different kind of organizations so, you know, having the initiatives, plus highlighting what you are doing. And then there's the organization which is about including inclusive naming and these are the participating organizations in that. There is for AI inclusive org for building an artificial intelligence community which is more inclusive to everyone. There's of course outreachy internships which I hope most of you are aware of where they provide internships to work on open source. Places trans hack trans code red hat lesbians who take Django girls Linux chicks chicks, they've been women genome women women in Drupal. So what does that tell us that tells us our ladies yes. And that tells us that there are many initiatives. So I'm going to since we are doing back back and forth questions and going to skip this Jamboard. So what that tells us is that there are many diversity initiatives, and they are there, and they are working and they're making a difference. Now what is it that they face what are the challenges that they face and then how what role do we have right in addressing those challenges so let's let's look at that. One of the challenges that these initiatives, or in general, the D initiative have problem problems with our lack of awareness. Like I said, people are not aware of what is going on lack of allies. So we'll talk more about allies that is a very, very critical aspect of D initiatives. Yes. Thank you Jim that shout out. Yes. Perfect. We'll talk about it. Yes. Lack of positive storytelling so like I said it's very important to give stories of successful women and underrepresented minorities highlight them and project them and present them. Enforcement of quotes of conduct, because of who is running the organization or just, just no interest in the enforcement of it. In the current scenario that we're seeing in different places some organizations just pipe their extrovert, you know, explicit commitment to diversity. These initiatives also like they have this we're committed to this, but then what they do in actions damages the community in a huge way. So we'll look at some of those examples. So lack of awareness, or maybe even denial can be about the status of diversity in OSS that's the first thing that I always deal with whenever I present in academic conferences, my papers. Thank you for sharing that Rupa. So, you know, people always are. No, you're not right it cannot be two to three percent it cannot be that low. Are you really sure that there is. There is that kind of hostility in the community, are they really facing this so there's this, you know, that is really I in my opinion that's just lack of awareness I don't think people are sitting out there saying, somebody is being, you know, treated badly, and that's why we don't want to do anything but it's just that they're not aware. So, increasing the awareness for diversity, increase about the value of diversity like we talked we started the talk with talking to all of us about what what would it mean why should we even care about it. What is the impact on the overall value of a brand and product if you want users for your product then you should value diversity because you don't want your users to be homogenous right. Lack of awareness about the level of hostility discrimination harassment faced. Also, the impact that this hostility has now that is something that I'm working on and I've done you know some research on how tragic it is for women to get you know, rape threats, death threats in these online communities where they're coming to contribute, how it impacts some completely leave that community some completely leave open source some completely leave tech. You know, that's what happens, and people who might be doing it in the moment don't think of the long term impact that it can it can have on women and then overall in the inequality. So let's talk more about the allies concept right so we face lack of you know we face this issue and so it was just pointing that out earlier of you know, lack of allies. For all of you who are here I hope you want to be allies and you we all need to be allies to others and never think about being an ally as something that Oh, I got it and I know what to do, but think of it as as being a journey organization called better allies this is a screenshot from one of their something they shared. So being an ally is a journey understand your privilege. Why am I privileged in this situation. Why is someone else what is the difference that give that is giving me this privilege. What is going on with others. What is happening so you know that lack of awareness well let's flip it and say we will be curious and we will try to understand what's happening. And you don't have to do a huge big thing you can start with a single act. Be okay with making mistakes. Nobody is asking you to be perfect ally in the first go right you you you start with this idea of I want to make a difference I want to learn and I want to if I even make a mistake I want to learn and then I move forward keep learning and keep improving and you know it's not a one time thing it is it has to keep happening. What are the things that allies can do allies can speak the name of people who are from underrepresented groups and they are not around they're not in a meeting, share some opportunities for them, endorse them publicly say you know good job this was good you did very good, invite them to high profile meetings. Sometimes people are just excluded sometimes they're excluded because people who are inviting don't even think about them. And that's part of unconscious bias. So if you are aware of it you can invite them if you are thinking about okay have I missed anybody then you will you know you can be a better ally. Share their career goals with decision makers if you are in a position where somebody's asking you how you know who am I going to promote or you know whatever decisions are being made. If you're aware of somebody's career aspirations share those with the decision makers recommend them for stretch assignments and speaking opportunities give them opportunities to share what they know and and get some visibility. Positive visibility. Another thing is amplifying stories and that's not just about positive stories it's also about negative things so you know what we were doing was mentioning earlier. This one is the first screenshot of a tweet is about Dr. Timnit Gibro who I'm sure most of you are familiar with was fired from Google and this is something someone tweeted after that Sanjana that remember don't be evil if you see something that isn't right speak up. And that's exactly what Dr. Gibro did. But then she was fired and all the people who supported her were also fired. So that is an example of a ruthless decision making by an organization that impact the impacts. The ripple effects the reverberating effects of it are much more higher than what they can see today and how much they're damaging people by doing that. Another thing to think about the flip side of the allies think we want to be allies but we also don't want to be performative allies. So we don't want just to be allies who can claim that they are allies by using certain strategies but are not actually systematically examining processes that are that are oppressing people. So we have to look inwards also that you know we have to make sure there's recent very recent study that I've shared here talks about performative or an ally ship and people are not looking inwards. So you can amplify stories positive or negative as an ally you can raise voice in advocacy and you can work in enforcement of COC anybody who is part of a community sees something. It is if there's a good code of conduct it is not a big thing. It is not a difficult thing to say what you're doing is violating our codes of conducts please don't do it and that gives support to other people. The impact of others seeing it is very important. And this is the RMS issue that like Jim earlier said we're all looking at it. We keep research we did that what they did in 2019 despite despite the well-known toxic city FSF announced on Twitter that RMS is returning to its board of directors and that takes the free software back to dark ages it's really back stepping doing things that are very clearly recognized earlier. So what are the allies doing in this situation this is example of allies amplifying these negative decisions so OSI they gave out their own response. Josh Simmons gave his own response I'm shocked and appalled by the news out of the planet that RMS unknown harasser is returning to the FSF board. So you know that's very important. So what do we do right we see all these challenges what are the ways forward what should we do we know these challenges exist we know diversity is important we know there are issues in things that we are doing right now so what can we do. That's why I call this as a time for disruption it is enough right. It is long overdue as a community open source software. needs to be prepared and ready for disruption and we all can play a part in it. We all I will call upon each one of you to commit to changes. And so you know it is very important to disrupt this to disrupt this cycle that is going on that is not serving people and we can do it so we have you have to think think about it now I'll talk a few more minutes about something else and then I'll get back to you. What can you commit where is your commitment from minimum to maximum what will be your commitment towards diversity. Equity and inclusion in open source software for myself. These are my commitments I'm committed to continuing my research so that I can keep publishing empirical evidence to increase awareness so that nobody can say this is anecdotal evidence right. Provide actionable results for OSS giving recommendations that are not theoretical but that are things that community. People who are community moderators can implement. I also another big aspect of my research that I not sharing here is I educate my own students master's students. In to OSS and you know getting people in who can do all kinds of things not just coding. I am also committed to developing formal mentorship alliances with us as communities for my students and for anybody else who's interested. Yes, thank you Bianca. I am also committed to disseminating my research at diverse venues like this right here I always presented academic conferences, but I think the type of work that I do I need to present it to people who who are most close to it and who can make a difference and so you know the value for me is in diverse and then I'm you know committed to continuing my research and also supporting future development of safe spaces for women of OSS and anybody who's listening to this or is here and is interested I would be very happy to collaborate with you and continue and share what I have you know going on so please do reach out to me. One of the things that is very important is when we say we're going to do something so we come into doing something. Well this this is space that we're talking about it has to be iterative problem solving DI is not a big bang approach you can just do a big thing and be done with it that's not how it works, you have to be consistent in our commitment and we have to continuously improve. So what we want to do for that is if we are committed for this disruption, if you all are committed with me for this disruption, then we have to measure it, we have to measure, is it succeeding or not. And so for that, what we will do is what gets measured gets done. So we will define DI dimensions that you will monitor, you will say this is what I'm interested in measuring. You will select metrics for diagnosing wherever you are in whatever environment you are in select metrics, what kind of representation is there the first step is their diversity, you know, who all is here, what is missing. How do we recruit what is missing, how do we keep people they're very, very important. Look at the pay that is a very good metrics to see how we are treating people. What is the pay structure who is getting what. And then once you identify those things think about how are we going to prioritize them, which is what we're going to work on. Select ROI measures to advocate future investment the return on an investment measure here cannot be about money. Right. So that's why it's very important when you are going to ask for money next year for these initiatives you have to have these documented for the continued advocacy. So your return of investment measures need to be either one of those metrics that you see above, or something else that reflects moving the needle. And that's how you can in future ask for more money. And consistently adapt these measures to changes in the environment so for example right now going forward we will have to deal with the long term impact of COVID on people and their lives. It's a remote work situation, people working from home people's children staying at home all these things as situations are changing we have to, you know, build these in into our metrics of, of diversity and of inclusion. The baseline metrics established responsibility and accountability published the results like I said in the github slide that publishing your diversity report and reaffirming your commitment is very important. You want to become aware first thing the roadmap for this disruption and seems like you all know a lot about diversity but so I think you are already on step three. But if not, then that this is step one for you to be here to become aware, and then you have to create awareness among others because you will need allies you cannot do this alone. So commit to disruption, that's what we do today, and then we implement the disruption, we measure the impact, and then we improve and implement again. So with that, what is your intention, what are you going to commit to, and you can, we will use the Jamboard again the same Jamboard, and I will ask you all to either share your name or not completely up to you but you know, I would like to see what do you commit to. So the Jamboard link I'll share it here and it's the third frame in it, and I'm going to open that up on the Jamboard here. So let's go to the third slide here. And you can again use the post-its, the sticky notes, and share your commitment. I am putting you on spot. And then while you, while that activity goes on, would you like to feel questions in the question and answer box? Yes. Would you like me to read them or can you? Yeah, I can't see that. Okay, now I just open the tab. Okay, so Shilpa asked about sharing the list of such safe spaces that you encourage women to start from. Yes, I can share documents, I can share also online resources and some of the other people in the talk might be able to share that too. There are open source communities which are committed to diversity and which are very good for women or underrepresented minorities to start from. So yes, that resource is available. And I can share it with you if you get in touch with me offline. I will share a list with you. Okay Shilpa. Thank you so much for inspiration about possible solutions. Have you looked at other IP creator spaces with better representation. For instance, the electronic music community is an interesting comparison because it relies on OSS IP to create musical IP. I personally find as in the streets, inspirational. Thank you. It's a very, very good idea. And, you know, that's all it's it's very important for us to learn from what already is going on. There are certain even open source the AO3 archives of their own community which has been very successful in having a robust community. So I think I will, I will note this down and I will look at this particular example also but yes that I mean that is the way forward is to learn from what has worked and what has worked well. Right. While it is natural to so thank you Kyle for that resource, while it's natural to focus on established open source communities have you done any research in early stages. For example, do women start projects at the same rate and they just don't get adoption or is there a gap at the starting phase. Very good question, Alan, and no I haven't looked at start projects started by women but that's a very good, you know, future study thing for me to consider definitely I think I think that would also be very interesting even in like you know I've been looking at some of the venture capitalist projects and start the projects from women are doing good or not so I'm very interested in that statistics also I think that's a great point to look at the project started by women and how they do, right, yeah. Rupa, sometimes women. Shua, were you saying something. No, no, no, no, go ahead. It just said they're sure would like to answer so I thought, oh, I am plating the things. You are answering. Oh, okay, okay, okay. Sometimes women might have their commitment situation change and realize they might have to step back that could be what is happening just try to reach out. Yes, yes, so there is some research about from CMU about you know the women who quit and yes the changing dynamics and that's not actually that's not even just women based early on when I was talking about you know the people who were what were the motivations for people. So for men to like when they were younger and they had their first job and they didn't have families then they were more engaged in open source software so that's definitely an element that is present there. You know that responsibilities change and things change and so people are not always able to commit in the same way, yes. I mean, when I'd like to hear how do you and I works is woven into academia say computer science curriculums, and I'd also be very interested to hear if academia has started a conversation about moving away from exclusionary terminology and moving towards inclusive language. Would a CS 101 talk about masters today still or they are they using main secondary or alternatives. Yes. Not enough. So first, you know quick answer. But, and you know, I think the, the inclusive naming initiative is a great one that I am all when I saw it I already wanted to share it with other people within our organization. There is you know the terminology of how you want to refer to you those are the types of things that universities are right now adopting, but I haven't seen much changes in curriculum and I think that would be very good to see. What would happen again with diversity being in in you know computer science education in curriculums. People bringing alternative alternative opinions there and alternative ways of doing it. So, you know, listening to your question here I commit to sharing that with people in my department and other other other places, who I work with. I think that's the most concept of you know make sure that we are improving our curriculum in terms of terminology, but in general what you're saying you know the, the, the, we're not doing very good with education, right, we're not doing very good with getting women into computer science education. And so one of the projects I'm working on right now in proposal stages this broadening this idea of computer science education to maybe tech education where we include more people and more things. So, I think the concept needs to broaden, but yes thank you for asking that. I also had the experience once that I tried to contribute but they were not going through my pull request answering my questions and comments I had the knowledge to improve what was lacking but it was just not happening so you didn't find the support that you needed yes. That's a good point Jim, I just read your comment to revise all your books I don't like the books thing but sure I use a lot of open resources. I think has a kid me as awareness of open source affected the inclusivity of open source sometimes I find at least in scientific software some of the same power structures are regrettably transferred over to open source, yes, you're right. It should right and I'm, I'm trying to think like hazard change I think it should it should change effect the inclusivity of open source. Just even if it is by the fact that you know more people are able to join in from different places. And, but I mean, I think the crux of that question is in challenging these existing power structures from being replicated like how how do we do that part is the difficult part. Is there data that you can share how the positive impact for having code of conduct and enforcing them. Yeah, so the article that I shared on my slide, the codes of conduct article it just got published in the go back to it. In the software quality journal they had a special issue and open source software. So you can, you can look at this here, this article right here. If you don't find it, just email me I'll send you the article. How often do community projects actually want to be in their project isn't it a natural product push from a project that anyone who wants to contribute can can can anything be done a project doesn't think it's valuable or is that just a fourth target. Well, no, no, I think we need to push back that's why disruption the word disruption right Allen because, yes, there are projects that don't want diversity or they don't think that that is valuable for them so we need to show them how it is valuable for them that's why that awareness point creating allies especially creating allies in places like that we have to show them what the value is. We have to show them how they and their product will benefit. And then, you know, so for type of things is what has been happening. We don't want that now we want to stop it and make it. Yes, it's Vandana at utk.edu. So it's via and DNA at utk.edu. Other projects can be initiated that can create diversity in our communities and the world society for OSS projects as in like what kind of initiatives we can do. I think allies initiatives is a great one. I, I really think we should be working on, you know, the actionable items that I shared so I think the codes of conduct support the safe spaces, I think that's key. And I have personally focused on online, but I think safe spaces as a concept in person, as well as online, I think would be very useful. How to measure contributions by advocates and evangelists that's a that's a good question that's goes back to you know thinking more about the return on investment type of thing right that what are we, what is being invested and what is getting out of it so I think we'll have to look at Bianca will have to look at the literature of, you know, influencers, social media influencers, and how those things are measured, and then kind of move from there and transfer that learning here. Right mentorship in the mentoring initiatives are excellent absolutely and and being aware, one of the papers that I shared earlier also talks about complexities that can happen in mentorship. So no biases that we bring from our, you know, wherever we are from whatever we had our experiences. Sometimes they get transferred into mentorship relationships as well so how to be careful about that. Yes, Julian. Thank you. Yes, is there any other question. Yes, I, I am still guilty of that Lucas of saying guys I correct myself every time but you know, yes. Thanks man thanks man yeah and it's important it's important for us to be aware and it's important for us to fix. Yes, you have I completely understand what you're saying I think I said one time to guys even in this talk but yeah, we have to be aware because it does. And you know sometimes people don't care about it but sometimes we will care about it and so we have to focus on the people who care about it and so we have to use inclusive terminology. Thanks mate yeah. Yeah, yeah so yes, having inclusive language is definitely better. And we all should make an effort to have our language inclusive. Yes. I once Googled you know what are the other ways in which I can say guys, you all just doesn't roll very effectively from my tongue so I you know I was interested in seeing that to folks folks is fine I think right. Yeah folks is fine. Yeah. I think again like I said in that slide with the allies you don't have to be perfect you're making an attempt and then you acknowledge that what I said I didn't mean to say it and so you can fix it right. Yes, I, I am not aware of a cheat sheet Lucas but that might be a good project to start right a good Wikipedia page maybe there. It's a good class project for my students to. Yeah, I didn't get to look at the commitment and so let me go back to that. So wonderful. I'm going to take screenshots of these and presented added to our PowerPoint. Let me just ask, are there any other questions and again you know I would love to hear from you I would love to hear from you on my email it's again when Dana Vandana at utk.edu. And I would, I welcome opportunities for collaboration and discussion and brainstorming. So please get in touch with me. And now I will, I will read some of these for all of us to hear what this is but if anybody has any questions please feel free to ask. I think we have a few minutes right. Five minutes. Yep. Yeah. This is a workshop that I recommend. There are from stage sharp there are the codes of conduct workshops that they do. They are I think very inclusive. You know it's the same. Let me share it again. It's the this this one right here. They have trainings of different types of diversity and inclusion. One to look for. Yeah. Yeah, Jim, exactly and you know I have a, if you will read some of my papers, you will see how many of these are there. I'll actually share one thing with you all. This study. This is the study that has taken me a really long time to do. This is also the study that was most difficult for me to write because you know, nonstop for a month or so I was just reading stories of harassment and misogyny. So you can imagine how that can impact a person to, and I have written this and I have written this, I think in a very good way. Yes, Juliana, it will be shared, and this paper is still not published. It is facing. I don't know who the reviewers are, but it is facing a lot of, you know, people don't want to it to show because this one, like I said, different papers focus on different things and this one just focuses on the misogyny, and the harassment and the discrimination. And that's a tough pill to swallow. And so I'm getting this bounce around I'm working on it, and the reviewer to is asking me to make so many changes that it dilutes the paper, and I'm not there yet and I'm not willing to do that so I might just even publish it you know just open and not get like publication credit because I think it is important because I have really gone through those 11,000 messages and picked up what the problems are and then analyze them in a theoretical way to understand the impact that women have and the consistency of it one of the big things from this was this persistent normalized experiences, you know it happens, and it happens to so many women, and they always, you know, it becomes almost normalized, which is a trauma response normalizing abuse right. Yes, Perry, I will do that. Thank you. Yes. Yeah. Thank you, Jim. Thank you I appreciate it. And you know I am publishing enough from all the other things that I will keep this one. Thank you. Thank you this has been really good I've really enjoyed interacting with you all and I hope you know it. I hope you will stick with the commitment that you made today and also. Yes, thank you for sharing that Steve that's that's that's a very sad reality but it's true. And you know I have met with people, and I have seen this I was thinking like you know I have gone through all these 11,000 messages and seen individual pieces of how things have impacted. And then you know to just minimize it by what percentage was it of the total messages, or, you know, so many were just me to you can't say this was discrimination me to when somebody says me to in response to abusive message or whatever it was instant abusive event, then that is an acknowledgement of further harassment, but they said no you cannot count your me to messages so anyways. Yes, Shilpa. Thank you for sharing that. Thank you Eric. Thank you. I appreciate you all being here. Wonderful. Well, thank you so much bandana for everything and for this wonderful discussion and thank you everyone for participating and joining and posting on Jamboard and with a wonderful webinar just so everyone knows this will be posted to our Linux foundation YouTube later today the recording and we will get the slides from bandana and we will put them up on the Linux foundation website by the end of the day today. So, thank you all very much again. Thank you everyone I really appreciate seeing you all and pleased to get in touch if you're interested in talking more.