 Okay. Hi, good morning. Welcome to this panel of how to strike balance and align your community work with your employer's objective. And I think this is a very needed to be discussed topic. And hopefully we can, you know, make this panel very productive. The goal is to build awareness and then also find solutions for everyone. And I think most of you have heard of, or used the tool, Stabilitix tool. And it's a, did I say it right? Stabilitix tool, right? And this is a tool to show contributions from various employers and then developers. And from that tool, you will notice that it's this 2080 rule. So 80% of the code are actually contributed by 20% of the contributing companies. So this is how OpenState is very different from, like, Linux Foundation open source project. And because OpenState is very much driven by or supported by vendors. And so this is the saying, you know, like, you want to dance like nobody's watching or work like you don't need the money. I'm still working on that. I can dance like nobody's watching, but so far I'm still working towards like working like you don't need the money. And, and so unfortunately, most of us have a boss, have an employer. And the best situation is you can work on something you love to do and get paid. And your employers are very happy with your contribution. But from a employer standpoint, they invest a lot in OpenState because they believe in OpenState. However, they do have their own business objectives they have to meet because they have boards, they have their, you know, executives that they have to, and customers they have to report to. So there, there are some, you know, different perspective in working in the OpenState space. So today, we would like to discuss that. And before we start the panel, I'd like to just to see how many of you are developers and how many of you are managers. Or if you're managers, you don't feel like raising your hand. I understand. Okay, how many of you are developers? Great. And how many of you are managers? Oh, good, good. I'm so happy. Yeah, both, or both, right? I mean, both. So, okay, so that's great. And so we're gonna start this panel. And I'm very honored to be here on stage with all these esteemed panelists. And I'm sure these guys, you guys probably know them already. They don't need introduction. But still, I'm gonna, because we're recording, so I'm still gonna have them introduce themselves. And let's start with Arkady. Good morning, everybody. My name is Arkady Kanesky, and I am from Dell. And I am the Dell Gold Member for the Foundation Board and also in charge of the Dell OpenState solution. My name is Mike. I primarily have been focusing in block storage area. Some of you might have heard of a thing called CIF. And so working on that initially before another project called Cinder that before it even existed, and Nova Volume worked on that driver. But on top of that, it was previously PTL for the Cinder for Cinder and the Liberty and Kilo release. And then I am currently a TC member at this point. And I work at the OpenStack Foundation focusing on cross-project initiatives to help with all those consistency woes. I'm John Callahan. I'm with Hewlett Packard Enterprise. My primary role with OpenStack is the application ecosystem working group and also the enterprise working group. So that's my primary focus this time. I'm Rochelle Grover. Most of you guys, if you know me, you know me as Rocky. I'm with Huawei, R&D Labs in Santa Clara. And my job is pretty much undefined other than provide value to the company with regards to OpenStack and help them get more value out of it. I'm Monty Taylor. You may know me on IRC as Mordred. And if you don't, you should get on IRC. I've been, I've worked on the OpenStack infrastructure program and other things set on the TC and the board and work for IBM where I lead a team called the OpenStack Innovation. Team Innovation, OpenStack Innovation, something, I don't know. We innovate, innovate all the things all the time. I believe your bio also included of creating the initial Launchpad OpenStack. Oh, yeah, yeah. So if you, I guess in the grand scheme of everybody claiming to be a founder of OpenStack, I do at least have one. If you go to launchpad.net slash OpenStack, it actually does say created by Monty Taylor. So Monty Taylor created OpenStack. Yeah, I think these guys are very humble. Actually, you know, they actually have been very successful contributing to the OpenStack community. We have TC here. We have PTL. We have board members here. So, you know, they definitely have been doing contributing a lot to the community. And they also, they are very successful doing their own job for their own company. And my name is Annie Lai. I work for Huawei IT product line strategy and business development. And I used to be a developer. Now I focus more on the business side. And in the OpenStack community, I'm currently representing Huawei School membership. I'm a board member as well. And also, I participate in various work groups. So without further ado, Arkady actually has prepared a brief, a small presentation for us just to kind of set the stage before our discussion today. So it's going to be about five minutes. And then he's going to talk about the various different objectives from the developers and then employers when it comes to contributing to OpenStack. Thank you. So first and foremost, I would like to emphasize that, you know, the whole topic of this discussion, thanks for any organizing that is that there is kind of a goals and objectives for us as a community creating the open source platform for the cloud. But it does not mean that every company who's participating in that has the same goal. And that's expected. If they're all having the same goals, you know, you will not be successful because everybody have to, you know, be identical. There is very little differentiation. But the goal of the OpenStack itself as, you know, foundation is driving that is that, you know, we wanted to provide stable APIs. We wanted to provide stable implementation, which you can use. And you can, you know, accelerate of your development of the cloud and developing the application and the value proposition on top of that. So, you know, there is a fairly remarkable recognition when you are a member of the OpenStack and especially if you, your contribution are highly valid. Both as an individual contributor as well as a representative of your company. But from the company perspective, I mean, you know, they're not altruistic. They have, you know, they have to one way or another show a return investment, whatever that means. So they need to, you know, balance it out, you know, what the resources they contribute. And when I say contribute resources, this is basically time of all of us working upstream in the community to improving the OpenStack itself. But at the end of the day, you know, they need to show some results coming out of that. And it's usually in kind of a variety of the forms, depending upon what your company, you know, goals are, you know, it could be that, you know, your hardware could be integrated batteries OpenStack. It could be that you develop some application type of that to make, you know, to make the delivery of what software you're developing faster and easier in a variety of other different ways. But the whole purpose of why the people go to the OpenStack, so they have, you know, tremendous speed to market. They have a recognition for the customers because customers already know what OpenStack is. So it's much easier for them to absorb it when you have, you know, your call it solutions, you know, interacting or properly integrated with OpenStack. So the thing I would like to kind of stress that in most the hardest part, which usually don't involve any of us, but, you know, involve what are corporate strategies is to define how the company is going to benefit from participating in OpenStack. What is it? What is the thing that they will develop either upstream as part of the OpenStack or add on which differentiate them in some form or shape? And what are you going to, you know, be able to deliver? Again, the fundamental thing is, is, you know, all of the products are, you have to face, you know, concentrate on what the customer requirements are and make sure you can meet them. You know, OpenStack have no sure to show the things which customer are required, but not capable of delivering yet. So there is a lot of what I would call short term opportunities for the companies to benefit from. But, you know, as everybody expects, sooner or later those holes will be filled, thank God, so not, you know, you don't have to have your own specializing to deal with that. So you have to have a value proposition which will, you know, have a longer, longer term. Now, with respect to you, as individual contributor to OpenStack, that's a very delicate balancing act. First and foremost, if you do not believe in your corporate strategy of how to utilize OpenStack, tough. There is no simple way of saying it because basically what you are saying is that while you love to do the work you're doing and contributing to the OpenStack and making it much better, you don't see how the current strategy which your company has really working. And depending upon your company culture, you might be able to provide the feedback to that or raise your voice, but that's, you know, from company to company specific things. A lot of the people are, you know, willing to do stuff on their own accord. And that's great. There are individual contributors that don't associate with the company. And you can certainly do that even when you work for the company. But do mindful that one way or another you still represent, you know, you still work for the company and you did sign some legal agreement when you joined the company. So be careful. Okay. So on top of what you know, you contributing your time, which you know, you have to balance between the time you have for the, you know, work for the company versus if you decided to freelance this OpenStack. Be mindful that, you know, we are, you know, your activities in the social environment, which is OpenStack one way or another reflects, you know, the company for which you're working because you're representing them. As any point out, the Stackalytic kind of gives you all sorts of information of how individual contributors in the company contributes to. But, you know, it kind of shows some social stuff, but a lot more shown not on the Stackalytics, but in all of the Twitter and everything else, which is happening around. So that's kind of my main thing, because be aware and be mindful, you know, to don't overstep the legal boundaries, which are between you and your employer. I'd like to add a thing there too, because there's a there's a really interesting, I wind up talking about terms of political science, more than tech, a lot of when talking about OpenStack these days. But it's worth pointing out that as much as that's absolutely right, there's a there's another option as well, which is if you if you don't agree with the one of the nice things about the about empowering developers is that in this particular case, if you if you are working for an employer, and you don't agree with with what they're doing, there's a bunch of companies here that have different strategies. There's a bunch of different strategies, and it's you can you can deal with this in a couple ways, you can just work for your your employer and do things on the in the evenings or weekends, if you don't agree, you can also move somewhere else, right, you can you can find the employer that you do agree with their strategy, right? And you do you do it does resonate with you so that so that that some of these balancing act problems and some of these, these having to having to be careful, you have to be less careful, like you can honestly earnestly of your own free will, say, yeah, no, I, I'm gonna, I'm gonna, I am gonna go do that, not just because somebody told me to, but because I think it's a good idea. It's it's like you sort of voting with your your feet as it is. And so I think that's always worth pointing out that you have that and at the moment, there are no shortage of companies who need people doing open stack things. But I'd also like to add that I would challenge you to be the change agent in your company. Now, there are some things that that you can't deal with. And if there's an ethics problem, you know, get the hell out. But be the change agent and don't be afraid to go for it to be disruptive because like Monty said, somebody else will hire you in a New York second. Yeah, I like to add to that. I mean, we, we, we work in the open source community. And we're like, hey, a lot of times I tell people open source developers are like artists, right? You're idealistic. You want to you have this dream and you really want to make it happen. So it's important to be true to yourself. Sometimes it's like sometimes just a paycheck. I understand it's important to have that paycheck. But if you cannot be true to yourself, then you can feel very painful to keep working in that environment. Something that I once heard from the open stack release manager, Doug Hellman, at one point, he was he was basically linking over Twitter, like to this article, I'm not going to name the company that just basically, you know, gave up on open stack was invested so much money. But the one thing that he took from that article was how much did they actually invest in their employees to allow the company to be successful. And I think when you look at all like the super users, and you talk to those talk to the even those large companies, it's amazing that how happy that their contributors are, their employees are, I mean, being around in the community for a while, I mean, listening to people, I mean, it's black and white there. I'd like to just say, in my case, I'm part of the non coding groups and a large part of what I'm doing back into my organization is really kind of informing them of the fact that we do have this action that's going on, we're recruiting a lot of the new developers to open stack. And it's a matter of getting some of our development people involved and bringing out some of their experience in being able to attract some of the new developers and providing them with the tools that they need so that they can get involved. And so as I've just recently changed management, the person that I was working with previously was had a lot of open stack experience. And two weeks ago, I found out that he was going to be moving to another division. I have a brand new manager and all of a sudden he's going about learning more about open stack and so he's supportive, but I have to inform him now what this is all about. So it's a little, it gets to be a little bit more of an educational format back to the organization that you're working within. You just really never know where you are sometimes. And as I think you educate them, they see the benefit. But back into the company and we can also benefit from the company to the open stack group. So it's what has worked well for me. Real quick, non sequitur. Guess which ones up here are the developers? What are you saying that you're saying that there's the tell by looking about what you're wearing? But here do I don't claim to be a developer. Okay, so I want to just add a couple of small things. So as mighty point out, you know, there is no shortage of companies who, you know, are doing the development of open stack and finding the company, you know, whom you which matches your desire of the outcome of the open stack is not that hard. So the other thing I want to do mention is that, you know, within your own company, find who is that person who is driving the strategy and who is putting together how the, you know, the investment and how all of the contribution which you are all doing for the company and open stack really resonates because providing the more feedback to that person making his job a little bit easier will benefit you directly. So the other thing I want to point out, well, you know, companies are kind of top down entity. The open stack is bottom up entity by as all of the open source communities are and innovation comes from the bottom, not from the top. So if you have an idea, don't be shy to bring it forward. And it will be surprising to you how it might benefit your own company, but you bring that idea forward. Yeah, that's actually I think the root at some of the some of the problems and some of, you know, why we're sort of on this topic is, you know, companies are very used to a top down, top down command and control structure and saying, oh, well, it's really this product feature we've planned for our, you know, for our big, you know, corporate event, then we're gonna, we're gonna make an announcement and we've got to deliver it. And so we're gonna ask everybody to, you know, pull all nighters and which by the way, if any of you are managers, that's a counterproductive strategy. All of the research basically says that. So if you ever have the urge to say, just work a little harder. Can you guys work a weekend? It's it's you're being a bad manager. But in any case, this is how this is how stuff is people are used to working. And that doesn't is very unaffective with with open stack to have somebody come in and say, well, my boss told me that I have to get this patch landed today. And we say, well, the patch doesn't work. And so, well, but you know, I have to get it landed because it's my business objective. And I'm like, I, I don't care. It doesn't work. And that, that's a, that's a certainly a friction point. And, and it involves, involves some, some, some communication and education that has nothing to do with, with the technology itself in both directions, right? Getting, communicating back up the chain, you know, talking to the strategist and getting people to understand that, that predicting, predicting when a feature is going to land is, is pretty impossible. Predicting that a bunch of features are going to land is a, is a guaranteed that is definitely going to happen. So, so figuring out how the, the corporate strategy and marketing departments and everything can, can ride that wave and, and, and, and use that as a value rather than a thing they're working against is, is a way that, that all of us that are on, in the trenches on the developer side can, can be really useful to the, to the company to remind people that, well, maybe you shouldn't announce four months ahead of time that you're going to release something because you have no way of promising that. But you can, you can, you can change your marketing strategy. You can change how you think about communicating with your customers about the value that they are going to receive in, in four months. We all know that in six months, from now, there is going to be another release of OpenStack and it's going to have a bunch more stuff in it. What that stuff is? No idea. I just want to say as a, being a former PTO, I've been pulled into very shady, uh, random hotel meeting rooms, uh, where there is lots of drinks and food being provided and, hey, we just want to have a quick conversation with you about our agenda. It doesn't work, but thanks for the beer. So it's actually a very good, um, conversation here because, you know, a lot of us where we are kind of, we have this dilemma. We want to create and then we want to follow the foundations, you know, policies and all that. And then also at the same time, your employers have, um, you know, expectations. So what happens if, you know, your project, the project that your employees are very interested in, are not accepted by, for example, Big 10, right? What do you do? I mean, maybe some of the panelists can share with us, you know, the strategy to deal with that kind of situation so you don't get fired if you don't do that job. By the same time, you can effectively kind of influence your employers to, you know, to act correctly. There's a, so there's a definition of, uh, of suffering, uh, which is that, that suffering is the, is the divergence between, between your expected outcome and, and reality. And that way to reduce suffering in, in your life is to, is to learn to not, uh, not, uh, uh, pre-expect what's going to happen today, but to in fact live, live in the moment and, and learn to, to treat reality as it actually is rather than how you, how you wish that it were. Um, so I, so I think that actually some of what I'm suggesting is that you have deep religious conversations with the leaders at your, at your companies to get them to, uh, to, to, to, uh, embrace, uh, trying to reduce their own, uh, suffering. Namaste. Okay. Also along those lines, um, at least some companies right now OpenStack is, uh, like the center of buzzword bingo. OpenStack and containers and, and the networking side we all know about NFV of various flavors. And I think one of the things that if your employer really wants something in OpenStack that's open source that they want to be proud of and kind of put their stamp on, educate them that everything doesn't have to have an OpenStack stamp. If you open source it and build your, build a community around that, that can be as good sometimes even better for the company. I'd like to just point out Walmart's OneOps is an amazing open source, uh, software package. And there are others out there that companies have put out there where they have built software around OpenStack, but they haven't felt the need to actually get the OpenStack, Big, OpenStack Big 10. So let them know it's an option and let them know that OpenSource in general is really good. It doesn't have to be OpenStack. I think with regard to Hewlett Packard Enterprise we're in a constant state of development and we're seeing all sorts of changes going on in the future. We have acquisitions that come and support OpenStack directly. We have OpenSource support throughout the organization. And we have the Hewlett and OpenStack initiatives that we've been supporting for some time with the releases that we have that are pure OpenStack support. I think that there will be a lot of changes that go forward as we ebb and flow and change with our, with the acquisitions and how they're positioned in the future. As far as OpenStack and the support I previously was at Walmart and saw exactly how that was used, how OpenStack was supported. And it's different for every company and it's different for every time and as live for the moment as Monty was talking about it. I think you really have to live for the moment and realize that your organization is going to have different requirements at different times and you have to you have to make the right combination of that software that's supporting what you need to be doing, driving forward based on what your networking situations are and what your apps are calling out for as well at the same time. So that's my take on it. The thing that I just popped into my head and I'll apologize that I didn't talk a lot is there's another thing to communicate up and out as you chat with people which is when we talk about investment when we talk about the investment that the company is going to make to deliver on their goals it's really important for your employers, your bosses, your managers to understand that there's a lead time. There's an investment that is going to have to be made and it's just a fact of life. Like you can argue that it shouldn't be this way but it is. So just accepting it and moving on is really the best thing. You're going to like bringing in new developers who haven't been working on anything to get product features done. Expect I mean if they're a rock star maybe in six months they might make enough progress in the community to be able to start to move their own agendas. It's probably going to be more like nine to 12 months. So you've got to be you've got to be thinking about long-term planning in terms of giving people the time and space to get plugged in enough so that having conversations about oh well my employer has this this special thing like it's really important to us that you know that Nova can be purple you know and you're like okay that I don't even know what that means that makes no sense but we can talk about it. When if you walk in first day and you're like dude I really need purple VMs we're gonna be like you're crazy and we don't want to talk to you ever again because that really doesn't make sense. And a really good way to do that and this is this is another important thing for communication is step one is step one for new people is is coming in and working on other people's problems right? It's it's coming in and helping with there are as we said earlier there are plenty of problems already we're already working on things right? What we don't need is new problems that's that's not really particularly welcome right now because there's things that we've been working on for three or four or five years that we need to finish off so the new as companies have come in the ones that have been the most successful in eventually achieving their business goals are the ones that started from a place of showing up and saying how can I help? What are you working on now that I can help with? And then they work on those things for for a period of time then eventually they're like great oh well I've also got this thing and then you're like because you've become a part of the team now and so now adding your voice and your requirements to the to the mix is welcome right? Because and that's a thing I think a lot of a lot of companies miss sometimes they get really they get really eager and they get really excited which is great be excited but like temper that excitement with some expectations of you know there's 2500 developers here they're all working all the time right now they need help on on their existing on clearing their existing things before they can think about new work you know and that's that's a that's another thing and along those same lines it's good to remind your employer that when they first hired you did they expect you to hit the ground running and be immediately productive on their huge monolithic code base most employers actually a lot of them have like three month probation periods remind them that it's the same way in the open source community you have to build trust with the people you're working with so you have to start by learning the code base and attempting to make positive changes in it just like you would inside the company but that the visibility is to the outside and also because of that you need to be more visible to your manager to let them know how you're building and moving forward in this process but remind them that it doesn't work that way in the company it shouldn't work that way in open source okay um yeah um we we spent a lot of time you know helping out developers and I want to make sure that this panel is fair so for the remaining of the time I'd like to help out managers because I do believe that these old sponsoring companies are they truly believe in OpenStack otherwise they have will not have spent a lot of money and resources in developing OpenStack and but however there's also a challenge for them because a lot of company structures KPIs they were not built around open source model so I want to kind of seek you know the panelists and then maybe some from audience that how we can advise our managers to kind of modify their KPI for employees in order to you know to help out to be fair and then really be productive for the development of OpenStack and for the benefit of their own company I have a really specific a couple small and it's not quite on the on the KPI's thing but I hopefully gets there advice for the for the managers in the room and I've had I've worked for more than one person in my time with the OpenStack and this has gone better or worse depending on who the person is I highly recommend if you as a manager of people who are working on OpenStack are not on IRC if you have not learned that do it right don't it's not it's not that hard it's it's a it's a chat system so so if honestly if you can't figure out being on IRC then then maybe finding a shift into the marketing org or something might be a and we but also helped improve for onboarding for IRC too there is like pictures and step by step for Windows Mac and Linux so we've got that but it's really that is where OpenStack happens and if you don't if that if that seems like a scary like a scary world often like it's like the dark place where the you know where there's like demons and what not going then then it's always gonna then then developing the developing KPIs developing ways to assess what people are doing is is gonna seem very opaque right and all of all of this happens in the open it it just may not happen in the way that you're you're used to consuming you can log on a really easy one that that people that some people like IRC cloud is a web-based persistent connection so you can it sort of feels like like Slack or hip chat in in those sorts of ways get yourself an IRC cloud account the channels that your employees are in and learn how those employees are are chatting with each other but also how they're chatting with with their teammates that don't work for your company because it that's the other thing that's missed a lot of times in tracking tracking this work is that your employees aren't the only ones working on your issues right and and so that your employees team isn't just the the people who also work for for you your employees team encompasses people all over the world at many different companies and that's that is just the reality that is you can get to know those other employees too and not from me I want to try and recruit you to work for me turns out they're free employees for you right there there are people that you aren't having to pay out of your budget and you get to know them collegially they're not going to do what you tell them to do but you can you can get you can learn to interact with them as humans right and and then there's a lot of tasks actually in the opens that community that you could pitch in on bug triage is a thing that a manager is completely able to to do in the and it helps to make your employees more effective at at doing their stuff so there's all sorts of ways that you can become involved that aren't oh start going coding nah you're a manager you don't do that anymore you know clearly and everything but um wait you have a point ahead and and you still code and you still code and you still code and you still code and you still code and you still code and you still code and you still code and you still code but yeah I I strongly recommend learning learning how to learning how to read the mailing list learning how to to to get on IRC makes a huge step forward in becoming comfortable in in in figuring out what's going on with with that that crazy upstream stuff so you're not asking your employee to give you a readout every day on what happened you can follow it yourself to add on to what Monty was saying so as I actually was a former VP at some point believe it or not of of a really big web hosting company so um the fact that you view like that there are demons there be in IRC is actually just you fearing the truth of where your project is going that's exactly how I felt as being in that position at one point so but I also I know like Stack Alytics was brought up on here and I actually want to echo Allison Randall specifically on the she's part of the open stack board of directors and you know she brings up the fact that in this goes with also with the discussions that have been you know being discussed on the mailing list for for development is Stack Alytics is being gamed so just just right because the it's all it's all open source people know how to what we're measuring on people know how to submit up you know little patches that here and there and get their numbers up so I just wanted to add that onto with Monti's point that you can't completely rely on that so that's why you need to be more involved all right yeah can we stick to KPI issues because I really want to find I really want to get some kind of answers actually okay that's um so this is this is this is managers wanting things out of out of out of open source caps who are going to do something else and we're role playing here but first things first Monti said that you can bug triage well you can also code review how many managers here have been developers before they became managers you guys know code you guys know architecture you can jump in on discussions of how to do how to implement something what the design is there there are specifications that are out there if you don't like the direction of the design and the implementation jump in and participate it's a lot easier than actually writing code for open open stack but for kpis I think managers often get caught between a rock in a hard place in that even if you understand what your developers are doing and are fully bought into open source the problem is that you have to educate your managers about things like if you have a product that includes this open source you have to allow your developers to upstream bug fixes and other other issues this has to be part of your kpi why because if you don't upstream your code base diverges your code base diverges you have support issues you have to have more support people on your team it costs you more money not to upstream and this is what the managers have to point out to their managers that not allowing the time it takes to get the code in the code base is costing your company lots of money the pki should be is the is the company code base converging with the open source code base if you can prove a trend of convergence that means your support costs go down and you're doing a good job these are the kind of measurements that should be in your kpi it's a really good and go ahead I've been talking a lot so I just want to add a couple of small things so first of all as a manager I mean you hired the people who you know either know how to do open source work already or you know interested in learning how to do it trust your workers set up what their expectations are not you know in the line of codes or in any of the other measures which are frankly completely irrelevant to the end goal what you're trying to achieve in the solution but then something which actually you know was measuring and is properly pointed out stack analytics is not one of that measures company creating their own project within the open stack and bragging about it well that's irrelevant the real question is what is the value to the customer and what does the customer want fixing the bug which have been sitting around for you know a year which customer would like to see taken care of is substantially more productive again you know as a manager and yes you're more than welcome to participate in upstream activities and it's actually very rewarding personally however trust the people who work for you they know better what need to be done to make things happen in the community that's right on I think also the the trending looking at trends is a really good thing especially as opposed to raw raw numbers if you've got you know bugs and or features that you're that you're interested in and you're looking at individual employee performance getting upset at an employee because you assigned them a feature and it didn't land in an open stack release is just insane right like because they they do not have control over that however if as long as you're giving them enough time to to actually participate and and I believe that the the theories recent stats are those of us that are they're sort of actually like in the regular contributor camp are are doing at at least in the in the patch a week thing I would say that really patch day is probably closer but at least if you're if you're not at least in in the in the patch per week then you're you're you're falling off the the the the the curve thing however that's not um when somebody first starts off that's that's going to be very different than then after they've spent six months after they spend a year so over time you should actually start to see your employees have a have a better track record of of of fixing bugs and landing features I don't have any problems in in in fixing bugs or landing features are getting my patches merged in in open stack but I've been around for six years right I've I've built up a lot of things so the one thing where I would say that looking at the trend over time are they getting better at at at getting their their goals landed if they're not then one of two things has happened either you're not giving them enough bandwidth to interface with people and you should be really honest with yourself about about whether you have actually given them enough time to to work on things but the other one is is that some people are not effective at working in the community and that's just to harsh reality a lot of this open source stuff has to do with interpersonal relationships it has to do with soft skills in addition to hard skills just being an amazing programmer doesn't matter if you can't handle a code review and respond to it right some people just dig in somebody tells them something's wrong in their code and they they defend their code rather than taking the taking the the feedback and and moving on with it so that is a thing to track and I'm I'm not just going to sit here and say you should let your developers do whatever they want to do some of them may not wind up being effective but but tracking that in in in short term like single numbers is is unfair to to anybody and it's not it's unreasonable but but you should if you're giving them enough time see over time an increase in their ability to to react to to participate to to be able to to to to show accomplishments that that you could enumerate and so I think really important I think we are running out of time I wish I had more time but I'm so like I think it was a great panel let's give a round of applause to our panelists and and they'll be around here right and so if you have any questions I apologize we didn't have time for you to ask questions so feel free to ask them grab them ask them questions after the session