 All right, so let's go ahead and get started, folks. Thank you all for coming. This panel discussion is on, altogether now, open source project, cross project collaboration. It's a discussion that's been coming up relatively frequently, particularly more recently, with a variety of projects needing things from other projects. We really hope that this is a true discussion amongst the audience and the audience and the panelists. We really want to hear your thoughts, your experiences in this area. We're certainly going to share some of the experiences in the projects that are represented up here through the panel. My name's Phil Robb. I help to manage the Open Daylight community. And again, we've been working with a variety of projects around the SDN space. And a good piece of that also ties up into OpenStack, hence why it's relative here. But I want to start off with each of the panelists kind of introducing themselves, obviously, describing what project they work on, as well as what has worked for them in doing cross project collaboration, meaning that what have other projects asked these guys to do? What have they asked other projects to do? What's that interface looked like? What's worked well there? And what could use improvement? Again, as we continue to build out deeper and richer open source ecosystems that bring together multiple pieces of open source and multiple projects, getting to do this better is just goodness for everybody. So with that, Justin, would you go ahead and start? Sure. So I'm Justin Pettit. I work on the OpenV switch and OVN projects. OVN is a new project that we started in, or we announced in January and started coding to provide virtual networking for OVS. And I think most people are probably familiar with OpenV switch. Historically, we're sort of at the bottom of the stack on OpenV switch. And so we get some requests coming in, especially around documentation. There's been a lot of complaints about how complex OVS is. So that's where a lot of the requests have been coming for us, for when we mostly need to work with the only people who are below us, which is the kernel. So occasionally, we need to deal with the kernel to ask for features there. But now with the OVN project, we'll be dealing a lot more with other communities, because we'll need to integrate with people like Open Daylight and OpenStack. Great. Thanks. Colin? So I'm Colin Dixon. I'm the chair of the Technical Steering Committee for Open Daylight, which basically means that I have no power and a lot of responsibility, which is a lot of fun. And I'm also the person who Chris Price comes and yells at when it is that we're not delivering things to OPNFV. So we interact with pretty much every open source project in this space. I think we haven't actually started integrating with OVN yet, but my guess is that if Kyle ever gets four seconds to code again, that will be the first four seconds of code that he writes. But we interact with OVS, and in general, we actually haven't had very many requests of OVS. It's mostly just sort of worked pretty well. But I think OVS is a pretty stable project at this point in time. Unfortunately, we had pretty good experts. But we sort of work with OVS, OPNFV, OpenStack, and just pretty much everything else in between. And my experience about what's worked well is when there are actual humans that sit in both camps that have cycles to write code. And what works badly is everything but that. So having Kyle Mestri sort of interact with Open Daylight at OpenStack has helped a lot with the neutron integration and other things that have been harder than language barriers. We can sort of get into that later. But yeah. Very good. Chris? I'm Chris Price. I work with the OPNFV project. We're a relatively immature project. Well, not immature. We're a new project, is a different word. Slightly immature as well. No, we are basically a consuming project. We are an open source collaborative activity that doesn't intend to write any open source software. We are a midstream activity. So we consume and we build out. We are targeting the NFV technology domain. So we're trying to essentially establish a platform that can be used in the NFV domain using OpenStack, Open Daylight, OVS, Linux, KVM, you name it. Generally, what we hope to provide is a stack that serves the telecommunications market. For us, collaboration upstream is extremely important. And just to echo Colin, the best collaborations we've had is when our devs have sat down with other devs and we've just solved problems. Collaborations that, well, still we're very young. I can say, we're finding our feet. That's my introduction. I'll hand over to Dave. Hi, I'm Dave Lenro. And I'm sitting in on this panel for Kyle, who was going to play the role of the deep OpenStack expert. In front of a lot of audiences, I would try to fake it and pretend to be that guy, but this is not the place to try to fake it. The reason I'm relevant on this panel is because I think I'm one of the few people in the industry whose really full-time job is to try to work across a large number of different open source projects and standards organizations and try to herd things in the same general direction. And while not necessarily deeply involved in the coding piece of it, I'd echo what Colin and Chris said, which is that you need to have the same people in all of the conversations. At some points, you end up with completely different conversations in every organization. And the chances of the pieces looking similar and coming together are pretty near zero. So I think we've got to somehow transition from a history where we were siloed and that made things more efficient and focused and whatever. And it was good to kind of ignore other projects. We've got to get into this mentality of paying attention to other projects and talking to them and showing up. Very good, very good. And again, so I want to make sure that everybody is totally open. If you guys have questions at all, please just come stand up at the mic. And I will constantly be deferring to folks in the audience for any questions or comments that you want to make in this domain. And if I continue to get nobody standing up there or raising their hand, then I will continue to pelt questions. I defer the floor to Margaret. Dave said something that got me. OK, so I understand the value proposition of trying to have the same people going to all these things. The problem is there's a lot of things. So you have open contrail, you have onus, you have open daylight, you have DBDK, you have OVN, you have, I mean, you can't go to all these things. And in the networking industry, we're getting, I think, so fragmented. And as a consumer of this, I guess, I'm actually, for those who don't know me, I'm Margaret Kiyos from AT&T. It's daunting. It is really daunting. And so since we, as a consumer of this, can't go to all these things, we end up going online and looking at ether pads, looking at things like that. But as everyone knows, what you see in writing and what's really happening is very different. So it's very daunting. So that's why I really, I don't know how. But I mean, what's the suggestion of all these factions and trying to get them either to have less factions and working together with a different faction or, I don't know, getting them to really work together? Well, I think clearly we can't have a huge number of individuals who all go to 47 different open source projects and participate heavily. But if instead of having thousands of people who only participate in one, everybody participated in, say, two, for example, you could end up with permutations and combinations such that you did have kind of a coherent thread between the entire community across the communities. The networking guy is suggesting a mesh across the. Full mesh. So I mean, this is gonna be another networking suggestion, but you know, these projects fit into a layering. And if you are mostly concerned with consuming one at the top layer, you can, you don't have to necessarily build relationships the whole way down to the bottom. I mean, right, I mean, right, you don't have to, I mean, if you're consuming open daylight to get network virtualization, you don't have to talk to necessarily to Justin and OBS because we're maintaining that relationship or if we're not, it's sort of, you know, our problem and you can ask us. And then similarly, you don't have to worry about the kernel so much if, you know, then Justin's gonna take care of that relationship. And so when it's that kind of path, for instance, say, OPNFV to open daylight and OBS and the Linux kernel and all the way down, that seems like it's, that's how you scale, you have hierarchy, right? You try and get to the point where you actually care about it. You interact at that point and maybe one or two points below where it matters. But it certainly is still a considerable amount of time. And I think that that's part of the trade-off which we're seeing in terms of engaging with open source projects. If you don't want to give more time to understanding open source projects work, you're unlikely to get more value out of the open source projects than you would have gotten out of a vendor product. It's not a free lunch. It's sort of a new spectrum of trade-offs that you can make as you acquire technology. And Chris being the representative of OPNFV and particularly those various components, what are the things that your organization is trying to do to build those bridges with those organizations? Are you taking that hierarchical approach? Do you feel you're able to take that hierarchical approach or do you feel as though you need to actually interface with those groups independently now and possibly help them build bridges amongst each other? I mean, what are your thoughts and what's OPNFV been trying to do there as a young project? Yeah, I think Colin has a good point from a what do I want to get from this perspective? You can go to a certain level of distraction or a certain level of depth to find the level that you want to get things from. As OPNFV is a project, we're a little more operational, so we're actually running and trying to deploy. We have projects like V-switch performance evaluation projects which are going to essentially hammer V-switching and try and figure out how to improve it. If you come to OPNFV and you're interested in V-switching, that's a great thing to do, but it doesn't necessarily solve the problem for Margaret and others to come in and really understand again what is OPNFV doing? Well, V-switching is an important component of the platform. Having high performance is an important aspect of what we want to deliver as part of the platform, so we kind of have to dig in. And in order to achieve that, we need actually a broad spectrum of individuals in the project. It's, we have engineers that are here at OpenStack working with the OpenStack community and focused on how we can get features and functions and how we can best leverage OpenStack in what we're trying to do. We have people that will work with the V-switching team to try and achieve the same thing. And we try and create a holistic view, I guess to some extent, at least for targeting the NFV space. We're not trying to boil the ocean and solve everything for everyone, but from an NFV perspective, we kind of know what workloads we want to carry. We kind of know what types of deployments we have. So it's a little easier for us to then sort of go down into the stack because we know what it is we want from the different levels of the stack. But then, if you start to look at Enterprise, then don't come to OpenNV. Well, no, I shouldn't say that. You could always come to OpenNV, but it may be that the expertise or the type of workload analysis you're looking for is not what we're focused on. But really, as we mentioned before, it's about getting engineers on the ground. We may think, oh, we need better performance from OVS. Can we send them a mail and ask for it? That doesn't really help. So we start a performance activity. We start to evaluate it. We start to say, okay, these are where we're seeing problems, and then we'll have engineers engaging with the OVS team and saying, we're having problems here. We really need to figure out a solution. And that's the way we're gonna approach everything. We wanna figure out what the best way to do stuff in OpenStack is. So when I come to OpenStack and say, hey, can you write this code for us? We come and we say, okay, we have a problem. Can you help us find the solution? And that's really how we're trying to approach everything. And as the engineers talking to each other about how to solve this problem, that's gonna bring us forward. Right, and that's the kind of advice that's very helpful when we're dealing with somebody. So it's not just, the getting trafficking out of the VM is too slow for us. It's like reproducing the use cases, and then we can actually then try and recreate it ourselves and give us, we can try and find the specific problems. And this is obviously a space that we're working on quite a bit because the network performance now has become so important, and they're different trade-offs for getting traffic, for example, in and out of VMs because in the NFV case, you may trust the VMs, and so that you can just do direct memory copies whereas traditionally, when we've deployed VMs, it's been in an environment where people don't trust the tenants because they're selling CPU time. And so really, there's sort of new areas that we're having to explore, and so having these sort of interactions is really important to understand how new use cases are evolving. And so this week, there was an interesting meeting between the OPNFV community and the telco working group within OpenStack, and one of the good things that came out of the discussion was there was a sense that the OPNFV folks would not be successful if what they did was came and prescribed the solution and the technical implementation. What we really heard from those guys was we wanna hear what the problem is and what you need functionally and sort of quantitatively, we need to get from A to B, not change this line of code and re-implement this, and if we can tell them what we want rather than how to do it, the relationship can work a lot more smoothly because we give them the latitude to do things that make sense in the overall architecture and community where they kinda understand why something might be the wrong way to do it that wouldn't be obvious to somebody that was from an outside community. Repeatable, sorry. Repeatable, automated, high level tests are the greatest single idea that the software development industry has had, possibly ever. And that, so that's another thing about how to interact with other communities and how to make this work is, don't bring me a written piece of text about I could try to do X, like bring me a script and better yet, like a script that runs and has a description of what's dependencies are, and it could actually run in my CI. That's like, and I realize that's like a tour's magnitude of work, but not only does it actually fix the problem, but it keeps the problem fixed for the rest of time. Very good, yes sir. So a couple of quick comments and I'm not a developer and I come from a networking infrastructure vendor position and knowing where networking is compared to other components of this industry and how it's very dominated by a couple of major vendors which is very different than most of the other components of this. Looking at what happened and why these other industries change and where they are now versus networking, I think big part of it was the leadership that Linux provided. And knowing how this progresses against the bigger vendors, unless somebody provides this leadership and writes this code and even if it is initially the wrong direction, right? Once there is an alternative and we learn from it and we move back and even if we need to rewrite it at that point of time, I think it's very hard to move the ship away from every vendor trying to differentiate themselves and hey, we provide the right solution, sort of approach, right? So and this is where I guess focusing on the open and the V as the lead to take us down this path because I appreciate that you don't want to get involved in writing the code and solving the problems yourself, right? I totally understand that. But I struggle who would, right? Yeah, to comment, I guess, maybe I misrepresented a bit. I mean, we don't intend, what we don't intend to do is write the code in OpenFV. We intend to engage in other communities and help them write the code. We don't just throw papers at them and things. We intend to bring developers and solve this problem. So from that perspective, when it comes to creating the platform, we intend to come with developers and we intend to come with problem statements. We don't, and I guess the difference is we don't intend to come with blocks of code we want you to implement or copy into your repos and we don't intend to come with papers that we expect you to follow. We intend to collaborate and develop and then pull that back into our platform. And one of the challenges that we have in OpenFV is one of the features coming. Well, the features are coming after we've come and talked to OpenStack and got the implementation and then pulled it back downstream and then managed to integrate it across the platform. So our feature development cycle is ridiculously long. Well, not ridiculously long. It is realistically long. But it's one of those situations. And one of the things you mentioned also, Linux provided leadership. Linux provided leadership by providing a common foundation that allows differentiation. And that's something that in OpenFV we want to enable vendors also to differentiate but to provide that common foundation for them to do so. And I think that's really important. But I think also there's been quite a bit of innovation lately in networking. I think historically that was certainly true. But with SDN, I mean, not that it's fulfilled all the promises that it made, but I do think that there is much more programmability in the network than there used to be. And so there are a number of open source projects that are providing that. Thanks to Nisera, by the way. Oh, thanks. The last comment, and I didn't mean it has to be OpenNV. OpenNV, it could be the telcos. And honestly, I'm looking for the AT&T's and the Verizon's and the NTT's to do that, right? Well, and again, there isn't a difference between that there's not an us and a them. It's all us now, that's the whole point. So there's, I mean, just, I mean, there's 2.3 million lines of code in Open Daylight Repos, which I can either argue isn't an asset or a liability, depending on the day, but we are writing non-trivial quantities of code to try and solve real problems and provide a platform for people to deploy networking functionality into their networks in as open and vendor-neutral way as we can figure out to do. And so I think, you know, I mean, the Linux Foundation always yells at me because I don't make this analogy, but we really are trying to build Linux for networking and there's a lot of people trying to push that down the hill. So I think you're exactly right that we need it and we're trying. I don't know if we're gonna be the most successful project in the world at it. I certainly hope so, and I put a lot of my life and my time and my lack of sleep and into trying to push us there and I think that we're gonna get there, but that's exactly what we're trying to do. So I don't think it's for lack of, it's not like people don't understand that's what we need to be doing and aren't trying to do it. And so I encourage you to come to Open Daylight and see what we're doing and figure out if we're meeting what you would like or not. Very good. Okay, yeah, thanks, I'm Brian, AT&T. So I think the thing that Chris said about the differentiation is a key point because what we're seeing, and I think this is to Margaret's comment, is that the desire to differentiate results in solving the same problem 10 different ways in 10 different communities and just basically provides an environment for fragmentation. I think as a midstream community, OPNV at least should try to encourage consolidation, try to encourage a common open source solution space where people can plug in their differentiators and their products if they want to, but stop trying to solve the same problem 10 times. Because even if we didn't deliver a line of code, we're gonna deliver tests, like you said. We're gonna define what we want and we're gonna make sure it works, right? And if we have to do this in 10 different communities, we still have the same scalability problem. Any comments from the panelists before Margaret goes? I mean, I would also love for there to be fewer fragmentation, but that's a, there are strong economic incentives that are driving a variety of actors, many of whom I try and negotiate with and help. And I think actually we've done a better job than the people on stage are actually responsible for a much smaller number of viable alternatives than you would have gotten if the four of us hadn't been there. And that doesn't mean that it's enough, but I think that, you know, we understand that we're trying. I mean, if you want one open source solution, I mean, I think how long did it take for Linux to be the open source operating system? There would be many in the community that would argue that it's not. And I guess my only point that I would make, Brian, is that that particular problem is not one that open source as a concept has solved very well either. I mean, I used to give presentations where I would say, if it's worth doing once in open source, it's worth doing 10 times, because there would be 10 different implementations. And when we still have that, right? It is the case that though often there will be a coalesce over time to something that is dominant as Linux is, right? And so I don't know that as far as an open source construct, we can hope for much more than that. And particularly in NFV and SDN both, they're new, they're disruptive, there's lots of ideas about how this might work. And the benefit of that is there are lots of people scurrying off in varying directions, trying things, and that's great for technology work through, but particularly for those trying to use in the early days, that's why it's called the bleeding edge, right? I'm sorry. Yes, I think one of the hardest things we have to do in OPNFV and in any of these communities is sort of find the right balance between diversity and freedom and making some choices. If we don't do any curation, you end up with a bag of stuff and it's not a platform and you can't build an ecosystem around it. At the same time, if you anoint winners too early, you don't allow some of the evolution and Darwinism to play out. And I don't think any of us in any of these organizations have exactly dialed that in perfectly, but most of what we're trying to do at the leadership level is find that line between freedom and curating. So I still struggle with the development and co-issue of where do you do it. The reason why OPNFV was created is because I got the impression there was a sense of frustration that when you're an open stack, it's so huge and it started in a certain market and they have this other market that is huge that's not gaining enough attention or not understanding the true needs. And as this industry and the networking industry is pivoting, we're all sort of all over the place so that was one of the values of why we want to get OPNFV together so we're not all coming up with our own little ways of doing it 5,000 times that we could hopefully do it in a more collaborative environment and this is the only OPNFV, it's the only environment with networking people and the users of networks can get together to figure out how to do this. So saying that, I find it, maybe it's because people say certain short phrases and you walk away thinking what they meant, but I don't know, Paul Carver's here. Paul Carver from AT&T and I were in one of these neutron meetings and we sort of started discussing it about how important it is for open stack to start, I mean we didn't quite say this way but I would say this way is, gain their act together because we're starting to build a lot of our businesses on this which is a lot of money riding on it and that it's important that the organization tries to help us to address our needs so we sort of got attacked. I mean I want to say sort of, we got attacked and one gentleman said that, well, you had the super user Comcast that submitted whatever, 6,000 or whatever, and that finds a code and if you really want whatever, you should bring all your developers there. And then so we got into some discussion of participating directly and also bringing code. So that's why I find it interesting because I was in some of those discussions where they said, work with us on the blueprint, don't come with a preconceived notion of how to work it and let's work it as a community. The problem though is they don't understand, they not all, right? There's a set of folks that don't understand networking, their view is what is the wrong with VXLand, what's wrong with LAND, and what's the big deal? Why are you guys, we've got whoever of the world having millions of users, why can't you guys just get on that bandwagon? So that's why I struggle with working with them, I mean of course you have to try but you get to a point like if they don't understand how do you move the ball forward as they keep discussing and arguing like, no, no, no, we have a better way of doing it than you. I can respond to that I guess. So there will always be people, open source is great because it's a place for many people to come with many different backgrounds, many desires and many needs. OpenStack is a great example of that, you could almost argue too many but that's really part of the wonder of participating in OpenStack. And there will be people that don't care about what you want, it's as simple as that. I think it's the community's responsibility to be accommodating of multiple needs. If there is someone that doesn't care about something you're doing, well that's fine, they shouldn't necessarily say it's wrong unless they take the time to understand it and then are able to articulate why it's wrong. And it's a culture that you need in any open source community, any open source project, acceptance and understanding is the two main foundations for how you actually go about doing things in those communities. And as we know, when the pressure hits, acceptance starts to struggle and you don't have time for understanding and so then you get into fun conversations which generally end up in red faces. But this is natural and this is normal and it's part of the bonding experience because you come out two weeks later, you're a little bit embarrassed and you actually have the conversation. But that's part of learning and part of new people engaging in these teams. I mean every one of these projects is a team and we all know how teams work, forming, storming, norming, someone and so forth. And in open source communities, norming is one of those long-term things which you may actually never really get to. You're just forming and storming and just moving forwards. And you norm amongst the people that are interested in what you're doing. But I guess it's a response, it's not an answer. This is a fact of the open source community and this is for us in OP NFV who are telco-centric, we need to be aware that we're addressing communities that there are components of them that don't really care. And that's okay. We accept that as long as they can accept that we do care, we're happy. And that's I think the line we try and find. Do you believe, do you believe? So knowing the key, I guess the leaders if I call them that on an open stack. And then of course we know the folks going on the networking. Do you believe that at least the leaders or the peak TLs or whatever you want at home have the right skill set if I call it that? To really appreciate what we're trying to do and to actually work with us on something that is accommodating versus not. What does that open stack expect? Thank you, man. I'm really asking that. I'm asking the non-open stack folks to do judgment on the people who are here, I guess. So Kyle is probably the nicest and most willing to sit down and listen and also technically capable of understanding diverse needs, humans that I've ever met. So like it would surprise me, although the implication at the beginning of the question was that Kyle disagreed with your approach. That would surprise me and certainly, and there's lots of us in the community that know Kyle pretty well and we're happy to sit down and figure out if there's things that need to shift. I will say that if you approach consuming an open source project like OpenStack, the way that you approach buying support for Windows, it will blow up on you. No, I mean, and you know that, but there is, but somebody has to sit down and write the code and if it's not going to be AT&T, then it needs to be a customer of AT&T or AT&T or some set of developers need to begin to understand what your needs are and start to code to those needs. And where those developers come from is an interesting question. I think that there's a whole bunch of people in OPNFV who a variety of your customers have decided are, yeah, vendors, sorry. Well, but like that's, but this is somebody needs to write, somebody needs to end up writing the code and showing up with a set of demands and without some cogent plan of where the code is, where the people that are going to write the code come from. I'm not talking about code, people are actually more important than code because they're capable of reasoning about and solving problems and getting decisions made about conflicting code. But that, but without that, it's hard. I can sort of, I understand the frustration that might have come up in that meeting. Let's just put it that way. Cause I get a lot of people that come and ask for things in Open Daylight. Like I went open for 1.4 support and I say, great, I do too. But I need warm bodies, right? I mean like, and I will do everything in my power to get those warm bodies on the right track, connected with the right people, helping them unbreak Jenkins at two in the morning. I will do anything it takes, but I, but you know, it's that there needs to be somebody with a vested interest who's going to sit down and actually do the work and figuring out where that comes from. And essentially in the networking space in open source is something which we're just doing now. And I think that we're, that more than the technical things is what's causing the friction that I think everybody is seeing. So we're gonna work it out, but it's gonna take time. Yeah, I think that the introduction of the telco industry into OpenStack and the creation of OPNFV are all an interesting introduction of a new culture into the OpenStack community, right? And part of this is getting through the impedance mismatch between those cultures, right? A vendor centric, you know, vendor's going to supply me type of culture versus an open source culture. Again, focused more around cloud data center than the needs of a large telco operator, right? Are separate. And so part of it is, as these gentlemen have said, you know, you get developers working across those communities so that that impedance mismatch can be worked through and the dialogue can actually be productive as opposed to not really understanding and to borrow a networking paradigm, you know, you're conservative in what you say and you're liberal in what you accept, right? Before we pivot, I just wanna actually directly answer the question. I have found the PTLs and the core members from OpenStack that I have interacted with to be both highly competent and collaborative and open for discussion. I found some that haven't really cared about what we're doing, but they're willing to listen. So I think, yes, we have some very good people here. As a direct answer to the question that we did avoid answering for some reason. I started with talking about that. You talked about Carl. So first of all, Colin, that request for OpenFlow 1.4 support sounds very familiar to me at the OpenVSwitch project. And the answer I always give is when you show up with patches to edit, we'll be happy to take it. I mean, that's kind of the answer to any feature request. We always love to have things so that are new. But so one of the things that has kind of surprised me at this conference is any number of people have come up to me and said, so how can OVN and Open Daylight work together? What are some points of collaboration? It's not something I thought about much. And I wondered if you or anyone else on the panel had some thoughts. Do you wanna start or I can? Well, I mean, I think this was the obvious question that came up when we announced the OVN project. And I think we can, my answer to, oh, is this completely duplicating the effort is, no, not really. I think that there's something that needs to manage OVN. OVN is controlling the OVS's and I think doing a good job or we'll do a good job at creating virtual networks and adding new features. But we're not a CMS or a cloud management system. We're low level guys. So if you love making database calls, sure, you can configure OVN, but most people aren't gonna do that. And so, whether it's OpenStack or OpenDaylight, that's managing that, I think makes a ton of sense. And it's actually, I think there's some precedent for OpenDaylight, not just acting as an open flow controller. I think that's pretty much that on, which is that most traditional SDN controllers were basically control plane replacement. OpenDaylight is a control management and orchestration plane replacement, which is at times a really terrible idea. But also sort of, I think we're at a point in IT where we're, one of the really interesting questions about SDN and the broader thing we're doing in IT right now is what happens, what would happen if we merged all the layers together for a brief period of time to figure out if we drew the layers in the right place 40 years ago and it's still correct, or if we would re-layer them. And I think if we end up the other side of that and we have the same layers we had before, to some extent, this whole experiment will have been a failure, but I don't think that's a significant risk. Well, I think we've already abandoned the layering that we used for 40 years when we went into cloud. The problem is we haven't replaced it with any, you know, in networking in the OSI model, we were very good at calling layer violation because the layers were clear. Now it's not at all clear where those layers are or what the hierarchy is. And so I think that, you know, one way, I mean, so OVN is pretty clearly, you know, sort of just below the management plane providing really a control plane. And so you can imagine using OVN to do a bunch of the control plane features that are in Open Daylight, but the truth about it is most of what Open Daylight does these days is actually more of a management orchestration plane. With the exception of open flow control, we tend to stay away from the control plane because the interfaces that we have are actually to control planes when you're outside of that. And so I think there's a very natural way that OVN would be another sort of southbound way to control as much of your network as you can. You can also imagine using Open Daylight to bridge OVN onto other things. I mean, you want to advertise routes using BGP based on what OVN is configured. Great, you can do that with, you know, some relatively minimal amount of code inside of Open Daylight. Yeah, very good. As we talk about how to actually introduce new features, right, be it new features requests, I'm making the assumption that people who are bringing these requests have engineers that are able to actually work on this. So with that assumption, you know, granted, there's one way to do it in Open Stack. It's actually still somewhat undefined in Open Daylight. I'm not quite sure what it looks like in Open V-Switch. Can you talk a little bit about both the necessity for that process within each of the projects and is there any cross-collaboration that can be done across projects such that the same types of information and the same types of milestone intersection points exist across those projects so that those that are actually looking to interact with multiple projects to build something can do it more successfully? Yeah, I'm gonna take a swing at this because this is pretty much the first thing we found when we started our project. It was a soft pitch totally for you, Chris, yes. Perfect. So how do you work upstream with so many projects that do things so differently? Yeah, good question. No. You're one of us. Yeah, we'll let you know. Actually, in Open FB what we decided was there is no answer, there is just action. So we established what we called community groups. So we have a group of people that are focused on working with OpenStack and what they do is they work with the OpenStack community and we have OpenStack community members coming into OpenFB to sort of help facilitate this where we learn how to interact with the community effectively. And each community has to be governed in its own way because it's made up of a certain class of people who have a history, who have evolved from where they are from some place to where they are and they still have a view of where they're going. And none of the projects are the same. None of the projects have the same maturity and you really have to be able to engage. And we talked about engineer to engineer, it's critical. Everything else is really just a way of framing the discussion between the engineers at the end of the day. If you can have any process you like, but if you don't have engineers flowing through that process and having engineers communicating back in the other direction you're never really gonna get anywhere. Open-minded engineers. Well, yeah, as open-minded as you can get them, yeah. But no, it's a real challenge. There is no cookie cutter. You really have to have a group of people that understand the community you're engaging with and are able then to articulate that community's needs back to ours so that we understand what we need to do in order to engage well with them. My experience has been that almost every project you're interested in interacting with has meetings that are open and if you show up to the meetings for a couple of weeks without talking you'll get a pretty good idea of what you should be doing once you start talking. And when you start talking people will actually participate and interact with you and that like if you spend a couple of weeks trying to get to know the feel of the community you're trying to get involved in it usually doesn't seem near, it usually works. Basically if you show up and you're not an asshole it tends to work. And if it doesn't in open daylight come find me and I will figure out how to make it work. And I think that's probably true for the rest of the people on stage and their projects and I'm sure other people in the audience who have their own areas of their work in but usually there's somebody feel free to reach out to me or anybody else in open daylight feel free to reach out to Chris and OPNFV and we'll make sure that you find the right people and you can engage in the right places. Very good Thomas, my apologies we're actually out of time. All right but folks thank you very very much for participating and please help me in thanking the panelists. Thank you.