 Hey, good morning everyone. I'm welcome to the Open daylight open stack state of the world Session thanks everyone for coming First thing here is my colleague Kyle Who as you may know is was a lot recently elected neutron PTL is not going to be here mostly because he vastly underestimated the amount of work being the neutron PTL would be and he's off doing other things, so Cat Kyle a break on that so What I'm going to do is just dive into this. Oh, let me let me just say who I am. I'm Dave Meyer. I Work at brocade I'm currently chair of the open daylight TSC technical steering committee and This talk is kind of about What we've learned what we did over the last year in open daylight What kind of things we learned and what the future what we're going to do going forward so as a As a kind of a way that to run this is if you have a question or anything just you know Anytime let's we can do it in line and just get up and stop me So with that, let's see So what is it? I mean there's how many people have heard of open daylight. Oh Wow, okay. Good good marketing. Where's the Lenox foundation? Excellent. I Had some personal learnings actually open daylight You know I'd written a lot of code and put it on github and stuff like that But I've never really been in one of these really large-scale Open-source projects, so I learned a few things and I'll kind of tell you what those were about and then You know a little bit about integration with Neutron few metrics about hydrogen And something about what's next which is helium so we don't have We don't have the alphabetical order releases like Open stock has we have something that is like following the periodic table, so it's going to go high It's gonna go hydrogen helium lithium, you know So check out the periodic table. You can find the names of the next releases And what's in the queue? so that What is it? So open daylight you guys know what it is apparently basically it's a it's an SDN We're trying to build an SDN ecosystem Platform Robust code extensible open source all of those good things Get the industry to accept it as a as a sort of standardized platform that people can not only use as open source Components but to build into their products So an example of where that was really worked well as in in an open v-switch right a lot of products are Based on open v-switch and now open open stack as well And then we want to you know We wanted to build a community because as I'll talk about this a little bit later But you know one of the things I've learned over the years that you know without community you really have you know a very diminished kind of project And I'll show you where we think that the community members can you know add or subtract value So as I said, it's a SDN platform how many people know what SDN is supposed to be I mean, you know The actual actual definition of SDN seems kind of elusive, you know, I mean you can it's you get as many definitions as people you ask But we're trying to kind of capture all of that So it includes northbound abstractions that people can consume rest and others and also southbound Multi-southbound so that's a little bit different than say some of the popular SDN controllers that are out there right now Like floodlight like floodlight is it open flow SDN controller We had in mind when we designed open daylight that there would be multiple southbound. So there's open flow. There's net conf There's there's BGP LS a bunch of other ones. So and it's designed for that So and then above that above that all of that infrastructure We have applications that people have written against all of that So and as I said whatever else we need to make it work Which turned out to be a lot more than we thought but you know There it goes and so the architecture or project framework is sort of like this The idea is that the there's a controller, which is really the core of the project You can think of it sort of like the kernel right so you have to have a robust rock solid kernel Or controller if you like that allows That allows that's extensible and allows people to build applications above and allows people to plug in new functionality below so as you can see in this picture and this is kind of the canonical open daylight picture The idea here is that there are applications above they talk through standard API's that are the open daylight API's and those are like everyone out Like everything else those are evolving as we're talking, but Rask Raskamp things like that and then the controller itself has Network services inside it has like topology management and things like that that you that everybody needs if they're doing networking And platform services like Plug-in managers and things like that and then southbound then there's a service abstraction layer called a cell There's actually two of them one is the hard-coded cell and then there's something called the MD cell or model driven cell The model driven cell the idea here is it's really kind of a nice idea The idea here is that you have yang models that describe the southbound and then you generate code that links the rest API's To the southbound plug-in using a code generation based on yang so it's fully modular that way and pretty nice But it's also complicated. So, you know, and you know, that's shaking out right now So and then the southbound plug-ins are going to be things like open flow I2rs and the fullness of time and things like that and because unlike unlike open stack open daylight is under the eclipse license and The reason one of the reasons for that is because we wanted to make it easy for people to bundle vendor specific plugins notably southbound plugins And and have them be able to bundle these into their products easily So an example of that would be I don't know if Cisco is going to do this but it'd be like one PK southbound plug-in would be a natural thing to kind of think about there or something like that and then there are data plane elements that that we have been working against both physical and Virtual so OBS. One of our projects is OBS DB and there's also a lot of physical Switching infrastructure underneath it. So that's kind of what it's about Now who's who's who's in the game Well, like all of these like many of these open source projects There's platinum gold and silver membership and this list. I don't think it's complete I mean because of the flux and the number of people who are joining I think it's close But not exact. So you can see who the platinum members are most many of them were founding members of the of the project and then we have a couple of gold members and Silver membership is growing pretty quickly There's a diversity of vendors and others involved in the project. So what we did was You know, we worked for about a year and then recently I'll show you this but We created a simultaneous release and the idea behind the simultaneous releases that all of the projects that wanted to be bundled together had had a project timetable in which they had to get in and they were All put together in a couple of different release vehicles and we call that hydrogen And it was supposed to be released on the 28th of January The additions or you can think of these as like release vehicles Were base virtualization and service providers So the idea was and this didn't this didn't actually work exactly the way we wanted it to but the idea was there'd be a base edition I'll show you a picture of this But there'd be a base edition which had the basic functionality in it and then since we had Projects that were around virtualization and data basically data center virtualization We bundled those up together and made a virtual Virtualization edition and then likewise for service provider and I'll show you what those look like Virtualization is where we did the open-stack integration And by the way, Kyle did a lot of that work and it's unfortunate. He couldn't make it But he you know, he did a lot of work on Neutron integration before He became a politician or whatever it is. You are when you're a PTL Running around talking to people. I guess is what you do By the way, that's sort of what it's like to be a chair of the TSC to you just run around and talk to people your Your your basic utility goes down or something like that so a So we had this nice simultaneous release plan and you know the typical things milestones and release Release candidates and all of that stuff and we were supposed to have the formal release on You know 12 9 that was our target last year Right and and we had all this and we're just going towards it and you know when we got there we found out that uh It didn't quite happen the way we wanted to we did a little bit of learning By the way, this is all you can find all this stuff on wiki.opendaylock.org. It's all up there So let's see We didn't make it we didn't make the actual date, but what we did deliver were 15 projects in the simultaneous release It was kind of this is kind of a and this is still a live issue inside open daylight and inside open stack Open stack seems to have solved this a little bit But the problem that you face when you have When you have a lot of projects that want to get in is that you have it's basically the same thing that you have in any Software development organization. It's sort of like you got a trade-off features for stability And we I don't think we've really hit that the sweet spot there yet We were working on that quite a bit open stack seems to have that kind of At least well-defined with sort of the idea of core and ecosystem so we had the simultaneous release it was supposed to be on December 9th, which apparently is the same as February 3rd But that's when we finally got it done and finally had artifacts that we put up and so that was cool And everybody was pretty much really happy about that and it was done right before the opens daylight summit the first one We had and it was really there's a lot of energy around it. It was really quite exciting It's sort of like here a lot of energy around everything so but in the process we learned a lot of stuff and and not enough to make it go smoothly yet, but we learned a lot so the first thing is We had a kind of an impressive list of projects in there and some people might say well That's too many because you didn't trade off properly against stability against you know You know features versus stability again same thing But the projects were the controller of course, that's like the kernel as I said VTN is a virtual tenant networking That was a project Came from NEC open dove came from IBM Affinity management service that came from Plexi List mapping that came from context stream that was kind of interesting They were not actually a member and there was a big a lot of confusion Early on with open daylight whether or not you could contribute code or be part of it if you weren't a member and of course It's an open source project. So you can Memberships about funding it Governance and all of that stuff but to be part of it to be a developer even to get on the TSC and for that Matter to get on the board. You don't have to be a member So that was list mapping how many people know what list is You know locator ID separation protocol It's something that Dino and I did when we were at Cisco, but the context stream guys picked it up for a sort of an NFV style service chaining application Yang tools was part of the Yang Infrastructure that we use remember I said that the MV cell was the model driven cell had used yang very heavily to generate code and Both northbound and southbound we split that off as its own project all the yang parts of that Which were substantial it uses a lot of modeling Defense for all that's a that's a security Application BGP LS and PSAP that's that's kind of a service provider traffic engineering thing Open flow and open flow plug-ins. Of course our open flow OS OBS DB was is something that's being done by the right had folks. Although when it started interestingly enough It was some folks at Cisco Madoo and Brent Salisbury who was at the time at you Kentucky and now is it Red Hat SNMP for SDN is kind of an interesting thing there were some folks in At Itry, which is a kind of a research Institute in Korea where they had switching infrastructure But for some reason they couldn't develop on the on the switch platform itself So they had SNMP. They wanted to see if they could use this control infrastructure to do flow programming on Using us SNMP and that's what this does Deluxe is a UI and System testing and integration we made a sound product project because that's just a gigantic thing So that's kind of the projects that were in there. That's kind of a lot a lot of different things that were in in there Now, how did we how did we wrap it all up? How did we build these release vehicles? So this is the another one of these canonical pictures that we that we always all use so These are all the components kind of wrapped up into everything right and I'll show you how they how they how they Break up into the different releases, but this is kind of the picture of everything together So there's so you can see there's the controller and it's different components And then in the blue layer underneath it those are sort of all the southbound plugins and then above Above the API recite API that they're showing there are all the things that our application So let's see how they how they fit together So this was the base edition. We kind of figured that you know the basic thing that you needed was net com because you got a you're gonna you need some way to configure the devices and then open flow because Most of the controllers out there were open flow only and we wanted to have that capability So the idea is you had the base edition then you'd add stuff to that to make virtualization and service provider So that was the idea. So what did service provider add? Well, it added inside the controller. Oh by the way, this it's The controllers written in Java and it uses OGS I OS GI Extensively right so where you see the affinity service and the lip lists service kind of outlined in yellow inside the controller that's where they added OS GI bundles to give the controller additional functionality to Support their application So it'd be like you had a KLM or something like that except for its OS GI in this case So we added the Lisp affinity service with I'm sorry the affinity service and the list service Which both of which added functionality to the controller again through this through OS GI bundles and then on the southbound we had SNMP BGP LS PSAP and less and Then above that we had the DOS DDoS protection thing that came from Defense for all By the way, stop me if anybody has a question It's important So the so the virtualization edition added OBS DB As a southbound how many people know what OS OBS DB is Okay way to configure and talk to OBS And then it we also put the affinity service open stack So this is where open-stack integration came in make makes kind of sense And VTN manager and dove manager so VTN and dove were Different ways of doing data center virtual multi-tenant virtualization one coming from any C1 coming from IBM Now, I'll just say right up front. These things didn't play nicely together They all VTN and dove wanted to wanted to talk directly to the switching infrastructure And there is not an abstraction in here that sits between those affinity could have been that but they but the plexi folks Just didn't get to it and you know We taught we've been talking to them about that affinity the affinity metadata services kind of an interesting Actually an abstraction layer so the way it looks it's drawn here is actually the way it's bundled But actually VTN and dove could run over Affinity but we just didn't get there and then at the application layer We had this VTN coordinator which took advantage of the VTN manager that again was an OS GI bundle that got loaded up into the controller We had the DDoS Protection stuff and then we had open-stack Neutron, which was the interesting part for this So how did that work how did open-stack Neutron work? So Open daylight it's you know the reality is it's a pastor right because what we wanted to do is allow open-stack to talk directly to the open daylight controller and then Exposed those API so basically what it does is it it exposes a single common Open-stack service northbound so on the top of the controller right by the way We have this terminology northbound and southbound so you can think of the controller in the middle northbound is on top You know this is where applications use rest API's to talk to the controller in southbound We have what are basically protocol plugins the problem with the terminology is north and south is kind of relative to where you are You know as an east and west up or down left or right that terminology is not very good, but that's what we're using right now so So the API that exposes that we expose just matches Neutron directly, and that's why I was saying it was kind of passing it through And there's multiple implementation of Neutron networks inside open-stack itself so the open daylight open-stack plug-in just passes through which is Which is a good way to start because it just made the open-stack plug-in a lot simpler And it pushes the complexity down into open-stack, and then on the left I'm sorry down into open daylight So on the left here you can see kind of what the picture is so we use Neutron ML to Mechanism driver, and then it talked to rest API's inside open daylight, and then Neutron service was inside there So and you know that it works fairly well The integration with btn and dove again is a little bit harder because they both want to control resources themselves Directly, and there's not an abstraction layer that would allow them both to kind of do that independently So you have to choose one which is kind of not optimal But that's one of the things we need to we need to learn a little bit better or deal with on the other hand If you're just running OBS OBS instances, it works very well. So what's the status of the integration? ML 2 driver got in that ice house, which is nice supports VXL and GRE There's nice dev stack support it got merged upstream and it basically you can run out open daylight It's a top-level service inside dev stack, which is really nice if you're if you're using that Neutron API services now in an open daylight So we made a lot of progress here and it it provides a Neutron API and you know, so there's multiple implementations it can handle and the initial ML 2 Plugin for focus mostly on the core Neutron functionality and still uses some older components, but It's moving pretty quickly So where are we? with this one our idea is We have a quite a few updates planned for helium remember the release that we have now is called hydrogen By the way, you know when we get to lithium and you know beyond beryllium plutonium, I don't know how that's gonna work out, you know, but it's what the community wanted so we have a Various updates planned for helium and you know including Plugging plugging changes for stability improvements, which is one of the again This is one of the big struggles in open daylight right now is stability versus features And we want to be able to notify From ODL to the mechanism driver once ODL is set up the host the port on the host We can't do that right now Security groups implemented with open flow rules. That's kind of a logical thing that you might want to do L3 routing handled by open daylight and don't do that removes the need for the L3 agent, which we want to get rid of that and The obvious is a did anything else that comes up plus You know fixing all the bugs that are in this thing So a lot of this work was done by Kyle kind of sorry. He's not here He could talk a lot more about what was interesting to him in that but You know, we made a lot of progress against this So we have a lot of open-stack Utilization of open daylight and actually it's the integration really Demonstrated something that we wanted to kind of work on a little bit Which is what is the relationship between open-stack and open daylight and so open-day lights Intended to be kind of a control platform that would sit underneath open open stack But exactly how it's kind of like people waving their hands a lot about yeah, that sounds good a lot of PowerPoint pictures But now we have this so it's it's much better than PowerPoint Let's see So that's where we're at with the integration, but we learned a lot About just doing open source in the process of this. How many people have been in the open source community for you know two years or longer? Okay, not everybody you guys are mostly like me. I'm kind of new to the open source community and so here's what I learned Let's see. Oh, I wrote a blog on it's down I'm sure you guys can get these slides at the end of this and if not, I'll just send me an email and I'll send them to you but the first thing I learned was that Community building is the core is a core objective and in fact the idea that you could do innovation through collaboration is extremely powerful and If you didn't know that You could just look outside the doors there and to kind of get the idea you know so Building a community around open daylight and how Anna and integrating with the open stack community is core Objective because you know, but by the way if you don't have that you might have a nominal open source project But it's not Leveraging what's interesting about open source. So the idea that you slap a some kind of open source language License on your code and put it up on github doesn't make it open source in that deep way Code is calling in the realm. So I'm sitting on the TSC call and I'm going wow, you know nobody cares about anything but code here and That is a key thing and in fact When the the hydrogen release happened right just immediately prior to the open daylight summit Well, it's really tough to keep open stack and open daylight straight here up here The Linux foundation had this idea that what they would do is they would build build these challenge coins They're kind of like military challenge coins So everybody who was a committer to hydrogen the hydrogen release got one of these coins and the coin I wish I could kind of show it to you a little bit But it's kind of a cool like thing, you know, and so everybody got one who was a committer because code is calling in the realm coin and Then Engineering systems this is Madhu hanging out in this Google hangout while they were doing OS OVS DB development the engineering systems that we were that we were using were so sophisticated in terms But just basically open source or open source like tools And they were and it turned out that those tools were as almost important as the artifacts that we were building And look and by that I mean engineering systems but by that I mean tool chains as well as how we communicate IRC all of those other things and There's kind of There's there's kind of something like if you if you follow this down to the logical conclusion of this You kind of learn something and this is kind of what I learned And by the way, well, here's the thing Engineering artifacts aren't the thing that is the source of sustainable advantage or in it or or Innovation anymore, and you know working in a vendor that makes things. That's not a popular statement but What is important is what I said earlier is the engineering systems the culture and the people and processes and the open source thing really embodies this Excuse me if you could compile this down one more time you get this What you build is not really as important as how you build it and that I heard this I heard this in a different way yesterday or the day before Maybe it was yesterday When the in the keynote the guy was talking about the good cheap and fast triangle And what he said was if that's morphed into fast fast and fast because you can build good and you can build feet Cheap out of fast. That's the same thing. It's it's a corollary at this so I'm in So I'm talking to some people In China about this and they were saying well, you know open daylight is pretty unstable I'm wondering we're going to be able to use it and I was thinking how do I talk to it is you know By the way, it's getting a lot better, but it was like every other open source project of the scale It's there's it's dynamic in the early days right chaotic So I was trying to explain to some folks in China and here's what I came up with tell me if you if you can grok this So I was trying to make an analogy to the early solar system. So in the earlier solar system You know everything's chaotic, you know people are trying to get their code in Rocks are flying around everything And and anything with enough, you know anything with enough gravity, you know people like enough Starts to you know Collide with one another and we just saw this in open daylight and in fact Everything is fluid and molten like the earth in the early days asteroids blasting into everything epic collisions That's what happens in the early days of open source But in the end of it, you got a really nice blue planet that we like or the Linux kernel or open stack So you got a kind of um You got a kind of uh, you know wait that out a little bit in the early days of this See how am I doing on time? Let me give you a few metrics from open daylight so you can get an idea of From hydrogen so you can get an idea of how it happened In the early days, we had a couple only a few projects at the end we had 16 projects what it is I mentioned and When I made the slide there were seven new projects, but now they're more So open daylight had You know a quite a bit of code in it and it broke down like this It was it's written in Java. So of course mostly Java the C++ stuff is really External in some ways they're like proxies So you build a you build some chunk of blob of this thing over here And then you put a plug-in southbound plug-in an open daylight to talk to this whatever this other thing is and then it proxies the service Some see some Python So this is something that's kind of interesting and again. This is the stability versus Features thing there are 1.5 million lines of code or 1.05 million lines of code in hydrogen That seems like quite a bit but The comparison that's being trying to make is that you know open stack took quite a bit longer to get to that scale Now is that good or bad a lot of people are gonna say that's good I'm on the fence and again, it's the same trade-off. I think we looked at this a little bit But the project itself has growing membership, you know, it's up and to the right Okay, sorry about the animation here. I don't like that. So what's in helium? Okay, so here's the helium simultaneous release plan So we think we're gonna have Helium, you know out on this time schedule and as I said earlier hydrogen Well, you know didn't quite make it but you can find all of this stuff on the wiki Just look for it. Just go to wiki.opendalight.org and you'll see this release plan there But so far we're moving along pretty pretty On cadence we want to get on it You know we want to get on the cadence like open stack as like maybe over six months or so And we want it to be deterministic because developers need that determinism to figure out When do I have to be ready to go into the simultaneous release when you know when when do I have to hit all these milestones? So we want to get on a more deterministic cadence. That's about six months so in the queue There's all kinds of stuff in there and now you might argue that's a lot of stuff and it is some of these things aren't In the incubation yet, so there's Gotcha proposed projects So application policy plug-in is something that came from Cisco It's I think it got renamed to group group based policy So it's a group based policy Then there are all a bunch of other projects here that people have proposed Oh the pack a packet cable PC mm manager that one did make it through incubation I'll show you that in a second and some of these other ones are kind of interesting somebody wants to do A simulator on top of Open Daylight. There's data persistence. We don't we don't have good. We don't have good Data store models yet it the original controller used something called an infinite span Which is a which is fine if you're doing If you're doing database like and transactional like operations, but networking doesn't really really likes to see what's called eventual Consistency more than strong consistency Oops, okay so There are a few really interesting ones that are that are coming up. So this first one SDN I SDN I I really I really I think this one's kind of interesting because what they're trying to do is do So now here, you know, what's east and what's west but you know Confederation it's about confederation and they they intend to use BGP to do this as a first cut Open contrail they have a plug-in that actually you can talk Open contrail southbound. They're a protocol southbound. So you could use the northbound neutron plug neutron interface for open daylight to talk to contrail There's some subscriber awareness stuff people are updating the L2 switching infrastructure Secure network bootstrapping infrastructure people are trying to figure out. How do I get these switches and controllers to trust one another? That's by the way, that issue is completely ignored and in most of the SDN Implementation of literature Triple a there's no triple a service in SDN. We're trying to get that service function chaining while everybody wants that So what what has actually been through to incubation this application policy plug-in thing Which is actually now renamed group-based policy There's a toolkit that has various different things that you need just to just to support development Documentation has actually become a project. So open daylight was kind of let me put it this way slim on documentation I Dynamic resource reservation Negotiable data path models is kind of interesting. This is TTPs Table typing patterns from the ONF. So this is a way of telling flexible switching infrastructure How to how to arrange the tables in the switch? Root parent was about how to do better release management and op flex is is a kind of a policy based Protocol that came from Cisco as well So those are we have all those in incubation and I all of those I anticipate are gonna go into helium and just in case you're Wondering one of the things I did And I don't know actually know how this works in OpenStack But one of the things I did when I first got on the TSC And became the chair of the TSC as I opened up the TSC calls to anybody who wants to be on them and and the mailing list because I was really not a big fan of the way say the IS IESG operates in the IETF. It's sort of a star chamber a little bit You don't know what they're doing how they make their decisions. There's an open-source project. There's nothing that's secret in here so if you get on the wiki wiki.opendaylight.org and scroll down towards this TSC page or something if you click on there There's a whole WebEx information When what the WebEx is where the IRC channels are and all of that and tomorrow What we this is the agenda I have for tomorrow and what we're gonna do is mostly about creation reviews because you have to do a Creation review before you can get into the simultaneous release and tomorrow's the last day to do that So these four projects are are up for tomorrow and that will probably take most of our time So please join if you want to listen what's going on or you have by the way you can talk too I don't restrict the discussion to okay. I'm down to I'm down to like two minutes or whatever I don't ever restrict the discussion to only the TSC members and we have many people in the community who Who are just active on those calls and they're not TSC members So if you like it if you like please come along and come along for a ride Let me just say a few things There's some quasi-technical things. We got to continue to build community. That's that's really important Um, we have the code color quality and and coverage, you know stability performance bug fixes You know, we really need that and again I'm not really sure how that gets handled in open stack I know that I know that ice house at least in the case of neutron Really focused on this really focused on stability and performance and bug coverage. So we have to do that Distributed systems. We don't have that nailed down. We like I said we have in finish band, but we're looking at Acca Staffing well release engineering. We don't have it actually a release engineer documentation Engineering systems that Linux foundation supports us Andrew Is really great We need fewer humans in the loop when we did the last release. It was way too manual Wasn't automated enough and that had to do with the fact that we botched up a little a little bit of the numbering of things and Basically, we need more code that writes code in the in the project fewer humans more automation Not that I don't like the humans that we got a lot of nice humans Yeah, and of course, you know sustaining engineering is always a problem, right that, you know, nobody wants to fix bugs There's millions of bugs performance and scalability code quality and new projects and I think I'm out of time That's that by the way By the way, if you if you have any questions about it if you open daylight if you want to if you want to come along and play You want to do some integration with open stack? There's plenty of people, you know Talk to me send. There's plenty of lists that you can join. It's plenty of IRC's come along for the ride