 Hello, everyone. Thanks for coming to this talk. Thank you, Shane and everyone else who organized the Apache Way track for including me. Excited to be here. Talk is titled from dev at to user at via the Apache Way. Subtitle lessons on the front lines of the ASF incubator. So the title comes from essentially the journey from bringing a new project, spinning it up, and then making the transition from a loose collection of ideas and code into something that is suitable and exciting for casual users, right? The Apache, we don't just want to make code that helps us, we want to make code that helps everybody. So some background on me, I've been programming for 25 years, but this is the first open source project that I've been involved with. So whatever I know about open source, it's been from this experience being in the trenches and from the mentorship and kind of learning on the job of working on streams. So streams was created to fill a niche or to fill a gap. Essentially, there is a standard called activity streams whose purpose is to allow for activity and profile and related data online to be expressed in a network agnostic fashion. So what is a friend or a follower or a post or a video, these are more general concepts than simply whatever one specific network or enterprise software is idea of what that is and should be. So the activity streams vocabulary expresses how those can be expressed in a generic fashion. And the W3C is working on putting out and has put out and is codifying this as hopefully the lingua franca of the social digital sphere going forward. So this is a, I think, important, exciting sort of project that is ambitious to say the least. There's a lot of data silos in the world today. And the more data silos that you bump up against as a coder or a data scientist or a user, the more challenging it is to keep track of everything that's going on and to be effective at whatever you're trying to accomplish. So Apache Streams tries to help solve that problem. And I will kind of talk through some of the different things that we've learned in the process of incubation. So I'm proposing these, you know, I've broken it up into four different parts. So these are areas that a project needs to develop and make progress on in order to become self-sufficient and relevant. Their vision, the code, it's community over code, but the code still does matter. A community won't be attracted to work on the code unless if the code is horrible. Documentation also, very important to attracting users and growing the community. And then the team, the team I think will be a nice transition into the next talk, which is about software is easy, people are hard. And so I'm looking forward to that and will be part of the panel and look forward to good questions there. So we can't neglect any of these if we want to have a successful project, of course. This list isn't comprehensive and for each one of these, I'm going to propose a couple different dimensions of maturity that a project can go through. For each one, I'll talk briefly about why it's important, kind of characterize the different phases of progress that I've observed that offer some examples and anecdotes from my experience working on streams. And then also just note some tactics for a project trying to get from one phase to the next. With each one, I'll also pause to kind of survey the room and look if anyone wants to add a comment or ask a question at that point. That'd be great. We can be interactive as we go. Otherwise, we'll get to the end and then we can talk about any of the aspects of this that anyone wants to talk about. But yeah, the focus is on essentially how can a incubating project learn the Apache way and grow up and become self-sufficient. So we'll start with vision. And I want to talk about both your mission, discovering what it is, and how you characterize that mission statement and the way that you market the project. So the mission is really what are the project's goals? So at the outset, it might be not clearly known. So projects entering incubation can be all over the place in terms of where they start, all the way from fully baked platforms that are already in production at many places. Some might be offshoots of other projects that are already more or less working. Maybe some single-purpose apps that are going to expand what they do. Maybe just a grab bag of barely related features that get put together into a project. Or like Streams was, an idea that had little or no implementation at the outset. So regardless, the project still has to decide what its mission is going to be. The next phase, maybe your project is uncertain about what its mission is going to be and needs to work that out. So the project team is going to disagree about different aspects of implementation, probably disagree about the scope of the project. What should the project try to do? And what should it avoid doing? And these discussions are part of the process of refining the mission and deciding what your project's ultimate niche is going to be in the open-source ecosystem. Ultimately, you want to have an explicit mission. It's important to ultimately determine the larger goals, scope, and limits of the project and state them explicitly, and achieve more capability-wise if the team's time and attention are focused on the most unique things that you do. And an explicit mission will help to hone and maintain this focus. The journey of streams in refining our mission had multiple phases, depending on what year you read our board reports. You'll see different descriptions of the project. But I think we are happy with where we've gotten to and what we're working on is now more focused on what's truly important to making that happen. And so then the statement of your mission. And I have come to realize it's very important that your vision can be summarized in a single sentence. And your technology and the solutions that you have to problems really don't have a place in your vision. Your vision is about why do you exist and what are you trying to accomplish? So often when you're determining your mission and trying to state it well, you start out with a statement that's too complicated. And it's obvious why this happens. If you're trying to solve a multitude of problems, then the various aspects of the way you're solving those problems will leak into your vision as you're trying to figure it out and as you're trying to describe it. That's expected. And you should work to enumerate everything that your project does and everything that it's trying to do. And in all those cases, highlight how your project is distinct from alternative projects or complementary projects. But don't get carried away on your landing page or in your initial user ramp-up flow. Don't let it be too complicated, right? Then when you think about all the different messaging that users or new developers will experience as they learn about the project, your reason for existing is going to take more than a page most likely. But if it seems to a novice or an outsider to be just a collection of bullet points and not a compelling narrative, then it won't resonate as well. Really, you want a coherent mission statement and a coherent set of messaging that is meant to excite people and to inspire them to use the project and ultimately to help to join the community, right? So I think it's important. And what we learned is that the essentials of the vision need to be written in a narrative, sequential, kind of outline fashion. Pictures are really important as are real-world examples. How do you actually get started? And that your entire web presence or your entire GitHub repo is going to have a lot of pieces of vision throughout it. You want to have really strong summary level content at the top, which is what people are most likely to find as they get started. And lastly, if your vision is bigger than your current implementation, you should work to ensure that all the pieces of the vision have a working placeholder at least, even if they're kind of the simplest just what you need. In order to withstand scrutiny, like you say you do that, do you really do that? Yes, we have something that does that. We're always working to improve the technology. Does anyone have anything they'd like to add before we move on to code? Yes. So the question is, should the vision be primarily technology-focused or also reflect on culture? I think by virtue of being part of Apache, we have kind of a great lineage instead of materials about culture. And the project vision should reinforce those and link to them, especially on the mailing list as you interact with people. You can continue to kind of point out and lead by example as to in order to kind of build that culture. I think the vision statement such as it is about what the project wants to become. And if what the project wants to become is characterized largely by culture, then it certainly has a place. Front and center. OK, so talking about the code maturity, four different kind of things I want to touch on. Licensing, functionality, synergy, I apologize is the best word I could come up with on short notice to describe, but we'll get into it. And comprehensibility. So licensing is the code actually legit Apache code. And out of the gate, a lot of projects don't meet the minimum code standards for an Apache release. That's fine. That's why we have the incubator. The next stage is having code that is releasable. So actually getting a release through an IPMC vote is a significant milestone in itself, so I've been told. Now getting your license house in order to graduate is another thing, because your project source code does need to be demonstrably free of any licensing issues and needs to get there by the end of your incubating journey. So functionality, yes. So the question is about license, auditability, and technological tools to help. For projects of the size of streams, that's pretty important. For maybe smaller projects, someone can wrap their head around all the dependencies and enumerate them and write the notice file by hand. That's fine, but when you cross into the realm of hundreds of things, all of which have independent versions and the group IDs might change, I do think you want to be collecting all of that information into one central place that gets looked at, at least prior to each release. And yes, Apache RAD and Apache Tentacles being two Apache foundation-supported tools that will help shed light into whether you have license problems. Functionality does the code actually work, right? So the spectrum starts at broken. There's Apache projects and not just an incubator that sometimes go periods of time with compile or test problems in the master branch, right? So to an outsider, this doesn't look good, someone who's coming and trying to get started. Regardless of why it happened or how long it's been happening, it's not good. You want to avoid this and get to a baseline working phase. So the project code, I think, is officially working when you have unit tests in every module and they get run routinely and the standard of there should be unit tests applied to all new code submissions. Getting here takes time. Assuming you don't start out with test-driven development from day one, you will build up some technicals that need to work through it. Ultimately, you want to get to where the functionality is solid, right? So projects with solid code have automated, continuous integration. They address build issues that are brought to light immediately. They probably are using integration tests that check end-to-end workflows for proper function. And features that aren't being maintained or keeping up get scrutinized and are deleted, right? You don't just want a lot of stuff sitting around that is not being maintained. And note that on this and many of these other dimensions of maturity, it's not necessarily a straight-line progression. You can work your way up to a certain degree of maturity and then fall back for a time because you're focused on other things. But general is keep trying to make things better. It's okay to be imperfect. But always try to remember that all of these various aspects of maturity are important and reflect on the project, reflect on the contributors, reflect on the foundation. Okay, synergy. I don't even know how to describe it, but I'll kind of talk about what I'm trying to express here. So code bases can be fragmented. So when a project is first being assembled, not all the parts fit together all the time. So you might have groups of modules that are incompatible with other groups in various ways. Or the same function might be implemented in multiple modules. Your interfaces aren't yet stable or you don't have any interfaces yet, you just have classes. So it's normal for projects to have a phase like this and for new capabilities that are going into existing projects to kind of start out a little bit disconnected. But ultimately you want the balance of the code to be connected. So you'll find modules that you don't need during incubation. You'll find interfaces that you desperately need. And you'll go through a lot of refactorings that hit tens or hundreds of files. And eventually you should get all the code connected. Very few orphan modules or classes that are completely standalone, everything, kind of all part of a larger system. So just understand that until you get there, you have these parts of your code base that someone can come across and think, this doesn't, you know, I don't understand this. Why is this not hooked up the way everything else is? And that those sorts of things can rub a new user or a potential contributor the wrong way. You want fully integrated code. So a really mature integrated code base will have exhaustively considered and debated interfaces, predictable class names, predictable package names, and patterns for things like configuration and logging and instantiation and testing and reflection that are universal across all the code in the project, right? And that are versatile in terms of what they can do and how much tweaking they take to try out something new. And this is really worth doing. Smart projects that are built smartly are more fun to dig into because they teach you new tricks. And learning as you get involved in a project is a good motivator for continuing to go deeper. Comprehensibility of code. This is, can newbies understand the code? So, you know, sometimes code can just be confusing. So has anyone checked out a new code base and had this like strange feeling that you just landed on an alien planet, right? So you get glimpses of familiar things and there's still some organic matter and you know, there's still an atmosphere. But on the whole, you really have no idea where or what or why anything is the way it is. You have to learn. So, and again, it's only once you come to understanding the reasoning behind the design and the implementation of a project that you can develop intuitions that are useful in diagnosing a problem or adding a feature. Projects though can take steps to seem less foreign and seem more familiar, at least to Apache committers who are coming from another Apache project, right? And I think that that's worth looking at and trying to strive for in your project. Ultimately, you'd like your code to be welcoming. So each newcomer still has to explore for themselves, but there's a noticeable difference between projects where understanding what's going on comes easily and projects where it doesn't, all right? And I think that comes down a lot to consistency. If all the parts in your project that need to do a specific activity do it the same way and you have documentation behind the, not only the individual parts of the project, but also the patterns, then someone can read about it and come to understand why you log with a certain package, why you do configuration a certain way, and understand how to actually do those things the way they want to, rather than just how the code base and the test ship out of the box. The fundamental thing here is to try to push your project closer to consistency in the design and in the implementation with every issue that you close, look for opportunities to take what you're working on and make it a little bit closer to the best practices that the team has discovered. Any questions or comments about code before we move on to documentation? Yes, yeah, these are just kind of a grab bag of different things that you're gonna need to make better. They're not in any specific order, although on each slide it starts from bad and goes to good. Right, so both vision and code, or vision code documentation and team dynamic all need to be developed constantly and there's no shortcut to, if you don't develop one of them, you'll wind up paying for it later. Okay, documentation, so here are the kind of the four kind of dimensions I propose. You've got availability, utility, recency, and aesthetics, all of these, really important for the public perception of the project. So availability just is, are there decent docs available? So worst case, they're straight up missing. Now, you need a project website and you can get away with boilerplate copied out of the proposal and links to your code and your issue tracker and so on, but not for very long. What I found is that if your website sucks, you're gonna hear about it and no one's gonna hold any punches. You have to take that criticism as seriously as any other that you hear about the project. The developers and users that you wanna recruit are by and large not tolerate missing or bad documentation and it's the number one thing that people see when they learn about your project. So even once you've got some documentation, it might be lacking. I would say you've achieved documentation level one. When your website contains details about everything that your project is, your readme's are not just a link to a license and once in a summaries, right? And your project's website and other artifacts are ranked towards at the top or close to the top in searches for your project and keywords related to your code. The docs that you have are never gonna be the docs that your users want also most of the time and that's because the practice of documentation assumes certain use cases and motivations and those are in flux in an early project and every time a new user shows up they have a new take on what this should do in their mind, right? So you need to keep creating, updating and reorganizing documentation constantly. So what you want to get to is rocking documentation. So in this ideal world, your project's documentation should anticipate everything that a user will wanna know, present the right information in response to expected search terms, right? So for example, developers are gonna Google your package names, they're gonna Google your class names, they're gonna Google the exceptions they run into. Ideally you want the results of those to land on properties that the project controls. And so Java docs are good for that or Scala docs or fill in the blank based on your technology but unless your team is actually annotating your code and providing good class and method level documentation the page might as well just be blank. They need to actually be filled out to some degree so that someone landing on that module understands what its purpose is and what it should be used for and what else it's connected to. And also a lot of techies are gonna chance on your project via GitHub, right? So you wanna make sure that your readme's on GitHub are strong and helpful. And you wanna think about people looking at the source code directly as well as people looking at the webpage. So ideally you want the content on the two connected or mutually reinforcing. There's a question of recency. So are the docs current? And sometimes the docs are just plain inaccurate, right? And inaccurate or stale documentation is actually worse than no documentation. And actually documentation that goes stale even just a few versions can become so wrong that a new user will fail to do what they're trying to do in steps one, two and three of their experience get frustrated trying to run the software at all and move on. So presuming you can get past that, you still need to be aware that documentation, its natural tendency is to become dated and even small oversight and version numbers or class names that changed, right? They lead to problems. Some of these problems can be easily overcome by an experienced software engineer, but not everyone is gonna have that experience or have the patience. And those experiences hurt your funnel of new talent. So to have current documentation, you wanna make updating documentation standard practice during code review, and you wanna perform a survey for incorrect content anytime leading up to and during the release process at a minimum. Documentation utility, right? So how helpful actually is the documentation? You can have documentation that's not particularly helpful or irrelevant, so things to think about. Does your documentation actually tell the world why they should use the project? Does it help a new user get up and running seamlessly? And does it explain the line between what it does today out of the box and what it's capable of doing and helping for power users who are willing to write code inside the project or using its interfaces and using its internals? So you wanna have helpful docs. Docs are your first line of defense when someone has a question and they're your best chance to win advocates, right? So assuming that your code can provide user value quickly to somebody running it, then you want your docs to explain exactly how to make that happen so that there's no room for confusion. Ultimately, you want your docs to be inspiring, right? So the parts of your code that are geared to programmers also require marketing to them. So when and why are your libraries the best choice? Why does the system work the way that it does? Why is it really good and if it's really good? What improvements are on deck and coming? And how does your project stack up against other projects that claim to do similar things? On streams, we've constantly been asked how are we different from Storm or how are we different from Spark Streaming? And there was a time when the project was attempting to and did implement some of the sorts of core features of those platforms in order to pass data around. But over time, we decided to stop doing that and focus on our bread and butter, which is moving data to and from various APIs while exposing standard APIs and data formats to the programmer. So it's important to have pages that explain how you differ from the usual suspects. So aesthetics, often an afterthought, but are your docs visually appealing? And they might be ugly. A lot of the tools that we use day to day, especially as engineers, are just ugly, right? So we get used to it, but isn't it just a breath of fresh air when you find a new tool that isn't ugly and is beautiful? Basically you don't want your new users thinking like meh or ick as they encounter your project's branding for the first time. You have them thinking, ah, okay, like this looks nice, I could get used to this, I wanna send this to my friends. So there's acceptable level, right? I would say, but even if you're using tools like Apache CMS, try not to settle for just like bare bones, very basic white background, same icon as every other project, right? You could pick a platform, there's several that are in use and supported by Apache and that are, or where you can find other projects using platforms that have header menus at a minimum, right? That allow for custom styling and theming so you can pick color schemes to match the iconography that you use. You wanna make sure you write frequently asked questions, getting started guides, examples, that sort of thing. So the question is, does the ASF have resources for designers to help projects? I'm not seeing any hands come up, but I would, if we come back to one of the themes from the talk this morning and a later theme from later in the day, it's not just about programming. So you can use the, we would like a better design as a way to appeal to people who would otherwise not join an open source project. They have skills that can help you out, don't be afraid to ask designers or copywriters to help with your project and make sure your project content actually explicitly invites participation from people with those skillsets. That's great, yeah, the comment was, just like you have tickets for what you wanna accomplish in your code, you can have tickets for updating copy, for creating logos for any range of non-code related tasks and list those and use that inventory of things you'd like to have happen to help motivate people to join, to try it out and to realize that you're not looking for a huge commitment necessarily, just for progress on specifically identified tasks that are gonna be appreciated by the community. Ultimately, you want your documentation to be really engaging and this comes from investing time in themes and style sheets and platforms. And it's important, it affects your reputation as a project, even if you think that it shouldn't. So, leaving aside the utility of your project to programmers or users, humans are just more likely to share links to pages that we enjoyed looking at, right? It's just human nature. So, if you have an ugly side, it's less likely to get retweeted than if you have a pretty site. And as it happens, web crawlers actually care about these kind of style things as well. Any questions or comments about documentation? The next and final grouping that I had planned is to talk about your team dynamics. So, kind of the three top level items to talk about with team, I think, are team presence, right? Team coordination and team planning. So, present, so is anybody actually reading these messages that I'm sending? And sometimes the team can feel absent, especially when it's small. Everyone is on a specific day or week or month that seems busy, disconnected, or unmotivated. And you're gonna hit periods like this, right? The way through it is you have to find specific topics that will spur discussion and specific goals that'll galvanize people to wanna reach them and get commits moving. And then find another one. And then find another one. And then find another one. That's activity is what breeds inspiration. When things are happening, people want to be a part of what's happening. They don't wanna miss out. And they don't wanna lose their chance to weigh in on the direction that the project is taking. The team can just be distracted, right? So even when everyone has great intentions, email threads and pull requests can take longer to wrap up, right? Then we might like, and I'd say, sometimes prompts offline to your buddies via Slack or text or phone, right? Can keep things moving. Like, hey, I really just need a plus one on this thing. I want you to take a look at it. Here's why I think it's important. Can you get this to me by tomorrow, right? You know, worst case we do, there is lazy consensus. It's not what you wanna do, but sometimes the project can't suffer and become static for want of active participation by the team all the time. Again, you want your team to become engaged in everything that's being worked on by everybody, or at least paying attention. Activity renews interest. Try to, when you talk about the project on the list or as you advocate for it, as you try to bring people in, keep the focus on short-term milestones, medium-term milestones and long-term milestones, right? Here's what we're doing now. Here's what we're doing next. Here's where we're going. And get buy-in on those kind of ideas which you can then tie back to specific issues that are gonna make it happen. And always be recruiting for new participants. Going to conferences like ApacheCon and just talking to people about what they're working on is where the most participation that I've been able to kind of drive for the project has come through in-person meetings. Sometimes it takes more than reading the front page of a website to kind of get someone really interested in something. You have to be willing to talk them through it and spend time listening to all of their questions and concerns in order to actually build a strong advocate. So the issue of coordination, and this is about like, how is the work actually getting done day-to-day? There can be work getting done, there can be participation, but what about if people are mostly working in silos or even if they happen to be working at cross-purpose at a certain juncture, right? Or if people are more focused on getting code merged fast than getting good code merged first on the first try. This comes down to integration. So I would suggest to try to break out of those patterns when they come up, try to reach a quorum among the list about where to focus next. You can start with the upcoming release. What are we trying to do in the upcoming release? You can create an agile board if you're using Jira. You can order your backlog when someone asks what to do next, right? The default answer can be go look at the backlog. And group issues into upcoming release sprints, even a couple releases out. Also, as a team try to assess the difficulty of new features and debate the assumptions behind why those features are needed and how they would be assumed to work. And the implications of taking on those problems actually before you start coding. So some projects will use design proposals on Confluence to do this. And then use the mailing list to actually kind of explicitly express the intent of the team to follow those design proposals and debate where they're lacking. So I think a fully integrated coordinating team, right? The implications of every feature request discusses is discussed and there's an appropriate kind of implementation and testing plan going into the work. A fully integrated team would treat every new feature as an evaluation of the pre-existing patterns and the project, right? Or think about them as an opportunity to implement something a little fresh or a little new. But using a design that's already been carefully considered, not just treat each new feature as a thing we're gonna bolt onto the side. It's gonna be added in a reasoned way. A reasoned way on which there's hopefully a consensus of the team that cares specifically about that part of the code base. So then planning, so how are you actually deciding what to do? Some, you might just be doing random planning. So code submissions are coming in ad hoc. Sometimes they aren't associated with tickets where the tickets are low quality. And if projects in this mode, you could look at the email threads and they'll be debating why things are the way they are. Or whether they ought to change. I think some of this comes back to having built consensus and having created good documentation. So a team that's planning in a tactical sense, not in a random sense, will be collaborating together to get tickets resolved and to get releases out the door. The decisions will be being made on the list, that's good. And then they're written up and shift into persistent reference materials. So then when a question comes up on the list that's maybe rehashing something that's already been addressed, sharing links back to that material is perfectly suitable answer. And I think the tickets need to contain enough details of the work to get done that they can potentially be picked up and attempted by anyone who's familiar with the project, not just by the person who had the idea about the change they wanted to make. But really, your team as a whole wants to be in a strategic mindset, not just the tactical mindset. This comes down to you wanna be planning your sprint objectives and your release dates in advance because of specific reasons, because the team has decided that a specific aspect of development is important to them and the other stakeholders that they're talking to. And then you wanna be doing your releases in a more strategic, major, minor maintenance type of way, depending on the degree of change that's needed to implement them. And based on whether there's breaking changes to, and how big those breaking changes will be. And you wanna get to the point where the stuff that gets merged is getting merged as good quality, not just as a first take. In the early stages, first takes on stuff get merged, but ultimately, you wanna raise the threshold of quality across the board and hold all contributor pull requests to a universally understood kind of high standard. So experience. This is I think one of the experience I mean specifically with the Apache way. So I didn't know anything about open source or the Apache way when I got here, right? Maybe your team has great coders, but they don't know anything about open source. There's learning that has to take place and the core team is gonna have to teach them patiently, both in public settings and in private settings, coaching, like the skit we just saw. It's very normal and positive for learning moments about the Apache way to occur on the list, but some messages are best delivered and or maybe best received without an audience. That's kind of a thing to keep in mind when you think about scolding somebody is maybe you could do that in a way that would come to a good result a little more subtly. So when it comes to experience, the best way is to get more experienced people working in the project, right? So you're gonna have some homegrown talent, but also add people to the PPMC who already understand the Apache way, even if they aren't gonna commit as heavily as the team you already have because they're gonna bring that institutional knowledge and that experience and that mentorship to the group. And as a working on a project that's made quite a number of missteps and mishaps, the number of them will shrink over time as the people that are observing and participating learn and start to emulate the best practices from the experienced folks that you bring into the community. So I think we have a little bit of time for questions and I think probably many of these topics will be fair game in the panel that's coming up next. So feel free to hit us with them and you'll get more opinions than just my own, which is great, we collaborate. This is the link to my website where the slides will be going up in my email address. And so does anyone have any media questions or comments or thoughts on anything that I just talked about or about my experience going through the incubation more generally? Thanks, I appreciate it. So to address that one thing you didn't address what I just saw in the incubator board report is we have an official, unofficial maturity model for projects that the incubator is not mandated in the incubator but essentially is a specific set of criteria that you have discussion in the definition of all the code open and all license checked. So there is a model for that that we have been working on. These one guidelines, the ASF uses it and certainly a way to express that. And I had considered like getting all the different codes and kind of tying it on here but everyone should just look at and read the whole thing. On streams we're using it as essentially our guideline to whether we're ready to graduate and we have a couple issues left where we're still working through to make sure that we have a check box everywhere. Right? Correct. So what is your personal experience? So the question is how does my personal experience writing commercial software differ from open source? So I've kept writing commercial software all this time too, right? So my company uses Apache streams among other technologies to do what we do. And it's very helpful I think to have parts of what you do that are not maintained by you solely, right? So if you think about the hundreds of thousands of lines of code that it takes to actually stand up and enterprise software or SaaS application, really you're better off the less of that software is in your own repo, subject to your own time constraints or subject to your own inability to focus on things that actually are important for the maturity of the software stack. So my take has become the more open source that you use in your project and in your commercial endeavor, the more quickly you're able to move all things considered and the less technical risk that you have of having bugs not be found, of having performance issues that show up that you didn't anticipate. I really strive to use an open source package first and actually I strive to use an Apache package first if at all possible for those reasons because I trust that what I'm using is gone through a certain degree of quality control as it was being built and then that it's subject to ongoing maintenance and debate about in anticipation of where the project needs to go. So I'd say 99% open source code, 1% in-house code is probably a good ratio to strive for. And if you write code in-house, the quicker you can push it into an open source project provided that it fits there and that it's already using the same patterns and libraries the better, all things considered. Okay, thank you very much.