 Cool. Okay. Hi everybody. Go for it. Thanks Chris. Let's just jump through to the next slide please. Yeah, next slide. Next slide. Okay, great. Next slide. This is the schedules have been published. Chris, do you want to say a few words about this or Taylor or anyone else who's on the call? No, they're just both live for Asia and North America. Also including some co-located events like EnvoyCon that we're putting together for the community. So just wanted to raise awareness that all these things are out. And more importantly, if you go to the next slide, we have the dates for 2019 locked in for our three major events. So feel free to pencil those in your calendar. So any questions on that? I have a question. I was involved in the program committee for Copenhagen. And one of the things that was a lightning rod issue in the selection phase was whether there was a double blind selection process. And it was promised at the time that that would be used next time, meaning later in the year. Was that used for the Shanghai and Seattle conferences? And if not, please can we use it for the next conferences? Alexis, it was not promised that we would do it. It was promised that we would discuss it in detail. And the co-chairs, Liz and Rice and Janet Kuo, discussed it in detail and decided that they did not want to do a double blind selection process. And so I would encourage you to email them or me if you want to discuss it further or we can have it on a future call. But I think there's a lot of downsides to a double blind process. And it's actually extremely rare in a comfort world and not something that I personally would support. So a bunch of questions. One, was that decision in the rationale written down anywhere? No. Okay. And I am strongly in favor of double blind. It is done effectively everywhere now in computer science academia. Computer science, academic computer science had some of the same arguments against double blind that you probably make. They have since moved to all double blind and they've seen much better results. So there are a lot of arguments in favor of double blind. And it's something that I think we should be considering as a TOC. I really don't think it should be up to each individual conference to decide whether they are double blind or not. If we are going to not be double blind, that is a decision that we should make for all of our conferences. If we are going to be double blind, that's a decision that we should make for all of our conferences. Okay. Everybody, let's not make this a debating topic for today. I was very curious about what the process was this time. I too, am strongly in favor of double blind, but I don't want it to be a discussion point for today. So Dan, thanks for explaining that. We shall follow up on this for another occasion, please. Could I just request that, Brian, if you could send some links to that process and the justification for it to the public list, I'd be happy to film. Yep. Sure. Absolutely. Right. So next slide, please. All right. So this is very important. And I'm sorry that not everybody who's normally on the calls is here today due to flights and stuff. But we had a bit of a huddle after the last TOC call because we hadn't had a proper, you know, private face to face for some time. And, you know, some topics that we discussed in the last one came up again. And there's a strong feeling now in the TOC, now that CNCF has gotten bigger, that we need to change how we work. And it's been a problem for a while, but it's become an urgent problem. And so we have the nine of us with votes essentially decided to make a few changes to how the meetings operate and a few other changes as well. So we're going to change how we build the agenda. And we're also going to require that minutes be taken. And the way that we're going to do that is very similar to how, for example, a Kubernetes sync would work or an open source project might have a weekly face to face hangout, which is to have a living document. You can see an example linked to here, TOC public notes. And in that document, you take, everyone can contribute an agenda if they have right access to the document. And the agenda will be shaped a day or two before the call by, you know, the chair and folks like Chris and Dan and so on. And to create, you know, some structure and order. And then we'll take notes on the same document during the meeting so that we can then have a system of record for what was agreed. And so on. Thank you, Doug. I see you there on the chat. We will use this in the next meeting. Okay. Not today. We just didn't have time to get there. I would really appreciate it if one or two people who either are TOC contributors or part of the executive staff or TOC voting members could just help out to get this way of working up and running. Otherwise, Chris and I will have to do all ourselves. And we will go a bit crazy or crazier in some cases. Okay. Does everyone understand that everyone's done the goals and purpose of this? It should be relatively familiar. This price is not nothing radical here. Any questions? Awesome. Please can we go to the next slide? Okay. And we also want to change how we work. We have had a lot of meetings where we sit through presentations. And in addition to that becoming a bit samey, we feel that valuable time is being squeezed out by that process. And we should spend that time in a more focused manner on some other things. Which means, of course, that we need another way of doing the real-time presentations. Some of them will be moved to non-real-time. So we're probably going to drop the real-time sandbox presentations. But keep the ones for incubation and graduation. And we want to divert the time available during the meeting to focus discussions, aka working sessions. And we'll have the first one of those today around the topic of end users. So we'll get a chance to do an alpha round of this approach. And then either the next meeting in two weeks time will be around the topic of helping projects, which appeared in the community not for the first time a few weeks ago. And there was some discussion on the mailing list about it. And Chris has created a document. And I think we can work together towards some ideas there together. We also might change the cadence and the length of the meetings. Please bear with us while the TOC members figure out exactly how we want to spend our time on this. Don't be alarmed. The purpose is to scale better. It's not trying to exclude anyone. We're actually trying to make it more inclusive and more scalable. But it will take a little bit of time to figure out how to get there. Does anyone have questions about this? Could I make a request or offer a suggestion? Yes. This may have happened already, but I would propose that projects, for example, that are coming in for sandbox both, really need to not just throw 30 minutes of YouTube up, but actually need to write a document that has some specific format and is kept relatively short enough to digest. I think we will get a lot more review if content is formatted in a consistent written way. It's a lot faster to read a good document is to sit and watch a presentation on my video or stored video for 30 minutes or an hour or whatever it ends up being. Thank you, Bob. I can not agree more. I love that. I love that idea so much. Please, let's do it. Bob and Camille, could you send me an email or send it to the TOC public list? I don't mind which. Just a few bullet points or things you might think should be in such a document. We're in the process, as Chris is pointing out here on the chat channel, of making some updates to some of these processes, especially around sandbox. Just please, please, please send suggestions to the public list or to me and Chris and whatever. Just send it. Send it. Send it. Thank you. We'll do. All right. Anything else on this slide that I've missed out? No, I don't think so. Okay. Please, can we go to the next slide? Okay. This is my grand moment to reveal that I'm sitting next to Cheryl. Hi, Cheryl. Hey, Isis. I know everybody else on the call. Would you like to introduce yourself or would you like Chris and Dan to introduce you first? Or both at the same time, maybe. Chris and, do you have any questions? No, go for it. I mean, we already published a blog post, but happy for you to take the lead introducing yourself. We're excited to have you here. I'm excited to be here. So, if you haven't read the blog post yet, this is my second day as director of ecosystem. And my mission is to make our end users more successful and productive and help the community ensure that it has some feedback and get feedback between the projects and the end users who are using them. So, my background is as a product manager at a container storage startup called StorageOS who are a CNCF member. And as an engineer at Google, where I was probably the best example of an end user myself and I was a developer who did not understand Borg very well, but was still forced to write code on it. So, there we go. Yeah, that's me. If you've got any questions, then let me know. Does anyone have any initial questions? We've got a few more slides on this topic, end users, by the way. I have a quick question. This is Matt Farina. End users can often mean a lot of things to a lot of people. Can you give some clarification on what we mean by end user here? Matt, if you don't mind just because we had to hone in on the definition over the last couple years with our end user community and such. And I don't want to claim that this is perfect, but it's worked well enough, which is companies that don't offer cloud-native services to their customers. And so Pinterest is a perfect example of a company that's using Kubernetes and other cloud-native technologies internally, but they offer the Pinboard platform to their customers, as opposed to GitLab or Google or lots of other folks who offer something. But I don't, where it gets a little fuzzier is we've sort of arbitrarily said that telecom firms are not end users. And then there's been some other ones that have been somewhat close calls. Do you have anything to add to that, Cheryl? Would you like to propose an alternative definition? I've thought a lot about this myself, because I come really from a developer background. I'm very involved with the meetups, and so I speak to a lot of DevOps engineers. So far, the definition of vendors who are not selling cloud-native services makes sense to me, but I wonder if it's broader. Is it a particular type of role within those places? Is it more engineering focused, or is it? I don't know. I've spent a lot of time thinking about what exactly end users mean. And you know, when I think of end users, by the way, for those of you who don't know, I do SIG apps over on Kubernetes side. So I deal with end users a lot, and I work on Helm, which is, so when we think about it, we're trying to hone in sometimes on individual types of people to figure out what do they need and how to set them up to be successful. And so, you know, it could be organizations, and that deals with organizations and management interactions. But do the engineers and the people themselves on the ground have what they need? And sometimes that's a different question, and it has different needs. And so that's some of why I was asking, because I'm curious to figure it out and see where we're at on this. More just because I was curious and wanted to know that had any specific opinion on what we should do. I think my personal bias is more for the engineering and developer side. But I'm also aware that there's a lot of work at the organizational level. So I think I'll have to work out that balance. And I would just add, sorry, Cheryl. The end user, oh please. And also whether we're talking about end users who are CNCF members or whether that includes end users who are not CNCF members. It's a good distinction to make. And there are also end users of other foundations like the Cloud Foundry Foundation who are associated with the Linux Foundation, therefore, but aren't necessarily in policy so there's all kinds of people. I was just going to add the two places where the definition has needed to come up the most are one on our end user community where we do require that they not sell cloud native services. And the thinking there was to try and essentially keep the foxes out of the hen house. And so that have to have the end users who were potentially customers of vendors to have a place where they could communicate bluntly but in a vendor free environment. And then the other area, so all of those people are CNCF members or supporters by definition, but the other area where it's coming up more and more is in submissions to QtConClubNativeCon where we do ask people to check a box to say that if one of the speakers is an end user. And by that again, we mean not a vendor offering services. But this stuff isn't perfect. So I think we're very open to modifying the definition particularly now that we have Cheryl on board. All right. Please can we go to the next slide. This is a screen as showed that you may have seen before of what was the feedback from the TOC in June and early July leading up to the GB off site. And this was the slide that I presented summarizing that feedback. Cheryl, do you want to say a few words about what your thoughts are on this initial feedback from the TOC? Bear in mind this is somewhat random and I speak as somebody who is very much aware this is a day two for the new role. So any comments and then we can go on to a bit of a discussion session afterwards. So I think the ones that resonate most of the conversations that I've had are the first three. So understanding why end users use the projects in the first place beyond the there's always a there's always a reason or purpose beyond because we want to use this technology. But the end users all have different ideas of exactly what they want. And from those they need guidance and templates on how to prove those things and then how to go how to actually execute and implement them. I think there's a feeling that there's a lot of projects and people don't have the time to investigate how they all work with each other. So that's something that I plan to spend a lot of my early time early time in the next month or two talking to end users and understanding what their goals are and what they are missing in those areas. Okay, I'm actually going to ask that we go to the next slide please. Thank you. So I was hoping that now it's 23 minutes past the hour just for the next five or 10 minutes we could solicit some feedback and maybe have a conversation between Cheryl and perhaps others from the CNCF executive team as well if need be around what are some structured activities that people in the TOC community whether you're a voting member or just somebody who does in once a month would like to hear about from end users or feel or is important. Personally I think one of the most innovative aspects of the CNCF in terms of its governance is that Craig designed it with the idea of three useful bodies in mind which the TOC and GBO just two and the end user group is a third and it's been somewhat difficult to make the most out of that without full-time people helping to kind of move things along and get information out of folks so this is a great opportunity I think. So a question for members of the TOC is how have you been communicating with end users so far if you have? Has it been informal or is there any expectations for how you would do that? I mean I am not the end user rep but I am basically in the role of an end user but the team that I run it does not produce vendor software for cloud native so you know I sort of live it in day to day as well as just being kind of heavily involved in the community so you know that's sort of how I personally stay involved with end user stuff. This is just kind of add to that like the two areas that I think are the most important for the TOC to be involved with end users is in either the gaps or the some of the issues that we're running into in the end user space with implementing cloud native technologies is one and then the second area would be some of the projects like I know here at MasterCard we're working in like some unique security areas that there are no solutions in cloud native today and we want to contribute that back to the community but we don't want to do it as a heavy-handed like you must do it because MasterCard did it way but more of a this is like really good for the community to look at and we want TOC input on on that does that make sense? As a project maintainer I would love some kind of opportunity at the KubeCon to have a meaningful discussion with some end users. So when you say a meaningful discussion you're talking some kind of forum or a track focused on end users? Yeah some kind of forum that can involve direct interaction I'm not thinking of a track so much in the sense of talks there are good end user talks at the conference or that have been in the past but more of an opportunity for project maintainers and end users to discuss the you know how end users are using the CNCF projects and where the friction and gaps are especially in using them together but also just independently. So this is this is Matt Ferrain again I'll just pipe up you know I know some projects are proactive in finding who their their big users are and going out and sitting down with them and interviewing them and having conversations you know some of the projects will actually do that but I bet you a lot of maintainers on a lot of projects are engineers and aren't those folks who organize and get around and sit down and get good actionable feedback from their end users and so for all those who don't organize and it might be nice to have some way to have that helpful for us I think is a way to put it. What sort of formats do you think that should come in because as coming from product management background I've spent a lot of time thinking about how to take lots of unstructured feedback especially as we're not working with three end users there's already 50 or 60 companies and putting that in a structured way so that we can communicate and we've got the same idea about what everybody needs do you have any thoughts on what format we'd like to see that in you know I've got a thought again is this something we could actually go back to the end user leaders on and ask them what format they'd be up for presenting because I think we've got a bunch of folks here work for vendors and things like that and maybe letting the end users to a certain extent drive the conversation might be helpful because then we vendors aren't trying to push it into a way that fits into what we're already looking for it's definitely going to be we're going to iterate on this for sure I think both sides need to understand or have input on the best best way to communicate user needs so if you have I would definitely go and talk to more of the end user leads as well but if you have any thoughts now I'd really appreciate I mean I will say that I wish cube con had a lot more end user talks I am I you know I am basically telling people so I'm sending a lot of people to cube con and I think the people that I that I'm sending that work directly on kubernetes it will be somewhat useful because they need to know you know a lot of what's in the ecosystem but if I were you know it when I have people that are like I want to learn more in general about you know cloud native architecture I want to learn more in general about you know when people actually use these things the experiences they have I think we are way short on covering that and that is you know like the real experiences of people really using these products in anger in prod and having problems is the most interesting thing that end users can provide and we're just not I don't I just don't feel like we we get a lot out you know I don't feel like we're highlighting that that enough in the conference in the conferences right now at least yeah I agree can I add to that as an elaboration that not only in the conferences but also in these to see meetings it'd be nice now and then to get it kind of round up you know necessarily a long spiel but 10 minutes just to give us an update on new things the end users are talking about right now yeah definitely heard yeah I would also add that when you are looking at users at cube con you have to you're getting a certain kind of user that may not be representative of the broader user that may have different issues I would liken it to talking to customers in your own customer briefing center whenever you if you work at a big company you've got a customer briefing center whenever you're oh if you only talk to customers in your own customer briefing center you are getting the customers that are invested enough in you to travel out and everything else you actually want to get those customers who are in this case users who are not invested enough to make that trip because they've got they see some really serious barriers that haven't been addressed and I would encourage us to play some more away games and to go to dev ops conferences to go to programming language conferences where people who are actually trying to get shit done actually collect and have some kubernetes have some some buffs have some some meetups whatever you want to do at those conferences and you will get a whole different I think invaluable perspective on where the project could go to address concerns that might not be being addressed right now whatever the opposite of preaching to the choir is I'm Kosuke and I run another open source project thinking tonight and I do two things that I wanted to share the one is when I go out and travel I try to just go visit some random companies and then ask them how they are doing these open and they surprisingly opened up to a lot of things and then I capture that then bring back to the project the other thing we do is we ask people to come to this day which like a lot of people from our side show up and then they share whatever the challenge is and you can usually have like a dozen or so different folks coming I find that these privates are that really help them open up a little more so in combination with those things like the funded thing that very good to be valuable they had to your point they did you do capture different kind of audience yeah I totally agree with that totally agree with that I think the end user community but there's the monthly calls but that's about it I don't know that there's much activity between those calls that happens perhaps if there's any anything that I'm not aware of there then feel free to enlighten me but improving the communication channel so we're not talking to the same people over and over again or only people in a certain geography on certain verticals because those are the people we already know I think that will help the other thing that I see from running I run the cloud native London meet up so I speak to a lot of developers and engineers and DevOps people in this space is there's a lot of projects and there's a lot of confusion about how why what what's the value of having lots of different projects if the CNCF ends up with a thousand projects then does that help them I don't know the answer to that but yeah so there's a there's something that I thought about which is playing more of a product management role across projects obviously each project has its own product management team and requirements and so on but as end users you want to have feedback across multiple projects and that's a role that I can have some input into that'd be great I think so anyway would you be able Cheryl to attend these coc calls regularly do you think that's something you plan to do yeah definitely great definitely would it be would it be possible to actively contribute to the agenda from time to time and just say give a sort of short update or I mean I don't know what the topics are going to be but be great to hear from you yeah regularly I think every once in a while hearing what end users are talking about what the trends are that they're seeing and what they're excited by and will be a good actually another thing that we could do is so I was lucky enough to be invited to an end user call not too long ago and I saw a really good presentation from I think it was the University of Michigan that Chris had organized for that call and you know it'd be nice to show that one again to people on the TOC meet I mean I'm not saying we should do that every time but now and then that would be an example of something we could do yeah definitely this is a particularly interesting use case yeah I think end users often think their stories are not worth talking about they're not interesting enough to talk about so it's something that I care about a lot is making those stories heard excellent does anyone else have any other questions or suggestions for this topic area for today I've noticed that Cheryl's taken very copious notes mostly for myself but I've been I've been writing down all the things that I've been hearing and thank you all for your comments does everyone have your contact details so we can spam you with further suggestions I think Chris put it on the chats okay my email address should be see hung there it is yeah it's in there good good yeah while my inbox is empty take advantage of it right excellent thanks DC right next slide please okay so the idea is to do another session a little bit like this maybe a bit longer actually next time covering helping projects as a theme and I'm not saying we'll only do this on the call but I want to touch on another theme together so this is a topic that's come and gone lots of different people with different points of view I'd like to make it very clear that Chris and other members of the CNCF staff do a lot of kind of below the waterline work that we don't always see personally I'd really like to hear more about what that is actually but as a big picture item I do know that it's a recurring question from people how can we help projects more and I hope that we can go through it in detail in the next in the next session Chris do you have a document that we can add further suggestions to I know you started compiling some thoughts from the mailing list discussion there yeah one second I thought I linked to it but here is it in the chat thank you thank you very much Taylor would you mind adding that link to the document please there's the slides so people can see it okay let's just jump to the next slide please thank you so Chris you're the person who's been doing a lot of this could you talk us through a little bit about what you've done personally also with other members of the team how many interactions there have been and generally give everybody a sense of what's the state of the art in terms of working with projects so we can we can move on from there yeah sure um you know basically the way it works is when a project gets on boarded to become an official cnc of project whatever level they are they're basically asked if they want regular meetings with staff essentially do a monthly you know 15 or 30 minute meeting depending on their needs I would say a little over half of projects essentially opt for the meeting the others generally say they're fine and we'll reach out if we need anything recently shared a calendar with the TOC in terms of those meetings if you wanted to actually join one of those we're structuring and doing well so we have a service desk for anything that projects require asked from funding and outreach interns to documentation help migrating a website from you know you know whatever WordPress to Hugo or whatever so we probably accomplished about a hundred or so tickets through that over the last year in we recently did a survey in the first half of the year which I linked to over here which kind of discussed thoughts from maintainers in terms of how the staff was doing in certain asks so we published those results there we're about to kick off a survey later this week for kind of the second half of the year so we could have results ready for kind of kubecon timeframe other than that I think that's basically it so you know regular meetings check checkpoints service desk regular surveys cool so I have two two comments on that first well first of all actually thank you very much and just to remind people that you know this is a topic where a lot of effort goes in from different sources we don't always see it so I think it's really important if we're going to discuss it in detail that people start from appreciation of what's already being done my two comments which are shown here are number one I think the service desk is okay it's clearly working if you're getting a hundred requests through it but it's it does still strike me as a little bit on the passive side it would be good to come up with specific proposals and solicit feedback from the projects and maybe other communities and users the toc etc around that and also with the survey you know I think there's there is survey was pretty good but it did feel a little bit like a glass half full or at least maybe two thirds full one third not so full you know you can interpret the results of the survey in different ways and I think it's really important with these surveys to ask very direct questions and my number one question that bothers me is if you spoke to a project lead from the CNCF project would they recommend other projects join the CNCF and if so why would they kind of shrug their shoulders and say well it's been okay it's a neutral home we did add that question Alexis by the way for the next survey going out yeah and if we can get the the high profile and well-known figures associated with the best projects being the loudest and clearest and best exponents for the projects then I think that will help a lot with all kinds of other things and use a happiness etc that's just my view does anyone else want to chip in this is not a major topic for today we'll discuss it next time any other things that you should be aware of in preparation for that yeah strongly agree to Alexis that should be our guiding metric is how we're doing by our projects okay thank you anyone else anyone else right next slide then uh we haven't quite got over the line on this but we did also discuss offline some um some ongoing concerns about the sandbox and marketing um I think a lot of this is just kind of a side effect of the CNCF having become a more successful and bigger organization than anybody who truly anticipated when they got started a couple of years ago so we have to be as best as possible as gracious as possible in dealing with this but we really do want to make a clear distinction between incubated and graduated projects which we see as having getting over quite a high bar and deserving of a large amount of attention even if they're not the only project in the particular area um and sandbox projects some of which were thought of yesterday and are very very new and may not succeed so we're just trying to adjust some of the process and marketing in order to align with that um and then we also want to understand better how to you know get the CNCF to scale because the TOC has become becoming more and more of a bottleneck and we've really got to empower the community more intelligently than we have done without necessarily creating you know alternate political universes um this is going to take a bit of time to figure out but as an example you know I was asked to look at a project yesterday that does something in security and it'd be really good to send that along to the working group for security and ask them to come back with a readout on it um but we're not quite at the point being able to do that systematically yet okay so that'll be a topic for the future uh next slide please I think we're into the end of the session now we're not going to do the key club presentation today I apologize for that especially to the key club team I saw you did some really nice slides for people who want to look at the slides they are in the deck and we will revert to these projects very very soon in a slightly revised form in terms of the presentations etc can we go on please to the next slide not going to do working group dates today cover that we're going to come back to that backlog move on and then just a reminder on the conferences I think and then the last slide is the new agenda style for next time I hope everyone understands what's being requested here does anyone else want to have anything to add in terms of wrapping up for today and preparing for the next presentation in two in two weeks time the next meeting I mean okay I guess silence is a scent which is one of the joys of this organization thank you all right thank you very much everyone and we'll wrap for today unless anyone wants to dive into something else thanks alexis thank you alexis bye thank you