 Okay platform is product. Thank you all for coming So this is a work-in-progress, right? We Oh, right. So first thing Pardon don't know those are verboten Thank you all for coming Would you please we're gonna do a quick fist of five? I would like you to raise your hand. Oh, nope my bad Sorry, the city of Boston would like me to announce that Know that the the locations of your surrounding emergency exits Find the ones nearest you please in the event of an emergency Please calmly exit to the public concourse area The emergency exit stairwells leading to the outside are located along the public concourse and Follow the directions of public safety staff. Thank you Okay, so no, I don't want to get to that yet Please raise your hand with a number of fingers one through five Indicating how well you feel you understand product management as Applied to a non-traditional product like a platform. So one through five and I'm gonna take your photo We're gonna do just a quick like what do you think? Okay? Get those hands up. Let's get keep them up Just yeah, and and totally totally Vulnerable there. Okay. All right, so I saw some I saw some pretty big numbers. That's awesome okay, so Pivotal customers have found a lot of value great success in managing their Pivotal cloud Foundry platforms as an internal product within their company And we wanted to enable the broader cloud Foundry community to be able to apply product management techniques to their own platforms So it came up with this MVP platform. Sorry. No Panel as product our value proposition is we assume that Delivering panel as product will result in better community product management understanding By measuring an increase in product management self-assessment So we've got the before and we're going to deliver a panel as product and we'll see the after But to deliver a valuable MVP we want to start by considering our problem space starting with our users So we're gonna do a very very brief personal workshop. So please raise your hand We we have some assumed personas because it would take very long to do user interviews and do a proper product workshop Personnel workshop, so we're gonna do a quick survey. Please raise your hand if you identify as product management Hey, we got a few okay great And that's distinct from project management. So raise your hand if you are a project manager a PMP If you are a camp to chart expert, okay, awesome. We got some of those If you're an application software engineering, you make like business apps or stuff. Okay got One or so of those Raise your hand if you're an infrastructure or platform engineering Hey, all right. Yeah, I kind of identify as one of those myself Are you an IT leader raise your hand? Yeah, oh, yeah, okay point here bosses you go And if you're in UX or design, are you wearing fancy socks today? Okay, oh cool really, okay, that's great Raise your hand if you don't fall into one of those other personas or if you feel like an outlier. Oh Yeah, sorry Hannah Okay, great. So we've got a smattering that's terrific. We're gonna try to deliver a panel I've actually got some stack ranked questions here based on personas and such I'm gonna try to make this as valuable as possible to you so that our fist to five at the end looks a little better First panelists Would you please introduce yourselves? Oh, that's me. Yeah, we're gonna we're gonna so we're gonna run through very quickly with a Who yeah, are you where do you work? Yeah, what do you do and what does your wanted poster say? Yeah, okay My name is Kyle. I work at CSA insurance in Phoenix. We're a triple a insurance provider We're a hundred year old company. So we have lots of old products that need managing Let's see what I lead the DevOps and cloud platform teams That's where who and then the last one the wanted poster So I gave a previous talk and I described myself as a as a Frustrated optimist that I'm struggling through trying to bring all the value of this platform as a product to People who are reluctant to absorb value So if I imagine my wanted poster meaning I'm gone that will mean it would just probably just say patience ran out That's probably all I would say Thank you Kyle. Jay. I'm Jay snipe. I am a product owner at Liberty Mutual Right now I'm focused on defining products in a roadmap for our commercial insurance group Also, Liberty is a hundred year old insurance company with a lot of the same pains that Kyle has my wanted poster We'll say disruptor of status quo nice Kevin I am Kevin Mackett. I am a product manager at pivotal and my primary role is helping build teams that run pivotal cloud foundry like a product and my wanted poster I'd be wanted for Breaking processes that are there for processes sake Good afternoon. I'm Sean Cury. I am the minister of chaos at pivotal I As Kevin does I go out to the customers and I help enable people understand how cloud foundry can be run as a product My wanted poster would be wanted for brevity Hey, I'm Clayton cloud native midwife at pivotal I just as those last two gentlemen I go out to customer sites and help build a platform product teams Applying this stuff in the field. It's very exciting and my wanted poster I would say colors outside the lines So questions And I'm kind of directing this at the folks that actually run their own platforms Kyle and Jay what distinguishes your platform from a turnkey vendor product? Maybe we'll go with Jay first When we introduced pivotal cloud foundry well, we've been on cloud foundry for over six years now So for us what was really important is to look at the platform and how we delivered value as a Enterprise consumer of it rather than what value it gave to us natively So what we really started to look at is the user experience for the developer when they wanted to consume the platform So how do they build and integrate with their pipelines? How do we integrate our enterprise identity solutions into the pipeline? How do you make it easier for a developer to get a vanity URL, which is painful So really taking a look at that end-to-end experience and that value comes from what we build on top of the platform itself Yeah, I mean it's it's really easy to identify The lack of value that any turnkey supposedly turnkey solutions providing the enterprises full of them from API gateways To whatever else, you know push button deploy code things are generous I should say push button generate code deploy our nightmares so, you know the I Mean, I guess the biggest value is the cultural impact it has in an engineering team right to take ownership and of writing code shipping code monitoring applications Being closer to your customer experience and being able to be more adaptable to shifting demand So in my mind that the main Value is platform as a product is that you know the the apps value is really, you know Temporal we don't know how long it's going to be valuable that particular app And so you need platforms that are can respond to shifting demand very quickly And you can change your mind on one thing the next day and not you know It's like oh we have this three-year commitment to this bit of infrastructure that we just bought but the market went that way and your host to Use a technical term your host Okay So, how do you know you're building the right thing for the right people? And I'm gonna ask Kevin this one because I think he's got a neat technique The the most important thing is understanding who your customers are so when we are working with platform teams He's a couple different exercises one of the things I like to do is go through and build out personas We do persona workshops and just understand Exactly what kind of customers that we have sometimes we look at individual developers and kind of different types of Needs for developers. I didn't experiment my last engagement where we actually set up personas as App development teams and just understanding what kind of things are important to them What kind of technologies they may want to use and use those techniques to help inform the The roadmap and the backlog Can you give us an example of a cup persona or two? so I think one an interesting one I was showing at lunch actually was the the cloud native Mavericks, so being a group of engineers that are in a in a innovation lab that they really want to have Just enough to to get going and and not have too much of a corral We had a similar kind of persona that we wanted to differentiate Which was just cloud native not the cloud native Mavericks and those were folks that Wanted to know kind of a specific traditional way of doing things have a little bit more informed opinion and Understand what was wanted from enterprise architecture and and what had happened traditionally in the company so they wanted a little more guidance and Helped us to find things like what you know giving them an account on the platform and providing them with the CI CD framework and giving them an example spring app that was Opinionated to the specific customer so as as a starting point Awesome, so yeah, it's just a helpful exercise to keep up on the wall in the in the room that everybody's working in Hopefully all together to just remember who your customer is Just the one thing that I really like To ensure we're building the right thing is our demos Right get get your product out there show people what you're building and get them all into the room So that they can give you feedback immediately Awesome. Yeah, keep those feedback loops tight And I want to throw one more thing out just because you had mentioned MVP earlier the concept of a minimal viable product It's you build just enough to get your customer in the door So we like to come up with thin slices of functionality Get it out before we've invested a ton of time into something and get our dev teams in in the platform For whatever the the new feature or whatever it is that we're providing and really confirm our assumptions So we want to make sure that you know, we're not going off on some You know six-month project to build something and then it's not at all Well, my favorite line is that's exactly what I asked for and not at all what I need So we just need to need to get the customers in the door as quickly as possible You know, it's almost like we practiced this in order, but we totally did that's a perfect segue. Thank you, dude So Jay How do you measure that success what You know, how do you determine the success of an MVP or you know delivery? Do you how do you measure value delivered? How do you gauge sentiment? What what's what is what is? Measuring success look like for you. Sure So we really started to look at data-driven decisions and starting to measure things incrementally as we deliver different value so for us a real quick use case here as we first started looking at the user experience and really when we talk about a user from a Pass perspective, it's our developer. They are our customers. We need to make that experience good We need to keep them working on business problems and not platform and enterprise problems So when we first measured the amount of time it took for a developer to come to Liberty Get the necessary IDs and access they needed to start building and deploying on to our pass pattern It took about 22 days We started looking at how can we start to cut that time down because our ultimate goal is to be down to Deploying on to pass in a single day new developer. No identity Just all they have is really an organization that they exist in how do they start deploying? So we started with identity. We started to introduce secrets Into that same pipeline Although there are different teams supporting that they all come to the same Console to start that process. So they're not going to one request to one team and then one request to another There's a single point where they start and they really end there as well So we've measured that down and over the past year. We have gotten it down to a day for a simple use case We can deploy new user Interproduction in a day. That's amazing. That sounds like a startup Okay, so Kyle what is measuring success look like for you? Yeah, a lot of the same a Lot of the same measurements there I'd say, you know From where our engineers are coming from the the bar isn't very high. So that you know, it's It's kind of easy to beat but we want to challenge ourselves, right? So and obviously the platform unlocks a lot of that just kind of nature of the design but you know the ecosystem we've put around it is important to us So there's probably two dimensions that are most important to me the onboarding process for teams So, you know as we onboard new application teams, you know We're literally kind of you know starting a counter and seeing like how soon are they getting you know They should be in slack the right slack channels They should have the right access to you know PCF spaces and users and all that type of stuff And so you know a pipeline should exist even if it's a you know Just a templated app in in the in the build pack They're gonna use or whatever like, you know, we want that to exist out, you know in a day, right? And so we're we were we're measuring that and then as that You know pipeline expands as that team starts, you know their environments proliferating and they're getting the prod You know, we measure that time from the package build time So engineer commits code package is built to the till the time it takes to land all the way in prod. What is that? build to prod KPI for us Our most optimized pass of 38 minutes for you know from the build all the tests to run Deployed through all the different environments and into prod 38 minutes So then we baseline that for that team and and there's a little you know monitoring alerts on that if if all of a sudden It goes up to an hour, you know, people are getting slack alerts like what why did you inject friction on purpose? Is it is that we're getting value out of it or is it you know just? You know noise So yeah, those two dimensions are what we pay attention to awesome So I'm gonna I'm gonna cram this last question and was to and I'm gonna make it one If we could pass the mic back down, we'll start with Shawn this time What do you think is maybe the? No, I'm just gonna skip to that one If somebody came to you and asked hey, how do I even platform as product? What's the one thing you would tell them to start with? t-shirt T-shirt awesome, okay Kevin t-shirt brandy. Okay. Hi. I would say start with knowing who your customers are. That's Instead of building what's interesting or looks good on your resume or whatever to like Know who your customers are sweet I guess I would say from an enterprise perspective where we know who the consumers are of a product like this I would start with what are their pain points so you can start solving those for them first Um, maybe I'll I'll tackle it from another angle. I'd say investment is not is a Difficult it it's a new form of investment that enterprises aren't used to You know they well at least our enterprise is very project-based investment, right? And so you like go build this thing and but we're talking about a platform that enables to build all the things and So it's a different form of investment that it takes and it's a different makeup of people, you know to know your customers that will cause a particular type of Person to be able to do that management and it's of engineering background to be able to navigate You know the demand is it like out of left field your request from application teams? so there's a lot of unique roles and and An investment challenges with with just calling it I mean making stickers in the t-shirts Yes, but then I also need to hire people to staff that product in a way. That's probably pretty That is new for our enterprise, and I think a lot of enterprises Okay, so audience questions, please raise your hand and I'll bring you a mic Hi My question is around the minimum viable product How does one determine that and you know when we talk it when you're talking about rolling out slices of functionality? in terms of legacy Infrastructure you either had an ESP that worked or you had an ESP that didn't and there really was no layers of it same for an app server or a Database so I'm trying to wrap my head around slices of functionality And I'd like to ask whatever panelist answers to try to keep it brief So the things that we think about when defining MVPs are What's the body of work? That's the least amount that we can do that delivers some value So how do we get our app team into the platform? Even if it doesn't completely work, you know, we tend to look for folks who You know are more interested in kind of being on the bleeding edge obviously But can they like for an initial stand-up of the platform it might be? That they can push an app, but they don't have You know a lot of the services and things that they need installed, but just kind of confirm okay, we can get an app in and we also consider functionality and Trying to exercise risk as early as possible if we've got something in a project. We're looking at where like this could be a problem we want to try to Exercise and mitigate that risk as soon as possible And another concept is we like to try to have parallel work streams as well But to go to the specifics of your question It would really be that instead of doing like a full-blown High availability kind of situation where you can push to production and have everything ready to go It's doing it in like a sandbox environment or something first and we're gonna do Kind of getting just enough to be able to get someone in Exercise it may be a very specific set of use cases knowing that you know, it's not gonna do everything that's needed for the whole Enterprise it's not going to be production ready to go It's just getting them in and getting feedback from them and testing those assumptions because it might be You know something like if you're gonna implement some kind of a new mess messaging queue or something like that and you Go and build the whole thing out and say okay. We're open for business. We're in production in It's like well. Yeah, that kind of doesn't do exactly what we need or you know, it's a problem It's getting just enough in just the environments would be one of the one of the recommendations and great job That's just enough of an answer who else? Hey, so How did you guys gain these skills? Is there any? Sub community within the cloud foundry community for product product management. Yeah, I mean, I don't I don't know that There's any sub community. I'm very lucky that I work at pivotal and I have wonderful people who've been doing this at labs for a Long time teaching me how to do that. I know we have a product managers playbook I Don't know if that's public, but I will take a look and if it is I will post it up on Twitter And if anybody knows about sub communities, it would be Sean Kerry You can also John Cutler has a great product manager you could follow on Twitter And I think he has a video about Applying product management to an invisible like an internal product Who else? Yeah As you were talking you gave me an idea of another question one is in product management How do you view competition and prepare yourselves for it and the other is? We talk about failing fast. How do you prepare for backup recovery scenarios? Is it different than in the non-pass world? Cool. I didn't the first part of that question. What was the first part competition? Yeah, so Yes, it's kind of a hard question to answer because I think competition exists at an enterprise Among other projects or products. So it's a matter of making sure the right Conversations are occurring when a team is making a decision on whether or not for example, they're going to deploy to Pivotal or PC a cloud foundry or if they're going to deploy to something like docker So it's really getting the right folks in the room to have those conversations to make the right decisions For the application itself. I guess that's where I would look at competition from an enterprise perspective is Understanding, you know, or do they even just deploy on AWS for example on an EC2 instance? And then from a business continuity perspective, it really depends on how you manage and build out your platform itself From a cloud foundry perspective, you could run it across multiple regions in AWS You can run it across multiple regions in your own internal data centers and run things active active or Have a you know pilot or a warm backup to go to We do both at Liberty. I could talk a little bit about the competition factor so There is that in I encourage it like if you want if you want to deploy on something else be my guest but I make the barrier to entry for PCF very very little right and Now the barrier to entry to get to prod is much more rigor But just getting them in the platform getting deployed cloud very very little and what I've found is Combination of the of the value That they get out of the platform The business value it provides the ease of use and the enjoyment they have in the platform The demand drives itself right so we don't we don't really lose those fights You know with nobody's banging down OpenShift store because of the great developer experience, right? so Although it exists there, and it's a you know, it may be an option for some teams I just really put the power on the hands of the app teams like just try it and see see if you want to stay here All right. Thank you very much. I'm sorry dude. That's a minimum minimum viable panel right there Minimum viable panel to be an advocate for the platform So if if I could get a a last So we have a before fist of five We're gonna grab an after fist of five how well would you say you understand product management as applied to a Non-tangible product such as a platform raise your hand with a number one to five fingers product management For a non-traditional product Okay, I saw some numbers go up good deal. Okay, so I'm gonna I'm gonna I'm gonna early declare it maybe successful Thank you very much. Oh Fancy that's supposed to be a thank you hands Thank you very much to the audience for help us to helping us to deliver this 1.0 of Panel as product. Thank you distinguished panelists for contributing your knowledge about product management applied to a platform and If you would like to continue the discussion please remember that roadmaps as any good product manager knows our conversations not commitments, but We're road mapping to present 2.0 of this panel as product at spring one platform By pivotal in September in Washington, D.C. And if you were to use the discount code up on the screen here Or in the slides attached to The presentation on the website you get a discount to come to spring one. Thank you very much