 I live in the United States of New York or I should say the greater New York metropolitan area. I'm in private practice. I consult. I do a lot of stuff with extreme programming, scrum, agile, and the like. In my spare time, I always like to find some new challenge to do. And my new challenge is learning to fly. So if you ever find yourself in the New York area, especially like on a Friday evening or Saturday coming in by air, you might want to consider taking the bus. Let's get started a little bit. This session, as you probably saw, will mostly be workshop. Everybody's got to contribute something here because listening to me speak is... Well, it's not that fun. Perfect. Okay. To start off with, I want to talk about the idea, the concept of portfolio management. How it comes into being. Now, everybody on here is probably familiar with these levels of planning in an agile world. Yes? Yes? Okay. Typically, when we start teaching people what agile is all about, we usually are more concentrated on things like the tactical end of things. What do we do for an iteration? What's that all about? Two weeks of work? Gosh, can we do anything at all? We talk a lot about scrum. We talk about stand-ups. Okay. How work gets accomplished. And usually, when I start off with teams and they get those concepts, they can do them fairly well. They get results. But the problem is that the organizations never really get the true benefits of agile until they start looking higher up in the food chain and all those strategic sorts of things, engaging people from the beginning so that they're not just doing the stuff right, but they're actually doing the right stuff. So we're going to be concentrating on roadmaps and releases. And we're going to apply that to a much broader situation where we talk about portfolios. Okay. Could somebody give me a definition of what a roadmap is? Raise your little pause. It's a plan of intention, yes. How's that different than a release when we plan our releases? Yes, that is true. Releases are typically going to be shorter. I'm going to make the point a lot more later that roadmaps are all about business value. Okay, there are a way that we can see ourselves going forward talking about how the business is going to operate and the things that the business needs from our development efforts. Releases, on the other hand, are kind of like on the cusp between our strategic thoughts and our tactical actions. Okay, so we're going to talk about the fact that release planning has to do with coordinating dependencies on understanding a little bit more because the foreseeability is better. Things are right up in our face. So start off with that. I also want to start off by making a really big point. Okay, and if you leave after five minutes, remember just this one point and it will make your day. Okay, the chart that you're seeing here is from Johanna Rothman's manager project portfolio book, which is actually excellent. Okay, and she has this systems thinking diagram pay particular attention to where we talk about projects. We're going to talk about the difference between projects and the way that we plan products. Okay, projects typically are much smaller. Projects are the things that we do things like release planning for. Okay, now the problem is that a lot of companies start off and they have like a budget the next year. Right, anybody in here working a company like that where they have governance and we've got so much to do. Right, and too many times what they do is they start off and start planning the way that they're going to be executing on all of their resources over the next year. They put together little cases and sometimes it's a business case, sometimes it's much more informal, and then they make commitments and they have certain dates and they carve up this stuff and they say, oh gosh if we can only plan properly, everything will come out right. So they start carving up stuff into these little teeny projects. I have a company, you've heard of this company, I don't want to state their name because there's a recorder going on here, that plans its stuff down to like four or six week duration projects where they're actually trying to get functionality done. This is usually a backlog of like one or two epic type stories. Okay, not a lot going on in them. What they fail to do is understand the issues involved in keeping this many projects alive. It's sort of like they're doing a beat-off, a big design up front as they get all their work done and as conditions change. If there's new business opportunities, if a project takes longer to do than we anticipated originally, if somebody gets sick and we don't have the resources that we thought we had, all of the work that they did in the beginning of the year falls apart. And what happens, what this diagram will show you, is that as these projects start falling apart, we start getting a lot more things going on, a lot more emergencies arise. And suddenly we have new projects which start superseding the old ones, which make the situation even worse. And the way that we combat that in an agile setting is to start thinking about products and product backlogs. So if you can remember that one simple thing, keeping the number of projects down will ease the amount of work that it takes to manage it. Just like with sorting, if I sort n number of items, it's going to take a certain amount of time, if I sort twice that number of items, it doesn't take twice the time, it's an n squared problem. So we've got to keep the number of projects down if possible. This is the counterpart to that diagram. Notice how this diagram gets kind of complex, because all the negative reinforcing feedback loops are out there. When things go well, we get all these positive feedback loops. We find things like people's time contention goes down, that we actually get more projects completed. We're able to focus better. All right. Oh, you know, one thing I never did, let me go back. Oh, this is working again. That's great. I never defined what portfolio is. I never defined that. It's partially because most of the definitions that we have are kind of poor. Typically when we talk about portfolio management in an enterprise type setting, what we're talking about are projects that are organized by a timeline of dates and the things that they're going to deliver. Our job is to make sure that we can always organize that well. Okay, so today what we're going to be talking about is how we can put together that portfolio to begin with, and we'll talk about the need to manage it in a kind of an agile fashion, where we actually go back periodically and review what we put together on our portfolio, and perhaps monthly have a portfolio management update session to take into account new realities, either from the business side or from the technical side. We're cut off a little bit over there. Okay, I have half a dozen terms, which may or may not be familiar to you. If they are, I apologize. I want to distinguish between what product lines and products are. Okay, now the examples we're going to use later come from a particular engagement that I had with a GPS manufacturer. They had a lot of product lines. They would have things where they developed stuff for automotive use. They had things that they developed for people that go out in the ocean, seafaring use, aviation use. They'd have consumer lines, they'd have professional lines, and they organized their company around those product lines. For work today that we're going to be doing, we're going to have only one product line in mind. So bear in mind this is a beginning course. It gets much more complicated as we go. Products, on the other hand, are things that are purchased by consumers. They're identifiable. They have names. Products will be sold. They will produce revenue for us. Products do not necessarily take development work. There are some products that are out there which are distinguishable in the marketplace. An example might be Microsoft Windows, where they actually have a home version, a professional version, whatever version. Windows Server 2008 is actually Windows 7. There's very little difference technically between them. Most of the difference that's between them is in the marketing. Products, the other important part about it is that products are sold based on perceived value. A long time ago I worked for a company you probably heard of called General Motors. Gosh, this is back in the days of the stage wagons. At the time they were trying to get away from having antennas that were on the fender of the car and they want to put them in the windshield. Well, it actually costs less for them to put the antennas in the windshield invisibly than to actually put the antenna on the car. Yet in the consumer's eye, this is a desirable feature. So they actually charge quite a bit for that option. So products have very little to do on the consumer side of things, the people that actually buy it I should say, with how much it costs to manufacture. And this will be a very important point when we go and start putting together our portfolios. Projects, you probably know what that means already. Those are the technical pieces of it. Roadmaps, I've kind of talked about. The term waypoint. Are people familiar with that term? Probably not. Waypoints come from navigation. They're actually a very old term. They're becoming much more popular with GPSes now. But it used to be that if you were sailing on the ocean, there might be a waypoint that said when I get to these streets, I got to make a left. A waypoint might be when you're driving, you'll know that when you go to Cincinnati, you should take I-80 now. So they're identifiable points. Our waypoints will be spots on our roadmap. So we're going to continue that metaphor. And release plans I think you know enough about. So the difference is between the roadmapping types of work that we do and the release planning sorts of work that we do. I said before roadmaps are value-based. They're how we can start planning for deriving profits, revenue, and then profits over time. Most of the waypoints they could get desired on a roadmap are put together by people like in marketing. And these people tend to have glasses that are very rose-colored. They start asking for impossible things in impossible timeframes. We're going to have to deal with that. When we do our roadmaps, we are not going to repeat the evils of going through a big design up front. We're not going to look at all of the development realities. And we're not going to talk about what effects matrixing will have, which I'll talk about in just a couple of minutes. So roadmaps are kind of idealized when we get started. Again, they're a much higher level plan than our releases. Release plans are where we start taking the technical realities into account, things like dependencies, things like resource availability, and unhappily matrixing sorts of issues. So I look at planning in general as a kind of a game, a game of constraints. These constraints will start off where we have to address a whole bunch of issues as we do our planning. We're going to talk about risk, and actually the tolerance that your company has for risk. If you're a young startup company, your risk tolerance is going to be really high. You're out there and you need to do something. You're passionate about it. If you are a large established company with shareholders and whatnot, usually you have very low risk tolerance. They do all these safe sorts of things, which is kind of not so fun. We have constraints on resources. Nobody puts together a company with unlimited resources. So the stuff that the marketing guys and the product managers put together, we have to face the reality at some point. There are uncertainties that are related to risks and the expectation kind of stuff. In my mind, these things form kind of a region. People familiar with linear programming per chance? Yes? A couple. It's a mathematical concept. We can put together a mathematical model so that, for example, if we were producing oil, we could look at the market demand for oil versus its price. We could put another bar that says, here's what it costs for extraction and lots of other kinds of things and carve out a region in space or I should say on a plane usually that tells us the best way for us to maximize our profits. Use a simplex method to usually run through those equations. That's great. It really is great. It has no applicability for this discussion because of the uncertainties that we're dealing with. They're so high that we can never get a reliable mathematical model to help us. I really wish I could. In order to do this properly, though, we're going to have to admit that we don't know enough. We're going to have to be agile. One of the basic things about agile is the idea of honesty and transparency. We're not going to try to fool one another into thinking things are the way that they're really not. Deep in my background, I actually have a degree in law. When I think about how you go about justifying a roadmap, I always think about it based on these two legal types of terms. You're going to learn something new over here, too. In the U.S., there's the idea of strict scrutiny by a court. When something really nasty is out there, like somebody can take away basic liberties, put somebody in jail, something like that, the courts, when they review those sorts of laws, have to be very exacting. They really want to make sure that we don't take away liberties from people unnecessarily. On the other hand, they apply a different test when they're talking about laws that maybe take, or about money. So if there's a law on taxation, the courts will apply this thing called a rational basis test. The rational basis test basically says if you can come up with any kind of argument that kind of says why this makes sense, we'll say that it passes muster. And we're going to be using our rational basis test when we go and put together our planning. We're not going to make people put together fictitious sets of numbers that prove down to five decimals of accuracy that these are the numbers that they really are. They're not. There's so much uncertainty in this stuff. There's so many swags that we put together that we're going to have to just live with that, find a rational basis to kind of explain our work, and then move on. We'll learn as we go. Typical sort of thing. The actual factors that we'll be using, using just in a couple of minutes, deal with the benefits that we're going to get from the feature sets that we put together, the costs that are going to be involved in however we go about planning our roadmaps and releases, and then the risks that are inherent. Now, risks are a funny thing, right? Because how do we deal with risks? So I'm going to talk about this on the next slide a little bit. But I want you to first think about a risk as something that's out there, right, that we need to deal with, and we're going to have to put some number against it. Now, I don't know how it works around here, but if you buy a house and you're not sure if the person selling you the house actually has the D to the house, many times they ask for some sort of insurance policy, right? And your risk of that person selling you the house not passing you good title is taken care of by the premium that you pay. Well, that's a way that you can quantify risk. So we're going to use that same kind of concept as we look through the various categories of risks. Now, it's kind of interesting. I'm going to talk about the concept of net present value. It's actually invented by somebody that you wouldn't expect to do that, but it was invented by Karl Marx, back in the latter part of the 19th century. He talked about fictitious capital. I guess he wasn't too kind about it. In any case, net present value is all about figuring out the time value of money, right? If I want $10,000 in five years, I don't have to put $2,000 a year away, right? Depending on what the rate of return is, I might be able to put together $8,000, let's say. So the time value of money allows us to look with present-day currencies, I was going to say dollars, but they could be rupees, with present-day currencies and to figure out which is a better idea. Especially if the sorts of things that we're talking about take an appreciable amount of time to develop. So net present value is a great idea. It's the discounting method that we want to use. For today, we're not going to use that. I don't think that everybody here is prepared with Excel sitting on their lap. I'm going to ask you just to remember that there's a concept that you can do with it, it's easily implemented in many different settings. That's pretty much the way they want to work with it. The problem with it is that we shouldn't rely on the perceived accuracy of our spreadsheets or whatever we're going to be using. The idea, though, is to go back and be able to compare one against the other, not to try to predict what the exact value is going to be. You've got to stay agile with this stuff because there's guesstimates involved, or as I'll say later, we don't call them estimates for nothing. Otherwise we would call them exactimates, and they're not. When we go through, and I should read some of these things, and we look at risks, we always are trying to figure out, gee, in a situation that could occur, what can I think of that might make our project fail? Now, you're actually looking at a scene that was recorded from a very dangerous airport in Honduras. You'll see in a second what actually happens. There's a big risk that somebody came up with, and they had to figure out a way around it. I'll tell you that the way that they dealt with the risk was to put in that stoplight. When we look at risks, there's basically six sorts of issues that will come up. We can talk about risks to the business. We can talk about the risk that a product that we put together changes our image in the marketplace, that a product that we put together changes the relationship that we have with our people perhaps. We have market risks. We say, will anybody buy our product? Will we get to market and be first to market with a product? All of these things will have an effect on how much we're eventually going to get. There's resourcing risks. Naturally, do we have enough talent on board or can we get it on board quick enough to actually do it in a timeframe that makes sense? Dependencies, either external or internal. We're going to have to figure all this stuff out. Of course, the technical sorts of things that go along the way. Make it worse. Many times, we work in organizations that organize themselves into functional silos. Anybody here work in such an organization where you have somebody and they're committed 50% to this project, 20% to this one, called Matrix. We have functional managers and we have project managers. Horrible. It makes the situation much worse, because we need to be able to focus and think about how to do stuff without trying to take other concepts like the amount of task switching that goes on as we try to spread people around thinly. Matrixing is an evil. You want to avoid it. Even though you're going to have to consider as you go through it. We'll talk at some point, I forget when this comes up, about how much harder it makes the problem. To do portfolio management is kind of easy. There's four basic steps. First, we've got to put together that roadmap. We're going to put together a roadmap initially based on things like net present value. What we want to do is figure out the projects that are out there that we really need to take care of, which are the ones that either give us the best value over time or that make sense to do to give us sustaining revenue as we go forward. Net present value will help you a lot in that kind of analysis. We then got to go and go down a level, just like with most kind of agile planning. We want to go from roadmaps, which might have an epic. This release of the product shall do wonderful things. Then we might want to take three or four or five epic stories out of that. We're going to split it up. We will do a little estimation along the way and come up with some release plans. These are very high level. We don't know enough at this point to even commit to doing a project, but we need something to get started with. We're going to do just enough to get us started. Now, you're going to apply the release plan against the roadmap. You're going to mix it. You're going to stir it. We're going to shuffle cards around. We keep on doing that until we can find that reasonable area that satisfies both the business side that it makes sense to do the product along with the technical side that says, yes, it looks like we can do this. But that's not the end of the game, of course. Then you have to start instituting a periodic review. Think of them as sort of like an iterative cycle on steroids, so that maybe every month, every six weeks, some period of time, we bring the management team back together to look at what's going on inside of our roadmap to see if conditions have changed. Perhaps we've gained enough knowledge now that we've put together the first couple of iterations to realize that this isn't really the product that we want to put together. It's a very agile concept to fail products early. Rather than spend all the money that it takes to develop it and then find out you have a loser, as soon as you figure it out, get rid of it, apply your monies to other work. At this point, let me see, how many people are in the room? I want you to self-organize into six different teams, and we're going to do it against the mirrors. Find some mirror area to work with. Everybody up? You've got to get up. I'm not going to assign them, but split yourselves up into six teams. If we can't move that stuff down there, then we'll have to make it four teams. Find yourself a good mirror to work from. You're going to need the space. There's going to be some materials to work with, but before you do that, before we start putting together our road maps, I want to introduce the products that our wizards up in marketing and the product line managers put together. Seems like I have a lot of sixes today, so another six products. First, let me introduce you to the company. Our company is called Location, Location, Location Software Solutions. And we produce GPS materials. We actually build the hardware. We've got some, how many teams? Four development teams that do hardware. We actually put together the embedded software on these hardware platforms. We have a full team that's involved doing that. We put together applications for mobile use. That's a pretty big part of what we want to do. We have three teams that have done that so far. We also have stuff that works on web applications. We have two sets of teams that do that. Here's the first product. The first product is a set of things that are augmented reality glasses. These are like goggles that you put on. What we want to do, if you want to, I could do a dramatic marketing of the vision. What we want to do is give people the option of buying these augmented glasses, which they'll wear while they're driving. The turn by turn notifications will come up in the goggles. But the great part about it is that they'll now have shopping opportunities. If we know something about our customers, their needs, wants and desires, and we notice that it's been at least 15 minutes since they've had Starbucks coffee, it'll tell them where to go to get the next Starbucks as they're driving. It'll show up on the thing. This is kind of like what it looks like if you went through it. Now, our wizards up in marketing also have some ideas about what the future for this might hold. You can take a look at it. It may or may not be interesting to you. They've put together some numbers, but you're probably going to want to look at to tell you what sort of market they perceive for this product in the future. And they also have some risks. I'm going to have you look through these. You're going to have to make pretend as you work through these problems that you know the answers to this stuff. Not everybody in the room is experienced in doing GPS's and building hardware and stuff. For example, it was actually kind of interesting talking to people that really did it because they really pounded me bad on what I told them to do. Anyhow, you're going to have to work through these sorts of things. Figure out what the potential problems might be along the way, like privacy concerns. In case we went back, we asked our development teams to give us some estimates, really high level estimates. In order to come up even with some high level estimates, the things to do and what I want you to do is split up that big vision. Make three, make four epics out of it. And then start estimating it. Okay? Put together something large. And in fact, for purposes now, we don't know how to convert things like story points into duration, because we don't have velocities to work with. So just put together your estimates in terms of the teams that you saw and some arbitrary quantity. So my estimates, and you can use different ones if you want, looked at the teams that were available and said, well, we don't have any mobile app kind of component to it. The hardware people, they got a long time to work on. They have four teams and it might take them, well, it's going to take them more than a quarter. You'll have to figure out for yourself if that's two teams working for two quarters or whatever. The embedded people, of course, have some work to do and the web app people have work to do because we need to get the data into this system. That's our first product. Second product, I'm sorry that these things are being cut off a little bit, is a product that gets installed on these high-end sports cars built by Subaru, which I guess isn't that popular around here. But I know them well. I drive one of these. The product that we're going to put together is called King of the Road and it sits on the built-in GPS, which is a hugely overpriced option on that car. I don't have one of those. What this will do is allow people who buy these cars to participate in road rally events. I don't know if we do that out here very much. Road rallies. People get some destinations to go to and basically it's not supposed to be this way, but it's who can get there fastest. And as you can imagine that's not usually popular with a lot of police departments who don't want people speeding. So it's also very hard to put these events together. So our wizards up in marketing said, hey, it would be great. We could have like a website that people could log into. They'll give them some rally instructions to go out. We'll let them run the course and then they can post the results. Sounds like a really good idea. Unfortunately, we have some issues. I want you to look through this market potential. One of the things about this is that the people who buy these overpriced GPS's already are very elastic in their prices. They'll pay anything. So we've got a good amount of flexibility in that regard. But if you look we only sell about 2,000 of them per year. So you might want to take that into account. We also have a whole bunch of risks. PND stands for portable navigation device. So it might be your cell phone. It might be a handheld device. We also have some risks of course that we may have lawsuits from the police about this. Be creative. There's going to be door prizes for the person who comes up with the most creative ideas. So we're going to be watching for those people. You will see that there's some estimates down there. These things by the way if we have six teams I have two sets for each team of these slides. So you can go back and get the quantification to look through this stuff. Third one. Wow, this is an interesting product called TrustMeNots which is for creepy guys everywhere who don't trust their significant others when they're out of sight. It doesn't apply to anybody in the room of course. TrustMeNots are a brand of undergarments with an attractive lingerie with a built-in GPS and a 3G radio in a nice locket. At a distance unlike the traditional chastity belt, our product gives the illusion of love and trust, blah blah blah. The market potential for this is like bad. We will get a lot of bad press if we ever make one of these. But on the other hand no press is bad press. It also means we have a tremendous amount of advertising that we paid nothing for. So you're probably going to take that into account. There are some real significant risks. If you notice over here we have a risk talking about we got to have it ready four months before Valentine's Day. Why somebody want to give this for Valentine's Day? I can't understand. That's going to be our big selling season. We probably have huge risks of privacy from people who use this and find out what it really is. And we have some estimates from our developers about this. Fourth product that we have is called Way to Go. This is becoming more of a commodity. But what happens is since we control the hardware we've built stuff into our GPS's that report back to a central office. So we know from all the GPS's that are out in the streets how traffic is flowing. And we combined that data with some historical data and we're able to give our people much better road guidance. It gets them on their way faster. There are risks associated with this too. We recognize that this is going to become a commodity going forward. Once a product becomes commodified the margins lower. It's much harder to make any money with it. You'll see though that there are some interesting sorts of potential. I want you to work through that. Fifth product is something called paint by cars. The idea about this without reading through it, you can read through it at your leisure is that you get into your car and you drive on the streets and the lines that you get from driving on the road become a picture that you now go back and paint. There's a web application built in with this sort of thing. It's for family based fun. We also have driving routes that we can suggest to you so that you can actually do it if you're inexperienced at it. You'll see that right now marketing says that we probably have zero money that we can make it this. However, we'll gain reputation like a cool company. That's going to be worth something to us. Final product is something called matte maker. This is a product that we're actually not selling directly to end-user consumers but we're selling it to the people that put together the maps that are used for the GPSs such as a company called Navtech. What we're going to look for are all the places that they don't have maps for. If you happen to be driving and we notice that there's no road out there, this will get sent back, we will be processing this and figuring out where the roads are before the satellites are able to map them. We can anticipate making some money with this. We have some issues about how to maybe do this even though we have some people that work through it. Now, for the next 30 minutes I need you to put together something called a waypoint summary. A waypoint summary, think of that as a mini business case. It's going to describe the cost. It's going to describe the benefits. It's going to quantify the risks. It's where you, if you were doing this for real, would put together your net present values and you'd be able to start to have a common place to compare. Yes, we're going to be working with estimates, not exactimates. For the meanwhile, we're not going to assume any kind of matrixing. Now, I used some assumptions and again, you'll see these on your sheets. You're perfectly welcome to go make up your own assumptions. Again, the more creative you are, the more likely you are to get the door prize. These are what my sheets look like. I do not expect you to come up with these sheets. What I want you to do is understand from these things that I was concerned about. You'll probably be concerned about a lot of other things. But you will see that I have some benefits that I can associate over time. Some costs for doing some development. I've identified some risks. Actually, the risk involved in putting those augmented reality goggles together, we may not have the talent on board to do that. I've quantified that by saying if I had to go out and have somebody else do it, what do I estimate it would cost? What's my guesstimate as to my ability as a company to produce this product in the time that I need? If I need to, I can go and have a business partner to produce the thing with me so I can quantify that. I can quantify things like privacy concerns. Where I'm going to be looking at what an average settlement might be for a lawsuit that says you've compromised my privacy. Notice that things like the privacy concerns probably don't have to be affected until we actually have product out in the market. I'm going to be looking at things like safety factors. If we start getting lawsuits, what's the difference? What would it cost us? Put some numbers together. When you're done with these numbers you're going to want to visualize them because otherwise you've got a bunch of numbers sitting on a sheet and probably for today you're not going to want to do spreadsheets you're going to want to do something more creative and you're going to self-organize and figure it out. These though are very useful to allow us to visualize how much we're going to have to spend now to make money later. We can get to that by working through some numbers. Other charts that are out here they're all in your packet. Take a look at them. Each of them will have some little tidbits in them. Do not copy my examples. Also, don't go further in the packet than you're supposed to. I've got to show you the slides before they become real. Here's that great product that's out there. Look at the risks on this. You'll have to figure out if it makes sense or not. This is actually one of the important takeaways. We don't necessarily know when we look at a product what to expect from it. We may have a certain desire to make a particular thing because technically we know how to do it or technically it's something we'd like to learn how to do but it doesn't mean that we're going to make money doing it. I put together some numbers out here for you to look through. When you are done you're going to go back and look at your numbers. These were the numbers that I put together. For each of the products that were out there I forecast a business value based on the net present value of the various streams that you saw for benefits and expenses. Notice how some of them were negative. If I went back and ordered them by their net present values, by the business value over time it would show us things like those augmented reality glasses. Not only do they seem kind of cool but they're probably going to make us a ton of money. Other products such as the trust me knots and king of the road, what do we do with losers? Kick it off. We don't need it. We make money by not doing it anymore. Just stop. When we're done we can then create our initial roadmap. This is not the take away from this session. We're going to do better roadmaps later. Now's the time. We've got about 30 minutes to go out and do this work. I'm going to roam around. If you have questions ask me. I will be your marketing, product owners, whatever. On your teams you probably need to if I was going to do this self-organize and have some people that are going to represent the hardware end of things. They'll be able to understand and make better estimates than the people doing web applications. You're probably going to want to have a product owner on your team. You can ask them questions. The people that stand up to be product owners, be brave. Give your best shot at this kind of stuff. Let's get started. There's going to be a benefit that accrues when we sell the unit. I believe that we're charging them extra per day to use it. There's some revenues that go over time. There's going to be on the initial one with the augmented reality glasses. That's going to vary over time. You've got to think through. I don't care what the numbers you come up with are. What I want you to do is to think about all those different vectors and risks. There's actually when you look at my notes, I can give you 16 different categories. Inside of those big six categories of risks. Not every risk occurs for every product. But as you look through it, you might want to use that list to jog your memory. Think about the fact that oh, we actually have a market risk out here. How are we going to deal with that? How can we reduce our risk that nobody wants our product? Business benefits. The things that show up on road maps are not things like oh, I like this better because it takes me less time to enter it. It's what I can sell that for. Because what you're talking about a benefit might be my code is easier to read and reuse. Now that's a really great technical benefit. The next part of this exercise is how to go and match up the road maps, however incomplete they are right now are release planning. Now the basic problem is that we have resources in time and we need to know are we going to over commit our resources as we develop perhaps several products in parallel with one another. Now there is a product out there. I hate to mention its name. It's Microsoft Office and Project. Microsoft Project does this sort of stuff. There's nothing wrong with planning using Microsoft Project if you keep it light like one get bar for each product and then assign resources against it. But if you go and start using actuals versus estimates Microsoft Project I will hunt you down and hurt you. This is not the way we do things in an agile world. So remember we're going to use a rational basis. We're not in a court of law here. We're going to expect things to change. That's why we do these periodic portfolio update management sessions. This is a road map. I want you to kind of look at this as a model for the sorts of things that you might want to be thinking about. The way that I go about putting together these road maps is with the use of 3M products which is what you are going to do right after I get done jabbing. I like to use 3M products and look at a duration as some sort of a length. And then I want to stack stuff up on the 3M cards so I can show the various sorts of teams that are involved. So if I need three times the number of teams for doing my embedded software than my hardware I might have a different color which is three times higher. The wonderful thing about doing that is that you can then put your cards up on this playing field and move them around to find what your road maps are going to be. Road maps also will have lots of things along the time frame. So if you can read this sort of stuff as we started moving these things around you are going to see events on there like I can't read that exactly but we are going to have to make a product announcement to the press at a certain point. We are going to be expected to show off our stuff at a particular trade show. We are going to have an advertisement in Family Fun magazine. These are going to become very important when we figure out, those are the waypoints those are going to be very important when we figure out how to match our release plans to try to make those events come true. So I now want you to get back in with your teams, do some of that stuff. The result of doing that should be that you can come up with a better road map. If you go back and compare and contrast and looked at this what is not really earned business value it is projected earned business value. If I go back and compare this chart just get a general idea of the sweep of it and the first year and compare that to this chart which shows that I have negative revenue for my first year. You can see that it is not just net present value that makes the difference. That is a starting point. It is actually considering all of the variables. Get with your teams, pick up some of these products over here. I do not want to take these home. I would like to do a little retrospective on what has occurred today. Now, you guys you are the guys with the mobile iPhone app for doing paint by vehicle. Paint by bicycle. What did you learn out of this? Anything today? Did you learn maybe not? That is actually very good. What these guys were concentrating on was looking at local markets to understand better not just understand but understand better what the product implications are. You are going to be doing that even with product owners that are going to be local. They are going to have those rose colored glasses so as you do a little bit more analysis into something maybe you do a product focus group. You learn more about what the product means. You will be able to use those results to guide you putting together your roadmap over time. Not everything is the way that it appears at first. Many times we use our preconceived notions of what is kind of cool or what we think would be great in the marketplace and then do a couple of number calculations and figure out that was a really dumb idea. If I could restate what you just said. What you were learning out of this was how our costs get spread over time. It is not just one lump number and when you get back home you will be able to perform NPVs against this because the further they spread out over time the harder it is to understand. Not every risk occurs when you start development. Some risks occur after it is released. I want to thank everybody for putting up with me for all this time. We have lots of topics to discuss. This is just the beginning. This should be just enough to get you started. I have a lot of work ahead of you to do this correctly. Thank you very much for your time.