 So now we discussed continuous experimentation, we'll look briefly into the idea of measurement in software engineering. Because for continuous experimentation we somehow need to measure the outcomes, we usually don't want to automate everything and then have someone sitting there and look at the results, we want to automate this as far as possible as well. And then measurement in general, even beyond continuous experimentation, is a really important tool to, for example, improve communication, improve decision making, improve overall quality and processes and so on. So it's a really important tool and that's why I thought of at least mentioning briefly in this course a bit of the theory about the background of how to measure software generally. And if you want more details, this is generally governed by the ISO standard 2502N, so that's measuring software quality, there is a whole family here, so the O1030 and O40 also into how do we evaluate this and so on. But if you're ever interested in this, try to read up on this, there are also papers, scientific papers, there are books that give you a bit more background. Because nowadays the issue with measurement is we have luckily a lot of tools, dashboards, web tools that do this, automated to some extent, but it's good to have the basis, the background a little bit. Now in general, what we can measure is of course our product, our system and the standard talks about software and systems quality or product quality and it talks about the so called quality in use which is different. So this is sort of static properties of the software of the code, how secure is it, how maintainable is it, for example, how performant is it and these are the things, how well does the software perform, how high is the quality when it is used by an end user. So here for example we have measurements of usability in this standard. So these are things you might want to consider. So your security can be really nice but it's not usable, so different aspects and of course these are related to the things we have already discussed in architecture. So many of these properties you find there as well. Then of course we might want to measure our process. So how good are we with for example handling deadlines, are we always late for the deadlines, what kind of, how many bugs are coming in, how many change requests that might be something, how well are requirements traced to tests for example. So a lot of things that are actually more on the process level than the actual product but it might nevertheless be interesting because the idea is of course the better we are in our product the better the product, the better we are in the process the better the product is. So there should be a relationship here and if we improve the process we hopefully also improve the product or at least the speed or the cost at which we develop this. So these might be things that could be a lot more but I think these are the things that are central for this course that we might want to look into. Now this standard family proposes some things you can measure. There are some examples also on the slides but I will not cover that in that much depth. I will go shortly into how the standard looks at measurement generally, how it breaks it down. And the important thing to notice here is that what you want to have, what you want to get answered is some kind of information. You as a stakeholder as for example as a scrum master you have an information need. And that is for instance how secure is our software or how many bugs are there. And on the lowest level you have your actual software product, you have the entities for example classes, the code, the different files. And the question is essentially how do you get from here to here. So how do you get your information need answered based on what you have and what the standard calls this down here are the attributes. So these are the attributes you want to measure. For example what is the size of my code, how large is our product. That's something you can measure. And then you start with the so called base measures. So that's a direct measure of one of these attributes. For example you could measure the lines of code of the software. How many lines your code has gives you an indication of how large the software is. Another way that you could use is to just measure the file size. Measure the overall file size. This of course gets more complicated. Do you want to measure the lines of code with comments or without and things like that. So this is actually quite tricky. You could also measure other things, number of classes, why not, number of functions. So this is not at all trivial if you go from a basic example such as size. Now the issue is with base measures we usually can't say a lot. So size itself doesn't really matter for us. So usually you need to get to a higher level and you have so called derived measures. And that's a function of one or more base measures. For example one way of telling how good our product is to look at bugs. But the number of bugs is maybe not very interesting. So it's more interesting to look at the number of bugs compared to or in relation to the overall size of the product. So there's a difference whether you have 10 bugs and one function or you have 10 bugs and 100 functions of course. So this is where you get to the derived measures. You usually need to have multiple of these base measures together to get to something that actually tells you something. Now this is already quite good but usually it does not quite fulfill your information need. Usually you get to something that is like an indicator. That's an abstract way to interpret this measure. For example we have number of bugs per software size, per lines of code. Then we might have certain thresholds. So we say well if we have more than one bug per 100 lines of code that's bad. If we have more than one bug per 500 lines of code it's okay. We have more than one bug per 1000 it's good for example. So it's sort of the traffic light system of rating this. And usually you end up with some kind of so called information product like a dashboard that actually visualizes this. And that's something you can give to the Scrum Master and say well look at this our software quality the amount of bugs we have is red right now we need to do something. So usually that's really when you get to something that is useful. Now just to make this complete we have attributes we want to measure them in a base measure and we need to have some kind of method of measuring them for example counting the lines in a source code, counting the amount of bugs in the bug reporting system in the last five days. Here you have a function in the mathematical sense combining number of base measures for example lines of code divided by number of bugs. So mathematical function here you have some kind of analysis model for example the traffic light I've given you more than a bug per 100 lines is red for example. And then you basically have decision criteria here usually that lead to some kind of visualization or product. So this is how you look at that and the important thing is always that for one and the same information need you can have a lot of different attributes or base measures that you might want to measure. So I've talked about lines of code but as I said you could also look at the number of classes why not and it's not always straightforward to decide which one to choose. So this is definitely not a trivial task and it takes quite some thought put into that how useful the different measures are. And then if we get to the product we have like a dashboard that we can show to the managers or the developers there are a number of final things you might want to consider that's what we go away from the theory even more the practical thing. And one of them is that people we have talked about human factors people adapt. So it is a very bad idea to for example give salary raises or generally praise people based on the amount of code they write because they will most likely just write more code it doesn't mean that it's better maybe they just do more line breaks because you are counting the lines for example. So you have to take this into account if you actually want to use the information product to evaluate people then they will adapt to that and not necessarily in the way that you want to. So that's something that is extremely important and in particular if you really have the interest of improving the quality of your process or your product you also need to take into account what other things you are not measuring because the people will probably try to optimize the things you are measuring with good or with bad intentions. But the question is then is there anything missing that might actually jeopardize this whole process. And then a final thing is you need to have confidence and you need to establish confidence in your information products. I think most of you have been in the case where you have written tests and there is this random test that always fails or it always passes it doesn't matter what you do people will start ignoring them over time. So if you lose the confidence in your measurements they are entirely worthless and that can be risky because the tests might actually still tell you something but if for example if there is a test that every day is red and then it's green again and then it's red no one will look at it and maybe at some point there is actually a critical error that the test captured but you didn't look at it. So that's a really important thing and that's why it's important to not just go in and measure everything but really put some thought into how do we measure things how do we set up this entire system so that there is high quality we don't have this issue of for example failing so-called flaky tests we don't have this issue of meaningless measurements that are always very good or very bad but they don't mean anything so this is something we need to put thought in. And finally consider that all of this is a software product so we are measuring our software but we are at the same time potentially developing a new system a measurement system and of course to this everything we have discussed might apply so you need to think about how do you actually build this system how do you do the architecture in particular if this is measurements that go at runtime like in continuous experimentation how do you build this in a way that it doesn't slow down the overall system so it gets increasingly more complex but I thought this is important to discuss this that you're aware that these thoughts exist and not just go in and use a random tool that you find to do some measurements but try to plan it. Okay so this concludes the very short episode of measurement there is a lot more to talk about but we'll wrap up now this module with a discussion of the concept of DevOps which brings together all the different topics we have discussed so far.