 Hallo und willkommen zu der Femme-Channel. Der nächste Speaker ist Antonius Vri, talked about the dark arts of floating point numbers and why multiplying a number a by n is not always the same as adding it by itself n times. Auch für alle Deutsch-Sprechenden zuhörer, der Talk wird auch auf Deutsch übersetzt, das bedeutet, um die deutsche Übersetzung zu hören, könnt ihr im Webplayer einfach die Sprache ändern von native to translated. Und jetzt? Viel Spaß mit dem Talk. So, hi, I'm Antonius und I'm going to talk about floating point numbers, which in computer programming languages are generally the numbers that have a decimal point in them, like the ones on the left, mostly the ones on the right, the integers use a different number type except for javascript in which they're both floating point. So, if you've done pretty much any programming at all, you've probably interacted with these and you might have noticed that they're a bit weird sometimes. For example, I'm going to start with this. This is a very big number, 1e16, let's say 1 with 16 is the zeros, and I'm now going to take 25, it's a pretty small number, I'm going to add the big number and then I'm going to subtract it again. And that shouldn't do anything, that doesn't make sense to add a number and subtract it again, nothing should happen. But actually if we do this, we get back 24, not 25, which is a bit strange, I think that's fair to say. So let's try some other stuff, let's do add 1 to the big number and see what changes and nothing changes, like you're not even supposed to do this, you're not supposed to compare floats for exact equality, because there might be the rounding errors. But these two floats are exactly equal, I added 1 and literally nothing has changed. So this is strange, right? So for this talk, I cannot thought about all of this very cool and very complex stuff that I could talk about, but I'm really not sure where you're at with your maths knowledge, because it can vary quite a bit. So I actually decided to start very basic and so the beginning of the talk is going to be very simple and then it's going to get progressively more complex. And I hope that this is a good compromise. The beginning might be a bit boring. If you know this already, the end might be a bit hard. If this is the first time you're hearing this, because it's really not a lot of time to understand, but I hope that this has got compromise. We're going to start part one and to start my explanation, I've brought like the display of like one of these old calculating machines, right? So it's just a bunch of zeros right now. The reason why this is useful is that it has a limited number of digits and computers when they store numbers only have a limited number of digits, right? So this is going to help us see some of these things that happen. So this is a very simple machine. I can add some numbers and then I can also add a second number and then I can add them together and there is a result. It can't really do much more, but it's got these other two interesting buttons that I can use to shift the whole number just one digit to the left and also back to the right again, right? And you can see it goes across the decimal point there. So und just because this is going to be a little bit important, I'm going to show real quick what this does. Because it turns out with our number system, which this machine uses, shifting a number left multiplies it by 10. So we've got a 2 here. I'm going to shift left and we get to 20, which is 10 times 2. And if I shift again, then we get 20 times 10, which is 200. And that's das. So, that's nice. And this also works in VR. So, right? So, shifting right again is going to divide by 10. So I'm going from 20 to 20 and from 20 back to 2. Okay. This is actually a fundamental property of our number system. Because what this means is that we only need the digits from 0 to 9. We don't have a digit for 10. Because in order to write 10, we use the digit for 1. And then we shift it once. And once we've done the shift, it gets multiplied by 10. And then we can represent 10 by that. We don't need a digit. Okay. This will become important later. So now, let's do a bit of maths with this machine. And because I'm an Astrophysicist, and actually just because I need an excuse to be a bit mean, because I want to show some stuff, we're going to enter the mass of Damals, which is one of the moons of Mars, in kilograms. That's kind of heavy. It's a 1 and an 8, followed by 14 zeros. So I'm gonna start typing. 1, 2, 3, 4, 5, 6. And I've arrived at the end of the Display we have here. Right? That's a bit, that's a bit inconvenient. Because I want to type the full number. And we have, we have 8 zeros that I'm missing from this. So what do we do? I'm just gonna write them down. Right? I mean, you know what's meant by this. I've just told you, when you see the number on the display, you just have to add an extra 8 zeros. And then you get the actual number, that I meant to write. Actually, now that I have this, the set of counter up there, I could have saved myself some rock. Right? I could have typed 1 zero less and written down an extra zero. And I could have, it's not supposed to do that. And I could have saved myself another zero and another zero. And so on. And now we're back at this. It's a 1 and 8 with 14 zeros after it. It's kind of, it's very easy, right? And the other thing that's, that's kind of interesting to notice is that shifting and moving zeros around is kind of the same thing. Right? I can shift to the left. And that's the same as taking a zero out of the counter up there and moving it back in the number. And I can also shift the number right, which is the same as putting a zero up in the counter. And actually, so, and actually now that we're talking about shifts, maybe we could do this, right? We could shift at one extra bit. And now no longer makes sense to think about this as the number of zeros that you have to append. But it still works. If you think of it as the number of shifts you have to do to the left, which is this way for you, the number of shifts you have to do to get back the number, right? If the display we had was a lot bigger, we could actually shift the whole number out to the right, to the left, like this, like you see on the screen. So shifts just as well as adding zeros and it's just that with this, they kind of look the same. Okay. So once we've got this, let's do some actual maths. And let's go back here. And I'm going to say, I want to add one kilogram, one single kilogram to the mass of demos. And I'm going to do it the obvious, but wrong way first, which is I'm just going to enter one. And I'm going to add them together. And so now, if I look at the full number, if I do the shifting, I didn't add a one. I added a one with eight zeros. And if you go back, this kind of makes sense, because right here, the one that I entered, the kind of display setting was at eight shifts. So of course, anyone that I enter has eight shifts. But I don't want one with eight shifts, I want one with zero shifts. So what do I do? Well, if you remember what I just said a few minutes ago, I can change the counter up there by shifting the number around. Specifically, I can shift the number to the right and increase the number of shifts that we get just like that, right? So I can just keep shifting it to the right until the counter is eight. And now they match. And now we can do the addition. And if we now do the same shifting out the number thing, then you can see that it works out exactly. We've literally added just one kilogram to this mass. So that's kind of an important point. If we work with these shifts, when we do an addition, we have to make sure that the shifts match. And now you can also, I guess now you can kind of see why it shows the mass of this very specific astronomical object, right? Because if it had been just a little bit heavier, if I wanted to add 0.1 kilograms instead of one, then the one I wanted to add would have fallen off the back of the display, right? So the addition wouldn't have changed anything, which is very similar to what you saw in the second slide. And to take us a bit further, if we had added something like 1.2 kilograms, the one would have stayed because it would still have been in the in the digit range, but the point two would have gotten cut off. So if we then subtracted the mass of time loss again, then we would have gone from 1.2 kilograms to just one, because the point two would have gotten cut off. And that's similar to the other case on the second slide, right? But to really understand that, we are going to have to go one level deeper. And the only thing I'm going to change here, and this sounds kind of small, if you remember how I said shifting a number to the left would multiply it by 10 to the right, what? Yeah, to the left would multiply it by 10. Well, I'm now going to change that factor to two. And so we don't forget, I've changed the font as well. And so let's look real quick at what that does. I've entered one, I'm now going to shift it. And this looks like a 10, right? This looks like this many fingers. But it isn't because the shift factor is two. And one times two is just two, let's say two. Some of you know this, this is called binary. And the reason that computers like to use it is because now obviously we no longer need a digit for two. And we don't need any higher digits, just like we didn't need the digit 10 for a normal number system. And what this means is that we only need zero and one as digits and computers like that, because they're only having two things. All right, so let's use this to actually understand the 2425 example I gave on the second slide. And I'm not going to actually use 1E16 as a number because it's very large and because we need a lot of digits. And that's not super easy to understand. But I'm going to show you an equivalent, an equivalent thing. And so the very large number that I'm going to use here for this specific display with this number of digits is the one you can see right here. Say one and then it's got like seven zeroes after it. And we've got nine extra shifts. That's the large number and this is our number 25. That's 25 in binary. You're just going to have to trust me. And we want to add them together, which means, remember, we have to match the shifts. So this is at nine, this is at zero. And to, we now have to get this to nine shifts, which means we have to shift the spec just like us. And you can already see that there's one digit. The one at the very end is falling off the display. Right? But let's keep this just for proper decorum. We're going to do the addition like this. Then we're going to round like this. And then I'm going to subtract the very large number again like this. And then we can shift the result back to the front so we can read it. Now as zero shifts. And you can see, this is very similar to the number we started with, except for this one that's missing. That got rounded away after the addition we did. And so there's now one missing from our 25 and we are 24. So basically the reason that we went from 25 to 24 is the same as with the decimal system, where we would have gone from 1.2 kg to just one. It's that the last digit has gotten rounded off to zero. Except that it's not the last decimal digit, it's the last binary digit. And so by the time it gets converted back in a decimal and shown to you on the screen, it's no longer as easy to see that it's just a digit. That's missing. Okay, so I've just said a weird thing with the rounding and that was proper decorum, which sounds a bit weird. So, so what did I mean by that? Well, up until now everything we've talked about is kind of general properties of floating point numbers, right? You don't need a specific system. But for what comes next, we're going to need more specifics. And we're going to look at one very specific system of floating point numbers. That's pretty much the only that is in use today. And that is the IEEE 754 standard. No matter whether you use floats in Ruby or Python or JavaScript or C or Rust or C-Sharp or I don't know, whatever. If you use a float on a computer, it's almost certainly one of these. And so what does this tell us about how we have to do floats? Well, it specifies digit lengths, of course, for different types, right? So that's standardized. It also tells us a standardized way to write floats. So if I take the 25 again, it says that we have to shift the float such that there's only one digit and one of the decimal points, like so. Since we can shift them around freely, this helps us kind of get down to only one valid representation for each number, right? And because we do it this way, we don't need all these digits in front. And since the first digit is always going to be a one, right, it can't be a zero. Otherwise, we could shift the number until there's one in the first position. We don't even really need to store the first one. So it's a very efficient format. And yeah, what else does it say? It also says that when we do an operation on a number or on two numbers, we have to compute the actual result of the operation with as much precision as we need to represent it exactly and only then round it down into the number of digits that we actually have available, right? And it also specifies a few rounding modes for this particular round. So you could always round the number up or always round it down or do something similar to what you're used to, which is round to the nearest number, right? And so with this extra knowledge, we can now finally look at the treat that I made a year ago that inspired this whole talk. And that's that if you take a number times two, that's the same as adding it to itself, right? That's two times A, A is the number, that's the same as A plus A. And if I take the number times three, that's the same as adding the number to itself three times. And so on, the four and five. And it turns out that if you do this in floating point, all of these are exact equalities. And except for the six times A case, which is not an exact equality anymore. And so what's going on here intuitively is that on the right side, there's all these addition and every addition is a separate operation. So it computes the sum as a very exact result and then it has to round it down. And then it adds in another copy of the number and it has a very exact result and then it gets rounded down. And so it does this multiple times. And for the first few steps, it just so happens that all of these rounding errors kind of cancel out and work out. And only for the last one, only for six times A, it finally gets out of step. And so six times A and the A plus A plus A plus A plus A are almost equal. So basically, it's like the last digit is a little bit different. But they're not exactly equal. And that's kind of the reason why people tell you to never compare floats for exact equality, right? Because small, little rounding errors like this cannot up and appear in your result and mess everything up. But I think it's still interesting to look at why it works for the first few cases and why it doesn't work for the last one. So, let's start with this number. This is the number that we're going to try this out with. It's at four shifts and it's got this binary pattern, right? So, it's pretty simple. And so I'm now going to add it to itself, like this, add them together. We get this, remember that we're supposed to have only one digit in front of the decimal point. So, we shift it back and increase the number of shifts. And now you can see that in this shifting operation, we've lost a digit of precision at the back. In this case, it doesn't matter. That digit is zero. And because we multiply by two and multiplying by two is always a left shift. And multiplying by two is always a left shift. The last digit is always zero. So, when we shift it back to make sure we only have one digit in front, it doesn't change the position. So, what this means is that right here on screen is the exact result of two times A or of A plus A and there's no difference yet. So, let's do the next step. I'm going to push this up. On the up top is the number that we just computed. Below is our original number. I have now had to shift it back, remember, for additions to match the number of shifts because you went from four to five. Sorry. And now we can do this addition. And you can see there's already a digit hanging over the end. But we've also got two in front of the decimal point. So, we have to shift again. Like this, we know at six shifts. And now there's digits past the decimal point. And now we have to round. And what this means is that the result of three times our original number is not something that we can represent in the simple display of our machine. We're going to have to round like this. And what this means is that now when we take this number, which is almost three times our number, but not exactly, and add on another copy, the result can't be exactly four times the number that we started with. So, let's see what happens. I'm going to shift this up, right? And then add another copy of our number. This time shift it twice because we now check six shifts and do the addition. And that's our result. And so to compare what should actually happen after the multiplication by four. Well, it's basically multiplying by two, two times. So it should look like this, right? Remember how with the multiplication by two we only had a zero left over at the end that got cut off. So here there's two zeros that got cut off. And again, four times the number is an exact result. And now it just so happens. This is the result that we get that we actually get out. And the error that you see on the very right as the result of the rounding error that we had for three times a. And now it just so happens that this rounding error gets rounded off again once we go to four times the number just like this. And it turns out that there's a few different other cases for this. But with the default rounding mode which is a round to nearest mode it just works out that the addition onto the three times a always cancels out. And four times a is still exactly equal to a plus a plus a plus a. And so since four a is an exact number just like two a going from four to five is the same as going from two to three. But again five can necessarily be represented exactly, right? There might again be some digits that get pushed off the back and that have to be rounded off. And then finally when we go from five to six the neat little rounding trick that made everything just work out so for the step from three to four doesn't work anymore. And so they no longer match. And then the rounding errors just keep accumulating the larger you get. So why do all of this? In most cases you might be fine with the old advice that we started with which is don't compare floats for exact equality always using approximate comparison. And well it's fun, I hope to look at this in more detail. And the other reasons are depending on what you do with us you might run into problems where rounding errors accumulate or rounding errors work out just the way that two numbers you subtract accidentally subtract to zero and then you divide by zero and you get infinity and then you divide by zero again and you get NaN. So in these cases it can be very helpful to know how how floats really work and also to know well maybe understand a little bit why you're supposed to only compare for approximate equality and not exact equality and so you're not confused when this happens the next time. I kind of shot myself in the photo bit with this talk because I went in with oh my god I can finally talk about all of this really really interesting and kind of weird floating point stuff and then I started out with the very basic terms and I'm kind of at the limit for what I want to do in a single talk just for my brain as well as yours probably so I hope that this was worth it the very basic explanation at the beginning and yeah that that you're not disappointed that we didn't get to maybe some of the even more fancier details that you might be able to get to and I hope that on the other hand my very quick explanation of especially the more advanced stuff was still understandable to fill in some of the gaps like the other cases for rounding for going from 3a to 4a or a few loose ends that I've left in the talk in a few places and that if you've seen this for the first time can be tied up very neatly with some other concepts that you may have seen before there's going to be a supplementary materials page if I manage to put this up between the time of recording this and the congress actually happening I hope and if you have questions or feedback or want to tell me that I included too much basic stuff or that I included too much advanced stuff you can find me in these locations but for now I want to thank you for listening and yeah yeah, thank you very much for this very interesting talk next up is at 6pm why one should not implement time-downs themselves showing us a few curiosities from the IANA time-zone database and now the Harriet News Show have fun