 Ychydig. Ychydig, mae'n rhan i'r rhan. Ychydig. Felly, mae'r cyfnod yn ddiddordeb gyda'r cyfnod. Mae'r cyfnod yn ddweud yn gwneud yn gwneud. Mae'r cyfnod yn ddiddordeb gyda'r problemol 2038. Felly, mae'n rhan i'n meddwl. Mae'r cyfnod yn gweithio yn 2038. Mae'n glas. Mae'n ddiddordeb cyffnod yn ydych chi'n gweithio am fawr. Felly dyna, mae'n ddiddordeb gyda'r cyfnod mae'n ddiddordeb gyda 2038. Mae'n ddiddordeb gyda'r cyffnod, yn olyg mewn oed. Felly, cyhaftawch, cyfnod i chi wnaeth bwysig. Yn gweithio, mae'n ddiddordeb gyda'r cyffnod gan gweithio'n ddiddordeb gyda'r cyfnod. Mae'n gweithio gweithio, ac mae'n gweithio i chi'n fawr. So, what's up? When are we all going to die? What do we need to do to not die? What the current state is? And please, this is a boff. As I always want these things, I do promise I will send a summary of the discussion to the mailing lists later. If somebody can please take notes in Godway, that would be lovely. Time T. It's really, really simple. It simply counts the number of seconds since the beginning of 1970. For many of us, that is really, really long ago. Some of us in the room were born before the beginning of time. Let's try not to mock them too much. Of course, when the Unix people came up with this back in 1970, 32 bits of time T was plenty. It was never going to run out. It's like 640K is going to be enough for everyone. Who'd have thought it? So, we have a problem in that it's signed, and therefore it's 31 bits, therefore it's going to wrap, and it's going to wrap quite soon. We have a second problem in that this is used everywhere. I'll go through that a bit more later. Now, imagine Y2K, except this time it's going to be worse. What could possibly go wrong? Now, again, I'm not trying to exaggerate this too much, but for most of the things that we had to worry about for Y2K, they were big, obvious computers. You could point out them. They were mainframes typically. There were a few other bits and pieces around you had to think about, but that was the call that people cared about. It was, oh my God, are the banking systems going to work tomorrow? Are the cellular networks going to work tomorrow? That kind of thing. Dare I say, it was a bit of an anti-climax. That's not because it wasn't a problem, but people did think about it and work on this well in advance. A little bit of an anecdote, the company I was working for at the time. I actually volunteered to be on call overnight for the new year Y2K. Obviously, a suitably accelerated pay scale for that evening because I was totally happy that I knew all of our code worked, and if it didn't, well, the mobile networks were never going to work so nobody could get through to me. Unfortunately, my boss saw this coming and said no. Our big problem, as I said, when we get to 2038, is that we're not just going to be looking at big, obvious computers. In 20 years, if the sci-fi authors are correct, we're going to have the sensor cloud. We're going to have things that are too small to see, and they're going to be computers, or they're going to be sufficiently like computers that actually we need to worry about this because, of course, you won't be able to go, oh, that computer needs fixing. Your washing machine will have 100 different computers in it, and you'll have no idea how to talk to them, let alone to verify if they're correct. Plus, of course, planes, cars, underwear. Who knows what, but if it goes bad, it goes really bad. So, when does it happen? This example I actually took or lifted almost straight from the man page for date. Other people are thinking about this. So, on 19 January 2038, potentially we all go back to 1902, depending on how badly things worked. So, what, in fact, needs doing? We have lots and lots and lots of things from the bottom of the stack upwards. In the worst cases, on-disk formats for file systems. On-disk formats for all kinds of other data as well, but the file systems we're using today typically are not Year 2038 safe. Do also new file systems like X4, VRTFS, and so on. The newest X4 file system versions are 2038 safe. They use 64 bits of time, which is 34 bits of seconds. So, that will last, oh, until we're all gone, and 30 bits of nanoseconds, if I remember correctly. Most of the normal file systems that you're likely to play with today, however, will die. I say they'll die, they will do weird and wonderful things, probably beyond just tell you the wrong time, but depending on exactly the code in the file systems, behaviour will break. Obviously, this is a problem. There were lots and lots of places inside the Linux kernel where 32-bit time t is passed around. That's being fixed. I'll come to that later. There were worse, there were lots of kernel interfaces, and above that, there were lots of libraries that use a 32-bit time t and related variables, related structures that depend on it. Finally, obviously, all the applications out there that might be using it, a lot of people may not even realise that they care about time, but the interfaces that they're using from the libraries underneath embed it all over the place. So, Linux kernel, I'll go through what progress we currently have. Deeper, Dinamami, I'm probably going to pronounce that horribly wrong, I apologise, and Arn Bergman are running a project under the kernel newbies umbrella to at least work out what needs to be done here. They have a page and there was a mailing list hosted by the folks at Lenovo where people are currently discussing this. Right now, while we have plenty of time to think about this, Arn and others are using this as a good way of getting people into early simple kernel development. You don't need to know a huge amount to be able to go through and pick up. That looks like a 32-bit value that's being used for time. We should fix that and at least document it all so people can work on it. They've started adding some 64-bit code in the middle, they've started fixing up drivers, how my slides don't fit, they've started adding new versions of interfaces, so Ioptals, SysCalls, and a few other things that you might not think about. I'm not going to go into all the details right now. The fun thing is, of course, that we can't just replace all of the existing SysCalls because everything else will break right now, not in 20 years. What we're looking at doing is adding more equivalent versions of old SysCalls, so the SysCall table is going to carry on growing if anybody looks at that. There are lots and lots and lots of tasks here. If you go and have a look at that page, this is a great place to get involved right here, right now. There's a lot of help available, and people would love it if you join in. So, Geolibsy is obviously the next thing up the stack. Essentially everything to a first approximation depends on Geolibsy. Those guys are being as thorough as they normally are. They have really detailed plans that they've started in their wiki. They haven't started work on this. They're still drafting the plans, but, again, they're going to be adding 64-bit time support in a new set of library calls without, obviously, breaking 32-bit support. There's more coverage of this in a very good LWN article, which I've linked. Obviously, the Geolibsy folks cannot even start testing this until they have a 2038 safe kernel. Again, there was a huge amount of work to do here. If you're interested, those guys are also looking for help. Please go and dive in. So, elsewhere. Other work, again, if you're further up the stack, you can't really test much of this until you have the layers underneath you working, but it's not clear how much progress people have actually put into this. I'll admit I intend to concentrate on Linux. I haven't looked at the HURD kernel, at the FreeBSD kernel or anywhere else. These people are all going to have to go through the same kinds of solutions. Hopefully, people are going to come up with a new, compatible way of working. A lot of the interfaces we're looking at at the moment that have broken our POSIX interfaces, I'm assuming that we're going to end up with new POSIX-approved interfaces like this, but there's not much sign of it happening yet. Now, lots and lots and lots and lots and lots and lots of libraries are going to need updates here. Anything that embeds a timetee clearly will need fixing. Timeval embeds a timetee, so you may not realise. Timespec embeds a timetee. Almost anything that is using libc exposed timing functions is going to need upgrading, is going to need updating. You might not realise it. The libraries underneath your application might not be likely to need it because, again, anything that embeds things here is going to break. That's why this is such a scary thing. It's not just a set minus d, time bits equal 64 when we compile. Anything that is embedding stuff here is potentially breakable. We're going to have to do lots and lots of mass rebuilds and people are already talking about doing automated scanning for these things and trying to pick up where ABI's break, for example, because, of course, any library functions that embed any timing functions at all are going to change in size. If you've been clever and kept as neat, then you might be great. If you think you've been really clever, you've possibly made it worse. Because, and I'll go start at the bottom of this one, whatever you do, do not bod you round this. People have had clever ideas like this so many times over the years where they think, oh, well, I know what a timetee looks like, but I don't like using timetee, or I can't guarantee I'll be able to have it on all platforms, so I'll use an int. That's great. It works for you today, but it also means that the poor buggers 20 years later who've got to go scanning for embedded timetee won't find that code. The really clever people who start thinking, oh, I need a more flexible structure, I need a better way of storing this that is bigger, I know I'll store this in a float to a double. Oh, my God, please don't. Really, it's the worst thing ever. So, if the things that people can do now is help with the kernel and G-Libsy folks, there are people already working on this, there is an almost unlimited amount of work. Yes, Dimitri? So, today, I should be using timetee and expect that it will change size, or if I'm writing new code, should I be using, like, is there, like, time64-t or something like that? Right now, your best bet is timetee. Just use timetee and assume that in the future I'll have an ABI break. Exactly. Okay. Expect that things are going to change, but using timetee and keeping on top of it is the right answer. I use quite a lot of structures in small embedded. Is it worth me padding a 32-bit in front such that my structures still align when we move to 64-bit T? Maybe. It depends. What are you doing with that structure? Well, I'm thinking more on the lines if any comes backwards and forwards. So, when I move to 64-bit, I'm not going to hit any word boundaries, any issues of block devices, and so on. So, I'm going to at least get the same performance that I'm expecting now with 32-bit T. Maybe. You're saying about having something so that it would still align. Alignment is highly random and unpredictable across architectures. So, placing something in front of it now and then hoping it will still align when timetee becomes 64-bit is a very huge bet. I was thinking more along the lines of my coms packets. So, it's a bit. What I will say is, trying to second-guess this might only make it worse. What you may find is that, by the time people have thrashed this out and gone through discussions all over the world, that instead of just going to a simple 64-bit value, instead we end up with a 128-bit structure with extra bits, and at that point you've then left your padding, but you'll still have to do work later anyway. So, second-guessing is hard. I appreciate that, which is why a lot of this is be aware this is coming. Get involved. Don't necessarily expect to be changing your code for this today beyond taking out the bodges. So, check your code, check and also check your dependencies. As I said, there is a lot of code here that is going to break. So, as I said, that is the core of the message I want to give. I'm not planning on talking about this much. I will answer questions, but please, does anybody have any more questions, any more points they wish to raise? Probably. First of all, I want to say how annoyed I am that you're giving this talk, because during Y2K I took a lot of mental notes and I had this plan hoping that everyone would ignore this, and in 2034 I was going to launch this consulting business. You've seen that, did you come in late? Oh, are you saying? 15 years ago I registered that domain and I've kept it going with a holding page ever since. But you realise that for that plan to work we were all supposed to be silent till about 2034, 2035, right? So you're screwing up the plan. I'm being altruistic and I am destroying my own pension plan here. Yeah, all of us, you're hurting everyone. The plan is to stay quiet till 2034 and then we launch our consulting business. I'm sorry, it's too late. There's no management meeting this anyway, so that's fine. But in seriousness, so I think one of the things we face is up and down the stack, right? And I'm wondering if anybody working on, you mentioned the sort of scanning for this, there are sort of two related areas of computer science that are free software analysis stuff that are going on. One is the license scanning people. So they're obsessed with finding licenses. Most of the license scanners are proprietary, but there are some free ones. Has anybody looked at applying that technology to finding time junk in code? And the second question related is anybody working on static analysis stuff and program slicing computer science stuff that would find where time t is being thrown into inch and other sorts of things. I've heard that the coveyty people are looking at this, for example, coveyty. Not free software. But one of the best-known commercial products here already. I'm absolutely certain that people are already looking at this for the free software things. I'll be brutally honest. I prepared the slides the other day and I haven't had time to check back since to really look into that. Definitely it's something that volunteers to help with would be great. Run Andy, run. So there's a class of bugs that is really slightly further out, which is unsigned time, which exists, for example, in every single PGP packet, unsigned time. So we have a few more years to deal with that, but also it's definitely not in a time t because it's unsigned. So we have to be looking for... Some of those bodges aren't just because it's going to be smart, the format is actually different. Oh, absolutely. I'm going to jokingly put my head in the sound and say, don't worry about it, we're never going to get that far. Well, at least I'm not. Secondly, yes, the kernel folks are looking at fixing this up better. So rather than just, oh, we need to put this down the road, we're not looking at any bodge work to say let's just extend this or let's... Lots and lots of the Y2K fixes are just let's change where the epoch is. So assume anything with a two-digit date must have started, must be relative to 1950. It's at a 1900, that kind of crap. We can't do that. We mustn't do that. We're better than that. We have plenty of time to see this coming and do it right. Now, the reason why, again, a lot of us are making noise about this is most of the systems that we are going to be running that we know about in 2038 are going to be 64-bit clean. You know, Debian and all of Linux distros, of course, are going to be fine because hell, we'll have rebuilt the world. The big problem that we have is that the internet of things, we have a vast number of machines. We're going to have by then probably trillions of little machines which we won't be able to identify. We won't be able to ward it. We won't know that they're safe until they fail. And we won't be able to rebuild them. So this is why we need to get this fixed as soon as bloody possible so that people building those systems will be able, or not so much will be able to, but will just be building on top of things that already work right. The 2038 is also the last estimated deadline, but anything that stores dates in the future, like expiration date of your carton of milk or appointments for next year or scheduling packs, whatever, or expiry... when your child reaches all voting age or that sort of things will start failing nicely in advance if it uses time to... Sure, absolutely. I think it's also an interesting question like which systems or libraries use time T which shouldn't be there, or it's not really necessary. For example, you said time beam functions. If I want to know like, okay, five minutes have fast, so wake up and do something, maybe I just do not need the 64 bits because it's whatever. I just need to know that like, okay, five minutes later than I started. So hopefully absolutely, libraries should not necessarily be exposing this up the tree anymore, but of course internally they will be using 32 bit times. For example, it's how select exposes things for your event loop, that kind of thing. We should be able, if we're careful, to fix those without polluting any of the namespace up the stack. Obviously, as I said, we're going to be looking for ABI checkers to absolutely validate that. I was just curious if you had any thoughts about, seems like there's a proliferation of these statically linked, things like Go and things like that. So just your thoughts there because it seems like we need to fix the libraries that they are statically linking in have to be fixed before we can even hope to rebuild all of those. Absolutely. It's a great example of the problem that we need to fix. That's looking say five layers up the stack or something. We need to make sure every layer underneath it is fixed early, so that they get the time to make sure their stuff works and B, that gives us plenty of margin to make sure things out in the wild are already fixed before it breaks. Yeah, so on a related thing or I said for the internet of things stuff, there's a lot of people who have been worried about already people shipping insecure devices or things that have no security support and it doesn't matter to the device producer because they expect to have already gone bust or moved on, there'll be three products down the line before anybody cares. Actually enforcing the ability to do updates safely, reliably, automatically on these devices is part of the solution to that problem and possibly also the solution to this problem but actually making that happen is hard. So going back to the time deltas so you want 64-bit safe, 32-bit safe and overflow safe time deltas, right? If you're calculating the map around it. Yeah, absolutely. So when I comment before I was avoiding mentioning this but you brought up IoT, now everybody knows that I see GPL violations as the problem of everything but it actually is a problem with the IoT devices so even if Linux gets fixed 90% I would expect of IoT devices on the market that use Linux are violating the GPL which means there is no source code for the Linux they're running, there's no instructions on how to rebuild it so even when Linux gets fixed and even if it's a Linux based device we probably won't know how to upgrade it so I would add to your slide of things we can do now, of course I would, but the GPL violations actually relate to this because if we have all these GPL violations in the next 20 years they're all running Linux, we won't have the source code for them, we won't know how to upgrade the Linux that's on them and we'll be stuck. Yeah, absolutely. That also adds to the urgency of getting this fixed as early as possible so GPL violations are really bad you know I agree with that I'm also more worried about this kind of thing killing people using other worse problems. Right, these GPL violations could kill people and we can't fix them before that, right? But I agree with you, the quicker Linux in particular gets fixed the fewer times this GPL violation problem will cause the problem. You've been saying that people should just carry on using time T for now, would that also extend in your view to people say designing a new on-disk format at the moment or should they do something else? Would you do something else? That's a very good question well brought up. No, in that situation of course your on-disk format is not something you're going to be exchanging with somebody else in the same way that you do through a library or through an API. Please God, if you're doing a file system design if you're designing an image format if you're designing a packet format today think about this people are going to have to deal with your format regardless so you don't have to interoperate in a clean way the same way everybody else does. Definitely think about this there are good guidelines in fact if I remember correctly on that Colonel Newby's page and on the Geolibsy page about how to think about this now if you're designing such a thing yeah, thanks very much for bringing that up. Don't necessarily pad out well in that situation the padding out yes, is if you absolutely want to store an on-disk format or something or an in-flash format that is somebody else's if you're designing your own think about it, do it right doing it right obviously is not always the same thing for everybody. Regardless of whether we fix Colonel and Libers or not and whether we even have ability to rebuild something I just checked my Nexus phone which has latest Android which is still supported by Google and it still contains Colonel version 3.10 so it is problem for all the embedded devices all the embedded devices it's a really long term absolutely that have got vendor kernels and binary blobs all of that kind of problem this is why we need to be fixing this ideally 20 years early that might be hard, 15 years early is a good goal and that will give us plenty of leeway before everything goes on fire Any more points? Thank you for coming along please dive in and help aren't actually specifically asked for me to run this session at Debconf there's a bunch of people going to be running similar sessions at other conferences or the distros conferences and everything over the next couple of years to raise awareness of this well again it was awesome to see most people in the room knew about this before we started it's not necessarily clear everybody knows the ramifications and why it's important to attack this early so thank you for coming along tell your friends as well and dive in and help if you have the time if you want to learn more I've linked the various pages I've linked to so the kernel and Glibc pages they link to each other as well there's lots of good information there's lots and lots of scope for people to get involved and make a difference now thank you very much