 So I'll shortly speak, I will talk about 3GPP specification version control. It's probably most important for people working and actually implementing the 3GPP specifications. And I'll illustrate it with a use case. For example, when we did the GSM network at the Congress, at the beginning we had an issue. And the issue is that the phones would not register with our network. So every time you tried on the phone to register with the network, it says, I'm not allowed to register to our own network with our SIM cards. And the mitigation was just restart the phone and then it would work. The problem is that when the phone lost connection with our network, for example when the people went back to the hotel or wherever they sleep or where they were just out of reach, the SIM card would try to connect to other networks because it wants to roam. But for the phone, it said, I don't know the SIM card. I don't have roaming agreements with this network. I don't know this IMSI. So it would say that I reject calls saying IMSI are known in our IHLR. And this triggers on the phone state where the phone is not allowed to register to any network again. Until it reboots. And the permanent solution for that was simply take the SIM card and write in the SIM cards in a list called Forbidden PLMN, the networks where you don't want the phone to register to. So now all phones without SIM card will never register to vote a phone. So the SIM card will never be blocked. But it took us a bit of a while to find out that. Now, almost everything is defined in 3GPP specifications. And this procedure is also defined there. So the section is the specifications is 24 or 8. This is the specification describing how the procedures for GSM work. And I will not explain the numbering system of 3GPP. There is a link at the bottom. And it's already complicated, but I will even add a layer of complexity because I will talk about the versions within the specifications. So the procedure here is really clear. Whenever you do a location update and it is rejected, if it's rejected with cause number 2, IMSI Note in HLR, it's written consider the SIM USIM as invalid for non-GPRS services until switch off of the SIM, until switch off of the phone. So this is what we found out. Or the SIM USIM is removed. Generally, you have to switch off the phone when you want to remove the SIM or put the SIM back again. So it's the same thing. But looking at things, there is an OR. OR the timer T3245 expires. And it's quite an interesting timer because we didn't see this happening. So I looked at, again, the same specifications which define the 3245 timer. And this says that it's a timer where you pick between 12 and 24 hours. And after this 12 to 24 hours, you erase this forbidden list which is saved in the phone and you perform the cell reselection. So it's a really, really large timer. Probably this is used for vending machines or fixed appliances where rebooting the machine or rebooting the phone is just really hard. You cannot do it remotely because you don't have connections anymore to the machine and you cannot send anyone on site very often. But the problem with that is that it says also that this timer is specified in another specification. In this case, it's 31010. And this is a specification for USIM, so whatever is written on the SIM card. And if you look at it, this parameter is in an optional file which is on the SIM card. It's an optional field. The behavior is an optional field and only the operator can write it. So the user cannot say, I want this timer to be enabled. The operator has to enable it. I checked a few SIM cards, it's deactivated by default. So that's why we've never seen it in real. But maybe for vending machines, they have this option to do that. Then I looked at the specification again and you see there is a small A. The subclose, so the clause A. And if you look at it, it says that if you don't use timer 3245, you use another timer which is called 3247. And this time you draw, the timer is between 30 minutes and 60 minutes. So it's a lot shorter than the other timer. And after this time, you again do the location update procedure. So that's really awesome because this saves you between 30 minutes and 60 minutes. You will re-scan all the network and connect back to it. But this is a specification for 2018. When we had this problem, it was 2012. And you can see this clause A didn't exist back then. So the problem with the specifications is that they keep adding features with the specification, which is normal because you want to have new radio, you want to have advanced RTE, and so on. But it's really hard to know what is new and what is not new. And this is where specification with versionally control is really important. And some specifications, and most specifications, you have release nodes. And one release is just one line. And if you can try to look for T3247, there will be no comments in the release nodes. So it's pretty hard to find when, in which version was this feature introduced. And this is where I will talk a bit about specification. So all the specifications are written in Microsoft Word documents. You can download them, all the documents really, in zip archives. The release nodes only contain one line change telling which feature has been added or which feature has been changed. And sometimes you even don't have the release nodes for the previous versions. So you really have to download the old version and see the release nodes for the previous version. So if you have the last version, it doesn't mean that you have all the new features. And it's really hard to track what the changes are from version to versions. And there are new tools which really allow you to do that. So the specifications are really written by humans, for humans. And only the people who write the specifications actually kind of know what are the new features which are added or not. To change that a bit, I wrote a small script which downloads all the specifications. It extracts them, converts them into text files. And text files are quite useful because you can grab into them quite fast instead of PDF files. And because they are text files, you can version them. So I put everything in a git repository. The script is really just one make file which takes care of everything. And you can find it at 3gr. And the result is that just the archive, so everything compressed is around 100 gigabytes. And everyone tells, yeah, there are thousands of specifications. Well, now we know there are 3,200 specifications. And when you count the versions of the specification, you are 52,000 specifications. So downloading everything, extracting the documents out of the archives, converting them to text, takes around three days. And in the end, you have 230 gigabytes. I have hard drives here with the dumps. So you don't have to do everything here. You just copy my hard drive. It's the easiest. Or if you have a lot of time, do it yourself. The problem is we can also not make all everything which is processed online because the specification version are copyrighted. You are not allowed to redistribute them particularly when they are modified. Transforming them into text removes the picture and so on. So it is modified. So I am not allowed to distribute the end documents which you can grab. You have to do it yourself and wait a long time. Once you process, every time you want to update, it takes around half an hour to one hour. But then you have also all the updates which are versions. So let's try to use it. Now I put versions on all the text. I am trying to search for a section 4.1.1.6a which is the new section in the 24.000 text. And here you see the commit. This feature has been added to release 13.5.0. So the first number is always the release which tells the major steps. So for example, LTE has been introduced with release 8.9. New radio 5G will be released with 15 and so on. Afterwards, you have the small versions. And the last number is almost always 0. Except for the first documents I haven't seen it change. But now we know in which revision it has been changed. So it's quite good. So here I just looked for the big change but you can use bisect to find smaller changes. And you can also look at the diffs and you here we can really see that they added the paragraph. So this allows you to see when it has been introduced but also have the complete diff of the feature. So you know what has been changed. And in this case, they've added also this timer in the authentication reject and a couple of other places. So when you're implementing this feature, you can see the differences. And now you know, okay, I have to implement this new timer here, here, here and here. They've added this feature. Problem with this also is that it shows the diff of the complete version. It's not only this feature which has been added but a lot of other features. And you don't really know which feature belongs to what. This is where we'll talk about how they do revision control. If you see in the change lock, you have an entry line. And now we know it's in the version 13.5. So I took the version 13.5. I looked at the release notes and the closest I could find is correction of the integrity of the reject causes without integrity. This was the closest description for this feature. So on the left, you can see other numbers which are related to these specifications. And I'll decode it rapidly. So the first number tells you at which meeting this feature has been introduced. The second number tells you, and the first number tells you who introduced it. The second number tells you which pack of feature it is. The third number tells you the change request. So it is really this single feature. And then you have other numbers. But based on that, you can go on the FTP, you just add these numbers in the path. And you find exactly the document which introduces this feature. They are called change requests. This is how a change request looks like. And this is what they work on. So it really looks like a commit. But for humans, you have the source file, the commit ID, which is on the top right. The branch name is the CR request. You have author, the commit title, the commit message, and it's a Word document. You have the diff. So this is how they explain what they change, what text they introduce, everything like that. And it's done in Word documents. What's interesting also is that you find more information about why they added this feature. In this case, we now know that they added this feature only for rejects which were not integrity protected. So there were attacks which have been demonstrated where you really reject the phone. And you could do a denial of service because you just roam around with an empty catcher. You deny the phone and you request it. You will never connect again. So this is why they added this feature because generally when you are a rogue base station, you cannot send authenticated messages. So in this case, a real network can really reject yourself. So this would not help in a case with Word Phone. But any malicious base stations, you would be able to recover from the situation. So that helps quite a lot to understand why they added this feature and gives you a kind of background. Then we saw on the top right, you have a commit ID and we have revision six. So it took them a bit of time to write this patch. You can see, you can actually search on portal.3gpp.com or don't remember. You can see a list of all these change requests. So where is my mouse? Here you have the change request. Here you have the affected specification. And here you have the revisions. So the revision and the commit ID. So it's this particular document which they talked about. And using that, you can actually build a graph on how they are doing the commit. But everything is done manually. So I just took it, you have the source file in the branch. They use this branch. Then the branch is the new feature which has a change request. And you can see the different commits and the technical specification group. There are around three of them. It really depends when you look at it. But there are three technical specification groups which cover different aspects. Within these groups we have around four work groups which are more specialized into something. So a change is done by one work group of one of the bigger groups. And they talk about this change, about the document and they really meet every month or so or more. Also they are really their calendars. And they group all there and they discuss and they change it. It has been revised. And it took them six version before everybody agreed on this patch. And what they do afterwards is they push it to another branch which is a CP. CP tells you which group from the subgroup is deciding on it. So the small subgroup said it's a cool feature. We'll add it to our group. And the whole group is meeting regularly but less regularly than the work group. And deciding if this feature is good or not. And so this was the group, the meeting 71 which you saw in the change log. And they thought, yeah, it's a good feature. We merge it. But this is what they discuss about is not this single request, this single change request. This CP includes around 20 to 30 change requests and approve all the 20 or 30 change requests. They can also reject individual ones but they generally talk about them in packages. And this is... And it's done by humans. The whole tracking and all the numbers done by humans. C195. Oh, you mean here and here. So C195 is one meeting. And three months afterwards they had another meeting. And so here they said we agree on this feature but it's not complete. And on the next meeting we will... Some guy said okay, I want to add this more and so on. So it's hard to see here but actually I'll come to that afterwards when I talk about the CR pack. Oh yeah, the CR pack. So it's important to see also about not just single CR but it's... This CR pack is for one particular domain or one particular aspect. And it's important to look at it because so the CR 2927 which I talked about is right here. But if you look at the other CR you will find the exact same thing correction of handling NASA jet messages but for another specification. 2431, 301 is for LTE. So they added the exact same feature in LTE. So it gives you a clue where this timer is also important if you want to implement something in a new feature. You can also see that they talk about almost the same thing. Authentication reject and authentication and ciphering reject in another CR which is 2928. But this was at some point rejected. Not pursued. Meaning that the guy who wrote the specification they said it's almost the same thing but it wasn't as complete. So the one which has been accepted actually integrated this feature. This is what you can see to the agree. And then they discuss later on or the next meeting they agreed again because they merged this request. They merged this change CR into the more complete one which has been agreed later on. You cannot see it directly so if you do the branding by yourself you can see it. And actually... Did I write? Oh yeah, there is a database. There is a machine-readable database about how the CRs which CR and which internal document reference to. It's also maintained by humans but at least it's machine-readable. And there isn't even an online tool which allows you to search for CRs and you will see in which documents they are and you could actually plot such a graph using their database. But again, done by humans it's not automatic. So the conclusion is that the 3GPP has quite good version in control but really done by humans and for humans. It's not machine-readable but it's kind of possible to make it machine-readable. Thanks to Git it's one of the things. But as you could see Git only works on the end specification. I don't track all the changes and so on. I think this is done for human and they had so many exceptions. Even just for the specifications I've seen half of the script is just handling exceptions and I don't handle all exceptions. And a last note. I'm also disclaimer, I'm not part of 3GPP. I never attended a meeting. This is just what I found out while trying to work on this script. I'm sure there are a lot of errors and I didn't talk about the meetings and all the other documents. There are plenty of other documents which are interesting but even harder to read. And also what I talked about are the specifications. They are not standards. So the 3GPP only writes specifications and then each country or nation state or whatever takes the specification and makes standards after them. This is why generally when you go on the Etsy page to download the PDF you don't find the latest version. It just takes some time for the Etsy. So the Etsy is the most known one which is responsible for the European standardization. They take a bit of time to take the 3GPP specifications. They adapt them. So I don't know what change they make to it but then they make them official. But these specifications are also available. So maybe if I have some time I take the same script, change it a bit, download these specifications. And then actually I could diff what the Etsy done compared to what 3GPP done and actually see what the differences are and which country does which change to the specifications. It would be interesting. They probably change from American English to British English. So Etsy is really the best of them. The other ones they are not really complete. For example in the USA they don't care about the core network. They only care about the access networks. Yeah, that's it about how versioning control it works and how the script can help a bit if you want to implement new features or at least keep trap of the features. Any questions? How long did it take? So writing the script itself took around one week. That wasn't too long. The longest is just waiting that things happen because it's a lot of files. Adding exception took another week for the most important documents but I didn't add exception for everything. So I will with time whenever I open a specification switch I need and I see something is missing. I look where in the conversion this thing has been missing and where it's been removed. Also I want to add a couple of features because here in the change lock you actually don't see who is the author. You don't see the date. So which working group and which is the date. I can extract this. So in the change lock commit I could actually add the real date and in the commit message I could add the list of CRs which helps you to find out which is the important CR. So these are features which I could add in the end. And finding out how it works. This was when I was working at Qualcomm I had to track. We had incidents about ATE and I had to track when this attack would be mitigated thanks to one of the specifications. So this is where I learned a bit how the meetings were working and where I really at one case really tracked every document and found out when it was added and so on. You mentioned file size of hundreds of gigabytes. Was that the extracted text or the PDF? So here I could make, there is a thing which shows me the statistics and the PDF. I don't do PDFs. So I take the archives I extract the doc files and from the doc files I convert them into HXJMN and then text. So no PDFs. I don't know the size of the PDFs. But from, I don't have the hard drive here, but if you make statistics it will show you how many doc files and how large they are. So it's 200 gigabytes of text. In the end, yes. So it includes the, yeah, it's 200 gigabytes of documents and text. But it's not plain text only. Also the doc files. But I recommend really to do, to use a file system which draws transparent compression because out of 200 gigabytes you are at 115 gigabytes. So half of the size and works quite well. It's quite fast. Saves a lot of space. And this is only for the TPP. If I do the same thing for the Etsy, we'll have as many files. So my question is about licensing. So you mentioned license terms for the specifications themselves. My question is about if there are, if there's some, some restriction or the same restrictions for the change requests. Or if it would be possible to, to include most or relevant information from the change requests in a public repository of metadata. Without any, any issues. I don't really know. I think they were under the same copyright. They're all publicly available. And I didn't talk to the TPP if they would be interested in this tool. They just do it by hand and meet every time with printed documents. So, so I don't know what the lessons terms for the CSR. I would think they are the same. You're not allowed to transform them because again of the diagrams and so on, but I didn't talk to them. Yeah, I would be surprised. There's the same copyright statement in there. So I think it's the same. I think the only way how you can make this like the result of your process public legally is to find some people at 3GPP who find this interesting and make them hosted themselves because all the information is provided by them. It's just nobody is permitted to redistribute it. So if they would run the resulting Git repo on one of their servers, then it would be completely legal. So at the very least, it would be possible to have a public repository with links to the, to the actual CR. Oh, if you look at the portal. Yeah, it's portal. You have links to all the documented and so on. So it's not bad. It's a bit slow, but they added they improved a lot. This is new. It's not, it's not a tool website. Right. But I mean in the generated Git repo, you could, you could link possibly even deep link to change request level information on this public portal. Yes, you could not change request itself because it changed request and zip files. So I could. Okay. I could link to the zip files. So the group of change requests and then it's up to you to do it. But yes, it would be possible if I extended to do that. Would be super cool for 3GPP to, to take some interest in this and export. I'm not sure they're aware of this, of this project or listening. Well, they should be. So exactly like, what are the different steps that take three, three days actually? Like what's most of the time invested on most of the time is converting the document files to XHTML files. So this took for me took one and a half days. I don't have a super fast computer, but downloading the files the first time took takes around one day. Updating them takes around 15 minutes. So you can, the script allows you also to update. So it just goes through all directories. And so is it a new specification? I will download it, extract it, put it in Git version control. But yeah, one day for downloading one and a half days from converting from doc to XHTML. And then a couple of hours for converting it to text and putting it in Git. I guess you're caching the files when you download it somewhere. So you kind of, you don't need to re, re download them in the script. Yeah, they have two ways or how you can access this specification. One is over HTTP. So there you, you it's slower. It takes, so it's faster to download the first time because I don't know the HTTP server is a bit faster. But afterwards it takes two hours to update. The second method is over FTP. FTP takes a bit longer the first time, but because they have listings, it's, it's really fast to update. So it only takes, it goes through all the directories, but it's just those listing on the specification. No date change. I don't have to request every document to know when it has been changed. So updating over HTTP is, is a bit faster. Over HTTP is possible, but it's slower. And for Etsy documents, you have to sign in to get the archives with the docs. So I'll see how I'll handle that. Since this is a tiny little bit off topic, but also on topic from what you chose to, to the project brought us to this. Because as anyone knows who uses a dead SIM card from an operator, it's really annoying when if it goes away from your test network and then gets blocked. Because you have to power cycle to get back on your own network. But just I had a question about what, what is, what is integrity? What is a, what is an, a reject then with integrity? Okay. Because if, if, if you are, if we're using our own SIM cards, then their network doesn't know the keys. So obviously can't. Oh, um, yes. Or how is that working? Um, this, the, so in DSM, it doesn't matter because there is no integrity protection. It matters in 3G and LTE or MTS and LTE because there you can have integrity. But what is important actually when a rogue base stations gets a Vodafone client on its base station and then rejects it. So the guy from Vodafone knows this is a reject, which is not integrity protected. So the Vodafone client is protected. It is not about protecting the non Vodafone against the Vodafone, uh, against the Vodafone network. So there's a kind of a balance which they decided on. They don't want millions of subscribers to use this timer every 30 minutes and log on again. Because if you have a rogue SIM card and expired SIM card or so on, you would have one update request every 30 minutes. Thanks to this timer. This is why it's only using the timer for non integrity protected networks with their own SIM cards. And because they know their SIM cards, they could protect it. So but we at the Congress network, we have our own SIM cards, right? So could we use this feature to, I mean, yes, we've put Vodafone in the Forbidden PLA man list. But could we also benefit from this feature? I can, so this rejection was just a use case where it's important to track features. But we can talk afterwards about how to use it. How about the timer?