 Hi everyone, I would like it to say good morning, but it's already noon, so hi everyone It's noon and I'm glad to see so many faces here because the last time and Stephen Doesn't give me to like here last time. I gave this talk a flock last year We were happy for people in the same room a video operator to people who came to listen to the talk and I'm presenting there, which was nice because in parallel there was Korea's Talk everybody went there. That's perfect because people love new technologies and it's great to have them there but I'm talking about all technologies and Actually, we have 40 years anniversary of these old technologies. So all the functions that we use for granted get PWNAM and get group stuff came in 1979 right and That's amazing We use something that was designed at 40 years ago And it's still in use in production all around the world and basically we don't know how to get rid of it But unfortunately with any old technology comes a nice thing called Off by one error because it's 41 year this year not 40 And really all those API's if you look at them I come back here So name service switch which kind of obstructed the way of using it came in 93 so the Authentication API subtraction came at at the end of 90s So we really talk about stuff that that really is not from this century it's maybe even out of this world and Kind of last API's API's made in in 90s That's what we deal with all the time and those who were Or familiar with Finland history. This is the stuff that finish Students are doing every year Upon graduation or before they get to the matriculation exams. They all jump on those tracks We're the these kind of funny stuff that they can find draw these posters and Ride across their towns Throw in kentus and there are people trying to pick up these kentus in Helsinki for example, this takes Something like two hours because there are many Colleges and every one of them is at least two or three trucks like this, so it's it's a fun and these were really the last students made in in the 90s so 1999 and 2000 and they ended like when it was two three years ago. That was my elder daughters Graduation thing but really if we look into it the evolution was really slow, so that was kind of a bit of fast evolution in 80s a bit of aggressive transition to Simplification of those technologies in 90s, but since 2000 There's a bit of stagnation, so we reuse the protocols were used technologies on the system level that Proven to be working and it proves to be hard to change that and Agreeing something that applies on again on the system level while at the same time on the on The application level there is a huge diversity and an approach is the web basically changing everything web 2-0 all these semantic stuff that got replaced by Rest API's and so on and we got into interesting situations in 90s and Early the thousands we got bunch of projects that implement the system level thing for remote access in something like LDAP protocols, so there were a bunch of Projects implementing how you query post-ex-data from LDAP how you plug it into your system How you use Kerberos? There were two PumpKirby 5 Implementations at the same time right there was a smart card handling with PumpPkcs 11 Every big system and of server-side provider Dean to introduce their own NSS and PAM stack modules, so it was flowering There but with this variety of implementations of kind of standard API We got where it is. Yeah, we got to a Typical let's say linux distribution that has bunch of these modules, so This is Fedora like two releases three releases before right we have SSSD provided Lib and SSSS says that we have bunch of modules inside to FC the standard ones We have Samba providing its windbind connector We have Lib and SS LDAP from an SS PAM LDAP and a ONL CD projects That give access to LDAP stuff and on the PAM stack. We also have those those modules the problem here is that it works for Single machine and it works and of nice, but if you need to deploy configurations of this to a fleet of thousand servers thousands servers or worse than that Workstations, which might be a bring in Bring out stuff in your office. You you will have to deal with the Evident configuration changes that happen on a particular machine, especially if there's somebody root other than you like on the work workstations you need to add complexity of the Local configuration tweaks because if if this is workstation then most likely I'm Wanting to use it against my home network as well as Corporate network as well as like with Fedora. I have Fedora Accounts right which also requires certain handling because Fedora provides the Kerberos authentication to access some of its endpoints and so on so we got also Requirements for the universal access for user and group data beyond POSIX so POSIX API for example for retrieving user information never in 40 years Had any kind of field for email address for new generations emails becoming Something that they might start asking what it is because they they moved on to stuff like what's up and Social networks and maybe don't have any email at all, but in 40 years that email Matter it it did not exist in the POSIX API So the first thing you do when you connect your web applications you use some sort of identifier and that Identifier needs to come needs to be Some something unique many applications. We're using email addresses for that because that allows you to also get back the Notifications to the user over the channel that at least be predictable in what it is and If you map on the same database that contains your corporate users or your Fedora Contributors users you don't get through the same POSIX API access to email address as basic as that so there are requirements that might look contemporary but really they They are not fulfilled by anything and So another example is home D Which is a project from system D Family of projects where they try to store much more information about the user like user related Keys that would be used to encrypt to create encryption of the home directory and some additional details about it Which cannot be resolved through the traditional POSIX API for advanced but on the other side with this variety of implementations we get a lot of Issues and those issues are typical so we get bunch of modules these modules need somehow be configured but they also were written here with Often with the concept of not taking into account that they are loaded directly in the application and therefore their state actually is capable to See what's happening in the application and not the same side Application itself can see the state of that module. So if your module for example loads Credentials needed to access some remote Database like LDAP binds the application technically can Look up into what is loaded into it itself via glibc nss modules and Pick up those credentials, which it's supposed not to have access at all. So there were approaches to have privilege separation talking small shims over the Unix domain so it gets to a demon that runs in background. That's effectively how SSSD works or how windbind works or how NLCD and this has palm LDAP D work. So they all have and a separate demon, but Many of those don't have like palm care B5 doesn't have it You need to load all this stuff in your application and sometimes you might have libraries also loaded with this That clash with the libraries that you're supposed to use an application ABI problems are real and they do exist and but the other part is many of these modules over 20 some fish years became broken Abandoned in the upstreams Completely removed the the owners might The people who developed them might switch to a different topic which is perfectly okay. That's their life and From the distribution point of view. This is probably the worst thing you're you're becoming the one who maintains it so streamlining this kind of a stack of Thunderweed Becomes an interesting thing Especially if you need to support this for 10 years or even longer and Also a fanciness Weird out 40 years is a different fashion now It is something that many Students or used to be Graduating and going to work. They look mostly at the mobile wall and see potential there not working with this boring Protocol stuff that might have no future from their point of view Again, it's her with us. Well, we just faced the real problem of All struggling with real maintenance problem for various reasons So life may be easy on the web side Because there is some rest and there's a stress here and The funny part is that I think it was two years ago that the year We celebrated the decade of SSSD in fedora So the core 11 was 2008 2009 so yeah a decade of Of that pass it so fast We gained a lot of functionality gained a lot of Support for a lot of things it's now one of the pieces of the client side That that exists everywhere almost in all distributions. It's packaged and assumed to be Used there are some things that are kind of debatable, but that's okay. We we Go forward and improve through debates here, right so on the other side If we look at all this stuff, then fedora actually quite conservative If you look at rail 7 and rail 8 development rail 8 actively deprecated some punch of the modules that still available in fedora because there is a voice of Customer in fedora asking to keep it and maybe there are people who are willing to maintain it unlike on the Red Hat Enterprise linux site where maintainers choose where the battles are and For some of these things it's it's better to focus on something else so for example the pumpkin b5 was removed from rail 8 and As I say, there are two pumpkin b5s. One of them was dead Well Maybe five years ago as soon as red hat removed pumpkin b5 from its Soon to be at that point rail 8 And publish it better in three months after that the original author of the presumed dead pumpkin b5 in dead end Restored its work. I don't know what was the reason that he started doing this again But he started doing so now we got the dead module restored on that zombie, I don't know We all zombies we walk for years the same was with one of the pks is 11 things but Here the progress was quite Quite big on the Smart cards infrastructure side, I'll get back to that part Thanks to our colleagues and Yeah, the other neck and the others on the unification of the smart card access and So all of these this functionality is not lost. It's all provided through SS Same in fedora just that we have options But on the other side configuration of those options happens in I Would say there's a very success Failure rate and sometimes something very simple might might be looking innocent to the package change a package maintainer that changes it and Causing the problem. So for example, look at this the Off-config thing existed since 1999 So quite a long time the off-config has standard standard thing The if if package provides a module for example this finger print. This is the real bug from fedora three years ago, I think the module installs its own stuff and updates the configuration and When you remove the module, it's supposed to remove itself Otherwise, it will be some something will be referenced in the Pam configuration that does not exist which might not be pleasantly You know usable for for the planet stack itself and it might just fail to allow you logging This is exactly what the bug made with it Unconditional removal Disable fingerprint Was causing the problem with the removal because Update doesn't have the off-config doesn't have the estate of what what the other options were there You you call update and it wipes out the configuration from what was there in simple cases It works in more complex ones when you have windbind, but you have a SSD or something is not there So at some point off-config Was also needed to be Rewritten from Python 2 to Python 3 fire that kind of migration Needed to happen because Fedora moved us Yeah, I know I know it was set story in in in in all of this and Effectively the decision was made. Let's let's do the whole refactoring of this space with a new tool and new tool was created and Yeah, it it caused it some gripes around it But we sort of simplified things Among the configuration things but didn't do really the actual Setup of the things that get configured. So if you set up your palm stack to use SSSD It doesn't mean that SSSD is configured. That's for some something else. So for example Where is it? Let me get Yeah, get back here All select sets the NSS configuration to the predefined one PAM to predefined one, but something else should create SSSD or Samba configuration to to use and That's done with the let's say IP client install or realm join dependent on the target there and Finally ansible roles that that create something and become kind of ubiquitous nowadays. So With all these on the single host you kind of get more or less predictable Scenarios predictable configurations you can extend them if you want but for the majority of people This is not and it allows you to reduce the scope of of maintenance and support because if the tool knows about the basic kind of configuration and Handles it well. It allows to move move away from the configuration itself into let's say SSSD in this case all the other steps Like handling all this configuration for the certificates and eventually move them centralize it like IPA does or AD On the authentication part the grid this was a big advancement over the previous state by effectively having a unified view on to what what is the Smart cards and certificates in in the system and how they can be used Effectively to P11 key in almost all cases except no PG Whatever is above is Using P11 key one way or another which means that you can do nice things like specify the very simple URI I Think it came in around 2015 this specification for the URI was implemented in a year after that That allows you to just say okay. I have a smart card With this token in it and if I specify this URI or I have tools that actually point me to this URI I don't need to handle all the configuration all the all the things it it magically appears and Works so nowadays it works in almost everywhere We have fairly Good stuff But on the Kerberos side There was also progress. This is not Kerberos, right? This is not a three Three had a dog that you expect but it probably looks like scary stuff the off of those owls attacking you when you try to actually Configure and use Kerberos like I saw on many mailing lists. We we also dealt with the content of how many 30 35 years history of decisions made 30 35 years ago, especially around the DNS Using DNS for the resolution specific encryption types that were there inability to operate in multiple realms Environment where you have multiple providers sources of the truth of your authentication data, which is for example, very simple. I have free AP at home I have free AP at Fedora Those are two different the hounds one of them I'm using to log into to my system another one I'm using to log into Fedora Things when I do commit for my packages and then I have a third free free APA at Red Hat at work where I used to access all the services in Red Hat So kind of I have three of them. They are different. They do not trust each other and That's good, right, but I want to have access to all of them at the same time because Here I'm logging in into the machine then I am building a package and Building a package for Fedora might be actually part of my work During the day and also accessing the red hat resources also might be there So I need to have some sort of a common access at the same time and depending where we are trying to authenticate that proper credentials should be taken care of and This this work This problem was known for years and there was an attempt to solve it in Kerberos with So-called director credential caches then Trying to increase security of that with the kernel key ring based at C caches all of them had their own limitations Finally when it was I think again ten years ago or so Heimdall introduced the Kerberos credential manager protocol and It was picked up by the MIT Kerberos by 113 version again six seven years ago But that's the client side the server side was really never implemented well until We did this work in SSSD to implement the storage and nowadays Fedora can use and I think it's configured by default to use the KCM provider from SSSD which can be used independently of Using any other features of SSSD. So if you don't need to have LDAP stuff That's fine. You just use the KCM part for Accessing Fedora infrastructure for example and Since these caches cache types they they existed at the same time they implemented and you can use them on your kind of Depends on where you should use it It becomes quite The only problem we have with with for example key ring is it's it's great. It's it's useful for Having secure access to them, but it's not namespace it So you cannot use it in containers containers will see the same key rings that Is in the host system or in other containers if they share the same host ID for that carry there is a work on going to create namespace for the key rings This work is still not complete the patches were flying last year Quite a lot. I think that there are basics out there but for example file systems networking file systems do not know how to pick up from the Container riser things and there are some existential problems how you run these Networking file systems mount them within the containers how you treat them how treat IDs and scope them so KCM allows you to kind of sort this problem because in the end It's a unix domain socket towards the client between the client and Unix domain sockets already namespaceable so you can separate them naturally already Okay, and one thing I wanted to say is that Aside from normal fedora use where if you create a new fedora installation and you don't copy any files By default fedora will use KCM in the Kerberos operations, but fedora toolbox container Actually notices that you you have access to the KCM and you have some tickets there and Automatically pulls it in into the container. So when you in the container you will Automatically get your Kerberos tickets from the host system. This works on fedora core as for example Quite nicely, but only in the fedora toolbox container or something you built on top of it so there's Something to to improve the other problem we had again if you have multiple parallel Kerberos installations that don't really talk to each other don't know you need to somehow handle the DNS resolution of the of the services and Kerberos can be made not dependent on DNS to the level that Everything that works automatically, but it's really very boring situation. It's better to have in some cases better to have the dynamic reflection from the DNS to Get the load distributed between different KDCs and get Some some way of saying hey this set of hosts belongs to one realm Another second belongs to the other one. The problem is that it's it's a Typical deployment of this 30 40 years old Behavioral so people tend to for example use SSH Server access using just a host name not the fully qualified host name and that doesn't really work well with Kerberos So there was a method that kind of expanded the host name to through through some magic and That method was broken If you switched it kind of to always require the fully qualified name for security reasons and whatever This kind of canonicalization and then this is going Public with Kirby 5 1 8 18 but Fedora built of Kerberos has the DNS canonicalization tri-state patch for year or two already and allows to basically be smart if something is not resolved or you can fall back to another method or you disable this behavioral and Depend on the Non fully qualified name or always fully qualified name there depends on what what you deal with this is especially handy if you have gained multiple Realms and parallel and you have one rules and one of them and another rules It also helps open shift basic applications where host names host names for the containers They typically are not fully qualified within the We have been looking at the other things there is a mechanism to proxy access to the Kerberos Key distribution centers if even if the client is not really having direct access to them proxy over the HTTPS connection this KDC proxy it doesn't work if this KDC proxy information needs to be resolved dynamically through DNS currently. It only works via Explicit configuration and there are some Low-level details why you cannot do the discovery of multiple of those at the same time and choose one or the other So that the work is ongoing mostly in upstreams of Fedora Sort of special here because it gets hammered with the upstream the newest upstream code Faster than the other distributions in this area. That's happened to be a problem my and Robby's fault So sometimes it's a bitter experience sometimes we actually Get smooth and sleeker stuff on the other hand a Lot of work has been done in Kerberos community to finally say no for with crypto and say no for the stuff that is known to be broken and You cannot really imagine the level of opposition from admins That say we have stuff that works. We never touch it Even if the actual system that provides that stuff Already unsupported by its vendor a typical situation is something like Windows Server 2003 Which apparently is so so widespreadly used still by the Big organizations for built-in-the-air forests active at the core of the forest they keep the oldest thing and Never updated or updated or something really conservative the key with 2003 server is that it doesn't support any new crypto, so it requires desk three and and desk single desk was used for NTE hashes in the windows NT networks where the password was stored So it's it's inherited from that. So there was finally an attempt a couple years ago to build up Request for commands for removing both of those from Kerberos making it last last notice and Kerby 518 removes Completely this support but on the other side we have system-wide crypto policies which allow us to force applications to not use those Again if the support for the crypto primitives exists in the crypto library Somebody can can try to use them outside So we remove that conflict except the RC for Which is market-deprecated, but it's used within the SMB protocol and within the active directory In couple places where you cannot get away So there's a work on going to fix that But I will talk about this a bit later. So the other part is introduction of crypto based on relatively recent research as in 20 years and 20 to 10 years let's say this way so so-called spake pre-authentication implemented now in Kerberos for quite for three four years and enabled by default in Fedora this has a good Feature that your passwords cannot really be Attacked over the internet so Kerberos originally was Created as a way to work over the insecure networks, but it didn't have a mechanism to prevent Impersonation and one of those mechanisms was to To have encrypted timestamp, but you still have the window of opportunity to get there So spake closes this window of opportunity for the attack and if you disable all these older methods We don't disable them by default yet But if you disable those old methods and your client support spake like all recent figures You prevent the dictionary type attacks on your infrastructure through the Kerberos quite serious The other part is that you could have a mechanism that says Okay, I used smart card or I used multi-factor authentication to obtain this ticket and you can use this information from the ticket at the server which you try to access to Say you cannot access this service if you don't have for example multi-factor or smart card Obtain a ticket so something like using a password is not enough to Obtain it you really have to raise your security to access resources Free APA implements this through so-called authentication indicators and you can associate the service with an indicator and then Application that runs this service will never see requests from anything But users that actually fulfill this because it will be stopped at the moment where a ticket needs to be issued by the key distribution center There are yeah policies now that allow you to handle a bit But we are it's becoming a bit wobbly because we are still in the process of Defining what you could do how you can reassign certain properties of the ticket that is being issued by KDC According to how you pre-authenticate if you got something like OTP which is multi-factor authentication Ticket then you might have some sort of extension of Lifetime of the ticket so it could could live longer or be Renew it for longer time or maybe the other way around. It's really policy for the administrators to Yeah What is what hardened? Yeah, the question is what is hardened here. So it's it's a spake basic authentication and time stamp encrypted Authentication either of those basically use a pre-authentication with password in both cases just We define as hardened Just because it is not clear password Basics negotiation We use some pre-authentication think about it that per authentication at least happened before we actually authenticate On the edge side We have Kerberos 118 which hopefully will land in Fedora 32 or in rock height before branch it We kind of working currently on the breakage on a bi level between different things we get Ability to Look up into the content of some structures and the ticket before the ticket is issued So the free AP a Driver on the KTC side can actually modify certain things insert or remove or deny with a greater flexibility than before which allows us to say to Microsoft Systems the Microsoft Windows systems that this user who used PKI in it To authenticate with the smart card on Linux machine Actually is the smart card authenticated user and the Windows will recognize that and will do Certain things it does for smart card authenticated users not implemented yet This is where we are going on the both directions so that you kind of get fine-tuned settings so for example one thing you could do is to add to this MSPAC structure and Information saying that this user has a higher level access Security level access by setting a special well-known Security identifier that says this user is not really accessing as the lowest privileged one but can get access to the files that protected with some higher high security labels on the file systems We were not able to do that before we would be able to do that with Purpose 1.18 1.18 also introduces the resource-based constraint delegation support which existed in Windows for quite some time Finally completes this it's a bit complex topic, but it allows you to close down Decision-making on who can access work Across multiple trusted realms We have certain things cooking and impersonate whom in in that presentation and use the credentials of Ticket obtained from that in the options Also Kerberos 1.18 merged interesting module Called Nego X which is really like a shell where where you put something working in it Microsoft put so called PQ2u Thing which allows it to do peer-to-peer authentication for the machines that do not belong to the same domain While at the same time proven identity of a user by binding it to online identity like your Open ID Connect identity so going outside of of the old technologies and maybe allowing us to to get Some nicer path forward with the new and fences Don't know how it was for how this will work out, but there is a possibility at least now with the infrastructure One thing we were looking for spake is to actually handle long-standing problems of Multi-factor authentication where you need to have some Secure channel between the client and the server to pass the token for validation on the Kerberos side and The work for the base of this passing is sort of in place It needs rfc to explain the multi-factor part of what is put into the spake exchange and then implementation, but this is already getting getting to To the horizon let's say this for you so and on the longer-term work Yeah, having the Support for all this fancy new tokens the web open Thing for Kerberos It needs a lot of Specification first understanding how this web-based exchange and proof of Access and authentication can be mapped into Essentially non-interactive non-browser thing then how it mapped into particular Identities again identity management where the token itself might be issued by Absolutely different party you if you're buying this you be key for example or any anything other it has nothing to do it has kind of a inside it a Certificate with Authority that has nothing to do with you or your organization and you need to allow mapping that to to your accounts and Somehow so this is not written even specified somehow Let's implement it Okay, so there are dragons one of the bigger and of Things that happened in divergence between Fedora and rail was that in rail Open held up server was removed because red hat mean that really there are better ways to support customers with open held up using open held up community Partnership there on top of rail apparently there are so many customers of those companies that actually run braille But the support better comes from people who actually develop open held up But in in fedora we have on this at this time like for Alternatives for the directory held up basic directories One of them is two of them are folding into kind of one But with different purpose of 389 DS server as a generic held up server Free APA as an implementation of a particular view on the directory and identity problem set then Samba AD Which tries to implement? AD compatibility and open held up and 389 DS does a great 389 DS project does a great job on introducing the visual Handling and tuning of the Server it's it's all available in fedora, and it's really really making the experience of using Directory server and seeing what's behind the bonnet is Much easier there. There is also a command line tool in but integration with cockpit is Kind of interesting and I would recommend if you look at this even with free APA You might want to look into the cockpit plugin on the free APA side we complemented work done for the Kerberos by removing some default settings aggressive defaults for tips mode Trying to Follow the system why crypto policies not everywhere. It's still work in progress, but we are going there and with Fedora 31 We actually got a First step of our free APA Samba integration in place so you now can have Samba file server running on IP master in a support of an IP client enrolled into a free AP environment in a supported shape Before that it was like a bit of Hacking here in there still there are things to be fixed to be Produced but we also forced to use Kerberos only here. So we closed down the password-based authentication for example and Try to to enforce and get better Better security there There are a few things interesting for example for Fedora infrastructure folks like the hidden and advertise and advertise it Replicas which allow you to have something that just used for back taken backups, but skipping the law from from the Clients that try to authenticate or search for it. They will skip it and The other part is the extended group management. This is what we was talking yesterday in the Fedora account system rebirth thing, so I hope we Get improvement on this, but this is also useful in other places where you need to grant two-step level kind of access doing the group membership by Designated persons rather than the administrators of free AP itself and And yeah, we working on having Access to Windows machines using IP users. There's some progress there We have prototype that shows that what is possible to do there You can even manage be an administrator on the Windows domain controller and manage it all by being IP Not the other way around because that's already supported And that's a path to actually have IPA to IPA trust which I hope will be in the next three Something like 3D Fedora releases Hopefully and the final part here is improvements and Getting the defaults higher Better in the certificate management give more configuration options if you set up new integrated Certificate authority support IP addresses and certificates. That's I think open stack often requested and We have bunch of utilities now that we use the same infrastructure for all three big parts So IPA health check does something like 300 different checks on the machine to verify that something is Good of wrong and warning you about it It doesn't provide remediation yet Hopefully at some point we will look at what to do with it but the same for a specific for the directory server and for the doc tag for the certificate authority and There is a work in progress. It's not yet in Fedora On the doc tag side to implement the local acne service so that you can ask over acne protocol for IPA Back at Certificates regularly Same way as you do with lads and crept and others not yet in Fedora. I wonder why they haven't yet cut the release for that there's sizable unsupply integration a plug in a in a 40 minutes there will be a talk about it go to Dio 6 which is actually the next room where the whole security track will take on and Listen there actually there will be like like video live demo not video there and Finally Samba Disabled SMB one by default. We got a lot of bug reports But what we can do we have to remove SMB one simply because it's insecure and it should be gone The the only problem we do not have implementation of the POSIX Semantics on top of a newer protocol. There is a work on going Hopefully by end of this year. There will be something upstream that that we can do the work on going is happening in a collaboration between Samba upstream and Microsoft which is willing to put this into Specification as an official specification not extension of part of the specification and actually Putting the bill for client side implementation of that in Linux. So Steven French and Another developer works on this on the Linux kernel module they they actually work at Microsoft on that and Yeah, we are getting a lot of improvements, but many of them are not visible to users. They may be important to administrators. There is some improvement work on the using active directory Implementation and Samba with MIT Kerberos. It's not done yet. We're still blocked by some fundamental pieces not implemented And the lack of time Which I guess is My problem right now. I have two minutes. Thank you