 Hi everyone! This is the talk about advancements in free IPA and we basically will report on a year-long work that free IPA and SSSD teams did. Well, it's not the finished work, but it's something we want to share with the world. And let us start with the next slide. And who we are, both me, Alexander and Trevinia, free IPA core developers, we are doing different parts of IPA development. And we work at Red Hat for quite some time already and also contribute to Fedora and some other distributions in our work in spare time. Next slide. Free IPA itself is what we call identity management solution. It's kind of hard to explain to people. Some people are often see it as an active directory for Linux, but that's not really exact comparison. But there are some things we try to do. We try to provide centralized infra to manage post-exidentities across a bunch of Linux machines. And we combine a bunch of other open source components, like 389DS, LDAP server, MIT Kerberos, BIME DNS server, SSSD, Samba. And we also have a bunch of Python codes that constitute the free IPA management tools. And in fact, free IPA actually depends a lot on different components in the core OS. And they work together. And if something is not working right, this can be really used as a cannery to detect any kind of breakage in the packages. Just today we found that an update of OpenSL PKS 11 package, a minor version, broke some of our DNS second integration in Fedora. That's kind of life we live every day. And one of the bigger showcases of free IPA is that many open source projects do use it as their own identity system. So GNOME runs free IPA behind the scenes. And Fedora uses it behind the scenes. So Fedora accounts is built on top of free IPA. Next one. And if we look at the free IPA at the operating system level, then we kind of provide three parts of functionality. So first, we give access to the POSIX information about users and groups. And that's done mostly through SSSD. All clients enrolled in IPA environment, they run SSSD nowadays. And authorization of the users to access certain things is also done by SSSD, which parses so-called host-based access controls, HPEC rules, and decides whether to allow or not certain access. Then the authentication part is done differently depending on what you want to have. We certainly promote Kerberos authentication as the central one because it allows you to do what often called single sign-on, where you obtain a ticket grant and ticket initially, and then present it to obtain individual service tickets and access other services. So practically this is single sign-on to system services. And we will show some of these or consequences of using that in the Fedora environment. But also IPA provides centralized management of different other attributes. So one of them is public keys for SSH access, which also broker it through SSSD on the client. So SSH, I can see that the user has this public key that is presented, is known to the host, because SSSD delivers it from the centralized place where the user places it. And the same happens for the host keys and so on. But next slide. On the most important part of this presentation is how to integrate free IPA, or IPA as we call it shortly, to non-operating system services. This is typically what causes most of the issues on the main list when people try to integrate with various web services or some solutions that include, let's say, email and other tools that do not really use POSIX environment concepts. They use their own concepts of users, groups and access controls and so on, but they don't really rely on presence of those users or groups in their execution environment, like Linux system where they run. And for this one, the typical use is what we see in the next slide as the kind of user-free IPA as a backend. So glorified LDAP store where different applications try to look up data. And in many cases, it's just what they call LDAP drivers. So many web frameworks, they have access for the identity with what they call LDAP drivers. But some of the applications we work with, we made possible to, instead of going directly for LDAP server, use SSSD as this kind of source. And Fedora accounts, for example, uses epsilon identity provider for its internal web authentication thing. And epsilon has plug-in to talk to SSSD, ask SSSD about users, about their attributes and groups they belong to and then make decisions on it. Similar thing exists for another open-source IDP identity provider keyclaw project, which is based for Red Hat single sign-on product. And there are a few modules written over years by Jan Patura from our team for integrating this with Apache and Nginx servers. So modlookup identity or a port done by one of his students to Nginx, this Nginx HTTP lookup identity module. They talk to SSSD and they have quite sensible and flexible way of defining what you want to talk to. But that's identity, so who you're looking at. The authentication of it happens typically, again, through binding to LDAP and verifying that credentials passed by the users are correct. Or using Kerberos with the protocol that Microsoft extended when they needed to go with Kerberos and single sign-on in the web world. One colleague of us wrote this, the Apache module, which I think named incorrectly. It should be modauth.js API there. But then there is a PAM authentication in both Apache and Nginx with the modauth.nz PAM, which effectively allows you to use the SSSD PAM stack for all of these operations. The issues with most of these modules is that authentication through those literally only allows you to take it Kerberos or to username and password. And it's not interactive in terms of doing back and forth and asking additional questions and so on. And it's real big problem for years or maybe for decades already. So on the next slide, we have listed a few disadvantages. Aside from technical things, really one of the biggest problems is that application authors, so the people who write those applications, they really have problems working with LDAP and Kerberos. These are complex things and I cannot blame those people who misunderstand or don't work with them correctly because it's easy to misunderstand what Kerberos does. It's easy to fail on security of your solution if you try to do everything yourself. So some frameworks do allow extensibility, but some are not. And documentation about these protocols and specifically implementation isn't really great. It's quite complex to do, secure and save implementation use of it. So we tend to for reuse and of course typical integrations, struggles like with LDAP, there are plenty of solutions where you can only specify a single LDAP server that promotes stuff like using or front-ended LDAP servers with some sort of a solution that distributes the requests to the back-end servers. And that in itself is a security nightmare because that's the single place which kind of should know all the security credentials for communicating with the back-ends and so on. And in many cases also Java is used almost everywhere, but the Java frameworks, they have really outdated Kerberos support. Many features that were added as a part of free IPA involvement in Kerberos development, they simply have no support for those. You can switch Java applications to use system-provided MIT Kerberos or Aymdahl in Debian, but it's a pain. Some commercial solutions do not allow this kind of switching, they do not certify these solutions, and therefore those Java applications do not see some of the authentication methods that we support and use and they simply not working. Then when we were doing support for key log integration with SSSD, we found out that there's literally a huge problem with Java-based applications to talk over Unix domain circuits. You literally need to bring like 10 to 15 Java packages to just talk Unix domain circuits. So it's not maintainable, it's not really working well. And finally, with the microservices approach, you're not really in a situation where you should assume that system-wide domain enrollment details like access to your active directory environments and so on is really possible to achieve. So it's always a problem that practically means, okay, you have to make a decision between being secure, scalable, and just working and take two of those typically. So in past maybe 10 to 15 years, there was movement in the web services to have something that was expressed in so-called OAuth authorization framework. And OAuth 2.0 version provides a number of methods that actually rely on, some of them are insecure, but most of the things that used these days, they're pretty secure if you use them right away and there are good solutions to do that, much better than for integrating with held up on Kerberos for web. They rely on browser. So HTTP protocol has feature for redirecting a browser to talk to different systems and this redirection is not under control of attackers often and can be used for reliable and secure operations. That's also a disadvantage in a sense. So if we switch to next slide, of course, the OAuth authorization framework in itself is giving you a lot of advantages actually. And one of the biggest ones is that everything moves from application which becomes OAuth client in this context to identity provider, the place where this client talks to. So you have one place to focus instead of every single app. If your app is done with the OAuth or OIDC and all these things on top of OAuth authorization framework, then in your application you don't really care how actual authentication happened. All you care is that you get a grant to operate and you formulate your application operations in terms of these grant to access user information. And these are OAuth to subjects, not the system level users. So it gives a bit of good flexibility to map these POSIX users that free APA expresses to something else. They not necessarily should be POSIX users and have access to resources on the hosts themselves, but we can simulate that in a sense. So next slide. And the other part, this was a backend, free APA is a backend for all these external things. The other part is free APA as a consumer of external resources. So one of those is the consumption of external identities. And in free APA we have support for trust to Active Directory forest, which is quite widely used. But we also can consume authentication from those Active Directory trusts. And there's also another one. So a classical typical thing that you can have a radio server somewhere in your network that your other equipment network equipment talks to. And it allows you to authenticate some of these users to Wi-Fi access points in your environment or some other place, but it also can be used to authenticate to systems. We have integration with radios on the Kerberos level. So this is pre-authentication method that basically allows to kind of proxy the request to the external radio server wait for the Ansper and treat that Ansper as a kind of signal to Kerberos KDC that the ticket needs to be issued. So this support has some limitations. There's one radio server endpoint per each user. So only a scheme where you can pass PIN plus token or just a password really to radio server. And there's no conversation. You ask it, you go to Ansper, it's either yay or nay and that's all. You cannot ask again and go in cycles without breaking the user experience. So we have this and we started thinking, okay, maybe we can do something around this. And next slide. We look it into and okay, the OAuth2 basically says, OAuth2 client has no idea how user authenticated to identity provider. OAuth client simply asks, grant me, authorize me to access something to certain resources and how IDP authenticates the user is beyond this. So if we forget about this and let the IDP handle actual authentication, all we need is to be able to trigger these operations and really go and make sure that our client and our server side is able to talk to IDP. But the problem is the whole OAuth2 framework is built around a browser. And many IDPs, when you try to log in or do something, they enhance a user experience with running JavaScript, running using all the fancy things with the HTTP and HTML and Web assembly and so on. So this craziness there. So how we can avoid running the browser on our site, on our server side is all this thing is a real nightmare. So last year at the Nest, there was a talk about doing similar OAuth2 authentication where there was actually a project running on the server side, this temporary server where your browser would connect to to perform this operation. And it's a nightmare. You have to run this server on the server where you would log in into and you haven't logged in into yet. So it's a... Yeah, we went through this for several years trying to find out what we could do and we kind of figured out that this is... We already had this problem. Next slide, please. It's really the same problem that we have with the captive portals. Captive portals want to talk to you before you get Wi-Fi access. And if you want to log in on your system, like GNOME environment, you want to log in into GDM. In the GNOME session, you have to run this before logging happens. You have to run untrusted code in the browser somewhere. We still haven't solved this problem for the browsers. How we can run this untrusted code prior to logging and so on. So we decided, okay, maybe we shouldn't solve this problem. We will spend years solving it because the first time we were talking about this in 2015 with the browser team and it was admitted that this is the issue. Desktop guys were saying they are looking into this problem and they still look at it, but it's tough to solve. So we started looking into different parts. Next slide. And before I go into the technical details, Trevino will log you over SSH. I think this is going to be fun. Yes, in this demo, I'm going to show you something cool. But let me switch to a different build desktop. So what you see in the screen right now is just a Firefox browser on the left and regular console on the right where, as you can see, KPA is already installed. And it's already deployed. And this is Fedora 36, as you can see. Okay, so in addition to that, I have some persistent users already in the system. For instance, KPA user 3. And this is a normal user where password is set to true. That means that this user has a password. And then I have two more user. And then where the main difference with the previous one is that the password is set to false. So it means that this user has no password. In addition to that, for this particular user, we have a pseudo-rule pre-configured. I can show you that by running the pseudo-rule. We have the Fed's rule. As you can see here, this rule is a pseudo-rule and allows the GitHub user 1 basically to do all. And then, yes, I think we have another one. This is called Azure User 2. And this is pretty much basically the same as the previous one where the password is set to false. Okay, so keep in mind that this pre-configuration, I'm going to log into the system. First of all, using the IPA user 3. So then I run SSH IPA user 3. And then as usual, everything is normal here. It's asking for the password. Okay, so I managed to log in and get a ticket. All right, so as I said, until now everything is normal. But then for the next two users, I can figure something, SSH IPA, and the login is going to be a little bit different. And that's because I added something that allows me to log in with credentials from an external IDP. So let's try. This is one, pretty much the same command. And there we go, surprised. Yeah, now I'm not asked for the password and then I'm asked to copy this URL and go to the browser and get that URL and then copy the PIN. I'm trying to do that in the URL. Okay, so this is doing the authorization for the user. Once this is done, I can first enter here and I managed to log into the system. But this time with a learning password, as you can see. And not only that, if you recall, I mentioned that this user has a pseudo rule, right? So let's try. We can test it with pseudo.l. And as you can see here, this user is granted, has granted privilege for pseudo for everything, right? And actually, when I use pseudo, for instance, to create a folder, yeah, it's working and it's not asking, it's not asking for the password again. I don't get again the authentication prompt to validate the login again, all right? And I can show you why. If I go, if I exit from here. This is because basically we enable the BAM SSS GSS support. We can easily check this by running off select command. And then as you can see here, we can extract enable feature, which is with GSS API. And this is what is allowing me to go through the authorization of the pseudo without BAM prompt again for the password. And finally, let's try it with different external IDP, such as Microsoft, so that I can use the other user we have. So again, the same thing, but now I'm redirected to Microsoft URL. This is asking me for the code because it's here. So all I need to do is copy and paste it. Okay, so the process is completed. Then I can go back to the terminal and first enter here. And the SSH was completed as you can see. And we got the ticket. So with this, now we can forget about the storing passwords. This is a fully password less mechanism as we can associate tokens for non-administrator users. And then I think Alexander can give you more details now about how this is working. Alexander, do you want me to show something else? Yeah, we can get back to the slides, I guess. Yeah, so next slide. So basically what you have seen is an implementation of the OAuth 2 device authorization ground floor. This is effectively a way to support dump terminals like our SSH, which cannot talk HTML and cannot be a browser. This is being done by basically offloading the browser task to a different device. So let's say it is your phone or your other computer or maybe the same client system that you're used to run SSH. But certainly you have this browser on it, not on the system where you try to connect to. And in the case of FreeAPA, the actual implementation is a bit more complex than this one, but remains to be similar to this. So next slide you can see an actual way of doing it. And this box in the middle is actually not the system where you run, where you initiate your SSH communication. It's instead FreeAPA server. The server runs Kerberos KDC performs something similar to radius authentication by calling out a helper. We call it IPA or TPD because it handles one-time password and radios in the same system. And now it handles also OAuth authentication. And it reuses another helper that is provided by SSSD, which is OIDC child there. And that child is what talks to the external IDP. So somewhere in the cloud or on other system, there is an identity provider that has support for device authorization grant and so-called device code portal. And the message that you see on the client side by the Kerberos actually is by the Kerberos client is the message provided to you by the identity providers to point to the device code portal. And then your browser is going there to talk there. David is saying that he needs a bigger screen. Don't worry. These diagrams stay in FreeAPA documentation and this presentation has links to it. We will provide it. So you will see them. Yeah. So it's all there in the design docs and also in the workshop. So a major part of it will go through the workshop in the implementation. So the actual flow that happens there, if we get to the next slide, is that the SSSD does implement a special pre-authentication module called IDP. So this client pre-authentication module on the Kerberos side tells Kerberos KDC, oh, I support this auth method. Consider it. And KDC may consider it by checking the details about the user. If user has a mark that it can use IDP and certain IDP is associated with it, then KDC will respond with the message and ask you to authenticate that specific URL and press Enter afterwards. So there is some communication, as I said with the IPR TPD helper. We get communication several times, ask the IDP to verify these credentials, wait for user to authorize the, or grant us to use resources and on successful authorization response, we issue Kerberos Ticket to the user. And user runs browser elsewhere. We still have to have a browser there. That's unavoidable, but at least it's running on a system where user most likely has the browser already. And this Kerberos Ticket is a normal ticket on the next slide where it could be used for anything. So the only thing that we do additional there is we associate authentication indicator with this ticket based on the method that was used for pre-authentication in Kerberos. So if that was radius method, we take radius indicator. If that was smart card authentication, we use PKI init as indicator. And for external IDP authentication, there's new indicator called IDP. So basically what was shown by Trivenia is Palmer SSS GSS using the Kerberos Ticket. And his configuration was actually insisting that the Kerberos Ticket must contain IDP indicator. And because that user, GitHub user one had IDP indicator in the Kerberos Ticket, it was allowed to access sudo. You can use that with model of the GSS API as well because this Apache module also supports indicators in the code. So that's possible type of integration we switch from externally authenticated and authorized user to a Kerberos Ticket and then Kerberos Ticket can be used for normal system level and maybe web applications. And indeed, with free IP, we don't have support for all through logging into the web UI that we have for IP itself. So you have to obtain the ticket first and then logging into the web UI. Next slide. So this whole stuff was tested against a number of public identity providers. We tested against Kiko, found some bugs in the device authorization, grant flow, implementation, fix it. Then we tested Google, GitHub, Azure, Octa, all day work. What doesn't work and actually the IDPs that do not implement device authorization grant, which is fun, but that's the reality. So GitLab is using some Ruby-based library for all the implementation that doesn't have support for device authorization grant. And Ypsilon, which is what Fedora accounts is using, also have no support for device authorization grant. I hope we will get this fixed going forward, but this is the state of affairs nowadays. Next slide. I'm trying to speed up a bit. And this is the second demo. Okay. Yeah, so let's jump into the next demo. So let me switch to another real-world desktop. So here what you see this time is three browsers, is that? The one on the left, it corresponds to Next Cloud, and it represents a customer framework application. Okay. Then on the right side here, we have a key clock to be used as our identity provider this time. And then finally, the third one is simply the free API web UI. And then on the background, I simply have a terminal log into the free API server. So what I'm going to do now is to create a new user in our main provider, which is key clock. Here, I'm going to click on add user. I need to log in. Okay, so I will call it ITELS2, for instance. Okay. This is creating a user. And then I can see how it's created here. I can even reset the password. All right. The password has been created. And as you can see here, suddenly the user should appear in free API, magically. Yeah. There we have it. ITELS2 is there. All right. This is because key clock is somehow connected to free API. And now, actually, I can use this user that I just created to log into the customer application, to log into NextCloud using key clock. This is going through the OpenIDC framework. Okay. All right. As you can see, NextCloud is using key clock as provider of identities. So then it's redirected to key clock to do the authentication. There, here, I can write ITELS2 with the password I created. And there we go. I'm logging into the customer application. So it was able to authenticate using key clock as provider, as a user that is at the same time, populated in free API. And then a user that has access to system that are managed by free API. Okay. So in addition to this, we can configure OAuth2 for this particular user. We can go to the IPA console. And then we can write this command to enable the user for key clock as a standard IDP. So the user is ITELS2 this time. The notification type is going to be IDP. All right. And the standard IDP is key clock. Okay. So with this command, we're indicating to the ITELS2. I think you need to modify the external user identity because you put it ITEST instead of ITEST2. IDP user ID here, yeah. So you need to do user mode again. Oh, yeah. You're right. Yes. I'm missing this one. Thank you. Otherwise, it wouldn't match the ID of this user on the key clock side. Yep. Exactly. So yes, as you can see, we modified the user. The user that we created in key clock. And then we are adding the authentication type IDP. And we are using the standard IDP configuration key clock. Okay. So now, if we try to log in into the Android system, by using these two users, it should get a similar, yeah. Okay. There we go. And then we need to go to key clock and provide the device authorization code. Yeah. This is just for the code, which is here. Okay. So this is ready. Then we can go back up to center. And as you can see, I managed to log in into. So I can't show you the standard IDP indicator that's under what's mentioned in before. I can basically do this command. You need to indicate the operation. Well, Trini is doing is actually using Kerberos utility directly instead of going over SSH. So the ticket that he got there is the ticket from, is the Kerberos ticket directly. And you can see, if you do Klist dash capital C, that this C cache contains configuration detail that shows the pre-authentication method that was used for actual authentication. And 152 is the IDP method that we created. It's registered upstream and in the system. So MIT Kerberos knows about it. And SSSD package provides it. Okay. We have three minutes left to the formal time and a few more slides here. But basically what you saw is a generic backend dream, I call. There is this set of RFCs and an effort to define how cross-domain identity management should be done. So it's kind of a protocol over REST API over HTTPS that allows systems to talk to each other using the definitions of what users are, what groups are. It's called SCIM, SCIM v2 for sure. And we implemented proof of concept tooling that sits between the identity provider and free IPA or actually LDAP server or Active Directory. The main on top of SSSD and performs this SCIM v2 translation. And next slide. And this is how it looks. The project was published today. It's called IPA Tura. The Tura is ice shizzle in Finnish. A type of ice shizzle. A tool for breaking ice for making access to the water in winter. So effectively making a bridge between water and air. Between those two systems, the web and classical ones. The current scope is fairly small but powerful. So we support free IPA LDAP and Active Directory for reading SSSD and writing through IPA, API in Python. There is rudimentary password authentication support with PAM modules. So you can have passwords in IPA stored or in LDAP and Active Directory and SSSD. If it authenticates you, then let's say key clock will accept this as authentication. But in this example, in the demo we used password on the key clock side and kept IPA clean of those passwords. So next slide. And this is published in free IPA slash IPA Tura on GitHub. And we really want to see people testing this against other stuff. And for connecting this to key clock, we have a key clock storage plugin, user storage plugin by Justin Stevenson. So it's working well but it doesn't have much of functionality yet. So we really have the proof of concept point. It works but obviously needs more work to be production ready. And I think this is all we have. There are some future plans. I'm going to get through them quickly so that we can have some Q&A. So next slide. So yes, the idea is to actually have a transparent bridge completely between IPA and OAuth IDPs so that we can have some stuff like IP enrolled apps like cockpit on each server to accept OAuth authentication of IPA users somewhere and login these users into cockpit or into web UI that free IPA provides. And then securely transit from OAuth grant to Kerberos on behalf of the user. You can do this in security nowadays but we don't want to compromise. We want to have something that really ensures that there is no token stealing and so on for the particular thing. That will take time to get through. Then we kind of have on the next slide a way to work with the FIDA 2 tokens, the web open tokens. You can already do this in Fedora 36 with if you're IDP like Kiklog or actually Google and GitHub and Azure and Octa. They all support web open now. So you can do tokens like the Ubiqui tokens with free URL IDP. We don't need to do anything on our side just that your browser have to support this. And that's all. But we plan to have FIDA 2 tokens natively. First locally with FIDA 2 to replace PalmUTF module and we work on this with some community members. This is not yet at the way it could be tested or something. Hopefully SSSD folks will get there soon. And then once it's ready we integrate this with Kerberos for free APA so that all these things can be used centralized way in the IPA environment. Then next step is to integrate with normal login so you can get there with all this stuff and finally get totally password less except maybe admin account on IPA deployments. And finally with the Scheme V2 support the IPA tour needs a test against other Scheme 2 providers support more IDPs at more authentication methods because we really need to do Kerberos and some other stuff as well. It's a lot of work but I hope we generate enough interest to see community coming and helping us. So with this, thank you for listening and it's quite late here. Let me look at the questions. There are two questions both from David. How does this solution scale over large systems? It is basically a single process per single request. It's launched and forgotten. So that process runs until it finishes communication. The IDPs side of the story is fairly scalable I guess like GitHub or Azure handle millions of this authentication and so on. And on IPA side it's fairly lightweight so basically lip curl with JSON processing and so on. So it shouldn't be quite high. For the other question that David asked whether this is prioritized on Y side and on the CPE I have no idea yet. We need to talk once we have something to use with. It's a chicken and egg problem for the integration. We will see. Any more questions? Or will you finish? If you have questions just feel free to ask them on free AP users mailing list. I guess this will be a good way to have a discussion about possible scenarios and use cases people interested in and thank you for listening. Thank you. And enjoy the rush of nest.