 Hello everyone. My name is Andreas Schneider. I'm a free and open source software developer working at Red Hat. And I work mainly on SEMBA, but I also have some spare time projects. I'm working on the SS8 library and some testing tools. You should know them if not then look them up and also do some hacking on Android stuff. But we talk about SEMBA today. So what is this talk about? I will give answers to the following questions. Why took it so long to get SEMBA AD in Fedora? So we look at the history. We look at what's still missing and how you can get your hands dirty. So RHEL currently only packages the SEMBA file server. And Fedora also packaged SEMBA FS only for a very long time. What is the reason for this or what was the reason for this? So enterprise distributions are required to ship with MIT Kerberos. And this is a special requirement if you want to do businesses with governments, especially with the US government. They have a policy that only MIT Kerberos can be used. So enterprise distributions like Red Hat or SUSE, they are required to use MIT Kerberos or to ship with MIT Kerberos. And everything in this distribution needs to be integrated with that Kerberos version. So distributions can't mix Heimdall and MIT Kerberos because the on-disk format, for example, differs. And also there are symbol clashes if you load both libraries into one application. So SEMBA AD with MIT Kerberos would allow enterprise distributions to ship it. So to fix that we needed to port SEMBA AD to MIT Kerberos and that was the requirement to get it at least into Fedora. So we needed to port it. So let's take a look at the history. So it started in 2013 and was four years of work. And in 2013 we thought about how do we get started? So if we port SEMBA AD to MIT Kerberos, the question was how do we run our tests? SEMBA has tests with me, we have a really huge test suite. The question was how do we start with that? How do we integrate an external process, the MIT KDC, into our SEMBA test suite? So in February 2013 I started to look into Socketrapper. Socketrapper already existed in SEMBA and was written by Jelma. And it couldn't be used with MIT Kerberos. I will explain what this more or less is in a minute. So we needed to bring Socketrapper and we had also different wrappers on a new level to be able to do all this testing. So the CREP project was born in 2013. So CREP is a set of tools to create a fully isolated network and system environment to test client and server components with synthetic account information, host name resolution and privilege separation. And this is done by several libraries, one of them is Socketrapper which can be preloaded into any executable. And with that you can create a network, set up servers and start that they can intact. You can think of CREP like it is the metrics for applications. So I gave already some talks also here at FOSTEM. So you can go back in the archive and you can find out what it is. So CREP in the meantime powers the complete SEMBA testing environment and it allows us to do integration tests with complex active directory environments on a single box on a developer machine without needing any privileges. And it's very easy to use. So for example here is a test run of SEMBA with MIT Kerberos. As you can see we have a pretty big test suite. So a self test test 2030 test suite in the meantime is much more and well it takes more than three hours to run. So in SEMBA as I said before we are test driven. So how do we get there? So also in February 2014 we started to run SEMBA or we started to run SEMBA with CREP. So one year later we started development in 2013, one year later we were able to use it in SEMBA. So it was one year of development to bring them to the next level and then we started to use them. And when we integrated them it still took a few months till we found and fixed all the bugs which were still in there. Okay, that was testing. Now let's look at SEMBA and MIT Kerberos. So at the same time in February 2014 Günter Deschner and I worked on the possibility to start the KDC, the MIT KDC and use it. So we created a SEMBA-MIT module to be able to access the SEMBA database with MIT Kerberos. For that especially Günter created a shim layer to convert between SEMBA and Kerberos structures called STB. So it is a simple abstraction of SEMBA KDC routines and provides a conversion between either Heimdall HDB and MIT KDP which are the KDC database plug-ins to extend the KDC functionality. So we created a MIT KDP SEMBA module and we had it working in May 2014 and were able to access the SEMBA DDP from the DC. So we were able to start MIT Kerberos KDC and we were able to start find and fixing bugs. But this was all manually driven during that time. We integrated that and well we started to work on changes especially in the chance act as API module to work with MIT Kerberos. We needed to start to port all the Kerberos code in SEMBA which was only working with Heimdall. We added much more regression tests for example for the KDP SEMBA protocols not only writing tests only for MIT Kerberos but we extended the tests and run them also first against Heimdall and then MIT Kerberos or vice versa to extend the whole test suite and then well we started to discover bugs in MIT Kerberos and reported them upstream then we needed to wait till fixes are available so that stalled us also often for some time. In August then we discovered a new problem. So the service discovery in SEMBA especially in the tests were not working MIT Kerberos library Libcap 5 normally looks up the KDC via SRV to server DNS records and that didn't work. Well because there is no in this environment there is no DNS server available or SEMBA provides one but we couldn't access that because of limitations and then we wrote and extended the C-Rap project with a new wrapper called Resolve wrapper and that was a way to fake a DNS server and tell the Kerberos library how to actually discover the KDC. So in April 2015 MIT Kerberos patch set crew up to 140 patches already and we started to push them upstream to reduce our daily rebase fun we had because people did not look at our branch and things were breaking often on a daily basis and we needed to rebase every day to keep up with the development and during that time we still had 69 test tubes failing so we thought it would be done soon. So in August we started to implement DC provisioning in SEMBA tool itself so that the user can set it up and in our tests we do it differently. We also noticed that we need to implement the K-Admin D service completely on ourselves because MIT didn't have support for access control lists so we needed to implement ourselves that we can use access control list from SEMBA AD. Then I ported for example the backup key service to GNU TLS and that was the next thing when you always when you port something for example here to GNU TLS you recognize well there are features missing in GNU TLS I'm not able to use it right away so I normally go to the main developer, talk with him and start implementing the missing pieces. So then from around February to April 2016 we added much more tests in the time to MIT Kerberos but MIT Kerberos then missed important hooks for testing we needed pre-send and post-receive functions in the KDC to actually do the testing based on Kerberos packets sending and receiving them. So the first step was again we need to implement them and MIT Kerberos need to wait till they are upstream package them so that we could use them for development. In April the development got completely to a halt because of the bad luck bug we had in SEMBA maybe you heard of that infamous bug. So we didn't do anything in this regard till around August and in August we recognized that credential passing in libSMB doesn't work as it should be and it was working fine with Heimdall and we didn't really understand why because this was really totally broken so we needed to rewrite it to pass down the realms especially through windbind to have support for trust. So that took quite some time and then in the end of 2016 MIT Kerberos 1.15 was released and we thought yeah now we have a release with where the SEMBA AD stuff is working and we can base on that but then we found out well this version has a major problem and then in the KDB API they removed the function which allowed memory management for KDB modules so we were leaking memory and yeah then we needed to work together with the MIT Kerberos developers to find a way to bring that API so that we do not break stuff for a lot of people who implement KDB modules. So generally to March 2017 the last test there was only one test failing and this was about external trusts and we discovered one issue in the MIT KDB module so that was our fault we needed to get that right and the second issue was a bug in Chansec redirecting requests to the correct DC. It took a while to find that out that we always talk to the wrong KDC but we were able to fix that, that test was passing and then in April 2007 and finally the code hit master we did not do a daily rebases you had everything upstream and it was working quite well. So last September SEMBA 4.7 got released and it was the first release with MIT Kerberos support for SEMBA Active Directory. Let's look at SEMBA AD in Fedora. So in November 2007 finally Fedora 27 got released with SEMBA AD. So it took a while to package it correctly so thanks to all the testers the first thing was we need to package it, try that the installation is working that we have system deintegration, the provisioning is working correctly. We found bugs around SEMBA tool with MIT Kerberos and also people are using either the integrated DNS server or bind so we needed to get things right especially in Fedora and together with the bind version in there and I was lucky that I had several people who were willing to test this in the better releases of Fedora 27 and reporting bugs and we could clean up the package. So what works at the moment? So all the important stuff is working quite well so single DC works very good. Forest trusts are working currently not with free IPA there is one bug we need to fix and also external trusts. The problem is that this stuff is currently not really tested very well in SEMBA self-test and we need to get there. So I hope that we can implement more tests related to trusts in the near future. The groundwork already was the code for that went to master lately so we fixed several issues in this area. So now the question is how to install it, you can just call dnf install somebody see then normally all the components required to run it are automatically resolved and installed then you can install it using SEMBA tool domain provision it will be an interactive session where it will ask you the questions for the realm for example and then you can call systemd to start the service. SEMBA tool is not, I would say it's not very user friendly if there are errors you get normally a Python tracebacks and we need to work on them to give useful information to people who don't understand Python tracebacks but give useful information to understand what the problem is. And if you are hitting something like that when you are trying it out please open a bug and we need to know about that and describe what you did and how to reproduce it so we can fix that or if you know Python please provide patches. So SEMBA provides a module to use binddns which is called binddlset I would suggest to avoid that module because it took a while to get that correctly in Fedora working and there are still some issues that module is really a horrible hack and it directly works on the database it needs hard links that you do that and it means bindd has full write access to the whole database of SEMBA we need to look into the bindd and LDAP project which is used by free APA and they just talk to LDAP and maybe we can extend that or change the code that it also works with SEMBA that would be the correct way to do that and there is already a project maybe we can work together with SEM to implement that correctly SEMBA provides an internal DNS server if possible use that and do not use bindd if you care about security unless you know what you are doing so look into that stuff administration so currently the SEMBA DC can be administrated using the windows graphical tools you can connect remotely to the SEMBA DC and use them to administrate it the other thing is SEMBA tool on the command line we saw several contributions lately to improve the usability but we are not serious not serious yet but for example if you are working on if you know Python and if you use SEMBA tool we are happy to see patches in this area to improve the usability of SEMBA tool and Alexander lately hacked a little bit and wrote a SEMBA module for cockpit it's very basic at the moment it's just a proof of concept it works very well so you just click here on run initial SEMBA AD setup it will call SEMBA tool domain provision will ask you for the thing it needs like the realm password and stuff like that and then it will magically in the background provision this thing start it and then you will have an overview about the configuration what networks port it is using and some details about SEMBA AD so if someone is interested to help with that I think Alexander is open for patches okay what's missing yeah as I already said usability improvements for the SEMBA tool then PK in its support should work but it's we don't have tests for it and we don't have any documentation how to set it up so I don't have time for that right now so if someone else needs that and wants to look into that and can write a tutorial how to set it up I'm happy based on that I can implement the test then audit locking got added I think to 4.7 4.8 4.8 audit locking we just tried to build and then item 3 didn't work there yes so this was added it works well with Heimdall I started to implement that for MIT Kerberos but there are some issues because Heimdall is integrated or Heimdall is running in the SEMBA process so it's the same process and locking works just fine but as MIT Kerberos the KDC is running as a different process we have two processes where do the audit locking and the interaction doesn't work but yeah that will be probably in 4.9 then Kerberos impersonation support is not working as for you to self and as for you to proxy Kerberos fast port is missing but we need changes in MIT Kerberos for that SEMBA AD with MIT Kerberos does not work as a read-only domain controller that would mean that we can implement that or that we can use that it would mean that MIT Kerberos needs to provide a libkdc and this means we need to specify an API and then MIT Kerberos needs to probably restructure a lot of code to make that possible smart card support is missing again it should probably work with the T4 MIT Mk8 modules yeah Alexander said that it should work with the default MIT stuff but we don't did any tests in that area so we can't save 100 or we cannot be 100% sure and we normally say at some of the bug by bug feature by feature and the other module we have is untested code is broken code so it's untested it's probably broken yeah and that's it we have still 5 minutes left for questions if you have any questions why SEMBA went with Heimdall Kerberos initially so the question was why SEMBA went with Heimdall Kerberos initially so when the development started for SEMBA Active Directory SEMBA developers evaluated what Kerberos implementations out there are available and which has the most active community during that time Heimdall Kerberos was very active and the developer was willing to implement features which were missing and required by SEMBA so when our feature was missing we could describe what we need and he started to implement that stuff during that time MIT Kerberos was pretty dead but that changed in the meantime so now Heimdall Kerberos is pretty dead I think the reason is probably the main developer moved to Apple and disappeared the community around Heimdall is recovering yeah so there is still a community around Heimdall and they do releases but there is no real active development going on and MIT new releases yes there are new releases but feature wise and MIT Kerberos changed in the meantime I guess also one reason is free IPA and Red Hat investing a lot into it currently it's very active they work on new stuff and it's easy to get features in there so when I implemented new stuff I just did pull request and then I get refuse normally and was into work days and yeah so that choice is not related to export restriction of cryptography technology from the United States so the question was if it was the restriction of export control at the time I don't think so no same story as with LDAP at the time open LDAP guys were unwilling to accept any changes from us so we did all LDAPs yes Volker just commented that it was the same with LDAP during that time open LDAP was not willing to do anything the SEMBA team needed to actually use open LDAP as the back end so the SEMBA team decided to implement LDAP on their own that's how we ended up with LDB based on TDB and we have much better collaboration tools nowadays so it's much situation is improved on that front Git was like this yeah and in the meantime as Alexander said we have much better tools for collaboration with these projects whatever we need so Git is now available it's easy to contribute question you mentioned that PK Innet and Smart Character were not tested or completely supported is that just because of MRT or was it not working already with a handle so the question was if PK Innet and Smart Character support is not what's the reason that I say it's not supported the thing is it should work because it works very well with MIT Kerberos but the question is does it work with SEMBA yes it should be but as long as I don't have tests which prove it I cannot state that some as working well PK Innet is working or Smart Character support so if you say I want to try it and you know how to set it up probably I would say yes it will work I'm happy to get how to, how to do it then I will happily integrate it also in our testing infrastructure but I didn't have the time to dive into that set it up and test everything you said at the beginning that MIT Kerberos was required by enterprise and government in the US would you have ever read that? so I should go into details for why MIT Kerberos is required by governments I don't know why but as far as I know it's the US government since a long time has a rule to only allow MIT Kerberos in their environment no other MIT implementation so this was why all the enterprises because they wanted to do business with the US government they went with MIT Kerberos but it's limited with the US government yes I think in the meantime there are more governments who based on that probably went with MIT Kerberos it's also practical distribution wise it's a practical to maintain two Kerberos implementations because they do differ on the ABI level whatever you link with I would guarantee that if you choose alternative implementation front time it will even work it will most likely crash they do work and be compatible over network connections that's the whole idea of the protocol but having them in the same time Fedora has hindal packages Fedora 27 that's the first release where it came out and we have been in an interesting situation with these kind of packages because we simply cannot at distribution level test and build those packages against different implementations there will be some change with the sort of modularity game that Fedora tried to run where you have different modules with different build parameters and there you might try to get it but on the generative strut it's a very tough problem to solve yeah, so as Alec I think we just stopped this last minute yeah, to repeat so you need to decide on one library because a lot of stuff is depending on it you cannot mix two MIT Kerberos libraries on the same distributions because of ICER symbol clashes if you have installed two and or both are loaded into a process which symbol do I pick as the on-disk caches differ so if one applications write something on disk and the other one wants to use it and uses different MIT Kerberos implementation then it cannot read the on-disk format they are the same on the protocol level but not on the implementation details on the system okay, time is over