 Veselim v tem vseh vseh vseh začetno vseh z vseh, tako, njih ne je veliko zvukov, neč spetno. Značajte z vseh z vseh, nekaj kaj se vseh zamo, če vseh vseh vseh? Tako, vseh vseh vseh vseh zamo vseh vseh. Tako, tako, ne musim vse z baho izgledati, da sem pridomil, pa se počutim, krišno, sestem, sajstroj, menešment vsega, da sem podal, nekaj pa nekaj ne vsega, zelo vsega, ugradaj, sestem infrastruksu in tukaj, There is also some basic configuration management and overall provision systems and stuff like this. Spacework is open-source project since 2008. It has history far earlier than 2008, but in 2008 it became open-source project. Spacewalk je tudi izgleda za redet sattelaj 5 in susemenazir vseh produktov. Spacewalk je tudi izgleda za vseh produktov. Zato sem bilo pošličen. So there are some things you can do with spacewalk usually every day. Like package management, data management, some basic configuration management, like provision system using kickstarts. And there are other features like automatic integration with automatic back reporting and open-scape and proxying stuff. There are many possibilities to achieve what you want. Why would you want to spacewalk now when there are so many other options for provisioning and content management and so on? Spacewalk is still actively developed. There are not so many features nowadays, but we are focused on bug fixing and stuff like that. In fact, spacewalk products derived from spacewalk like Satellite 5 and Susan and Azure are used in production, so using spacewalk you have some guarantee it will work in certain quality. Most of the times. We have also an active community, but of course not that large alike some projects which are more alive now. There is a list of some spacewalk components. There is a web interface which runs on Apache and Tomcat. There is also XML RPC API, which is quite old, but it works. Spacewalk uses a backend, PostgreSQL or Oracle database. For provisioning, Spacewalk uses Kobler, which is, of course, one of the projects that is not heavily developed now, but yeah. Daskomatik is our asynchronous engine for doing basically things which cannot be done in real time. You don't want to wait for. There is also a dispatcher, which is the thing used for a synchron. I will go back to the server client architecture here mentioned. If you schedule on Spacewalk some action. For example, you want to install something and you schedule that action and the client must pick up this action. And this can be done several times. You can manually run a region check command on the client. And the client will ask Spacewalk to if it has any scheduled action, if yes, then it will pick it up and run them. And this is not very comfortable. So there is a demon that runs periodically that check. This is a very old code, but it's not very comfortable as well because it runs only once per hour or four hours or something. This is not very good interval as well. So there is also dispatcher, which is basically demon on Spacewalk, which chats with clients using Jebr protocol. And basically says them, hey, you have some action scheduled for you, so please run a region check. So client comes, reads this message and runs a region check and so on. So here are some resources for Spacewalk, like Spacewalk wiki, which recently moved to the GitHub and Spacewalk bugzilla for bug reporting. I think this is the most important things. And now we should go to the workshop. And, yeah, I have here two flash drives with some QCOW images with Spacewalk client and Spacewalk server, even with some content. If you want to follow this workshop, you can compile these images and set up it in your LibVirt or something. And if you don't want to follow, so you can at least go to the IP address on the bottom of the slide. Here is some live Spacewalk instance running somewhere in the cloud. So you can click through the web UI or do something with it and hopefully not break it. So if you want to copy those images, there are three images. And there is a Spacewalk server and Spacewalk server with content. And the content has like 11 gigabytes. I have USB flash drive, so it could be quite fast. And Spacewalk client, these are real six images. So please, anybody want to try? Okay, so I have here two virtual machines, one for the server and second for the client. I will kick them on. If you will have some troubles with setting up these images, please ask. So I will hopefully check. And root password is Defcon. So I have two machines, one for the server and second machine for the client here. I will go also to the web UI. I will have to wait a bit. It's not running yet. Come on. It takes ages when you wait. It's the usual thing. Ah, finally. So on this resolution, it doesn't look very good. I don't, but it's shame, but never mind. So this is the main overview page you have here. Overview of your systems and which are, for which systems are, are updates available and which are security updates, bug fix updates, and so on, so on. And some other things. Currently, I don't have any systems registered to the Spacewalk, so there is nothing. I have, I'm using that QCOW image with content. So I have here CentOS6 base repository synced to this Spacewalk. It's like 6,000 packages. So, yeah. Maybe you don't want what the channel is. Channel is basically repository now. It's called channel for historic reasons. This concept comes from the times where there even wasn't yummy repositories, and yeah. It's very old concept, but basically you can understand it as a repository because finally there are also repo data generated for each channel, so you can consume that channel as a repository. So now I will go to the client and will try to register this client to that Spacewalk server. Yeah, I can do it multiple ways, but I will do it the way, like I will create the activation key in this formula. I will call it CentOS key. And the key itself will be CentOS key. There will be unlimited usage, and it will subscribe you to the CentOS6 channel I previously created and synced. Okay, so I will create that activation key. Now we have activation key and we can register our client. I will do one little thing here. I will set IP address and hostname of the second virtual machine to make sure spacewalk.example.com. And now I'm going to... If you can register your system to the Spacewalk using standard old tools, like RHN register, which have all the RHN name in its command name, because the same tool was used for registering to the RHN itself years ago, or even now, it's possible yet, but it will not be. So I will specify the server URL and we want to go to the XMLRPC handler, which is used for that, and specify, we will specify activation key. What was that key? One CentOS key. I'm doing something wrong. Is it XMLRPC or RCP? Yeah, thank you. Just guessing. Yeah, it is it. Client is now also CentOS. Yeah, the client is also CentOS. Now, if we look to the API, we see we have the client registered and subscribed to the base channel. There are no updates for it, because that client... Because we have in Spacewalk only the base repository, which doesn't have any packages. Maybe the client has also packages, which are newer, then we have available from CentOS channel. Here we can list packages we have installed. We can install new packages. We can verify packages using RPM minus V or something. We can subscribe the client to another channel. So I will just decide. Let's try to install Firefox on this client. I will make sure I have disabled all external repositories on that client. So I will run some enable, so I will disable them. And this is the Spacewalk repository, which was initially used for installing the R&D registrar, and these packages needed for registering to the Spacewalk instance. So I will disable it as well. And now I can see I have only enabled repository ID CentOS 6, which is actually the channel. The channel on the Spacewalk server, which are accessed using the YAM plug-in, which contacts Spacewalk and fetches its report data from its channel. Basically the channel works as a regular repository. Now I will say let's install Firefox pretty fast, because we are downloading in from second local virtual machine. So now we should have it. That was the first approach. We are installing directly from the channel, which we are handling as a repository. Is it already recorded? I think it should be. I will search here, and yes. Install today at one. Yeah, it's now. So now I decide to remove this package. We don't like Firefox anymore, so we delete it. There are some options to schedule it. I think to action chain, which is some other abstraction to work with. So we are simply scheduling for deletion. So we have here scheduled action, but when the system will pick it up, we don't know. The system, when it's a demon runs the RH in check, it can be anytime, but there is no guarantee it will be now. If we run a RH in check manually on the client, so it will be picked up, and the Firefox will be deleted, and the action will disappear. Yes, but as I said before, it's not that usable. So now we try to enable OSA dispatcher on this client. There is OSA dispatcher on the server, and OSAD demon on the client. So we try to enable OSAD demon on the client. So we will try to install it. Yes, and we can install it because OSAD package is available from Spacewalk client directory repository, and we are consuming only the base channel of the CentOS 6. So let's create another channel and get that spacewalk repository to that, to here. So let's create CentOS 6 spacewalk client channel. Now channel, and then we will create repository, and we set up a parent channel for this channel because at the time we can have only one base channel. There is a concept of base channels and child channels. One server can have, at one time, have one base channel and multiple child channels. So if we want to consume the base channel of the CentOS, and also these spacewalk client packages, so we have to create another child channel. So we will create it by setting up a parent channel in this formula. Yes, so we will go here and we create a new channel. Now we need to get the packages to this channel somehow. We will do it by assigning the repository, like we will find yumspacewalkproject.org and we will go to the spacewalk26 client and we will find that repository. This is it. So we will again name it somehow in the URL to the repository. Yes, it was created now and it was assigned to that channel for synking. And now we will run the sync of this repository to that channel to get these packages. We can do it from the dvpy like I'm doing or we can do it manually from command line. So now it should be running and downloading. Is it still the case that we have the possibility to assign multiple repository in single channel? Yes, it's completely possible but it depends what you want to achieve. Now we see we have already sent to a spacewalk client synked with 25 packages but we don't have the system subscribed to the channel. So we will fix that. Now by going to the system software, software channels and enabling this child channel which contains these packages. Child channel, yeah. If we now run again the repository install it doesn't work. Yeah, maybe we will have to wait a minute or so to generate repo data for that because it's done synchronously. Do that synking. Yeah, maybe I will try. Is now we have access to that channel so it should work. But it doesn't work because it requires JeberPie package which is available from EPL. So it can be fixed quickly. I will again create another repository. I will copy the address from this EPL but I will set a filter to download just the JeberPie package. So I will merge it to that existing spacewalk client channel and set another repository called EPL. I will have to change architecture. Can you handle package templates at all? If you have two repositories going to the same channel. The NVRA has to be unique, I think. But there are cases that they are not unique but then we check some of these packages. Yeah. I forgot to include the JeberPie only here. So now we try to run it differently so that the repository is syncing from the command line on the server. So we will run a command like this and it will just run repository sync and we want to sync this channel. And the script will look into the database which repositories are assigned to that channel and do the rest of the things. Yeah, there is some error but this is caused by bug in the current release and it was already fixed. But it doesn't matter in this case. So we downloaded this one package JeberPie to that and added to that channel here. So now we have 26 packages in that channel. So I will run the installation of the wrong machine. Yeah, package. Maybe I will just have to wait for a while. So now we are successfully installed usadi demon from the channel. Now we will have to set it up like SSL certificate of the spacewalk server to successfully communicate in-between them. So we will set it up like this. We will go to sysconfig origin-sadi.conf And here we are setting SSL Or maybe it's not needed because it will use that one. Yeah, it's needed because I registered it using HTTP without certificate. But I first need to download that SSL certificate from the spacewalk server. The spacewalk server has a public directory where is this certificate stored. It's also packaged in the RPM but could be downloaded separately. I will download it separately and place it to the user share and hn. Okay, here we go. We have this file. So now we will set the certificate here. And of course I don't remember the name. Now we can try to store usadi demon. Make sure it will even work. It will not for some reason. So let's look to the wiki page and let's search how to set up the study. Okay, so I'm not sure what exactly is wrong here but it doesn't work. Okay, never mind. Yes, I think I disabled firewall for this workshop but I already tried it but it worked but I had to do some step which I don't remember. Okay, never mind. Maybe we can go back to it later. Yes, we can ping spacewalk example.com from here and we can ping it from here. We are contacting spacewalk example using this, yes. Using this certificate. Okay, let's skip that for now. So it would allow us to instantly or almost instantly do pickup actions scheduled for current system like installing packages and so on. Now we tried to do a bit of configuration. Well, this configuration in spacewalk is not very advanced. It's not like the modern configuration engine in joints but it can do something. There is a concept of configuration channels which is very similar to the regular channels but in these configuration channels we have configuration files or binary files or whatever. So I will take some configuration channel here and name it like that. Like that, like that. We will add some file. For example, I will change ETC modD to something. Okay, I have to go to the crit file, not upload file. Okay, so like that and we will set the message to, for example, hello Facebook managed server A. So we create a first revision of this configuration file and consumed from this configuration channel and here we have some list where we can list these files if we create another revision of these configuration files like I will add here some new line we can also do some comparison of the configuration files to see the differences. Well, the new line isn't recognized, but never mind. Now we want to deploy this configuration file on that system. It can be done quite easily using configuration and subscribing to the client to the configuration channel. This configuration channel is completely different channel than the base channel for consuming packages, basically. So we can subscribe them both at one time. There are some additional setup which is not needed for us now and now we can finally schedule the configuration file deployment but I'm not sure where is the correct page for that. Yes, I need to subscribe to the channel or I already did that. Well, I lost a bit. Yes, I need to enable on that system the deployment configuration files capability. Yes, I remember. So I will install the RHN-COG packages by... I'll install RHN-COG like that. There is a couple of them. Yes, OK. And I will have to enable which actions I can do with that system and I will enable them all. Here we go. Now we can deploy that file on this system finally. So I will select this message of the day file and schedule them for deployment. OK, scheduled. Now if the OSR dispatcher would work, it would be picked but I will need to pick it manually. Unfortunately, but never mind. OK, now I will try to SSH to the client machine again and check if we see the new message. Yes. We already deployed, we successfully deployed the configuration file and this is basically it. There is a possible to also deploy binary files to whatever locations and override these centrally managed files by locally managed files for per system. There is really a lot of possibilities how to customize this procedure. So that we already tried to manage that system by installing packages, removing packages, by synking new content to that spacewalk in final repository. We already tried to check or we already tried to deploy some configuration file to that system and yeah, there is there are things, there are other things to do. For example, we can provision that system again like we say we don't already like CentOS 6, so let's try to CentOS 7, but it would require another repository synked with kickstart 3, but I have here only the CentOS 6 synked, so I will try to reprovision this machine to the CentOS 6 again like re-install this machine. So it will be quite simple. First we will check the systems page and here is a kickstart kickstart menu distributions and we see we have already CentOS 6.8 kickstart 3 here there is some configuration for it and if we want to actually apply to some system we need to create a kickstart profile which is here so I will create kickstart profile like that which will subscribe our system to CentOS 6 and we will kickstart it with kickstart 3 CentOS there is 6.8 which was created which we saw here distributions so now I will next password leftcon ok now we have a kickstart profile there is much things to set custom kernel options and commands some variables like which channels we want to subscribe after the kickstart is done and so on and variables like I don't really understand all these things which are that possible but there are possible there are some parameters like there are some default ones like setting a partitioning file preservation like languages and GPG keys whatever you need install packages after installation what needs to be done which activation kick should be used for register that systems to that channels and also you can specify some scripts which should be done before kickstart or after kickstart here and the script could be dash script or python script or whatever script now we can look at the final kickstart file here can see it is generated of course so now I will just say hey I want this client to be reinstalled using this kickstart profile and I want it now so I will schedule kickstart and finish now we will just have to wait to client to pick up that action or if also dispatcher would work it would be instantly so I will run a region check and it doesn't work yeah it worked there is probably something missing very easy to fail okay that's not helpful I will enable the apptall repository because I think it needs some package from them from that like cone which is quite important to kickstart I will yeah I am repose I am not going to sing packages one by one to that to space walker just enable the repository and let client to directly download it from the external repository in this case sorry so I will just and now I will schedule the kickstart kickstart again I will pick it up again yes I will switch to the console after it yes the system is going to reboot in 3 minutes yeah there is some error I don't want to see that so I will try to look at that client console so let's wait for I don't know 2 minutes so have you succeeded with the image system is going to reboot in 3 minutes okay give him some time to go out okay yeah I can reboot it maybe but we can wait one minute so I am basically done with what I wanted to show so now what do you want to see I can see some things here and what do you want so I am not sure the reason why I am here technical accommodation already and recently a handful of weeks ago I got a customer with satellite 5 I have absolutely no clue about satellite 5 and spacewalk and whatever so I started with the documentation I am eager to see how the upstream version behaves because I have another customer which potentially liked new spacewalk don't really like satellite 6 and still stock on satellite 5 and they told us we will switch to spacewalk when you stop supporting satellite 5 okay so for me it is also interesting to see what are the differences between satellite 5 and spacewalk is it possible to do the same stuff with spacewalk as we do with satellite 5 or there are some restrictions is there stop in spacewalk I think there is less restrictions in spacewalk but basically the same things can be done here it is very similar so it is more or less the same code with a different logo more or less from the okay I know what is the problem here because spacewalk.example.com cannot be resolved in this time after reboot so I will just put the IP here and it should work with IP and yes okay like that maybe again it should definitely work but it is because the virtual machines names set up and there wasn't really any set up I saw that spacewalk is supporting and working based distribution mm-hmm so you have packages for rail and fedora yes is it possible to handle soon as the end of the box with spacewalk I think could be I'm not sure how much it's possible with spacewalk, I've never tried it but I think on some level it's possible but there are no finalized packages which could be downloaded by a customer and it just works so he has to do one thing by himself okay same issue no this was something different I think I can fix it now okay maybe I can it was because I enabled the child channel to be an enabled on the kickstart time and it failed on some people data error I haven't seen before okay so I have here it can go I have a backup of the image so I can try it again and do it somehow simpler so let's kill this machine and go far, a little bit images so we have this machine again and do we have network yes okay that's good I will delete the old system because we now have them twice here and the first one doesn't work I will enable client repository and into kickstart yeah this is a problem with kickstart because we mainly test real kickstarts but sentOS kickstart have some differences and it's not one-to-one about activation keys in skip disk app activation keys you can I'm not sure about that I'm not sure about that because it worked for me even if I haven't this check but I will try I will schedule kickstart again yes but I'm trying to remember yes okay I know I have to schedule it again because it failed again okay this this error and I don't know if it's relevant here although okay let's try it again so does anyone want to help to be setting up that containers or images and spacewalk server and client devcon admin I feel try to to go to the IP address I mentioned in the slide here anybody to the spacewalk instance no looks good yeah finally okay crash file fetching packages so we will see looks really good let it work what is your personal relation to spacewalk I'm developer of spacewalk are you one of those guys currently preparing 5.8 hopefully it's gonna be good release it's your job it will be one of the key products or key things to fade out origin so it has to work so does anybody have a question or won't help with something come on I left my notebook in the hotel spacewalk recording too so if you would have the time and could rewrite some feature or make it better what would it be I don't know maybe the fact it's based on old technologies now because it was written years and years ago and now it uses old versions of libraries and old versions of everything I don't know maybe this would be good to rewrite to use it more modern and modern things maybe there is of course not much time for that the configuration management is very basic here and it's not powerful enough to compete it's not to use script in the post install to install an object or to install an entity or let them have all the power of modern config management it would require to install an object to install an object to install an object and it would require to integrate other projects into spacewalk and we would basically rewriting cattle or something it would just be work what anybody did already for writing the project is innovative as well so set alive file stopped once set alive it was out of order so we are not pushing development for spacewalk or set alive file but of course it's an open source project and I would compare it to picking also for it's alive brought in quite a lot of improvements so it's slowly moving on but for that what it is it works so maybe the biggest drawback is old libraries maybe the old channel virtual machine but something we will address actually the Java virtual machines is pretty good but the library it's working with poster which is also a good thing so in 20 years you needed Oracle in the back end it's a relatively easy thing when you see he was able to kick start a box in about an hour correct with some difficulties let you show the same thing from the former guys so starting almost from scratch the only thing he did before was synchronizing but everything else he did inside of his hour and that's not really possible in form of whatever it is but usually you invest a day or two to really be capable of provisioning so it's a simple tool because one job doesn't do it really bad we're gonna use that in marketing but of course it's limited it has no real content management it has no real future content management was used 15 years ago so it can't be good but it sounds real cheap it was never really intended so the point was to be able to deploy the basics in a big channel in fact then it was always intended if I wanted to control these various things done but no I think space box will be around in a long while we're happy to see here in a new oracle using it in some capacity and we have to figure out what we can do I have a point here but in a few years we have curiosity I know they touched a little bit he touched a little bit on this but are you looking at grade management just as part of it be able to operate like this and let that be swapped out in those configuration management which they swap out so what is the with we have satellite 6 but as you mentioned you're not going to say you can't register the machine back to into a core, into a tower in the back of the in the back of your kitchen but also they have this understood I thought that it was said earlier on that satellite 6 is just space walk no, satellite 5 satellite 5 is space walk satellite 6 is form ok and then also we met Povet and then satellite 6 in the future we'll be getting ansible but with space walk and satellite 5 but there is some guidance as far as how to take space walk and if you want to do ansible things afterwards it's not that I think you're at a 2-3 channel just to get the SSH key on the machine stick that in there in your ansible utility or if you're a tower user do you have basic configuration management in your robotics through space walk in that step you want to plug in the towers in the enforce go importance and so much is said so much goes for ansible so you can actually hook up your ansible inventory out of space walk is that a problem or what is it how about yourself what's the screen we have some people do you have any development do you want to get one all the space walk development now now too and Jordi are here you can I need to put this I thought it was yours no, that's for the room I saw how it works with redhead ripos it's just name, label architecture check some type for the data basically all and here repositories IFS signed one repository which points to the directly to the center and that's enough to create the kickstart source the repository has to contain if you look to that repository it has to contain images and then it creates a kickstart yes if the repository doesn't have these things you cannot use it as an install source there is additional option like sync kickstart because in space walk reposing it's not a default it's just to specify separate option unlike satellite sync satellite sync does it play default satellite sync is completely controlled by RHM I read it basically but satellite sync can be used also in space walk for synking from export channel for example or can be used for synking from another space walk there is a lot of possibilities how the content can be there is also support for DPN or something but it doesn't work very well it works the basic installation of packages works but the advanced features doesn't it would require more work nobody is willing to invest it's not priority for that the install base is still pretty large there are quite a lot of customers still working in satellite so just to give you a little bit of feedback I heard one of the key products so I heard it carefully I heard it like that it's still very used heavily for that it's extremely important that we are able to lift the customers to 5.8 and to attach them to CDM to make sure that we are capable of switching off RIJ as soon as possible switch on off the front end in July so all the rail system attached to RHM will lose their support basically and satellite 5 should be lose the connectivity beginning next year hopefully so satellite of rhm.redat.com will be switched off for this reason we have to make sure that 5.8 succeeds where do we get 5? the customer will be able to do it now it's still somewhat tricky we have quite a lot of customers running satellite 5.6 so even all the ones even rail 5 but it's still supported but do rail 5 rail 5 is going to it's end of life not really end of life so there is an extended life but still yeah and we have a subscription that we still get support for another 2 years it's a nightmare for our current developers they have to change something in rail 5 yeah and yes it becomes a nightmare for us in support imagine when they are running rail 4 still luckily I don't have any more rail 4 in high customer base but rail 5 is still still in use and we have currently to convince them hey there is another subscription and it's a good idea to buy it otherwise I think there are still running some rail 4 and even over and when they open cases and you try to tell me yeah it's like picking but it's time you know it's okay maybe in big organization it's not so easy to upgrade probably the problem is today most larger companies you need to get budget to work on something and budget to upgrade something is usually not not given by the manager so I have one really big customer so my position is 10 I have one customer and they are already finding the season team for more than a year to get budget for a rail 7 rollout management doesn't see any value because they replace rail 6 it works nothing nothing new feature which really brings the company forward so they don't spend money for such an upgrade but from 4 to 6 at least rail 4 rail 4 was really painful that was my personal experience rail started to be really useful with 5 and 6 was a great great stuff that worked 4 was a little bit painful I wouldn't be surprised when we have people requesting support even off the end of life just because it works the only thing which was missing I had a feature request during summer for TLS 1.2 luckily we have a page describing that we don't do it it's simply too complex we would need to replace someone as a senator surrounding packages to make it happen that's something we don't do so when a customer is relying on security it has to upgrade to at least rail 6 that's great because in bad cases we will not replace as you saw as much they pay basically we support everything every major release of rail so the customer is not urged to install something else he has to buy a rail subscription and we don't care if you run 5, 6, 7 we support everything we support but still we have our limits you are somewhere in support delivery I am here you don't know there is a translator an open ticket so let's say the first part you are one of those magic people in covering radio translation ok let's say first as a support and some info about products and subscription for the more complicated issues just think the engineers the next step and you are based in in Zurich so in Switzerland I have been here in December I have friends ok cool I have one customer opening German ticket and they do my best to switch then for waiting on owner because I am able to speak German so it's not necessary no I don't speak German I speak Italian and Spanish so in English where all the people are where mostly we are only girls now on translation team now we are going to have a guy for German friends and I am supporting Italian, Spanish and English we are already meeting it's a nightmare the latest issues it was a lack of everybody out ok at the moment the majority of customers is still running so customers are talking to each other and when someone collected experience with satellite 6 they tell their colleagues and we have quite a lot of customers they still stock on 5 they wait until satellite 6 is really production ready which wasn't the case at least for the last years it's getting better and better but still there are issues inside I think it's much better we don't have so many cases now because we are kind of scamming us for the first time I remember last summer it was the operate from 6.1 to 6.2 it was a nightmare it was something we were not really testing and it was something which brought up issues, key issues in satellite 6.1 installations so when you once had a broken suit or a broken publish process satellite was still working but you were not able to upgrade it to 6.2 and that was really a nightmare and for us in support it was not possible to fix those installations so in most cases we had to engage engineering I think there are so many possibilities waiting on engineering maybe forever this is a problem for the support team because we don't have feedback at the end maybe five years four years and you don't know to say for some help this is the biggest problem let's see the support because they are calling and you don't know what to say you don't have anything anything back at the end it's a big issue but at the end some customers they say we stop using it doesn't make any sense I think that's really an issue with standard front line it's a little bit better when you have a technical command so my job is really to reach between engineering and the customers so to also make engineering and you are also supported in your part of the support team so but for a normal front line engineer you get a satellite case and you never get rid of it the case is open and waiting on engineering because you don't have the guys don't have anything to do they work till point and then it's a Godzilla and you don't have the time to really reach out to the business unit or whatever it is to get really an answer so many bugs are you also involved in the satellite case? a bit a little I'm not working on it actively but I heard some stories how things were recently I was also on redhead summit where I was talking with some customers directly and I heard some feedback on it I got a customer with satellite 5 end of December they applied to do a transition to 6 but I was trying to take it slowly but on the other hand I have to be fit with this product so I installed at least once a satellite 5 figured it out how the sync works I attended the presentation of 5.8 to get a little bit of an idea what does it because they didn't use it before ok yeah and this customer will clearly do an upgrade to 5.8 because I don't think they will be able to implement 6 in the next few months they have 4 data centers spread over the whole world around 2000-3000 service so it would be a major effort to replay satellite 5 of 6 and when they see old issues around satellite 6 it will be hard so I really have to support satellite 5 for quite a long time I have to understand for reasonable idea so thanks for the invitation yes, some customer setups are so big so if for development some customer database their data you see thousands of service yeah I think their system are influenced sometimes you can see some thousands of systems there sometimes we are touching the limits how so even one more system cannot be handled here so we can find me on IRC go to an engineer and all the good people are on IRC as usual sofija, you can find me as Sofija on the GSMM and on all SPR the channels of almost another Sofija so as nickname I mean if you ever need something it was a hard start started lost family with rendered in the beginning it was really hard to find people supporting luckily I'm the second talent Switzerland so I got a little bit of support from my colleague and in the meantime I had my services so it worked out but you never know it's good to know people in the supporting field sofija we have Daniela Daniela on Sofija sofija and I think we are both here for our onboarding as well and here go italiens what we can say italiens I'm Greek so I know the spirit is the same massage we have we're crazy we need to leave the room for your effort thank you if you want just around because we're going to be here for the next 10-15 minutes the tracks so go fast bye I guess you're done in here too would you stop those you are free I would like to say that here on those A's and 112 and 113 I think I can close it and close it thank you