 Hello, welcome to my talk rolling your own security team for fun and no profit at all We will come to that later while we are focusing on fun. It will be clear at the end So let's quickly go through the agenda First I gonna talk a little bit about what we actually do as in the Arch Linux security team Then we will a little bit look into how we got there and take a small Look into our security tracker, which is our central platform and then discuss the core of this talk basically what we have learned in our journey so far and Finish everything with a small Q&A session. So a little bit about myself I'm Leventa Poyak and I'm a full-stack software and security engineer and also love DevOps duties I joined Arch Linux in 2014 And I'm active in a security team packaging Development and also DevOps. I've been elected as the Arch Linux project leader in 2020 So let's talk about what we actually do in a security team So basically our main goal or our primary value is keeping the arch users safe this is what everything is basically about and If we think about what that means the the main Interaction from a user point of view with our distro is Software installing packages because that's where the software comes from And you're doing that by pulling the packages from our repositories. So Basically Yeah, we were responsible to ensure safety for all users by taking care of the safety of our packages What we on top do is? We also consult infrastructure design choices nowadays and also review some configuration related to it So this is because we also run different services like the a you are and other things and also own machines that we use in the whole distro process and Releases so it's also equally important to focus on our infrastructure ourselves and On top we also coordinate information What that means is basically from a user perspective You see advisories being issued from our side Which basically warns users that there has been Certain update released that fixes shot some security related shortcomings So we also warn about the impact of such Problems so users are aware and can prioritize updates for example, but also We coordinate embargoed issues Barcode issues just to summarize it shortly if you're not familiar with it It's basically coordinated disclosure where you have Upfront information about the security vulnerabilities that is not yet public. It's just shared Across various entities that can prepare updates review The problem or the fixes that are related to the security issue and also have a fixed date where This information gets public so for example every distro can In a timely manner release updates at the same time So we also coordinate that with our with our package maintainers Yeah So We now know what we basically do today, but There is a lot of evolution that's led us Yeah to get there where we are today So let's have a quick look into the timeline and basically how we got here and yep It was a bit before 2014, but I think January 2014 is a good date to pinpoint Some community efforts that were that existed back then Basically community members Filed bug reports and issues about CVs Which means CVs are an Identifier for a specific security vulnerability. So it a certain issue can be referenced in a Uniform way and easily recognizable with an ID and yeah here So the community filed bug reports and patches, but there was nothing really structured We're still very thankful Even up until today that our community still helps us and files bug reports. So thanks a lot there But there was some kind of demand for something established something structural something persistent so this led to In March 2014 that Ellen basically made a call for action to fund or to fund such a security team and Yeah, have have really something established inside our distro that is Dedicated to be responsible for this kind of things It Yeah, the main focus that Ellen wanted to see here was our packages. So this is also as I explained earlier basically the main interaction point from a user perspective and This was what Ellen really was looking for it took until September 2014 until Me and Remy basically founded the security team. It was day zero kind of we came up with with well-defined workflows and structures and also templates for Advisories and really wanted to send out advisories to users things like that did not exist before and This was basically the day zero where where this team has been founded It took over two years to Finally come up with our own platform to coordinate all the work so in around December 2016 it was quite specifically December 2016 I still remember it because I made the big release during the CCC Which is an annual conference at the end of the year And it helped us streamline all our work So I will come to that Shortly and it also show a little bit how it looks like how we interact with this central platform to coordinate To coordinate everything which is package related So and this September we had our sixth birthday, which I'm really proud of This was a long journey so far and we evolved quite a lot And I'm especially proud of it because even through Ellen back then made the call for action to Have such a security team once we stepped ahead and said yeah now is the time we Are really eager to see this happen and we want to invest time and We are very passionate about it. It was kind of skeptical and said like in six months we will be gone And turns out we are still here and yeah, we've come to stay so that's a good thing um All our efforts also led to a lot of visibility and fixes and Improvements in our distro. So these numbers are just from our security tracker Basically, it is a bit higher because I I did not count everything That we released and issued before our central platform went live But so far we have issued over 880 advisories and We have released Over 1100 Packages with fixes that we tracked that had security vulnerabilities or Well after this release they hopefully didn't have those anymore The discrepancy is here a little bit because Sometimes we decide to not issue advisories. This is because Artists are a rolling release. So we only have one version and we expect users to kind of update in a timely manner um, which which means if we If we want to track all Fixes for every software that affects our packages and we do so hopefully But sometimes It takes a while or some some issues were discovered or at least the made public um Very far in the future after a release So for example if an issue was fixed for six months or something like that And we then track and add the information to our platform We basically assume that users should have updated within six months So we don't issue advisories if the severity of an issue is Yeah, also not too critical that we say we still want to warn users after such a long period of time So this is why this difference in the numbers exist um now let's look a little bit about how our security tracker works and You can reach it at security dot arch Linux dot org It it it helps us keep track of all the information Of all the issues and information related to CVs and Certain open source software We also use it to track Yeah related things like links change logs also commits because We always try to find commits so we can easily verify certain fixes and if they Those fixes really made it into a release Our platform also has a central to-do list. So we we um Have a much more streamlined and easier way to find What we need to do right now what kind of packages got updates released and things like that Before our platform before our central security tracker We needed to do all this manually, which was a lot of pain Yeah A lot of pain So we are very lucky that we now have a very well established central platform. Um, I will show shortly show some Some examples, uh, how it how it looks like also describe a little bit What what we did before we had this platform? um It also has or it also provides an api for external consumption Which means just an example the data could be pulled out of our tracker to create tools like giving users the ability to correlate the data on our tracker with their local systems, which basically Makes it able To warn users that Or make users aware that they have some packages installed which have known vulnerabilities As well as that there are updates available that fixes certain issues Which yeah, it was not yet updated on a local machine or a user um We the world tracker is also open source open for contribution And we are also happy always happy if we can extend it further So we are inviting everyone to take a look and if something is missing on the platform We are happily accepting patches Let's look a little bit at how it looks like this is basically the index the the main page the front page when That you will see when you visit our tracker It lists all the currently open security vulnerabilities You can also go through all the fixed ones and get also more detailed information If for instance you click on one of the issues one of the cvs you get a very detailed overview Including descriptions severity and things like that. It also has cross references So it shows what kind of packages were affected by an issue or if they were some advisories already released This kind of handy actually so makes navigation and Yeah, collecting information about certain aspects very easy and convenient From our team perspective, we have a very easy to use form-based input Where we can add details to the tracker Back then this was not that easy. It was very annoying and cumbersome I will show in two slides how it looked like So This is a huge improvement. I can really assure you We also use all this information to correlate and and assemble a final advisory So it lists all the issues that affected one release summarizes everything States which version is the one you need to have installed at least installed to make To make those issues disappear and we also have a summary basically an impact All the issues what kind of impact do they have for a user? So it's easier to understand from a user perspective without reading through a wall of text sometimes What is the actual impact? Why should I update or Or can I just delay it a little bit because it does not really affect me or it is not too critical So I don't need to Run home and install updates on all my systems or something like that And back then we we also created this one manually. So This also gave us a lot of free time We also have a logs so audit logs where we can see diff base changes So it also makes the work a little bit easier because if we Want to have a discussion about something for instance if a severity got raised too high Or one of our members thinks it should really be high Because it affects I don't know for example If if it was Detected that is not alone just a local issue But it is a network facing issue then you also want to of course erase the severity And sometimes you want to have a discussion about why some data was added in a way like it was So this helps also to To initialize such a discussion and know who Entered what kind of data? Yeah, this is basically how it looks or what it looked like before we had our central platform This is a simple wiki that we used we inserted all the entries manually We keep track of updated packages manually We wrote all those advisories manually which often were not really Fault tolerant and we had formatting issues and copy paste issues and stuff like that And it also consumed a lot of time. So it was really annoying to do and As you can see this this platform really helped to streamline the whole process for us So let's a little bit talk now about What what we have actually learned in the six years? And what what we have learned by developing the tools we have today and why we did so and the stuff like that So one of the main central Core values is that optimistic optimization is crucial. It is not something optional. You you need to always optimize We'll talk about that very shortly. So let's let's quickly go through this list It is basically a high level summary of what we have learned in the six years and then we will get into detail shortly after Another very important Aspect is having motivated people. This is also something non optional You really need to assure that you have motivated people that are passionate about what they're doing um And if you are dealing with people it is equally important to always be excellent to each other We also talk about that a little bit in depth in a couple of slides But you really need a healthy environment and this is very Very important Also at You also need clear responsibilities So think about your responsibilities and Don't overextend in early stages What I mean by that we'll come to that Also in a couple of slides. So now let's let's focus on the first topic, which is optimization Often in my career And over and over again It happens that people or if you if you want to improve something and you say Look at this this you really need to improve some aspects and then you get a lot of Or a better handling of something you get more free time You you don't need to do busy work and stuff like that. What busy work means. I will explain also shortly, but They they also say they always say They're too busy to improve Which is which is the number one biggest mistake if I if I would Need to name one in my opinion and it is it is really a fundamental part of every tech process Or at least it should be a fundamental Central part of every development process At the end you need to think about it a little bit different. There's virtually never too little work Uh, especially in the enterprise environment where you're hired to do something There is really never At a period of time where you say oh now I'm bored. Now I can optimize. This is just not gonna happen You always have work. You should always do something and so If you take that into account then the only conclusion is that optimization needs to be a fundamental core value of your world development process And this is really important and if you if you look at it like that Then you will also never say that you're too busy to optimize because that's just a lie to yourself and just don't do it It's a very huge mistake So why do we want to optimize? The primary goal is that we want to fight busy work a busy work means Like work you're doing that has no value itself so basically you're just doing the work to achieve something different but To picture it a little bit, let's say we need to dig a hole and You just have a shovel and no further tools And you need to to bring or to carry the sand mile away or something so At early stages when you have no additional tools You can just use your shovel and just walk over a mile every time you dig deeper To to yeah to carry the sand over But this is as you can imagine highly inefficient you can do that at the beginning but This whole process and just walking over and and dropping a shovel full of sand Is literally busy work because it has no value You're just doing it to achieve your final task, which is having the sand somewhere else So this is busy work And if you want to optimize away this kind of busy work You would for example come up with a push cart where you could put in a lot of sand and just carry over a tons of sand In one move so you reduce the busy work without losing any core values So this is super important if you want to grow So Identify and eliminate your busy work parts But you still should always also assess your workflows There are central parts of your workflows that are that that have value itself So you don't want to automate that What I mean by that is if we take an example in in what in the work we do in the security team There are central Parts of our workflow for example the verification step that has huge Huge value that you do not want to lose By by having it fully automated What I mean by that is we have a final look over Or Let's get one step back If our platform In the to-do list of our platform There is an entry that some package got updated which should now carry the fix For a vulnerability And normally you could automate it away and automatically send for instance the advisory right away and stuff like that But then you lose this core value. I I'm about to explain this this verification step So we don't eliminate it that part. We still verify it manually that okay This is really the update There was no mistake done the commit is inside the release and things like that So this is core value So this is why you need to look at your workflows and only automate where you don't lose values We we had that several times already is that some patches that needed to be back ported were applied In Unsufficiently so basically a fully automated process wouldn't have courted and we would have warned about About having something fixed which in fact was still vulnerable So this is why why some manual steps in your workflow can be good. So don't just Blindly automate everything away This also has a little bit to do with peer reviewing critical parts We we we don't just manually verify it like person and then go But we we try to always peer review all the things at least by two persons or more This helps to catch errors that or Fail or just overlooked stuff Sometimes it happens. We are all humans errors can happen and we should think about the process where we can Reduce the amount of errors that could potentially happen and peer reviewing is a really great thing to do It also helps cross cross a knowledge share and This is also very healthy for your world process It is also very important to document everything So write everything down in a wiki in a kanban board or something like that Even if you don't have time right now to solve all the optimization steps you have come up with It is still important to document everything Process we have talked about it Optimization is a fundamental part of your workflow. So by documenting the results of your thoughts of your analyzes You already did a specific Part of this job and by documenting it you ensure that you can just pick it up and solve it later when when you have You are able to to Pick up this part of your optimization and and solve it So even if you don't have time for it right now, you should always document it. This is a good thing So This is It about optimization. We can now a little bit switch over to to the human part to to motivation Motivation is the key to success Basically for everything and also for For running a team and fresh team Motivation is one of the most important things There are different kinds of motivation That that we want to to talk about a little bit There is intrinsic motivation and there is also extrinsic motivation. What I mean by that is And and you can if you're interested in that topic, you can just look up information in the internet. There is tons of information about about motivation intrinsic motivation to summarize it roughly is If you're passionate about something if if the motivation comes from within yourself from From some some values some some some ideas you have really it boils down to having passion and and being self motivated And contrary to that extrinsic motivation is just That there are some goals you want to achieve and everything else is just Yeah, you're not passionate about the thing itself, but about the goal you want to reach For example in a corporate environment that would be getting a salary raise or having a title or something like that The problem with that is that it always needs reinforcement Shortly after you get the salary raise This isn't enough anymore and at one point you want to have Another dose of motivation and then you need another salary raise So what you really want to find is intrinsically motivated people because they're passionate about the work and they will do everything to improve it and This is Extremely healthy for a team. So it is very tough to identify intrinsically motivated people, but is a key component if you want to have a healthy and passionate team It is equally important to maintain motivation So just being motivated at the beginning is not enough. You need to maintain the motivation You need to to keep being motivated There are different ways to achieve it It is also a very personal thing because intrinsic motivation is personal But you want to give Utilities to people that they are able to maintain motivation We come to that Shortly when we talk a little bit about identifying Responsibility sense things like that But all that also boils down Or at least motivation is part of that That that you need to to make sure that you're not burning out or any single one of your team members is burning out There are different ways How you could burn out In a startup phase and this totally fine. It is also Beneficial if you want to start establishing something that you work like let's say 200 percent but this this can't be sustained for Extended period of time you will burn out. So you need to identify that in time and and and reduce the the problematic parts that would lead to burnout Just an example about myself at the early stages. I had some scripts running to identify package updates And basically I had an alarm clock or some kind of ring That would appear In the middle of the night when some new updates were released So I could wake up and just enter the data in a wiki and It was really cool. I was really passionate about doing it about Getting the information out to the users as fast as possible And it was also I believe healthy for for the startup phase, but You can imagine this is not something you want to do forever. This is not cool And you need to identify it and then just just Solve it in a different way which which yeah, which avoids people burning out. This is extremely important If we are talking about people another important topic is communication So you always want to ensure that you have a friendly and respectful environment and communication um The key here is nonviolent communication And you want to have a healthy environment that ensures it. This is also a very huge topic. You can also look at nonviolent communication um but if we want to summarize it in very small bullet points, I would say just there Everyone has basically needs and and you want to solve something and But you may have different point of views or different things you want to solve as your need The the problem is we are all humans and I believe nobody really wants to to fight over something or be disrespectful to something This is just something that happens And you need to take that into account Especially if you're dealing with text and not the voice or personal discussions, then you're also Reducing some side channels. So what I mean by that is you don't have emotional layer You you can't see the expression of the other person. You also don't have maybe the voice you can use to to To hear out that something is meant in a joking way or something like that if you have only text this this increases the likelihood for misinterpretation An easy thing is just Just look at everything that people don't want or people are not evil minded people Try to just get their points straight So you should keep that in mind and don't assume people are disrespectful or something and contrary to that if you write some text or you communicate with someone in some Different ways always try to do it in a way that assures it could not be easily misinterpreted And maintain a healthy environment. This is super important It is also important to be approachable What I mean by that is This also correlates a little bit with with ability and with being friendly and having a healthy and non-violent and communication environment um You established by that a little bit that You don't always need to run Around and see where you need to poke into I mean to some degree you want to do that, but you also want people To know that they can approach you and they can actively consult you when they have some new topics that should be discussed So be approachable. This also means have clear communication channels Like select one primary communication channel, whatever that is For us personally, it is IRC, but I'm pretty sure this wouldn't work out for other people so Just find for your environment and for your team What is an appropriate communication channel and stick to that and make it clear? How you can be reached? This is very important also to Little bit implement a security-centric mindset into different teams and that they themselves Take it a little bit more into account or at least come over to you and consult you in certain aspects It's also super important to be self aware. Um What I mean by that is and especially that applies for security, but also for everything else Um, it's totally okay to not know everything. Not everyone can know everything. This is literally impossible but the problem is if if If you don't respect your Your boundaries where knowledge about topics starts to get blurry Then this is a very destructive for security and also for general Tech-related topics or in general for everything basically um Of course, if you start pretending even through you Should be aware that this area starts to blur out at this point now Then this this this leads to two problems to to Overlooked security implications or something like that You want to be aware of it and then take the steps to either consult further Persons who may have more knowledge in a certain topic or at the very least educate yourself or further so you can make an Yeah, and an educated Response to something This is super important. So so always be self aware. It's totally okay to not know everything Um One thing you also should Think about our responsibilities. So define your core values. Uh, think Think especially at the early stages don't try to solve everything that you come up with just Think about what is the core value? What is the core problem you try to solve? Uh, with with Yeah, introducing a new team For instance, when we have uh, when we founded the security team The core value was packages. Um, and also very In a very limited way um so basically Yeah, basically just keeping track of cvs and App stream fixes and applying those. Um, but but you can extend it. We come to that very shortly Uh, but if you if you have your core values The reason for that is that you want to avoid over saturation. The problem is In the early stage, let's say when we are only two people The problem is if you want to solve everything right away and also take care of infrastructure and everything you can imagine Then you just don't have enough Time to solve all the tasks and you will miss some task or work gets delayed or updates get delayed or advisory Or issuing advisories get delayed. This is why you don't want to oversaturate yourself. Um, Think about the core values and extend them later and stick to them until You feel confident that now you can add more Areas to your responsibilities Before you're and before you're sure that you won't delay the work you're currently focusing on Um, yet just don't add more responsibilities uh It is also important to uh at a test step to to explore further areas. Um You you can do that by even either getting more team members But just throwing more people onto it is not not It is also not sustainable on a long shot. Um, you you want to extend your team But still while optimizing every single point of your workflow and tooling over and over and over again otherwise, you are highly inefficient and You're just doing lots of busy work and you just throw more and more people onto it Which creates other kind of problems if you then end up with having 30 people doing busy work. This just don't do it Explore further areas when you get free time by optimizing or by getting One two new members into your team And always think outside the box. So look around you in for certain topics or areas what I mean by that is All Directions basically mean if we think about packages that we talked about Um, look a layer below and a layer above if Let's say if we want to look a little bit further away a layer above it then What does packages mean a package is not just or not always just an upstream source It can have additional content basically for instance a system d unit file Which describes the service and This can contain for instance Information what users should run the service or what kind of additional hardening this unit activates or configures so A layer above could be hardening those unit files don't run services as root and you don't require restrict further down capabilities and and Other hardening ways This is a layer above we want to think about a layer below A package. What does a package mean if you if we look deeper into it? For instance, it could be a binary a cc plus pass binary or something which is compiled And what does that mean if we compile code then we also have compiler flags and linker flags This this could be also a way of hardening your packages and in revisiting the new new compiler flags that you can use like Pi or other kind of things So so just always look around and think outside the box, but but not just for one area You also want to explore new areas So look at the left and right of you. What I mean by that is Go away from packages if you have enough time and think about new topics like infrastructure Is a totally new area you want to grow into and this is what we did when we gained more More yeah more free time that we could then invest in new work So always be clear and explore your responsibilities when it's appropriate Taking or speaking about responsibilities The current team. This is who is responsible for all that that I described um Are our six core members here and at that point I want to to yet to send special thanks to every single one of of our team members and I'm really proud to have such a great team and to to I'm really lucky That that we managed to have such a nice team You can really rely on every single one of the team members and everyone is really passionate about the works that are doing and Thanks again. So if you see any one of those at the conference or something like that Send some some cheers or buy them some drinks beer amata something they like So yeah, this is give something back to them. They they rock um Let's a little bit summarize what we have learned so far About everything we have talked about in in this uh slide so far so it's a summary and You you I I guess you could have you you see how Passionately I also talked about my team and how proud I am this is uh also because They're intrinsically motivated. They're really passionate about their work and this is one of the key learnings try to find intrinsically motivated people who are really passionate about their work this is a lot cooler to work with people like that and it is also a lot healthier and It makes things easier to expand further areas and to evolve into something bigger and something better So try to find cool people Another important factor is optimization optimization is not optional and don't say you don't have time for optimization Then you're doing something wrong always have time for optimization. It is super important Otherwise, you will spiral down in a tech depth State which is very tough to get out And the further down you you you go the tougher it gets to get out of it. So just Just don't just don't always optimize plan in enough optimization time into your process depending depending on areas it could a little bit differ but Be clear how much time you always invest in optimization And maintain a nonviolent communication environment Be friendly be be excellent to each other and if you see Things can easily get heated especially if you or Yeah, especially if you're dealing only with text Attack that into account and try to de-escalate things and we're all just humans and we all just Yeah We don't want to fight each other. This this is really not something we should do at all We should be friendly to each other. We should spread love and this this is also especially important for For teams for internal and external communication be super friendly Then people will also like to approach you from their own and you don't need to run Around and search where you need to get involved because People like them to talk to you and to communicate with you You always try to also de-escalate yourself and revisit a discussion I also had it several times myself that I noticed that at one point I just misunderstood something or things like that and at the end it turned out that This Shouldn't even have been felt like it needs to go in some kind of heated way Because it was just a miscommunication And and and try to to step back and look again at the information and and try to To bring it into a way which is friendly And this helps a lot Also define Responsibilities think about the core responsibilities of yours and stick to them And when you optimize your processes your toolings and you get You get a little bit of Yeah freedom to to be able to extend the responsibilities then think about or then You can't think about it earlier And document it but then only start to implement or make new responsibilities to your core values If you have really the freedom to do it appropriately what I mean by that is like I mentioned earlier Where you're sure that other work you already have as core responsibilities don't get delayed It's super important. It's better to focus on something which you do appropriately than focusing on everything you can imagine and lose this Yeah, lose the the value itself because you can't sufficiently do the work anymore because it's just too much So At that point I want to thank everyone for listening to my talk. I hope I I could get my points Over to you and if you have any questions then feel free to ask them I also invite you to to come around and Chat with our team and maybe get involved in these different aspects For distro security. We also we always very we are very welcoming for everyone in our community And at the end is every every every part of this is a community effort. So Yeah, come around and Think about how you can get involved if you wish to So if you want to contact us here are some Contact information Thanks a lot. I hope you enjoyed the talk All right, welcome to the last q&a of the day and my name is mackie lannu Or my name is marcus and I go by mackie lannu and I have no other than anthrax with me And we're to do some q&a. So let's just dive into it Um Has arch considered becoming a cve numbering authority? Uh, yes, we did. So far. We just did not Apply for it. So I hopefully we will fix that pretty soon because we sometimes have Yeah cases where we should get some cve numbers ourselves So this would aid a little bit the process of ours Right sounds great. And that that question was by fox born by the way Um, next question by tam's Um, do the arts maintainers ever manually review code Is security a factor in deciding whether to package some software? um I guess partially it's up to the maintainers but at the end There's sometimes that software which is basically un-maintained upstreams and things like that um, so we try to identify it and See if we can get rid of it Sometimes it's challenging because you also have lots of dependencies and yeah, but um We at least try to have an eye on it and get get rid of it if it's really un-maintained and has a lot of vulnerabilities All right sounds great How to close source proprietary packages in the community Repository work, for example discord Um, are there any special procedures with respect to security trust distribution for these packages? Um, I wouldn't say there is this very strict guideline about that but First it boils down to us or being allowed to Redistribute something like that. It depends a little bit also on the license of such projects And on maybe if we get an allowance to do so with which we always are very keen to ensure that this is the case On top of that when it comes to to trust and security, this is a tough question especially because I think open source is extremely important, so It's always like, uh, why is the source not open? This is not good Um But yeah, it I it boils down a little bit to if just how widespread is the software and things like that So I don't think it would make sense to just yeah package everything Yes, um And the next question is by the same guy as the last question daniel are parks Um, what does it take for a proprietary package to get included in the official ripple ripples? Well as mentioned, basically it boils down to Licensing if we are allowed to or we have an allowance to do so And yeah, just have some sanity checks on if we really want to package proprietary software And um, is it really an established software in terms of is it widespread? So this at least gives some Indications about if it's trustworthy or not Also, you can vote on your packages right to to get them accepted and stuff like that Next question by you one one zero six Is there a fixed rule when no asa is done? You mentioned six months, but that sounded a bit like an example It was quite literally an example. We we don't have very strict rules about that Mostly if if something just takes very very long time And as mentioned the severity is not too critical that we still want to make sure to warn people about it We we I guess we always have a small discussion about it like look this package is fixed for Long time. So we let's not issue an advisory or something So there's no strict rule. We basically decide case by case Yes um Next question by oak Um, what software do you use for the security tracking workflow and by workflow there? It means the security tracking and finding and patching um For tracking itself, we now have have our central platform But this right now at least just a part of the world workflow because we also need the information To be pulled out somewhere Like mist or mitra or something like that Right now, this is one case where we are right now optimizing in and having it more streamlined better integrated currently it's more like security team members are responsible to um, yeah to to Fetch themselves the information get updated about new cvs and And all the information and appended to our platform our security tracker But we have identified this this is the current pain point of our world workflow So we are right now started to basically get it addressed and have automatically pulling in data from various sources um Yeah, I think that hopefully answers the question Yes, um next question by al-qasar 79 Is it part of this? Is it part of the security team? Teams mandate to verify the security fixes or is that an individual package maintainers responsibility? Well, I pretty much hope to package maintainers themselves also validated But it also depends on what what means validating A package maintainer ensures for example some patches get applied or something, but as mentioned also on slide 14, I think We still have a verification process that we really try to ensure I mean we as a security team that we really try to assure that certain patches certain fixes Were back ported into a package because sometimes errors can happen and patches not Properly applied or something like that So we try to always have an eye on it and verify that it's indeed got fixed Sometimes there are cases when we have some Reproducers that we also throw those against a fixed version and see if we can still trigger something So yeah, we do it manually Okay, next question by Hoyt Does arch Linux have any paid staff if yes, how many and where does that money come from? No, not at all. So basically we're all volunteers Just free time for every one of us Yes, so we don't have any paid staff But yeah, our income is basically from donations So if you like our project, please donate Go get some merch So the next question by arch guests Concerning arch infrastructures mirrors archlinx.org, etc How is the security handled in their pentests? Vuln assignments um Well, there are different aspects and this is an area we are thoughtly now Or since since a little bit Already involved there and we Well, basically there's some stuff that the DevOps team itself is also doing like monitoring known issues that Are affected by packages we have installed on our own ifrah Which is by pulling out the data of our security trackers And and do timely updates on that. We also actively say like oh, we need to Update our machines because like for instance some nasty kernel issue got fixed or something else On top of that we also review Infrastructure related things But this is a little bit like Where basically the work of the DevOps team and the work of the security team is overlapping So it is also a responsibility of the DevOps team, but we of course help to To try to review configuration try to review design choices and also actively try to to find vulnerabilities in In our servers So yeah, right So the next question is from several people Diabonus Grille and others How can you get involved with in the Arch Linux security team? Uh, that's a good question. I think at the last slide of mine I've shown some channels where you can get in touch with us most notably the IRC at hashtag Arch Linux minus Security where we are basically always hanging around So the first thing that comes to mind is finding Finding CVs or known vulnerabilities that we did not track in our tracker yet also, maybe try to Review something we basically always post our advisories beforehand So this could also be something just looking over it if the data inside it is looks good if The patches we mention are the the ones that really affect this this problem And also sometimes filing bugs to our package maintainers to get some stuff sorted out So I think it's easier just dropping by in the IRC and Trying to find information that we don't already track This would be a first good first entry point Yeah, the IRC is a great starter. It is a great place to start Um, so the next question by arch conf 53 Um, can you talk a bit more about the api for the security tracker? Um, where is it? Is it publicly available or only for for the security team internally? Uh, no, it's publicly available. I guess we could optimize having a proper documentation on it because I think we know we basically lack proper api documentation But what you can do is either a pen slash jason or Dot jason to basically all the endpoints like if you open the index page or Like you open a cve entry and you just add a dot jason at the end Then you get everything in jason formatted way, which should be very easy to write tools to consume the data But this is also something we have documented in our issues We really should have proper api documentation, which we currently lack unfortunately Yes, um So the next question is from Coden alt Um, and that's basically how how you identify burnouts Uh, I guess that's a super tough question and that could be a whole talk on its own Uh, I also wouldn't say like I'm an expert in identifying burnout or something because it's also Very a very personal thing mostly it it also depends on a little bit What what what kind of work or what kind of part of a workflow Starts to frustrate a person this this this could vary a little bit. Of course, there are Things which is common like oversaturation, which I also talked about which you always want to avoid Uh, if people are highly oversaturated this will definitely lead to burnout Um, but it's very tough to identify. I mean if if people are highly oversaturated They either can recognize that themselves or sometimes people can also from the outside recognize it and try to Make people aware of that and help them in some ways um but mostly if it's not very obvious it's very personal and I think this is a point where every every single one of us need to work on that ourselves Because we burnout it's it's mostly personal things that we are annoyed about So we should ourselves somehow create awareness of what leads to burnout in our case right So the last question is from In the kit tanko, sorry if I pronounced that wrong What are typical vulnerabilities? And what are the sources and New users or everyone um, so typical Wernher abilities could be For instance, uh denial of service problems in applications which basically means If you can't crash the application or make it a misbehave or something which makes it basically unusable um This would be one example or more severe things like Remote code execution, which means if you have A service or something like that, which is network facing then sending some specially crafted content to a service can lead to basically Giving the attacker the control to execute custom code and by that they can take over the system and Yeah, exfiltrate data and stuff like that Um, these are some vulnerabilities. There is A lot of different classes of vulnerabilities Mostly I would say they affect all users if they're inside a package or something But It could also be that it only affects all users and not new users for example If a fix was applied to a package because for instance, uh Um Directory modes or Something like that then it may be is that the package when you update it It does not get the new modes applied Um So this could be one example, but that this is or this this should be a shortcoming in The package update itself. So basically it should heal itself, especially when it's um security related I Hope that kind of answers the question Yeah, so that wraps up this q&a. Uh, thank you for answering the questions Sure, very welcome