 Thank you so much, Sivan, for the great introduction and also thank you for mentioning Michael Jung, because I just couldn't fit that anecdote into my talk anymore, so you're giving all the context that I needed. Thank you very much. Hi, my name is Elisa Lindinger. As Sivan already said, I'm the co-founder and manager of Superlap, which is a Berlin-based non-profit and I'm going to talk today about meritocracy, open source and funding. Thank you for giving me the opportunity today to share a bit about my work, my past work to be more precise. So let's start and let me give you a quick overview of what I want to share with you today. We'll start with a bit of context about myself and also most of all the work that my colleagues and I did in the field of open source funding and also research on that field. And we look at the report that is at the center of this talk, which bears the title Road Work Ahead. And while the scope of the report naturally is more broad, today I'll spotlight a few of the findings that we came across that relate to meritocracy, especially how merit in open source communities is assigned and how funding influences these communities and also the mechanisms of meritocratic structures. The report itself closes with recommendations for funders or FOSS projects that I believe are also relevant for FOSS communities themselves. So that's where we are going to end as well. And as Sivan already said, in 2016 I started working for the Prototype Fund, which is Germany's first public funding program that supports individual software developers either on their own or like in small teams to work on their open source projects. I'm sure that many of you are already familiar with the Prototype Fund, so I'll keep it short. It supports people for six months to bring their idea to a prototype stage. It funds projects in four categories, which are Civic Tech, Data Literacy, IT Security and Digital Infrastructure. And our role at the Prototype Fund was, of course, to manage the calls for applications and to support the grantees, but also to evaluate this public funding program. And during this evaluation work, we found that digital infrastructure projects had a much harder time to actually successfully apply for our funding, even though we really wanted to support them. We just didn't ask the right questions. We didn't offer the right forms of support, apparently. And we really couldn't quite figure out what the final reasons were, why those projects kind of fell through. And we were very happy that we received an additional grant from the Digital Infrastructure Research RFP that also supported the work of AirYiri and Jessica, that was just presented. And this grant allowed myself and other people from the Prototype Fund to look into FOSS communities and where our work exactly did not work out as planned. Because this kind of happened as another project on top of our work, it has a different name. We called the project IDE Implicit Development Environments. And this is basically something that ended in a number of reports that were authored. One of them was authored by myself and the other one was authored by Katarina Meier. I don't want to make things more complicated, but the past years have been tumultuous with the pandemic and everything. So now I'm not part of the Prototype Fund anymore. I've found it a nonprofit that is based in Berlin in Germany, which is called Superlab. So I'm not part of the Prototype Fund anymore, but you should definitely check out the work that my former colleagues do there. It's a fantastic program. So what is all that about the ominous report called Roadwork Ahead? First of all, the title, of course, is meant as a homage to Nadia Ekbal's book, Roads and Bridges, which I highly recommend in case you haven't read it yet. And with Roadwork Ahead, we evaluated the needs of FOSS communities that work on digital infrastructure in the public interest when it comes to funding for that sector. It also shows how funding programs fall short to provide specific forms of support that FOSS communities need and how funding can lead to unintended consequences. If you want to check it out, you can find it online, a clickable version and also a PDF to download and read later under the URL recommendations.implicit-development.org. Let me give you a quick summary of the scope and the content of the report. It's based on qualitative interviews and follows a human-centered design pattern. We conducted 26 interviews in total with people that contribute to a number of FOSS projects in different ways, as employees, as freelancers or volunteers, as developers, as community managers, as fundraisers or generally as networkers and communicators. And because of our special interest in infrastructure projects, we focused on people or projects that work on standards, that implement them, that maintain these implementations or run a service on them. The report formulates four different project types that also need different things from funders to thrive, ranging from the one-person shop, how we called it. I guess it's more commonly known as the infamous random maintainer in Nebraska, to proper organizations with a legal structure that employ people, collaborate with other organizations or businesses and have a community of freelancers or paid staff even or volunteers. The report formulates 10 insights into the work these communities do, how they are organized, how they perceive themselves and how they deal with resources and also, of course, what their strengths and weaknesses in that respect are. Connected to these 10 insights are 30 recommendations for funders and how to better position themselves if they want to support open digital infrastructure. For today, I've selected three of those insights that show how meritocracy is at play in FOSS communities that we talked to, how it influences their strengths and weaknesses and their relationships with funders. So let me start with the first one. The first insight that I want to share is quite obvious if you think about it. The insight is there is a uniform path that leads people into open digital infrastructure projects, or if you want to turn it the other way around, paths into FOSS are limited, very limited. So what does that mean? People's paths into FOSS projects usually follow the same route. Individual people spend sometimes months, sometimes even more than a year to read mailing lists, follow discussions, they try to make sense of the code on their own before they actually start to actively take part in discussions and also before they start committing code. And the fact that people need to put in so much time and work is necessary for newcomers to prove their commitment and also to be taken seriously, so to gain merit in a way. Our interviewees were aware that this status quo, this path strongly favours people that are privileged to have access to computers early in their youth, they can teach themselves, they also have time on their hands in the here and now, they have resources that they can shift to make that kind of voluntary work in the beginning possible. And the prevailing sentiment is yeah, that's not optimal, but it is necessary to guarantee independence of the project and also a quality of work. So this rather arbitrary pre-selection affects the lack of diversity in infrastructure projects in several ways. First of all, it limits the skill sets that are present in the community. If people join a community by contributing code, the result is of course that only developers join the community. And people with other non-technical skills who might be very much needed once the project grows and needs different things, they don't have a true way, a normal way of entering the community. Of course it also selects for financial background. People without a steady income or the backing of a financially stable environment have a hard time working their way into a project to the point where they are accepted as a contributor. There is no guarantee for people without those resources in the back that the time that they spend on a project will at some point pay off financially whether that be in form of contracts or job opportunities. It's important to understand that people who make it that far, people who have passed all these unspoken hurdles and have arrived and have been accepted to a community can be part of the meritocratic structure of open source. And I'm not doubting that the work that they have done is worth it, not at all. But others that were not able to follow that path basically have no way to become part of the structure of open source and of meritocracy and to prove themselves in a meaningful way. So as a result, that process has a self-affirming quality. If only developers make it into communities, their work must be of value and hence developer positions need to be more supported than others. To quote from one of the interviews, the person said, what we really need is a community person to manage us and to facilitate communication and also to mediate. But the acceptance for that is missing in the community. We need the money to produce code. So the initial filter mechanism for FOSS communities is not merit. It's resources. People who can spend time, work often as volunteers or redirect the resources, I already said that. Let's face it, people who are culturally similar in our societies. So meritocracy becomes a self-fulfilling prophecy. The second insight is that non-coding work and value are quite unbalanced. So what happens if the path into FOSS projects is by contributing code? And what if, as a result, the community consists mostly of developers and engineers? Well, they are the ones who will have to do the work that needs to be done to keep the project afloat, which of course also consists of non-technical work, which is project management, financial administration, design, community management, event organization, coordination with other projects, companies and public speaking, for example. And this work in our interview was seen either as a burden or as an annoyance, sometimes as an unwelcome surprise and sometimes it was a nice little side hobby. So these non-coding tasks are taken over by developers involved in the project because there's no one else to take them up. And the developers recognize that they are not very much qualified for this work and they often are not very interested in doing it. And that's a source of frustration because it keeps them from what they like to do, the joyful task of coding. So on the other hand, it's not easy for them to delegate that task because no one presents themselves as, hey, I'm equipped to do that. Just, you know, let me do it. And in spite of frequent complaints in our interviews about this work, even in fields where there were community management roles, fundraising roles, administrative roles, they were not that much valued. They were often not part of the communication structures. They were sometimes not really seen as the developer's team, but at, I don't know, team adjacent. And there was one interesting thing that we came across. So when developers were undertaking non-developer work, they favored technical fixes for non-technical projects. And I'm sure you've come across that once at least. One example for that is the social fork, where a lack of community work and care work in a project leads to massive social friction. And instead of solving the problem on a social level, parts of the community actually fork the code and establish a new tech project partly to circumvent this kind of friction, these social tensions that have arisen or even abusive behavior. And the last insight I want to share is where the funders actually come in. So what happens if funders step onto the stage? Funders tend to support the existing structures that they find in a project. They also like to focus on project work, developer positions, for example, sometimes also design, and not so much the structural support that is needed to build a thriving community, to set up financial reporting and all the things that you have to do if you want to live in the long run. So on the other hand, what funders support sends a message of value. Funding becomes a factor that reaffirms this kind of assumed meritocracy, the value of certain positions in tech. There is another influence that funders have. They prefer to support new projects. I believe we all know that this kind of innovation driven approach. So like community work or finances, maintenance is also not on their list of things they support. Another example of a very dearly needed position of very needed work that doesn't receive the acclaim and not the merit that it deserves. There's one other thing that we saw in funder-grantee relationships, and that was kind of a vicious cycle. Funders on the one hand found that a community is heavily focusing on developers and they want to add and support these highly valued positions. First communities, on the other hand, needed the funding, so they tried to preemptively, I don't know, ideal figure out what the funders wanted to hear, which kind of things they wanted to support. And they assumed that they are focusing on the tech aspect, on the developer position. So even if they wanted to apply for community positions, for example, they did not do it because they believed that their application would not be well received, would not be seen as worthy. So I guess it's about time that we all talk, communities and researchers and funders alike, about this kind of imbalance to give open source communities the support that they actually need. So what can we actually take from that? It might sound like, I don't know, like a damning status report, a bad description of open source infrastructure communities. It really isn't. Open source is awesome. I'm the first person to say that, and we need these communities and their work. Because where open infrastructure is absent, privatized, and often exploitative business models predominate, we see that all around. And these business models commodify people, they harm the communities that are most vulnerable, both in the physical and in the digital world. And while the existence of public infrastructure and public open infrastructure does not automatically make the world a more just and more equitable place, it is an essential prerequisite. So we have to better support that work because we really need it. So I believe that there are a few things that we can do to make digital infrastructure projects and also first projects in general thrive better. And the first one is, of course, to also scrutinize these kind of hidden mechanisms within our communities to scrutinize how positions are valued and why or why not, and how we can balance that a little better. We also, of course, just have to acknowledge that community building is hard. So it's better if we share what worked and what didn't and if we add that to the exchange that we are already having. And the last one, and this is also something that we actually propose to the funders in our report, I think there are ways to share the load. We can't expect all the projects to fund their own organization and have their own financial department and their community management group and whatnot. But there is one aspect, one approach with fiscal financial administration which is the fiscal sponsorship where you have specific entities that have expertise in financial administration and that take that part over, keep the discussion going, but basically liberate smaller groups of the burden to have to administer their own finances. And I believe we could do that in many more respects. The open source design network, it goes into a similar direction where they pool expertise from designers that are well acquainted with open source projects where you can just send them an email and ask for support. You don't have to have a host of designers on your team necessarily. And I believe that is something to go forward to build a system, kind of a backbone with the expertise and the knowledge and the resources that can help FOSS communities to do what they are good at to provide the tools that we all need and should use much more. And with that, I'm already at my end. Thank you very much. Again, my name is Elisa. You can reach me under the following emails and also check out the work that we are doing at Super. It's superwith3rs.net. So thank you very much. Sorry, Elisa. I think there's another little connection issue. Let's see if you can hear me. If this comes through to you. I can hear you. I can hear you all right. We'll just have to wait until the whole thing has unstuck itself, I think. But I think we should come back in a second. I'm not sure whether that's on your end or on my end. Okay, let's see. People, I was the only one who had issues with hearing Elisa. Can somebody confirm in the chat? Ah, okay. Then I have to apologize. Let me just quickly reload. I'll be back in one second. I will just reload quickly. I'll be back in one second. I guess Bianca, do you want to come on? Hi, it's Paola from OFE. I just assigned Bianca as a presenter so we can just move on. Also, thank you for the presentation. It was great. Can you all see my screen and hear me? Yes, your screen is loading. We hear you and we see you. Okay. Okay, it's still loading. It's actually taken quite some time. Okay. Hello again. Thanks Paola for dumping in. All right, I guess we're back on track. So I will just let Bianca take the word for her presentation. And thanks to Elisa for the very nice presentation. Thanks Elisa. Thanks everyone for the opportunity and it was very nice to be, I think I'm the last one, right? I think it's very nice to be after, to be in this session with Elisa because our work is really very connected. And I had in the last minute included one slide to make this connection a little bit more clear. So I am Bianca Trincan-Reich and you pronounce it very well. Thank you. And I am a graduate student, third year PhD student at NAU here in Arizona. I'm from Brazil. I'm an American. And I'm also a research assistant. And this work is a study that was published, accepted in transactions of software engineering. It's about the pots of gold in the end of the rainbow. And we studied what means being successful for open source contributors. Okay, how did we go until there? And then I'm coming to the connection with the comments that I'm seeing chat and with the Elisa awesome presentation. Open source is changing. Open source is not the same from 20 years ago. But until now there are some famous preconceptions that open source is only for developers. And why are those preconceptions still existing? It's because usually, usually not all, but usually the metrics and the recognition are based on code. So they are based on how much code you can produce, how much code you can commit. And this helps this preconception to still exist. Okay. However, the landscape has changed. And now the open source communities and projects have and need different roles there. This graph is from a previous study that we have investigated the many hidden roles that exist in open source software projects. So besides the project-centric roles and the developer here in green, we have many other roles. And the community is being such a diverse ecosystem that we need to start recognizing all the many roles that exist. So as you can see, the coder is part of, we categorize the roles into community-centric and project-centric ones. So the coder is inside the project-centric and together with him or her or they exist also the designer, the documenter, the release manager and project manager, and many other roles that are there to help the developer and help the community to develop and deliver a better software. But besides the project-centric roles, we have also found many community-centric roles that you have also, you have for example, the community founder when you create the new community and also the community manager. And the community manager is growing and growing because the ecosystem is also growing. So we can see that having different roles, people can have also different motivations and different career goals. We are going to a diverse set of contributors. And this is the new slide that I included because I wanted to talk a little bit about the study that we investigated, the different pathways that contributors follow until they achieve the success. So this study was based on interviews. We had interviewed 17 contributors that were speakers at the open-source conferences. And we selected them because it's one of the many, and I'm going to get there, the many ways to measure success, someone who is speaking at the conference. And then we interviewed 17 of them. And besides an inter-interview, we asked them to tell the story of their career until they are now, since the beginning until the point they are now. So those contributors included contributors from small projects, big projects, companies that include open-source areas like IBM and Microsoft, but also big and large communities as Linux Kernel. And they told us their career story. And we could understand there are many pathways until like here in the circle, until the way they are now. So the colors indicate if the role is related to not open-source, which is the orange, if the role includes some consuming open-source, which is the yellow, or if the role is already a contributor to open-source, which is the pink. So we could see that many of them started lurking open-source. We consolidated many kinds of lurking inside here in the paper. We have more details. But some of them went directly to be a community founder. Some of the other went directly to be a community manager. So there are many pathways to be successful, many pathways to start being a contributor. And we also categorized those roles into coder or non-coder. So we can understand the many pathways. In the paper, we also discussed the fluid movements between being coder and being non-coder, or both at the same time. And after that, we started to think, okay, we interviewed 17 contributors. They were speakers at open-source. They are meant to be considered successful from the point of view of their communities, at least. But to think of you being successful, what is success? What means being successful in open-source? Is that only like being a speaker at a conference or not? Okay. Then why did we start to investigate that? Because the success must be recognized like more than only being a speaker, more than only just the amount of code that we produce. And what means success? Then we are doing a lot of social science together with technical and open-source. And then the success is like achieving at any point of your life and your experiences with the professional results that you desire. I can understand success by one way. You can understand by another. And this will drive our decisions per life. This will drive our education choice, our professional choice and the motivations and also the community that we want to contribute and our decisions to stay or to move to another one. And this is the goal of this study, understanding what means being successful, not the success of the project, but being successful yourself by the point of view of open-source contributors. And what did we do? We started with 27 interviews and included the speakers of the conference but also maintainers of like well-known projects and people who are already in a core position in the community. So we interviewed 27 contributors. We did the open-coded qualitative analysis. Then we did a re-analysis using an existing model of career success from the psychology and human resources. And then we ran a survey with more 193 contributors from many, many and different communities. And those were like not chosen like any contributors in the beginning of career and in the end doesn't matter. And we asked it what means success to you. After that we did our analysis of the survey and some quantitative analysis like descriptive analysis because we didn't ask, we asked open questions. What is success to you? The person could answer the first screen from the top of the head but could not remember all. So we did descriptive analysis to understand how the different demographic considered success. And then we came to this model which is a model with four quadrants created by two dimensions. You can see the interpersonal and interpersonal dimensions is the relation of the meaning of success if it's only to me, it's only for yourself or it relates with the external world. It's something that is related to others. And the effect, the horizontal way, the effect in one side and the achievement effect are the subjective perceptions of success and success. And the achievement are the objective. What can be measured? This big square with the 10 regions, cooperation, recognition, perceived, this comes from the existing model and we could fit all the 26 definitions of success to these regions. So we are using the existing model from the social sciences to explain what means success to open source computers. And I'm going to each of them. So the first quadrant means interpersonal with effect. What means that? It's like the feelings, the subjective perceptions, external to myself. So it's subjective, it's perception but it also interacts with the external world. And then we could see many definitions of success related to cooperation, cooperate, work with others, which is the core from open source communities and open source contributors, right? So having contact, something to me is like having contacts in different communities, helping the sustainability, provide opportunities for my colleagues to grow, mentor and develop the team who is with me. So those were categorized inside the cooperation. Also the outreach, like providing perceiving contribution, serving society, impacting users, making people's life easier. So you could see that many definitions were not only for the only yourself but also interact with others and this is what made us to decide using this model which was very interesting to explain all those perspectives of success. And the recognition. The recognition was really, really cited by the contributors being recognized by other people, being famous, having my name well known, being awarded for my efforts. So for the second quadrant we go to the other side, we also still interpersonal, still the relation with external world but definitions of success that are more objective. They are measured. They are possible to be measured. So factual contribution was like, something to me is bringing as much contributions as I can, as much objective measured or selling products or services upon open source. I'm successful when I can do that. Also like performance. When people said success to me is being able to release a new version or a new release of the project every six months or every one year. Every three months it's like also like pharmacy being measured by the results and set the goals and achieve those goals. And advancement. Advancement is an interesting one which includes salary increase. Salary increases related to money but money was one interesting definition of success that could fit to two regions of services and I'm going to get there. So the first region that fits money was when people said being successful to me is having my salary going up earning more money. Being in the top level of a community hierarchy or being part of a famous community. Look, this is different from recognition because here the person says being part of a famous community like being part of UNOX kernel being part of Microsoft, being part of Ugo and the other is like being recognized by me from my efforts. Me being famous. So those were very dynamic and nuanced conceptions of success and here we have like also receiving job offers and choosing the next step of the career being able to choose the next steps to me is being successful. And here we are now going to talk about the interpersonal side. So it's the definitions of success that are related to yourself. So the self-developing when I learn new skills successful to me is being able learning and being able to join any type or any size of open source projects so be prepared to serve as a core member as a maintainer or any other core member of the community and creativity is like having making something extraordinary bringing new ideas bringing new yeah, new ideas making something innovative bringing innovation to the project. And then the last quadrant which is affect with interpersonal is the only yourself and the perceptions subjective perceptions of success. In the model we have two two regions which one is satisfaction. Being successful to me is being happy is belonging to a place is like being able to express myself have more friends is satisfaction and security brings the money back. So security here is when the person says being successful is leaving from open source is not only contributing but making my own sustain from open source you know so here we have the money concept again but here is representing security is representing using open source for your own sustain and not having this in parallel or as a volunteer so being successful for those people is making a living from open source okay and then what what can you do with those definitions so we we use that implications of the study many recommendations for open source communities who to support the different to see recognize and support the different definitions of success so you can for example for people who who aim to be recognized by others you need to create like meritocracy and hours and badges for different contributions for those who answer questions who discuss issues who are building your content for training and then you are going to attend also the creativity because here people who understand that success success is being creative and you recognize that you are going to help them to create new content for training and you are going also to help another area of self-development providing like learning models, manuals and skill specific mentoring not only technical and the recognition here in the up can engage more contributions so you can help people to provide more factor contribution to the project so we understand that those those implications and those suggestions can be integrated in a way to provide what many different contributors want as the success so the success is a concept this is my last slide the success is a concept that is once it is multi-fested as you can see recognition money can fit in different regions depending on what it means for the person and can go well beyond only being a core member or a mentor at a conference or a motivator we need to recognize the different contributions and the dominant view that successful open source contributors is only hackers is not adequate anymore even the point of view of coders because when we segmented the analysis we could see that even coders consider many other definitions of success than only being the motivator and the perceptions of success they can mean future career goals so if a person has the motivation has the perceptions of success of making a living from open source and this person is not making this living in the community that is contributing today there is a big risk for this person to go to another community that offers this so we can use those perceptions of success to support the retention and avoid contributors to live and this is my next my next idea of research I am now as a next steps I am researching the connections between the perceptions of success the motivations the sense of belonging and the challenges being faced and the intention to live thank you very interesting again very I really had to make sure my webcam doesn't work so you will only hear me for one second I really had to make sure that I keep my attention a very dense presentation but full of information I am wondering there is a slide we can make it available there was a big discussion now of course happening in the chat around what are the right steps I am wondering if you have some recommendations that you would like to share because it is a very very good analysis very comprehensive analysis but I am wondering if you also have some thoughts on what could do to tackle it I think the first and more I don't know if the more important but the more important is to recognize that people can want more than being the mountain and then when you recognize that you can understand in your community what means success for them okay some suggestions they are not so complicated and they can help to retain many people as they were more cited so making explicit here in the right making explicit criterion rules for promotion is very important because when people want to get up to keep the top level in the community if they don't know how to get there they can be demotivated so this is one part of this side that is usually not transparent in the community how can you become a core member how can you only be known how can you get there and this can be connected to the recognition because you can create a pathway to the core member oh you need to keep these and that steps and also create recognitions for the different steps you know so I think the two more important things here is like making explicit the rules and also recognising the many different rules not only the coder I cannot hear you I am on mute everybody has to make that mistake once it's very interesting and a reflection maybe from my side just from reading a bit about this book The Rise of the Meritocracy because they are in that book and it speaks of course in a more general context but it's still I think quite interesting because they are in the book he speaks about a part of society that is not being let's say appreciated not being valued based on some definition of value you know one can make a decision on how that is being defined and of course it leads to a lot of frustration leads to a lot of frustration as you have also now mentioned in regard to open source and broader reflection but you know maybe today we also see something like that in broader society a lot of frustration on parts of society that feel they are being taken out of decision making process they are not being heard and are not feeling like they are part of a desirable part of society people in this room are very likely not part of that group considering what we are doing here but the impact on society that that has I think is quite significant and quite relevant broader reflection but I did have to think about it now that you talked about it and it is quite it seems probably difficult to argue that it is not a little bit dangerous to cite it I am thinking if people want to come in I can see there is a lot of people not typing in the chat it is also completely fine for you to unmute yourself and turn your camera on if you want to talk that is completely welcome to we don't only have to have this slightly asynchronous conversation between here and the chat otherwise also if there is any further questions maybe just give people a second to see if there is any further questions how it is quite early for you still right because you are in Texas I am in Arizona Arizona I asked the question before 838 okay that is alright at least you are fresh everybody else is already there but I was so stressed about getting up and being ready I have a small daughter here so I was like be there don't be quiet and don't I mean I will provide the slides to Silvan and he can share with all of you okay and then I have the links the link for the two studies so the first was the hidden figures the study about the pathways the pathways to success so we call it the hidden figures because making like the connection with the movie I don't know if you all watch it it's a movie hidden figures a movie about the women in NASA who are black women in NASA who were not recognized by their work and we thought about using this yeah I love hidden figures too and we used this title in the first paper because we wanted to make awareness about the many roles that exist in open source and the many pathways to success and then the second slide the second study which is this one the pots of code is like okay we preconceived that such successes like being in the spot but what else and this what became the pots of code my PhD student is related to increasing diversity in open source the diversity aspect that I'm focusing in is gender and women as the super population but increasing diversity as a whole and I'm using those human factors to understand what makes people stay and remain or leave so this is going to be my next study connecting the sense of belonging with all those human factors and the intention to leave to try to find some connections and yeah that's really fascinating and actually now that you say hidden figures the pandemic also gives one a reflection on hidden figures let's say jobs or activities that if you think about what do politicians talk about we need more software engineers stuff that we also talk about and then something happens and suddenly you realize that some of the jobs that might not be always in the public eye and not always be a priority are extremely important so it's also a very good reminder if I would choose like a next step I'm sorry to interrupt no no no go ahead just like having all those people here if you have like students or yourself want to I think that is not very explored yet in research is how to recognize the different kinds of contributions you know there are some Stack Overflow recognize a little bit like those who answer questions with beds but it's not easy why is not easy because it's not easy to measure the different types of contributions that one does when it's not related to code I think I read something when Elisa was speaking measuring code in GitHub is like almost lazy is like the easy one I can measure the commits I can measure how many patches you submit how many patches you can commit but what else and the people who work with community managers they work hard and how can you measure everything you need to mind blogs what are you going to mind to measure that it's not easy this can be our own research only this part recognizing the different contributions and dividing by different types of contributions because it's hard to mind it's like many many parts of contributions not only in GitHub it can be implications for researchers too yeah absolutely and Vittorio interesting what you write on meritocracy complex and dangerous and specifically on the point of rejecting the authority of science it's very interesting because I'm not an anti-vaxxer I believe in science but I also wonder if there's an approach that can marry let's say the critical view of how these values are being assigned but at the same time not being unscientific we're discussing for example this this presentation I cannot call unscientific you have to actually work yourself through all of these graphs I'm sure that's what you mean obviously Vittorio but I do think it's worth a discussion on where what how such a model can look like I see Johanna actually now asked a question to you Bianca have you worked with the chaos community in implementing exploring metrics related to these different we didn't start working on that but I yes I know the chaos I have introduced it and two or three of the interviewees are core members of chaos too so I didn't start working on measures for that because I had to like take care of the other part of the PhD but yeah I know the chaos and I really like this community and the work they do on metrics they are also working on something to recognize different kind of contributions