 12205 welcome everybody. Thank you very much for joining our session. It's a pleasure to see you. Thank you for your interest in Linux foundation research My name is Hilary Carter. I lead the research function in partnership with Steve Hendrick or VP of research Who will join me in in presenting how we contribute to the open source ecosystem? how we go about creating our research insights and how you can get involved so without further ado our Our goal at Linux Foundation research is to investigate the impact of open source projects Open source software open hardware open standards and open data and try to describe Through data how our projects are making an impact what the trends are and I use that data to Importantly inform decision-making at the enterprise level at the project level and so on allowing our communities to leverage data to inform how they allocate budgets where they Can dedicate resources and activations To try to close some of the gaps that research reports identify is it are there issues with Adoption other issues with Community Sentiment and diversity equity and inclusion issues, you know, where are the gaps where the challenges and use research reports to try to fix them So if you're not yet familiar with us, this is our home Linux foundation org slash research We were founded in 2021 on a tradition of research excellence at Linux foundation as early as 2018 publishing the Linux kernel history report and 2020 the FOSS contributor survey which was in partnership with the Harvard laboratory of innovation science and These reports are well received and I think the decision of the time was let's keep a good thing going Let's do more of this and add net new value to our community and create something together that everyone can benefit from and So we generate insights through survey work qualitative interviews and analyzing data sets from SCA vendors, for example GitHub repositories or SCA vendor data and Produce research deliverables that will Describe what those trends and gaps are today. We've more than 30 unique reports published so it's been a very busy two years and a rewarding Effort to be a part of because it is so value additive The way we go about creating research is using frameworks of analysis We have industry specific research reports to date. We've published projects in energy An education and training if you heard Jim Zemlin's keynote this morning. He referenced our 2023 jobs report That's actually called the tech talent survey beg your pardon. It was called the jobs report for years this year We're focusing on tech talent from the Enterprise point of view and how they go about seeking and retaining qualified tech talent. We've done reports in financial services And as well as media and entertainment the film That motion picture industry we've done a lot of reports along technological Lines that are tech horizontals and in blockchain and cloud AI and data cybersecurity Phenops and and more as well as reports that are Quite broad. We call them ecosystem-wide reports issues and to an extent cyber security could be Called an ecosystem report because it affects all industries and all technologies But looking at leadership and governance the role of the open source program office Diversity equity inclusion looking at the developer experience and the maintainer experience and issues like climate change We also fairly recently launched a new framework. That's looking at open source dynamics along geographic lines What are the differences and in communities at different levels? We can do city-based studies country studies Continental studies as we've done initially with Europe and we'll be doing more of these reports this year and Steve will tell you about some of that work That's in the fields presently Our deliverables include Reports as they're free. They're they're published under creative commons license. So you may use them You can publish them on your own website. You can share them broadly But because of the nature of our research and and a lot of involvement qualitatively They are published under non derivative license We publish blogs that talk about the whys and wherefores and the high-level findings and often as we're doing today have a live discussion at an event or will be part of a digital webinar all of our data is available on data.world and This is so that our communities can dig into the findings can conduct their own research Maybe come up with a different perspective that we didn't consider in our analysis but feel free to dig into any of our data sets on data.world and Take advantage none of the data that we publish is There's no personally identifiable information. That's a core ethic at LF research is to keep our respondents a privacy protected So so far Many of our projects have been supported by individual member organizations including VMware Intel Futureway POM NFT studio and others and We've also received funding from project communities like hyper ledger foundation cloud native computing foundation FinTech and open source foundation So that is how we are funded if you are interested in helping us We'll all describe ways that you can get involved a little bit later on But there's no topic that we couldn't explore At least giving it some some careful careful consideration about our ability to be successful in a research project But as I mentioned previously we've published lots of great reports in the areas referenced here on the slide Now I'll invite my colleague Steve Hendrick to explore the methodology That we apply in our quantitative Research process Steve come on up. Sure Okay, thanks, Hillary I am so you can turn that one off So if you wanted to know a little bit about how the sausage is made This is the right place to come Yeah Okay, so I want to talk a little bit about the research process that we in we utilize here in Linux foundation so It's actually pretty simple. There's a planning process we go through We plan out what we want to do with the research itself. What are our objectives? What are the headlines that we want to see come out of the research if possible? And that shapes development of a survey instrument in that survey instrument Is sort of the keys to the kingdom because the survey instrument is actually what you're going to throw out in the field And ask the questions you're going to ask And those questions to find everything that follows so the quality of that research instrument is Absolutely critical from the standpoint of being able to get collect data in the right way And be able to do the analysis you wanted to do and then you don't really know what the findings are going to be until you get to the analysis But dig deep in the analysis to find some really Come you know compelling insights from the research so there's four slides here on through the research process I'm going to show you the kind of the money slide after that if you have any questions You might as well just ask them as we go So this is research planning so we talked to stakeholders for the project, you know discuss their needs objectives We put a proposal together which has a lot of information about how we're going to do this what the objectives are Planning deliverables timing pricing all that We've revised the price the proposal because often they'll say We want it faster. We want it cheaper And then we'll put together a project schedule So research execution. I'm going to go that into that more detail here But there is an opportunity here to talk to subject matter exports if we need to before we do the project to educate us If need be from the standpoint of some of the more nuanced Important issues that we have to look into we developed something called a survey reference model Which is in fact the questionnaire for the survey plus a lot of other information that you'll see in a moment Then once we get that we present that once we get it approved by the stakeholders We go and we do the program we do the testing and put it out into the field Once it's out in the field We monitor it we do data quality tests periodically To make sure that we're collecting the data and it's coming in the way we would expect and there's nothing wrong with the survey instrument itself And we do sort of periodic analysis of the data just to see what's going on anyway So from all of that we create a whole bunch of results Massive results deck we go into key findings from that Will do SME interviews again at the end if there are particular topics that have surfaced in the research That we want to know more about and because ultimately we'll be writing a report about this and we want to be able to have Not only you know get the answers to the key questions and show the key insights that we found But also dig into it With relevant Commentary from people who are experts in the area because we can't be experts in everything that we do Okay, and then it's a delivery process In terms of how we write the report massive review cycles with stakeholders And then we finally get to the point of handing it off to the creative team And they make it look beautiful and we put all the data out on data.world So that's kind of a short story on how we do whether our research process is And it's a really nice one because that enables us if there's four of us who actually do research Hands-on with this particular execution model. And if anybody for instance, you know Has to stop working and run off to do some other firefighting anybody can pick it up without even a Any kind of loss of time because we have the same particular process that all of us follow and All the same kind of terminology and nomenclature we use to describe what we're all doing. So Anyway, this starts with the survey reference model. We do use survey monkey as a data collection tool I'm not a big fan of it for analysis, but that's okay because we have market site Which is a fabulous and analytical tool for doing frequencies and cross tabs some very basic kinds of stuff But it does talk about confidence limits In all of the analysis that market site generates and it's the only tool I know of that will automatically generate graphics in batch mode Without you know having to click you to death for every single slide that you want to create so that's the The way we actually execute on research Now let me talk about Sort of one thing that I kind of keeps me up at night and I'm always thinking about which is how can we improve our research quality And it really comes down to two things we can improve the way we do the actual research the research process And we can also improve the the quality of the people that we talked Because respondents are the ones that this information is coming from if you talk to the right people you get good information Talk to the wrong people don't So there are ways to deal with both of these. So let me just talk a little bit about What we can do internally from a research process standpoint to get higher quality One of the first things we can do is stop making the surveys so long You know we're famous for doing like 50 to 60 question surveys and we torture people and I most apologize for that We've got a we've got to stop doing that because it's you know 20 25 minutes in the surveys Just not what people are looking for so more frequent surveys And certainly shorter surveys is sort of the order of the day There's also from this internal perspective a framework for asking questions in a more somewhat standardized way Is really important because that will allow us then to be able to Compare results Much more easily and have a much more compelling way that we can describe the information because it's not just a framework for how you Ask the question. It's also a framework for how you're going to analyze the question when the data comes back And so from the standpoint of looking at current state future state sort of issues when it comes to our questions Or as I'd say here in the slide Adoption of technology penetration rates how to do growth share matrices and compelling ways correlation between questions and samples Forecasts and confidence limits around forecasts all of that can really benefit from having a good solid framework from the standpoint of How you want to ask the questions and how you're going to be able to analyze them And so my my objective here is to kind of move us from this idea of having Delivering today, you know data and perspective, which I think is sort of step one to really being able to get to trusted insights And we know we can get deep insights by doing a lot of cross-tabbing and deep driving deeper into variables to understand What's going on in a segmented way? But trusted insights is a more challenging Issue and I'll talk to you about how we're going to get there, but from the standpoint of this whole issue of research If you have better data quality You're going to have Information data itself that allows you to make better decision-making That's going to have an impact on how your what your strategies are going to be and ultimately Give you a lot more industry relevance from the standpoint of how you can do your analysis because with improved data quality, especially over time then it gives you a ability to do some sort of different kinds of longitude analysis To see how change things are changing over time, which is really interesting so what I sort of figured out not too difficultly is that when it comes to research quality one of the biggest and hardest things to address is the quality of the respondent that you're talking to and So I've been doing surveys now for 30 years. I've done close to 150 surveys, I guess in my career and Respondent quality has always been an issue so How are we going to deal with that? well, one way is that we can engage more extensively with the LF community and We do we try to do that already But it can be very challenging because people are overwhelmed with surveys these days You know your bank sends you surveys everybody you fear your grocery store sends you a survey Everybody wants to know your perspective So people are tired of taking surveys So we have to do is we have to be able to rise above that So we have to have some way to be able to actively engage people and essentially Incent them to want to do work with us and we like the idea of being able to incent them I think it's a win for them and if they have a lot of particular expertise. They want to share with us It's a win for us So anyway The LF community is quite large. There's what about a hundred one point five million people in the community 1.3 1.5 Those are people we know about the community Okay, okay, well three million in the contact list Okay, right and seven hundred thousand where we know a lot about them. Yes So there's quite a few people out there So we would love to be able to engage with the people that want to work with us on quantitative research So it's not hard for us to reach out to them, but but we have to have a story to tell so Things we have to keep in mind as we do this is that we certainly want to make sure that we don't we observe all the privacy policies so that people are you know are Not concerned about Anything that any information that they share going to the wrong place So that's not that hard for us to do. We're already pretty good at it now We also need to have meaningful incentives to be able to say, you know, if you're going to help us We need to help you a bit And I'll talk to you about those in just a second in terms of what my ideas are And ultimately the last bullet here. We want to grow our LF research panel That's comprised of the community Who are interested in sharing information and knowledge with us And that's going to help us improve the quality of the research at the end of the day So that's the objective From the standpoint of the benefits of the panel to somebody who's a part of the panel We know first of all It's going to avoid people getting Request to take surveys that don't want to take a survey And so the the level of noise coming out of Linux Foundation in terms of asking people to participate in Monetary research will be able to go down because we'll have a panel of people who are interested in taking surveys So that will help us also avoid pestering people who are really not interested And we'll make sure that if you're out there, you'll be able to communicate your particular interests about opting in and opting out But we also need some sort of a mecca sort of a really compelling I hate to use the term economic, but it kind of comes down to that incentive here We already provide discounts on courses to people that take our surveys are pretty generous discounts And we can extend that across everything that Linux Foundation does so it's very focused on trading and certification right now But I think we can kind of get it across everything in our portfolio that we do We also probably can put together a way to do donations to charity If that's something you want, but the idea of having credits to a participant That's part of the panel from the standpoint of being able to then spend those credits on for instance Coming to events or taking classes or swag I think would be in compelling in some some of the right ways from the standpoint of incenting people as opposed to You know, you know here's a drawing for an iPad or something which I think would incent all the wrong people so So anyway, once we have a panel in place, there's lots of things we can do like a longitudinal research Going to the exact people that answered questions year after year Which is true longitudinal research as opposed to quasi where you go to different people year to year Alright, as far as the timeline for it, you know, we've already built the requirements document for what we need to do here We're in discussions with the technical people at LF about how LFX and CDP functionality is going to have to change to be able to support all of this And we're looking about a beta release last quarter of this year and then a real full-fledged production release beginning next year Alright, any questions on that? Alright, as far as upcoming research goes, State of Tickdown, you've already seen that came out today Very exciting stuff So I was driving the research for that particular project And I was very interested to sort of see how did 2022 influence what was going to go on in 2023 And so 59% of the people in the survey were concerned about sort of what the financial climate was looking like in 2022 And so the 60% of people that were hiring in 2022 when we did this last year went down to about 44% this year Which is actually not much of a slide The people that were considering freezing hiring went from 20% last year to 30% this year, so that bumped up But the really interesting thing was that the people that really were thinking about reducing their technical hiring Went from 20% in 2022 down to simply 8% in 2023 So despite a lot of the concerns about the economy, from the standpoint of technical staff, you know, hiring There was some but not nearly as much as I expected and there's a lot of optimism about what's going to happen as we go into 2024 So very cool stuff Upskilling was the other thing that came out of this report We always look at upskilling because, you know, we have a huge training and certification business But when we looked at it in this report, I took the opportunity to weave it into all different kinds of dimensionality from the standpoint of the questions that we were asking Because it was a pretty big survey going against my rule of make them shorter And sort of unilaterally, we found a ton of interest in upskilling these days, more so than even a college education And certainly I think that reflects the increasing difficulty that organizations are having to find the right skills for the rules that they have internally And so if you can't find people on the outside, you either can wait and take a year and a half or so to find somebody You can get a consultant or you can upskill And upskilling doesn't work in all situations, but it works a lot of the time And upskilling was I think the number one response in many different ways that we asked this question about upskilling So very exciting for us to hear that from this study Henry Jesbro is a professor at University of Berkeley And we reached out, he's a pioneer in looking at the economics of open source And has already written one or two books on the topic that are revered in the open source space From the standpoint of how open source drives innovation in the economy And it's just, it's fascinating stuff So we had him go out and do some more specific measurements around what's going on in open source And some pretty cool findings I think this was the infographic that came out of the report And so we all know that open source costs less I think on the bottom left-hand panel you see here that From the standpoint of being all in on whether you're using open source or some other kind of solution Whether you build it yourself or buy something proprietary or some combination of that and consultants That, you know, open source was about a quarter of the expense of other solutions And in some cases I'm sure it was higher I mean lower in terms of open source being less expensive than just, you know Everything else being four times the cost of open source But so this is a relatively conservative view but powerful nevertheless It was empirically based and it was pretty good data So that was great to see We had an OSPO survey that I think has been announced today as well This is something that we do every year And there was a this year's survey was as I'm sort of was relatively is a big survey I guess I have to admit But you don't have to do all the sessions depending upon how you answer the questions So hopefully it won't be a struggle for people But one of the really exciting consequences of having it being a larger survey Is that we can dig deeper into all of those different issues that are interesting to people About that are want to know about how OSPOs will make a difference inside of their organization So from the standpoint of the size of the OSPO responsibilities The impact on economics benefits the challenges of OSPOs And what the characteristics are of maturing OSPOs all of that is covered in the survey So I encourage you to go out and take the survey It would be great to hear from you Then we have another survey that was I guess went out in April But it's also being talked up here at this event And this is our global spotlight world of open source This is a worldwide study Most of ours are in fact I think OSPO is as well and everything else we've been talking about But this one is really fabulous Because it's all about how organizations are using open source How they're contributing to open source What kinds of open source they use from a technology standpoint And the value of open source to their organization So really really cool set of questions that we're asking there And back to Hillary Steve? Amazing One of the things that I love about this partnership at LF Research with Steve Is that he brings years of quantitative experience to the project My background prior to joining the LF was creating a lot of mostly qualitative reports And we've carried on our respective traditions and blended them And what I think is so important about the qualitative process Is the opportunity to engage our community Subject matter experts to give them an opportunity to have their perspectives be heard To go on record or to contribute anonymously And for us to meet the names and the faces Some of the challenges with surveys is that the data is anonymous But we do have opportunities to build personal contacts and networks And really understand who are the leaders in these respective studies that we do So one of the studies that we've published based on qualitative work Is the business value of the OSPO And to engage with so many different partner communities including VMware Thank you Suzanne for being a supporter of this study Conducting numerous interviews with OSPO leaders And here's some of the findings Most open source program offices tend to originate to deal with a problem Whether that was an inappropriate use of pieces of software Not complying with licensing or another kind of challenge Other challenges mentioned include creating an open source culture within an organization Educating stakeholders across the organization about the value of open source And how do you measure success for your open source efforts And for the effort of establishing an OSPO OSPO KPIs that are most widely implemented include How do you sustain growing numbers of contributors to projects And how do you measure the success of those projects internally and at the community level And what can OSPOS do beyond just simply managing consumption or usage of open source code Well, how do you take an OSPO and create an open source strategy Something that creates real business value as part of an overall open source sustainability vision Steve mentioned our working with Henry Chesbro the so called father of open innovation We had the wonderful opportunity to conduct qualitative round table with Henry Chesbro And a group of academics practitioners and people from enterprise at the open innovation conference in Eindhoven, Netherlands Discussing what we believed our hypothesis was that open source is a lever to be applied especially during an economic downturn Mostly because of the cost low cost innovation opportunities that open source creates And so these are the findings for that discussion that yes it is widely accepted to be low cost high quality alternative Yes open source has this tremendous opportunity that is unrealized right now to advance public sector initiatives So that the governments around the world can become model users and the most cost effective pathway To creating digital citizen centric services is often through open source software And to create opportunities for digital sovereignty not being dependent upon vendors from other regions This is a sentiment that is common in Europe wanting to achieve a European identity in their innovation initiatives And what are the barriers? The identifying lack of leadership and regulatory barriers that prevent some organizations from having their team members contribute to open source projects That is a limitation and something that we need to help work together across the community to change those regulatory barriers We published a project on open source wallets making the case for why the world needs an open source digital wallet We need new infrastructure that will empower individuals, organizations, governments to transact in a more trusted capacity And in almost a limitless number of use cases whether it relates to digital identity or custody of currency or custody of keys Trusted credentials, security of tokens like our real estate deeds One of the challenges in Puerto Rico following Hurricane Maria was that homeowners could not prove that they were the beneficial owner of their property And therefore they didn't qualify for assistance from FEMA for months and months and months These are the problems that are solved through digital wallets and tokenization and the open wallet foundation is helping advance the understanding of what are digital wallets, what problems do they solve What are the challenges around existing wallet infrastructure and the widespread protocols that are issuing and storing and exchanging digital assets presently And how do we reconcile a very fragmented environment when it comes to digital asset protocols today And how do we enable innovators to create their own wallets and not rely on one or two vendors So exciting opportunities If you read one report this year I would say read this one, this is really interesting This is how leaders across open source communities are overcoming challenges of fragmentation Whether that fragmentation is linguistic, whether it is technological or other How can we increase collaboration, how do we work together to solve some common challenges And what this report revealed was that fragmentation is real We are fragmented across open source communities in terms of language, culture, geopolitics And that impairs our ability to be as effective and as innovative as we might otherwise be Techno nationalism does pose a threat to open source collaboration And we need to find pathways to overcome those geopolitical tensions and reinforce that we are in an ideal world all one global community that wants to make code more robust and more effective Leaders also said we need to better align some of our projects that share common priorities, sustainability being one of them How do we avoid duplication? At the Linux Foundation alone we have four different projects that are working on digital identity That's a kind of fragmentation into itself The good news or fragmentation is concerned is that the longer the technology has been around the less fragmented the environment So if you think of COVID everyone was scrambling to create a COVID notification app at a time of crisis And there were solutions here and solutions there and a whole wide variety of solutions So new technologies have lots and lots of fragmentation Over time the winners emerge and the market essentially picks the winners We're also doing work to better understand challenges that our maintainers face In March of this year we published a report, Census 2 in collaboration with Harvard's Laboratory of Innovation Science Identifying the most widely used open source application libraries Why did we do this? Well if we understand what is the most widely used software we can prioritize our efforts to try to make it more secure In identifying those packages we try to identify the maintainers of those packages And reach out to them and have conversations with them and say what challenges are you facing? How can we help? What tooling do you need? Jim Zemlin mentioned the need for more time to work on projects But these are really vital conversations and research provides a mechanism for stakeholders like maintainers to be heard And to inspire the community to get involved to try to enhance their experiences Finally I'd like to just invite all of you to stay in touch with LF Research To sign up to receive our newsletters, to receive our surveys right in your inbox We'll, as Steve mentioned, work on creating a panel community where we can engage as effectively as possible And try to minimize some of the challenges for research like survey fatigue And provide appropriate incentives and engagement opportunities You may have noticed in our surveys and in the research reports how many different organizations have come forward to distribute surveys To sponsor surveys and to get involved And it is a most enriching and rewarding activity to collaborate with so many different organizations on unique research projects And so this is an open invitation to get involved If you are interested in sponsoring a piece of research, let's have a conversation Very briefly the benefits are brand visibility, having the opportunity to conduct a very specific study and answer a pestering question Contributing to findings and addressing issues as well as having early access to those findings So we will make the experience as value additive and as fiscally rewarding And invite you to consider LF Research projects as you form your budgets throughout 2024 And join us, thank you I would like to invite questions for Steve or me And we have one minute, thank you Yes it is, it is available on data.world in Linux foundation projects and the title of that report is called census 2 Identifying the most widely used application libraries, I think that's the subtitle Data.world has the complete list of the data sets And that data came from three SCA vendor companies, Sneak, Fossa, and Synopsys So in terms of contributing, other ways that can get involved in LF Research, if you have data, we are going to update census 2 We would love to work with more SCA vendors, we would love to get another data set Because all we can do is provide a measure, the more data sets we have, the better that measure can be And we will respect your organization's privacy in the process Thanks for that question It is 245 and I believe that wraps our session, thanks everybody