 So what we're gonna talk about in this last session is social and societal issues. And what that really means is think about everything we've heard about already, all these advancements, all these innovations, everything, all these great things that are happening. How do we ensure that the benefits of those things are not just benefiting business, but also benefiting society as a whole? And to kick things off, we've got a really great speaker who is going to talk to us about culture and specifically about when culture intersects with technology like artificial intelligence. So without further ado, I've said probably too much already because I don't wanna steal a thunder. Charlie Sol from Culture X. Thank you, Bob. I'd like to start off today with a question for everyone. It's kind of a dumb question, but please bear with me. I would like to see by show of hands who in this audience thinks that corporate culture matters. Okay, every single hand, let that be noted. Okay, I've got a different question, maybe a little bit more of a smarter question now. And this one links back to a research study. So for this question, imagine every conceivable driver of value creation in a firm. So the choice of CEO, the strategy, the strength of the supply chain, the product quality, the strength of competition, any conceivable factor that drives value creation in a firm. Now again, by show of hands, who would place corporate culture in the top three? Okay, fewer hands, but still quite a few hands. I would say maybe 40, 50%. Well, the reason why I asked this question is because one of my favorite studies about culture got 1400 CEOs and CFOs, not CHROs, CEOs and CFOs, a fairly large organizations. So the average or the median company that sample had revenue of 100 million, the quarter had revenue of more than a billion. What's cool about this survey is they got all the CEOs and CFOs in one place. And the study at Duke asked them this question, out of any conceivable factor, do you place culture in the top three? And this is how many of the CEOs and CFOs said yes, it's in the top three, 54%. And by the way, if you ask them, is it in the top five, 89% say yes. So we have evidence, everyone says culture matters, CEOs, CFOs think it matters. How much value does culture create financially? One of the best answers we have for that is by a professor named Alex Edmonds, who did this great study, does the stock market fully value in tangibles. It was a very rigorous study published in the Journal of Financial Economics. What it found is that for large publicly traded firms, a strong culture as measured by a strong employee experience is worth approximately 20% premium on market cap. So for large organizations, culture is a multi-billion dollar phenomenon. It can be an asset or it can be liability, but it's worth billions of dollars. Despite this, culture is not where it needs to be. So in the same study, the Duke study, the CEOs and the CFOs were also asked, is your culture where it needs to be? They had just said it was very important to them, but is it where it needs to be? 84% of them said no, it's not. But culture doesn't just affect CEOs and CFOs, it affects all the employees who are embedded in the culture. So what if you ask tens of millions of employees around the world, extrapolating to the billions of employees across the world, how would you rate your culture? Well, thanks to Glassdoor, this new platform, there's an opportunity for every employee on Glassdoor to rate their culture on a scale of one to five. Does anyone wanna guess the average culture rating, the average rating of culture on Glassdoor in terms of stars? Anyone wanna shout it out? One star out of five? All right, this would be a crisis. I would have really my word cut out for me if it were one star out of five. It's not that bad, but it's bad. On average, you're rating your culture 3.5 stars out of five. Now, if you're on Yelp and you're looking for a restaurant to eat at, would you eat in a 3.5 star restaurant? I would not. If you're waiting for an Uber and you see the Uber driver is 3.5 stars, what do you get in that car? Definitely not. And if you see a culture on Glassdoor that's rated 3.5 stars, what do you wanna work in that culture? I would not. So as far as we can tell for billions of employees across the world, they are living in a culture of mediocrity. They are spending the lion's share of their waking hours embedded in a culture of mediocrity. And this is what I've spent the past 10 years researching. How do we fix this? What's going on here? And I'll share some of those results with you now. These are the findings of one of the studies we did. Basically, my team and I have been researching corporate culture using these new AI techniques for the past 10 years. One of the questions we asked was what is the key to cultural success? Because it's this interesting phenomenon where if you say, does culture matter? Yeah, everyone raises their hand. CEOs and CFOs will say, yes, it matters. But it's not in the right place. It's 3.5 stars across the world. So what's going on here? Well, what actually drives cultural success and failure? So to investigate this, what we did is we looked at big data. We looked at nearly 2 million Glassdoor reviews from 1600 of the largest firms in America. And then we segmented them into culture champions, the top 10 percentile cultures, the bottom decile cultures, and cultures in the middle. And how we did that was a mixture of their performance on things that matter for the employee experience. So things like respect, integrity, diversity, and inclusion. And also things that have a direct, more direct impact on value creation. So things like innovation, agility, performance, cross-unit collaboration. And we did this all basically with free text. We analyzed about 2 million free text Glassdoor reviews. It's an interesting approach. When you read one employee's right on Glassdoor, they go into detail about hundreds of different topics and you can measure hundreds of topics with the same review. So after we had segmented the companies into the culture champions and the culture laggards, the bottom decile, what we did is we then investigated what's special about these two groups? So what do the culture champions get right culturally? And what did the culture laggards get wrong culturally all compared to their industries? And to investigate this, we looked at hundreds of different topics. So anything in the employee experience and culture ranging from psychological safety to how pet-friendly the environment was. Basically anything that employees will ever speak about on Glassdoor. And what is very interesting, but we were surprised by, is that the exact thing that the culture champions were most likely to get right is basically the same thing that the culture laggards were most likely to get wrong. So the culture champions for this topic, which I'll tell you what it is in a second, they're nearly a full standard deviation higher than their industry on average. And the culture laggards are more than a full standard deviation lower than their industry on average. So what this suggests is a strong driving relationship between this topic and cultural success. So what is this topic? It's a topic called Voice of the Employee. This is also known as the listening topic, the cultural measurement topic. Basically what it is, is an organization's ability to understand the voice of the employee across a large organization, across all these different parts of, in many cases, a multinational organization to listen to that and incorporate that information into their decision-making capabilities. So as far as we can tell, this is the key to cultural success. And even if you think that other things matter for culture, yes, they definitely do. It's not as simple as this is the only thing that matters, but this is what our research suggests is perhaps the single most important. And it also stands to reason, if you're trying to improve a culture and create a successful culture, that it's difficult to manage culture unless you can measure culture. So this is the cultural measurement. So from a few different directions, what we're finding is that the ability to listen to employees successfully and to measure culture effectively is critical to cultural success. So, hmm, well that's interesting. If that's the case, then how come all these cultures that are in such a bad situation? I'm 84% of CEOs and CFOs say the culture's not where it needs to be. Because every organization right now is doing some kind of cultural measurement, right? Every large organization is doing some kind of employee listening metric, right? Well, maybe it's worth examining how organizations currently measure their culture and listen to employees. Does anyone know by far the most common systematic large-scale way a large or even a medium-sized organization listens to their employees today? Engagement surveys, yes. Okay, so these engagement surveys, they look something like this, right? You have, in this case, nine questions, but in many cases these engagement surveys are longer. Typically they're often 40 questions, they can be as long as 100 questions. I saw one that was 120 questions. And respondents are asked to either strongly agree or strongly disagree, or maybe they're in the middle with these dozens of questions. This repetitive survey they're being asked pretty much is the same thing all the time. Okay, well, maybe this works, right? So if employees answer this survey as it's described here, this wouldn't be a terrible way to measure the culture. So how leaders often think that employees answer these long surveys that they're pending on for all this cultural insight about their organization is the question one, yeah, I agree with that. Question two, I disagree. Question three, I'm kind of on the fence, but actually I disagree. Question four, let me think for a second, strongly agree. And you might think that employees are paying close attention to these surveys and there's gonna be a lot of variance in their answers. So you're gonna be getting this relatively rich cultural data. There's still maybe better ways to do cultural measurement than this, but maybe this is a pretty good way. Okay, there's one problem. This is not how employees answer surveys and I can guarantee you unless your survey is taking drastic steps to reduce this, here is how employees are gonna answer surveys in your specific organization. They just go like this. In academic literature it's called straight lining. They just go in a straight line. I mean, there's a little bit of variation, but it's basically just a straight line and they're answering pretty much every single question with the same answer, even though the questions are normally about very different things. So employees really aren't paying very much attention to these surveys. They're not giving very rich cultural data. It's hard to get actionable cultural insights if you're gathering data with this mechanism. To quantify it, we did a study of 770,000 liquor scale responses. The typical respondent in a survey and along the liquor scale survey is gonna answer 89% of all questions with the same two answers and 67% with the same answer. So it's not quite what they're doing is just going through and answering every single question with the same answer, even though more than 10% of them do do that, but it's pretty close. It's called autopilot, it's called straight lining and it really makes it hard to get the information that you need to measure a culture and improve a culture. So at this point, I'll often show this to people. I love talking about this. I'll talk about this all day, every day in case you can't tell and I'll show this to people and at this point in the conversation, they normally say, okay, yes, I get it. I understand that this is a problem, but my question for you is why do you care so much? Why are you so passionate about this? And my answer for that, if you've been following the narrative of this discussion is because culture really matters. Culture is something that billions of employees are embedded in every single day. They're spending the majority of their waking hours embedded in the culture and it's gonna have a huge impact on things like happiness or sadness, fulfillment or boredom, even joy or suffering as we'll see in a couple of slides. So culture really matters. The key driver, what determines whether the culture is successful or not, at least one of them appears to be cultural measurement and the way almost everyone is doing cultural measurement today is fundamentally broken. So this is bad news. Not just for the people in HR who are doing the survey, it's bad news for the employees whose cultures are not being improved because their leaders don't understand what's going on in the culture. They can't do an effective cultural measurement. In terms of other actionable takeaways from this talk, I often get asked, okay, so I wanna improve my culture, what do you need? And we've written a few articles about this. We'll link to all of the articles at the end. Culture change is complex, yes, there are definitely different moving parts and different components, but I think you can boil this down in a nutshell to you need two things for effective cultural improvement. One of them is senior leadership prioritization and this often means the buy-in of the CEO, him or herself, it really needs to be at that senior level level. And the reason why you need senior leadership prioritization is because culture changes so complex, there are all these different levers that you need to pull things like distributed management, work design, all sorts of things, social norms, and typically in a large organization, the only person with access to all these levers is gonna be the CEO or maybe the top team. So one of the things you need for culture change is senior leadership prioritization and if you don't have that, try to get it. The second is effective measurement and we're finding this out every day and what I mean by effective measurement isn't just I'm gonna tell you what's going wrong with the culture in a specific part of the organization, it's also telling you how to fix it and the way you measure that is by listening to employees again because when you ask employees for their feedback and just listen to them, they go into a huge amount of detail about the context for the issues that they're raising and they'll even proactively offer ways to address them to leadership. So effective measurement, what does this mean? So we've seen what it doesn't mean. This is what everyone does for cultural measurement but it's not effective. What do we mean by this? Listening to employees, just asking them simple questions like what would you change about the culture and it's hard to read this one I'm talking but basically when they speak about what they like and what they dislike about the culture, they'll go into a lot of detail and they'll give you the answers you need to understand what's going on and to actually fix it. So I was talking to my grandma so I tell her I'm the co-founder of this AI startup from MIT that measures organizational culture and she asked me what the hell does that mean? She's from a fishing village in Alaska and I tell her, well at its heart, grandma, it's actually very simple. Effective cultural measurement is just listening to employees in their own natural language and doing that systematically throughout an organization. That's really all you need to do and if you can do that effectively and if you have senior leadership prioritization to change the culture, then you have a very good shot at actually changing the culture. So I'm actually, even though there's a lot of reasons to be pessimistic about the state of culture, I'm actually relatively bullish about the state of culture over the next 10 years or so and that isn't because I think people are changing or people are getting better but I think that culture on some level is an engineering challenge. Cultural measurement is an engineering challenge that we've made significant progress on. And I mean, yeah, so full disclosure, I'm the co-founder of a MIT spin-off startup that does this and hopefully my company will win in this space, maybe another company will, it's probably gonna be a lot of different companies but regardless of who wins in this space, what makes me happy is that this approach, this AI approach to cultural measurements is going to take the place of this outdated traditional measurement approach and what we see when we work with leading companies is if you have these two things, if you have senior leadership prioritization and effective measurement, you can get huge results. So one of our clients is one of the largest retailers in America, 150,000 employees and we began using this approach and their senior leadership team is great, they were bought in. We began using this approach with them for six months and we were measuring the culture and six months later, their employees across 150,000 employee organization were X% more likely to speak positively about the culture. 61%, across 150,000 employee organization, they were 61% more likely to speak positively about the culture. That's a big deal for those 150,000 employees, it's also a big deal for the finances of that retailer. But what matters even more than statistics like that is the really thorny problems of culture, that kind of the dark side of culture, which is what we've spent a lot of the past couple of years researching. This is toxic culture and this case, abusive management. So that's one of the forms toxic culture can take, it's not the only one. I'd encourage you to read the whole research series on toxic culture, we don't have time to go into all of it. But the punchline is that toxic culture is very destructive. So in our research, we go into quantitative detail about what toxic culture is, who it affects, which groups it affects. For instance, it's 41% more likely to affect women than men, it's a highly gendered issue. But in terms of what toxic culture is qualitatively, it's that pit in your stomach when you think about going to work the next day because your sense of right and wrong is being violated. It's that terrible feeling that you can't leave behind you when you go home. And it has serious health consequences, physical health consequences as well as all sorts of emotional consequences. There's great research from Stanford that finds that if you're in a toxic culture, your chance of developing a major disease like a heart attack is increased by 35% to 55%. So this is just a terrible phenomenon that we're learning more about every day that's not too well understood at the moment. But what we do know is that's very prevalent. This would be one thing if it was affecting 0.1, 1% of employees. This is affecting 10% of American employees. It's a pandemic. So toxic culture is very bad and this leads us to the next slide. How do you deal with toxic culture? So again, if you have two things, they need to improve the culture. Senior leadership prioritization, a great top team that cares about this and effective measurement. What we found is that in large organizations, you can make rapid progress and consequential progress. So when we started working in this case, this is a 30,000 employee health system. When we started working with them, they were 1.2 standard deviations more toxic than their industry average of other health systems, which you can measure on Glassdoor. And that's what a lot of our researchers is possible to get a good read of the culture just by using Glassdoor data. We applied this approach with them for a year. They went up to zero standard deviation. So now they're average. Another year, they go up to 1.2 standard deviations for toxic culture. So now they're significantly more culturally healthy than the average health system and their benchmark set. And this is huge for, I mean, forgetting the finances for a second. I mean, sure, toxic culture, it takes the toll on finances, things like health insurance for starters, attrition, more so. But leaving all that aside for a second, this is something that affects thousands of employees in that organization. And alleviating that toxicity is gonna make a huge difference in their life. So again, this isn't a sales pitch for my company. It's more of a sales pitch for this general approach of cultural measurement, this AI approach, which a lot of people are doing. And it's gonna yield significantly better results and it's gonna enable employees to be a lot happier. So in conclusion, why I'm optimistic about this space, despite all the reasons for pessimism that I've studied. So extensively researching toxic culture, seeing all those statistics about, the average culture being 3.5 stars. It's because I think that at the end of the day, improving culture is largely an engineering problem, which very few people think of. Improving culture is an engineering problem if you can tackle one of the main bottlenecks, which is cultural measurement and come up with a much more effective way of doing that, even with the same leadership. Even if people don't change, the culture has a much better chance of getting better because you're getting much better information. So it is my hope that over the next, I don't know, 10 years, maybe not that long, what we're gonna see across the world is this new kind of cultural measurement pickup steam. And what this is gonna mean is that employees aren't gonna live in a 3.5 star culture. They're gonna live in a four star culture, which is a Yelp restaurant I would actually eat in. So this is our research series. This covers all the stuff we've done over the past few years. I'd encourage you to check it out. If you want to email me, my email is there. Thank you very much. If you think Glassdoor is negatively biased, everyone who's familiar with Glassdoor thinks it's negatively biased. So Glassdoor is actually one of the most balanced review platforms on the internet of any kind of review platform like Amazon or Yelp. If you look at what percentage of employees leave five star reviews and four star reviews, so positive reviews, it's 60% and fewer than 15% leave one star reviews. So it's actually significantly less biased than you might think. What I also say is even if there is a negative bias, which I would say I might have a couple of issues with, even if there is a negative bias, you're comparing different companies on an apples to apples comparison. So say everyone's negatively biased across the board, but one company is negatively citing favoritism 20% of the time. And on average, it's cited 3% of the time. Well, even if there's negative bias, that's an insight that favoritism is breaking down at this company. Have you done any research into how a company improves or doesn't improve year-a-year? Well, in a way, that's what the research study I was talking about. So that's all voice of the employee is. Do employees feel like their voice is being heard and reacted to effectively by leadership? So I mean, there's other research out there, but I would say our best piece of evidence that voice of the employee really matters for cultural success is the study I just went over. Positive culture is definitely correlated with profitability. So there's a lot of great research from an employee experience perspective. This is what the Alex Edmonds study, this professor at Wharton now at London Business School, great piece of work. And his punchline was he found that culture from an employee experience perspective alone is associated with the 20% market cap premium using very rigorous analysis published in the Journal of Economic Finance. But then that's only one way that culture creates value. So culture can also create value through what we call direct value creation. And this way, we're still in the early days of figuring this out, but we think that this might create even more value than from the employee experience. So for instance, we did a large study of 500 or so of the largest employers in America. One of the things we measured was how agile they are based on Glassdoor reviews. We then took the 2.5% most agile firms in our sample that were publicly traded. And all we did was we compare their stock, their share price increase over a five-year period. So over this five-year period, the S&P 500 increased by 68%, NASDAQ increased by about 130%. The top 2.5% most agile firms in our sample increased by over 500%. So there's still a lot more work to be done, but there's evidence that direct value creation with regards to culture matters even more than the employee experience. How exactly does the AI analysis? Yeah, so it's really very simple, as I was telling my grandma. Let's go back to this. Yeah, so if you see there, all the AI is doing, it turns out it's a hard problem. It's a harder problem than you might think. But basically all AI has to do is two things. One, understand what topic you're speaking about, and two, understand if you're speaking about it negatively or positively or indeterminate the sentiment. And if you have those two pieces of information, you can do all sorts of analyses about the culture. You can do internal benchmarking, external benchmarking, drivers analysis, progress over time, all sorts of things. The heart of how AI works or at least how our AI works is it's topical classification. So things like, well, they've left the screen now. But you measure hundreds of different topics and you see whether employees speak positively or negatively about them. Okay, thank you everyone. Thanks. Thanks.