 So my talk is called woman hater is gonna hate analyzing rap lyrics using Python and beautiful soup I just came back from spending the summer in New York at hacker school and New York is the home of hip-hop and the home of rap. So I listened to a lot of rap over the summer Which I like a lot. It has a lot of rhythm. It's good for dancing Can everyone hear me okay with a microphone? Okay, but as a woman honestly when I start to listen to the lyrics. I'm like So here I posted an example of a sap rocky a pretty popular rapper. I had to leap out a bunch of words So I was curious as a programmer You know, can we quantifiably measure how sexist is one rapper with respect to another? Is there some that are better or worse? How bad is it in general? So the first question I started with is which rapper uses the b-word the most because me I'm sick of hearing that Um So what I did is I looked at rap genius, which is like a Wikipedia site for rap. Oh my don't have four seconds Okay, which is like a Wikipedia for rap where people post rap lyrics and they're annotated as to like What does this particular verse mean and so on? But there's no API you're not even supposed to scrape it, but I did So what I did is I had to do screen scraping and I use beautiful soup, which is a library in Python I think there's more advanced things but for myself I wanted to start with something simple and beautiful soup is very easy to just get started with I think scraping You have to read a bunch of doc and I was impatient So the data is I started out by looking at 32 artists. I looked at 50 to 150 songs each I looked at a mix of east coast west coast dirty south rap, which are the kind of main currents and rap I tried to get male female black white right rappers It's pretty skewed towards male African-American rappers because that's the majority of rappers Let's be honest and the total I got two thousand six hundred seventy-three songs I want to do more but that's what I wanted to start with just to get an idea of What I was doing these are the artists that I looked at you see Drake little John 50 said Queen Latifa Wu-Tang Clan Nicki Minaj like I tried to get quite a mix of like good and bad and old and new rappers and so on This will be online. So you take a look So step one I went to rap genius and basically the pages look like this for each artist They're just a big list of songs. So I use beautiful soup to just essentially get the all the list of URLs from the page You can use it to parse the DOM and say I just want all the links in this particular div And that's what I did. Here's ASAP Rocky. It's this nice face and then Pretty much all of those links leads to a page like this So I said, okay, I don't care about anything except the lyrics just slurp that into a string where then I can start to analyze it So step three is count them up do some stats So what I found was rappers are using I'm gonna call it the B word on average twice per song Like what is up with that? And so it's used about 5,000 times over 2,000 songs like too much So the best rappers were you see some some surprising ones were Jay-Z Kanye Wu-Tang Which I didn't expect to be they use it less than once per song The a lot of women are in that category a lot of socially conscious rappers like Talib Kweli dead prez and mortal technique Don't use it that much, which is good, right? Um So so you had rappers like a biggie small 50 cent two chains Ludacris I was very surprised by biggie and Eminem and I'll talk about them more in a second Pretty bad little Wayne little Kim Tiga Snoop Dogg. I was surprised by the women are actually in the worst on the community I expected to let down humanity which she did That's pretty much what I and then little John gets his own category Because he was the worst ten times per song Approximately ten times like you learn some new words little John So the question is is this a good metric of sexism and lyrics and it turns out the artists like biggie small and Eminem That have very violent lyrics about murdering women kidnapping them assaulting them Mugging them these score relatively low like Eminem I think was about twice per song and biggie small was three times per song So um, why would that be so just look I saw I thought okay Let's just actually delve into lyrics and someone like little John This is the kind of thing that he says and he's just a bit kind of not super articulate Like just he just kind of swears a lot where it's biggie small. Yeah, I'm not even gonna say that stuff out loud Um, and to think about this biggie and Eminem They're kind of recognized amongst the greatest rappers of all time. They're kind of the greats of rap They're very lyrically clever little John not so much And so it seems like biggie now, and they just have a lot more vocabulary to describe worse things really So it suggests that the usage of the B word turns out to be not the best metric like we have to refine this a bit more So further work. What's a better marker for sexism? Is it another word? some people that like do research in this field suggested that You have to look at groupings of word related words around certain terms I am actually planning on submitting a proposal to Picon a more extended version of this talk because I and I Analyzed other words other relationships I just didn't have time in five minutes talk about them and you can soon see the code on github Soon when I've cleaned it up because it's pretty it's pretty dirty itself, honestly And that's my github right there And I just wanted to leave you with not all rap is like this and here's a queen Latifah Actually talking about this particular issue in rap which you can just read for a sec So, thank you very much