 So I'm not going to read all of it, if that's OK, because Carla is very, very impressive and long. But Carla is the dean of Computer Science at Northeastern University. Prior to joining Northeastern, she was professor of Computer Science and Clinical Translational Science Institute at the Clinical Translational Institute at Tufts University and before that on the faculty of Purdue University. She's a fellow of the ACM and AAAI. She's numerous positions in computer science in her chosen field of machine learning and data mining, including co-chair of ICML, AAAI, Journal of AI Research, Journal of Machine Learning Research. And the list just goes on. But we're just delighted to have you here, Carla. So please welcome Carla. Hi, so I'm Carla. Everyone calls me Carla at Northeastern. The freshmen have a little trouble. Someone's calling me Dean Carla, but I'm really Carla. So I'm here to talk to you today actually not about machine learning. I spent 25 years doing machine learning research, mostly in applied machine learning from applications such as finding interesting objects in astrophysics data to trying to help find the lesions that cause seizures in people who have treatment-resistant epilepsy to trying to come up with maps of global land cover of using satellite data to track deforestation across the world's surface to think about climate change. But I'm not going to talk about that. So I actually want to talk about something that is my other passion, which is trying to make computer science and machine learning in particular such that everyone has an invitation to the table. So I always like to know who my audience is. How many people are students? Keep your hand up if it's in machine learning. And who are the other people in the room? Are you your researchers? And is anyone in the room a student who's not in a technical field? OK. Yes? Computer science and philosophy. I love that the diversity of thought that comes from that. We have a lot of combined majors at Northeastern. What I'm going to talk about are two national level initiatives. And why are they national? Because it's wonderful to do stuff at Northeastern, but then it's just a drop in the bucket. So what's our problem? And this was alluded to, or actually the data was given to us in the initial presentation. But here are the numbers. 23% of AP takers in high school are women. How many people took that AP class? It was terrible. My own children told me it would be social suicide to take it. So until we fix this problem, computer science in high school isn't going to work. 19% of bachelor degree recipients are women. 26% of jobs in the computing workforce are women. But I think they count in a very advantageous to themselves way, because how can it be that we've been only producing between 17% to 19% for the last 20 years? And somehow they ended up magically with 26%. Seems like there's something fishy going on there in some companies. 11% of chief information officer roles are women, and 11% of senior leadership roles in tech companies are women. So we created two national initiatives. The first one is called the Align Masters in Computer Science. This is a masters for people who studied something other than computer science. So you all have a friend that studied something other than computer science, right? Does everybody have a friend that studied something other than computer science? Do they kind of wish that they'd studied computer science, because they see how well you're going to do financially? So we created a masters. Let me describe how it works. And there's really going to be a call to action for all of you at the very least. And this is the very least. I want you to go tell a friend about this program. And by the way, it's not just at Northeastern. And that's important to note as well. So how it works is people come in and they do two semesters of bridge programming. Think pre-med. Like if you wanted to go to medical school and you majored in, let's take philosophy as our example. So you didn't take all those science courses. You would need to take pre-med courses before you could go to med school. So we've put it into two semesters. We do not have the students join the undergraduate classes for two reasons. No 35-year-old woman wants to be in a room of 18-year-old boys, correct, who know it all about computer science. And number two, it's too slow. So what we do is we figured out exactly what needs to happen from our undergraduate curriculum that you need to have seen before you can do well in the master's classes, at which point they join the direct entry master's students. At Northeastern, it's very hard to get into our master's program because we have a built-in work experience with a company. We only admit 16% of our direct entry master students. So this is a pretty hard cohort to join. So we also give them a work experience. We have at Northeastern a work experience that's called Co-op, where we work with over 3,000 companies at the university to place people into six-month job experiences where they're paid. IBM is one of the places that we place students. And then they come back and they finish one more semester of classes. And they get a master's in computer science. It's not a different degree. It's the exact same degree that our regular direct entry master students get. And some of them go on to get a PhD. And this is particularly exciting because we're hoping to get more of these students in. So let me tell you a little bit more about the program. Why is this work experience so important? Well, first of all, it pays really well, right? Computer science pays well, so it helps offset tuition. But it also alleviates a lot of the fear factor of someone who's changing careers. Imagine for all you people who are in tech that you decided to go through a program like this in chemistry. It'd be really good to get that work experience, that some help before you went out to find your job. And it also increases job prospects. We also are an unusual university in that we have campuses in other locations. So we have our main Boston campus. And we also have graduate campuses in Seattle across from Amazon. If you've been to Seattle, we're across from the banana cart, the free banana cart. And we're also in downtown San Francisco. And we're also in downtown San Jose. So students can start at any location and they can move around as needed, which is really cool for them. The other thing that we've done is we've made it so that adults can take this. The average age in the program is 28, 29. Classes are always in the evening. They run year round to short in time to graduation. And probably most important is that we've spent a significant amount of time working with philanthropists and companies to help us be able to offer the first semester for free. Because now, as everybody got in mind the person that they're going to go tell this program, tell their friend about this program, imagine if you could say to that woman, and there's scholarships for you to try it, so you don't have to spend any money. And by the way, how many people knew that they loved computer science after their first semester? Loved it. Really? Really? Come on, you're all computer science. You didn't love it and you went into this field? OK, so I loved it from the moment I tried it. And by the way, I was an English major when I started college and I discovered computer science in 1982, back when there were more women in computer science, by the way. So this allows them to try it before they buy. And the time to graduation is 2.5 years with a half-year work experience. So this is pretty good. So where are we? We have a lot of demand and we're scaling. Right now, we're going to have 450 new students in this program. We're trying to scale to be 1,000 new students per year. And we want to grow in proportion to demographics of the US. So we would like to have 50% representation of women. At this point, we are at 46% women, people who identify as women, and 54% of people who identify as men. It's pretty good. Contrast that to 19.5%. And by the way, the regular master's population is 25% female. Why is the population of the regular master's program 25% versus the 19%? Where are they coming from? They're coming from other countries. 75% of students that are in master's programs in this country come from other countries where there appears to be less in bias or whatever reason why people don't choose computer science. In terms of other demographics, underrepresented minorities are now 25% of all domestic students in this program is roughly 60% domestic and 40% international. So we really have representation across all groups that reflect the demographics of the population of people who graduate from college in this country, which is really cool. And they come from 80 disciplines. We have some of the more obvious ones, like math and physics and chemistry and biology. But we have English. We have theater. We have performing arts students. And they're all doing fine in the sense that our attrition rate is not that high. It's about we lose the most people in the first semester where they try it and then they're like, yeah, I thought it wasn't for me and I was right. I hate coding. But after that, we have an amazingly high retention rate. These are some of the companies that our graduates have been placed in. We've placed over 150 students in the program in co-op. We have 137 people who have graduated from the program. We started small. We waited for three years to see if it worked. We wanted to see would companies want to hire people who studied something other than computer science and then got a master's? And the answer is yes. And they clamor for more of them. And the reason that they like them is think about a pharmaceutical company. Who's the best person to hire? Someone who has an undergraduate degree in chemistry and a master's or a PhD in machine learning. That's your ideal employee because they understand both sides. I know we're going to talk about applications of machine learning later in a panel. Think about the types of applications people can do if they're already trained in another area of the ideal employee for Fitbit might be someone who studied nutrition or sports management before we have a sports management and a nutrition student, by the way, in our program. So these are some of the companies. And we have of the students who responded to the survey, which is 105. So we don't know where some of our students are anymore because they didn't keep their LinkedIn pages, but we're tracking them all down. 100% are placed. I don't want to exaggerate. We also did a survey to ask what's the average salary lift. It's $40,000. That's pretty cool for two and a half years of investment of your time. And we have over 800 students currently enrolled. So it's great that Northeastern is doing this, but it's not enough. It's not going to change the demographics of who's in tech in this country. So through the generosity of Facebook, we began the Align Consortium, which is we asked other universities if they were interested in joining us in offering this program. And the first three partners are Columbia UIUC and Georgia Tech. They're scheduled to roll out this program in 2020 and fall of 2020. And we're currently looking for more partners. And for the people in the room who are in academia, the person who's running this is Jan Cuny. She's joining us in December if you know Jan, which is really exciting. So I want to talk about the second initiative that we're doing also at the national level, because again, it's wonderful if Northeastern solves things at Northeastern, but then that doesn't translate into any major change in the country. We were given a very generous gift from the executive office of Melinda Gates this summer to start the Center for Inclusive Computing. This is a center that will run out of Northeastern, which will work to help 25 different universities in this country change their intro sequence in computer science to make it more welcoming. So we'll be giving out gifts of half a million to $2 million each to schools that graduate at least 200 people in computer science. The idea is if we can move the large schools and just move them by 10%, we move the national needle from 19% to 30% women graduating in computer science. That is going to change, particularly for the young people in the room, that is going to change your work environment. There's a tipping point of 30% that changes the atmosphere and changes whether you feel underrepresented or not. So schools will apply, and we will go and work with them. And how do we get them to have this change? What do they need to do? Well, what we're really trying to address is that 56% of women are getting bachelor's degrees, but such a small sector proportion of that are getting degrees in computer science. So what we need to think about is why. Well, there's been an incredible amount of research that's been done about what you can do to make an environment more inclusive, particularly for women. And of course, obviously you can put a professor on a class that would kill off anybody's interest in computer science. I'm sure we've all had one. But if you have it on that first course, it's really challenging. So the executive office of Melinda Gates, Pivotal Ventures, commissioned some research to be done to cull all of the literature that's ever been published on how to change the demographics of who's in tech, particularly through the intro sequence. And they came up with eight best practices. And I'm not going to go through all of these. They're on our web page for the center. But I want to talk about the two that we did. First, when I was at Tufts and at Northeastern to change who went into computer science. So when I took over as department chair at Tufts, 18% of our majors were women. And at Tufts, all students in arts and sciences have to take a math distribution requirement. And two classes fulfilled that, something in math or you could take intro to computer science. So most of the students that get to Tufts that aren't going into STEM fields are ready for Cal2. Well, Cal2 was a really hard course. You know how you can make Cal2 arbitrarily hard? tanh, sineh, everybody remember those? You can make it arbitrarily hard. Well, they had made it arbitrarily hard. So people were scared of it. So they would take computer science to fulfill their math distribution requirement. There were two computer science classes. One was for majors and one was for non-majors. And I went and I looked at the demographics of both. And all of the diversity was in the non-majors class. So I got rid of it. And I said, I know they're more scared of Cal2 than they are of our majors class. And then I put the most charismatic faculty member we have on that. In two years, 46% of that class was female and we were up to 33% women in our program. And a lot of them were doing joint majors of something in liberal arts, Spanish and computer science, philosophy in computer science, English in computer science. Because at Tufts, you can have a double major and it works out fairly well. So that's what worked at Tufts. What worked at Northeastern was something entirely different. There are no, we have distribution requirements, but we call them Northeastern University Path. And lots of courses fulfill every one of those distribution requirements. So there was no trick that I could play like that. So instead, what we did was we made sure that the people that were teaching the intro sequence really cared that everybody have a good experience and not just that it was a weeder course and that there is a geek gene that they're looking for. I think we tripled the TA budget so that everybody had access to TAs all the time whenever they needed them. And this worked. The second thing that worked at Northeastern was the creation of combined majors where people can take two topics at once. They're not quite a double major because it's hard to get out in time and have your work experiences. So we've created over 36 different combined majors. So you can combine data science and health sciences, cybersecurity and criminology, computer science and English and the list goes on if we don't have it, we'll make it for someone. This year, 40% of our applicants were female and 31% of the incoming class is female and we admitted in proportion but other schools also tried to go after the women applicants and so we didn't end up with the same distribution. So we're working on this and we're really hoping as I said to get over the five years by working with 25 schools of the largest schools, we're hoping to get a 10% increase in degrees across the country, moving us to 30%. And I wanna conclude and just say, it's unacceptable where we are. If you think about it, first of all, the most important thing that every single person has in their daily life is tech and it's every person uses stuff that you guys are gonna create. It should be created by people who represent all sectors of this country. So we can't stick with where we are. Second, I really believe change is possible and it's never too late or too early to break into tech and this is a sort of a, you're probably thinking, why is a machine learning researcher and I'm at a women in machine learning conference talking about this to me and I could have talked about my machine learning research although it's a little dusty because mostly I work on this now but I chose to talk about this because you're going into a field or you're already in a field where you're severely underrepresented and it would be really helpful if every one of you went and became like me. I recruit my Lyft driver. I recruit, you don't maybe wanna do quite what I do. I recruit everybody I meet. That's why I asked if there was anyone that didn't do computer science because I was probably gonna go and stand right next to her and talk to her the entire time and try to convince her to go into computer science and a couple of people from Northeastern here are with me and they're laughing because they've seen me do it. So, you know, talk to your friends, tell them about that it's that, you know, we're waiting for them with open arms and not just us, but Georgia Tech, UIUC and Columbia are all waiting in the fall for them to come. So if they live near one of those schools, they should go to one of those or if they live near one of our campuses and we're also putting this program online starting in 2020 for people who don't live near a school. And the idea would be that if you do the bridge programming, you've learned enough that you can go into any master's program. And then hopefully from that, you're gonna love machine learning and you're gonna end up getting a PhD and you're gonna end up becoming faculty and you're also gonna be a role model for other people who wanna go into machine learning. And I'm happy to take a few questions if there's still any time. Yes, and I'm hearing impaired. So pretend I'm your deaf grandma. I'm not a grandma yet, hopefully in 10 years. Or yeah, that would be even more helpful. So I've been in meetings like this and one of the main questions that would always come up is there are a lot of fields that are completely opposite of us, that women are completely outnumbering men. And we don't necessarily talk about it or like men are not hanging out with each other trying to make sure that they are represented in those fields. What is your main answer to that? Because a lot of people play it with like, oh, it's fine because there are fields that men are very few and women are outnumbering them. So what is your main answer to that? That's so interesting. So this morning I had to go to our Board of Trustees meeting which is at our Burlington campus and my Lyft driver was a wonderful young man who is studying to be a nurse. And we talked about that and he's one of three in 60 and he is uncomfortable and he would like something to be done about it. And his reasons for being a nurse, I mean, I'm sorry if I tear up or because his brother died of cancer and he saw how meaningful the nurses were and I talked to him and I said that, and I'm gonna answer your question but I think it's important to understand how I feel about it. I talked to him about one of my sons who spent quite a bit of time in children's and how wonderful it would have been for him to have a male nurse that he could have joked around with because in some sense because of how he would have felt about having the male nurse, it would have sort of made him kind of try to toughen up a little bit as part of male culture and it would have been a wonderful thing for him in that hospital to have had a male nurse. I think it's an equally important problem and one that I've talked to actually the head of our nursing department at it, how do we get more men into nursing? I think it's really important. I think one of the problems people often don't think it's a problem is because the fields in which women are overrepresented are not the ones that pay well. And that's really the problem. And unless we have a change in the distribution of wealth in this country, we're also gonna continue to have some problems but I'm not allowed to talk about politics. And when we were outnumbered, we were more in computer science, it was the time that computer science and well as well as now as well. Correct. My first job paid, I think, $17,000 a year and my sister who was working for the National Cancer Society corralling volunteers made more than I did. And now that job probably still pays about $18,000 a year, right? And our graduates are getting over 100,000. Yeah, question. Microphone would really help me and also help the camera. Sit down, come on. Right. Hi. Hello. So I'm actually... Maybe it would be good for people to introduce themselves too, because that's always fun. Okay, so I'm Belma Velich and I represent Refinitiv, which is a data company. We actually just announced our membership of the MIT, IBM, AI, Watson Lab. So we're really passionate about getting into research, being part of that innovation agenda, but also we're really passionate about women in technology. So my question is, are you open to actual partnerships with companies, maybe as part of the co-op program, offering work experience, or is it just students that you're interested in? No, we are, oh no, we're kidding. I spend all of my time mostly talking to companies. We're very interested and my head of strategic partnerships, Catherine, is here. We love companies to provide co-op experiences and we also have other ways for you to get involved, both personally and philanthropically, in helping with our mission. One more question. So we get involved with a lot of women's networks in general. I mean, I've been part of quite a few in different roles in different industries. Do you have a global reach as well? Do you reach out to these women's networks and actually try and mimic your programs or the programs that you're running in other schools outside of the US? Coming, we're just got the center this summer and the Align Consortium only started, I think, in April. So we really believe, I'm still a machine learning person. I'm a data person. I believe I have to make sure it works at home before I decide I'm gonna go out and have it work, not at home. So Align, we had to watch for a few years to make sure it worked and debug it. And by the way, we went through three iterations of the curriculum before we got it right and then the co-curricular programming to make people feel comfortable also took us a few years to sort through. So we're just at the point where we're now starting nationally but Northeastern has a campus in London and in Vancouver and in Toronto and we're opening this program in those campuses first in Vancouver then in Toronto and then in London and more to come because our university president, we are a global university and I don't know where we're going next. I don't have any control over that. I go where he goes. Yes. Hi, my name is Leah Pillsbury and I just finished a program sort of similar to what you talked about in Align at BU, the LEAP program for mechanical engineering and I really enjoyed it and got an opportunity I never would have had otherwise and in going through it, I found that the professors and the other students were very supportive but there are times when professors and now in looking for jobs, people who interview set things up in a way that I would say is unfair to women but they wouldn't necessarily realize that like either in the way a professor would create competition in a class or assign teams or now in interviews, the way people ask questions, the kind of questions they ask and I'm wondering how you've been able to address that in your program. So I can't address how companies interview you. I can't address, we have 500 companies in Corey College that we work with and we definitely give them feedback if they're interviewing in a way that makes people feel unwelcome. So we do our best with the companies that we work with to help them figure out how to do more fair interviewing and ways that are comfortable. Sorry if I wasn't clear. I don't mean feeling unwelcome. I mean like asking things in a way that would get a very different response from a man and a woman. I meant that too. I meant exactly that. I feel uncomfortable when I'm asked a question where I know I'm not gonna answer the way they want me to hear but I need to be true to myself. So that's what I meant. Yeah, not like a weird question like can I date you? But like in a way where I know that there's a correct answer and I'm not gonna give it because I'm gonna answer from the way I would answer that. And in terms of classrooms, there are best practices. And so with our Center for Inclusive Computing, one of those eight is about how do you set up your classroom atmosphere? So for example, it's well documented that if you have teams of four or more people, you shouldn't ever have a one of, whatever that one of is. And we do TA training at Northeastern and we'll also be sharing this with our partner schools where we talk to the TAs and we do paired programming and we gave the TAs all these scenarios where we said, okay, what if one person in the pair is underperforming and we never used any gendering language? And when the TAs went to answer how they dealt with it, they used the pronoun she and we then pointed out that we never said what gender and we do that type of training to try to help them figure out how they can be better TAs. So yes, we are trying to do all of that but not in mechanical engineering. I'm very sorry. On that in terms of the interviews part of it, I can assure you right now there are a lot of companies that are starting to realize this problem in the interview practices and so on. And so they, you know, a lot are trying to address it. And so I would recommend, you know what, maybe think about writing an article about that. You know what about your experience and I bet a lot of other people would probably comment on it as well. And if you can't think of a place to submit the article, you know, let me know and I'm sure we can figure out a place to try to get it some attention, okay? That's a fabulous idea, yeah. Hey, I'm Maddie Shane. I lead the data science and ML community for womanhood code. We have over 180,000 professional engineers worldwide. And you know, I want to give you kudos for fixing the pipeline problem using a push strategy or trying to fix the problem using a push strategy. And one of the counter patterns I've kind of like noticed is that sometimes when you're trying to break into machine learning as a industry professional, people would want to look for like research experience or PhDs or et cetera, but for a substantial part of our members that is not an option, right? So I was wondering, you know, you as a person in academia, do you have ideas for how to kind of like break that anti-pattern? So let me rephrase it to make sure I understood it, but not to change it, right? That's a trick people use, but I'm actually gonna say it to make sure I understood. So if I didn't get it, are you asking how people can get more machine learning if they don't have the luxury of going back to school? Yeah, exactly. So a lot of our candidates are actually like undervalued because even though they're doing research work or at least applied research, they just don't have the typical researcher background, right? And our hypothesis is that, you know, for us to fix the diversity problem, we need to build strong technical leaders and put them into positions in companies where they can actually effect change. So this is actually a real barrier that a lot of our members are facing and we'd love to hear your thoughts on how to solve that problem. So I'm not sure I have the answer, but here's my first idea having never thought about that before, but if they've been doing research and they can somehow document the results that they've had in a way on their CV or during or network their way till they can talk to the person that would be evaluating them to demonstrate that they've done research. So if you don't have a credential, it doesn't necessarily mean you can't do, like if you don't have a quote master's or a PhD, it doesn't mean you can't do machine learning research, but you definitely have to have some way of demonstrating that you understand how to do the research. And the question is how do you best do that? My first idea is to document it somehow on their resume, but also to document it by when they're trying to get a job that would be a research job in machine learning is to figure out how do they meet the people where they can actually sit and have the conversation to demonstrate, yes, well, I fixed this thing in this algorithm when I was working on this problem and here's the result that I had and here's why the algorithm didn't work and here's what I had to do to fix it just to show that they have the ability to be creative in a situation that's gonna cause them to be creative. But they can also, very hard to get research experience if you aren't in a program, but you could come back and get a PhD. Is this a thing that- You're paid, you're paid to get the PhD, you're not paid much, but you're paid a subsistence. Right, so I'm gonna ask the leading question, is there any opportunities for academia and people who are currently in the industry to kind of like collaborate and maybe borrow some street cred in terms of like research? I'd have to think more deeply on that, I don't have an answer off of the top of my head. I know that we're talking with several different industries about how, so one thing academia likes to say to industry and my apologies for this is that you're stealing all of the faculty and eating the seed corn and we're not able to generate enough people for you and so the question is, can some of the people from industry help mentor our PhD students? So yeah, actually I was maybe gonna comment on two things. Yeah, I think Laura will too. So we actually, we are, we do that quite a bit in terms of mentoring PhD students so even a program for that with universities, I can talk more about that but I also wanted to mention something else that may be helpful for what you're describing. I agree with everything that Carla is saying but to add to that, what's happened is because there's such a need for people in machine learning and AI, many large companies have, several large companies have started a program, they refer to them as AI residencies. So it's basically a 12 month program where people that have some machine learning skills but maybe don't have the PhD or maybe don't have enough of the work experience to come in for a year and work as part of a research team to try to build that up and in some cases it may well be that they may wanna go back to school or not but at least they have a way to get in and start building up. So IBM has, you can serve for residencies or AI residencies, so IBM has a program, we hire that, Google has a program like that. I think Microsoft has a residency program as well but there are quite a few of those and they're all about a year long and so it's meant for exactly what you're describing. Thanks. Time and I think I'm out of voice. So thank you.