 Welcome to the education technology session. I'm Caitlin Tagarelli. As I said, I work at Mango Languages. So I think that's how I became here as moderator of this panel. And we have it's Rachel as our support person. Hi, there she is. Thank you. So we are going to talk to some panelists who have had different roots from linguistics to education technology. We have Sam Cooper from No Red Ink, Emily Maline from Duolingo, and Anastasia Lukina from ETS. I'm going to let them each introduce themselves and tell you a bit more about themselves. I have some questions prepared to get the conversation going, and then toward the end we'll make sure that you all have a chance to ask your questions as well. So I'd like to start by giving each of you a chance to introduce yourselves. Anyone like to go first? Can tell us a little bit about, you know, who you are, what you're doing, and how you got there in a couple minutes. I can go first. Okay, perfect. My name is Anastasia Lukina. I'm currently managing senior research scientist in NLP and speech group, which is now called AI Labs at the Educational Testing Service. I started my linguistics journey as a modern Greek scholar, and I started it as a specialist in modern Greek and actually Byzantine studies, and I did my PhD in Oxford and phonetics, and my PhD was on phonetic variation in modern Greek. So I went to Greek villages. I recorded elderly speakers. I looked into how their speech compares to each other. After that I studied Oxford and I did a postdoc, which was also in phonetics, but moving more into more technical aspects and into machine learning. And after that I came to TTS as an associate research scientist, and I've been here for eight years since 2013. And what I have been doing here, we are building NLP-powered backhands for various educational applications. That could be automated scoring engine for speech. It could be backhands for reading tutors. It could be backhands for learning applications. And at this point in my career, I'm primarily managing other people who are doing research or engineering into how to build the backhand that then provides feedback on your speech. Great. That's so interesting. I'm excited to hear more about how you made that link from phonetics to NLP. Sam, would you like to go next? Sure. Okay, so my name is Sam Cooper. It might be a detail that I'm originally from the UK, but currently based out of San Francisco. Well, I work for No Red Ink, which is an education tech company but creates writing and grammar curriculum for grades five to 12. I did a BA in German and linguistics at the University of Oxford. During that time, had a student job teaching English in the summers in Germany, which set me on the path of becoming an ESL teacher after university. I spent five years in Spain teaching English as a form of language to really all levels from absolute beginners to extremely fluent people, all ages from seven to I think about 70-ish. So very diverse teaching experience. And when I moved to California for personal reasons, I had to find something new to do. And that tech was it. Great. Thank you. Okay, I see, I just see in the chat that some people are a little bit confused about what the back end is. Before we get on to Emily Anastasi, do you want to explain that? Explain the chat. Sorry for using the specialist terms. So back end is actually the piece of software in the back that would take your response, get the essay or your recorded response, do various processing, and then give back score or feedback. So this is what we call back end. The software that processes what you give it. Thanks. Okay, and Emily. Yeah, hi everyone. I'm Emily Moline. My background to Duolingo is, you know, similarly, I started out in a more traditional research effective kind of like Anastasia. I got my PhD in linguistics in UC Davis in 2018. My focus was on applied linguistics and socio-linguistics. And my dissertation was on kind of an applied research perspective looking at adult literacy, English teaching. So I'd always been interested in the applied aspects of language teaching, but from this more theoretical perspective, I also similarly to Sam, how I've experienced teaching English, both at the college level as well as also to English learners in Spain as it turns out. So for me, I was sort of just interested in exploring other options outside of academia. I was into the idea, I still am, that educational technology can have a really outsize impact for folks like us who have linguistics backgrounds and ideas about teaching and scaling language. So my first job was a kind of funny one at a marketing company doing professional naming of products that was me playing out with what it was like to use my linguistics to green other places. That didn't feel like a great fit to me. So when the Duolingo opportunity came along, that felt something more relevant to my background in applied linguistics and theories of teaching. So at Duolingo, I am a curriculum designer, and I think about how to apply the best practices of language teaching and pedagogy to an app. So the skills that I honed in the classroom and through my research, looking at the role of oracy in literacy learning, I now think about teaching to millions of people, which is very different and full of lots of interesting challenges. And at Duolingo, I have done specific work on certain courses like the English Courses Speakers of Spanish and the English Courses Speakers of Russian, which are separate courses entirely as you can imagine. But I've also consulted on various aspects like our teaching of speaking through live events and other languages that are teaching non-English languages to folks with the background in English, so a variety of things. Great, thank you so much. I'm really excited to hear more from all of you. So you talked a bit about your career paths and the first thing I wanted to ask you is how did you get your first job after graduating with your linguistics degree? What did you do to get that job? I mean, for me, I just, this is, I think you'll hear this a lot, but I was casting a broad net. I was applying to a lot of different things. As I mentioned, this first job that I had wasn't one that really felt like a good fit for me. I was there for kind of a fellowship for a year. So that's another tip, your first job that you get doesn't have to be your forever job also. But I would just apply blind to the website. I didn't know anybody. That being said, I also did a ton of informational interviews. And that's another thing that people probably told you to do. And I highly do encourage you to take them up on that. It was just a great way for me to get a sense of what was out there and what kinds of skills were needed. I started doing those when I was still in grad school. And I also really found them super valuable to understand the landscape of jobs. And it helped me to prepare for the do-it-and-go job. I knew that at that point, that was something that was more appealing to me than having done four or five informational interviews by that point. Great. So that was just like an application that you saw online that you applied to. Great. And did you have informational interviews with people that were at that company or in sort of other areas? So I didn't for that job, but it would have been a good idea for me to do that. And that is something that is available for you to do, to reach out to folks who are at that company before actually submitting an application. That's helpful both for you to get a sense of what they do, as well as to tailor your application and your cover letter better to more accurately capture what the role is actually going to be. Sam, would you like to tell us next about how you got your first job? Yeah. Well, my first job out of university was a teaching job. But there was a little bit of a gap between me graduating from the university and me getting that job because I had intended to go to grad school and graduated in a recession. The funding was not fair. So had to take a minute to figure out next steps and ended up qualifying as an EFL teacher. That's a separate course that you need to do in order to, you know, to teach you something more reputable. Schools, I had a lot of clarity going into that, knowing that I wanted to teach and I don't know that I really considered that many other options until several years later. So for me, the question was just like, okay, what country am I going to live in? Because I figured, hey, I have a German degree, I'll go and live in Germany, but turns out there's not a huge need for private English teachers out there in the same way that there was in Spain. So that was serendipitous. And linguistics kind of led into that very nicely because obviously when you spend all of that time really analyzing grammar and how language works, that translates quite well into explaining that to other people. So it turned out to be very easy to get jobs as a linguist who became an ESL teacher. Yeah, that's a great point. All that stuff that's kind of intuitive to native speakers is hard to explain. But if you're studying it, that comes up there and that's a really good, an important skill that we develop as linguists. Anastasia. I was trying to think what was my first job and it's had to pin down the exact first job because I had a lot of linguistics related drops since undergrad. I've worked as translator and interpreter for many years. And then I was teaching modern languages. I was teaching in modern Greek for three or four years. I'm trying to remember for how many years. And that was also head of modern Greek library collection in Oxford for five years. So I've had all those jobs and each of them, I found either through personal network or just seeing an application and applying. And definitely at the end, all those jobs helped me to get my current job because they all taught me different aspects. So I would generally encourage you to just be very open and to go for different types of jobs because when I'm now hiring people, I always like to see this diversity of, you know, you had a bit of language teaching experience. You have a bit of that experience. You had a bit of that experience. My real full-time job after graduation up to my PhD was the postdoc. And that was actually not particularly interesting pathway because that was postdoc in my own lab. So I knew the person who was hiring and they needed somebody and they just asked, what do you like to be on our ground? So that was a very easy transition, but the one that's easy, difficult to arrange. The next job was the same as everybody else. I was applying to many different positions. I got a number of interviews and the position at the TS was the one that I liked most. That seemed to be most relevant to my skills. Interesting. So I think it's really interesting to see how, for all three of you, there are lots of kind of different tactics that worked or different ways of getting jobs that worked over the course of your career. So in some cases, it's really about casting a broad net and actually applying to these jobs that you see online. And then other times there's a bit of networking happening. One thing that I would, I know it's not very common in linguistics, but it's very common in computer scientists and internships. Internships. And open eye for internships, because often if they look for an LP person, they might actually hire linguists. And we certainly always value people who did industry internships. That's good. All right. So when you are creating, updating your resume, either for your job search, or even now as you're keeping this updated, or have you had interviews, how do you talk about your skills as a linguist? So the thing that I feel to be true regarding, at least my particular role, I do a lingo and what we look for is that we understand that if you have a PhD in linguistics, that you can do linguistics. That is not called into doubt. It is if you are going in the academic route, and you're worried about tenure, and you're worried about standing in front of a room of people, asking you hard questions about your paper. But when you're applying to a job, people will assume that you can do linguistic stuff. What they aren't sure of coming from academia is if you can manage a project, if you can interact with colleagues, if you can deliver something in one week that you'd rather have one month to do. Those are the kinds of skills that I think you more have to prove in industry than in academia. It's kind of switched. So what I would recommend when putting together a resume or thinking about presenting yourself for those kinds of jobs, is mention your research project, mention what you did as part of your particular work in linguistics, but make sure to highlight kind of what Anastasia was saying, the breadth of your experience. So if you did teaching, for instance, in my job, that was very important. I didn't just do research that I was an active teacher in a classroom as well, and that I studied that, but that I also had that experience on the ground. So I made sure to mention that. I also mentioned my prior job, how I had responsibilities for delivering things on time and managing projects, which by the way, if you've done a dissertation, you have managed a big project. If you've done any research experience outside of particulars of what is just covered in your academic experience, that's good to mention too, but emphasizing that you have these working abilities rather than just the language ones, I think is pretty key. Yeah, that's such a great point. These transferable skills. I second that when we look for resumes coming from any field, but also linguists will look for, are you able to connect what you've done to what we might be doing? So definitely you have to do your homework, see what the company is working on, and write the target that's cover letters saying, this is how what I did is immediately related to what you are doing. And we also look for soft skills like, have you worked in a team? Are you able to work with other people? Are you able to resolve conflicts? Even if you work with somebody on a project where they told you to do something and you've done it well, do you have any ideas of information? This is a difficult question to answer from my job because my background as a linguist is sort of secondary to my background as an educator in the world I'm in. I got this job mainly because I have a lot of teaching experience and it's a nice bonus that I'm a linguist because that comes with obviously a much greater amount of declarative knowledge than native speakers typically have about the English language, which is very useful when you're trying to explain it to students. It also means that I think linguists have some advantages in things like copy editing and just being able to articulate this is why this isn't really working in a way that a lot of people aren't even if they're fairly accomplished writers. They might be like, that sounds weird, I don't know how to fix it. So I think we have an advantage in that sphere as well. I will second the thing about transferable skills if any of you do have teaching experience, that is a big thing that you can parlay into explaining all of your work skills, managing people, managing conversations, project managing and teaching have a lot of skills in common in terms of just having a million boards up in the air at once and trying to keep tabs on everything and who's doing what and as far as linguistics, I think the main thing that I bring to my job from my linguistics background is problem solving skills. Part of what appeared to me about EdTech is that it seemed to sit at the intersection of having a lot of teaching experience and also having that analytical linguistics background. There's a lot of problem solving and thinking through solutions and I think I probably adapted to the more technical aspects of my job, which are not that many, but they are there a little bit more easily than I might have otherwise. Can you speak to those more technical aspects of your job? What are some of these more technical skills that you think you do actually need to draw on? Do any of you need to write code or what other sorts of things come into these technical aspects? We do our own HTML, which is actually a coding language I was lucky to learn at school, so that wasn't too bad. The engineers do most of the technical lifting for us, but we're a startup and we have limited boundaries, so everyone has to do a little bit. They really try to limit what we do because it's not in our real house, really. A lot of what I do in code is actually plagiarizing our people's code and just changing the words, but manipulating code in that way is a little bit, I think, more intuitive if you have come through that kind of understanding how language puts together is helpful. More broadly, there's just a lot of understanding how to interface with machines, but I didn't really have to do as a teacher at all. I was at a very low tech school, so then understanding things like how the backend works, like a working knowledge of what's going to break the site if I do this, or just understanding how things work with the engineering and there's a lot to process. So even though our actual coding is limited, we do have to understand a few things. I'm sure it might be similar that we do in Ingo, but I don't know how much coding you guys do. Yeah, so it's very similar in that I do, as part of my job requirements, I do absolutely zero coding. I don't need to know how to program. I was, this was the biggest thing, question I had in going into educational technology was, do I have to learn Python? Do I have to learn, you know, a sequel or something like that, but I don't need to base language. Those things are not useful, but they were not at all part of my job description, job requirement. And I don't, similar to what Sam said, I don't have to use those things. I could, if I wanted, I maybe could, you know, they could be useful, but I most have to know how they work at a broad scale and how the people who use them all the time need to know how them to work. That made sense as the full sentence. So these are things that I pick up on the job, you know, understanding the process, what is possible to do with code. And so it again, in terms of getting the job, I would not at all say you have to do a special program or learn how to do this stuff, unless it says specifically in the kind of jobs you're looking at some NLP jobs. Definitely you have to know Python, for instance, but if you're looking for a curriculum design job, it's, I think probably pretty unlikely that you would have to have these skills. Again, just look at job description, it's listed there for the kinds of things you're interested. This is why informational interviews are also helpful. But again, having that knowledge of how things generally work is probably far more useful in showing that you can adapt to that and learn to that, to what you need to know rather than having to come in with those skills yourself, because you're not going to be the programmer in these contexts again, unless it's specifically said that in the job description. I think the other thing I'll say just in terms of technical stuff is it actually helps to know how Google Sheets work. I use spreadsheets so much. So that is something, again, that you'll pick up on the job. You don't have to say I am a spreadsheet wizard, but in terms of technical stuff, knowing the basics of Excel is handy. Yeah, just to add on to Emily's point, yeah, definitely you need to know what have you considered the basics. Beyond that, I don't think we require people to really know anything more than basic technicality, but if you do happen to know SQL or if you have worked with data, that is definitely something that you should mention because there's a lot of data in our tech and to the extent that we can do it ourselves, we like to, but one person on my team who happens to be very good at SQL, she's a person who's currently plagiarized all the time to do my SQL. So if that has been a part of your quite of experience, definitely bring that up because they're going to want to know that you can do that. It's a big plus if not a requirement. I definitely want to hear Anastasia's take on this, but I did want to kind of ask a follow-up question for definitely Emily, maybe Sam as well, but I was wondering when you talk about needing to know how, understand code and kind of how it works. Can you maybe give an example of why you would need to know this? So something I was thinking about, maybe it's like writing requirements for how a curriculum might be implemented. Is that something that you would do? Yeah. So as an example of what I need to understand, so like a duolingo when we're teaching stuff, when we're coming up with how to sequence things, we can't use a word in a sentence that you see on your device that you haven't seen before. Just from a pedagogical perspective, we don't want to show you something that you haven't been exposed to, but that might include, for instance, the exact spelling of the word, even though you've taught the concept, say a third-person singular s. Maybe you have long since shown plays and walks and things like that, but then you forget that you haven't added the exact word that has this s on it, and you want to show a Gibbs, right? And if you forgot to do that, you have to understand why that's impossible. And there's like a code reason, which is that the code doesn't care about the word. That's a human concept. The code cares about G-I-V-E-S, right, being coded into the system. So when you understand those aspects of what is possible or not possible, then you can communicate with the folks responsible for implementing it, which is definitely not me, and say, hey, could you write me something that would allow me to plop on an s to any base form of a verb or something to that effect? So that's the kind of way as I need to understand what is there isn't possible with code or how code kind of generally works in our system and so I can communicate about it. Thank you. Anastasia, do you have any... So I was thinking that at ETS, we hire linguists in multiple divisions and maybe I should give some background. ETS stands for Educational Testing Service. So we are the company behind TOHOL-IBT and GRE and many other standardized tests. So we hire linguists in our test development department where linguists maybe are writing questions, writing answers, questions and items for different tests. And there, there are no technical skills required. What they want is experience in language teaching, good copy editing skills, good understanding of how do you teach language. Then we also hire linguists into our research department where we do research into second language learning. And linguists in that department might do a wide variety of studies. They might look where the TOHOL-IBT test is really a good test in predicting your future success in academia. They might be looking into developing future tests. For example, how do you measure somebody's pragmatic competence? How do you measure whether somebody has cultural competence? And here we are not necessarily looking for technical skills. We are looking for solid research base. We will be looking for PhD preferably in that same area, in second language acquisition or if we are looking for cultural competence, maybe something that focuses in that area. And those linguists in that department do very much academic research similar to what you would do in the university. And so we are looking for skills that are same as you would be looking for in academic job publication records, good knowledge of qualitative quantitative design methodology, ability to present your work in the right way, ability to ask the right research questions. Now my group is more engineering applied in LP group. We hire linguists too. So what we do, for example, we want to give feedback on your pronunciation. So we're building an app that gives you feedback on your pronunciation. And we have engineers who know a lot about signal processing, but we also need linguists who would be able to break the cons of the pronunciation error into linguistically meaningful terms, like are we talking about absolute representation? Are we talking about specific contrast? Are we speaking about prosody? Are we speaking about connected speech processes? All those things that politicians know, but engineers don't know. Having said that in our group, we are definitely looking for very strong technical skills. So we usually look for knowledge of Python. We look for experience with machine learning models. And usually people who are behind, you know, people would be the ones who have done either postdoc or PhD with some machine learning component to it. And there are politicians and linguists who come with that background. But for this job, I would definitely recommend learning Python, learning shell scripting. Definitely learning something about machine learning. Does it see as research team, high-primer linguists doing research and language testing specifically or SLA in general? SLA in general. It depends a lot. We have a lot of careers posted on our website. You could see the description. They usually say what the background is expected to be. What did you say? Is it both? So you have some... So we hire people with the background in educational measurement. We also hire people with the background in second language acquisition. Thank you. Let's see. What about... So we talked about some of kind of the skills that you've brought with you to your jobs. What about some of the skills that you've learned on the job that you think are useful to what you've done or what you're doing now? I've certainly become much more of an engineer. I've learned a lot about engineering and programming just from working with colleagues and being very hands-on. I also learned a lot about test development and assessment. I was not an assessment person before I came in. And it was fascinating to learn about how those tests are designed, how do people ensure their fairness, their validity, even the terms themselves. What do they mean? All the things that I learned after I came to UTS. And of course you learn a lot about how to work with people, how to manage people, how to work in the industry. I've definitely learned like Anastasia was just saying those soft skills, which is a term I hate, I need to find a better one for it because they're as essential as your technical knowledge I think in your field about working with others and what it means to actually make something exist in the world. That isn't just a paper or a PhD, a dissertation that takes a very long time to make and is maybe read by 10 people max. But to actually make something that's out there that is used by lots of people and kind of what goes into that and the intense amount of collaboration and giving up of a sense of self in order to benefit the greater good for making something exist in the world. So that's sort of a very active collaboration and that process has been a big learning for me. In terms of like more technical skills I have learned a lot about the specifics of my experience and sort of my theoretical teaching was more about the basic principles of thinking about how language learning happens or literacy through oracy. And in my job I had to be extremely specific with thinking about the actual grammar structures, the phrases, the collocations, the words that go into the see-for levels of English and other languages. And so now I have a very good intuition for what like an A2 sentence is or A2.2 sentences versus A2.1. And so those are definitely skills I didn't have for starting at Duolingo that I think of maybe a stronger person who knows about curricula, language curricula in general. So I think both kinds of soft and hard skills can come through a job. I have become a better writer since starting to work in a writing and grammar company I feel like I have had a lot now that I'm making this curriculum it's like I wish I'd had this in school because we've really not explicitly taught how to write very well. So I've actually learned a lot just from doing the job but it's also a very different way of presenting information from being at the classroom. Well, you're live and you're talking and you see the kids and you're like, oh, you look confused. I'm going to try to say that another way. With what I do now it has to be totally clear on the screen there isn't any space to add that clarification. So it's really finding very specific and concise language. I was second what Emily said about collaboration that was a big shift for me coming out of the classroom where really it was all on me and like just being students now working with other adults and having to be in a team and working together also open long term projects like some of our projects take weeks months to complete it's not okay you've got a plan today lesson okay did today's lesson that's done next thing maybe it went well or maybe you didn't there's a lot more back and forth and like providing things and that's been an interesting shift as well I would say. You probably have the opposite too if you're coming from academia and a PhD where things happen very fast in industry I think at a much faster pace than they do in academia sometimes as well. So kind of depends on the project maybe this and I was going to ask if any of your learned on the job skills have kind of contributed to your current responsibilities and your current roles like basically what you've if you've been able to find some expertise in what you're learning on the job and distinguish yourself that way I have an example that I can share for context so I have a colleague who has a bachelor's in Mandarin and linguistics she's a very talented linguist and she came on to work with us at Mango and she just really honed in on our our authoring tool so basically the tool that we use to to enter and create content and she now knows that system probably better than anyone in the company aside from the person who created it and then she's always looking for ways to improve it for the teachers and stuff and she's become essentially like our our expert on that which is very different from what she she learned you know what in her linguistics degree and I see I see that a lot at Mango where people kind of find ways find niches for themselves and I was wondering if you guys had any experience with that or or examples of where you learn things on the job that have really contributed to what you're now able to do I would say most things I'm now doing kind of on the job yeah because really also they feel this changing so fast that also thinking about most of my colleagues those who graduated more than 10 years ago everything we learned in the university well the theory is there but the technology has moved on so fast the methodology has moved on so fast that we just all continuously learning on the jobs in fact everybody I know is constantly taking courses so Udacity courses to learn new methods to learn about new models to learn about new approaches yeah because really I can think nothing that I used to my PhD it's still parent some of the models maybe linear regression of course is still there but beyond that and certainly we see a lot of people moving they come as a linguist and some people become chief BQ of technology or maybe they become business CEOs there are lots and lots of possible career paths yeah so it's the same I do only go where we really emphasize like continued learning within our field not up not just things that within the job but yeah we all encourage to attend conferences like LSA or others that are relevant to our sub fields and you know are paid to go to them to attend them in terms of stuff that's unique to learning on the job in my case I had the opportunity I've always been interested in transition teaching and just oral language in general and so I do a lingo I was there was a project floating around that people were sort of like I don't know if I really want to do this someone else was like excited I was like oh I really want to come up with all the different sound teaching for English and I want to come up with all the different sound letter mappings and biological contrast even though it's not like you know my my my soul role not to be the sound teaching person but it was something I glad they signed up for and as a result you know have this knowledge of our English teaching courses as they relate to the sound aspects so that's an example of something that was sort of available that was I could express my personal interest in and kind of own as an opportunity is sort of a niche thing as as a linguist or or curriculum designer person at the company that's really cool yeah for me my trajectory over the last three and a half years well I think has been one of starting out basically writing content to spec so you know here's what we do do the thing and then that turned into okay now you help the team do that thing you're in charge of making that one thing but we've already figured out and since then having learned more about the EdTech and having completed more of that work that has turned into managing bigger and much more ambiguous projects and really solving design issues and thinking about user needs what do body teachers need what do students need especially this past year working for a K-12 content when I was learning it's been a lot of thinking about what can we do that's going to be helpful and really taking on more of that design work and figuring out myself okay what are we going to make here which is big and sometimes a little bit overwhelming because it's like canvas almost but yeah kind of increasing amounts of ambiguity and autonomy but probably fair to say they've never given me anything that was outside of my sense of proximal development in terms of teaching but yeah we also emphasize you know continued growth we have levels within each role I think even sub-levels within each level but we're expected to just move we were like increasingly take on more and more things but we're continually growing in the role yeah I think sometimes in the job process we're looking for a role that we want to fill with the mind that we can like grow and change in that role and within that company as well you know and possibly onto other career moves so it's good to see examples of how that can happen okay so moving on I have a lot of skills questions so I think we have a range of people with different backgrounds and linguistics here and so I wanted to ask a bit about the degree requirements for your job so does your position require a PhD or a master's or a bachelor's to either to get hired or to actually do the job so could you have gotten the job without one of these more advanced degrees and are you able to perform it or are people at a similar level able to perform it without that training I'll go first and I have a PhD I don't even have a master's I have a BA in German linguistics and a postgraduate diploma in TESOL which is a useful diploma to have when you work in TESOL but not really one that people in the US have even heard of so that's nice but definitely not helpful on my team we have a couple of people who don't have master's degrees but most people have master's we have in the past had people who had PhDs it certainly doesn't hurt but it's not a requirement I don't think my team would hire someone who doesn't have a bachelor's just because they don't hire people who haven't had teaching experience and I think one is a prerequisite to the other but certainly in my company PhD is not required not required now maybe for the pre bachelor's level ETS is very formal in this respect we have all the specific roles the research scientist must have a PhD even though they would often be open mind to what area PhD is in but PhD must be there but we have positions for research engineers and research associates and those two require master's and then I believe positions in test development might only require bachelor we don't hire without bachelor but we have a number of internships I've posted some links into the chat thank you yeah similarly Duolingo will specify if a master's is required or PhD is required but something that I think surprised me about coming to the company was that within sometimes for certain roles it's not so much about that you need the PhD to do so I have colleagues who are at different points of the career ladder based on their educational background so I'm a curriculum designer too I have a colleague who has a master's degree who does extremely similar work to me just with slightly fewer responsibilities who is a curriculum designer one and then I have colleagues who are senior curriculum designers who again we do really similar work just with they have slightly more responsibilities than I do so in that sense you might try to hire for a band within those a job announcement might say master's is the minimum and then if you have a PhD or PhD plus five years of experience you might just get hired on those different points in the career ladder but do similar things essentially so that's another thing to keep in mind is that there might be jobs similar to what Anastasia mentioned where there is a certain degree minimum you might have jobs that are like a PhD minimum but you also have jobs where you might fall just on a certain range in that ladder and still be encouraged to apply so that's one of the reasons why if you see like master's is minimum, PhD desired or something you should still apply because they might just be looking for someone who could be in that first role or first tier of the career ladder but still do similar kinds of work so that's a great point okay so my next question is what do you wish you had known about your industry before you graduated so maybe like what skills would you have acquired what classes would you have taken would you have started networking earlier any other hindsight for me so I only started thinking about ed tech my fifth year my final year of grad school when I I guess that was technically my first job in ed tech was freelancing for I think it was Amazon doing like a freelancer thing a gig on kind of like a natural language recognition checking so they needed to hire like a cheap grad student to do it was the first time I realized that I had any skills that were transfer whatsoever to anything related to tech I thought I just had a baseline had to know how to code and I just didn't for this they just need someone who knew linguistics and compared their human linguistic knowledge to machines and I immediately tried taking like signed up for the class of my department in NLP that we just hired someone and previously again I thought that's going to be way outside of my wheelhouse and I really enjoyed it I was finding it like super interesting and very transferable and then I just had to drop it because I had to write my dissertation that was more important at the time and so I just I wish that I had started earlier but at the same time I have been able to you know find this career in ed tech that you know didn't require me getting in I think if I could go back in time I would just say don't be afraid to don't feel like I'm pigeoned home I wish I could have said don't feel like I have to do the certain thing in a certain way because I always have but to say to be more open to a breadth of things to trust myself more that like ed tech is a possibility for myself that just because I don't have this background encoding or technical stuff doesn't mean I can't be interested or explore these things and kind of as has been mentioned like casting this wider net for yourself is I think something to and being creative something that I personally encourage I wish I had done a little earlier on but it still worked out I think I would have been more proactive about looking for internships I had various drops but I actually never had an internship while I was in grad school and I wish I was more proactive looking for opportunities and doing them more because that's a very good experience and another thing I wish I would have done is it's different in the UK where I did my PhD in the US but I would look for more opportunities to work on various projects not just on my PhD project but you know if somebody needs a research assistant I would try to embed myself into that to learn because this gives you wider network that gives you new skills and it gives you much more to speak about when you apply for because I know when I interview I like hearing and I've worked on that and I also did a bit of this and I also did a bit of this and I got to work on that and I started working on it and it's really powerful and I have a lot of the experience about that and it kind of helped me. You guys have mentioned this breadth of skills and experience a couple of times and it's a really good point. Sam? I don't think there's anything that I would have done differently about my university experience or my years spent teaching come to me and there was a lot of uncertainty about, okay, what do I even do now? Do I want to throw myself back into teaching? And I think just taking the time to figure out what I wanted to do and like for the first time, since starting my professional life, just taking a break and like sitting with a, you know, I have to find something like that, like trying to figure it out. Shout out to Anna Marie Trester and her career camp, which I did shortly after I moved to the stage just to connect with other linguists who are trying to figure out their next steps and also just get some clarity around job searching, what even are my options? Like what can I do if it's a linguistics degree that I have in my back pockets and doing a lot of the informational interviews then and talking to people was very helpful in figuring out next steps. So perhaps one thing I should have done more of was networking earlier in my career. It's just, it's not really a thing you do when you're a teacher. You don't really need to that much. But it's actually been very useful in getting this job and then afterwards just connecting with other people and hearing, hey, what do you do? Do you do a cool thing? Tell me about what you do. Just like, you know, approach of curiosity and be open minded. Another thing I would say is that tech job descriptions are very intimidating. It is not always clear what is in last tab and what is in nice to have. And my company might be a little bit more flexible than some. You know, as I said, we don't require advanced degrees. We don't require a ton of technical experience. My experience is that tech companies care about whether you can do the job. So if you think you have the competencies that they are looking for, don't be put up applying. But there are definitely some jobs that I looked at and I was like, I definitely can't do that. And in retrospect, I think the company could have done that. So yeah, there's a lot of big words and jargony things. You will totally learn them on the job if you go into this field. I would just say, you know, if it looks good to you, go for it. Don't be put off by the technical stuff. Yeah, there's a lot of behind the scenes stuff happening with job applications. And you never know what's going to be prioritized for a given team at any given moment, for sure. I know we definitely have situations when this is probably similar to what Emily is saying, where we end up hiring someone with a master's, even though we have a lot had a lot of PhD applicants, because that person fit better with what we were doing at the time. What's an interesting project that you're working on right now? I can't tell you about the current project. But I can tell you about my favorite project I worked for the last several years. They've been prototyping an application to encourage children to read. And the concept that we came up with was a child listens to a page of audio book, and then reads aloud, and then they listen to a page of audio book, and then they read aloud. And when they read the audio of their reading is recorded and is being sent to our service where we evaluate their accuracy, their words correct per minute, their reading fluency, and then they give teachers the reports about how they're doing. And the whole concept relied on the fact that it's not an assessment, it's not a text, we're giving you a fun book to read. Initially, for the first experiments, we had permission to use Harry Potter, but this was for first experiments. Later on, we used a variety of books like Wizard of Oz, Pinocchio, some Henry stories, some Sherlock Holmes stories. And we have to actually up, I can pause the link later in the chat. But the most fulfilling part of it was going to schools, going to summer camps and observing kids interacting with our app. And then seeing how they were getting into reading, how they were loving reading, and then going back and listening to their reading and thinking, what can I do to help this child read better? And as a phonetician, I would think of things like, you know, can I help them with intonation? How can I recommend intonation? What can I do is pausing. And then I would talk to reading scientists about their point of view of what helps people on through their reading. And that was really lots of fun to do. That sounds fantastic, yeah. My current project is expanding narratives, a library of texts. Over the time I've been with the company, we've kind of moved. When I started, we were really building out a guided writing curriculum, like this is basically a guided interface to help students write essays and tips and tricks along the way. And we finished building out all of the different types you wanted to do for that. A couple of years into my time here, and since then we've been trying to move more into literacy more broadly. And, you know, getting texts on the site that kids can read and analyze and engage with. So a lot of what I've been doing this past year has been sourcing texts and creating text analysis content to go about, which is super fun for anybody who has any kind of literature background to be doing. And the other fun thing about that is that we're always trying to one up the big curricular textbooks in terms of what can we do that's a little bit more engaging, a little bit more unexpected, not always necessarily the safest options. All this for once though in literally every English language arts textbook since the dawn of time. That has been a lot of fun to work on. Great, awesome. For me, I would say that I basically in a position where I'm finally able to help improve Duolingo's teaching of Indigenous and Threatened Languages, which is something I somehow forgot to mention in my background that my undergrad and master's degree were in Indigenous language documentation and vitalization, a big focus of that in the social aspects of those things. So it's something I'm just passionate about and Duolingo teaches maybe around 10 or so languages you could say fit under that broad umbrella. And they've traditionally kind of received the fewest resources because, you know, there's more folks who are interested in learning French and Spanish, but I feel really strongly about their importance, both the speakers as well as for the world and the ecological potential of the languages have. So I'm really happy that I get to work more closely on those courses. And recently, we just launched our Yiddish course for speakers of English. And that's one of my heritage languages. So it was really fulfilling to be able to put this language that many people think of as being kind of antiquated into the hands of people, many of whom are much younger than my family, who currently speaks it to our grandparents. And getting people using this app into re-engage with this language was really deeply satisfying and wonderful. That's amazing. Yeah, and there's such a big role for these for tech in the preservation of Indigenous languages right now and getting it to speakers. Awesome. Well, I want to make sure that we have time for the attendees to ask questions. So I'd like to now turn it over to the rest of you who are participating. So feel free to turn your cameras on and ask questions. Also, in the meantime, I'll try to scroll through the chat and see if there are any questions that we missed. But you can go ahead and ask questions, if you like. Hi, everyone. Thank you for coming and talking to us. It was really informative. Recently, I've been looking at different ways that can learn different skills. And one of the things I've noticed is there's a lot of certificates offered now for industrial design, ed tech, and curriculum design, those kinds of things. So it's just wondering if it's a good idea to kind of sign up for those courses, whether they're the online, the free versions or with a local college or something to kind of like, I guess, beef up our resume, because a lot of the job descriptions can ask for that or if it's okay to just go in with a linguistics degree. So I'm not a hiring manager, meaning I don't regularly screen resumes to approve or deny them. My feeling is that that is sort of not bad. It's nice to have it, but not a really a make or break for you or for the role. I would sort of be inclined, kind of, I already said this, to look for what kind of the job description is. And if, for instance, says it needs, you know, a certain amount of experience, doing a certain thing that the certificate won't replace that, you know, but that it could maybe potentially help you to get experience in that field. I would think of it as like a bonus or a nice to have, but not necessarily something that would probably replace that requirement if it's a hard one. That being said, I don't think it can hurt. It could also give you some exposure to whether or not you like that thing with the caveat that the experience of taking online course could be very different from actually doing the thing for better and for worse. So that's all to say, maybe, and probably you'll get more a richer experience than like talking to someone or reaching out to someone from that company and getting a sense of if that course is, or companies are interested in it and seeing if that's the kind of thing that would be beneficial or not. That was a great question. Thank you, Janice. You know, if, say, Mara, I don't know if you had anything to add to that. We don't pay attention to just certificates if they're not supported by something you've done. So if in the Highland likely situation, when you are really saying that they're not the candidates, we might look at it. We might appreciate certificates as an evidence that what we are looking for is an evidence that you are not just doing what your supervisor or what your advisor tells you, but you are willing to go and explore things yourself. So going and doing other courses could be evidence of this sort of being proactive and curious and exploring other areas, but I wouldn't focus on just getting credentials if they don't lead to some sort of output on top of that, unless you just yourself want to do it. If you just want to do it for yourself, that all means going to that, of course. Yeah, we haven't hired in a while. We're going to be hiring soon. I haven't seen the job post for us that's going to be going out. I don't think we're going to be asking for any credentials at all. We're pretty non-specific about what actual qualifications people need to have. So I say, yeah, I would agree with what Emily said. It might be a class with us, certainly not. I don't know how much stock are hiring, and this is going to put in online courses versus experience or demonstratively having the skills of the job. We also tend to not hire people who have that tech experience. I think only one person on my team has previously worked in ed tech, and the rest of us came more or less out of the classroom and kind of learned on the job. Thank you. There was a question just recently in the chat about the length of the hiring. We have very lengthy hiring process in general because it's usually open position and then there is the phone screen, and after phone screen we have what we used to call on campus visit. Now it's virtual, and after that we would usually wait until they have a couple candidates so the whole hiring process can easily take a month or two. Although it depends if somebody is really strong and if they must have them, they might move a bit faster. But I would say in general, these companies like ETS, if you submit it to your CV and you haven't worked for a week, it's not the reason to despair. Yeah, it's the same. It's the exact same as Anastasia. Can vary, but a month or two is pretty normal. And you probably wouldn't go longer than a month without hearing from someone. Although things can also change, and I think someone mentioned this, but it is really, you have to keep in mind that there's so much you can't see about the hiring process. Like maybe you applied this perfect dog or an amazing fit, but you're the hundredth applicant, and they already have three people in interviews. Like you're probably not going to get it even if you're amazing. And you also, you know, someone might say, encourage you to reapply. You really liked your profile. We're so busy. We get so many applications. We're not going to keep it on file and reach out. Like that's just really unusual because there's just so many really qualified folks. So you have to resubmit your application. Even if you're outstanding and amazing, it's not a reflection of your quality. You just have to keep resending your application that happened to me at Duolingo. I submitted like at the tail end of one hiring process and got a nice email to look out for the next one. And I did. So yeah, you have to have some vigilance about those things. Similar in terms of time, but really high on a voting basis. So I think for me, it was a month from start to finish, but that was all actively in the process. So I applied and then had to go through screens and then onsite, which I guess is virtual now. Among the technical interviews, kind of job tests. Like I think I had to break down how a teacher concept on a whiteboard at one point, which was not a part of any of my previous hiring experiences with teaching jobs. So that was that was new. But yeah, I'd say be prepared for there to be a lot of stages. And yeah, similar to the volume of applications is very high. So just just be prepared not to hear back. I guess we're definitely some jobs that I applied to in Ed Tech. And I just I still haven't heard back. I guess they're not interested because it was three years ago. We could just be prepared for that too. I know like at Mango, we have a certain window where we have to at least respond to people and and then kind of but how long it actually takes in the process could be a month could be a few months. It depends. I see Wei, I see your hand up. Do you have a question? Yeah, thank you for the wonderful panel. I'm wondering how this kind of job opportunities apply work with international applicants because you know, this is a training and the education position and I am a financiation. I think I know a lot about pronunciation and contrasts, but I am not a native speaker and I might make errors in my grammar. So in that situation is how are we supposed to contribute if we're interested in this kind of opportunity? That's my question. I'm not a native speaker. Yeah, I know the list. Not bad. We usually we don't look at for positions unless it's a position which specifically says native level of English. We don't most of our hires actually are not native speakers thinking through my group. We have about 60 percent of non native speakers. We look clearly for ability to communicate and English for ability to communicate research ideas, but beyond that, you don't have to be native speaker to be a good linguist. I think being a non-native speaker is probably a great thing. Working at a many antique companies do. For a fact, the head of our function is a non-native speaker of English and it's a pro to speak more than one language and what I do for sure. And any small errors you make, I don't think that's as long as they don't impact your ability to communicate. As you said, the thing to really be more concerned about, I think, is if you have work visa requirements, if you're truly international and you don't have a green card, that's something you don't want to go through the whole process of writing a beautiful cover letter and everything if you can't get permission to do the job. So if you are international, just encourage you to reach out to the company and see what are the actual requirements because that's something they're very inflexible on. So you want to make sure that that's squared away first, unfortunately. I would add to that that some companies will hire international people remotely. My team, my curriculum team happens to be all based in the US and they do prefer people who have experience of the US context of that particular job but we have people in other roles in Europe and in South America so you never know. I had experience interviewing for a company for a position I didn't take where they had an arrangement with the university for really strong camps. So if the position is really something you want, I would try to apply for that. Recruiters would usually ask you anywhere where you stand on the visa situation. Yeah, that's true. Those are good points. Duolingo does for instance have an office in Beijing and an office in Berlin that we're just opening up now. So we have other positions but they'll generally be listed in the job title that they are, the position has to be based there. I'll just add that Mango also has some flexibility there. So some jobs are preferred in the US or can only happen in the US but sometimes they will make an exception and you know they're able to do that but definitely agree on the being a non-native speaker is not an issue. It's a probably an asset. Let's see, there's another question here from Paulina. Do you want to ask your question? Okay, sorry, I get online kind of late so I don't know if you really mentioned this at the beginning so if you did it I'm really sorry but I would like to know, I have seen sometimes some positions related to tech that you don't really have options to make a career. It's like you get into, you have your position, if you want to move forward you need to kind of try to get a new position, find a new, like trying to apply for a new job somewhere else. So what about your companies? Is it something that you can get a promotion? How far do you can go in your career in this sense if it's an option? At the TS for each of the functions we have career letters. So for example as a research scientist you are usually hired as an associate research scientist and then depending on your performance you are promoted to research scientist then you're promoted to senior research scientist and then you might be the research director or principal research scientist and I think after that you probably have to go into the exact level if that's your thing. And the same as for engineers there's also assistant associates, just engineer senior principal and for other positions too there's a natural career letter but at least in the TS it's also the case that yes you can move along your function but you can also be proactive and move into a different function or move into a different job role. This is always an option but it's up to you to show that initiative and to show that you can be in a different role. Yeah Duolingo is also very invested in like keeping people at the company and giving them opportunities to progress and get promotions if you're interested in that. So there are you know we have conversations very regularly and I talk with my manager a lot about what it looks like to be at the next level and how to get there and what the kinds of responsibilities are and we have a very clearly defined like it's called a career ladder of what the sort of next steps are kind of like when nostalgia was listing that are very clear and yes the idea of having a career at this one company is definitely something that's talked about and promoted a lot. Yeah similar at my company we have I think levels one to five for individual contributors like me. The expectation is that you're just going to keep leveling up over time I've leveled up twice in my time at the company. We have goal-setting conversations with managers a couple of times a year and you know looking at okay what's in the next level what are you doing now how are you going to level up eventually at a certain level you become eligible for people management roles or team lead roles if that is an area of interest for you. And yeah I think I think that having us continually grow and not as interested and get to do new things so yeah I think I think I would be able to stay at the company for a long time and not not feel like I had got everything out of the well but I'm never going to get out of it. It's a good question to ask during interview what are the next steps and I know that for us there are multiple ways that you can build your career you can go into more managing positions where you really don't do as much research but you manage people or you could go more into principal research scientists where you more or less like an academic research who has their own research books so you might not be managing people but you are managing the research or you could decide to go more into business where you really work with business people and do more of the product management. Alex asked if you would be able to give an example of a salary range for some of these different levels so like entry level next level etc. I'm afraid that comes. I can't give certain really good numbers because it's not something I've been set into as a manager but way higher than anything you could expect to get in academia is what I'll just say at the outset. Yeah it's at the level that I'm at it would be normal to be hired the upper five figures lower six figures if you have a PhD and then if you have experience plus PhD more you have a master's degree I would say probably around the level of an entry level professor it seems crazy but yeah that's about you can expect relative to academia. That would say always negotiate your salary. If you can my company doesn't negotiate salaries they have a very specific thing that they use I'm not going to do numbers just because my company also ties your salary to your geographic location yeah because we are always try at least. So my company San Francisco is in line with San Francisco cost of living and depending on where else you are in the country like that's calculated as a percentage of the San Francisco salary depending on if you live in Austin or if you live somewhere where the cost of living is a lot lower it's going to be a different salary as well so I won't do numbers what I will say is that the the mazes at different levels have found pretty substantial coming out of teaching. There's another question from Samantha which is kind of related we're already talking about this a little bit and I'm going to add to it as well so her question so Samantha I hope you don't mind if I read this your question is is it the norm to negotiate salary and benefits at your company and I think another related question is whether if you can speak to asking for raises as well and promotions. So like Anastasia said you should always negotiate at the outset that's normal and expected once you get the offer right not before that so once you have the offer there's kind of you do want to ask around other folks that you might know to get a sense of the range but it's good to aim high that you're not going to get your offer retracted the worst will happen is they'll laugh at you and be like no sorry like that's way too much but we can do this right they're not going to just make the offer or anything like that the you can also something similarly great advice I was given when negotiating was that you should always negotiate for things that aren't just salary because there's also other things they might be more flexible on that could be relevant to your life like commuting benefits or stock options if it's a startup those things I asked about some of those things do lingo like for instance PTO and stock options were actually not flexible but some other companies they are so it's worth to ask about all the range of benefits not just the base salary in terms of asking about more I think was that part of your question like more asking about salary after the fact it's pretty clearly laid out how your bonuses and raises and things work once you're at the company and they're also the company's pretty transparent about why people get the money they do and how they work to be equitable across gender and salary and range and things like that so there are routes to talk about it if you have questions but it would probably be something you're just like say out of blue that could be seen as kind of gauche or not not appropriate but there are definitely ways to talk about it like with your manager for instance if you had questions yeah and I don't think that's always the case that that's transparently laid out so that's that in of itself is really helpful to know that there's a system there that's clear usually your first conversation with the company based on my experience would be with an HR person and they are more than willing to answer the sort of questions how promotion works how raises work these are open very company specific and very role specific same I think you spoke to this a little bit already I don't know if you had anything else to add yeah my company won't negotiate so and that's for equity reasons because as we know some people tend to get involved in negotiation but others do so there's very specific guidelines around who is paying what at what level in what geographic location and promotions and raises go go hand in hand if you level up you're going to be getting berets accordingly similarly benefits are they are what you get they're pretty good benefits I don't remember there's a ton of room to negotiate on those I will say that my company is pretty good at working with people when they ask for what they need so you should ask things like you know how we have a flexible slash unlimited PTO policy question mark it's a normal flexible than anything I've ever had as a teacher so don't don't be afraid to ask questions about like hey how does this work like one of my options I think coming out of a very rigid structure like working in schools I didn't even really know like what was available to me as a as a professional in the industry so ask ask any questions I would say about what what the rules I know I met for the teach like everything is rules in my head but like policies are because you might be surprised basically by what's available to you that's where it also helps to maintain your network and to grow it because this is where the best solution would be to reach out to somebody else who works there and ask them to tell you how it is to work on that company and ask all those questions well we only have a few more minutes I just wanted to ask you if any of if there's anything that we haven't talked about but you haven't been asked that you'd like to share or mention before we go I'm thinking one of the mistakes I commonly see people commit when the interview is that instead of being curious and learning about the company people come focused on their PhD and the person just talks and talks and talks about what they've done for their PhD without looking at our company website and seeing what we do and another major thing that happens every now and then as we always ask to have any questions for us and people will say have you done x I have this great idea you haven't done but of course it's right on our website first thing and this is not a good impression to give so do your homework don't try to come across as smart nor old but rather try to learn as much as you can about the company before coming to the interview that's something that trips up people surprisingly often well definitely I agree with Anastasia just said uh do you do your homework especially if you get the in-person interview you definitely want to come prepared and knowledgeable um but also for writing that cover letter too that's a big yes or no that would kind of make or break you um and I would say that just yeah I would reiterate the value of collecting a lot of experience that was something I did um I did a professional development program for ALT-AC when my last year of grad school so check on your campuses and see if there are things of that nature that really prepared me to feel confident to even think about myself as um a non-academic linguist um and I do still think of myself as a linguist or someone in academia um so and then yeah those informational interviews I did collecting that network like um other panelists have mentioned um really kind of collecting experience and not being afraid to reach out and ask questions um something I really try to do is do these kinds of outreach panels and respond to people who ask me questions on LinkedIn because I did so much of that when I was a grad student um and I was so confused about how all this stuff worked so I think many people you'll find are very receptive and very kind and want to help you out too um so yeah just cast that wide debt and um stay curious yeah I'll second what Emily just said because I was going to say the same thing um transitioning into industry one of the things I had to get over was like reaching out to people I don't know to ask questions because between being an introvert and being from a culture where gosh I couldn't possibly impose um you know actually I was surprised by um people being receptive just to talking to me about their jobs so if you see someone who has a job that you like would look on you know just and they reach out worst-case scenario is like they're like uh no um but you know you're not gonna burst into flames if you reach out to someone um on LinkedIn like Emily I found that I had a lot of help from connections I made early on in living in mistakes and I'm happy to pay back forwards one thing I would say is that it's worth reaching out for information interviews before you apply once once you're in the pipeline it gets a little bit more complicated for people who are already at the company to talk to you um because I don't feel somehow a little bit unfair if you're already in the process um so I would just say maybe if you if you do want to learn more before you apply do it do it earlier than later just if it doesn't create a weird conflict but yeah yeah don't don't don't be afraid to connect with people yeah that's great advice well thank you all so much for uh for your your answers in this discussion today I I think this was really interesting I hope this was helpful for everyone