 Hey everybody, you're here for the interviewing like an interview. We're interviewing like a unicorn. How to interview. I've had a lot of great teams. My name is Alan Grant. I'm the co-founder and CTO of pirate.com. Here we have Aileen Lerner. Aileen is, actually Aileen do you want to introduce yourself and kind of tell us about how you got into the whole interviewing.io? So I've had a windy path. I started as an engineer then I dabbled in recruiting for a while trying to fix some of the things that are broken which many of you have experienced some of those firsthand. And most recently I'm the founder of a company called Interviewing.io that seeks to make technical hiring a lot more merit-free. I'm Chekyu Duran. I work here at high-end. Hi, I'm Aileen Percival. I'm the CEO of women who code. And at women who code we provide diversity consulting. So best hiring practices, best practices around diverse minor pipeline for our sponsors currently. And we're hoping to expand that further in coming year. And my name's Obi Fernandez. I'm the CTO of Andela. We are training 100,000 young Africans to code and to be awesome people and improve the continent. And my function I'm in charge of recruiting respectively those 100,000 people. So I'm doing a lot of interviewing. And I'll be moderating the panel. Thanks for having me. Thanks to everybody. Thanks to all our guests. I think we're very interested in finding out from the audience basically what you're looking to get out of this session because there's a couple ways that you could get interested in being here now versus the other sessions. And that would be like, you know, raise your hands if you are in this session specifically because you want to learn how to interview as an interviewer. You know, meaning you're looking for work. Okay, so it's scattering your hands. Then raise your hands if alternatively you're here as an interviewer and, you know, an employer looking to, you know, get some insight on that. Okay, so it's useful to our, you know, panelists to understand how to structure the information that you share with the audience today. I think that one of the major themes we wanted to touch on during this panel is one that's near and dear to my heart because on the softer side of what we do, you know, I think it's very important to bring a dimension of empathy to this whole process of interviewing. So I wanted to give the panelists a chance to discuss their thoughts around, you know, how empathy applies to the recruitment process. What do you want to go first? One of the challenges that we see in candidates is that we don't help the candidates who they best move forward. So essentially it's like a process where you want to see the best of the candidates that are coming in and employees know who they are to actually help them manage the expectations or give them guidance on what you are looking for specifically because every company values different traits differently and hire one of the efforts that we have to give them an interview guide clearly explaining each round what we are looking for that we help the candidates showcase their best skills during each of those rounds. At what stage do you give them that guide? From the beginning. So right after they apply? Yes. Okay. And just for clarity, they may be confusing because there is a distinction of applying to get a job at hire and applying to use hire. So the way that hire works is that you create a profile and lots of different companies will reach out to kind of interview you. With Chakra Cycle specifically, when we interview candidates at hire, we've gone through several iterations of our interview process and initially we had this process that wasn't very empathetic. It was a little bit brutal where we would, as soon as we started talking to them, we would throw coding challenge after coding challenge and we would just really, really get down to trying to figure out if somebody's going to be a fit. And then when we get to the end of the process, what would happen is that nobody wanted to come work for us. They would get to the end and say, well, great, thanks, your interview process is kind of rough, but I'm not sure if I want to come here, you know what I mean? I'd be the right culture fit. And so then they would say, well, what can we do to fix this? How can we improve this? And so we actually started, you know, basically doing a phone call with everybody before the interview process starts just to kind of get familiarized with the company and then talk about what the process looks like. I called one of the founders. I think things kind of go a lot smoother. And then Charlie put together this interview guide and the interview guide basically says, here's the kind of questions we're going to ask. Here's, you know, what our culture is all about. You know, we do TED, here's how we approach things and there's a way of getting people to practice that. I really like this idea of having a program that sets expectations and creates transparency through the process. But it's also important to think that while you're going through this interview process, especially if you're in a growth phase as a company, you're probably doing this a lot. Whereas the people coming into this situation, they're only interviewing once every couple of years. And so they are coming to this without the experience and without having necessarily done this every week or several times a week for the past couple of years and so creating that transparency in the process. And then having the process will also help you evaluate people across the board more equally. A lot of programs has been covered already. I guess one thing I'll add is that in this market, I'm sure you guys have felt this, labor certainly has a lot of power. So great engineers are in very high demand and there are companies that still have to wrap their head around this. And as a result, they're making people jump through very early in the process. So there are companies that will send out soulless coding challenges and then very good people know that they don't have to jump through those loops and that company will no longer be a contention. So maybe that's a very different way to think about empathy, but think about how many options the people that you're talking to have and be mindful of the fact that you have to be selling very early on. So maybe having people talk to a founder is a very good idea. But don't treat people like they are going through a rolling door on an assembly line because you're just shooting yourself in the foot. I want to really fund the themes that came up here and try to make it a little more actionable for our employers and others. So I've got with me, they already have a process. You mentioned, Alan, that your interview process was brutal. How do you gain that understanding? How do you go from whatever process X that you have now to one that is more methodical? What are the steps to determine and change? I really like the way, actually, I named it when we were engineers and put it yesterday and saying that like anything with development, it's also a negative process. So you start out with something that works for you and you can just keep iterating on it. I don't think that it's necessarily the best approach to try to nail the perfect interview process right at the beginning. When you have an interview process, what are the things that you're trying to address? One, you try to actually make sure that your process accurately figures out who would be able to fit for your team and who wouldn't. So if everybody gets through your interview process, then you need to be very empathetic that it doesn't actually do you any good. But then you want to actually provide a positive experience so that you can get to know somebody in the right context and sort of bring them on board. And so the way that we started out was with just a few problems, a few questions and we would ask them on site and we would bring somebody in and we saw that that was working pretty well. That was our first process. We had a few questions and we asked them on site. Then over time we said, you know what actually, we can ask one of these questions in advance on the phone. What were those questions just if you remember? Yeah, so with us, the first question is an algorithm question. So it's not, you know, it's not a hard algorithm that you have to come up with but it's more like, here's an algorithm, I have to code this up. And then the second question is an old design question. So in that case we'll give them a game of some sort so that somebody at least familiar with, like Mindsweeper and I'll say, so, you know, write this case of thing in any old language that you choose. What do you do if they don't know the game? Well, so the easy thing about Mindsweeper is you can always sort of pull it out and say, so this is Mindsweeper. Here's how you play. I kind of walk them through it. One thing that actually I'll do on the algorithm as well is sometimes if somebody, you know, really can't get their head kind of wrapped around the actual algorithm, then we'll sort of walk through and say, well, here's the algorithm that you implement. So it's really not about knowing the rules and being able to kind of put it to the top. So I want to walk back a second. So there has to be a process. So you should treat like development. It's an interim process. So what is your test? You know, how do you get that feedback to know whether the process is succeeding or failing? So actually those two questions I thought were like related to the interview process itself. Has anyone tried like kind of surveying the candidate to see how they felt the interview went? Is that something that maybe would work? Well, so one of the main things that happens on the platform that I'm working on, interviewing IO, is technical interviews. And we always survey people after each interview and ask them how do you think you did. And we also ask their interviewer how they actually did. And I've tried crafting these, you know, perceived versus actual performance. There's no correspondence at all. People have no idea. When people bomb, they tend to think they did okay. And when people do fairly well, they tend to think that they bombed. So is there a hopeless situation? I think that surveying people about their experience is probably much more useful than about how they did. But I think probably the biggest test is whether they actually accept the offer or not. We also do this that way. But we also call people and if we don't hire, hire them on our team and ask them what can we do to make your day better and get a lot of good information and can't respond to you. What are some of the more interesting stories that have come out of that process? So one of the interesting stories that came up pretty back at BBC was, you know, we had a half an hour coffee session in the morning and people come outside. We had a few process times with four of our engineers so they get to talk with the company and learn more about culture and so on. So that really helps them get a sense of how big the environment is and they will project more of their skills during the entire day. So were you an informal kind of thing in the beginning of the coffee sessions? That came from... Exactly. It was feedback that they liked that. What are some of the things that you've heard that they didn't like? I've often had feedback that when you have several days before you hear back about something, you know, you start to kind of psych yourself out about that company or sort of come up with reasons why that company is not a good fit for you because you're wondering, you know, are you going to, like, why haven't they given you an offer yet? And so, one, if, you know, you're going to decide on Thursday as a team what to do about it, let the person know that, hey, Thursday is when we're making this decision and we'll let them know by Friday morning, something like that. So it creates transparency in the process so they're not like, oh, why haven't they responded to me yet? Or just respond to them right away. So sometimes, like, we have a separate set of recruiters at our company and it's a little bit of a technical interview. I can't make a promise like that to the candidate because I don't handle things. That's something like the recruiter will handle a lot. Now, I'll tell them we'll get back to you soon as soon as possible. So we have a comment from the audience basically expressing that sometimes the recruiters in the company, I guess it's a larger company, are disconnected from the technical staff that actually does the interviewing so it may be a little difficult to set expectations correctly. How do you, how do you bring these kinds of processes, you know, how do you make your recruiting group more understanding? Absolutely, that's actually a pretty common situation and one that we started to run into as well. One solution to that, you know, issue specifically, which is sort of recruiters not being, they try to recruit, being not as active and following up, is to be very clear who somebody's champion is until he's going to champion, you know, on the engineering side. And so at the early on in the process, that may be that whoever does the first interview, the person that gives the first thumbs up, then that person potentially says, well, along with all the other things that I'm doing, I'm going to kind of stay on top of this. And, you know, that can mean very different things in any organization, but usually that just means literally staying on top of the recruiter and saying, have we followed up? What's happening? When we first started at Hire, it's actually kind of an interesting, we've been learning experience, we started with a different name, we started originally with the name, the developer auction, and the whole hypothesis, the thesis originally was that we were creating a competitive dynamic to increase, you know, candidate salaries in case they help people, help the market clearly start to happen. What happened was we did see a competition, but it was on salaries. Companies pretty much knew what they were paying and they weren't comfortable going outside of those ranges, but companies started to compete on speed. So early on what would happen is because the batch lasted a week, sort of Monday through end of the week, people would log in on Friday and put in kind of the offers before the week ends. One thing they would see is that all the best people that would reach out to them would be put in their offer, they can see all the other companies that are reaching out and they see why there's like 15 other companies on this group. Let me try to get in there earlier. And the candidate chooses which interview to request and which one to say no to, which pretty much does what happens outside of Hire too, and in your regular day they cross, which is not as time compressed. And so what happens when you get to the end of that is people are just less likely to go talk to an interesting company. And so it essentially trained companies to start logging in Monday morning and putting in their offers. Well after that the competition continues where it's about getting the person to the next onsite, to the next onsite. And so I guess one of the best things that helps fix this is showing that it makes a difference, right? That when we move faster through a process we're much more likely to make hires. And kind of to address your question earlier, like how do you know if it's working, right? So I think one way of knowing if it's working is if you think about it sort of like in the original final, you have different steps like if somebody did well on the first screen and they were interested in doing that if you gave them the offer, if they accepted the offer they did well on the job. And ultimately you know if you're doing something right if you're actually able to hire the people that you want, the people that are doing well on the job. Then if you're not, then you can kind of go into that and see what's the problem. Isn't it that people aren't passionate in an interview or is it that when people pass the interview they don't want to accept the offer and you sort of get into that and figure out why. So I saw that everybody was coming in and the first thing that was happening is they were already starting to work on the old challenges. So in the surveys we heard it's like hey let's get to know the team a little more. We added the coffee break at the beginning and really just kind of smoothed out and you have a great welcome to it. How many of you are using an applicant track that says something like greenhouse or something along those lines? Anyone aware? Very few. Yeah. What do you guys think? Is that a wise move to invest in that kind of system? It would seem if you want to have data about this process. That makes sense. Yeah, especially if there's a disconnect between the recruiting team and the engineering team you can solve some of that with technology and that they're two great products on the market. One is greenhouse and the other one is called lever. And there you can just see a snapshot given point in time, how long it's taken to get from point A to point B to point C and you can also set reminders so that this person falls out and nobody follows under them. You can have the system pay you and say you've got to get on this and that will solve a lot of your problems. Fundamentally I think we still have to set the expectation that if it's the recruiting team's job to move at a certain speed it has to be reinforced and it has to come from the top but you can take away the friction around actually making that happen if you get a good ATS. How do you how do you help your team be good interviewers? Because I think this addresses both sides of the equation here because if someone's an interviewee and they understand kind of what the interviewers are how they're being prepped maybe that's helpful but I'd love to hear your thoughts on that. Absolutely. I can comment on that and also potentially give a little bit of advice for if you're in an interview process like ours what are the kinds of things that you know be positive signs. So as I mentioned in our interview process there's sort of two main questions that are the technical the main technical aspects. And the first one really assesses whether somebody in the program writes this you have this algorithm it's not a complicated algorithm but it's a little tricky to make the OO problem. In the first one what we do is to figure out how to get everything in the same page we're basically all calibrated. The way that we calibrate is that when one person does an interview if you're new if you haven't interviewed before then you sort of shadow that person and you can look at the other support cards. So a support card is where after an interview you say here's what went well here's what went not as well. And the support partner in this case is very very specific and I'll tell you exactly how it starts out. I'll give the problem to somebody and it's fairly standardizable we'll paste the problem into the actual use of the coder path the kind of screen sharing program together and we'll actually paste that in because we want to make sure that each time the question is given exactly the same way. And then the first thing that's on the scorecard is did the candidate clarify the requirements? So all of these things they're not required but they are a way of making sure they're looking at the same thing. So you know just say yes or no and if somebody jumps into it and did the problem right and can clarify the requirements obviously we're not going to penalize them for it but if somebody clarified the requirements and did some of these next things then that might be kind of a thumbs up. So then at the second part after basically did they clarify the requirements is did they ask that the root for solution is okay. So a lot of times with algorithm you can find kind of a trickier solution of solving in a more efficient way we're actually not looking for that. We're not going to penalize them for that because they count a better solution but also if somebody says like hey is root for solution okay we'll say yeah it's fine you're going to usually have very small kind of inputs so that's fine and that makes the problem much easier. And so what we're looking for there is we're basically trying to figure out not just whether somebody can code but can they do it in a collaborative environment. Now as an interviewer going in to do this for the first time especially if you haven't interviewed in a while and you don't know what's going on you don't know what's expected of you and it may not seem like you should be asking questions but generally asking questions is a really good thing like we look for people to talk out loud and get engaged and it can be a good idea to actually take a moment and just really get in line with the interviewer and get on the same page. I'd love to hear from somebody on the panel it's like shouldn't you go into an interview and actually say to the interviewer I'd love to discuss what you're you know what you're designing on you know just maybe be very very blunt. Yeah definitely being more clear on the expectations and how people are going to be evaluated is going to produce a better result. Some of the I guess innovative ideas that I've seen or would like to see tested out is some companies are taking it off white boards and actually coding on a computer some companies instead of going more towards algorithms are doing more towards like actual company code-based so real company issues and problems I would love to I don't know if any companies doing this but I would love to see a company actually doing the code challenge over IRC so without looking at the resume to see that they graduated with CS degree from Stanford or knowing what their interviewer's gender is I would love to see a company Yeah I would like so I think it was Harvard that's what my building imparted I have to use her platform come and say all that words I think managing expectations is very important even for the internet interviewer team because a lot of things what happens is different engineers are judging on different skills for the same problem and that it's much harder for any company to clear the bond because the expectations is not clear to drill down on this point just to make sure we're there so one I think might be considered best practice or at least advice that I've seen is to make sure that the interviewers have a specific thing they're looking for as an interviewee if you come in and you're across the table knowing that studying intuitions is kind of universally like a good skill to have you know wouldn't it be good to be just blunt and out front and say look I'd love to understand exactly what it is that you're judging me out because I know that necessarily that's not going to come up right the interviewer is not going to say hey by the way I'm looking to know if you're a good culture fit today I think it's much harder for the candidate to do that for a member of the interviewer to then achieve that conversation just take courage or something why do you think it's harder for them because that person is going at that point you're going to have I mean it might be a good thing but it could potentially not be a good thing but the person in the interviewee position is much more in a position where they need to be put in their best effort I don't think they should do it but it's I think slightly more my recommendation to candidates is to do that before you go to the onsite have a conversation with the hiring manager have a conversation with the recruiter understanding the interview process what the day looks like and what are the focus areas for each round I think if you have a conversation then it's much easier to kind of dice out the process and give them feedback as well as you and I with the culture of the company and how much they're putting in to the process I'm kind of curious about something actually now that you mentioned that so if a candidate was to ask that in order for you to answer you'd have to actually have a process where you know for instance who's going to do the interview and stuff like that how many of you work in an environment where there is a process and you would be able to answer that question again kind of half the room so maybe one of the takeaways from this panel is the very simple kind of process you know is there what is the best way to go down the road of having a process and then be buying a tool or you know some people like to invent their own things what's your experience with that so I think the first part of the process is basically something that's a standard repeatable process so doing some things in the same way so with us what started out as their initial process was just saying okay well we're going to always ask this question and we're going to always ask this other question and then that was kind of the early part of the process then we added okay after that we're going to have a pair of programming sessions so we can see what it's like to work with somebody and originally that was our process then after you have something where you know a lot of candidates go through it then the next thing that you might want is a tool as Amy mentioned the two that are you know getting pretty popular right now Greenhouse is more of an enterprise solution so it's a little more expensive and Lever is a very very low price point so really any team no matter how big if you don't have an applicant tracking system you can get started with that right and then at that point you know into the tool you would say well here's my stages and here's how I'm moving somebody through it and the second part for us from the process actually we had to sort of simplify it down so we had the first question we had the second question so we started to ask the first question on the phone screen well then before that we started to do the founder phone call and pre-sell sometimes we would do the second question over the phone as well as somebody's remote now in case you're going to read multiple touch points and that becomes a bad experience because you've got somebody that's like well I've been on the phone each time you have to get something scheduled and so then you sort of simplify things down but for us it's basically just doing the same thing repeatable so long enough to where you can actually see what works and what doesn't work and we do retros on this like in the general engineering style we'll actually do a retros on that retrospective where all the interview areas will get together and send you I was moderating previously Chakri is moderating that I'm curious for something it's kind of a different angle on the process aspect of it in your process are you optimizing to prevent false positives or false negatives or maybe you can imagine what I mean by that I'd love to hear some of the other perspectives on that we try to minimize false positives in some sense so it's more of an eliminating risk of hiring someone who is not actually called by it's not that we only optimize for that but I think that's more important because it's more costly on the company it's more costly on the candidate that's very important and going back to your earlier question about how do you mean hiring process successful I think it has to start with the commitment the team has to be committed to having a good process and taking feedback from candidates after all what doesn't make it a business and it has to be a top priority for the team especially and it has to be a clear champion who is accountable without that nothing gets done everyone is working so that really follows having strong commitment so I do think one of the risks with the false positives is that I work towards hiring women in a certain box so you're like oh this person graduated from here this person worked at this company this person has this many years experience and all of these boxes are ticked and it kind of goes back to that like nobody gets fired for buying IBM and you end up missing a lot of people who maybe have the alternative experience for who didn't graduate from top 10 CS school and so I think the risk in that is that people will push great candidates out of the box what's really unfortunate is that a lot of smaller companies take their hiring cues from companies like Google and Facebook and Microsoft and those guys have such strong friends that they can just have this revolving door of people so they can be very, very picky both at the top of the funnel and later on later on it makes sense but at the top they just throw away a lot of people that might be very qualified but don't look very specifically if you get the strength of your own brand and you're honest I would advise everybody to just think critically about whether you're blindly taking your cues from these guys or whether your process makes sense for you and if your brand isn't as strong as Google's then maybe it doesn't be who you to throw away everybody that hasn't gone to MIT and Stanford and you know has worked at Facebook and the challenge one of the challenges is that you actually don't know how well you're doing you only know sort of for the people that you've hired whether the person worked at it or not the people that you didn't bring on site that you passed on you never know if that person was fit or not and anecdotally so we have we have one engineer on our team that we did not take through the regular process who have joined prior to this point that process was established and this is one of our top engineers one of the most productive really really top guy and he did the test and he was very nervous and worked on it for a long time and essentially after seeing his results we would not have brought him here and that would have been such a huge mistake so one of our top engineers so the point there is that you really kind of don't know it's one of the things that we're actually trying to figure out because we've got a lot of companies interviewing candidates on our platform and we've got other companies that have done really well and so that's something that we actually wanted to try to prove to I actually have a friend that went through your recruiting process this friend was really excited by the way he hired early on and this friend had an interview with a coding talent this friend didn't feel that they did very well with the coding talent and then this friend didn't get an offer from hired I've always wondered I mean my friend has always wondered but I think the idea is that you know the coding challenges I think actually it's worth talking about the beginning of the pipeline there you know and how you actually screen like you know basically is there a chance that you're inadvertently you know failing at that level and then I was mentioning there's a lot of thinking for interviewers that do the most interview and I don't know the top of my head what does she want to do what are all the interviews what is the success rate of first you know on-site or on-screen I think people get better at like the hiring part the interviewing process the new job is one of the most stressful things in life and you're going into these situations where you're being judged by the unrealistic life experience are they gender dynamics to it as well definitely for bringing people and on-board the issues of mental plight for jobs where they have 7 out of 10 of their requirements and mental plight for jobs where they have 4 out of 10 of their requirements so I as a company is like if you can see the anyone like not having that just leave it off the job description and you know you're not suddenly going to get 200 resumes you might get 4 but hey we need that for one of those 4 are going to be the right one I also ask people to lower the barrier from application as much as possible so if you're going to ask an engineer for their resume or something like that you know if you're looking for someone who's going to have a job and probably don't have their resume pulled together they might just send you an email saying hey I'm willing to talk to you or it might have someone who completely has it together is like they're preparing for this and they want to send you their 30 page CV so like when we invite people to when we have people post jobs to the women in the community we ask that it's like 140 characters and includes an email for follow up and they don't have room to put that any 4 years of girls experience when actually you hire someone with 3 years experience because you know in that case a woman might not apply so I want to stress what she just said because I think it's really a really important takeaway that the less bullet points you put on the job description the better also I want to make a suggestion that hasn't come up kind of surprisingly but we've had over 13,000 applications to hire 60 people and you might say okay well how do you screen that you know because it's just like an unworkable load to go through we actually partnered with a company called Chlon.io and they provide an assessment test that takes about half an hour for the candidate to go through but it tests their problem solving aptitude basically IQ task management innovation and 12 different personality profile stuff so we have a particular personality type that we're looking for and we also have problem solving aptitude minimums that we're looking for so that helps us kind of cut it down I'm thinking that that process is fairly blind because when we get those results like basically you know there's no picture or gender is necessarily associated with it so it's especially blind for us looking at after we make a lot of times we don't even know you know what they are and it it helps in the sense of having a baseline expectation of how that person you know what they're capable of even if they don't interview well afterwards or even if something afterwards indicates that they might be good so curious about your experience with that sort of thing do you think that that would work as well with engineers where the market is in their favor you have a very selective program where people probably kill to get into it whereas here you have much less engaged audience and I can see people that don't well on paper may be willing to do that because this is their way to prove themselves so I worked at ThoughtWorks for four years and during the time that I worked at ThoughtWorks it was like one of the top employers in the world but it was still a hot market you know and like when you showed up for the interview they'd be like here's the wonderlite you know there might be issues with the wonderlite but it was basically an IQ test in certain ways I don't know I also think that like once you get someone to apply you can add extra layers but kind of getting those people to come in the door for the first time you want to lower that barrier as much as possible right so maybe to answer your question something you do after there's already one one thing I've noticed this sort of comes up sometime you will ask well why do I have to do this coding challenge or you know I have x years of experience why do I have to do this trivial problem and for us it's actually kind of also a way of testing out the cultural attitude so we bring in some people that are really fantastic and just crush the test and then you know might respond like okay cool like not too bad let's work through this and have a very positive attitude about this as opposed to others that say you know what I don't really want to do this and whatnot and that's what comes across and I don't know if I want to work with this person on a daily basis so I think the same thing can happen with an at-home assignment or a coding challenge as long as it's reasonable and you know the expectation has been reasonably set that we don't ask somebody to do any sort of testing or assessment until they've had a half hour phone call with one of the founders right and so with us we know that 80% of the people past that phone call won't make it to the interview screen it won't get to the offer stage but we're still ready to have that phone call in advance at the beginning because essentially it's just our way of saying like we're gonna give something to you but we do use them for us as well which basically goes through this process. One way that I've found to kind of combine both of these is to have interview questions that themselves are selling points so smart people don't really like solving toy arbitrary problems whose purpose feels like it's an idiot test you know FISBAS is kind of a canonical example of that but it comes in all sorts of variants so if instead and again on our platform the best performing interview questions once where a candidate said like I really want to deal with these people these are people I've never met before they don't see them face to face they just have a conversation. Our questions where you start with something that you've actually seen and then adapt that situation to something that would lend itself to an interview question so you can describe a scenario you've seen at work and then eventually you start writing code but you also have a person start thinking about the problems that you're solving and you know that if they're interesting problems those problems are going to stick in their heads so if you can don't care about people on tables or like beer or whatever most of the time I think they want to work the smart people and solve interesting problems so the sooner you can get somebody in front of a smart person and have them solve an interesting non perfunctory problem together the better off you're going to be. And as we're getting closer on time I want to make sure that we have a chance to take any questions that I saw on hand on this side of the room before. Yeah what's our comment again? I think it's I think we're starting to get together again. Okay so I'll take a question from the audience. Hi I'm Elisa, I'm a single go ahead custom software firm in New York and my question is just if you could address more in the very early stages of the hiring process a little more in terms of earlier recruiting tips especially more recruiting work for applicants. From the perspective of an employer? Yeah from the perspective of an employer recruiting from the earlier so we can talk a lot more about the interview process but before you get to that in terms of attracting more of a high quality candidate and especially more diverse candidates. So the question was how do you attract really how do you attract candidates and especially diverse candidates. So shameless plug women who code has over 25,000 technical women and we have job postings in the code review of everything. So I think your question is sort of around what in the the higher cost of this is due to sourcing right how do we get people into the door. I'm going to say that for me it's something that I did not know anything about and that's where this company came out of that was failing to source for many months and at this point what we do kind of have hired is we bring people in across many different ways right so from sponsoring events such as RailsConf to we advertise on LinkedIn so sometimes we can use networks like that to identify the type of talent we want at scale as opposed to having to mess with each person. Messaging people individually that can be effective as well if you find to make this a right fit and generally if you're going to do that then you want to find to make this a fit for a specific reason. Yeah I also want to advise you just from experience have a distinguishing something whatever that may be. So that is our purpose right we have an amazing social mission and that's helping us get amazing candidates in the door. People are cold emailing me saying I'd like to be a part of this. I had a hash rocket I had a beach front office and I was like hey come be a surfer and do basic code with us and do great things and that's what we were about it's basically about human psychology and having some sort of hope. If you're just a consulting shop in Ithaca and that's the only thing you have an offense but unless you go to an Ithaca and have family there why would that be your number one choice for the next one? An office right but if you have something else that's what you do and your purpose that helps you a lot. So then also on the university side a couple of things in addition to the don't have bullets that don't need to be there using superlatives women don't identify as rock stars or ninjas or the absolute best ever using words like learn quickly or things that are aspirational but still inherently positive work hard and play hard is another thing because then you're saying hey we're not only expecting a lot of hours out of you we're expecting you to use your social hours with people who are currently strangers and those are just things that don't resonate with one another. Real quick, are there any perks that you found are really good in this environment specific to where we are right now in the industry? I guess that it hasn't from what we've seen it's been less perk driven so you see a lot of things coming out sort of you know on a little bit of salaries lunches and what not but from what we've seen I'm sorry so you're saying what you're seeing is that the perks are not as important so what is important? So the main thing I have I would be like it's working on an interesting product working with technologies that they like working with the people that they like if they came down to the one most common factor I would say it's probably not that important Any more pressing questions like which you're going to die if you don't get out? How do you protect against unscrupulous they're probably reproducing what? Don't use your fingers That's easy to answer What do you ask in the 30 minute first session because it's technical or not Do you mean the coffee social? Yeah so the first part it's less of a screening it's more of a cell call so in that call we don't study it onto the next step unless you know for some reason otherwise you'd be wasting a lot of time So the idea is this is entirely a cell call for the counter's benefit so we talk about what we're doing mission, mission and so as we're wrapping up I want to just make a quick announcement so we've got a few things so when we're having a party tonight it's going to be it's going to be a legalized roof which is a really cool rooftop bar with a view of the city it's really close together if you go to rails.com slash party you'll see the details OV is going to be DJing there by addition to being the Addison Wesley editor of the rail series author of the rail display and I'm going to pass you on because it's also a DJ at 35 years I'm also the month's first party so come to our party it's 6.30, 9.30 the details are on the website we have a ping-pong tournament that we're hosting so anytime today come by the higher group in the expo hall if you win one game today then you're enrolled in the final tournament which is tomorrow and we're giving away a $2,000 signing bonus so when an mechanic gets a job on a higher you actually pay them a $2,000 bonus that is a deal they get from the companies so we thought it would be cool to give that away too $2,000 prize drift for ping-pong awesome alright I want to thank all the participants of our panel let's come around and applaud them