 All right Everyone's here, and this is where you get to ask interesting questions, and I hope you understand what interesting means Difficult tough questions Politically incorrect questions All right to put these guys on the spot So can we do a quick? round of Intro in a fun way. I don't know whether I'm up for fun What's a fun way? Go on then. Yeah You better start then. All right. This is Minaj He once wrote JavaScript that was in the Selenium project. That was a big thing. He's done lots of documentation He's helping organize his conference. He used to live in Sydney, but now he lives here But I've heard he's moving back. He's on whatever the PLC is project leadership committee I thought of like the Palestinian Liberation That's PLO that's that's a PLO and also just wrong country. That's a lot more than I expected. Thanks, then Stuff that you didn't know about yourself All right left Simon Stewart described as the undeniably hairy as you can see as he is now the project late for web driver and He's been in the show for past 10 15 years now It's been a long time as he describes 2007 yes as he did Thank you To my left is Titus Fortner the undeniably now a day older Titus He is a leader of the water project. He is the owner of one of the owners of the Ruby bindings He is a fantastic swing dancer, although he doesn't do it as much as he used to because man the scene has changed Young kids dance Yeah, so to my left is Diego who is doing the docker containers And I have not been around the community enough to know much more than that in the last year We'll have to Yeah, that's and he Ries by that when I asked him about it We have Marcus. He's a professional cricket player as you can see He has been longer than What I can remember that I know that Selenium existed in the project And he's one of the first persons I met when I started joining the conferences one of the warmest and nicest person that I have met around here and Thank you. I Think that applies to Dan Dan Quear the father of Appium and someone I met in London in 2012 and Who has always impressed me with his ability to compress just? Dense number of jokes into few number of seconds. It's very impressive. So that's Dan He lives in London, but he comes from the United States awesome So the next thing is we're gonna open up the floor for questions Pooja here is volunteer to help me take the mic Pooja should introduce herself like Nourish should introduce Pooja My left one of you introduce Pooja So, hi, I'm Pooja and I have been with community when I started Using yeah started with using and then I started liking the idea of oh I can automate everything What I was testing so I just fell in love with that. I Had been a co-developer earlier, but then I became just a lover and then on I started coming to this land community Where is Anand Anand brought me to this community and Manoj and Naresh made sure that I'm staying here And then I met Simon last year So just it's kept going and and the markers as well. So yeah I'll add to that that I met Pooja two years ago when she was in Simon's fix-a-bug become a committed workshop. I remember it was before the end of the first break He said she's the one we're gonna watch out for She's the one who's gonna contribute the most this is definitely true Thank you I'm gonna start with a round of questions and then I'm gonna pass it pass the mics to other people Yeah, so this question is something I'm hoping each of you will answer You know from your perspective What was the aha moment for you in in your experience working with Selenium project like what was the aha moment for you? Aha is in I now understand how it works or I Guess it got a lot easier when web driver came along the API has made a lot more sense In the old Selenium API. That was the answer he was looking for but I'll go with that Well, my moment was Very recently debut for yesterday There's a very memorable incident Don't wish to expand it. Well, yes, my war moments was when I made the first commit the definition of web driver and The next woman was Getting the commit or access to the code base My aha moment. I think Was when I discovered that there was a server socket buried deep inside Firefox that I could use from an Extension and it was possible to remotely control Firefox from outside the process and then I realized that we might be able to do all the browsers All right, so let's see here. I think my aha moment was only it kind of was a progression I like I found a bug in something and I'm like all right Well, I can do a workaround in my test or I can oh, hey Well, let me fix what in my company's library that and oh, no, it's something wrong with water Like there was a problem in water So I fixed the problem in water, but that broke something in Ruby in the Ruby bindings in Selenium So I actually walked a bug all the way up from my test to my company like fixing the company's bug Showed that there was a bug in water fixing the bug in water Showed that there was a bug in it had to do with how wind how you could transfer between windows in Selenium And that you couldn't query whether a window was open or not in the Ruby bindings at the time And I'm like hey Yari who was the maintainer at the time. I think I found a problem. What do you think about this? He's like, yeah, write the code for it. I'll all accept it I'm just like I'm gonna write the code for this and I wrote it and And he accepted it and that was my first commit and I was super excited about it awesome I had to think hard about that one. I remember that I started Selenium and We were using the open source so far the whole time and then I had issues like setting up the grid and When I saw the internals and I saw the things that you could overload and I knew java I thought maybe I can like do some open source tool that can help with that. And that's how Contributing to open source I think I my career has been a whole series of them, but you said first one, right? I Had an employee who went to GTAC in 2008 or seven 2007 and the he saw the steel cage knife fight and he came home and he said we had just Concluded or at least reached a milestone with our three-year project of quick test professional anyone remember that one QTP, right? so we wrote we had a three-year project in QTP that was just exactly a nightmare as you think it was and We came back from GTAC and he prototyped a page object using web driver that just contained the locators not not any of the the smarts just the locators the locator map of element to locator and developer was standing behind him when he Generated that when he wrote it and the developer said we could generate that locator map for you just for free out of our Front-end code and then you can write the behavior for it in a subclass And so my aha moment was when we threw away a three-year QTP effort and replaced it with web driver Achieving feature parody test for test within about six weeks Throwing away a three three year effort. That was my aha moment whoo-hoo moment and Then for even after that the front-end tests continued to generate the locator map for us to maintain our tests for us So pretty amazing All right one more question What was the shittiest moment for you guys? Yeah, we start on that end this day Marcus is good at that perfectly transparent the shittiest moment for me is every time I realized that I am not actually contributing to this code. I am organizing. I'm doing non-code things I'm doing what Simon talked about but for me I I You know this thing is paid for my house and I have not contributed back to it except in sort of a time fashion So it's just it's it wears at me and I want to change that I think my moment was when I was checking the grid internals and then I was thinking how to contribute and I thought that Nobody else should leave this and then I try to open source stuff that could make Life easy for people and also contribute back to the tree. That was I'm facing the grid All right, so as we were transitioning to the W3C I the way that I designed the the drivers and subclassed and and the way that I I set things up I was really proud of the design work that I did and how it worked I was running tests constantly and finding all kinds of bugs and The Microsoft team said at one point that over half of the edge bugs at one point were mine And then at the conference last year Simon and Jim were like, oh, no This is how we have to do the desired capabilities and capabilities in the same handshake package with this and that and I was like The way I designed this Will totally not work with that the way it needs to I Don't want to redo everything Hey Alex you want to do this and Alex said yeah, so I guess that's kind of the shittiest and the And a relief that someone else was able to kind of pick it up when I The moment of freedom So yeah, so I haven't done much this last year Because of that moment, but gonna get back into it here shortly I don't know what my shittiest moment I think my my shittiest moment was realizing that I am terrible at estimating how long six months will be In fact, I'm just a terrible estimator of how long things will be I think in 2013 I said Selenium 3 would be out by Christmas and It it was out by Christmas, but just not Christmas 2013 And when I started the web driver stuff, I thought I'd be done in six months And clearly I have failed that I told my manager at Google that moving to The W3C it would it was we called it the Great Leap Forward because it was a five-year plan Which it breaks it which tends to break people down into people who understand communist jokes and people who don't It's not a very good joke. I'm afraid I apologize sincerely But it took over six years to actually ship that so Yeah, that's it, but the rest of the time even when things have been difficult and we've been arguing with those projects and things have been exciting and There's been all the stuff going on every single time. I'm just reminded how lucky I am to be part of this project Mine was been interesting. I should actually be feeling brus to say that but I can open up No, not really But the first one right the first one I think most of you here can correlate With my experience, right? I actually chose the hard way of contributing code to the Selenium when I chose a JavaScript bindings and Without understanding anything on promises. I apparently made an API which something was missing in the JavaScript code and Jason labor the man who maintains a JavaScript code and I was seeking his help basically and He was quite nice guy. He used to help me on the IRC and I made up something on the API And I want to get his feedback and I didn't raise a pull request instead I sent the code to his Gmail and the reply I got was amazing and that was my shittiest moment So yes, the lessons learned. Yeah, don't send the code on emails for reviews and be open in learning All right, it's hard to pick just one So I'll sort of say that it's either When I was at the Dubai Airport two years ago and I thought I was coming to this conference And I found out that the last three pages of my passport were no use to me and that I would have to go back to London And not get to come here. So that was pretty shitty But I think there's one shittier and it's like maybe six months before that I was doing the opening keynote at a conference in Berlin And I brought the tapster with me the robot and I tried out the demo before my my talk My talk was at nine and I tried it up like 7 30 in the morning. It worked. I didn't move it I didn't change anything didn't reboot my machine And then it came time for the tapster demo in the talk and it's the opening keynote So I should also mention that there was an enormous traffic jam and the Berlin marathon and like some kind of you know Ten car accident on some major road in Berlin So what was supposed to be a keynote to 500 people had about 40 people there? But the room was still the size for 500 And to make things even worse when it came time to show the tapster demo where it was going to send a tweet about the conference I Went to to run the code and of course apium through an error And I I tried to debug the error like once or twice in front of everyone and then I just moved on but what actually had happened was my Apple developer Membership had rolled over at midnight Pacific time Which is 9 a.m. Berlin and I was using a real device which meant I needed to be able to sign to Use apium and the tapster with it And so though I had tested it only 90 minutes earlier that was but 11 30 p.m. Pacific time the day before Nonetheless, I found out only two hours afterwards after debugging it that that's why it failed Awesome now we're gonna turn the floor open for people to ask questions These could be things that you've been waiting in the conference, but didn't get answers So this is a good time to ask or you might have other kinds of things that you want to ask So it's pretty free flow We're gonna time each person if you kind of take too much time to ask question like how I'm taking time now and Then we'll time you out. All right So who wants to go first? All right Hi, I am Vignesh. I am part of a life science company biofarmaceutical and I work for that Test automation team So this is not only for the leadership This is for the entire community because I have been thriving to get an answer on this in multiple forums But there have been no concrete answer on this. So since I said that it's a life science organization It's not a typical functional testing. It's a validation testing. No, it's not an oxymoron It's a computer system validation testing where lots and lots of documentation is involved Not only for the from a testing per se but from everything. So and coming into the testing part So they have been doing the validation testing believe me. There is so much that could be automated but We are finding it difficult because of Selenium being an open source They have been looking out so any tool It's not only an testing tool But any tool that has to be brought into this computer system validation environment It it goes through a quality management process from the procurement to everything and it has to be Documented like what this tool is all about and everything So has anyone in their experience seen an organization or been a part of that Where they have used to like Selenium and validated it Yeah, I have Done work with banks in the UK who are subject to Sarbanes oxley compliance requirements and Typically open source code is Something that they start being a bit wary of and then they realize that because the code is open and anyone can read it And you can build the artifacts yourself. It's actually way safer than almost anything else So Whenever I've run into that problem normally it's been a conversation and suddenly people have realized like hang on This is a mainstream tool that is a de facto standard that is used widely And it gets adopted and I've seen the same thing happen with the spring library in Java as well as as well as Selenium and Normally, there's just a sort of small bureaucratic hurdle as they do that if however you need a proprietary closed-sourced unreviewable version of Selenium for some reason I Mean the protocol is entirely documented very clearly on the w3z. You could waste man years writing your own Apis that look basically the same And then you could use one of the proprietary vendors So you could use safari driver where you could use edge driver from the Microsoft web driver And then you would have a proprietary toolchain, but I don't see what that buys you that doesn't seem to be Incredibly beneficial, and I think most organizations are reasonable when you point these things out Simon as a follow-up to that Do you believe that once Selenium 4 is released as a w3c compliant standard with all the drivers? Do you think that will be an easier case to make in a case like his? from your experience Selenium 3 already supports the w3c Specification so it should make no difference The fact that it is I don't think the concern is around whether or not the protocol is proprietary or not, I think it's because There needs to be a need to point to an individual and go if this goes wrong It's your faults your we're the ones that you're going to sue that tends to be the thing and there's this terrible fear However, if your testing code somehow leads to a major security breach It's not the testing code that got you to that point There is something else that has gone seriously wrong and The review would probably end up pointing to somewhere else, which is why tools like Selenium Thanks, I'm and probably I'll reach out to you offline and talk to you about further on this. Thank you Does anyone else want to Hello, I'm vikash part of all state. So I have one practical situation with me, which I'm facing right now so we were using the Selenium test with Windows 7 Windows 7 platform and IE 11 and It was working fine and recently they believe to update the system So they actually updated system with Windows 10 now and our browser throws the windows pop up initially Which we have to enter the user ID password then it opens up So with Windows 10 and I11 there is different kind of pop-up is coming in which is not able to organize by the Selenium as of now So do you have any solution for the pop-up which we can handle through Selenium earlier? We were using authenticate using some user name password I Used I tried with Robot class it it is working, but it's again Works no focus, but auto IT is not working. Can I just ask one clarifying question? Yep, which is is this like a HTTP authentication dialogue versus an operating system level operating system or dialogue. I Mean, this is a Selenium conference If we go to win dev we might be able to help but it was being handled by Selenium with Windows 7 and I I Like UFT has given the solution is they have different add-ins for UI automation like that which what was this I? Mean this I don't I think this is a debugging session now It's not so it's not basic authentication. It's some OS level modal that is somehow I'm not understanding you are Talking in OS, you know, it's like you know when you open any kind of pop any kind of browser with any URL You know sometimes you know Windows throw the pop-up where we need to enter it might consider as a Authentication pop-up for I browser or maybe URL for pop-up for the URL where need to enter the user name password That application opens up so it was working with different one of the method which is Selenium provided authenticate using But it is feeling no from your description It actually does sound like it's probably one of the authentication dialogues that you get yes Yes, and and the basic authentication dialogue is there's this basic authentication authentication or NTLM yes There isn't anything in the web driver specification for handling that at the moment and You used to be able to get away by putting like username Cologne password at and the URL you were going to that ability got stripped out of Internet Explorer quite a long time ago Coming up at the Leon Face-to-face session for TPAC the w3ct pack one of the items on discussion is handling authentication dialogues like that So there's nothing at the moment But it's definitely a known issue The reason why I was in it wasn't in level one is because we just didn't have time to get around to it Even though we got it had six years and one workaround that I have used successfully is to Authenticate using a rest API get the token and add it to the browser via a cookie So that may or may not work for your particular situation But that's definitely something to to see if you can and actually for anyone that's doing any kind of authentication test your login in one place and every time after that if you can authenticate Without having to go through the browser and then use a cookie it will speed up your tests Each test by 10 seconds give or take I'll try using the test API. Thank you Good evening. This is a question regarding the drivers I know you've mentioned that we move the drivers to the vendors themselves right in some cases It's good like you know Microsoft is doing their part by updating it more frequently and getting quite a few updates and things like that but How do we kind of like deal with? Company like Apple, you know, who doesn't update that much and you know, we basically were waiting for them I mean, do you have any insight into that or any? thoughts on that I don't know if that's entirely fair and and here's why the Safari driver version is tied to the version of Safari and That only gets updated when Safari gets updated and Safari doesn't do a major release very often But they do do every two weeks a technology preview release and in every single technology preview release They have moved that Safari driver forward to be more capable and and more compatible They are the third Browser vendor to ship with W3C compliance enabled by default So they are moving faster than Google are with Chrome And I know the people who are working on it care passionately about improving it and moving it forward and making it better the problem isn't so much that it's That the problem is that Apple is an opaque company and it's hard to see this But if you look and see the progress that they're making they are moving forward incredibly quickly Particularly since like the Safari team as I understand it is a fraction of the size of the Google Chrome team And so they've got fewer people to focus on moving this forward But they are doing a stellar job and if you care about this Fari driver right now the tech preview is definitely the thing to be using Thanks. I'm here Simon one basic question here. So And in the the starting of this conference you basically said So in the starting of this conference you basically said You know, I mean you showed us statistics in terms of the contributors and all of that, right? So What are your? guidelines advices you know Tips and tricks whatever it is you normally in terms of What would you basically tell people who want to start off with contributing and I mean I have I mean I've seen a lot of people Have this question in terms of I want to contribute. I don't know how to start Where do I start and all of that so anything that you guys have in mind in terms of guidelines tips and tricks? I maybe we just go down the line Sure, so I Think documentation is always a good place to start that being said my first contribution was Building sort of an alternate reality for Selenium where it controlled mobile apps And then once the code got converted to a programming language. I was not familiar with I was less able to contribute at the level I was when it was written in other languages And so I started doing things like the graphical ease interface and I did that using cocoa and Objective C and other technologies I knew and C sharp and windforms and that kind of thing and then once those sort of got rewritten in Javascript as well Now I'm largely just a mascot or a spokesperson These days so I think there's like a number of ways to contribute so anything from answering forum questions to writing documentation To creating alternate open source projects that question the very meaning of the open source project you're trying to contribute to Yeah, as Dan mentioned documentation as a first place, which also we mentioned earlier and seeking some help on that We hugely appreciated if someone can step forward on that and more than that. Yes, we need some help Potentially on organizing the conferences be in the program committee. So we do have two conferences every year And yes, we might need some help there and coming to the other part Where yes, I had also had a lot of people asking on how can I contribute code? I think to me if you can clone the selenium repository and build a selenium jar for yourself I think you are one step closer and contributing to the code and that experience will actually tell you What selenium is and we'll have a lot more experience on back I'll give you some practical tips The first one is open source is about scratching your own itches. That's that the description we used to use Many many moons ago find a thing in selenium that you think could be a little bit more polished a little bit better And then put together a pull request to fix that the second thing is Everyone is really busy a pull request that is 15,000 files and we'll take two days to review We'll almost certainly just be left unreviewed for a long time So if you can make your first pull request bite size that would be great That would allow us to figure out How to respond to you in a nice way that you find meaningful and things like that And it also means that it's far more likely that someone will have the time to review your diff Pull it in and merge it and as you get more and more experienced your diffs can become larger And at some point you get the commit bit and then it doesn't matter how large your diffs are It goes straight in The third thing is we are chronically undemanded and so understaffed And so if you have a pull request that is just sitting there and no one is doing anything Please come on to the slack channel Or IRC if you're an old man like me I Realize I'm talking to the millennials here the kids. I Yeah, I mean just just log into ICQ and then Why talk? Yeah, log into the IRC channel, which is mirrored on to slack And just say look I've got this pull request. It would be really nice if somebody could have a look at it It's been sitting there for a few days. I know you guys are busy I move it forward there. So solve your problem do it in a way that is reviewable And then when no one reviews it just give us a gentle nudge My recommendation is to start with the tests Run the tests understand what selenium does Class by class what language do you want to take a look at I know that the Ruby test there could be more of them in different variations and When someone comes and asks like what can I do the first thing? It's like all right get the test run them and then when they don't do that and then ask what else they can do Like if you can't run the test if you don't understand what they're doing How can you add a pull request because I'm gonna ask you to write a test if you're gonna make a pull request in the Ruby bindings Understand what the tests are doing how they're working you'll gain a lot of knowledge about how selenium is doing what it's doing in addition what Was said I think similar to Manush that if you can try to do local meetups or go to the local meetups to share Your experience and what you have learned or if you want to learn more about selenium And the second thing is that in many cases when you are running a test or where they'll do something with selenium and you have a bug Don't don't find a workaround and publish a post that says how to go around that Like actually open open a bug an issue in the project with a reproducible case that actually anybody can run and Actually reproduce the problem you're having that's the easiest way to help to improve the product and to make things better In general, that's it. So and also when you open an issue, there is a template. Please use a template to report the issue Yeah, my my contributions to this point have been mostly my time I would definitely echo the sentiments in terms of the community involvement my first Interaction with the code of selenium was deep into my relationship with it as a user and I went to the fix a bug become a Committer in Boston and fixed a bug Wrote a unit test or two really really liked it and then found that somebody else had already merged it as part of the same workshop And so that didn't exactly put me off of writing code But that was at the point where I started following Bob Silverberg and Ashley Wilson around saying how do I get involved with? conference organization and that started taking up a hundred percent of the time that I would have spent Involving myself with selenium and now I you know they pay for me to travel all over the world two or three times a year So it's that's been my involvement and that would be my advice is to chase one of us around until you want to get Your trips paid for a couple times a year. Oh By the way when you're getting started Don't be a head against a brick wall like trying to figure things out yourself if you get stuck Come and talk to us. We very seldom bite. We're normally pretty friendly You know and we're here to help we really like some help so rather than you getting frustrated and annoyed and I can't get anywhere Just ask for help. Someone will give you a hand Hey When when Nuresh introduced the panel he mentioned that there were gonna be like, you know edgy questions and politically incorrect questions so mine is one of those which is Have you ever encountered somebody using selenium for like nefarious means maybe like DDoSing a website or like getting bad credentials and attacking websites same with apium and What would you say to such person? It's a wonderful question one that comes to mind is someone I worked with went to black hat in Las Vegas and a tapster robot in Conjunction with apium and the code I wrote to make the robot touch the screen Was using it to brute force passwords on older versions of Android phones So you would steal a phone put it under the tapster come back in the morning you would tell you the password That one comes to mind There's the classic example of Nike bought a tapster and all they did was make it go up and down and attach one of those Nike plus iPod bands to it to like test that like fake running nefarious And I don't know if like North Korea or Isis uses apium, but if they did I'm not sure what I would say I Guess if it was helping them accomplish, you know what they wanted to accomplish I don't know about yeah, I do know People use selenium for like scalping tickets and hitting ticket master and all sorts of things like that It's why in specification You'll see it and it modifies navigator the navigator object and says that you're under automation That also has a link to the evil bit RFC Because clearly if you're savvy enough to use selenium for nefarious means you're savvy enough to recompile the browser without that particular piece of functionality now The thing I'm most proud about is the reference to the evil bit is in a span that is marked as not being visible It's one of the two jokes in the RFC in the spec I did also help both google and facebook develop fingerprinting so that even if the browser is You've changed the navigator to not include web driver We could detect whether the browser was under automation because most of them have a tell Which makes it really easy to figure out what's going on I'm not sure this really helps or not, but one thing that we're working on is adding a code of conduct to the project itself And we have an organization that backs us up called the software freedom conservancy That is full of lawyers who can go after people that doesn't help too much with international crime rings But I think it could serve in the in in the future for us to be able to at least credibly say this is absolutely not Okay, as opposed to right now, we're you know, I'm not sure we say that as overtly as we could If I'm not wrong If I'm not wrong when Jason wrote The first version of selenium. He wrote it to basically fill time sheets That's exactly what it was for. Yeah, so that was not really for testing It was it's really to know he was developing the time and expenses app that ThoughtWorks were using He wasn't filling in time sheets nefariously Before that I think I mean time sheets are heinous anyway So maybe it was nefarious. I I just don't know actually retelling me not uses Web driver all the time probably even more than we do for testing to validate coupon codes Put them in shopping apps. We we respect robots.tech So we're not doing any we announced to the site. We're definitely not doing anything nefarious But we put stuff in shopping carts. We apply coupon codes. We make sure that the discount was taken away We do that hundreds of thousands of times per day So, I mean it's a fascinating use of web driver, but it's it's not nefarious unless you don't like coupon codes It doesn't like coupon codes Well, there was there are a few people that will come to the chat rooms and ask for help automating various things that We have we have turned down assisting a number of times. I think uh Yeah, they're what gmail Ticket websites by passing cap that we're like are you going to be violating the terms of service by doing this and they're like And we're like, yeah, good luck By passing captures a big one too I love getting that question. How do I get past the capture with selenium? Yeah, a lot of them are legitimate questions asking how to get past that and the answer is still no don't Well, it's it's like the whole the capture can't be very good if web driver can answer it There's more some questions in the back there All right, we go here. Yeah So I wrote a test Which I would like to run the same functionality on multiple browsers So in shell, I choose and five folks then went with chrome then went with ie browser So what I've done is I want to run the same functionality. Just I'm trying for 10 iterations So thank god in 10 iterations seven times. I got the all the post results. The test got passed However in the last three iteration not lost in 10 iterations, maybe three iterations the test got failed And the confusion part is the test got failed at different pages for different reasons And it got under the other seven times the test got passed without any issues in the same point of time I'm in a confusion like whether I went through the locks and I'm very confused like whether through the selenium api Or is it between the intermediary between my test code and the application because the driver is sending the commands from there Or is it from my test side because he's my test flaky So I'm just thinking if I write a test which satisfies all the conditions It should pass 10 out of 10 times, right? So why selenium is doing it and it is failing in some iterations at different points for different reasons I'm still confused where to debug and how to debug it I mean the first thing is going to be the synchronization issue different browsers will load different components at different times and so W3c is going to make the driver behavior more consistent going forward But most likely it's an issue with ensuring that you've got the right explicit weight settings Are you using explicit weights? Yes, I'm using it. That's why maybe the other seven iterations it got passed My confusion is why it is failing at different reasons on only some iterations. Okay, it's I mean Dan showed the slide of me going all code is shit. It's because all code is shit And also all networks are shit and all computers are rubbish Like the problem is somewhere something isn't working an end-to-end test by definition requires the entire stack To be functioning flawlessly and perfectly in order to give you consistent reproducible results If anything happens if there's a dropped network connection if you run out of ephemeral ports if You know, there's a passing You know interstellar particle that hits the wrong bit of memory in the computer Something somewhere is going to go wrong And and all your testing infrastructure needs to be rock solid as well So in order to debug the thing that I tend to do or I used to do is When a test fails capture a screenshot Quite often that will give you some interesting information If you can record a video of the test running and only keep the videos of the failed tests So you can then go hang on a second It was hanging for you know a mega decade until it did this and then at least you can figure out Whether it's the app that failed or selenium at this point in time It's almost definitely your app It is incredibly unlikely that that pattern of failures you're describing is selenium itself Failing is probably the app. So there's probably some database access Maybe a database connection pool ran out or some query timed out or it was Tuesday And so the cleaner turned off the server racks so they could plug the hoover in to tidy the room or whatever it is Which is a genuine test failure I had I was saying that maybe the apple developer's lessons expired That's a yeah, that's a funny I mean just debug the way you normally debug gather data Look at that data and then analyze that data to figure out like what it is that has gone on And it's probably a video or a screenshot will tell you a loss Already we started like all the application was moving the development through angular j s So maybe I didn't get the answer So like since we are moving through angular j s how we are going to have Selenium to be like supporting or developing our scripts in angular j s or what is the next step we are going to take Talk to the angular team I mean seriously like Selenium is a browser automation api. We've given you an api for automating web browsers But the libraries that you run inside your your application are not part of the browser They're a third party thing that we have no control over Things like xjs like to like make up random things gosh knows how angular organize its internal state the people who know how angular xjs j query react Organizes its internal state Other developers of those frameworks. So the correct thing to do is to ask them to document the hooks you should use for locating elements That being said there's a Protractor is a tool that's provided by the angular team that will make your life a lot easier And you will notice that if you're using an angular app your test will run much more reliably much faster If you hook in the way that the protractor documentation tells you to Okay, last question and someone this uh this week that uh has worked with the protractor says please don't use protractor right now so Right that's the last question for the day Okay, my question is like will selenium ever have its own test runner? No, it won't Selenium selenium won't have its own test runner because selenium is an is a library that provides for browser automation Um, there are so many test runners out there already. There's j unit test ng j unit five Um cucumber calabash aspect n unit Like does a mocha night watch probably not the related question is well, will it do test formatting and result formatting? The answer to that is no as well Because what would happen is each tool would then have its own test results in its own format And then it would be impossible to have like a unified overview of What what on earth is the health of this application? And ideally what would happen is You'd run all your tests using a standard test runner In java world probably j unit maybe test ng n units um in in the dot net world Mocha maybe in in the javascript world Um, and then you'd have the outputs in a common format and then you could pass them in a common format and it It wouldn't matter In the rush I have one question to ask Yeah You I've got one question. Are we done? Yeah, we're done But I have one question to ask Um, yesterday you may have seen me wearing a black t-shirt with white text on it Marcus, I see you're wearing a black t-shirt with white text on it Do you want to talk about why you're wearing it? Absolutely, I intend to wear something like this at pretty much every time i'm on the stage. Um, this is to promote Um, a movement also a website Called hashtag cause a scene started by kim creighton Who is Bent on improving the state of diversity inclusion in the tech world Um, as noresh pointed out, we have 26 percent Women speakers at this conference at chicago. We're shooting for north of 40 percent and this is My way of of saying, you know as a as a privileged white male in tech that i'm not going to Stand by and and let this continue without making some room And this is a very small gesture on my part But I think it's trying to be an important one and I wanted to get more people to wear these Um shirts with simple messages The other one says that i'm learning how to become comfortable with being uncomfortable Which is what I have to do in order to make room for these discussions about diversity and inclusion Which I truly believe in if you want one a Small example tiny tiny minuscule example of why I think this is important Learn about the development of airbags in In automobiles where they were testing them with a bunch of you know Five foot ten white dudes and they were killing people who were a shorter stature than that So if you don't think diversity inclusion is important That's a tiny little anecdote that could maybe open up an avalanche for you in In this discussion, so you'll see these more on this committee. I believe I realize there is an irony that we have a stage full of men here Um and like it's something that we wrestle with We need to find a way of improving and making the project more inclusive and better And I would really love to see You know a far more diverse Group of people leading this project like if I could step down and have some You know amazing lady or a personal color or Somebody of a different sexual orientation or something step in and fill my place I'd be completely comfortable doing that Um, and I'm a bit disappointed on ourselves that we only have men up here I'm hoping that next year when we when we do this the next time we're here in india We'll have an even more diverse lineup of people sitting on the stage Where's puja? Absolutely for the first time we actually have an indian on the stage. No, we don't on the plc as well We're making some improvement. I mean we are working really hard that the Menoja is not on the plc because he's an indian. He's on the plc because of all the work he's done for the project Similarly for everyone who is on the on the project leadership committee you know Even even the russians I was just trying to highlight that uh, you know in terms of diversity We we are making a small improvement. We still have a very long way to go But I was just highlighting the fact that we are improved and leave people with a positive light that yes, we can make a bigger change But i'm sure if anyone who is interested reaches out to anyone that is a core contributor Will get lots of mentoring and support. So please Contact us if you're interested and we can figure out how we can do better with some kind of outreach Also, how do I get a t-shirt hashtag cause a scene dot com. Let's spell out the word hashtag cause a scene dot com Excellent All right with that. I think we want to thank the committers for being here with us for over the last three days Hopefully you guys enjoyed the conference as much as everybody else enjoyed Right Manaj wants to say something Yeah, I just want to say thank you. Um, we should apparently forgot to thank anand bagmar If you're here Yes, he's been an amazing co-chair bringing up this program And yeah, come on, please I think we can give a round of applause for him, please Yeah, thanks anand if you can give a word about the experience. Thank you everyone. Uh, it's been amazing. I think it's been 2014 so four five years I've been associated with the planning team I've learned a lot And I think this is one small way that I can help others as well. So thank you everyone for letting me be here Shout out to andrew krug as well Last but not the least Huge thanks to hang on. Just want to complete one thing, right? Uh, a lot of times we get criticized that we're seeing the same people presenting over and over again I just wanted to highlight that uh anand had actually a very interesting topic But he said, you know, I'll keep this as a backup if there's some speaker who drops out then I'm available But let's not put my name as a speaker You know, even though it's more than qualified to be a speaker He, you know, he kind of backed out to just let other people come in and present And I think like gestures like this make a huge difference to the community Usually it's quite unusual to see anand being here in the conference and not speaking. It's the first time Yeah, so it's going to do and yeah last but not the least Huge thanks to narration team for putting up this amazing show We really know for close please Thanks a lot nourish for pulling of we know we started late and Finally, we're having an amazing day We did it All right I just wanted to say that chicago is going to happen october 18th and 19th and that we should be If you've submitted a paper for it We should know within the next week whether or not you've been accepted So good luck to everyone and I hope to see you in chicago All right, thanks everyone Like we said before the slides and the videos will be out The slides will be out. Hopefully in the next week the videos will take about a month. So you should get that Sign my cricket bat Sign the cricket bat So Marcus wants everybody to basically sign the cricket bat as a as a momentum for him to take back home That's right. I want a wallpaper this thing So please come up and signed And I think we do want to thank the sponsors I think they have been wonderful very patient. It's not been the best event for the sponsors from the perspective of location, but they have been very supportive So I do want to thank aptly tools Sauce lab mobile labs For being part of this event and supporting us We had very last minute another sponsor info scratch Which also came in to support the conference without and you're actually having a booth or anything They said, you know, we think we like what you guys are doing. So we want to support what you're doing So here's a small gesture And so that's how the community is built and we encourage I think people have already highlighted So if you want to be part of the program committee next year Or the year after typically will do every alternative years There is a call for program committee. It's an open call anybody can join the program committee. We actually never Reject anybody we accept everybody who comes in And then based on your contribution Basically, you either stick or basically get dropped off, right? Like that's I think the easiest way To not be judgmental to start with but results will basically show who sticks around and who drops off So that's the process in case if you're wondering, how do I contribute to this conference? You would look out that in in a in maybe a year's time We would actually announce that the conference is happening and here's the call for program committee Even before we do the call for papers And so that's a way to kind of join in and help run this conference Okay So one final thing. Thank you all very very much for coming. Thank you all for your comments Thank you all for all the discussions. Um, thank you all very much for the warm welcome But most of all, thank you for being part of the selenium community Thank you