 Hi, good morning everyone. I'm Puneet Kala. I'm working in Oracle in Bangalore office, but I'm here to share my experience that I've done automation, how I've started automation for open source organization. So I don't do automation for Oracle, but I do automation for Joomla. So in this session I'm going to talk about how we started using Selenium in Joomla. First of all, how many of you have heard or used Joomla before? I'm going to show a small video, just a two to a half minute video, in which they're going to explain what Joomla is. And then I'm going to start sharing my, I'm going to share my story. Where did we start from? What was the need? What was the requirement? And why did we switch on to Selenium? Where did we start from? We started from Selenium R.T. Then we switched on to Selenium WebDriver. We built our TechSoot. Then what was the limitations that we faced? So it's just going to be an experience sharing session. And then later on, I would like you people to start. I would even like you people to share your experiences. What are the difficulties that you people have faced in the past with automation and Selenium WebDriver? So before I start, I would like to show you the video that I was talking about. The new download every two or five seconds. Joomla holds a unique place in the open source transparency. The only non-corporate bank community driven CMS platform in the world. Joomla also implements modern programming standards using object-oriented programming and wrapping things up with a strong MPC design pattern. Let's look at Joomla's lead playing numbers. Joomla.org has seen fast paced growth since its beginnings in 2000. They have seen assets with 64 languages currently supported. Joomla has funded universal boards and has one most active global source input. With its own NINIS magazine, featuring over 465 articles published in the last few years and hundreds of thousands of topics, posts, and matters in it for. 2006-2009, they won 111. Joomla made ads for the first 400 groups of 6,000. Once you start with Joomla, you have sent it to the world for several months over 9,400 extensions available from Joomla Community as extensions for Joomla.org. Seeing how it was to grow 8% of the entire internet by Joomla, chances are some of the sites you visit every day are unmarked by this powerful platform. So, are you ready for those who Joomla.org for more information as we get started? Basically, the idea is... I'm not trying to say to you that we need people working on Joomla, but I want to motivate people to start contributing, not just in Selenium, but to start using Selenium and help other open source organizations. Now, how many of you over here have contributed to some other open source organizations? Is there anyone? Okay, which one? And apart from him, anyone else? Just like in the video they were talking about, if you look at the percentage of the web that uses CMS, most 23% of the websites that are there on the web use WordPress and Joomla, it uses around 3% of the websites are built using Joomla. So, we've got more than a million downloads every month. So, with this, we are trying to show that there are a lot of people, those who have started using Joomla, there is a lot of developers coming in into the community. And moreover, if you look at the contribution that we have had in the past 10 years, it doesn't remain constant, it keeps on changing. This is the first graph shows the contribution on the yearly braces and the second graph shows the frequency with which the code changes. So, if the frequency of addition and deletion increases per week, the chances of getting a bug into your product increases. And because of all these reasons, we people thought that, we people at Joomla thought that we need something to test our system, because for an open source organization, it's very difficult to have people sit and do manual testing. Sometimes you don't get time to even do manual testing and that is the reason why we wanted to shift on to the automation, automation testing part. So, most importantly, if the size of your application is huge, you need automation testing in your application. That is as simple as that. So, when did we started using Selenium in Joomla? Selenium in Joomla came into picture in 2010 by E.N. He started writing Selenium scripts, Selenium RC scripts. And then, but the framework that we were using was not maintainable. And we were finding it difficult to maintain the scripts with frequent UI changes. So, in 2012, Mark Texter was the guy who started using Selenium WebDriver. We had a good framework. We started using PageObject pattern. And then, because we wanted that our scripts and our test suite should be maintained easily. Now, before I explain this, how many of you over here are aware of Google Summer of course? Anyone else? Has anyone heard about the program Google Summer of Code? No? Okay. See, basically Google Summer of Code is the largest open source program that Google has done every year. There are a lot of open source organizations that take part in that. There are a lot of students across the world that participate in this program. And Joomla is also one of the organizations that takes part. Apart from Joomla, there are a lot of other organizations that take part. And there are a lot of Selenium projects that are there on the Google Summer of Code project. So, if you people are interested, and if you really want to contribute in open source organization, if you start from Google Summer of Code, you should look up for the organizations that are taking part. What are the projects that they are taking, that they have? And that is when we also started bringing in contribution for Selenium testing in Joomla. Our first Selenium testing project that we had in Joomla was in 2013. I was a student in that project. I did a lot of work. I had almost done the 80% of the backend part was done by me. This year also we had another project on Selenium WebDrivers. The main purpose of that project was to increase the code coverage on the front-end part of the application. So, with two Google Summer of Codes, what did we achieve? We achieved almost 80% of the backend coverage and mostly the major front-end part of the application is covered using Selenium WebDrivers. In our test suite, we have approximately 200 plus test cases. And that is fairly enough, but the problem is the test cases are so huge, it takes around 11 hours for our test suite to complete and it has become a pain for us. So, we're looking for solutions. What is the structure of our test suite? I'm not going to talk a lot about it because I think you people already know. All these things, we are using a page object pattern. There are page classes, there is a test class and there is a WebDriver API that we are using. However, we are not using Java, we are doing it in PHP. So, we are using a third-party library that is called Nearsoft PHP Library. How many of you have heard about it? How many of you over here work in PHP? I think I'm the only one in the conference who works in PHP. Although, so, we are using a third-party library, Nearsoft, that they provide us with WebDriver API. Then, and this is how our test suite is designed. Moreover, apart from our test suite, we also have a good reporting tool that displays all the errors and failures that we get after running our test suite because, and the reason why we needed it was, it became very difficult for us to look on to the console for all the errors and failures because for 200 plus test cases, we didn't add any reporting tool. So, we wanted someone to develop it. So, we did this, we did develop this tool this time in Google Summer of course. Then, we also were able to integrate Travis. How many people use Travis for their application as an integration, continuous integration tool? Okay. So, we were able to do that this year. So, with Averipool request, we are able to run our system test suite and look for the results. Now, there was another requirement that we people started facing, and that was, we have extensions for Joomla, and people wanted to implement their test suite so that they could test their extensions. So, we tried to change our test suite such that it can be implemented by any person who is building Joomla extensions so that it could help them so that they can test Joomla as well as the extension that they are building. So, we had to do some changes. Basically, what Joomla extension is. Joomla extension is nothing but those are the extra features that you want in your application that do not come by default in the application. So, you write the plug-in, you write the hook, and then that hook is being used as an extension. So, there are more than 8,300 extensions as of now in Joomla extension directory. The first thing that we did was, in every extension project, we told them to add Joomla as a sub module, a Git sub module, and then when a person will fork an extension, they just need to update the sub module and they'll automatically get Joomla CMS in the test sub folder. And then the automated test which did the installation of Joomla would also do the installation of the extension along with Joomla, so when you get Joomla installed, you get the extension installed. For building your test, the extension, all you need to do is you need to include the page classes for the extension and then just write the test for that. It can be used for any extension or it can be used for any website that uses our API or our test suite. There were a lot of limitations that we started facing. The first and the most difficult limitation that people faced was that our code in Joomla was written in such a way that people never used ID attributes in HTML and they rarely used. Sometimes they even try to, they even duplicated the ID attributes and it became very difficult for us. Our test became slow and it became, the maintenance became, became a big pain and an issue for us. So we thought of switching on to the WebDriver API. Then apart from that, we also have a limitation of Travis. The limitation that we people have is, Travis doesn't allow us to execute tests for more than 50 minutes and our test suite takes around 11 hours, so the only solution that we people had was to run our test in parallel. Now, the problem that we people have is, our nears of library that we are using doesn't support us to run our test in parallel and this is the biggest problem that we people are facing as of now and that is what we need. We need a tool that can help us in doing parallel testing. That is what our limitations are for current test suite. So according to, in terms of my experience, I would say the first question that we really need to ask ourselves when we start working on an automation project is functional testing really required. How many of you agree with this? I don't think so. Most of the times, we don't even think of this and we just start writing because our client asks us to do it or maybe our project lead asks us to do it but two months down the line, they expect us to show results but they don't get it and then they finally, then they finally just decide to cut off the project. How many of you fear this situation because most of us are working in a product-based company that doesn't happen to them? Now, the second most important thing is test automation strategy is the most important thing when you start working on an automation project. You need to be clear what all modules you are supposed to test. Are you going to test the complex part? First, are you going to test the complex functionality first or you're going to go for the basic or the simple functionality? Moreover, you need to be sure how many people you want on the project, how many arts of test do you want or how many test cases do you want. This is something that you need to be clear about right from the beginning. Choosing the correct framework. Now, this is again an important point that I would say you cannot just switch on to a framework just because it is very famous and there are a lot of people using it. It's a trend in the market. Codeception is a trend. I want to use it. That doesn't work. Based on your need, based on what your client or what your product requires, you need to choose a framework and that solves your purpose or solves what you really want to achieve. Plan to achieve small success and grow. Now, what do I mean by this? Basically, generally, what we do is we just start planning. When we get on to an automation project or a functional automation project, we just start planning for six months or one year down the line. In Oracle, I've seen that they hire around 15 interns and they want all of them to start working on that. I think that's not a correct approach. For me, a correct approach would be you plan for two months. You take one or two developers. You see how things are working out. It's like if a developer 5 is to 20, I would say, 5 testers to 20 developers. But it depends on the project that you're working on. There can be 100 developers and there can be just 10 testers for that on the side of the application. So, rather than focusing on six months down the line, you should focus on two months down the line. You should see how things are working for you. Are you really getting the result that you were expecting? Are you really happy with the results? And if that is the case, you should start thinking in a broader way. But don't just start thinking from the first place for a complete one year or two years down the line. Automation is not just about regression testing. How many of you agree with this? Why do you agree? Do you have any point of the gap here? Why do you think that automation is not just about regression testing? I had something different in my mind. Anyone else has a point to make? Was your question that automation is not about regression? It's also about new features or is it that beyond regression, new feature testing? Automation is a process. No, no, no. It's your question. Are you asking me, automation is not just about regression testing. It's also about new feature testing. Is that your question? I'm not asking you a question. I just want you to feedback on that. I'm not asking any question. So, are you agree with that? Why? Automation is not just about regression testing. It's not just about accuracy, performance, load, stress, anything. Right? It can be, yeah. It could be also that, in our case, there are a bunch of, you know, 40 mental testers. Whenever there is a build release, we work for a banking project where there are a bunch of 40 testers who would not spend creating accounts or various accounts which have 150 data sets combinations before they could start their actual testing, which is basically a test data generation. So what we basically help mental testers with is basically being able to create that test data. So it's not necessarily regression testing sometimes. It could be also a test data generation. But it could be anything that actually helps mental testers to realize their efforts so they can focus on something which is more important to function. Your input, why don't you agree with that? I mean, I agree with what you said, but that is automation in general. I thought you were talking about automation testing. Talking about automation, process in general. I think it can be, sometimes I would just say that automation has helped, it can help in a lot of processes as well. We had a, we were following a process in our team of documentation and it's not at all related to the product, but we used the Selenium to automate our documentation process. Yes, there was a tool in which all the details were there. We used to manually copy the details and then prepare documents. We used Selenium to do our documentation. Selenium would automatically create docs for us and then it would just fetch the values and then create documents for us. We don't need to waste time on manually creating the documents. That's not a best tool, but it can help you in a lot of other processes, apart from regression testing. How many of you agree with this point? Automation scripts just don't pay at the first release. How many of you are product managers here? Any managers? Leads? Yeah. So, we are on the first week. Yeah, you should not be bored. They should be maybe. Maybe? I would say maybe, I don't know. If we, for example, have every single change. Yeah, so, the concept of regression is when you start getting new releases and when you run your scripts and that is the time they really pay off, that is the time when you really get the results from your automation scripts. How do you think there will be some kind of change? Hmm. Hmm. Testers and developers need to be on the same team. Now, how many of you agree with this point? Why do you disagree with this? It's possible that there's no answer. What about, I think you agree with this? Yes. The reason why I agree with this is that developers need to have testers being on the same team if not in the same team. There's some problems that many of you have seen. There's this kind of mind of people that actually exist to be developers or testers. And testers basically find themselves on the same team that we found the problem with the companies that I agree with. We found the problem that was there. So, if people are always on the same team if not anything else, they get to talk to each other. But the goal there is always to be on the same team. So, if you are on different teams, so you don't get to talk to each other, you aren't just looking at each other as if you're on the same team. So, if you are on the same team at the end of the day, you may come to stay behind you, you must take both of us and get to the front. So, the partnership becomes collective. The responsibility is shared. Both success and failures have the same effect on both of these terms. If not anything else, I would say between that, developers and investors are supposed to be in the same team. They may not have the influence of each other, but developers will be able to experience in terms of how the application works or how testers need to be able to do something. Why shouldn't they expect everyone to be? But they can actually talk to each other and they can also think at some point a bit differently from the others. So, you've always been in the media, you can be a manager, you've always been there. As long as you don't make one, things will turn out right there. If it is between teams, a number of times you guys actually talk to each other in a very less... And even if you do talk to each other, it's on a very, very common basis in terms of getting to a conference room, start off the meeting, whatever. And I have seen that the more it acts in both qualities, the problem gets by each other. So, that's it. It should be understood. How many of you have this... This sort of team in which the testers and developers are working together? And who is this kind of... You have... We make a reaction from that one. So, do you people really feel that it is helpful? I think you don't agree with that. The person sitting next to you. You agree with the person sitting next to you, I don't think you agree with that. How many of you don't agree with this? Apart from her? Why? I don't think you just mix the team and the other. I think the biggest part is communication. I think when you're not communicating with the developer team and being a tester, you don't know what is coming back. I think it's the staff's nature, right? Essentially, I don't think it's taking power. So, there's been the ultimate goal in Solitude of the product. So, communication, I don't know how it fits in here, but ultimate aim is the person, the first customer of the product. So, I don't think it's even the necessary to talk to the developer and just find out so I can find a solution to work on it. If I have a solution to the developer, maybe it's on the same page, it's up around the same team. You can do a reaction on it during different teams. Then you end up in Waterfall because developers start a feature. And then when it's scheduled for release, QI finds out about it. And then you have this feedback loop where there should have been any feedback. The feedback should be the developers and the testers design this feature together, and that's iterative and iteration. Create each print, each from meeting, they're working on the same thing at the same time. And then they're trying to feature themselves actually testable and automated. Okay. How many of you are five-plus years experience here in automation? So, do you people have any suggestion or any experience that you want to share in general? Getting help on... And then you said, not have... You need that support that you're not ready to manage. Stuff like this, a G or the internet. So, you basically get all of the parts. You get all of those things, they're already sorted out for you. You just... You have to apply the part and then you have the system. So, when you have something like that, what do you have to do with PHP? The idea of our application itself is in PHP, so when 2012, the mark text was started off after hours, it took years of the library. And most of the things that we needed, we never thought that it is going to increase to 200 plus test cases and it is going to take 11 hours to complete. Frankly speaking, no one thought. We never even tried, we never even had a plan to invest two Google Summer of Code projects. It takes around six months, two students. We never had a plan like that. So, with time, everything kept on happening. Now, where do we stand today? We stand with this bottleneck that I was talking about. We integrated Travis to our test suite, but we started facing this difficulty that 50 minutes limitation is there. We tried to execute parallel tests, we were not able to do it. That is the reason why I talked about choosing the correct framework. What was our need? We didn't even thought of that. Firstly, we started doing RC, we had to change. Because there were a lot of frequent changes, our framework was not designed in such a way that we could easily maintain it. That is the reason why we shipped on to web drivers. Again, it's a difficulty, it is the pain of an open source project. You don't have people sitting constantly and thinking about it. It's just someone sitting in Australia is thinking something. Someone sitting in India is thinking something, two people decided and started doing something. So, and more I'll tell you, with PHP, I have been working on this project for two years now. I have not faced any issue as of now. We are using, we are shifting on to codeception. So, codeception uses a Facebook web driver API because Facebook uses the same PHP web driver API for their automation test and they constantly keep on maintaining it. So, that is the reason why we have not faced so much difficulty in terms of, we have not faced so much technical challenges. And I was, frankly speaking, I was so amazed when as yesterday we had a session of Anand where he was asking people about what language they are using. They were not even a single person who was using PHP. I don't know if anyone is here who has ever worked on PHP and web driver. So, if you see 82% of the websites on the web are built on PHP, that's a simple tool, I would say, but it has got a huge share. It's a big giant now, but people are not using PHP. That is the reason. How does that happen, since I started working on the web driver API sessions, there's a very occasional question that comes up on the Google e-mail. I'm seeing, please don't comment on the many questions on this. So, if you pick up something which hardly anyone on the forum is basically talking about and you start with a problem, who will you go for then? Your forum is a big place where you can be activated and you can create your goal and ask where you are. So, I'm not going to comment on that, because people ask you to choose Java. At that time, I had no question which of them would have been on the back of the forum. They were able to ask you where from Java. I think that Java, when it was on there, just put everything in there. Then it had everything that you were talking about. So, since you were saved, and you can't have anything that basically leaves you without any support, it's not going to be bigger or bigger. It's not going to be the same as that. It's not going to be the same actually. I think there's a lot of help out there for PHP, just because people have not used it before and you've not even tried to search for it. But since we are using it, I found a lot of online forums that people are using PHP outside for WebDriver, and there is a lot of help available. Mostly people are using Facebook WebDriver API. That's almost similar to what we are using on Neersoft. So, that solves our problem most of the time. So, is there anyone else who has any suggestions, experience that he wants to share with? Last but not the least one, how many of you have used the record and playback in your projects? No, it will never payback. That is what I wanted to say. Record and playback is never going to payback. I've seen people using record and playback. They have their own, they have their own complete library with record and playback and they want to... Since, because in Tumlai extension generally this happens, extensions are not huge. They can be a small extension. So, that is what they generally prefer. They don't want to hire or they don't want to invest time in building a test route. So, they generally prefer the record and playback. But it becomes difficult to maintain moreover, something here. I'm trying to say that people need this complete... It depends on the record and playback as a framework. That is not going to help them. I'm not trying to generalize, I'm just sharing my experience because that is what I've faced. I want you people to disagree. That is the reason why I've put this confusing point in my presentation. So, if there is anyone who doesn't agree with this, then you are free to speak out. So, basically what are our future plans in terms of our test suite? At Tumla, we are looking for the support from SOS or Browser Stack. We have already tried to integrate a test suite with SOS. We have support of PHP with SOS as well and even with Browser Stack. So, your point is... I don't agree with your point at all. And there's a more advanced library that is available, Codeception. And it uses Facebook WebDriver API. And then there are a lot of other modules that it uses. And it's really helpful for us. It solves most of the issues. It has a parallel test execution module as well. So, we can use that in our test suite. So, that is what our focus is. We are focusing on scaling our test. We want someone... We want to implement parallel testing. That is what our idea is. We are using Codeception and then cloud-based testing services and parallelizing test execution. So, I think more or less I am done with my presentation. If there is anyone who is interested in contributing for an open-source organization, feel free to contact me. I'll share my email ID. This is my email ID. I can help you out, not just about JUMLA. Any open-source organization that you want to contribute to and you're not sure where you want to start, where you have to start from, I can help you in that. This is my Twitter handle. This is our system testing working group. It contains a list of all the people who are involved in this. It's hardly four. One is in... The two of them are in India and the rest of them are in Europe and USA. So, I think that's it for the presentation. Thanks for coming. If there is anyone who has any questions, please feel free to ask me. Anyone has any questions or want to share any experience that you have? Open space. I didn't get it. We can tell you some points in the open space. I think there was some... Yeah. That's just to be a part of the campaign or not. It's hard to understand. I just wanted to share what I feel and I know that there are a lot of people who don't agree with that. My question is not for everybody at this time. We need to understand why you could understand that the developer and that it should not be the same team. I don't say that it should not be. I say they should be of the same team. That is what I'm saying. If you think that they should be of the same team, what are the points that you can... Yeah. Like I think he mentioned a lot of things about it. I would say there should not be a lot of communication gaps that are too high and too old. Sometimes it happens that testers are not even aware of what is coming next. Okay. You have a code release. You have a code drop. And there are just hardly two days for you left to release your product that you don't know what all changes have come in. And you don't have enough time to make changes to your script. Sometimes developers don't even really care about the results that you're getting on your automation script. How many times does this thing happen? How many times developers don't really care of the results that you're getting after running your automation test? I think there might be a lot of people who will agree with this. It happens a lot of times. That is why I'm trying to say you need to take testers seriously as well as the developers seriously. We need to be of the same team. I don't mean that we need to work together, but I mean we need to... We do not need to have communication gaps. We need to inform if developers... If the team has a plan of bringing in a lot of changes, UI changes or any other major functionality changes, they need to do a code drop maybe 10 days before because they need to give us time for making changes to our scripts. That is what I think. How would you feel when you get a new release, which has got most of the changes in the functionality and you don't have enough time to make changes to your automation script? How would you feel about it? They are automation testers. I'm just... As I am into automation part, I'm just talking about an automation tester because manual testers, it can be managed, but for an automation tester, it becomes difficult when you're not aware of what is coming next. Or even... Okay. A lot of people don't feel bad just because your application was in PHP or just need to be in the same... So did you have any kind of... No, we didn't have any kind of constancy. We could have even used Java, but that is how it started off. It's an open source project. You don't have defined... You don't have rules. I'm not asking you to define it. Yeah. I don't know it started in 2012. I got into the project in 2013, but there was something in the mind. They wanted to do it in PHP, so they thought we should do it in PHP. That's our standard that follow our applications. So they might have thought that our applications in PHP, we don't want to use any other code. I think he's talking about you. Yeah, I'm talking about you. So you said that you don't see too many questions on, like with other than jama.net. I thought you were wrong, but then many testers, I think, who will be, by the way, up to be, right? Yeah. I'm basically seeing to myself, so I almost read all the emails that get posted on the Sydney and who is full. So I'm talking about one specific forum which does not represent the entire community, right? There are a lot of people who, you know, will be using a lot of other bindings, but they never come to the forum, ask for any help, or they never come to the forums to offer help. Most of the questions that I have been seeing have all been, you know, harboring around Java, having mincopy, harboring around.net. Ruby here and there, I keep some questions around there. So I was like, you know, so whatever I'm seeing on the mailing list is what I want. So based upon that, so my entire question was, you know, so in case if you had a question on PHP, and you know, like everyone knows that if there's a question on web drivers, you go to a Selenium user's forum, but then no one is asking any of those questions, right? So I just want to understand, I mean, like when you are stuck in some situation like this, where do you go for it? Where do you get help? So at that moment, so there's not a point, you know, when you intend to start a holy war online, means I should know which one is better than which one is. Why? Because that is probably going to run for years together, and this will not come to a conclusion. Moreover, I have one more thing to say to you on that developers and testers being on the same team. What I really meant was, me being an automation tester, there are a lot of difficulties that I have faced. Sometimes the code is... I was like, so what? Sometimes the code is, there is no idea to be used. Okay, my test becomes slow. So I want to communicate my message to the developer team so that they, whenever they are writing, whenever they are doing some development, they should have this somewhere in the back of the mind that this is going to cause difficulty to the testing team. This is, there should not be any communication gap. That is what I wanted to say. There should not be any communication. You were talking about egos. Testers and developers, they have an ego between them. Developers don't want to listen to what testers have to say. Testers don't want to agree with developers. That is what I was talking about. That is what happens most of the time. But it's not your job to... It should be the team. But if the code was practically what happens in our... Okay, I think we are running short of time. So if you have any questions, we can have an open discussion. If you don't agree with any of this. Any of the points that I say, that I have told, you can... Alright. Or we can have a discussion. Okay, thank you.