 It's a debatable topic so I said I must come well-prepared since this is an arena of war and I had promised my daughter that you know what for the next talk I give I'm definitely going to use one of your toys luckily that came in hand with the light right. So contrary to popular belief and it makes noise. So contrary to popular belief testing is extremely important and testing is overrated. Now my talk is essentially not about going in the wild and trying to be a Rambo or a Gijo or a Commando deploying code on production logging into a production server and changing it, you know what that's going to get you killed. So don't deploy without testing, fair enough, perfect. The point is ensure that you do not get yourself killed or blowing your head apart by trying to do any manual QA, right. Now this is an, it's a story that I'm talking about, it's an experience report so it makes sense that I share my experience and if you don't like that experience, you're welcome to come and talk to me later on. So this is a story about an outlook and outlook towards writing cleaner code where it's not easy and outlook towards ensuring that we have code quality. I'm sorry Microsoft, that's an awesome picture I found, I just couldn't avoid that one. And an outlook towards customers, how often do we find customers, do we have happy customers, do we like them, do we like to work with them. The morning sessions talks were very interesting. They always spoke about how do we judge a cost for a project, how do we not estimate. The bottom line was always there was a customer involved, there was a vendor or developer involved, but nobody ever spoke about the relationship between them, should it be good, bad or ugly. That's the kind of experience report that we're actually going through. So way back in 2007, I co-founded Josh, 2007 I was working for about eight years, I thought you know what, I think I can do something on my own. I wanted to do something different, wanted to, I wanted to do something different. Wanted to see if I could start a company, a very niche company. So we started working in Ruby on Rails. Now this is very interesting because in 2007 Ruby on Rails was so freaking new that I got a call from the newspaper within two months of starting the company and I was ecstatic. And this reporter asks me, hey, I have called because I want to talk to all railway enthusiasts. I go, wait a minute, hang on, no, this is Ruby on Rails is a software framework and by the time I finished saying the word framework, the phone was already disconnected. So I said, okay, this is not going to be as easy as I thought. The difference was we were doing only Ruby on Rails. Now starts the challenge. One, I'm trying to start a company which is niche, two, trying to do only that niche and apparently people don't know the difference between railways and rails. So of course, after working for eight years, after starting, trying to start a company in a niche technology, you assume this guy must have thought through things. He must have had some sustenance money, he must have had some team, he must have done an MBA, I'm sorry, all those who have done an MBA, I apologize, he must have got some financial background, well, that's how we were in 2007, like a refugee camp. We had absolutely no money, I blew it up all partying earlier when I was working. We had absolutely no entrepreneurial experience. I did not know the difference between debit and credit. I did not know how to get customers because I was a developer, I was a coder, that's all I was to do, I should write code, that's something I knew I could do well, write code, but is that enough to get customers? Of course, working in a few multinational companies, you end up picking up liabilities like a car, a house, personal credit card, free company loans and all that kind of crap, and I did it. And I realized that once I'm in the sun, whoa, I got marks to feed. What do I do? All right, I've taken a decision, no going back. Get clients, whatever pays the bill, do what it takes, so far so good. But people said, dude, you're working Ruby and Ruby on Rails, it's all open source. I was like, open source, I have no idea what open source is. This is a time that I am trying to get a hand to mouth existence. I'm just trying to feed mouth, I'm trying to keep a life. So please, I have no idea what open source is, and I'm not gonna do it. So we started hiring a few people. We started hiring people who could help us code, who could help us get things done. It's a very critical word phrase I've used, just get things done. So we had developers, of course, when you start things on such an ad hoc basis, it is bound to lead to a war. So our first world war was in 2007 itself. This war was between our QA engineers and our developers. What is the scope of work? This is not my job. I heard this so frequently. I am a developer, I'm not gonna test code. And the poor guy who's testing it says, this is not finished. I'm like, too bad, I don't know this, your problem. So if there are anyone who reads Douglas Adams here, this is the typical example of a set, somebody else's problem. So people were putting the blame game between each other, and who suffered? We had unhappy customers, we had missed timelines, we had bad code. What tests? I don't know, I don't know what tests, we didn't have tests, we need to get going. But this led to a lot of problems, but it also built around a lot of maturity. Now, the team started to mature and say, all right, I'll write tests by way, you do your way. So the developer started writing unit tests. The test engineers said, he's eating my cake. Then the developer said, yeah, what do I do? Should I not write cake? Should I not write tests? What kind of tests? Then we started diving into unit tests and functional tests and integration tests and whole lot of tests. Like dude, this is gonna be tough. But it's important, testing is important. You want to do as much testing as you want. Now, the team was matured. And as in life, has there ever been just one war? That brought us to the next year, our second world war. But this time, we are mature. We knew what we were doing, so we knew how to fight. This war was slightly different. This war was between our QA engineer and the developers together fighting with the client. You didn't give me the right scope of document. The document scope was wrong. It said, go from step one to step eight. See, I've been there, done that. Customer wanted an apple. The developer thought to build an orange. What we gave him was a watermelon. As an entrepreneur, I take care of my guys. And I hear, customer is wrong. And it did not, it brought a lot of bad blood. What we realized over time that we need a process. We need to fix this stuff. We need to figure out how we are going to get things done. So it was important that we kept doing the right things. But along with doing the right things, it is equally important of doing things right. One talks of business ethics, doing the right things. The other talks of execution. So when you have nowhere else to go and when you just want to make some money and you want customers, you start off with whatever work you can get. This order started changing in 2010. When you realize that, you know what? Let's choose the right customers. Now, we're a services company. And does a services company ever say, you know what, as a customer, I don't think you're good enough to work with me? Wouldn't it be cool if you could say that? Wouldn't it be cool? Does anyone agree with me if you are in services line or if you are on the product line and one of your vendors tells you, oh, your processes don't suit mine. I'm not going to work with you. You know, believe it or not, it works. We started rejecting customers. We told a customer that for us, quality wins over time and money. If you are bothered about, how fast is this going to get done? I'm the wrong guy to work with. Talk to somebody else. If you are going to ask me, I need this finished in this fixed budget. I don't know what I want to build, but I want to do then. This was very evident in even stock in the morning. Evan spoke about all these things. But one thing I did not see in the talk and I don't know if I'm right or wrong here, but this is an experience. We started saying no to customers who said, it's a fixed time, the fixed cost. And lo and behold, things started changing in 2010. In 2010, we had the new order. We said, no fixed cost work. Every time a customer came to us, we gave an estimate of the budget. The boy out there for the talk before lunch, Vasco's talk about estimates. So when he was talking, he said, estimates, you should never have estimates. I went and changed my slide. I said, all right, man, let's keep in sync, man. We forecast the budget. Well, for all the practical reasons, I agree with what he's saying, but I really need to know how that works in practice. And I agree, fine. It's impossible to get a perfect accurate estimate. I think in fact it's an oxymoron, accurate estimate. But getting a forecast, all right, man. We can never forecast anything, but it works. We also decided one additional thing. Let's stay only in our area of expertise or where there's innovation. And if there's neither expertise nor innovation, is there something our team can learn from it? So these are the three words we kept with ourselves. So in eight years, since 2007, our team has grown from a measly two people, me and my co-founder, to a massive 30 people. And if I have a choice, I'll bring that to 20. We have struggled. We have struggled a lot to keep our numbers down. We are struggling because we want to ensure that we are doing quality work. I don't want to grow in numbers because the moment we start growing in numbers, then I go back to our third world war. I'll have to start cutting corners. And I want to build a team which actually does this right. Now, easier said than done. But when we were prepared to take this risk and call it quits, we gave ourselves two years to say this is the new order of way that we are going to work on. And if this doesn't work, you know what? I don't think I can work again. I've been an entrepreneur for eight years. I don't know how many of them have been in a startup and gone back to work. I'm not sure I can be an employee again. But I don't care. We give ourselves two years with the new order. We make it happen or we'll go down fighting. But we also upgraded our tools. We said, that's it. We're not going to test anything. We are going to have no QA or QA or testing people in our office. Everybody is going to code. Everybody is going to test. Now, we started adding a lot of project management tools. I mean, agile project management. I had to use the word agile somewhere in this conference. I don't know why. We started doing automated code quality checks, continuous integration. So we got rid of all our old tanks. And no offense meant to any of these tools. This is what we used to use for the first three years. They're perfectly suited where they are, but they were just not for us. So we got rid of these old battle tanks and got our new Oakley kick ass new tanks. Okay, I think my daughter put that slide in. Well, actually, to be honest, I was searching for on Flickr. I searched for latest battle tanks. And this came up. And I said, I just couldn't resist putting this slide up. Well, back to the topic. We got our entire new arsenal with using these kind of tools. If you look carefully, most of them are services. Not a single one of them is installed on your machines. So at my office right now, we have no server room. We have no infrastructure. We have a nice, strong, heavy lease line or only infrastructure that we have invested in. Everyone works of all these tools. Now, some of them are pivotal tracker tools. There you go, I have a marker in this. I have a marker in this forward. I have pivotal tracker, Trello, which are good project management tools. Has anyone used GitHub for project management? For project management? It's interesting, but we've actually used GitHub for some project management. It works for very, very niche customers who say, yeah, please get my job done. I have no idea how to do it. I don't care about the technology. I need a solution. And there when we are tracking it, it works. But that's built only after a level of trust that, you know, Gautam, if your company's doing this work, I know they'll do it right and they'll not make an ass out of me. They'll not try to make money out of it. The other tools for continuous integration code climate and Slack. Slack, I need to put a question mark there because I got an email today that Slack actually had a security alert. They got hacked in February and there's a blog post out on Slack today. It's talking about a security alert. But Slack is awesome. This is where the new age arsenal is required to help things out. Now, we bought enterprise licenses for most of these. Now you might think I get free accounts on all of them. What is the need to actually go and buy enterprise accounts and if you need to buy enterprise accounts, why should I buy them? Push the cost onto the customer. But no, we bought it because just like testing, code quality, test coverage, it's equally important for us to ensure that we handle the code quality. We control the tests. We are interested in the quality assurance of our code. This gives our customers a lot of confidence that all right, these guys are doing something right. In fact, it's become so weird that we've been teaching some of our customers now. This is the way to do it. Sometimes you find a customer like, you know what, you keep the code and we have to educate them saying no. The one thing that you want is intellectual property on the source code. So never give the keys to anybody. Control it. And they're like, no, it's okay. And they're like, nope, gotta do it our way or the highway. This particular phase helped us out too. When we were doing project management and I'm sure you all have seen Pivotor Tracker before, it has all these, I like it because it's a one page thing. I'm not going to get into Pivotor Tracker promotion activities but you have a current and a backlog. The backlog's pretty interesting though. No estimates. That's another thing I learned in the morning. And we realize that we've been doing this for a long time. When we give a forecast to a customer, we don't give him a fixed cost. We give him a budget. This is gonna come to like maybe three months and X amount of money, but plus or minus. It's fine. We have even had the case when we have finished work before time. And when we've worked before time, we've actually got a customer giving us lesser money. Great. And he says, are you sure you're doing this? Because this is my budget. You're taking lesser than the budget. And I'm like, that's fine. Because we work less. We've finished it faster. You take the benefit. The benefit wasn't trust. We got three referrals from that customer, from other customers, which is God's more business. So we ensured that we were using the right project management tools. But again, a practice that we learned. And we work with a lot of startups. We allow our customers to pause iterations. By this I mean that, you know what happens when you're testing a tool, you're testing a product, you put out a feature, and your customer tells you I need to see what my customer feedback is going to be like. I don't want to do any more development. I don't know if I'll have more development for you, but I want to retain the team. We pause the iteration. And our iteration is about two weeks. Sometimes we pause the iteration for a month. That means two iterations. It's okay. What does my developers do during that free time? No, they don't write test cases. Test cases are part of the work. They write open source tools. They contribute to open source. So even though my team is actually working on things, they're also giving their open source contribution. Sometimes even 100% of their time is going only on open source contributions. We had started an initiative in our office for open source Fridays. Anyone who's anyone is free to come there and we sit the whole day just doing open source activities. We tell our customers, we work four days a week. That fifth day is an open source day. It helps developers get out of your project, think of something new, learn what others are doing, come back, improve your systems. This is the way we actually went and evaluated various other continuous integration tools, seeing that we set up free accounts with them, seeing where our test coverage can work, what tools work, and this is how it actually helped our customers. So bang, this was approved. The next part was about continuous integration. We use CircleCI because it's a hosted service. Any build, anywhere, I get an email. This email goes to customers. That's my customer out there. It goes to a customer and this customer gets a build failure mail. He's happy if the build fails because he knows that somebody's doing something. Of course, the person who broke the build is never happy because I'm around. But approved. The third part we started doing is code climate. Code climate helps me check the test coverage and I get an email like that, hey, the test coverage just went down by 0.6%. Okay, not bad, if it goes down by a lot, I am going to raise some fingers. Now, the talk here is not about which tools to use, how to use, what to test, but it's about code coverage. In that same email, I also get an amount of code coverage and 83.2%, not bad. We consider 85% over 85% test coverage is good. Now, this was good to go to, but just because I have continuous integration in place, just because I have test coverage in place and I have some project management tool in place, is it enough? I need to check my code quality and I get another email. If anyone gets, writes some code which degrades the code quality. The code climate checks on JavaScript, on Ruby, on PHP right now, so you can get all these tools, you can get a code quality metric and now somebody gets killed, done. Of course, when we are doing all this work, even if we have a lot of test cases, even if we have everything, there is nothing called a free lunch, right? Some conditions always apply. This condition is a question raised early in one of the talks about UI testing. What do you do if you have to test the user interface? And that is one testing which we leave, which I believe is best left to the human eye. That's a cat. I'm just trying to see who all of y'all have really got your eyes open, okay? But when you're looking at user interface testing, the visual appeal, nothing beats visual appeal like humans do it, look it, see it, what it looks like. We use tools like Bootstrap to ensure that Angular material or Angular, we use these tools to ensure that any front end framework has cross-browser platform standards, it has good compatibility with CSS, and all this is so far so good, but the final touch requires human intervention. Now, in some time, I'll explain who actually does this particular manual testing. We have these tools, browser stacks, or slabs, which do all the cross-browser testing, the visual testing, but again, there has to be some manual intervention. Having said this, we got around to changing the way we work. It's not just about everyone coming together and coding. Everyone coming together, writing your test code, figuring out and hoping for the best. Just because I have my code coverage in place does not mean things work. So we started doing no daily stand-ups. Again, in agile conference, I don't know how interesting this slide will be there, but we don't do no stand-ups, no emails. Our iterations are straightforward. People just put the tasks in Pivotal Tracker or Trello, and we just keep doing this, but we do not have a very agile, intensive process. We keep our customers. Our customers are completely allowed to talk to our developers. We don't have any team leads. We don't have any project managers. Our customers talk to our developers, and there's only one person whose head rolls, mine. So if something goes wrong somewhere, my neck is on the line. And this way, we ensure that if we've logged everything, we attach this to a new concept, which I don't know if this is part of agile, but agile billing. We have agile billing. We don't charge our customers any advance fees. Why should a customer not pay for something that he's not sure of? So we say, you know what? I'll put my money where my mouth is. We'll do our coding, we'll do our testing, and we'll show you that we are actually good enough. So if you're not happy with it, compare us. We've not had a failure case till now. This also helps us in being very proactive. By not having any estimates in our current iteration, we can actually go back to our customers and say, you know what, instead of using this particular approach for the same task, I'll use a different approach with better performance. Should I or should I not do it? I think it should be done. I think it will be done. And the customer's like, okay, who do it? I'm glad somebody's being proactive and coming back to be telling me, you know, it should be done. But who's gonna pay for it? They're like, we are gonna do it. I'm not asking you, I'm telling you. And after the iteration's over, if the performance of that particular feature is good enough, why will the customer not pay for it? Now, once we start doing this, we start building a huge level of trust with our customers. Because now the customer knows this is one guy who's part of my team is not gonna take me for a ride. When he says, my code is completely tested, it really is. And this helps us keep everyone on our toes because this is a metered experiment. That means we build for our time. So if there is actually no tasks in the iteration, we are, the clock is still ticking, the meter is still ticking, the customer pays for it. So the customer knows, all right, let's get them some work, let's ensure everything's in place, and we have to find out that the customer's always happy. Now, everyone turns out to be a winner if the customer values the developer's time and the developer values the customer's money. Once we start doing this, we are all winners. And, you know, it's a war. There are no winners in war. Nobody wins. So I had absolutely no idea how to conclude my talk. So I said, who better than to get the Dalai Lamar to say, well done, partner. I'm open for a few questions. Thank you very much. I'm sorry, could you just use the mic? I'll just repeat it for everyone else otherwise. What are the test automation tools that you're using? I saw many numbers there, but I'm not aware of the test. So I intentionally did not write any testing tools, but we use a lot of RSpec. We use a lot of Cucumber. We use Jasmine for front-end testing because we are a real shop, essentially. But RSpec, Cucumber, from the heart of the universe. And Jasmine on the front-end side. Yes, sir? It's good, okay, even I like small teams. But eventually, you know, as a consultant, okay, for all of your advice for teams like 100 plus or 200 plus. So how do you like to scale this model? Right, so for 100 plus teams, it has two factors which actually matter. One is the core company culture. So once you, like for now, we don't have any hierarchy. But for the largest teams, you definitely have hierarchy. But if the push comes from the top down, that I want to ensure that there is guaranteed to be only automated testing. Everyone has to test and everyone has to code. Everyone must have the code quality metrics in place. And the people at the top start looking into it. It actually starts percolating the culture. To give you an example about how we actually changed the culture back home at Pune because it's open source Friday that we started. Initially, it was just an internal thing. Slowly, we started getting people from outside coming in for that work. And people used to come and tell me, hey man, can you please do this on a Saturday because I have office? And we told them, you know what, this is supposed to be part of your culture. Your bosses and his boss's boss and your company's customers should all be aware that if you are using open source tools, you should have some time to contribute back to open source. So if the change comes from top, I'm pretty sure it can be managed in larger teams. But I don't have the experience of managing that in larger teams right now. Follow-up question, see when a code quality drops because of a developer, so does everybody in the company know that this is going now? Absolutely, you have to be embarrassed. Embarrassment works. In fact, it's not just in the company, this mail goes to our customer too. So we just keep it open. All right, if there are no other questions, thank you very much. Thank you.