 Thanks all of you for joining today and today the topic let me just start my this also wait a second today my talk is about building innovative products of course this is my personal view of how to do how to build innovative products how to adopt innovation at your company and how to be personally more innovative right but there is no one right way for innovation. So I am sure you all will have your own stories to share so probably that will be covered during the question and answer session you can also ask questions you can share your ideas and all right. So with that I will just start building innovative product what I will be covering today will be examples of innovation what is innovation why innovate in today's world why is it important to innovate in today's world ingredients of innovation like how do you get in innovation working in your organization ideas for implementing innovation right like how to get it rolling in your organization as well as what is not innovation how not to do innovation right and finally I go on to how to innovate in data science field right primarily I am talking innovation in perspective of technology right I am sure that there is lot of innovation happening in all other fields right and there will be lot of things similar in other fields but I will be mainly covering technology area right. So few example of innovation right you have things like Airbnb which came up with a new business model for hotel industry or travel accommodation iPhone which use multi touch brought multi touch to people and transform the way we interact with our software with our phone right Amazon web service which reimagined how IT infrastructure works right Alexa which is a household platform for voice assistant also a platform as well urban farming spaces rocket Uber right this is just to give you a context of what kind of innovation we are talking about right and going on why is it important to innovate today one of the reason it is important to innovate today is because big tech firms may not understand whichever field whichever company you are in big tech firms may not understand your business today and therefore they may not be able to take over your business but definitely there will be many startups which understand both the business as well as the technology and they may eat you for breakfast in future so if you don't want to be breakfast of some of the startups you better start innovating right and it's definitely not something just for big tech firms right so any company any field can innovate so typically what holds people or companies back when they are trying to innovate right typically it's existing well-performing business model right which gives you a good stream of money today it it really make any changes right just for an example Nokia was very happy with their stream of money and stream of business model give them money all the time and they were just happy with the status cure they were not exploring what else could be done right while on the other hand Apple was exploring a lot of different ideas one of them was multi-touch they were exploring it for tablet they wanted to create a better tablet they looked at multi-touch eventually they said that okay we'll not launch tablet first we'll launch a phone first right even though iPad came after iPhone actually iPad was the first thing that they were Apple was working on right so those who explore those who explore different things they eventually over to their business model and are executing on the same business model all the time right so it's important to explore ideas today and why why is it important today compared to like 10 years ago because earlier innovation used to happens in lab and there used to be another team which will be go-to-market team which will commercialize your innovation over four or five 10 years time business models did not use to change so often but today the time to market for a new business model is like two years uber a bnb and it's not always technology sometimes it's just a business model which has changed right technology is a enabler for that but all these things come to market within two years so you'd really don't have to wait for a long time between innovating and actually getting the getting the thing to market and commercializing it right so given that that's time to market is shrinking every company has to think how can they become both an innovation and an execution powerhouse now you might like one of the comic characters more but basically i'm saying that you need to be an execution powerhouse and an innovation powerhouse execution powerhouse execute on today's business model and they do very well in short run they'll give you a lot of money they'll get your bank balance filled but the ones who innovate and explore will do very well in long run right and your company needs to do both of them right so now that we know what my view of why innovation is important i hope you agree with that now let's look at what is innovation in essence right innovation in essence i think we all know about scientists we know that scientists do innovation we also know scientists do experiment actually innovation is experimentation so people have some people have a wrong idea that someone is so smart that they come up with a fully baked idea which they just throw into the market and people just lap up and take up that idea adopt that idea it doesn't happen that way innovation first and foremost starts with a problem you care about so let's take apple's example apple really wanted to create a better computing interface than a laptop or a computer things that you could carry with yourself and interact without using without a keyboard so they were exploring ideas they really cared about this that they really wanted a computer which was better and because they cared so much about it they were really exploring a lot and one of the things that they were exploring about they must have tried like 10 15 20 ideas you never saw the ideas which failed but the idea which succeeded was the multi-touch idea right they started exploring about it because they really cared about that problem so really the whole innovation starts with a problem that you really care about right then it's basically a experimentation validation and and really how many experiments you can conduct depends on how much is the cost of your experiment if every experiment costs you hundred thousand dollar you'll probably only do two experiments in a year right but if every experiment costed you like thousand dollars you'll probably do hundred experiments and there's much better chance that you will succeed right so problem you care about generating good quality ideas validating those ideas very fast and keeping the cost of the this whole iteration very slow because what happens is every time you experiment you learn something new the idea of experiment is learning right so every time you learn the next idea would be better right and for all this you have to be really close to the user really really close because the thing that you can innovate best about is dependent on the pain that you understand the best the best pain you can understand can I have a can I have any answers from any of you which is the pain you understand the best yeah the one you face yourself right other one the next one is the one which you see someone in front of you facing right and the third one is the one which you the remote one is the one which you read about in a document right so really you want to be so close to the user that probably you are the user if you can't be the user see the user in front of you right or probably use analytics if you're building a product a very close view of what the users are doing how they are using your product every experiment you do you should see whether the user are happier or they are sadder after that experiment right and and very important something which is often neglected is the fact that innovation doesn't happen by by a process it happens because of people right people are the most important aspect of innovation and that's why you really need empowered people who love to do what they want to do like they and they really care about the problem the same pain that you are feeling right if they don't care about the pain then like you're really driving them like resources rather than intelligent beings then you will never have innovation so very important part is empowered people right so so these are the five things right but what do you get out of these five things okay i probably cut that slide out yeah so what you get out of that five steps is an insight every time you conduct an experiment experiment one two three four five six seven eight nine every experiment makes you smarter in understanding both the pain and the solution potential solution right so so it's it's not very difficult so why should the problem be the problem that you care a lot about it's because you will inevitably fail in an innovation project and this is something that many big companies fail to recognize because big companies are used to doing projects and the projects are execution they are used to calling up a vendor when the tells them okay you do this and if a vendor tells them that oh you need to experiment for this you say why did i hire you if you are not the expert to tell me the answer i want answer that doesn't that's not the right way the innovation inevitably is failure prone you have to try 10 ideas and one of them will work it's not an execution where you're this is the requirement you have to execute it you have to build this bridge from here to here right it's not like that so if you don't care about the problem you will give up after the first few failures so it's very important that you select the right problem and if you don't select the right problem another thing is you will not give it enough resources so say suppose you are a medical company and your main thing that you do very well is inventing inventing medicine right should you be trying to create a voice assist should you be trying to create a chatbot for booking meeting rooms no it sounds very interesting it sounds very exciting you will never get enough resources for doing that why should you care your company's dna or the things that you're really good at the thing that you really care about is not a chatbot for booking a meeting room that could be still a side project but i don't mean to say that you don't have to try that but select the problem you care the most about as a company as a team as an individual right so yeah i just talked about this what's your company's teams and individuals dna very important never select the wrong problem to solve right yeah so how do you generate good quality ideas so of course this is my personal opinion and my personal experience you might have other ideas but these are few ideas for generating good ideas be very close to users very very close you should understand the pain very well and collaborate very well with them after every experiment you should see whether the users face lit up lit up or it became like this like so you should be able to see that literally one of one of the videos i was seeing seeing recently there were some people who were trying to create a app for for goggles and they went to a goggle shop spec shop and they literally were sitting there and they were as the people were trying the actual app in the shop they were looking at their experience and every time someone went back they improved the software deployed it and again the next person who came to the shop they were seeing that so that's the level of closeness i'm talking about you must understand your users that closely right and and and that's very important to get good ideas otherwise you'll end up solving wrong problem explore a lot you have to explore a lot of ideas explore a lot of technologies if you don't explore you won't get a good solution right so and do it cheaply you don't have to spend $50,000 on that and thirdly you want to keep the cost low so really if you're in tech field be in the know of the open source development you don't want to do redundant work something that has been already solved some problem that has been already solved by someone else you want to reduce the cost of your experiment start off from that base of open source rather than trying to build up the whole thing by yourself right these are few ways to generate ideas yeah um yeah as i said be close to users otherwise you'll solve the wrong problem another reason why you have to be really close to users for innovating is because as soon as i have i have the pain about some problem as soon as i write it down on a document i have already lost 50% of the information that i have in my mind as soon as i write it down so if the user to the product team the information flows like this users pain is understood 100% by the user is understood 50% by the sales team 25% by the management team 12% by the product owner 6% by the team leader and 3% by the developer right and your day to day decisions will be completely off the mark if this is the loss of information that you're having you really need to be much closer to the user right and instead if you have user who's talking to the team lead product owner and dev yes you still have loss of information 50% loss but much better than 3% reaching the last person in the line right so so you really want to be very very close to the users ideally you want to be the user yourself so i'll give you example right uh you know uh the slack anyone uses slack here yeah so you know how slack started the company which created slack was actually doing some other business and they were on verge of they found out that the their idea was not very good and they were on verge of closing off the vc uh or the funding person came in and said maybe we should take out the money which we still have in the startup and close this and cut the loss and at that time the founder of that slack said oh by the way while we were doing this our team has developed a collaboration software that they use to collaborate with the whole team on day to day basis it's fantastic must see it once and they saw it and that became slack so the reason slack users created a fantastic collaboration software was because they were solving their own problem of collaboration there is 0% loss of pain information when you are feeling the pain yourself directly right similarly uh similarly facebook take facebook for example facebook created facebook all the people were actually college student and they developed it for college students and it's not a coincidence that they understood college students very well and solve the problem for college students well first so if you actually are the user of your product that's the best case scenario but if you are not still try to use your product try to talk directly to the users right collaborate with them don't get the documents from them okay uh another reason uh that you need to be close to user is that initial any plan that your user gives you about solving a particular problem is never perfect every idea every plan is imperfect so your initial well-thought plan from user is probably only 50 60% good and you will not cut it with that plan so once you have that plan your first experiment in or iteration or sprint or whatever you call it uh prototype whatever you call it first experiment your adoption score will be very bad right even though it's 50 60 percent right but you will learn something from this and what you learn will improve your next experiment and the next experiment and and the next experiment and slowly and slowly your adoption rate will keep on going up and up and up right so in this whole process when the learning comes out it comes out because the user is closely collaborating with you either through a process of analytics where you are seeing the user's reaction to your product using if you know things like mixed panel or you could have other analytics systems if you're able to see what users are doing with your product how they are using it what they are not doing what they are doing uh or even talking to the users the learning will come from there and that will improve your next next step next step next step and that's why if if you had a a chain like this between your users and product team you can never have that kind of a collaboration with the user after every experiment you want to go through the whole chain up and down forget about the innovation thing right so how how you are close to users as I said talk to them analytics be the user of your product right yeah idea as I said explore a lot and have a long-term process every idea that you explore one in 10 or two in 10 will succeed so don't evaluate your innovation projects the same way you evaluate a execution project you are not building a bridge similar to a bridge that has been built 100 times before you are exploring a uncharted territory you will have to try 10 ideas and out of that one or two will succeed right yeah as I said be aware of open source you want to reduce your costs of experiment never drive blind use analytics talks to user take your product for test right right that's the validation step right experiment has to have hypothesis and a validation okay so as I said that the cost of experimentation has to be low and the speed of the experimentation has to be fast if you want to eventually succeed so how it does not come cheap there is a trade-off you have to put in lot of resources into building up an infrastructure which allows you to automate lot of stuff so that you can experiment very very fast every experiment if you can do it in one week fantastic instead of four months to do an experiment if you could do it in a week and that's why you need to have DevOps and any kind of automation that you can think of reduce the cost so that you can do more experiments yeah as I said no innovation happens without the people you really need people who love what they do and they are empowered you're not sitting behind the scene looking at oh how many hours did you work today like are you still working today or are you are you now just dozing off or doing something like that can I look at your timesheet and this and that you have to trust the people that you hire right and those people really have to love the work that they do right and these kind of people can actually help you build groundbreaking products you cannot get groundbreaking products by not trusting your own people right so so yeah so like agile stand-ups there should not be a question and answer session where you're trying to check were you just just not doing anything yesterday what were you doing today morning you shouldn't be doing that just the five to ten minutes you treat everyone like an adult and a person who you trust right and that helps a lot right empower them let them also it should not be I'm telling you do this sometimes people who are working in the team can come up with a much better idea than you came up with and empower them let the best idea win don't don't don't think that these people are resources for you just hand and hand and legs right don't think like that otherwise you can't innovate right okay so we saw the ingredients of innovation according to my experience right now let's look at some ideas of if you're in a big company and you really want to do a lot of innovation how will you do that right a big company has a benefit that it has a lot of people right and one of the ways that companies do innovation is they say okay we'll open up a lab we'll have some person who will be the in charge of all the innovation and this person will plan out all the innovation what are the innovations we are going to do that doesn't work very well right then this person is the bottleneck this one person can think how many ideas right and how much context can this person have no you don't want to do that what you want to do is you want to build up a system in your organization so that any one of them in any department of yours can think up of an idea for innovation and start up a quick experiment in 20% of their time right with a small simple approval maybe just the main person's manager's approval right don't require the CEO's approval or higher management approval for doing innovation experiments let anyone start an experiment at a very cheap cost that means that you must have cloud you must have easy ways to spin up a server you should have easy way to do a lot of other stuff you should give easy approval for such things like 20% of time people are trying to do so why that matters a lot because when someone comes up with an idea in your own organization they are in the same context of the problem they really care about the problem because they came up with the idea they care about the problem you're solving the problem you care about already but you must make sure that the idea is also somehow helping your company so if you're a medical company as I said and someone says I want to create a chatbot for booking meeting rooms probably you don't want to spend time on that right it should align that that thing should still align to what the company is trying to do so typically it will be things like try to think bold bold way right the DNA of your company if you are building a very big website for something and one of your team which is UI team someone comes up with idea oh we can improve the UI by like 10% or 20% the user experience by doing this thing that is the idea you want people to experiment on right this kind of idea which like is both personally meaningful for the person who has said that and is also meaningful for the company right right problem to do this but simple approval process just the person's manager should be able to approve it no need to go to director or svp or vp level to do that right so you want to do this right and if that idea comes out nicely in their own 20% time they have done really good job and it's looking promising then go on to the next step give them a good runway right out of the 100 ideas that came from your thousand employees maybe 10 came out very well out of that 20% time and then take these 10 ideas give them enough runway what runway means is that that idea will fail it's a guarantee any innovation idea will fail 10 9 times before it succeeds right you can keep that as a bulk figure you might only fail three times you might fail 20 times before you succeed but give them sufficient runway right so two shorter runway no innovation idea will succeed so give them that support of full time if it's a very great idea from 20% free time give them a full time approval right so like this is what I have talked about thousand employees 200 ideas then in their 20% time early distributed experiment then 20 ideas simple approval runway time late distributed experiment and some will be runaway success out of those late ideas that you gave the runway to few of them will be runaway success and no ceo no head of innovation could have thought of those ideas right like you look at aws amazon was running its gigantic website their engineers were already virtualizing all their servers they were already doing a great job inside someone must have said by the way if you're doing this for ourselves probably there's a mark outside right and that comes out rather than having to imagine what should we do what should we do it came out organically right so you want to you want to promote that kind of organic innovation in your organization right then those are the distributed experiments right but you want to have some big bets big bets the things that your company really depends on right like if your company is an investment company then probably big bet will be about investment how do you want to do in 3 4 5 ideas that probably high level management will approve you select the right problem again for the big bets higher people for purpose for this right you really want to hire people who live on the edge of technology who really have that experimentation mindset rather than an execution mindset and give them sufficient runway interim failures don't ask them that you better succeed because i'm giving you a hundred thousand dollars to run this project or five hundred thousand dollars to run this project don't do like that mean mentality that start with very less so it's like this if you were trying to create a new food dish right and you were trying to create this pizza pasta combination would you the first pizza pasta combination you make would you start selling it at scale first or would you start first dishing it out to your friends and family see how they respond then improve improve improve till you get a perfect dish and then once you get the perfect dish you scale it out you sell it to everyone you don't want you you want to give in that run runway to make to achieve the failures first before they get the success right so that was ingredients of innovation some ideas of how to implement it now sorry now we'll go to what innovation is not right and again this is based on my personal experience it might be different for you right so it's not a eureka moment maybe by my talk till now you would have realized it's not a eureka moment no one in the world is smart enough to just think wow i have this great idea we'll roll it out tomorrow and it will be instant success whatever idea that you have if the right idea is a vector i i hope a line like this in three-dimension space then your correct answer is this your initial idea might be somewhere here it will never be here the first idea will never be here because you will have to be extremely lucky to get the first idea right at the point for innovation so your idea will be slightly ahead you will learn from that adjust it a little bit then learn from that adjust it a little bit and eventually it will align to the perfect idea right so it's not a eureka moment if your boss comes and asks you you're very smart can you come up with an idea which will which will be the innovation of this year tell him that it doesn't work like that right smart smartest people can't come up with the first idea which succeeds okay neither is it hard planning so as i said that every experiment you do gives you some learning from the users interaction from from lot of other stuff so every experiment gives you learning how if you don't even know the outcome of your first experiment how can you plan it out up to the next 10 experiments you can't right so you really need to you can't really plan it out the users came and gave you the requirements and you planned it out so well that the innovation project was successful doesn't happen that way right you can't do that because you don't even know the outcome of first experiment so yeah innovation is this idea generation validation high speed and low cost of experiments and enough runway but how do you scale this i already talked about that those ideas yeah i would just like to highlight because this is tech talks and all of you are aware with agile lean design thinking this whole idea of quick idea generation validation and high low cost high speed essentially this is the core of agile lean and design thinking you want to prototype quickly you want to really quickly learn from things that you're doing you don't want to scale out very quickly this is agile right some people think that they have this very big project requirement they just break it down into sprints and run it sprint by sprint that is not agile if after every sprint you don't have a learning if the if the sprint did not go to the user and giving you feedback it's not agile it's waterfall in agile clothes it's a wolf in sheep clothes whatever you call it right it's not agile just by saying you're doing scum doesn't mean that it's child if you're not learning after every iteration you are not agile right so if you see high high speed and low cost of iteration that's agile DevOps but DevOps generally is only your build system your cloud deployment automated but there are a lot of other things anything related to automating and increasing the speed of experiment is essentially sort of related to agile DevOps right again you could adopt agile DevOps in a form but not in spirit it's very important that you're trying to increase the speed of experiment right not just getting the tools there right and closeness with user now this is my favorite because agile manifesto talks about being close to users talking to them directly having interactions with them but when the agile comes to big companies the first thing which goes out of the window is closeness with users we cannot afford the user to come and talk to the product team they're too too expensive too too busy they can't do that but if you can't do that firstly software scales if you did do that you can really pay the cost because your software will have much higher chance of success right so so don't throw this thing out and say that you are agile your user is not near you don't have any analytics you are not talking to the users the product team is very far from users you're not agile right so this looks like that yeah you have to see spirit and form of agile right yeah another thing about innovation which is very popular nowadays is technology people say i'm doing blockchain and that's why i'm innovating or i'm doing ai that's why i'm innovating i want to somehow use this tool i want to somehow use ai i don't know what problem i want to solve but i want to use ai right that's not the right way because what happens is it's not a problem that you care about when you face the initial failures you will retreat so what's more important is you first decide the problem that you really care about right all these big technologies they are icing on the cake if you don't have cake you don't need the icing right start with the problem you care about first typically the solutions you build will be simple first and complex later when the facebook started it used to distribute each post of your friends to you every time right later on they started using data science to like figure out which ones you are more likely to click on but they were doing pretty well even before data science came into picture so data science is the icing on the cake if you don't have cake you can't eat the icing by itself right it's very unhealthy right so it's not the particular technology right um innovation needs sufficient runway so before innovation flies it crawls it walks and it runs if you don't have enough runway or sufficient runway no innovation idea will ever succeed if you're not ready to see your innovation crawl first walk first run first you will never see it fly so typically what happens is that companies expectations are like oh i want this innovation to fly i'm going to spend so much money why will it not fly and when they see it crawling they already throw the baby out with the bath water right so that's not the approach to innovation you cannot treat them like execution projects where you already have figured out everything and you're just executing you have to be ready to see it crawl first walk it's like a baby right it starts with crawling eventually it will run and it will fly also right so you need sufficient runway okay another thing that i have seen in my personal experience is sometimes people especially in big companies they think i am a very big manager i'm actually a director and i have been asked to create an innovation project at least there must be 100 people in this project then i can make a big impact it will be this that this that so they get like 10 big managers and under them five small managers and under them like so many people and vendors it's not it's not the right way so it's like this when you are creating the first dish which is a combination of pizza and pasta you only need one cook and one person to serve people and collect feedback and maybe one or two more people but once you have found the right pasta pizza combination the dish you need chains and chains of people logistic and things like that you need a lot of people once you have found the right dish but if you start with oh i am the director i must have 100 people in this project you're never going to succeed you must start small fast moving self-sufficient team they should not depend too much on other teams they should be fast in experimenting and all those things right only once you have the final solution then you scale it yeah you don't check uh the success rate of every innovation idea so you try 10 ideas your one or two idea succeed what's the failure rate 80 percent that sounds really really bad 80 percent failure rate horrible but you don't look at the success rate of innovation ideas or experiments if you're carrying out 10 experiments only one or two will succeed and those will be the home runs they will give you big rewards right and the ones which fail right they are training for your people free training because they will become more mature after that experiment they will have better ideas for the next round so there is a win-win situation here whether your ideas or experiments fail in terrain or they succeed in failure case your people become smarter in success case you get a lot of money right so you can't judge it the same way you judge uh you judge the execution projects yeah for data science projects so basically i am a data scientist at Thompson Reuters so and i also uh earlier started a startup for a year i ran that as well so and i have been in this data science field for a while so for data science projects specifically i will say that all previous still apply you still need to have a problem that you really care about you need empowered people you need close you need to be close to the user so you understand the pain very well you need to generate idea validate idea quick iterations all the all the good stuff right learning and sufficient runway but other than that in data science there are there is again the execution part and the innovation part right the execution or the quick wins are typically uh using open source models libraries and platforms so you don't when you when you use these you can quickly uh reduce the cost of adopting data science right you can quickly get the results first right uh rather than building everything but some problems are not yet solved in these libraries and that's why you still need research right and it also gives you a better understanding of new answers of data science right so you should have self-sufficient multidisciplinary team where there should be data scientists data engineer uh api person ui person you need all those people right in a small team and uh yeah so so you have two streams here you really want to get quick wins and you also want to do research right uh yeah and you really need to get this right you need to reduce the cost of your experiments and you need to do a lot of experiments right every time you do experiment you become more knowledgeable better at solving the problem right and uh and if you get this right everything else will figure it out self itself right uh one of the problem similar to the technology is the innovation problem where people think oh i'm doing blockchain or i'm doing uh data science so i'm innovating sometimes people in data science especially the people who are earlier adopting data science they feel that data science itself is the innovation uh but data science is like the icing on the cake i mentioned this before you should have a platform ui and product analytics you should have open data and on top of this book initially was not using data science but they already had a nice product and they were collecting all these logs uh even google was collecting all the logs of queries do you do so all this log initial later on comes up and it adds the icing on the cake to make it even better right but you don't you cannot have data science just sitting by itself giving you innovation even if you are doing something like fraud detection right uh you really need a product first uh which by itself is useful right and you need to start with simple solutions uh start with simple solutions is very important for data science initially you don't have data and what data science will you do when your product has no data right so initially you start with simple rules simple solutions and you keep collecting a lot of data you keep building up your data uh data um this right uh data uh warehouse and all that right and once you have a lot of data replace introduce your ml machine learning your analytics right and replace all the complex rules that you're using in your product with machine learning right because complex rules are very difficult to manage if you have a simple rule which just works uh leave it there you don't need to put ml there right so as i said data science improves with data there are various avenues for data if you're new not using the open data you're missing out big time are you using the wiki data wiki pdr twitter if you're not using it you're missing out on a lot of data right are you using your own ui api and other things uh that is part of your product chain are you using that data in your data science if you're not using it you're losing big time basically similarly uh what happens is that okay data science basically learns from example data right uh what happened uh what is the thing that i should recommend to this person right and they have kind of looked at what other people like and things like that so uh sometimes you don't have enough uh okay enough training data so like say suppose you want to recognize all the alphabets in images right and you have very nice uh good images but you don't have bad images for alphabets so what you do is you take a good image introduce lies in it and still get it to uh recognize you rotate it share it you do a lot of augmentation of your good data to create training data for your uh for your uh for your system so basically your system performs much better in that case right so uh so that's that's the core idea your data science improves with data you must leverage all sorts of data sometimes what happens is people people are so focused on developing things for other people they forget the data that they already have in their own product uh yeah uh sometimes that happens uh because you may you may not realize what if you don't realize which part of your data science product is being useful to the users and which is not being useful and how people are using your product then you are missing out big time on this and big organization have this problem because no one is looking at the big picture and small startups would typically not have this problem right okay so with that uh I've shared my experience and my uh understanding uh it's open for question and answers yeah augmentation of data that sounds synthetic synthetic augmentation that sounds that sounds dangerous like doctoring your data uh yes that's right uh but you have to be very careful when you synthesize your uh training data so like I gave the example of alphabet a you have some good images but you don't have bad images you deliberately add noise to it so now you're creating order to recognize this uh yes yes that's right that's true yeah so when you when you augment your data synthetic uh data set uh what you have to be careful is whatever you're trying to make your model learn it should not be uh impacted by the augmentation you do right so what happens is it's very difficult to get I mean I ideally you want to get all the original data like all the bad images all the shadowy images all this images that images you want diversity in your data but if your data is not diverse for some reason you don't have resources to get that kind of data you add that uh that thing in but when you do that you have to be very careful that you don't uh impact your models learning yeah yeah it is risky it has to be done with care but everyone does it google does it uh facebook does it uh all all these people do it especially image recognition natural language people do that alone but you have to be careful what when you do that yeah any uh yes any other questions yeah so basically the data that we want to look at this conversational data right right right right so uh so I don't know if you cut through anything so a lot of our data creation right beyond what's available online and whatever right customer services actually getting people to write and create questions, right? Right. Because then you're employing people and whatever. So any, I don't know if at your work you're doing something around conversational data? We don't deal with conversational data. We typically deal with like documents and, but we do extract out some things like, if analyst has recommended that you should buy or sell something at this price and that price. We do extract that kind of information. What you're trying to say is how to create that kind of data for conversation, right? Is there like a generational aspect to it? Has anything been done, where have you done actually? For example, give you a news casing, I throw a PDF, like a product PDF, the system, and the system sort of creates questions about it. So we've been trying those last years as well, but it hasn't made a lot of success yet. So there is this model from Google called QANet. This is a question and answer system basically. But for that they actually get people to create the data. And one of the good ways is using crowdsourcing. So what you really want is to ask very intelligent questions, very distinguishing questions to understand what a particular thing was. Like I guess if you are talking about conversation, you must be talking about things like intent, what is the person trying to do, what are the parameters about that intent, and things like that, right? So what you can do is you can try to create a crowdsourcing script, which actually easily shows an image to people and then lets them select something or type in something which is your input for training. But yeah, it's a tough thing because there is no specific answer to that. Because with Google for QANet, they actually deliberately created, you might have heard of Squad dataset. So Squad dataset is a question and answers dataset. So when they created that dataset, the person who is creating the dataset is actually very, very careful in crafting the datasets questions and things like that. So what I would suggest is look at the open source things that are available, look at the datasets that are available from Google, Facebook, look at crowdsourcing your data if possible. That might reduce the cost significantly. So it's something we do for our data. We do get actual people to tag data, but we also very often go to crowdsourcing. So not mechanical Turk by itself, but there are several platforms on top of mechanical Turk, which allow you to do that kind of this. So you really have to make that transform your problem into a crowdsourcing problem. Yeah, hopefully that can help. Some basic stuff you can always make that work our way and all that things. It's not that all you have to do is put product right in front of you. You're looking for reverse Japanese. It's basically Japanese and it's human knowledge. Right. Yeah, great. If there are no more questions, then thanks. Thanks for joining the talk. Yeah, thank you.