 Jadi, jika kita berfikir bahawa semua perkara yang saya percaya, ini sebenarnya adalah algoritm yang sangat generis yang boleh digunakan ke produk yang boleh digunakan, jika kita mempunyai langkah internasional. Jadi, ia tidak terlalu spesifik untuk mungkin semua produk. Jadi, ada yang boleh digunakan yang boleh digunakan. Dan perkara yang terlalu filam ini adalah bahawa比如 bagaimana algoritm terhadap dua perangkatan yang ditutupkan, mereka boleh mengalami potongan secara secara berbasis yang lain. Model semera-sebirdatah membantu dapat menganggalkan perangkat komplementar, perangkat komplementar. Kemudian dengan semula akaas.. Dan ia tidak akan lelaki yang lebih baik untuk mendukung televisi item kawasan nanossy, ini akan menjadi model yang lebih pudar tidak baik duga-duga untuk membuat wajah lebih jahat dan masyarakat membuat senjata yang lebih gampang untuk mengganti dan mendukung Wayangkan nanossy di kompetisi. So itu yang saya dah berdekapan, Saya bercakap tentang algoritma yang dapat membuat perempuan peribadi. Kami sebenarnya adalah perempuan perempuan perempuan. Ia mempunyai perempuan data yang sangat basic. Namun, ia masih perempuan perempuan yang penting untuk membuat perempuan perempuan. Yang penting adalah membuat perempuan perempuan. Sebelum saya berjaya untuk beritahu yang kami mempunyai, kita boleh menggantikan perempuan yang kami mempunyai. Ini hanya untuk perempuan data yang kami mempunyai. Jadi apabila mempunyai perempuan perempuan, perempuan perempuan, perempuan perempuan mereka akan mengakutkan perempuan. Perempuan perempuan perempuan perempuan akan mempunyai perempuan. Jadi, ia akan baik untuk mereka sesuatu. So, you want to do more comparison, do more evaluation for recommendations, right? Most of them, you will have to do it as a detailer. So, we forgot to do personal recommendations. A way you can do comparison is when you are showing the component on the homepage, you compare the click through rates in real time. Sorry, online, against other components within that same page. And the other components must function the same way in the sense that they must show products that actually, when you use a click, it leads to the bloody news page. So, once we have that kind of comparison that can be done, we can run it with test and see whether showing the personal recommendations lead to higher click-through rates from the homepage to the item detail page. Whether it leads to better click-through rate than not having it there. That's the answer question. So, comparing click-through rate. Ya. So, for the homepage, we have an aggregator click-through rate from the homepage to the item detail page without the personal recommendations component as well as with the personal recommendations component that lead to every point that we get a higher click-through rate if the component is shown to the user. Hai, I'm Shuman Cho. I'm passing my master's on a list. I have one question. A couple of questions. I have a few slides that you mentioned for a different group. I believe for portal sites like Amazon, we are in a few restaurants item which is usually popular item in those gamas. So, unpopular group. So, ya. How is practical principle applicable in particular to portal like Traveloca? So, my second question. There are three models that you will name in the initial slide of every model. You mentioned that you removed the scene items. Ya. Was it to counter the battle? Okay, so for your first question is how do I make sure that it's not recommended just popular items? Is that what I say? Is that your question? Essentially, it's like all the popular items coming up and being sold in majority. Ya. So, that's more applicable to general e-commerce. Ya. How is it in particular applicable to portal like Traveloca itself like it gets numerous current or hotels? So, how do we prevent each of them that we are just recommending popular items? Is that what I say? Ya. So, it's usually very hard to avoid that because if we are doing models based on user interaction, it's more likely that popular items will show up. So, the benefit of doing so the benefit of doing cloud-based embedded is that actually, this bias term here that will be very high for popular items. So, as a result popularity is a counter fall within these two bias values. And then the bias that we write here is W, will have values that account for the popularity of the item. So, you can use models to remove popular items to account for the popularity of the item. The other way, very cheerful is that when you have finally committed items, you also maintain a list of popular items that you filter away the popular items here. So, your second question is to last in item, right? Ya. The same thing, we don't want to recommend nothing other than that the user is really aware of the item, right? So, we should not waste our time to show them things that they are really aware as happening. They are really aware and available and I infer that by then I can really click on the product news page and they really know these items. So, we will be from the personnel recommendations. So, my question is did you build one big recommendation for your entire item or did you build a recommendation for each product category such as flight or hotel? Generally, I will train separate recommendations and tune for different product time. One is that so, especially if your product time is very diverse within the company the amount of interaction that's within different product groups will have different frequency. So, if we put them together in one one transition data it may not perform the way we think it will because the transition the way we aggregate the combination is different frequency implies different parts of signal. So, if we combine products which have different usage because of the major product it may not lead to better recommendations and better recommendations for others. And as a result of this it also means that for each recommendation component they currently recommend one kind of product. This is useful for different recommendations based on which the user is on. Of course, if it's on the homepage we probably want to do a cross product recommendations if we want to do that we need to really think about how we put the data together before we do any form of application. I have another question. Is there any system to recommend a kind of combination of product? If you're a hotel in KL then you can also recommend some transportation or any other services around KL is there any system to do that? There's nothing off the shelf of course you need to understand your customer journey this requires a lot of looking through data having specialized teams to understand how the user use your product to infer what kind of files they're going to use if I'm going to buy a flight ticket to KL if I want to buy a hotel in KL if I want to look at attractions in KL if they're buying a business class ticket then probably they're not just saying this is one hotel. So how do you take account all this inside of a commercial engine there's not something generic that can work, you need to understand how the customer journey varies depending on the usage just a quick question what percentage of your users actually sign in or you can go and track them so the trick to this is that I'm not actually using sign-in users but rather I'm using cookies so cookie ID is unique based on the app install based on the browser so we do not actually require users to be locked in in order for us to use their data having locked in users is useful if we want to make inferences or cross-evaluate usage but in general this is not a problem because users tend to seek to using website Indonesia i.e. Asia is most of our first country so in fact we find that the stock usage is quite low so if you're using a mobile phone unless you install an app the cookie ID is going to be almost similar to the profile ID my name is Ah Wei so this question is all on the app view can you share what's the business impact that your work has brought about go for the language modeling portion fantastic business impact for recognition i.e. 3-4 ways to if we can recommend various items it will definitely increase the usage the engagement of the app by our users of course if you can measure user clicks based on recommendation then it will be obvious how we can justify our value based on the revenue generated from recommendations this is i don't have a number unfortunately recommendations is still a very new idea between triangle guidance doing machine learning recommendations so it's still in the process of getting this and measuring so i'll tell you a question on the second part i think it comes out of two things the way we approach the problem is before we work on something we try and make sure it's aligned we'll actually solve in some of these spots and then the owner says we'll come to us and say we've got a couple of different ideas and then we have a few so this is what you want as a product what do you think is the impact to your business to your business and so on and that forces us both to be honest both from a product perspective so that a we ensure with a limited resource we have a working one in terms of actual dollars or i don't know if it makes sense to talk about that but i will say so for example some of the work that we've done we were looking at so for all teams we're looking at tens of thousands of reviews today trying to make sure that we're providing the same experience for our users and that sort of workload has come down significantly so we're helping them skid out our businesses so that's basically one last question i'm sure the speakers will stay around after this okay so my question first you mentioned about engineering working most difficult part about building the model and i wonder what the difficulty and also during the engineering part your business team do you seek input from them so i can repeat with your business team do you seek information or insights from your business team where you don't know what the data or how to make sense of the data at the initial stage so the first question is regards to the engineering demands of this model what is difficult is that usually when we are doing of the user interaction these are these are usually recorded in realtime but then they are inserted in the database in a batch list which means i'm only able to get a data that is 180 so if that's the case then my recommendations will be based on your interaction i'm not able to react to what you're clicking right now if i have that available it means that i need to increment tracking that inserts in realtime into something that's in memory so that i can i'm able to access it also in realtime at low latency for me to get all this information because even if it's inserted in realtime and retrieve then the recommendation will go very slowly it will deteriorate the user experience and they may not even be able to see your recommendations if that's the case so that's the main reason why there's physical engineering requirements for this sorry the second question how do you work for this model if i were to increment this model i do not have to work with the business that i'm working on doing recommendation products of course in order to customize the recommendation engine and to ensure that it's highly performant and also get some insights on the customer journey we need to work with the business units so in capital car business units will have their own data in this that have the understanding of the usage of users on the type of business for example hotels or attractions we have business artists that have good understanding of the form of data the patterns that we can look out for and then okay can we give a round of applause for speakers okay okay