 Untuk berkongsi dengan kita, ia adalah perkara yang cantik untuk membuat komuniti berkongsi dengan kita. Terima kasih. Jadi, pertama, mari kita meminta Manish untuk berkongsi dengan dia, bagaimana dia mempunyai penyakit dengan mereka. Manish adalah penyakit data dengan Ojek, dan Iki adalah Manish. Terima kasih, Eji. Sebuah keterangan yang saya akan mempunyai sebuah keterangan data. Dan saya rasa kita mempunyai sebuah keterangan yang lebih keterangan, akan diperkenalkan nanti. Baguslah. Mari kita mulakan dengan mengapa kita panggil ini adalah sebuah aplikasi. Kamu akan mendengar tentang aplikasi, tapi sekarang kita panggil Ojek sebagai sebuah aplikasi. Dan satu keputusan kepada faktinya, adalah jalan yang telah digunakan, bagaimana ia bermula sebagai sebuah peluang berkongsi, dan bagaimana ia mempunyai kesannya. Jadi saya akan mengambil kepada kamu beberapa jalan di sekitar itu, bagaimana kita menggunakan peluang berkongsi dan bagaimana kita mempunyai beberapa keputusan yang anda lihat dan menggunakan. Jadi sebagai peluang, saya rasa masyarakat adalah untuk menggunakan teknologi untuk masyarakat sosial. Itu di kisah yang kita lakukan. Masyarakat sosial adalah untuk mempunyai maklumat, maksudnya maklumat untuk banyak penyakit. Itu adalah bagaimana ia bermula. Sekarang, apabila kita mempunyai Asia-South East, kita membawa masyarakat yang sama. Dan sesuatu yang anda tahu di sini, adalah bagaimana kita mempunyai potensi sebuah peluang berkongsi. Saya akan berkongsi beberapa keputusan tentangnya nanti. Jadi itu di hati apa yang Ojek melakukannya. Kita juga mempunyai peluang berkongsi. Peluang di sana adalah peluang penyakit, bagaimana kita dapat mempunyai peluang berkongsi, bagaimana kita dapat menjadi peluang penyakit yang berkongsi untuk banyak peluang penyakit. Di mana peluang penyakit tidak begitu popular dan ia berkongsi peluang penyakit untuk orang-orang. Jadi, seperti yang saya katakan, di hati itu adalah peluang penyakit yang berkongsi. Dan itu yang peluang sosial adalah tentangnya. Jadi, dalam keputusan, jika anda melihat apa yang kita dapat mencapai di Indonesia, anda akan dapat mencapai kepercayaan kepercayaan dari negara dengan menyediakan kepercayaan untuk orang-orang. Selain itu, kami telah memberikan peluang pengalaman AHA dan itu yang kami mencari. Kami mencari kepercayaan untuk 2 juta peluang peluang penyakit yang menggunakan platform selama-lamanya. Apa yang berlaku sebagai peluang penyakit terlalu berlaku menggunakan peluang penyakit yang lain. Jadi, ada peluang yang sangat terlihat disebabkan di sini. Kami memiliki 18-plus peluang tapi hanya untuk memberikan beberapa jika, anda boleh menjual makanan, anda boleh menjual kawasan, anda boleh menjual pengalaman, anda boleh menjual rumah anda dapat juga mencapai pesanan. Jika itu tidak cukup, anda juga boleh mencapai pesanan di rumah. Apa yang berlaku adalah bahawa kami telah melihat peluang penyakit terlalu selama-lalu 3 tahun. 125 juta peluang plus peluang per bulan, sekalipun 6,600 peluang berlaku selama-lalu 3 tahun, tidak banyak peluang berlaku. Untuk menurut orang setiap orang di Indonesia sebenarnya menggunakan Gojek. Dan itu bermaksud bahawa kami telah menurut mereka, kami perlu menjual banyak peluang berlaku selama-lalu dengan banyak peluang penyakit dan banyak peluang penyakit, bagaimana kita boleh memutuskan dan membuat semua ini mungkin bahkan dengan setiap peluang penyakit. Jika anda melihat bahawa ini sangat efeksi dengan apa yang berlaku. Sebenarnya, sebuah peluang ada 250 peluang penyakit. Jika anda melihat bahawa dari persenjataan perspektif, kami selalu berlaku dan berkata, setiap persenjataan berkata, setiap peluang penyakit. Jadi, bagaimana kita menggunakan peluang penyakit dalam beberapa aplikasi dan peluang penyakit? Jadi, di sini saya menunjukkan satu contoh untuk peluang penyakit. Jadi, jika anda membuka aplikasi, anda menggunakan peluang penyakit dan peluang penyakit dan kami melakukan peluang penyakit untuk menjelaskan perlukan yang anda ada supaya ia mudah untuk anda menjelaskan peluang penyakit. Saya tidak fikir bahawa peluang penyakit boleh berkata bahawa mereka mempunyai peluang penyakit sehingga ia susah untuk mencuba apa yang berlaku. Setelah anda melihat bahawa anda sekarang mempunyai peluang penyakit sebab anda mempunyai peluang penyakit untuk mencuba. Di sini saya menggunakan peluang penyakit semasa anda mencuba peluang penyakit. Jadi, kami menggunakan peluang penyakit. Ini adalah beberapa listan yang boleh diperlukan pada kisah anda. Dan kami menggunakan peluang penyakit untuk peluang penyakit dan menjelaskan perlukan yang berlaku semasa kami mencuba menggunakan peluang penyakit. Sekarang anda mempunyai peluang penyakit dan anda memperkenalkan peluang penyakit. Sekarang adalah bagaimana anda perlu menggunakan peluang penyakit dan anda perlu mengikuti sumbangan, penghidupan, dan kemudian yang kita bawa untuk menakutkan peluang penyakit untuk mempunyai peluang penyakit sehingga. Sudah tentu, ada banyak peluang penyakit Sebelum banyak orang mencoba dan menggunakan sistem, menggunakan sistem. Jadi, kita perlu menjaga perhatian dan perhatian di tempat ini juga. Ia adalah di mana kita mencoba dan mengetahui bahagian, membuat siapa yang benar, siapa yang besar, dan hanya memutuskan penggunaan di general. Dan itu adalah bahagian yang menarik. Kemudian kemudian ada alatasi, yang menerimu untuk mengalakannya juga. Jadi, bahagian itu dapat dibuat dan kita mahu memaksaikan. Kerana itu benar-benar menjadi cara pengalaman. Ia adalah pengalaman yang baik. Jadi, kita akan memaksaikan itu juga. Sekarang, apabila itu telah dibuat dan perhatian telah dibuat, kita akan bergerak kepada, jika ada pengalaman yang teruk, seseorang mungkin membuat alatasi, dan kita akan membuat alatasi untuk membuat alatasi. Jadi, kita menggunakan alatasi yang ada di sana dan memperkenalkan alatasi. Selepas itu, apabila alatasi telah dibuat, kita akan melihat bagaimana kita dapat membuat alatasi untuk menggunakan alatasi lagi dan lagi. Jadi, ia sebenarnya menjadi alatasi. Dan juga cuba dan melihat bagaimana kita dapat membuat alatasi kerana itu membuat perniagaan perniagaan, bagaimana kita menghubungi, bagaimana kita menghubungi orang-orang. Jadi, itu adalah alatasi yang satu. Jika kita melihat perniagaan perniagaan perniagaan di Gujik, ia sangat teruk. Jadi, sejak ia bermula dengan perniagaan perniagaan, ada alatasi dan perniagaan untuk semuanya yang kita lakukan. Jadi, untuk menjelaskan alatasi, bahawa perniagaan perniagaan adalah tim yang penting, yang mencuba dan menjadikan, atau membuatnya sangat efisien bagaimana alatasi dan perniagaan berkaitan dengan orang-orang. Jadi, kita cuba menjadikan mereka bersama-sama dan cuba menggantikan perniagaan itu. Alatasi dan perniagaan perniagaan berkaitan sekarang pada alatasi. Saya akan mempunyai alatasi dan perniagaan. Saya akan menghormati alatasi dan perniagaan tersebut. Makanan perniagaan, anda yakin kita akan mendengar pada pada masa depan. Kita berjumpa di alatasi atau berkaitan, apa yang harus kita bekerja, bagaimana kita boleh menghadiri suatu alatasi, mereka juga menjadikan perniagaan untuk perniagaan. Kemudian, kita melihat daunan dan kita cuba menggunakan banyak kemampuan untuk menggunakan alatasi dan bagaimana kita boleh menjadi sangat efisien dalam perniagaan. dan juga cuba mempercayai perniagaan perniagaan kita. Jadi jika kita faham kenapa orang menggunakan ujian, orang tak akan dapat memberi mereka pengalaman yang terbaik dan dalam perniagaan perniagaan juga, itu adalah perniagaan yang besar. Kemudian kita juga cuba memperkenalkan perniagaan yang terbaik di dalam perniagaan. Jadi sekarang, transport adalah perniagaan yang besar kerana itu adalah perniagaan yang banyak yang lain. Jadi itu adalah perniagaan terbaik. Jadi jika kita melihat makanan, itu adalah selalu tentang personalisasi, itu adalah mengenai, dan itu adalah mengenai perniagaan yang berjalan. Jadi itu adalah beberapa perniagaan yang terbaik di dalam perniagaan. Kita juga mempunyai sebuah kawasan yang besar, dengan mengharapkan perniagaan yang besar, di mana kita cuba memperkenalkan cara kita dapat memperkenalkan perniagaan yang luar biasa, cara kita dapat menguruskan aktiviti kemahiran di London, yang berlaku kerana ada banyak kemahiran yang berlaku di Promo. Dan kita juga memperkenalkan perniagaan yang berlaku. Jadi ada beberapa perniagaan yang berlaku di London, Terutamanya kita tidak dapat memperkenalkan strawberry dan thелен wise. Semuanya HOMU, ia ada lain yang hanya yang kita bukan pakai kerana memperkenalkan, perniagaan, perniagaan,�ì-t이�. So characteristics跡 yang ter ogrom untuk hidup, rancangan hari ini akan memuntukkan bahagian tremendous. MAPS adalah inisiatif lain di mana kita cuba menggunakan pembunuhkan pembunuhkan pembunuhkan pembunuhkan yang kita ada dan cuba mendapatkan routin yang betul dan ETA yang betul kerana semua orang itu adalah penting untuk pengalaman yang boleh dibuat. Kita ada beberapa pembunuhkan pembunuhkan pembunuhkan Ops. Platform adalah yang besar. Di sini, kita mempunyai pembunuhkan pembunuhkan pembunuhkan pembunuhkan, yang seorang pembunuhkan yang membuat pembunuhkan pembunuhkan pembunuhkan pembunuhkan yang lebih efekat dan efektif dan bagaimana mereka dapat membuat hidup pembunuhkan pembunuhkan pembunuhkan pembunuhkan pembunuhkan. Jadi, ada banyak sistem yang dibuat. XP juga adalah perjalanan untuk mempunyai pembunuhkan pembunuhkan pembunuhkan. Kerana setiap produk yang diberikan, kita mempunyai banyak pembunuhkan yang berlaku. Jika pembunuhkan pembunuhkan tidak kisah, mereka tidak benar-benar mencari, ada selalu risau membuat keputusan yang teruk, jadi itu ialah kesan berlaku pada platform eksperimen. Spokak-spokak untuk hari ini, jadi pertama kita akan mempunyai, jika anda ingat, pembunuhkan pembunuhkan pembunuhkan pembunuhkan, jadi kita akan mempunyai Jabaa dan Peter, bercakap bagaimana kita menunjukkan pembunuhkan pembunuhkan pembunuhkan. Lama yang kedua, kita ada Zelle. Jadi dia datang dari pembunuhkan pembunuhkan pembunuhkan. Macam mana kita membangun pembinaan masin kompleks, dan dia akan bercakap sedikit lebih tentang itu. Kita ada J.D. yang sedang berada di sini untuk kebangunan dan kebebasan pembinaan dan bagaimana kita dapat lebih baik membangunkan diri kita. Dan itu sebuah pembinaan yang mungkin tidak terdapat banyak, adalah bagaimana kita dapat menghubungi perniagaan data dan bagaimana kita dapat menjadi lebih efekis. Saya berjumpa dengan anda dengan satu video. Siafri, mana awak? Siafri memperkenalkan pembinaan di Jakarta. Maaf, saya lupa beritahu. Kita mempunyai pembinaan di Singapore juga, apabila pembinaan pembinaan ada. Kemudian kita juga mempunyai pembinaan di Jakarta dan Bangla. Siafri memperkenalkan pembinaan di Jakarta, dan dia adalah pembinaan. Jadi saya akan tinggalkan dengan ini. Mungkin anda akan melihat pembinaan pembinaan dan pembinaan pembinaan. Mana awak sepatutnya pembinaan sekarang? Dia ada sesuatu. Bagaimana dengan pembinaan ini? Ini adalah contoh klasik bagaimana anda perlu belajar keadaan real-life dan mengadapkan. Saya rasa semasa anda memperkenalkan video, bagaimana anda memperkenalkan video. Apa yang kita berharap dalam pembinaan ini, adalah kita belajar keadaan 3 pembinaan, bagaimana kita dapat memperkenalkan, bagaimana kita dapat menjadi lebih efekis, dan bagaimana kita dapat memperkenalkan pengalaman pembinaan pada saat ini. Jadi ini adalah pembinaan pembinaan 3 untuk kami. Dan, kita benar-benar memperkenalkan pembinaan di sini. Maksud saya, pembinaan pembinaan adalah pada pembinaan setiap pembinaan yang telah dibuat. Jadi, itu adalah sesuatu yang menarik, seperti yang saya katakan, pembinaan pembinaan sosial itu adalah cara yang terbeza. Jadi, itu adalah sesuatu yang sangat unik, menurut semua orang yang lain tidak ada keadaan. Jadi, itu adalah sesuatu yang menurut saya sangat berpikir. Hai, saya adalah Shafri. Saya pembinaan Pembinaan Pembinaan di Gojek. Saya pembinaan Pembinaan Pembinaan. Dan mungkin salah satu sebab saya ialah ayah saya adalah pembinaan pembinaan yang terbaik. Jadi, cara pembinaan pembinaan saya menjadi pembinaan saya, saya akan beritahu saya perkara yang saya merasa seperti... ...saya ingat apabila saya bersama dengan Pembinaan Pembinaan Pembinaan dan saya membuat masalah untuk rahsia matematik semasa pesakit yang memiliki percuma yang sangat berbeza dengan percumaan saya. Dan, dengan sangat berkawal, saya cuba memberikan masalah ke percumaan yang berbeza ke semasa percumaan yang berbeza untuk berdiri. Jadi, saya bertanya untuk pembinaan untuk membantu saya. Contohnya, sebab membantu saya untuk berbazir, dia memasukkan saya sebuah matematik pada kualitas. Dan kemudian, untuk pembinaan Pembinaan, dan dia mahu saya mencari solusi untuk diri saya. Mungkin dia bermakna sebagai joket, tetapi saya sebenarnya menghantar masa untuk mencari buku dan saya memasangkan bahagian-bahan idea dalam buku. Itu adalah apabila map saya bermakna memasangkan. So, satu lagi cerita mempunyai tentang matoliknya di sekolah saya apabila saya mempunyai matoliknya di sekolah ini. Jadi saya mempunyai Danwoo Featwork dan Dr. Fernlaffi di kanak-kanak saya. Tetapi saya masih mempunyai masalah melawan di kanak-kanak saya. Jadi saya rasa apabila saya memasangkan makanan saya, saya akan buat apa saja yang saya mahu. Jadi seperti apabila saya mahu mempersiapkan mahu dan mahu di sekolah ini, saya memasangkan makanan saya dan saya memasangkan saya berada di luar dan memasangkan sumber 5C euro. Jadi saya cuba memasangkan banyak skolak-skolak. Sebenarnya saya dapat kemungkinan untuk menemui penerbangan universif dan saya dapat kemungkinan dari penerbangan universif Danwoo Featwork. Saya memasangkan matematik yang terkenal dengan kemungkinan kemungkinan kemungkinan kemungkinan. Selepas saya membuat kemungkinan saya berada di Manila untuk 9 tahun, saya bekerja di KWAN yang adalah kemungkinan yang lebih terkenal dengan matematik dan kemungkinan kemungkinan kemungkinan kemungkinan. Walaupun ia terkenal, KWAN adalah bahan-bahan data sebelum berjalan bahan-bahan data yang sangat terkenal. Saya memasangkan kemungkinan kemungkinan untuk kemungkinan kemungkinan kemungkinan. Selepas saya bekerja di KWAN saya dapat mengalami kemungkinan kemungkinan yang lebih terkenal dengan matematik dan kemungkinan kemungkinan yang lebih terkenal. Saya hanya mahu buat lebih banyak. Saya mahu bekerja untuk menerima banyak orang lain. Itu apabila saya menemukan bahan-bahan data yang besar dalam kemungkinan yang sangat besar. Saya ingat bahawa saya berada di KWAN apabila saya menerima video dan menjelaskan bahan-bahan data dan sosial. Saya bermula dengan sebuah video dan saya berakhir dalam kemungkinan saya selama tiga jam, mencari bahan-bahan data dalam kemungkinan yang lebih terkenal. Itu apabila saya mencari bahan-bahan data untuk kembali ke Indonesia. Saya sebenarnya kembali ke Indonesia dan akhirnya menerima bahan-bahan data. Saya menarikkan bahan-bahan data sebab ia menyebabkan beberapa bahan-bahan data mewekkan kemungkinan untuk menghiasai bahan-bahan dan ini adalah bahan-bahan yang sebenarnya terkenal untuk orang-orang. Saya rasa Gojek mempunyai sebuah tempat yang unik sehingga kita mempunyai banyak bahan-bahan data dan itu bahan-bahan data. Kita ingin mencari bahan-bahan data untuk kemungkinan yang terbaik. Di Gojek, kita menggunakan bahan-bahan data untuk memlepaskan bahan-bahan dan menyebabkan bahan-bahan dari bahan-bahan data. Sekarang kita mempunyai bahan-bahan data dari Gojek Singapura dan Jakarta menjadi dunia terbesar. Mereka datang dari latihan dan latihan negara. Dan satu perkara yang saya suka tentang mempunyai orang-orang ini adalah semua datang bersama dan cuba mencubanya masalah dan peluang. Bagaimana dengan sebuah wajah? Bagaimana dengan sebuah kopi? Masalah yang mencubanya adalah mengambil kita. Mereka tidak selalu mudah mencubanya, tetapi kita mempunyai banyak peluang untuk mencubanya. Bagaimana dengan sebuah kopi? Baiklah, mari kita beritahu Siafri. Ya, seperti awak lihat, peluang perkenalkan, peluang perkenalkan, tantangan data yang kita buta untuk bersesuaikan, kita ada banyak peluang untuk mengambil perintah. Dan saya rasa kita.. itu sangat mengerti dengan diri. Jadi, ya, tidak perlu mencubanya secara selesai. Tapi ya, kita mencuba, Terima kasih kerana mencari penyelesaian. Tolong berikan komen untuk serta-salah. Jika anda mahu tahu lebih, saya akan mencari ke Petra di Javan, yang akan bercakap tentang bagaimana kita mempunyai penyelesaian dan penyelesaian. Terus presentation ini. Saya telah bekerja di跟你i daerah di Marketplace yang berkumpul selama sepuluh tahun dan prim-prim, saya terpaksa menggantikan halikasi dan menghantikan pertanian berlaku. Jadi, tanpa lagi, kita akan beri licauan. So, the agenda of this discussion would be, it first start with an introduction to Gojek marketplace team and the different components involved in Gojek marketplace team. After that, we'll look at the formulation of matchmaking program at Gojek, after which we'll be talking about the allocation system, rank or drago ranking also first generation allocation slash matchmaking system and the acre is the new and improved version of rank. Peter will be talking about rank later in the session. So, what is Gojek marketplace? You know that Gojek has at least 18 plus product offerings in Indonesia, but at its heart Gojek is a marketplace status. The entire objective is to match service providers with service seekers and how we do this is core to our business and critical to our operational efficiency. So, let's look at an example of customer making a go-kart booking and I'll walk you through the entire process of how Gojek marketplace has different activities in the entire state of the transition pipeline. So, marketplace starts working even before you try to book an app. The drivers are shown information regarding the high supply area so they can use this information to strategically position themselves for better allocation opportunities. And once marketplace, once the customer launches the app and enters the pickup and destination location, the price estimate is computed and shown to the customer. It not only depends on the road distance between the starting and the destination location. It also takes into consideration the dynamic real volatile demand supply situations in the marketplace and then adjust your price accordingly for better operational efficiency. Now, once the customer decides to make a go-kart booking, the Gojek marketplace allocation system looks for closest drivers surrounding the customer and then among those list of drivers tries to choose the best driver for the specific job. So, once the driver has accepted his booking, marketplace shows continuously updated information about the estimated arrival time of the driver so you don't have to wait in the pickup location for a long time. So, essentially, Gojek marketplace is a combination of all these factors. This entire pipeline happens millions of times a day and every efficient allocation counts. So, you can consider Gojek marketplace as a highly complex machinery that takes into consideration the ever-changing real world and acts accordingly to provide the best user experience possible for you. These are not just an exhaustive list of functions in the Gojek marketplace team, but these are some of the critical ones. Pre-dispatch function refers to any operation that happens before a customer makes a Gojek booking. And pricing is a complex machinery itself where it involves a variation of pricing structures and it takes into consideration the supply demand situation in a given place at a given time and adapts accordingly. So, dispatch or mass-making is nothing but allocating the right drivers to the right customer. And driver incentive is a function that operates to keep the driver's incentives to do the right behavior and keeps them motivated. To do specifically, we'll be first focusing on the dispatch and mass-making problem at Gojek. And let's do a deep dive into it. So, what is mass-making? Mass-making is nothing but providing the best service provider for a given service seeker, which is a customer. In terms of ride-rolling, this is just providing the best driver for the given job. This visualisation is what we call the heart of Jakarta. And it's completely plotted using Gojek, drivers, latitude and longitude information from their phones. And this normally beautifully shows the incoming and outflowing traffic in and out of the central business district of Jakarta over the period of one day. But it also highlights the magnitude of the task involved. There are millions and millions of transactions happening every day in Gojek. And especially for two-wheeler ride-booking. And this provides an enormous opportunity to optimise and create a big impact for our entire operational efficiency of the marketplace. So, why not just use the closest driver for the allocation? Let's look at a few interesting scenarios to understand the situation. Imagine you are going in a go-kart from this specific building at Aksa Tower. And there is a driver, a couple of blocks away from you. The straight-line distance or what we call as a co-flight distance is about 300 metres. But in reality, cars don't fly. So, this is the road they took to arrive to Aksa Tower. And depending upon the traffic conditions, the time may vary between 3 minutes to 5 minutes and so on. So, in this specific scenario, you could have chosen a driver who is far away but travelling in the Shintenway Road and coming towards Aksa Tower. This is another interesting example. This is what we call as a cloverleaf junction. And these are very common in Indonesia. Let's assume that there is a customer here trying to make a go-kart booking and it finds a driver travelling in this specific direction. Even that the driver is close to the customer, in order for him to reach that specific pick-up time, he has to make this complicated sequence of manures and reach the pick-up location. So, these things are just highlighting the complications involved in making a particular match for customers and overriders. So, why don't us in the classes draw it to you? These are some of the factors that we consider. So, the traffic conditions in certain places are really, really volatile and you have to take them into consideration when you're allocating drivers to customers. And the driver preferences, since Gojek is a multi-service type product, Gojek driver has an option to do a right service or deliver food or any other choice. So, certain drivers have certain preferences and they like to capture them during their allocation. Drivers may have favourite locations like at the end of the day if they want to go back to the home they don't want to take a ride which leads them to the other end of the city and customer might have their own preferences. Certain customers are much more patient and are okay to wait for a driver but certain customers try to cancel the booking if they see that the drivers are too far. These are just a few examples but they are much, much more to consider. So, two things I would like to highlight in this presentation is that the magnitude of the task involved and the number of factors to optimize gives a great potential to make it as a machine learning problem to solve jangk jangk or driver ranking system it is the first ML model to be deployed in Gojek and at scale and like a first step the model was a simple supervised machine learning model and it has this familiar steps where we perform ETL based on the historical data store and initiate a model trading based on the model string we deploy the model as a web service for real-time inference looks all good So, although I have masked most of the technicalities involved in this pipeline it was a pretty sophisticated system and in the sense that we were able to achieve one click deployment for the system and the CI-CD pipelines automatically integrated with load testing functional testing components and the model trigger and train invariant validated the end result of the model training and put the push of a single button deploy into production and we generally used more than 150 features to train the model some were location based driver preferences customer preferences but the entire objective of the model was to improve the operational efficiency of allocation which has did one thing and it did really well once the model was deployed it was a huge success it improved the operational efficiency significantly and it also was recognized well with the business users but what happened let's take a look at what happened later at Gojek business really changes fast things that work well at a given point of time of couple of months so at the same point of time Gojek started expanding to overseas markets outside Indonesia like Singapore Thailand and Vietnam and these specific geographies had different objectives and different behavior of drivers this was a challenging task to accomplish for Drank and the next biggest problem for Drank was to retreat fast and experiment faster most of the time the offline metrics such as AUC or Waterloo School does not correlate very well with the with the real world metrics that we compare to and even simple situations like adding a single more feature would require enormous change because the entire pipeline from ETL to model training and the web service has to be replaced so the architecture of Drank it had all this type of things and it made it really difficult to improve the model test different modular components and add new features or try other different variations of models and respond to external behavior Drank wasn't really designed to react fast to external behavior it had a traditional pipeline in the sense that I trained on historical data and what to try to do was to just optimize a single objective some of the external behavior that changes fast I can give an example of say drivers using fake GPS the behavior of drivers even in certain fraud things try to capture the behavior the behavior changes very fast and these these types of fraudsters do affect the network efficiency GPS being spoofed to different location and when you assign the driver to a specific customer the real arrival times for those drivers are very long and Drank was simply not designed to solve these types of problems and we took this time an opportunity to think deeply about certain problems and areas of improvement that we could handle for Drank and those are how we should formulate the problems the feedback groups involved in training a machine learning model and then deploying it in real world production and in order to explain these things in detail I would like to invite Peter to talk about how we converted these problems and address them well and eager Thank you, David So Drank was the history so I get the more fun job talking about what we are doing now what the current application system is so yeah, Dara has already introduced so that's what we call a new application system so what were the things our main objective main reasons why we decided we needed to to rebuild I think Dara has already hinted we needed more flexibility Drank was a very powerful tool the model did a very good job what it was intended to do but it was a little bit like we had a like a rocket launcher which we were trying to fire blindfolded if we hit the target there was a lot of luck involved so this is kind of also the reason it wasn't wasn't my name but it's kind of it refers to I think it's a specific reference but it's something like this sort of interface of a human and a machine so we're trying to get the best of machine learning but keep a lot of flexibility for human intervention sub-aptives manually override model if we need to we also needed it to be a lot more responsive the allocation problem is very non-stationary so we need to train refresh the model very quickly we need real-time features into the model yeah so one thing that we've spent a lot of time thinking about was how how are we fair to drivers we don't want to just if one driver has a bad day or just has one or two cancellations we don't want them to be decryptised for the next two weeks or something so how do we make sure we're reactive enough so that we're fair to drivers that are performing well we should be allocating them more drives yeah so we need to get more real-time features into the model and the next objective was to make it multi-adaptive so going from one model to many models each optimizing for different models so why did we think this was so important well probably as Jared has already said the business is always changing you know it's often from week to week they flipped up and say no we should optimize for this no we should optimize for this if just one model it's very difficult to do that and also as we've expanded we've launched Singapore Thailand Vietnam these are very different places we need to have a different model a different objective we also need to test different ways of framing the model I think this is probably the main takeaway for me from building this moving from rank to yoga is that when you have a supervised learning model how you define your target is by an order of magnitude more important than anything else I would say particularly if it's not a sort of cookie cutter application machine learning if it's a sort of slightly bespoke solution I guess how you define the target super important so that means you can't just rely on offline metrics like AUC if you have two different models that are prediction two different things it's meaningless to compare those metrics so you need to be able to test those online and iterate very quickly so that was a big driving factor for us to build yoga control feedback loops this is an interesting one I think what we found with rank was performance initially was quite good but then it degraded over time what we found was once the model was sort of been refreshed on data that it had itself generated we got these kind of amplifying effects and there were some biases maybe in the model which were getting amplified and it was degrading our performance so we needed more control in order to address this I think I think the reason for the bias or the big reason for the bias was that we had one model so we had a sort of monolithic prediction problem and we decided we needed to break up this problem into smaller prediction problems with different models and then we could control how we actually combined the outputs of those different models finally similar point but as we will know if you ensemble multiple models it often performs better than one model so we wanted a way to experiment in different ways to ensemble different models so we decided we need a lot of models because we got multiple adapters now we already have 18 plus products many, many cities and every one time we will have dozens of experiments different versions of the models of different features who know experimenting so it's yeah when we we build one model deploy it and now we realise we actually need hundreds maybe thousands of different models you know how are we going to scale this up we need to for each model we need to do the feature question the training, the deployment the surveying, the ensembling and the experimentation how do we manage this big complex system and I think I can't go into all the details of how we did that but I just like to focus on probably the most important tool that we had most important sort of abstraction to make this problem more attractable it's what we call the feature store or feast I'm going to give up on this and good news for all of you is that we open source feast a few weeks ago so this is basically our system for it's on GitHub don't need to take pictures so yeah this basically just it doesn't do the feature creation but it does the feature storage so it's a very useful abstraction we found just obviously for sharing features between different models and for ensuring the consistency of features between training and production I can't take any credit for feast it was built by the platform team here and I probably can't answer any difficult questions but I'm just one happy customer so that's just the perspective I'm going to give you so what did feast allow us to do with Yega in Yega so this is more or less what we had in the old days we drank we just had one sort of monolithic pipeline four days feature engineering model training then this would trigger trigger the GitLab CI pipeline to deploy the model so it would end up in production now we have one model we want to go to hundreds possibly thousands of models we can't just multiply this a thousand times so this is more or less where we are today we've broken up this pipeline at the bottom with the feature store so generating the feature engineering if it's it could be batch in airflow or it could be real-time streaming features in link or batch of beam so that happens separate process or ingestion so the feature store and then we can train the models putting the data from feast and then we can run that we can do the model training and whatever tool we prefer and then we have a sort of model store that's analogous to the feature store using MLflow currently and then for deployment we found that Go CD was a better fit than GitLab CI and then on the production side we needed a system to integrate all of these models to fetch features obviously first then call the models then ensemble them that's Meister, Jäger Meister so that's the assembling and also the logic for the experimentation and then in the center we have lasso which is a sort of orchestrated service which is open source also or soon to be open source but I'm not going to go into the details of this because you'll hear from Julian in a few minutes and she'll be able to explain this much for them better than me so you'll notice we have the feast at the top there so that's where we need to serve get the features in real time for each request and then send them to each of the models asynchronously so so what this feast has allowed us to do it's made it much easier to get real time features into the model because we don't have to worry about the inconsistencies between training and serving it makes it easy natural to share share features you define it once you get suggested in the feast and then all of these models and other models different predicts can use those features and also an extension of this is that you can also have another model that is predicting something maybe some driver levels fraud score for instance that can then be ingested in interface as a feature and then that's a feature for our models in allocation so I think this has really helped improve the collaboration in the data science team the guys that are building this driver fraud model they might be in Jakarta but we're in Singapore but feast allows us to collaborate in a very productive way because we've broken up this pipeline at the bottom and choose the tools to make sense and this has made it this much more modular which has made it much easier to scale to dozens and hundreds of models and then finally it's a we found it's pretty performant so this is the serving latency this is when we first rolled out feast for Jega we were monitoring the latency of fetching these features and as you can see it kind of dropped at the cliff which was very good news for us we were having a lot of timeouts before we make the feast we solved that problem I can't give you a good reason why we managed to achieve this but kind of my point is before this part of the reason why this is so slow is that we in Jakarta the data scientists were manually loading features into big table we were using as a data scientist you don't really want to be you don't want to be doing that so feast gets the division of most of us and that's correct as a data scientist I don't have to worry about the the serving latency I just gave my features and I'm happy so this all added up I think it's had a lot of success a lot of business impact I would say that one biggest most important being as I mentioned is being able to to try out different models different framings of the problem in terms of different targets in the super-resolving model and then it quickly easily put that into production see the uplift so we've got a lot of appreciation from the from the business from our bosses which is nice I think obviously they care about the uplift in the metrics but I think they also recognise that we've built a robust system that is going to scale with the business and as our needs change we're not going to re-build again so yeah that's it for me I think we have time for a few questions now for we move on to doing it's high presentation just wondering can you give an example of what the model actually outputs and a few examples of its inputs yeah so jarod jarod mention the main sort of features we have they tend to be driver level features or customer level features or the sort of do do type features as I say they're the inputs we have a lot I can't go into one of them the output yeah I mean it just depends on what the business metric we are optimising for is so we have a lot of different models optimising for different objectives yeah I mean obviously the number of completed bookings and just converting the booking is a big one but there's lots of other things like you want to reduce 50k and so on yeah we have forwarded it's been a big problem driver lot of our drivers seem to like Pokemon Go a lot they go to their location because I'm busy so trying to obviously that is influencing the marketplace in a very bad way so we try to to deal with that quite successfully yeah there's really whatever the business says they care about this week but that's Thanks I have a couple of questions so if you were doing you were saying that it improved your operations what were you doing before like what did it improve upon that's the first question the second one is what features break the model the most because you were saying that things change really fast after a few months things stop working what features or it was this problem formulation we tried we were not looking at the right metric to optimize for without the the acceptance rate of drivers it's a good indication of network efficiency but really what happened later was that we we found that in order to improve the network efficiency holistically we have to look at different objectives and metrics and have certain constraints on them that is how we differentiated with the other some of the examples of real time changes are like Peter mentioned it's about driver behavior changes sometimes if a driver is having a bad day because of if I can issue with his mobile phone or something that is not able to accept what power is that might that might lead to poor driver statistics and we don't want such things to diperyak get the driver's diperyak day so done Ya I think as we mentioned forward it's a big one that it's always fast changing it's always an arm race Ya I think we're in danger of overrunning Alright, thank you Peter ladies and gentlemen please halfway through and it's already 8pm Next we have Ziling we share about lasu about service administration I'm Ziling So just like how I did I'm sharing in front and then I'm talking to you now and I'm probably doing cleanup I specialize in the end-to-end deployment of machine learning but it's not only deployment of models but also the building up of a platform for the dear scientists to bring their models to production in as fast a manner as possible Ya So today I'm going to talk to you about building complex machine learning workflows here at Gojek Ya So I hope that sounds slightly exciting to you guys Ya Thanks Okay, so let's talk about model deployment Right When you think about putting models into deployment what is it that you think about Sorry Ya So you expect that with your model up somewhere in the cloud you receive some sort of request and then it will return some sort of response and this will go through some sort of model Something like this Right So request into model and response So a lot of serverless machine learning deployment solutions they focus on this you can deploy a model and it focuses on outputting whatever the model whatever the model output So it's just in and out But realistically what you'll be having is something like this You'll have some sort of pre-processing of the request and it goes to the model and then the response will have to be post-processed before it's returned to whoever is querying the service But it's not usually this simple This is like I don't know Maybe you built this in university or something This is hardly ever good enough Right So A common pattern is that you would have some sort of features in some TV somewhere These would be batch features and you retrieve this during the pre-processing stage and these would be passed the model and then the response would be post-processed and this would be returned to the person that's querying it So this still looks pretty simple pretty doable You could do it in a really small microservice But then it gets more complex What if your model requires some sort of fallback logic What if your model is something that's complex and could possibly take very long So for instance if let's say you had some sort of deep learning model that couldn't possibly return a response within 60 MS 99% type Right What do you do with the responses that fail to be created on time You need some sort of fallback logic Right And this fallback logic would have to go inside your app You have two models Right What if you have two models and you want to do some sort of traffic splitting between the two This is actually a very common pattern Let's say if you wanted to segment your population and you wanted population A to get results from a model that you are So in a base model and you want some test population to get results from a model that you are experimenting on Right And then what if you had even more models Right And it's not even traffic splitting this time You want to ensemble them You want your request to go to each of these models You need run some sort of post-processing step to ensemble these models before you output a response to the user So what I'm trying to say is that it's never deploy a model to production It's never as simple as putting a model in a microservice in a cloud and expecting that what it produces is good enough for whatever application you're doing Right It'll probably look something like this in the end It gets It gets complicated pretty fast So case in point what Peter and Joe showcased to you earlier with lasso Sorry, not lasso I mean Yeager I'm too excited about my own thing So if you look at what they had what they had was multiple models some sort of experimentation service they had teacher getting they had post-processing they had on zombie And if you think about building something like this Sorry Okay I'll talk about it later So it's very difficult to think about a model in production as a model You have to think about it as a predictive unit So this is a term that was coined by the developers behind Selden who is building something similar but it's way more featureful So what I'm trying to say here is that when you want predictions from something that something is usually not a model but a system a system of different entities doing different things Right And can you imagine if you build this entire system within a monolithic application Right All that complexity would become something that's incredibly hard to handle even for software engineers Right Much less data scientist And you want data scientist to not have to grapple with code as much as possible I mean you want them to code They need to code to make their models but you don't want them to be building massive massive microservices sounds a bit strange large services to house their models Right So the problem with something like this is that a lot of code ends up being boilerplate Right You end up reusing the same code across your applications and then it comes very hard to make changes in one of them and not have to propagate them across all those other applications It's also very difficult to do orchestrate all these flows within a single application because it's not immediately visible what each part is feeding to another Right And it also means that you're locked into a single language in this project Right Which in machine learning can be handicap Right You would want like for instance let's say you've trained an XGBoos model Right Usually you'd serve it with Nowadays you can serve anything but back when this was built you could you could only serve it with Java or Python while the rest of our the rest of the part of application you wanted to go Right So it was it was difficult Right What's the next slide So Logically What you want to do is break it up You want these entities to live within their own services and interact with each other somehow Right But the thing is that when you broken it up how are you going to define the flow Right Where does it where does the logic for how this input flows into this input and how the request flows to all three models Where does that live Do you put it inside each of the applications The problem with that is that you have to it becomes very difficult to make changes you make one you add one model you have to change that model you change this you change that and it becomes extremely difficult to maintain this system and that's not what we want So enter lasso which is something we built to as a solution to this problem It's a microservice orchestrator which is an extremely big word and very fancy sounding but I hope they explain why it does someone Right So it's supposed to orchestrate flows between different microservices In this case would be services that comprise predictive unit So By orchestrating these microservices as you can see lasso would have access to each of the services within predictive unit It's able to execute workflows similar to how your airflow dag would be like You define some sort of direct acyclic graph your request to flow through this graph and you get some sort of response So what lasso is is that Sorry lasso is comprised of workflows So workflows as I described to you are directed acyclic graph of tasks So there's a bag of tasks that lasso is supposed to execute and all these tasks have access to an intermediation storm and when a request comes in lasso will orchestrate the lasso will execute the tasks as they are required to Let me show you later So lasso will execute these tasks in turn either synchronously or asynchronously to produce some sort of response So what's going to look something like this You'll have some set of options So you're able to set which endpoint you want it to be at I'm able to set a global timeout that's proved to be very useful for their scientists and you are able to define a bag of tasks as I showed in the previous diagram So what is a task? A task is basically something that does something if the conditions are met So the way we've decided is that you don't draw a graph You define a task that only executes when certain triggers a file So it's able to depend on the content of the request other task statuses So for instance if an upstream task has succeeded or failed and also other task output And the task we have a variety of tasks that are currently built into lasso We have tasks that can echo values that can make HD calls and also execute some sort of lambda function to transform the request or outputs or other tasks So it looks something like this So you have the task type the configuration of the task you have additional inputs to the task output path that you want to put it in and then the conditions to run this is extremely important and then if you want this task to be exit then there will be an exit flag So how does this whole thing solve the problem? Thanks to lasso microservices can have distinct rules You don't have to have a microservice do a bunch of things and do a bunch of things in a mediocre way You can write a task that a service that processes data and a language that processes data for instance, Python You don't have to write it in like which is like flipping your head out And then there's also encapsulation The microservices don't necessarily have to know about each other Only lasso knows about them So they can basically care about the world around them All they care about is whatever they get and whatever they have to return And also similar services can be used for other predictive units So instances of services can be reused to be like teacher-getters Many teacher-getters Ya Ya And of course there's no dependency on the single framework or language as I've extended the first point where you can have a predictive unit that comprises some of microservices that are in Go, Python, Java whatever your developer is comfortable with or whether it's best suited for that task Ya It also means that it's extremely easy to edit and iterate on workflows because you don't have to write additional code and the configuration of the workflow sits outside of the definition of those services Ya Oh ya, and boilerplate can be moved into lasso So for instance, you're planning to move feature-getting into lasso so that you don't have to write a service for that anymore So it'll be a task Ya, so lasso brings to table some other nice things It's fast and it's scalable It's extremely lightweight It sits at the heart of Ya Ya So I know at least that it's able to handle the load that the Indonesia marketplace puts on it So that's pretty impressive Also because it's written in Go it can be combined to a single executable that's very easy to deploy We have a help track for it So all you have to do is to help install, provide it to a contact map that has the workflow specification and it works That's it Ya And then if you've seen from the previous slides we also support both templates which will be similar to your ginger template Right It allows you to have logic inside your workflow Right So you're able to do things like input and output validation You're able to transform requests and responses And the nice thing about the workflow sitting outside of the service definition is that you're able to version the workflow separately from the services Ya There are some could be it's the world It's not all sunshine and roses in lasso land There are trade-offs between latency and type safety So I mentioned the Jason store earlier The thing is that Jason can be very expensive to pass It takes very long Right So what lasso has done is that it opts for a lazy Jason passer that doesn't really care about how well-formed your Jason is In exchange you're able to pass the Jason extremely quickly Ya So to handle validation you have to write it in the lambda instead Ya It can also be very intensive because we use it in the Jason store Both templates are also not intuitive to write if you guys have a dungeon-jump the thing is it's the same and ya most can get believing and also it's currently HDP only It's not cool, right? Ni, I know GRKC Ya So I think that's it I have a final slide with my I didn't handle it because I didn't know what to put in the final slide but Ya I hope you enjoy what I'm saying Does anyone have any questions? It's a very important talk to say to you Ya, I remember we're not going to go shop immediately so Peter and Java are going to be there Jalik is going to be there They're already charismatic speakers but I will say this that this is the first time I'm clearing the official version of why Yeager's name Yeager I always assume that the team like to drink that's why they call it drank and Yeager So okay Ya, let's go with that Pacific Rim Okay, cool I'm actually going to prove it a little bit and talk more about thank you so much burning out data science and why is this an important topic is because who what's the most important thing when you are trying to launch a data science product What is that sound? It's people who said that thank you and the reason why I say people is because honestly like the telepool here in Singapore and Southeast Asia is you know and to a large extent the world like you don't support people they're going to burn out they burn out you don't get data science cool so a little bit more about me my name is Jairitan it's pronounced Jairit without the T here is my contact info if you want to get in touch with me later my current favourite animal is the penguin because they're just so cute look at that one even real life penguins are very cute so why do I care about burning out as a data scientist well my task here at Gojek is simple I support 46 also data scientists and I'm here to help them in their career and to make sure that all of them feel supported by the organisation my background is very all over the place I used to work as an economist at MAS if you've heard of it it's right across the street I'm a bond breaker I have strong opinions about the scholarship system please come talk to me I worked at Flippert which is an app that has since gone on to do to greener pastures I was the first data scientist that worked on growth hacking at Facebook I started out as a data scientist on groups then I transitioned into a software engineering role on AP testing then I quit and started my own consulting business published a paper in the Lancet on Bayesian Method Analysis and now I'm here supporting my teammates and I have to say these people are on point I've been around the world that's a song from the ADs in case you didn't know you know what and I have to say that the talent here at Gojek is Stella and I've really been impressed by all of my teammates so I'm living proof of the fact that you can have many careers and you can read many yourself even if you're not Madonna so what happened to me at Facebook I burned out in 2014 my best friend died and I didn't take any time off because I was young back then I was in my 20s I still am and I was just powering through and then in 2015 Facebook grows huge like that's when they had a doubling in size of headcount at Facebook by 2016 in January I was physically exhausted and in 2016 in March I actually to take 2 months off of work and in November of that year I left Facebook and then soon after Trina as a legal instructor focusing on mindfulness and then started my own consulting business now what is burnout and how do you recognise it there are 3 points here it's exhaustion, cynicism and attachment excuse me exhaustion, cynicism and feeling ineffective and unaccomplished so exhaustion is pretty obvious I put a picture of you that here because I think you would ask you to but you can read the symptoms by yourself but it's important to know that exhaustion can refer to both physical and emotional exhaustion so if you feel like you're irritable and like that you're short with your colleagues that's an example of exhaustion cynicism and attachment refers to a lack of enjoyment when you engage with your work you start to feel pessimism you start to isolate yourself I did this a lot at Facebook at the end of my career and attachment refers to things like I go and lay or I don't really want to engage my colleagues the last point is feeling ineffective and note that feeling ineffective can also be a cause of being ineffective so it becomes like a vicious circle so congratulations if you exhibit 15% of these symptoms you may be suffering from burnout and that's not a good thing because burnout equals attrition I mean from an organisational perspective but for you burnout means burnout and that's bad too so why didesitis burnout and I think that there's something unique about burning out as a didesitis and I'm going to I'm going to give you a couple of hypotheses precisely six one this field is full of media hype so there is a myth that didesitis can do everything but in reality can do somethings really well like it can play go but it cannot really stand in for product or design which I think we often as didesitis get involved in conversations where PMs or directors ask us so should we launch or not and then you're like yes hypothesis two it's a lack of clarity about what we do and this has to do with media hype but it also has to do with the fact that we're a relatively unfeel and we do so much we do machine learning we do statistical inference I love like I train as a statistician that's where I I think I shine at software engineering we do data visualisation and also sometimes we're expected to tell stories and not all of us are going to this is not an exhaustive list yeah not all of us are going to be good at all parts of this of this list hypothesis three data is where product and engineering disagree because engineers tend to be realists and product managers tend to be idealists and data science often gets caught in the middle how many of you have fixed a logging bug? no oh my gosh really you all are living but like I have fixed a logging bug so often where the PMs define one thing as something and the engineers just increment the wrong part of the app and like all the metrics crazy all the results look crazy and then like I'm the one that has to like chase it down that's where product and engineering that's an example of where product and engineering disagree at least sometimes a question later this point is important to know that the data science life cycle is very very different from the product life cycle or the engineering life cycle because why we have actually engineering you tend to go from prototype to implementation to maintenance to department maintenance and improvement but for us there is a front loaded analysis portion because we're not going to execute unless we have a fairly good idea that it's going to produce good results and how many of you have been involved in daily split stand-ups where you're expected to just say I'm still researching because same as yesterday very exhausting the other thing to note is that we're often often by the organization if you work at a startup the data science person is usually not the second or third hire it's usually like the 20th hire and then when you get there the engineers kind of look at you like you're a bit crazy and when you report to engineering you're reviewed and ranked against engineering standards which tend to ignore the front loaded analysis portion and when you report to product you're reviewed and ranked against product standards and who knows what those are I'm sorry I love my I love my product managers and everything but sometimes I don't really see it at all so who advocates for data scientists that could be another crossover now oh excuse me the last thing is the relative lack of mentors and managers and I think really when I was at Facebook this was the number one reason like because data science was such a young field then like I was 12 you know I'm not 22 and my my managers were just 14 you know I'm exaggerating my manager was actually younger than me at that point and you know bless his soul but he tried his best and you know the issue here is that a lot of us data scientists we have an obsession with being technical which somehow has become has come to mean like I know how to use deep learning but we don't realize in order to leverage ourselves we need to leverage ourselves in terms of talent rather than technology right because when do you really encounter a problem that like deep learning or like whatever the like I come from statistics so like right now everything is Bayesian like when when does that Bayesian approach like bayou you know the 20% win not really so give these are my hypothesis how do I think that we can prevent right now the normal thing is to set expectations right I when I work with my team here at Goja I continually help them to remind stakeholders that data science cannot answer every question in particular don't get data science to answer really hard business questions that are at the intersection of product engineering design and UX research we are a voice at this table yes I agree we can provide a lot of valuable insights we can provide a lot of valuable strategies on how to grow but we cannot solve everything the other thing to note is that we have to be cognizant of where we are in the data science life cycle which means that when we are in the research or analysis space we have to really be aggressive at telling other people to you know give us some space to breathe to say it nicely I was about to use like an expressive at each phase requires different skills different cadence of check in and different project management techniques we start to answer very broad and open questions at first data scientists answer broad open questions at first that kind of converge on an implementation solution right and we have to to be able to recognize and advocate for ourselves another thing that you could probably do is to ask for organizational clarity as I said being a data scientist that an early startup sucks because you're too late to be founder and early to be a comfortable employee a lot of you are laughing because probably it's true and being a data scientist that can suck because you're often co-operating product management engineering but you have none of the authority and you may also have to compete with other data teams like data engineering or product analytics I think in the face of this you can ask yourself what data science as an organization and your teammates what you all can uniquely do and focus on those things and once you get clarity of what it is that you should do other people should listen to you when you are the authority on that and I think that this requires a lot of self-awareness and the ability to advocate for yourself the last thing I think it's the last thing but I'm just crossing my fingers I'm hoping that this is the last slide sorry but I'm not putting my foot in my mouth it's to find mentors you find mentors inside the organization so you get more senior data scientists within your organization to mentor you but you also outside the organization use these meetups to find counterparts that you can bounce ideas off of and that's what I hope that you will do with DSSG tonight but the more hidden and more valuable thing I think is because we're such a young field you may not necessarily get mentors with that much more experience than you as I said I'm only 15 they're not that way I'm wrong I'm only 22 and this field is only 6 years old who else is here to help me so one of the ways you can overcome that is to become a mentor yourself and peer mentorship I think is where you start to learn and that's actually how I got started in this business helping my teammates and shepherding them through tough times and in doing so I put trust and they also mentored me when I needed them so this is not an exhaustive list by the way of why or not happens and how to prevent it but I'm hoping that it will spark a good conversation and if you're interested in being a manager or a team lead here at Project please come email me this is my name jaiatproject.com and we have one last video but I feel like the sun's low off and we're it's all right now well we can watch this video but before that do you have any questions for either me or shepherding? can you give an example of when you were asked to answer a stupid sorry complex business question that really wasn't all posed I'm not covered by Fist of India anymore, right? so I worked on Groups and one of the last products I worked on was a Groups app the standard of Groups app how many of you have heard of it? exactly right? because then they were just like basically every week they'd be like oh did we hear on our tricks and they'd turn to us and be like so should we launch or not I'm like I don't know it depends on what you think the business value of this is and whether you think that having a standard of Groups app is part of your entire strategy of having individual products be broken into their own separate apps and then they'd be like should we launch or not? you know you get into those weird conversations which are very on loop way and you have to start recognising this when you are asked to behave like a product manager to make product decisions because your product manager is not bold you should you should find a way to figure out and say like hey like I have provided you with the best information that I can we need to have a multifaceted approach to decision making at this company because you can't simply just look at me and say like data is all the answers we often do I mean we're geniuses but we often don't any other questions before we play this video? yes I had a question about lasso Yes, Jilin Regarding Thank you So it's regarding pre-processing very often it's a part that actually is duplicated you have pre-processing when you're training a model first time and then you have pre-processing for live requests and I'm not entirely sure whether you were using the same code the same components basically in feast or in live pre-processing Oh yeah, excellent question that's where feast comes in please like and star us or get heart okay so the way feast is able to help you in this part of the machine learning workflow is that we do our pre-processing outside of the app it's done inside the stream so in our case we do it in deep so if it's real time data it passes through the stream it's transformed and ideally what comes out of it is a pure feature it's whatever you need for your model so whatever goes through that stream would ideally be something that could be shared across multiple predictive units yeah so does that answer your question? Absolutely, thank you Okay, thank you we're going to play this last video and then hand it back to the organizations of TSSG How do I play this video? Click the button But there's no button here Oh, okay Sorry 18,000 plus islands but 22 billion minutes wasted in traffic and the land of me Go Jets, super app me, my transport, delivery food, Paris, massage ooh, nice payment bills, rewards shopping, business you get the point I do 100 million orders every month Wait, what? No Yes, you heard me 18 plus products for 261 million people Who are you going to call? Me I do it all faster than you can be still with me Good, finger linguist Say, makasimas to one of the largest JRB Java and Go clusters in Asia 1 in 4 in the nations have me and their pop Everyday my riders cover 16.5 million kilometers That's more than 21 round trips to the moon Does your app go where no man's gone before? I didn't think so Oh, sorry, Neil Most importantly 2.5 million people rely on me for the income every day I help fill values run businesses and move an entire nation I know Where do I get the energy? Wait, now going to other countries too Vietnam, Singapore, Thailand and Philippines? You ready? Now What were you saying? Right, you were looking for a job Say something Super Come help us First, thank you to Gojack for sporting venue food having as big a share as us How many of you here have actually looked at the fees Github Who is the last committer? For more fees check round talk to her Gojack also has an excellent medium blog Go check it out They have tags They talk about life They talk about culture Everything about that So, I mean I'm sure the speakers will be hang around a while as well So just do whatever you want if you want to head home tired it has been a very packed meet up with a lot of content So after that that's it Thank you everyone