 Baiklah, mari kita mulakan. Baiklah, terima kasih kerana datang kerana bercakap dengan saya. Nama saya, Gerald. Saya baru-baru-baru berjaya dari NUS. Icon dan ikon berdua-dua. Sebenarnya, saya selalu melakukannya memperkenalkan diri saya dengan cara yang berbeda dan keadaan yang berbeza. Apabila saya bercakap dengan pekerjaan, saya adalah seorang pelajar yang mempunyai ikon yang mempunyai. Apabila saya bercakap dengan orang data, saya adalah seorang pelajar yang mempunyai ikon yang mempunyai. Jadi, saya tahu sedikit tentang cara lawan berlaku, sedikit tentang pahlawan. Dan cara saya melakukannya dengan perjalanan data dan analatik adalah hanya mencari mencari apa yang saya belajar dari sekolah, apa yang saya berminat dengan R, dan apa yang saya fikir bahwa orang yang berminat dalam perjalanan kita sekarang. Kamu memasukkan perjalanan data dan analatik dan orang akan menunjukkan bercakap. Ia sangat menarik kerana jika kamu fikirkan tentang ini, apabila saya pertama meminatkan perjalanan data dan analatik di dalam perjalanan ini, itu mungkin 3 tahun lalu. Saya terlalu belajar untuk melepaskan modul-modul saya untuk lawan. Jadi, kamu belajar perjalanan yang sangat standard, lawan berkriminal. Kamu tidak belajar apa-apa perjalanan. Kamu tidak belajar program apa-apa saja. Kemudian, satu hari ini, ada satu artikel yang keluar dalam perjalanan. Dan ia berkata, Well, menurut perjalanan perniagaan lawan kamu, AI akan mengubah semua lelaki. Jadi, kamu tidak mempunyai kerja apabila kamu mempunyai kerja. Dan saya seperti, Oh, saya tidak akan mempunyai kerja. Jadi, apa yang saya lakukan sekarang? Jadi, saya seperti, Okay, sejak saya menolak kode, saya suka membuat peluang dan sebagainya, jika tidak, saya hanya akan mencari apa yang ini berlaku. Dan apabila saya mencari apa yang ini berlaku, apabila saya mencari apa yang saya menarik di dalamnya, dan ia berlaku mencari ini berlaku dengan beberapa lelaki dalam sekolah lawan, yang juga menarik di kedua-dua lelaki lawan dan juga perjalanan program. Jadi, ia adalah cara saya menerima. Dan, Well, mana yang kamu mahu, saya rasa? Kamu mahu perjalanan data, data perjuang, data teknologi dan kemari jauh. Sangat bagus. Okay, tapi kenapa kamu tidak mahu memilih pilihan dan memasukkan kata-kata? Lekas? Baiklah. Baiklah. Itu saja pengalaman pertama. Saya akan membincangkan diri saya dan sebagainya. Jadi, seperti saya katakan, kita ada kesan yang baik untuk orang di tempat hari ini, jadi saya hanya akan memulakan program ini. Saya ada beberapa pertanyaan yang saya mahu beritahu. Sebenarnya, hanya dua yang penting kerana tidak cukup masa untuk beritahu segala-galanya. Apabila Taha dan organisen saya mengalami saya, lelaki lelaki dan segala-galanya. Saya berfikir saya akan berada di bilik sebuah orang data. Jadi, apa yang saya perlu beri? Apa yang saya perlu bercakap tentang? Kamu semua sudah tidak tahu? Atau bagaimana saya dapat berkongsi pada mesyuarat ini? Saya berfikir saya hanya hanya... bercakap sedikit tentang hal yang saya belajar daripada sekolah lawan, alasan domen dan bagaimana bagaimana bagaimana kita melakukannya. Apa yang kita lakukan apabila saya sedang melakukannya, dan apabila saya mulakan. Jadi, siapa yang telah menjadi sekolah lawan? Minus lelaki lelaki? Wow, itu sebuah hal. Baguslah. Ya. Jadi, pertama pertanyaan yang saya mahu beritahu adalah, apakah sains data akan berjaya menggunakan lawan? Saya rasa sebuah keselamatan adalah pada sebuah hal. Kerana kita tahu bahawa sains data dapat menggunakan semuanya. Apa yang saya bermakna dengan sebuah hal? Kita akan menjelaskan itu. Dan lawan itu menarik kerana... ia adalah salah satu perkara yang digunakan dengan sebuah hidup manusia dan sebuah rakyat manusia setiap masa. Jadi, anda mempunyai kes. Anda membuat aplikasi. Sebuah rakyat semua mencari atau mencapai sesuatu rakyat untuk orang lain. Rakyat untuk rumah. Rakyat untuk dukungan. Rakyat untuk lakukan sesuatu. Rakyat untuk tidak lakukan sesuatu. Dan jika sains data akan bermakna menggunakan sebuah rakyat, ia perlu dapatkan membuatnya dengan cara yang menyerahkan kemahiran lawan. Yang penting, hidup manusia akan berubah. Kita tidak boleh hanya masuk ke dalam dengan model yang mempunyai kemahiran yang mempunyai kemahiran, dan mencari untuk menggunakan semua ini. Ini adalah pertanyaan bahagia dan sebagainya. Jadi, pertanyaan kedua saya akan meminta, bagaimana bagaimana sains data dan sebagainya? Profesional dan sebagainya sebenarnya memperkenalkan proses ini. Adakah anda perlu menjadi seorang lelaki untuk melakukan perjalanan kemahiran lawan? Saya pasti anda dapat mengetahui jawapan saya. Saya mungkin tidak perlu berkata. Jadi, semasa kita berada dalam perjalanan kemahiran lawan, saya berfikir saya harus memulakan beberapa kemahiran. Jadi, apa yang saya akan menjelaskan, itu hanya menurut saya beberapa perkara yang lelaki. Apa yang lelaki adalah sebuah pertanyaan yang telah dibuat tentang lelaki untuk beberapa tahun. Jadi, tidak ada jawapan, tetapi hanya sebuah perspektif. Dan bagaimana itu menjadikan apa yang saya rasa, apa yang saya rasa kita boleh menggunakan perjalanan kemahiran lawan. Jadi, saya hanya akan menjadikan beberapa perjalanan kemahiran lawan dan sebagainya. Pertama, perkara pertama yang saya mahu membuat adalah, perjalanan kemahiran lawan dapat menjadikan kemahiran lawan. Pertama, ini sebab perjalanan lawan adalah sebuah masalah data prosesinya. Jadi, apa yang saya maksudkan dengan masalah data prosesinya? Pertama, ini adalah sebuah prosesi yang anda dengar sebab lelaki yang menggunakan perjalanan kemahiran lawan. Jadi, saya ada sebuah jam untuk menjadikan apa yang lawan adalah dan saya hanya akan menjadikan perjalanan yang sangat mudah. Lawan, dalam perjalanan, adalah sebagainya. Anda menulis sesuatu perjalanan dan anda mencubanya sesuatu... anda menulis sesuatu perjalanan yang mencubanya sesuatu kemahiran kemahiran lawan. Apabila ia menulis sesuatu perjalanan, anda menulis sesuatu perjalanan dan algoritmasa mencubanya sesuatu perjalanan. Dan ia hanya sebab lawan berlaku dalam perjalanan yang lebih langsung dan lebih berlaku. Tetapi, juga, ia mempunyai perjalanan yang lebih besar. Jadi, beberapa orang saya bercakap menggunakan energi bahawa lawan seperti sistem operasi secara society. Dan, orang yang bekerja di sini seperti lelaki yang anda boleh fikir dengan mereka mencubanya kemahiran lawan yang besar. Itu kemahiran lawan, atau lawan. Jadi, ini adalah contoh lawan. Saya hanya mengambil ini dari website Pino Code. Ia adalah lawan yang dipercayai di Kriminal Beach of Trust untuk... ia adalah sebabnya apabila anda mempunyai duit. Jadi, jika anda menikmati perjalanan yang baru, keadaan yang mengambil orang-orang di beberapa perjalanan yang telah dipercayai dengan kemahiran yang tidak berlaku, ini adalah sebuah perjalanan yang berlaku daripada perkara lain. Jadi, kita lihat perkara yang tidak berlaku. Pertama anda akan melihat yang di atas ini, adalah perjalanan yang lama atau teruk atau teruk, atau teruk, dan sehingga... setelah itu, ia tidak jelas, tetapi sebabnya ia tidak jelas kerana ada sebuah bilik untuk membuat kemahiran yang membantu kita memakai kes-kacauan. Kita akan memaksa. Kemudian, kemahiran yang dipercayai bukan setiap statu, tetapi kemahiran yang anda lakukan. Jadi, yang kemahiran yang sangat mudah adalah sekarang, apa yang lelaki membuat? Jadi, saya tidak membuat ini sendiri. Jadi, cepat. Cepat membuat kemahiran yang lelaki membuat. Jadi, saya bermakna, anda tahu yang lelaki ini adalah OJ Simpson? Sangat terkenal. Apa pun, Ya. Ada banyak kemahiran tentang apa yang lelaki membuat. Ada beberapa orang fikir kita salah, ada beberapa orang fikir kita... Jika anda melihat televisi Amerika, anda tahu, kita bergabung dengan orang kemahiran mengetahui apa yang kita mahu. Pukul yang baik. Ya, tapi apa yang kita benar-benar lelaki adalah, ini benar-benar sebuah banyak kemahiran data. Ada sebuah kemahiran yang seluruh. Anda masuk dan menemukan apa yang paling penting kemahiran di sana. Bagaimana jika anda memakai kemahiran di sebuah kemahiran, kemahiran yang anda dapat menunjukkan bahawa kemahiran ini telah membuat kemahiran, atau telah membuat kemahiran kontrak dan sebagainya. Jadi, apa yang kita benar-benar lelaki dengan kemahiran data? Sebab itulah saya katakan kemahiran data adalah sebuah kemahiran data. Jika anda melihat model yang sangat mudah dan sangat mudah untuk kemahiran yang anda lelaki membuat kemahiran. Apabila kemahiran di dalam bilik, anda akan tidak lelaki untuk menghidupi objek. Anda tahu. Pergi, pergi, pergi. Ia juga berbunuh dengan orang dan kemahiran. Bagus. Ia adalah terbunuh yang terbunuh. Ya. Selain beberapa masa, anda mempunyai pengalaman dan anda tahu bagaimana kemahiran akan dilakukan. Ya. Anda tidak lelaki kemahiran data lagi. Anda hanya beritahu dan beritahu klien anda apa yang akan berlaku di depan. Bagus. Bagus. Ya. Itu benar. Bukan itu yang kita lelaki. Bukan itu yang kita lelaki. Bukan itu yang kita lelaki. Bukan itu yang kita lelaki. Ya. Anda benar-benar betul. Sebab itu saya perlu beritahu perkara. Ia hanya sebuah kemahiran. Menurut saya kemahiran itu sebuah kemahiran yang muda. Jika anda lebih senior anda menjadi api cara mengbekang klien dan lawan. Tetapi orang-orang yang bekerja di lawan yang mengambil orang-orang yang berlahiran. Jadi beberapa model memanjurkan. Sudah semuanya stabil dan senang. Pada saat you memanjurkan sesuatu? keadaan kekuatan yang bermain x ialah sumber 1 tanpa sumber 3, iaitu keadaan kekuatan yang kita dapat, iaitu kek serangan. Maafkan saya, saya melihatkan kue ikon. Jadi saya melihatkan ini. Jadi jika saya memakai sumber ini, sesuatu seperti ini, keadaan kekuatan yang bermain x ialah sumber apa yang berlaku, sumber f ialah sumber legal. sumber a ialah sumber fungsi yang mencaya berapa you menang. Jika anda menang, dan sumber f ialah sumber fungsi yang mencaya berapa you menang. So just math fluff really just to make it more formalized but just small point we assume that the legal fees are fixed whether you win or lose which lawyers will know it's not true but just for simplicity let's just say so. Why am I talking about modeling this in terms of math? We'll get to this. So basically a small part of what lawyers do is to advise on these two things. What's the chances of winning? How much will you win? And of course in exchange for that they charge you this, the cost. And there are other costs that you incur when you fight a legal case of course. Right, so an important concept that lawyers will go through in year one of law school. These are the only two Latin words I will use today. It's pronounced steri decisis I think. So it's basically saying that when you have a common law system that we have in Singapore, the current case depends on what has been decided previously. So the quants in the room will notice as an autoregressive process. AR1, ARP, whatever. So what it means is that when you have a case, lawyers will have to look through previous cases and the previous cases are really, really important because if the previous case is the same and your case is the same you have to decide the same way because of this concept of steri decisis and I've taken an excerpt from a relatively famous criminal breach of trust case. You don't have to read it, you don't have to understand it. Just look at the blue parts. These are past cases that have been cited by the court in its reasoning in coming to its decision. So these are the past cases that look at. So in a sense that looking at the past data points plus here there's only three and you'll also notice that some of the cases are pretty old. Ya, sorry. So this was the case decided this year and a very famous case I don't have to say which one I think. And they still have to look at, in theory they still have to look at cases that were decided 40 years ago, 50 years ago if it is similar enough to the present case. Of course things may have changed and that will be part of the reasoning that they use. So I'm going through all of this stuff to just come to this probably controversial equation. Probability of winning for case i is equal to some function of the past decision that have been made relative to relevant to the case. I think about law in that case. The facts of the case. So what do I mean? If a case in the past is very similar to your current case it becomes very, very important. It may even be the case just looking at that one data point you would decide what happens in this present dispute. So the facts of the past cases are so important. The facts of your case is important. The facts of the past cases are so important. Procedure which is how the case got here. Who first, whether there was any interlockritary, basically intermediate applications and who are parties involved, who is the lawyers involved, who are the judges involved. Are the lawyers like senior counsels or are the lawyers, not senior counsels. And the most important thing which the decision will know is all important error term because we don't know what goes on all the time so that's an error term. So it's some function of all these things and all these factors that goes into how decisions are made in the course of law and to power this decision making process which is one of the most important decision making processes we make in society. Lawyers look at the past cases. They look at what academics have written. They look at a lot of factors to come into this decision. And okay. So I'm just going to give you a real example. This is probably the most influential scatterplot ever plotted in Singapore's history so far. It was plotted by the Supreme Court. I didn't plot this myself. The Supreme Court plotted this graph, this scatterplot. It was trying to answer a very simple question. How many years should I give in jail for this drug trafficker? So common sense will tell you that the more drugs are trafficked, in general, the longer you spend in jail amongst other factors. So the Supreme Court plotted this graph. Then there was this analysis that was made. So don't have to read everything again. Basically just look at the blue words. Correlation between quantity of diamorphine and the length of the imprisonment imposed. However, such a correlation is weak. And it's somewhat weak. The correlation is somewhat weak. And the final sentence that was imposed is about eight years for trafficking 8.98 grams of diamorphine. So it's quite a lot of diamorphine, by the way. You can make a lot of people very sick with that amount. So, take a look at this graph again. I'm sure some of you are already wondering, so what is the correlation? Like, what is the number? Give me the numbers, right? So let's play a game. It's called Guess the Correlation. Let's play this game. How many of you have actually played this game? It's a real game. I play it when I'm bored. Yeah, because I'm like that. So looking at this graph, what's the correlation that you think is the correct one? Just have a mental answer in your mind. Sorry? So, who says A? A, okay. B. More. C. And D. Two hands. Okay, three. Okay, right. So, if you pick C, I know your statistician because C is the global most likely outcome, right? According to some studies. But actually the answer is D. It's 0.81. According to, I fired up Jupiter and I got the numbers. I also ran a very simple one variable regression. You know, it's significant and everything. So, so this is this is the correlation of 0.81. And, well, if you're wondering whether in the Supreme Court they looked at the correlation numbers, the answer is they didn't. Because numbers are not considered very important in currently in the judicial making process. Well, it's interesting to me because when I look that it's lost and I'm like, okay, the judge is saying that correlation is weak. What's the correlation? Oh, it's 0.8. So, is it weak or is it not weak? I'm a bit conflicted now because my ikhwan side would say, well, it's pretty strong as 0.8. You never get 0.8 in real life. But in in the law, we would say it's a weak correlation. And I think I'm not really trying to say that the judge was wrong or anything. I'm just trying to say that, you know, data can actually help. You can have some numbers to support your decision-making process when you are looking at something like that. So, when you're thinking of, you know, what have the past cases been like and using that to decide the current case, you can actually use some data, you know, data science concept and skills. So, this is interesting because in that case, the prosecution who is the authority that tries to get people in jail. Okay, no, that's not fair. They charge people. They're police basically. They were trying to argue that the sentence of the order was too low and they went up to appeal and they were basically saying that, look, it's out of the line, it's out of the line. It's too low. Look at the other cases. But the judge disagreed and said, you know, eight years it's fine. So, this brings me to my next point. First point is that law is partly a data processing problem. Second point, current methods of processing data are inefficient. I can get away with saying this now because there are not many lawyers in the room. Plus, I think the times have sort of changed because if I said this three years ago, I would be excommunicated from the community and you're not allowed to practice law anymore or whatever. But, I think lawyers have increasingly seem to acknowledge this issue and the thing is that lawyers are really smart, really hardworking people. So, you know, it's not that people are lazy or anything, the methods haven't kept up with the times for various reasons. So, anybody know this? So, this is from a famous TV show, Suits. And a general rule that I apply to explaining what law is about. Whatever is on American television, the opposite is true. So, why do I say it's inefficient? It doesn't remember when this is from, this is actually from one of the early episodes. I think, episode four or five of Season One. And, and this guy an associate for those who don't watch, he's like a young, just started lawyer who's like, you know, he has photographic memory. So, he's like the protagonist with a hero in the story, right? And he somehow manages to get himself a job reading this whole room of documents or briefs to check for typographical errors, problems with it. So, I would tell people that when they ask me whether Suits is realistic, I would tell them that if Suits is realistic, this guy would still be reading these documents in Season Six and starting in Season One. Right? So, in the show, he did it in a day which was like, doesn't happen. So, when I say inefficient, you know, sorry, it's common. Everything American television is, legal practice is not. That's a good road to go by. So, how do lawyers do things like proofread documents? Well, they literally read them. So, if you talk about how do they do due diligence, due diligence is this process where if a company wants to buy another company, he has to figure out whether they're worth buying. So, they have to look into their documents, have they done their shares properly? How many contracts do they have? What's their value? The assets? A lot of other things. And, what they would do, now it's a lot better, by the way, but what they used to do like 10 years ago is, read all the documents. If the company had 200 subsidiaries who has operated for like 50 years, so that's 10,000 effective years of business. They read everything that, essentially everything that the company has to offer. Then they would, of course, charge the clients for it. That's the most important thing. So, they read everything. Discovery is this process where you're litigating it from the other side with the other side and you want to know certain information that could help your case. So, you ask them, can you please disclose this thing so that we can find information to sue you on. As you can tell, that's probably not a very friendly exchange and they probably don't want to give you stuff. So, what people would do is, they would give you like a huge amount of documents, hoping that you wouldn't find anything. So, they would just swam you with documents and then you get your associates. Look through these things, find me the smoking gun. So, of course, they would also read the cases manually and that's how it's done but that's really from maybe about 10 years ago. So, it's not because lawyers are lazy or they're all really hard-working and really smart people. It's just that it's not the way it's taught in law school. So, when it's taught in law school, what really it's not the way it's taught in law school, what really it's taught is, well, you read everything because if you don't read anything, you'll fail your exams. Just read everything, don't be lazy, be hard-working, read everything and then you can pass your exams. So, why is this the case in legal industry? So, some responses that you always hear it's awkward. We've always done it this way or like numbers don't help, you can't reduce a lot of the equation. And part of the day is this US poster, Nate Silver if you have heard of him. So, they have their own matplot color scheme if some of you have used it. And sometimes people would say certain things like only way to be 100% sure is if you read everything yourself. Which is true. The thing is all of this is actually true. Up to very recently, there weren't the tech really wasn't reliable enough to do the kind of things that lawyers do to a degree of reliability that respected the needs of the legal industry. They respected the needs the reality that we are talking about real human lives and rights. So, it was only in the last five or six years when everybody started talking about AI and everybody started talking about data analytics that and there was this explosion of techniques that was being used. The attack really seemed to me at least come out with a reasonable first cut. When I say first cut it means that basically it means if lawyers had to make a decision the tech would just browse everything and give them initial brief after which the lawyers can make their own qualitative assessments. So, there was this interesting research that was done just like two weeks ago by a US startup and I'm sorry for showing you a pie chart. It's not my pie chart. It's their pie chart. Ya, they just found that basically the key finding is that lawyers miss cases and judges realise and this affects their client's chances. So, this idea that you should manually do everything because manual is the only way to ensure that everything works it doesn't work 100% as well. You miss cases and that makes it a problem. So of course this startup has their own interest because they offer like a legal research tool. Ya, that's why they say this kind of things. By the way, we don't do legal research. So, part of this reason is because there's this thing called the billable hours model just to get into a little bit about the commerce side of things. What this means is that you charge based on how many hours you spend. That's all. One hour, $1, $2, $2. It's great. It's great. Ya, you know, it's how suits is so you have high street officers, tailored suits, billable hours. Great. No, imagine if like, you know, if programmers did this billable hours, please note that your project cost will be directly proportional to the time complexity and the memory complexity of the code we write for you. The more complex it is, the more work we've done. So, the more you should pay us. So, this is kind of like how weird it would be if we talk about it in efficiency terms, which is I don't actually have an incentive to reduce the time complexity or the memory complexity of the algorithm because well, I just want to get the hours and any association in the room you have and you have a quota of hours to hit, right? You have a quota, right? If you don't, what happens? Let's leave it as that. So, there's some inefficiency and the proposition really is quite simple. Data science can reduce it. It won't eradicate it. It will just make things a bit simpler. It can reduce it inefficiency and so if I want to talk about a general problem that we have from a data perspective, it's when we have a document collection D, this is just a customary abbreviation, right? Capital D. For each document we want to see something like is this document relevant? It's a classification problem. Does this document contain something really specific like a jurisdiction clause or a currency clause? So one of the famous examples in the legal industry nowadays is they will say no, when Brexit happened, Brexit, the British pound fell dramatically like 30% in a day or something like that and banks will like oh no, what's my exposure to the pound? So they will ask the lawyers what's my exposure to the pound? And the lawyers will say oh okay, give me about three months. I need to read through all your loan agreements and see what the currency of that loan agreement is and they will like what, no, we need it in 20 hours or something because it's a live thing for exchanges. Crazy, it's so fast. So you want to do something like a paragraph or sentence classification task in the contract look for the particular phrase that says this loan is denominated in British pounds or not in British pounds so you're actually trying to extract some information and these are all things that people have done using no analytics and data science so what does the collection review about state of the law our opponents people are doing data mining on judges, data mining on the opposing council it's like if you read the book called Money Ball people are saying it's Money Ball for Law it's quite exaggerated sometimes but this is what people are doing so examples are if you can cluster into specific treks of cases lots of people will be familiar with this they'll be like oh the common thread that runs through the cases or there are a few different lines of cases that support different propositions so it's sort of like clustering problem data set that's involved is a huge data set natural language data set there's judgments which I have shown you contracts every day it's brief any legal document basically is a possible source of data and I always go around telling people that look the law is probably the most important natural language corpus that we do in the world and lawyers are the only ones right now working on it whether it's for the better or worse for society you decide example is this this is a case from a Singapore website so I've just shown you the summary at the top this is a very big case you've seen all the senior councillor fighting and it just gives you some information who are involved at the bottom there is the full judgment of the case so you can just go to this website SingaporeLaw.SG can't get it wrong and how we do data extraction it's just as you expect it's NLP and then that's more NLP so NLP stands for natural language processing for those who are not familiar not but the commonly confused neuro-lingual programming thing that I don't really know what's about so both MSLUP and in general I think a mix of both conventional and AI methods are used conventional I mean regular expressions your basic string matching conventional not the stuff that you get millions of dollars to do but very important and then there's AI methods like your bag of words models the space models there are 5,000 models now they turn semantic analysis I can try more keywords try me so there's a long way to go in this in the sense that we don't really know yet as of now how we could get computers to just extract effects very reliably it's a tough problem and there the thing is that there's no natural language those of you who work with natural language there's so many different ways same thing can be expressed and this is what keeps lawyers at job actually because you need a human at the end of the day to just come and read through make sure it's correct if you run TFIDF you don't always know what you can what you will get you don't always know you can't know for sure any experts in NLP if you guys are here please raise your hand I'll go look for you after the talk so don't take it from me there are lots of companies mostly in the US that are looking at this stuff so document review in fact contract review especially case tax is the one that did the pie charts previously J.P. Morgan of course everybody knows what's J.P. Morgan so they were one of the banks that came up with in-house a system to read contracts for them to find out what the currency of the loan is and then they probably announced on the website we have now saved 3600 no no no 360,000 hours that we used to give to our lawyers using the software that we have so if you take 360,000 hours times the average hourly rate of a lawyer which is like $500 it's the amount of money they're not paying lawyers anymore it's quite scary for me but they're doing this and due diligence there's a firm that has recently moved to Singapore they are quite good Luminance is the name they basically run NLP to try to look for stuff in due diligence documents and I've spoken to some lawyers who used it and they were quite impressed because they said it really helps them first of all and it really keeps the cost of the legal matter down and one of them actually said I like this I like this word cloud thing that they show me because it helps me you know see what are the main threats in the documents and I always tell him you know word cloud is what we do at level 1000 and there's so many other things that you can do but that's the thing because even the most basic stuff there's actually some value can be unlocked from that if a word cloud helps then it helps you don't have to always use the most complex things and it's really the case that given the way the legal matter is the most simple stuff actually can really bring quite a lot of value to the industry discovery there are a few companies as well so they basically just throw all the documents into something the lawyers are involved in the process they will teach the AI or software what kind of things they are looking for they would tag initial training set of what they classify as relevant to the discovery and non-relevant to the discovery and then I'm just guessing that they use some form of backwards model the system we're just trying to find documents that are similar to those that are relevant and not those that are like not relevant sorry so for research there are quite a few companies oh you can just google this legal tech landscape all the companies will be there I've only taken like a selection yeah Ross, Rebel, Lex Machina these companies all these companies are from US I think because I guess US is just more I don't know more accepting of this stuff but research so what these people do is essentially if anybody has done semantics search semantics a mix of that there's also question answering so Ross is marketed as something based off IBM Watson and the idea is that you type in a natural language question and like what is the applicable law on bankruptcy and the system will tell you oh the applicable law on bankruptcy is so and so forth so it's powered by some NLP and some AI in the back end so these are the companies that are doing it we are not the only guys of course so right so this is the part where I'm supposed to show you what I've done I have to preface this with a few gain qualifiers because some things I can't share otherwise my co-founders will kill me as much as I want to tell you yeah this is like the bipolar disorder that I have between my lawyer side who says protect everything and the coder side who says share everything yeah so very early proletariat what we've done this one I can share because it's so early you can probably know what it is straight away but this is CBT coronavirus of trust and money stolen is on the x-axis years in jail y-axis the curve is just a simple regression line so we look at the cases extract some factors from the cases and plot a regression line and scatter plot so this is similar to what I've really shown you that the Supreme Court has plotted earlier so we're trying to use this to show lawyers that this can actually work so remember it's quite interesting because we spoke to pretty senior lawyer and he asked me this question which was like you know why do I need regression analysis because if I have enough data everything becomes normally distributed so let me just use a normal distribution and then when I heard that I was like I just spot and said oh yeah you're right because there's no way I'm going to change his mind anyway it's just this kind of was just too far but just an experiment that we did and it's interesting because if you plug in we tried plugging in the facts of the city harvest case into the model and it predicted a pretty high sentence so a lot more than what the court gave but this was of course not trained on the right data anyway just just for fun and I think that we did so this is also CBD cases and this is more recently based on the data set that we have just to see the distribution of sentences I mean this is just all data visualization simple stuff because I can't first of all it's because I was training economics so I always talk in graphs and charts and secondly because I can't talk too much about the model process the modeling process right so in 2008 there was a comprehensive overhaul of the Pinot Code the maximum sentence for criminal breach of trust was increased from 3 years to 7 years so I wanted to see what happened when you increase the maximum sentence and if you look at the graph it actually says nothing so it kind of makes sense like increasing the maximum sentence doesn't by itself affect all the sentences after that so you can't kind of see that most people get pretty far on the low end of the spectrum when the maximum is 3 years they kind of get 0 years anyway maximum is 7 years they still get 0 years if you're wondering why there's this guy who's like getting 6 years even though the maximum is 3 years in the pre 2008 era it's because he did it twice and the judge said you serve consecutively so you get 6 years so that's how it works so another thing there's this interesting trend in the in the court that they're trying to talk about sentencing people based on in their words principal factual components principal factual components of their offence and when I saw that I was like oh principal components okay so yes principal components analysis they were trying to reduce the facts of the the offence how much money you steal do you plead guilty did you commit some violent were you very violent in the process of doing this crime how many times do you do it so all of these factors reduce them into certain principal axes and use that as a guide to how to sentence the person how many years this person should get so based on a data set that we've already did previously like this I manually this wasn't PCA but I manually reduce this based on kind of roughly put together index of what factors go towards axis 1 or factors go towards axis 2 and then I put this out and it seems to kind of work because when you are high on both you also get a high sentence and when you are low on both you kind of get a low sentence but of course there is some it isn't so neat as the data that you obviously see on blog post which is like you can draw a perfect circle around each around each cluster right so this is something else that I was doing for academic research hit map typical hit map the x y axis is the in a sense the judge so each row is a judge one judge one person x axis is the year and how green it is on the left side is how often the plaintiff the complainant before this person wins the case and of course how blue is it on the right side is how often how many cases this person has decided so I was just curious to see whether there was any clear trends of people who are like all the time they are green one person here and there are some people who are all the time red so all here and this tells you that basically based on a two dimensional correlation over time and over person some people you usually would lose if you go in front of this person usually if you go in front of somebody else to win of course this is something that could be quite controversial I know people in the UK are doing this there's a firm in the UK that goes around saying we have the best data on lawyers if you consult us we'll tell you which lawyer to pick because this lawyer always wins in front of this judge and your chances are great I don't believe in that because I've gone through legal training I'm not going to tell you you look at this map you can choose the best lawyer it's not the case because it all depends on well sometimes the lawyers will get lousy cases or the judge would just happen to you know be assigned all the cases where the complainants have clearly stronger cases so this was just one of the intermediate steps that I took in the research to kind of have a rough sense and then after that I was I applied this and I put in some controls don't need to explain what controls are no right so I put in some controls for no facts of the case and try to see the impact of just the judge the impact of the judge alone when you control for the facts of the cases control for circumstances of the cases is there still a significant correlation and according to the numbers there wasn't only a few one or two people out of like 500 had significantly different chances for complainants than the other judges so it means basically that the judges are quite neutral very good if it was at the every round I wouldn't be able to publish it so nothing that we would do we're doing this is this is to do with divorce although the judge although the courts always say that you can't reduce the law to an equation well in the area of divorce there's one thing that they've applied a very mathematical framework to which is when you divorce you need to split your assets hopefully not nobody has to go through this but you need to split your assets because it used to be one part one whole and now you've got to split them into basically two parts and the court has to decide what is a fair division do I give husband 40% and wife 60% do I give husband 60% and wife 40% so this is a division and basically they're calculating a ratio so we looked at all the publicly available past cases of divorces and what the judges have said in splitting the assets and then we plotted some data this is just a selection of the data but the direct ratio refers to how much monetary how much money you put into the marriage so if let's say the main asset is your house it's worth $1 million you pay $500,000 your spouse pays $500,000 it's a 50-50 direct ratio for example indirect ratio is the intangible stuff so if you take care of the kids more you get 60% if you do the housework you get more if you are irresponsible criminal you get less so that's the intangible part so they will kind of they are really trying to reduce it to a number actually and then they will take the average of these two numbers and if they need to they will adjust further and they will find the ultimate division that they think is right for you the numbers here are expressed in terms of the husband share so let's say 1.0 it means that the husband was found to have contributed 100% 100% of the money so you can kind of see very quickly that based on this there are set of dedicated divorces we are quite in a skilled situation when husband tend to contribute more of the money but when it comes to the contributing to the household it's equal and in fact tends to be that the wives contribute more to the household so pretty stereotypical agent kind of setup and this is shown in the cases interesting so this was part of the data that we crunched to come out of this thing that the courts are referring to as the outcome simulator because they don't want to use the word prediction they want to say I'm going to simulate your outcome I'm not going to predict the outcome I'm going to simulate your outcome and this is the front end how it looks basically you go to there's a model behind this you go to the app you enter in some facts about your marriage that we have found that courts tend to deem as relevant and then at the end of the day it gives you just a simulation because the exact model I can't share but let me just say that we follow established data science practices and methods and algorithms and it gives you the predicted ratios the simulated ratios so just some of the challenges that from doing all this stuff I kind of think about which I'm not sure whether it's the same for all types of data science because I really only look at legal data sets there isn't enough data I'm sure this is probably the same you typically have few hundred in Singapore oh ya so this comic is from everybody's favourite XKCD dangers of extrapolation so that's a big problem as you all know because at the start I said that you really want to be sure you can't just extrapolate and then we say oh you know you get less money from your marriage because based on 25 data points you can't just do that and there isn't enough public data there's a lot of data actually because there's 6000 divorces that go through the courts every year and that's just a non-Muslim divorces but out of that only about less than 100% actually get published of course there are reasons why obviously you can't just publish everybody's divorce matters unless you anonymise them of course but I guess this is the other the other problem data was collected in the time before data science you know it's collected in terms of in handwritten forms sometimes a lot of data that we have in the law it's not even digitised yet or it's scanned but not OCR there was a big push in the last 20 years to just get law firms to digitise their stuff and even then there was a big fight oh I need my copy because I learn better that way and it's also true so there was a big fight and there's still a big fight going on at the digitisation phase and if it's not digital we can't do anything from a data science perspective and of course there are all these typos it's annoying I will run all my code and then it will crash for some reason because there's a typo that I assume wasn't there and I have to go and rerun the whole pipeline and it's especially a case when it comes to names because when you talk about let's say I'm presenting lawyer's names the same lawyer has 10 different names in the database because like if for example sometimes the Indian names have the son of s slash o and some people write it as s s dot o dot some people write it as s o without any slash or whatever so you know I mean of course you could just scrub the punctuation off and everything but sometimes you are trying to code fast and you don't really realise these things happen so it's not very clean and another thing which is probably unique to the legal sector is that lawyers can reasonably disagree on anything so when your definitions are not set your data definitions are not clear you can't just it becomes difficult to get data that people agree on so simple example is we were trying to look at word count has the average word count of certain types of legal documents like judgements that judges gone up over the years simple as that then we were talking about what words do we count in the judgement do paragraph numbers count do headings count do quotes count do appendices count and all these things like if the citation of a case has dots is one letter, one word or the whole thing, one word and then you know you can fight about this and if you don't if we don't get to if we don't have final say the lawyers can fight about this to essentially no end and the problem is they're right so you can't just say I veto your decision as the data scientist and say I just want to have this count even though you try to tell them oh you know as long as it's consistent the average is consistent across it's kind of the same it's not that important but you know they still want for valid reasons to have the exact count and it becomes a problem even for word count and another another place where the disagreement occurs on the outcome so if you're trying to predict the outcome of a case lawyers will tell you but it's not just a binary thing it's sometimes it's like a half-half sometimes it's there are five different levels of is it a complete win a total win a very complete win did we win on both liability and quantum did we also win on cost there's all these shades of winning and losing that's not very clear in law and sometimes for example in a criminal case if you if you are your client is found guilty but you've reduced the sentence that's also a win and that's very valid so it becomes a question of definitions and it becomes something that you have to really work with the legal team and talk to them about it and understand what's going on so some data related problems imbalance I'm working on a data set now the positive rate is like 1% less than 1% imbalance so as you know sometimes when it's imbalance you have all these other problems that come up you have to do all these countermeasures which are quite annoying conflicting data so some cases are just wrong and the courts will say this this previous case I find to be wrong henceforth no one shall follow it and sometimes it's post facto which is only after like 10 years they will say it's wrong and that's because social values have changed and so on and high velocity the law keeps changing that's what keeps lawyers and their job if it's fixed nobody will have a job because it's just the same you learn it once and it doesn't move but it keeps changing so your models have to change as well that's another challenge and yeah there aren't enough people working on this area so it's really niche area because people have the misconception that you need to know the law you don't actually have to it helps of course but knowing the data is really good enough yeah so there's nobody but a little bit of time so brings me to the last point you know other professions you can help so this is the famous friend diagram for data science you know there are three skills and it's not possible to learn all of them trust me I've tried very hard and no I don't know all of them the lawyers are here substantive expertise hackers engineers you are here and the quants are here and one thing the lawyers really need to help with is the math and also the code because if you do this on Google also complete you actually see this I did this about two days ago you see the World Cup is there I just why are lawyers bad at math comes out it just comes out okay I'm aware that we are at Microsoft so I tried this on Bing but it didn't come out so maybe Bing is more sympathetic to lawyers and don't ask me about the second one I have no idea I make no comment I make no comment on that I think they are really good so maybe I have no idea and there are really this big open questions that we need data scientists and AI researchers come in to help us answer I don't know the answers to any of this like how do we get the facts or applicable law interesting parts of things from the text itself is there a way to do it is there can you just throw everything into LSTM with attention and everything I don't even know what that is by the way how do we deal with missing data there's a lot of missing data do you impute them there's all these techniques data synthesis so the famous example there's a cat you flip it around it's still a cat you rotate it it's still a cat you can use these five different cat photos to train the cat classifier but if I have a case and then I maybe I tweak it a little bit using some vector math is it still the same case is it a legit way of doing data synthesis I don't know I'm just trying to see what I can it works if I try it out architecture this is deep stuff other models that are better for legal purposes maybe they are better reflective of how lawyers think just an idea for NLP everybody knows you just use an LSTM for some reason it works for image use and CNN I'm kidding of course there's a lot more no one says to that interpretability the judges lawyers always want interpretation that's important because if you want to change someone's life you got to explain it to them that's the reason so it has to be interpretable but you can't just all most of the stuff that we have is black box quite black box you can't just neural network into neural network into actually boost and then here's the outcome it's not going to work that way and finally the most hard question really is the human system how do we imbued data science culture into an industry that's really led by qualitative thinkers i.e. lawyers they think qualitatively very complicated very deep qualitative thinking but still not qualitative by nature so the point is really that interdisciplinary partnership today work this was a very famous paper published 2 years ago very ambitious title the authors some guy from Amazon computer science is a lawyer psychologist what is this whole bunch of interdisciplinary people working together to create something that predicts what the European Court of Human Rights will say using NLP for example and it was interesting because when this came out the press reported this in a very interesting way you'll see this is what the press said oh yay artificial intelligence judge developed by UCL computer scientist all judges will henceforth be replaced and let me tell you what they did they took about 600 cases ran vector space model so they took i think n-gram features so one hot encoding n-gram 1 to 4 then they did TFIDF and then they reduced it using spectral clustering basically LSA your latent symmetric analysis into topics fed the n-gram features and also the topics into an SVM linear SVM did some grid search 79% accuracy published AI will replace all judges I'm not kidding you read the paper so this is the kind of work that's being done and I'm not trying to make fun of it at all I'm saying that it's a field that's so new so full of opportunity that if you just did something simple you could still be world class you could still be leading the field in that because nobody else doing it and it's just really new so the last point is really that there is a really big problem that can be solved here so I couldn't get data on Singapore so I just got this data on the US legal market it's $437 billion in 2016 I can't give you 2017 because that's behind a paywall $6,000 for a report crazy and that's just how big the legal market is and we haven't even included those that can't afford the lawyers you know how many of you have ever paid a lawyer to do something for yourself not for our parent not for us for yourself just for yourself there's one there's one there's one out of about oh there's two so two out of about 50 so why haven't you okay maybe I should not ask that question that way but you know it's not the cheapest thing to ever buy and for some people it's just something they can never afford so if we do some quick math you know if we I took the median income so this is an economist thing but median income of a Singaporean about $3.8,000 you take away 50% on Wi-Fi, taxes and other stuff that you have to pay you know you divide it by 200 hours a month so you get you basically earning $10 a month $10 a month $10 an hour effectively and average lawyer bills $300 an hour and I'm I'm sure the lawyers will agree with me this is a conservative estimate right and then objections $300 so if we just divide this $300 an hour with $10 an hour that you make you have to work you have to work 30 more than 30 hours to afford 1 hour of a lawyer's time that's how it is nowadays and that's not even including the other copies you got to pay the other costs you got to you know food for printing and other stuff and if you lose you got to pay some of the costs as well for the other side problem and it's known in the legal industry as an access to justice problem an access to legal services people can't access the legal services most people can't afford a lawyer that's the fact and you've asked the lawyers as well they will tell you that if you don't have a lawyer your chances are really seriously affected it's not as if you can just go to court and then you just win you know if somebody has for example if you're getting into a bankruptcy court the banks will have you know make some lawyer and you'll just be there and you'll be like even if you have a great case your chances are affected of course a lot of us will say this because they will want you to engage them so this is really a big problem that is hard to that what I really said at the start it has to be meaningful and the meaningfulness for me and why we do this is really because you know if data science can reduce the legal inefficiencies people will have better lives you'll have access to justice it's a thing that it's slowly trying to gain slowly gaining attraction in the UK this 17-year-old came out with a chatbot that helps people appeal parking tickets and like he claims that 300,000 people have not paid the parking tickets because of that something like that and yeah to recap what I've said first point is that data science can be meaning fully applied to law so partly it's a data processing problem and current methods are not totally efficient I mean I'm very interested in efficiency as an economist I want to make people better off without making anyone worse off not even lawyers data science can reduce the inefficiencies to some extent not completely for sure but to some extent and you can help as a data scientist if you're interested in this thing you can actually just go pick up some legal stuff run your own thing there's more problems than people working on them so we will come in a sense competition but this is a big problem of using data in the law law is to help with the math to distribute partnerships and there's a big problem alright so I'm going to end here any any questions because I have I have books to give out because if you don't get them I will keep them right I will keep them right it's okay any questions for me any yes anyone take biophones or be dependent in any context independent rate but in a legal case or legal scenario context is very important yes so do you have any pre-trained word admitting for legal opposites or good question so I need to repeat your question for the recording so the question is in using vector space models like TFI, DF word embedding context is very important and in the legal context indeed legal context is also very important so do we have any pre-trained models for legal users the answer is not that I'm aware of like internally in my startup we have some simple version of it pre-trained but there's no you know glove or word to vex that's already pre-trained for you to use and just plug and play into your application I'm still waiting for somebody to come out of it if you can come out of it let me know so you know it's always on news like it's always on 20 news groups or something and you know it's not 100% applicable so that's why we have to make some extra effort every time okay what do you want I'll collect it at the end yeah so any other questions there was another one yes so just how you show some trust about some of the research you did the one with CBD cases and number of years so for that I'm going to ask how do you gather the depth and how do you attract people to go see so do you do it manually or are you already using NLP a mix of both like I'll be lying if I say I did everything automatically because the state of the art as far as I know doesn't go that far anyway so it's a mix of like I said both conventional methods like Rejects and even just passing you know HTML passing and also some NLP involved but I'm not like the foremost expo NLP so I would just say that there's more manual than automated also because we want to be sure there's no training set and try right now so right now the manual part serves as a training set for future stuff yep oh right so the company 4 of us are from law school 3 of us have graduated already I've just graduated but it's kind of like a mix so we take the event that I went pretty seriously pretty literally so amongst the 4 of us it's a guy with law and CS it's a guy with law and business I'm the law and econ's guy and there's some guy who's like law and he quotes on the side so it's we have a mix of skills yeah and to repeat the question he's asking whether we have a mix of skills in the startup and I said yes yeah any other questions yeah sure you're talking about some of this work giving people more access legal access can you see position where it will actually create a bigger divide okay so he asked whether there's a chance for legal tech basically and legal data science to create a bigger divide and the answer is it's possible like any technology it's just a tool how we use it is a different question altogether so you know if if it happens that the tech is being used by the big companies to you know size up who they're going up against more so than the small little guys and of course it would be oppressive so it's it's really a question more of the human side of things than the tech side of things like the computer as far as I know there's no AI that knows good and bad right now so it will just do whatever it's what to do right am I wrong I look like you look like so the question is whether legal tech can make low cheap and accessible the question is I mean I mean my answer is it's what I hope will happen so in the sense it's already happening there are all of these what they call access to justice push access to justice hackathons so people will come out of these apps that are like you know essentially most of them are chatbots I don't know why but chatbots and anybody can just go on and ask a few questions about oh you know I have this problem and chatbots will say oh what's your problem it's housing problem or where do you live and all this and then the chatbot will kind of just try arch and and direct them to the right authorities in some cases it's just the authorities already not lawyers so this is happening and um sorry it's just not at a widespread scale yet but um something about economics of the law industry recently is that not just the people kind of fought it but even the people who can fought it they are pretending that they can't afford it so they are trying to tell lawyers can you please lower your cost and so there's a huge pressure on lawyers everywhere to you know there's like basically a price war uh sometimes and everybody is trying to look at ways to reduce cost so it would in my opinion all it would take is for a big law firm you know you are your magic circle slaughter in May suddenly realize that with the technology they can service the lower sections of the income income what they call it income classes the income records and maybe you would see in ten years time big name firms actually using technology to service the masses it's possible and it's kind of the standard theory of how disruption works by undercutting the current players at the mass market level any other questions yes so you mentioned like the data rights it's very under rotation you can generate more data points so it definitely seems like you need more data rights so you talked about trying the techniques like what have you done so far or what are you thinking about doing what have I done well nothing so I've been reading the papers so I haven't had chance to try it out because all of these are like research questions and I spend my time working on stuff that I'm being paid to do so they are usually not in the same direction yeah but it's always a little bit of my mind if I had more data I could probably do a lot more things what I've tried to do yeah I've just read the papers on data synthesis googled around not much honestly just one guy any other questions yes so the question is do I have any comments on the technology infrastructure and the legal sector and beyond just data science and is there anything that should be done I'm going to give you a biased answer everything should be done so like I said you know stuff are not digitised some stuff is still in hard copy if you go to the state courts I hope you don't have to go there but if you go there you'll be given a hard copy form and then you will submit it somewhere but in Singapore we are quite forward in this area quite fortunately the courts are quite forward in adopting this new system called the e-litigation system it's revolutionary but the idea is just really we file documents electronically but think about it it's actually a huge challenge because the level of reliability that's required in litigation is very high you can't just have a simple system that's open to tampering you have to have a really robust one so that's one thing just filing stuff online there's a lot of beyond data science there's a lot of other it's just very simple automation processes that are actually being applied to the legal industry now lots of people are looking at it so if you fill in forms you all know that you can just write a very simple script that fills in the form put in some placeholder there's a template fill in the form this could have been done 50 years ago but now it's people are looking at it and it's because it's an economic incentive people are asking for cheaper stuff so people have no choice but to come up with all these templates beyond that everything else is more or less related to data science because of the fact that you kind of need something more advanced than put in logic to deal with the new ones of the law quite usually so even simple stuff like reading contracts for example you can't just have a mega reject that catches the different ways it's expressed and you can't just do that unless you spend like a million years reading all the contracts but if that's the case why don't you just tag them yourself so most of the other stuff that I know of there's some form of machine learning inside some form of NLP that tries to address for the uncertainties in the legal industry yeah sure blockchain blockchain is interesting because when you say blockchain like lawyers get jittery because they've been told 5,000 times that blockchain will disintermediate and everything and lawyers are intermediaries so lawyers will be eradicated by blockchain I don't currently see a clear path to that to me I don't know much about blockchain because I spend more time looking at this stuff data science stuff blockchain is just a data structure to me and it's just a very special type of data structure that holds things in a certain way so when you need a good data structure that has the requirements you use blockchain and one of the requirements one of the things that lawyers are saying you can use blockchain for in the legal industry is logistics so whenever you ship stuff all of you have shipped stuff via Amazon or whatever there is this contract of sorts called the bill of lading that you sign or somebody signs on your behalf and I'm getting kind of wrong but if Stephen Lawyers please forgive me apparently there is this whole ecosystem around tracking where your goods are on the ship and you need to verify that the person who claims the good at this embarkation port is the true owner of the person who is the true recipient of what was shipped across and you have to verify and there's a lot of this integrity in the system kind of thing so I'm looking at using blockchain for that there's a lot of talk about that I've attended three conferences where I talk about blockchain bills of lading I've not seen it being done I don't know like I said I don't know about blockchain but it's the talk theoretically it's supposed to replace everybody just like AI yeah I don't know but technically you talk about blockchain is mainly if you have to move the natural contracts over the fact that you try to internalize the or the or documents or text that has really been assisted how can we repeat it I mean okay you talk about when you talk about blockchain that means it's about something about the future when you try to move into electronic contracts and it's not about handling those documents that already appeared oh okay the new contracts so your question is can we use blockchain for the new stuff the future stuff yes over like the old stuff maybe turn blockchain I haven't really talk about it I haven't talk about it myself but what I hear people are doing they just talk about smart contracts so ethereum smart contracts you know lawyers will tell you that smart contracts are not real contracts by the way I tried to quote the smart contract like solidity and everything and there were so many restrictions you can't do everything that you can in like python you have all these restrictions and I think the blockchain needs to advance a bit further at least to me before it can be used for full contracts like the kind of contracts you see 300 pages software license agreement M&A and purchase agreement currently I think blockchain has a very few few functionalities that can replicate what contracts do but maybe only about 10-20% of the simpler stuff so for future contracts I think the tech probably needs to go a bit further this is what I think based on what later I know I don't know about blockchain any other questions? I'm sitting down but you can still ask no? so do I get to stop? alright yeah thanks for coming so much thanks so much really really happy to be here so this is this is serious if you know any of the answers please look for that's kidding so I have a nice week and go and quote some legal stuff I really hope that more people can get interested in this