 Merci beaucoup, tout le monde, pour être ici aujourd'hui. Merci, tout le monde, pour votre invitation. Je vais vous parler aujourd'hui de l'enseignement de transfert. C'est quelque chose que nous avons tous l'expérience dans notre vie habituelle. Si vous étiez à apprendre une nouvelle langue, vous trouverez que c'est plus facile de apprendre une autre langue après ça. C'est aussi été montré que si vous étiez à apprendre de la musique à jeunes enfants comme il y a longtemps, comme 9 mois, vous serez capable de apprendre une langue plus vite après ça. C'est cette idée que nous avons tous l'expérience, que quand nous avons appris quelque chose dans un domaine, nous pouvons apprendre mieux et plus vite d'autres choses dans d'autres domaines. Mais avant de parler d'enseignement de transfert, je voudrais parler un petit peu de hockey pour ceux de vous qui ne savent pas ce que c'est. Et ensuite, je vais vous expliquer comment l'enseignement de transfert peut être appliqué à la médecine et comment peut-être être une solution pour des problèmes très grands. Donc, qui nous sommes à hockey et à quoi nous faisons. So Hockey is a new startup company that was founded last year and we have offices in New York and here in Paris. And I'm the chief scientist and I lead here in Paris the scientific team. Thomas Closal is my partner in this startup. He's an oncologist, so he's a medical doctor. And he used to work in the field of predictive medicine. He developed techniques to predict response to chemotherapy using epigenomic data. And I used to be from academia since I was working in ENS for the last two, three years. And I've been working a lot on applications of machine learning in the field of medicine. Right now, we are a team of about ten people and we have exceptional machine learning engineers and top data scientists that performed among the best in all the international competitions such as Kaggle. And what we do is that we develop algorithms and softwares in collaborations with biotech companies, pharmaceutical companies, hospitals and academic labs. So I will show you the four major topics that we are covering in the company. So what we do is that we develop these machine learning techniques to address fundamental problems in the drug discovery and development process. And in this process there are several steps that are very time consuming and that cost a lot. If you want to develop a drug today, it will cost you around one billion dollars to perform the drug design, drug development, clinical trial, all the process from the idea to the market. And it fails a lot. It fails more than one out of two, more than 50% of the case. And to achieve, to transform this process, this workflow end to end using artificial intelligence, we focus our development on drug design. How can we use machine learning algorithms to predict molecular activity at the chemical level? We work on clinical trials. Now that you have a molecule, you want to test it in a population, how can you use the data you have accumulated until the phase three, for instance, to select the right subgroup of people that will respond to the drug. And this is also, there is a clear need for predictive models. We work on real world evidence, meaning that we work, for instance, with hospitals, with data coming from general practitioners. For instance, we work with Institut Curie to work on clinical data in a free text form, meaning that this data is written by the doctors and it's a huge amount of data, millions of documents. And we also work on images. This is a little bit transversal because it can be used for clinical trials, for real world evidence. And this is maybe one of the field right now, medical images, where deep learning can have the more impact in the next few months or few years and a lot of people trying to do that. So now I'm going to talk about transfer learning, give you some ideas about what is transfer learning, how can it be applied in medicine and then after that we'll take some time to discuss about more general ideas about collaborative artificial intelligence and the data sharing problem. So just before starting, in medicine data is often has very specific properties. The first one is that it's something that should be kept private. Medical data is something that is not to be shared to everyone, one has to be very careful. Second point, it's often coming in small data. It's really not common to go and see a doctor or a lab and he says have data for one million patients. It never happens. Often you have data sets of a few hundred images. So these two specific features are really important to bear in mind and I will show you why these two key features put in perspective this idea of transfer learning as really something that can be very useful. So first, what is transfer learning? So this is a domain of artificial intelligence and machine learning which is focused on disability that an algorithm can improve its learning capacities. Either it can learn faster or it can learn better to achieve better accuracy on a given data set through previous exposure to a different data set. So you have the task that you want to perform on your data and then you have this other data that can be helpful to solve your task. And you can use transfer learning so that your algorithm can learn other stuff on this other data sets. It can be different, it can be different types of problems, it can be a different task, it can be anything but there are some things that can be extracted. One point that is very important also is that deep learning architectures are very well suited for the transfer learning approach and the reason is that these deep learning architectures they operate by different layers in a hierarchical way and often what we realize is that the first layers can learn some things that are very general on a data set, very basic such as discovering contours or basic structures, basic textures of a cell of any medical images and that can be transferred then to another problem. And I think this precise point is one of the key hidden reasons behind the success of deep learning. As I mentioned in the earlier introduction it's also a cognitive phenomenon in humans that learning things in one domain can help you learn things faster and better in other domains. And personally we will discuss about mathematics later. Personally I think that mathematics is really a good example of that. That people who are educated in mathematics can learn faster many other things after that. But I'm speaking in the home of mathematics so I'm safe here to say that but maybe not everyone will agree. Anyway, so it's very interesting that we are entering this era of artificial intelligence where we can have this connection between a cognitive phenomenon that seems very powerful and very abstract decided that if I learn mathematics I can be good at law or philosophy. This very abstract concept is now really present today in everyday life of people doing machine learning. So this is an idea, I think it's really exciting. So I just want to give you two examples of how can it work in general for deep learning and then we go into two, three specific examples in medicine. So very generally what people do every day in the deep learning community is something like that. It's called warm restart. So what they do is that they want to solve a problem. They want to be able to classify cancer versus non-cancer from medical images for instance. So they have a bunch of say 100 images of each class. If using 100 images of each class you want to train a deep learning algorithm from scratch using completely random initial condition of your neural network weights it will most likely be a failure because you have too many parameters to train and you have not enough samples. So what you can do is say well I can't do anything please come back to me with more data. This is something you can tell to the doctor that gives you the data. But what you can do as well is something that may seem completely crazy. You say you start from your random neural network your initial condition then you train it on a very large database called ImageNet that most of you know I assume it's a database with a million images 10,000 classes that shows plane, cars, cats, dogs, whatever. And the task here is to classify all these pictures. So you train your neural network on this database. It gives you what we call a pre-trained neural network at the end of training. This neural network is niable if you give him a picture of a lion or a car to tell you if this is a lion or a car. So it has learned things. It has learned how to see things but not especially medical images. So why are we doing that? Does it have any chance to work? Then once you have this pre-trained neural network then you can use it as an initial condition to train your deep learning algorithm on the data that you are interested in. And basically you do the same as if you were starting from an initial condition but you just change the initial condition. The training process is all governed by the same back propagation algorithm that you like. But you just change the initial condition. And if you do that with just a few hundred images you will see that the improvement in performance is really astonishing. And I will show you some examples after. We can also do other things. There are many, many other possibilities. I want just to show you another one that is quite interesting. It's the idea of shared representation. It's the idea that you can train your network to classify plane and cars and lions. Then it gives you this pre-trained network. Then you start from an initial condition that is random on your medical images. But what you do is that you penalize your loss function. You had a regularization that tells you that your new network should look like the one that was trained on ImageNet or that the features, the activation of the neurons should look like the ones that are on ImageNet. So it's a process of regularization that has the same kind of effect that the warm restarts but sometimes it can work better. So let's go into some concrete examples now. Okay, so there is always in teaching this debate should I start with examples or with the theory. So I decided to start with the general theory and then I show the examples. We'll see if it works. So here is the cover of nature February this year. So it's like two months old. So already very old. And the cover of nature, it shows a paper that has really shaken the world of dermatologists. So we have a lot of dermatologists that come and see Okin right now to see how we can help them. And basically these people in this article they applied deep learning techniques to classify skin cancer using pictures. And what they show in the article is that they obtain the same level of performance in terms of classification accuracy than a group of experts, dermatologists. So in 2017, this is where we are right now. These kind of systems, if they are trained on a database that is large enough, they are able to perform as good as doctors and there are other papers that we can discuss. I can give you the reference that shows that in other problems they can even perform better. And here, this picture is just an excerpt from their article in nature. And what they show right in the middle here is called deep convolutional neural network inception V3. What is inception V3? Inception V3 is the name that Google gave to the neural network they trained on ImageNet. The same pre-trained neural network I was talking about. So you have to know that these pre-trained neural networks on ImageNet there exists a zoo of models. There exist many models. There is inception V3. Maybe there is a new one that has come out. You have Microsoft that has another network. You have Facebook. You have also academic labs, etc. Each one has its pre-trained neural network on ImageNet. Then what these guys did in nature they took that as a starting point and then they do the warm restart. So they keep the architecture and they just do the gradient descent on the weights. And the real tour de force in this article is the size of the database. I told you that it can work with a few hundred images. That's true. But if you want to achieve on this very hard problem the same kind of performance as expert dermatologist then you need around 100,000 images. And this is not easy to gather. So this is really very impressive. So this is a very concrete example and right now all the dermatologist community is really trying to understand how things are going to organize around this kind of technologies. I want just to give you two more examples of things that we do in the company. So this is a work that we've been doing with a biotech company that is developing new drugs on a disease called pulmonary fibrosis. It's a very bad disease that can kill people in a few months. And there is no real treatment at the moment. And their goal was to develop a system that can grade fibrosis. So that you have low grade it means that the fibrosis is not spread and it's not very important. And high grade means that it's very bad. Ok, it's very severe. And when they develop new molecules they want to test if I give this molecule will the grade be lower after a few weeks? So they do clinical trials, pre-clinical trials and they want to assess the grading. And usually they do that with a team of expert pathologists that look at the image so these images are taken from biopsies of the lung and they are scanned at very high resolution. So here there is a challenge here on the side is that images are gigapixel images meaning that you have 100,000 per 100,000 pixels. So it's very big images and we were given only around 100 images with 5 different classes. Ok, so you have like 20 examples per classes and images are more than 1 gigapixel. And basically we are not still at the level of the agreement of pathologists but what we realize is that if we do the neural network training from scratch then it performs around 65% accuracy in the 5 class program but if we use a neural network that was pre-trained on this carplane, lion, etc. task then it performs way better. So this is a very precise concrete example of how transfer learning can give a very important improvement of the results due to this problem of having only a few data points. And this is a very common problem in medical problems. The last example I wanted to discuss here is a project that we've been doing in the last few months. It's a challenge organized by the Kaggle platform. So Kaggle is a platform for data science competitions that gathers all the best machine learning and data science engineers and teams over the world. So it has more than 100,000 users and the platform has been acquired by Google recently. And on this platform there was a challenge with a price of 1 million dollars. And this challenge is the one that we try to play and it's called early detection of lung cancer from CT scans. So what is the game? You are given 1,500 CT scans and for each patient that you have just a label that tells you if this patient has or will develop a cancer within 12 months timeframe starting from the exam. So what is a CT scan? A CT scan is a 3D representation of what's inside your body. You have like 200 slices and each slice is around 500 x 500 pixels and it looks like that and you can reconstruct in three dimensions if you want what's inside the lung. This is very challenging because we have only around 1,000 data points and the signal is really, really small compared to the volume of data we have because the signal is hidden in the idea that this guy has an audio so it's a mass in the lung that is very important to diagnose this type of lung cancer and this nodule is just a few centimeters so it's just a few dozens of pixels patch, 3D. So you have like a very, very small tiny needle in the haystack and you have a lot of information about the lung that is completely useless and that is completely noise for your machine learning algorithm. So it's a very, we call it a very strong weak labeling problem and in this problem everything that we tried just using the data given in the competition completely failed and I mean the guys in the team are very strong and they try a lot of things and I think it's almost impossible to solve this challenge with only this data. So we had to find another way we had to find a way to bring knowledge from outside because it was really not possible to design something that would work. So the final ranking is tense so we didn't win the 1 million dollars but we still win a small price but most importantly we learned about the strategy to solve this kind of problems. So I put aside the fact that it's a 3D problem and 3D convolutional neural networks are not as easy to manipulate than two dimensionals but I would say it's a side problem. The real idea was to bring external knowledge through another data set. So we used this idea of transfer learning very strongly in this case and it was the only way for us to get something that worked. So there is this other data set called LUNA that has been designed for another task with other patients in another challenge the year before. And the other task was to work on segmentation of nodules on CT scans. It was not about early detection so it was something with different labels different patients it was a different data set. So what we did is that we trained neural network to distinguish the nodules so we split, we extract patches in three dimensions and each patch we have this annotation that tells you here you have a nodule we can give you this diameter or we can tell you here there is no nodule and this is the kind of label that we have from the LUNA data set. So this is this is an information that is localized in space. Compared to the annotation on the KGEL data which is completely not localized. They just tell us if this patient has a concern or not where is the nodule etc. So there is pretty much nothing you can really learn. So you train first this three dimensional convolutional neural network on this LUNA data set so it has five convolutional layer with pooling so that at the end with 64 channels so that at the end you flatten everything and you have just 512 and 12 vectors ok and from this vector of dimension 512 it means that you can have a representation of a given patch of 64 pixels of size ok and now what you do is that you have trained this network you have the weights, it's here you take your data from the KGEL data set you extract patches so you do it with overlap but you just scan everything on the patient all the patches then you put it in the CNN you extract your vector of representation and then you do some max pooling so that at the end you erase all the spatial information so everything becomes invariant in space and from that you can do classification with these new features we could also have done some global fine tuning of the whole process but still it was very complex to make it work and at the end we do gradient reboosting to do the classification ok and this strategy we realized that among the top 10 almost everyone has done something similar ok so it was really maybe not the only way but I really challenge everyone here in the room to achieve a very a significant result without having this transfer learning approach so here we see there are some very important problems very difficult with a 1 million dollar price and the only way to find a solution that starts to work is based on transfer learning so this is not une anecdotal technique this is really at the heart of many many applications of deep learning so now that I gave you this compelling story about the use of transfer learning in the field of medicine I would like to maybe take the time heavier to dream a little and to try to move 1 layer of abstraction above and think about what this kind of technologies could make possible in the future right now if you think about classical statistics for instance each time you have a problem of hypothesis testing of estimation of parameters you start from a white page each time you have a new data set turn your problem and you start again with transfer learning the idea is that you can build up systems that learn from a given data set then they are able to perform a task and then you can reuse them to do another task and to learn other things and you can climb like that and you can build systems to become better and better when they are exposed to different tasks it's like humans it's like this sentence like standing on the shoulder of giants this idea that artificial intelligence systems are entering an era where they are not isolated anymore these algorithms can talk to each other through transfer learning ok so why is it so important so clearly what we have shown here is that transfer learning is one of the key for me one of the key reasons behind the success of deep learning and it brings the power of machine learning to small data sets that will not be amenable but it opens the way to collaborative artificial intelligence and I would like to show you how here at Okin we imagine the future so our idea is to build a collaborative artificial intelligence platform where each contributor would be a lab, hospital doctor, whoever has some data that is labelled suppose you are a doctor you have 500 images of breast cancer that respond to chemotherapy 500 images of breast cancer that do not respond well to chemotherapy you have this data this data is very important it needs to stay private most of the time and maybe it's not enough data to build a powerful algorithm but you know that other people around the world they have this kind of data so the idea is that as a contributor you can create new algorithms or you can participate in the creation of new algorithms and these algorithms can improve each other through transfer learning going from one centre to the other and then the community of users the people, the doctors if they want to use this algorithm to make predictions and say well I need to predict if this patient will respond well to chemotherapy or not can use the algorithms that are built on the platform and so the idea is that you don't need really anymore to have just a team of someone who has the data and someone who designed the algorithm and they work together but we are thinking for the future and we think that this can become a platform system and we can move towards something that is more collaborative and with the idea that people can keep their data so we have a prototype of that concept that is working that we are currently installing in several hospitals where each user who has medical image can create algorithm just without any coding skills can apply the algorithm to new new image that come in the clinic and share the algorithms that have been trained to other people on the platform but I think really here the final point that I want to make in this conference is maybe the most important way we are doing all these nice predictive models but clearly what is really lacking today and for the future in the medical community and in artificial intelligence for medicine is the is the motion, is the dynamic toward an actual data sharing solution right now the medical data is really spread across many different centers many different hospitals many different pharmaceutical companies etc all this data is really not at the moment shared in a common place and this is a huge problem because if we want to be able to train powerful artificial intelligence algorithms that can help doctors that can help patients that can help predicting the effect of a treatment to give the right drug to the right patient if we want to do that the first the most major obstacle is this data sharing problem ok and we hear conferences very nice people saying well we are committed to data sharing we are going to create this platform where everyone can put this data together etc etc if you say that people think you are someone good so it's good to say that but in reality it's not happening it's not happening right now and I think for many many reasons privacy intellectual property different things like that it's not going to happen soon at a large scale ok so if we want to do something powerful that exploits all this data that is already here we need to find another solution and I think that these ideas around transfer learning can be very very helpful and in fact it's very simple the idea behind because transfer learning is just you train your network on a given data set you use it as an initial condition for the next training so it's just about doing a gradient descent and following the path and just you stop at one point and you start again so the standard approach would be to gather all the data ok, so you go you knock on hospital A door you say can you give me your data I will put it in a secure place you go to hospital B, to hospital C or you can do that you can imagine doing that with general practitioners so it's not a few hundreds hospitals it's a few hundreds of thousands of doctors or you can imagine that this problem can arise with internet of things connected objects that measure our health ok, and we don't want to share our data with some cloud provider or whatever, but we would like to benefit from the artificial intelligence that this system could offer ok, so the scale of this problem is not only a few hundred hospitals it can be millions of users ok, so what can we do overcome this and say, either this is not feasible or we don't want to do that we think it's too risky to put all this data in the same hands so the idea is really simple is a kind of natural extension of the idea of transfer learning is the idea that you train your neural network on the hospital A so you have your 100 or 1000 images with a given task here and hospital B has the same kind of images with the same kind of labors so now that you have trained your network you can make it travel to hospital B so the data stays put in hospital A but only the neural network waits, travels ok, so it's brain traveling ok, you make the algorithm travel and then you put the algorithm in hospital B and you continue the training ok, so this is I would say so you can do that for all your users in your network and you do one round and you can do another round etc until your validation accuracy is good enough what you can do as well is doing something that will exploit all the computing power that can be put in each hospital by doing a parallel algorithm ok, so you can do like a parallel stochastic gradient descent so that suppose you are the coordinator of the process you first you spend you send the same initial network to all the hospital at the same time at the same time, ok each one does a training on his data set and then what they give you back is the gradient, the change of the weight then what you do is an average of this change ok, so in the gradient descent you average the contribution of each of the contributor who has data and then you send back these new weights to everyone to synchronize everyone and you do another batch like that ok, so these kinds of algorithms it's called parallel stochastic gradient descent and there has been recent research the last two years about how to improve that, how to make it work better, faster especially in the context where the network that we are making travel they can be 100 megabytes maybe 1 gigabyte soon ok, so these are objects that you should make travel not too often ok, so if you want to make it travel not too often, then you should try to optimize everything so that it works well it can be also you can use that in a peer-to-peer fashion, if you don't want any third party involved you can make the network travel between users and I think that this idea is very powerful for the future of artificial intelligence in medicine on a wide scale not only hospitals but as I said also for data coming from doctors or for consumers and it's going well beyond medicine as well because it's a paradigm capable to train artificial intelligence system without jeopardizing the privacy of data and I think as citizens here in the room we are all very concerned about that right now at the moment and hopefully there are technological solutions that can be developed that can have a huge potential for medicine but also for for the privacy of every citizen so I guess it's time for me to conclude I think my message was clear today so with transfer learning technologies all these artificial intelligence algorithms can cross-fertilize this idea of algorithm sharing has a potential to circumvent the data sharing problem which is to me the major problem right now for the emergence of a lot of new artificial intelligence applications in medicine and we hope that developing these technologies can have a big impact because we work every day with doctors and what we think first is how can we make it happen today thank you very much questions ? yeah so my question is when you use data to pre-train and then train for something else do you know how much the original knowledge is preserved after you did it yeah so you can destroy some knowledge you've learned but you can keep in the memory as well so you can keep both but it can destroy knowledge it doesn't work all the time of course and it still remains unknown from a mathematical standpoint how different the two data sets should be so that it works well because if the two data sets are too close then it's just a matter of adding some new points and it's not very useful ok ? so you need some kind of difference but not too much ok ? and it's like I would say it's like if you learn languages that are quite different if you learn Turkish and Finnish maybe it will be easier for you to learn Russian and Egyptian or whatever ok ? but if you learn only Spanish, Italian and French in good position I think to learn new languages yeah in the city thing why didn't you fine tune ? because I mean the transfer learning thing makes sense if you fine tune since it's different data yeah so we just can take random features no no no we use the features that were extracted from a network that was trained on the other we didn't fine tune plus the gradient boosting worked well and you may see the gradient boosting as just doing a fine tuning of the last layer if you want ok ? if it's a linear style the last layer is not fine tuning the last layer is learning the classifier is learning the classifier you cannot you cannot really call it transfer learning if you are only fine tuning you are just taking pre-existing features ok ? if you want I mean we call it as you want I mean to me it's still transfer learning my question is about privacy so you have this distributed scheme whereby as you say you don't need to don't need to actually share the data that you use for the training so this is preserving privacy so my question is if you only have access to the training network after and before some training what can you infer on the data that has been used for ? ok that's really an excellent question and this is a question we are currently working on in the team my intuition is that if you only have the weights of the network not the activations occur when you present the input but just the weights we say it's very very difficult to recover any data points maybe you don't want to recover the full data but you might be able to infer like how many patients were affected with this and that disease in a given data set that might be already too much information to yeah of course so that's an excellent question and we will be really happy to collaborate with anyone here to work on the theoretical proof of that question about what can we infer knowing the weights about the data that was used for training maybe there exists some very nice theorem already that I don't know ok I don't know everything but I think this question has not been very studied before but it's a very good question so the diameter of the nodule is used as the target for the training here so we have we split it into several categories like small, medium, large and we have another category which is not a nodule because in the database of Luna you have some by question is the outcome of your deep neural net on the competition was it was a vector where you had a number of diameters ok and that's what you used to predict no we used the layer right before and what means you said that you were tenth right on the competition but the best what was his performance are we detecting 80% ok so it was a log loss metric the data set was relatively balanced anyway it was like 0.4 of log loss which means in our cross validation it was like 0.83 AUC ok so you are 83% 83% AUC and this is useful yeah ok so you so this is meant to be used to do a general screening of the population and they want to reduce the number of false positive and false negative and this is a task that is really not easy for the doctors I mean I have a little bit of online and usually the problem is that for some of them they don't do this so we had examples in the database where these people were classified as having cancer or we'll develop cancer in the next 12 months there is a subtility without nodules but you have other markers like emphysemas we didn't use that you were based on nodules whatever we were only on nodules and we ran out of time but it was our to-do list and why you didn't try the whole network at the beginning we tried to do a lot of things on just the data set and you can try you will see you have two few samples and the information is lost in the middle of a lot of other things that are completely not related to the disease you have more samples than on the but the annotation in the Kaggle data set is global it just tells you if the patient has the concern or not and on the other data set you have a local information ok you can take the last question ok two questions the first one is related to the security one so in this process do you share your algorithm so in which process in the for example if you have isolated data centers and if you want them to provide you the output do you share your algorithm so we share the algorithm between the different users that are taking part in solving the problem but then the question is whether we share the resulting algorithms to the global community in the world it's something that can be done sometimes and other times is not done depending on the different parties at stake and different problems so of course it would be better to share the resulting algorithm to everyone but then if there are a lot of money that is invested on that and that people want to keep for them then you enter you have this problem that is all the time I think if you share the algorithm then as if if I'm a data center as if I have the control of the how to say several of the dynamic of the algorithm because I can provide some specific data such that I want some specific outcome yeah this is some kind of security problem that if I'm a malicious player I can provide some specific data such that I can have some control of the algorithm outcome ok so you mean a malicious player that will disguise as a hospital and take control of the that's ok so ok this could happen maybe in ok that's very that's a very clever question non-collective game yeah we are much on the ground right now and we talk with people on the phone and we say ok did you receive the network commercial system and this could happen yeah ok that's in the same direction of the other question like that is how this kind of system is really secure yeah ok that's very very interesting second question is normally we know that individual optimization or learning may not a sequence of individual learning may not lead to social optimal so how can you make sure that during this dynamic process you can how to say increasingly improve at the mathematical level you need at least some some proof or some yeah so I mean if you take the amnist data set with a hand written recognition problem ok if you split the data set into 10 different parts and you put one part in each hospital ok first I will call you crazy why do you send me but then you can train and you put back the weights and you send it back and you train etc and you will see it will converge almost as if you had all the data in the same place so it's right now it's more I would say it's a bunch of experimental results but also the process really mathematically is really like doing the gradient descent and there are also some you have some potential the fact that when you're optimizing some distributed problem then you are convergent to the potential of the problem yeah so there are connections and there is a lot of theory existing already on parallel stochastic gradient descent but ok I think that if we go into the specifics maybe there are some different things that should be proven to be sure mathematically but still in our first experiments things are quite working in the right direction yeah I think we should stop here and thank you