 So first of all, of course, I'm going to ask the audience before we start the panel if they have some general questions to ask to the speakers. Okay, no question. Oh, yeah, one question. Go ahead. How can all the techniques you use be used efficiently in communication? So that was a more specific question, not a general question. But how do you think the techniques that have been shown today could be used efficiently? Maybe this is the best question for Hongli in communication. I explained one example. I explained one example in SDN, right? Using machine learning to routing decisions. Yeah, that's just one example and there are many other examples. So in general, I believe anything related to decision in telecommunication, if it's very hard in traditional ways, then maybe data-driven machine learning can play a very important role. And it's getting started in all the areas in telecommunication. People are starting to collect data, et cetera, and try to define problems. This is true for our own company Huawei. In our lab we have several projects and quite a number of people working on this area. And I believe it's going to be more and more important. What about cellular? What about the 5G? Because the network and computer science oriented approach is quite clear from your presentation. But for 5G, there are also issues like resource allocation, resource management, and other things like, for example, antenna parameter setting. So far, such kind of problem has been done by human knowledge. If you have an antenna, you want to decide the angle of the face, the pilot's power, et cetera. So far it's based on human knowledge or based on simulation. But we think that, for example, in such a case, machine learning can help to do a better decision. And in fact, we have done some experiments for this and we observed some improvements. 15% improvements in overall management. So it depends on how you define the efficiency, overall efficiency, et cetera. But it's possible to define and also make improvements, for example. Is this just an optimization approach? Not optimization, but using reinforcement learning, for example. We can do a better control to improve, for example, throughput or improve some KPIs. Yeah, better than existing method. Yeah, so there are other cases like this. Another question from Maxime. So the progress of learning nowadays seems to be very fast and in a wide audience, there are many people who are afraid that maybe machines can become too smart. So is that, you know, for scientists like you, is this a realistic concern? And more practically, do you think your work has the potential to really disrupt some fields of the world as we know it? No. And of course, I think technology will disrupt the world. But people have that all the time. But in ancient machines, personally, I don't see any queue that they are closer today than they were 20 years ago. But maybe they are, but I just don't see that. Maybe a good question I was asking is that maybe people think that we are destroying our own future in terms of jobs. Which comes very often in many discussions, and I think in your interactions also with people. What we're doing is destroying our own job. Is it true? I can speak from the interactions we have with doctors. And one recurring question they ask us is, well, it's really nice your technology, but are you trying to replace us? No, and it's really the kind of question they ask today. So what you say is really important for us because we need to explain them that the algorithm is important in the interaction. With the human and not as a replacement of the doctor. It means that we need to figure out a way to design this interaction. And doctors are not specifically educated priori to interact with algorithmic results. So there is much room for development in this direction, I think. I was, for some random reason, I was at some big biomedical meeting in Lyon a couple of weeks ago. They invited me. I don't know anything about the topic, but the doctors didn't seem too worried. And I think they also have to take into account for things such as medical decisions. You have things of elusive ability. Reliability, explicability and all that that are not well mastered. Explanation, you know, responsibility and all that. So I think there's a long way to go before you can take a human out of the loop and I think you actually can't. I have a more general question is that AI is there since the 50s. It's not a new topic. Whenever you talk of why is the AI hype now, they tell you it's because of the computing power. Is it the only reason why now AI is increasing in terms of momentum? You're seeing it now everywhere, but it's been there all the time. I mean, there's always people who've been working on neural networks, stuff like this. No breakthrough and now, for some reason, everybody's on it. What's the reason? I speak too much. It has always been there, but they didn't work at all for a long time. Why? I think there have been several factors. Of course, there's the availability of data and the availability of computing power, but you also have the technical stuff like neuralness now. The thing that works now is essentially CNN. CNN has been here since the mid-90s, maybe early 90s. They worked quite well on specific domains. They don't work well on natural images, but they worked quite well on their own domains. They started working a lot better when there was enough computing power and data to make them work better. I think it's a mixture of scientific advances and a mixture of data and computers. But the scientific progress is there, and I think it's the key behind it. Yes, it's the combination of the three. Also, one more to add is some good examples like AlphaGo. It's good and bad. It gives people some confidence and also wrong impression. People can easily imagine that a machine can do anything. It can beat a human champion. But in fact, it's not always the case, but also I think it's important. If I look at the projection, things can only be better because computing power is increasing every time. Storage, I would say, ability is also getting cheaper and cheaper, so we're going to have more and more data. So there's always going to be a good combination for AI to explode in the next year. Is there a limit to it? Yes, you want to... I think that maybe there is a hype right now about artificial intelligence due to AlphaGo, for instance, just like you said. And that when people will realize that artificial intelligence is not able to do what is promised to everyone, then they may be downfall like we've seen in the past. And maybe it will happen in the coming years. We don't know. And maybe then after, it will still rise up for the reasons you said. But not maybe at the same pace as it does right now. I don't know. I agree, but I think you still have to take the opportunity right now because I think there's so much excitement. And there is scientific progress. But an example of expectations is the automated cars, right? So automated cars are not deployed. The only ones that I know that are deployed are the Uber cars in Pittsburgh. So in Pittsburgh, when you call Uber, you know, one time, sort of, I don't know, many times, it's an automated car that comes. And I was in Pittsburgh last week and you see them driving around. But there's a driver sitting at the wheel. I think he doesn't drive most of the time. He's sitting at the wheel. And it's not clear to me when real automated cars, if ever, will come on intelligent kind of intelligent cruise control type things. Yeah, but so who knows. But I'm sure we will have this thing. But the scientific knowledge stays. So you never go back down like that. So I think we have learned a bunch of things. So another question I have since we are in a mathematical, I would say center here, is basically the relation between mathematics and AI. So there's two issues I see there. The first thing is our industry is using basically a lot of AI cooking recipes without really understanding it. How do you foresee this problem of using things that we don't really understand with the risk that maybe sometimes it will not work for the customer, basically because we're not fully sure about the theory behind. And second is when do you think a neat mathematical theory, let's take the example of deep learning will emerge from your point of view in trying to explain. And the third one is basically, and I know that Gilles is maybe going to be going to ask him, is when do you think we're going to find the FFT of AI? Basically, where we could implement something of a mathematical, I would say theory of AI with its algorithmic fast free transform. So let's go for the first question is basically, do you think our industry is taking a huge risk? ICT, I think a bit less maybe because it's not nuclear bomb for the moment. But basically do you think we're going too fast in industry by incorporating algorithms for which we don't have a full understanding. And sometimes the network may fail because just the 5% of the cases that we didn't understand and predict. Yeah, currently training neural network is a kind of art, right? You need to accumulate some expertise or knowledge in order to train a good model. And there is no theory as I explained in my talk. And we hope there will be some new discoveries in theory to really guide the construction of neural networks. But it's difficult to predict how long it will take to have a better theory to explain everything into learning. But even without theory to guide, still people will try to different models and different architectures with more data, etc. So that means it's less efficient if there is no theory. But maybe still it can help people to build neural networks to apply the technique to different problems. Let me say differently, I mean would you buy a product where somebody tells you we have an algorithmic size, it works but we don't know why. But this is the case in many scenarios, right? But people don't care. If it works and we know the fundamental, it will not break something important, right? And it's still within one territory. If that's the case, I think it's okay. But the question is how can we assure that it doesn't break? Yeah, there is one issue. Some people like Professor Dietrich is proposing the idea of robust AI, right? Even for telecommunication, it's the case. You want to do something really bad, right? That's something still not clear for self-driving cars, the case. And even the accuracy is very high still but there is a risk. And then there's another issue because we always, when we train a model, it's always on average we can improve or we can achieve very high performance, right? And then there is maybe an issue. For example, if it's in self-driving cars, it's an issue about how should we put our trust on machines, right? It's not only an issue in computer science, it's an issue in social science or other fields, right? And that's also one important question. I don't have a very clear answer about that but it's beyond technology in some sense, right? But within the boundary of technology, still, with current deep learning or machine learning, it's average maximization and there is no theoretical guarantee from the viewpoint of math. Yeah, and clearly this question about having an algorithm that works well but you cannot explain why is critical in medicine. That's what you said is that if you have an algorithm that predicts the diagnosis or whatever but you can't explain to the doctor why is this decision the right one, then no one will use it. This is as simple as that. So we need to develop explainable and interpretable machine learning systems. And this is related to what I said before is the idea that we need to create this interaction between artificial intelligence and human intelligence and to create this interaction we need this interpretability. And for this interpretability to emerge I think my intuition is that it will require mathematics. It will require a deeper understanding of how it works. So at least I would say this is one of the big challenge ahead for the theoretical development. Well, I think you need really reliable things with empirically provable at least reliability so you can some confidence in whether it works or not. Otherwise people just won't use things. I think people have been, I mean they have been accident with self driving cars. So part of it was that people maybe were too confident in the decisions of the car. So I think reliability is a research issue. I think to me any research issue is a mathematical component in any case because otherwise you can't model it. I think it's a real research issue and after that the explainability is also a research issue. And so I think there are interesting topics on that. I think as you said there are people at CMU who work on this. There are people who work on this from an empirical point of view as well. You know having some sort of monitoring system that tries to look at the data and how good it is for the underlying algorithms. I think those are very important. I think there must be still plenty of areas where you don't need. When I step into a plane I have no clue how it works. I have no clue how the engine of my car works either and I still use it. So maybe for doctors it's a bit different and certainly there's a responsibility issue. You may not want an explanation always but if there's an incident you certainly want to be able to point at the responsible party. Whether it is the person who sold you the software or it's a part of the software. But I'm not sure you need the customer to always understand how it works. But for doctors it's probably true. And another question if we have to invest in mathematics to make AI work and have a full understanding. In which particular field of mathematics do you think we need more investment? Statistics, optimization, algebra which would be the part where you think we would need or require more and more. I would say a bit of breakthrough there. It's a tough question. Well if you look at the major trend certainly probability theory and statistics would play an optimization would play a very important role. But I would add there are other areas which would be important as well like reasoning in AI. So currently AI is mainly based on machine learning. But one view is that to further expand the capability of AI ideally we want to have reasoning or inference capability added into machine learning. And then clearly humans when we do reasoning we don't follow the logic in maths. And maybe we use a kind of different logic. Some people propose like natural logic etc. A lot of works I did before but actually no really helped improve AI. And maybe one possibility is a kind of new logic different from traditional logic in math. But very close to human natural logic maybe and really can help enhance AI. That's maybe one possibility. Another possibility is knowledge and this is related to symbol manipulation etc. I don't know whether it's strongly related to math but it's also likely math can contribute. And there are several other areas I think moving forward in math if there is any advancement may also help AI a lot. I would say that to my opinion it's really important to do a lot more of mathematical biology and especially mathematical neuroscience. And use mathematics as a language to describe on the same ground artificial intelligence system and biological neural systems. And using this common language I guess it will favor the emergence of new ideas in the field of machine learning through this understanding of how the brain works using mathematics. And it will require probably to develop fields such as dynamical systems on very large graphs, statistical physics of disordered systems that are purchased by external inputs. So it's non equilibrium thermodynamics and things like that. I think it can be very powerful to describe these disordered systems and have a mathematical view of how the brain works and then have this common language. I have no clue but I'll say something anyway. I think analysis is important in the mathematical sense is important. When you look at machine learning, I'm not a machine learning person so I'm very naive. But I never really understood kernel methods and all that until I looked into it and this thing about using functional spaces to describe data. In machine learning you have data people often talk about feature vectors but they are not vectors because there are not things. There are arrays of numbers but there are not things you can add or subtract or multiply by. And the way people went around it in machine learning communities by building functional space over those values you can reason about. And I think doing something similar in the deep learning realm for example is important. The few kind of theoretical analysis I've seen of CNNs. Assume that you have linear operators and some linearities over some input vectors and those are not input vectors. Images are particular structures and convolutions are particular operators and that's what makes CNNs work. So I think there's a lot to do there. I think there are a few people like CNNs. I think there are things that interest me that are intriguing also from a mathematical point of view. If you look at these image generation stuff, deep dream, whatever. And they seem to have pieces of images that are remembered from the training set. You see a face of a dog somewhere in the image or it has been the same for other images and these things. So why? And I think that's a mathematical problem. Do these things really remember somehow represent some of their input data? I think that's interesting. And of course there's all the business. I like CNNs but I'm not a great believer so I don't think it will solve the world. I don't think this is the solution that one should explain because I think people take as a take on the thing that CNNs work. I don't think that's the case. But I think still it's an interesting problem to try to understand as much as they work or not why they kind of work. There has been a little bit of work there but I think it has been not very so sexual. I think there are very few attempts that are convincing and why these things can learn the functions that they learn. And I think there's plenty of effort to do them. Especially if you are part of the converted, if you think that this is the solution to everything for the rest of the mankind. But I think it's generally interesting. Okay, are there questions? Yeah, actually I've seen a couple of papers lately that connect neural networks with renormalization in physics. Real space renormalization is where basically you have... So physics have this problem where they have situations where they have many, many particles, millions of particles. They won't somehow deal with the situation. So what they do is they break it up into groups. So you sort of replace the group with one sort of average, so to speak. And then you take groups of groups and groups of groups and so on. And there's an analog between that and when you have a neural network and layers. As you go from layer to layer you have features and features and features and features and features. I don't know if you have any say about that or are you familiar with this connection with renormalization in physics? I've read the paper. But yeah, there is much work to be done ahead. It's really more like an analogy at this stage, I guess. There's also been work where they tried to use, you know, spin glass type stuff and you know, polynomial functions with coefficients and random coefficients on the sphere and all that. It's interesting to do that but it's not so convincing because as far as I understood the paper in question, they removed the nonlinearities. If you remove the nonlinearities the neural network doesn't work so well. But then, otherwise, what they are trying to do is interesting. They are trying to get at the structure of the critical point of the... Yeah, but I think there are real technical problems in that paper. But trying to do that is interesting. Yeah, there are connections with spin glasses. Marc Mésard has been doing some connections with this physics. And they try to look at how energy flows per layer. And it's a question of our energy and what they call free energy goes in from layer to layer. So I think it's a good path but I don't know what's going to be the result. That's like the paper with Le Cun and Banarou. Because the analogy is the layer, the structure, the critical point for spin glass models. But it doesn't tell you at all that when you minimize you will go down to the local minimum. So you have to take it to the ground assault. But I think there are clearly interesting things to do. And even if they are not the answer to everything anyway, it's an interesting problem. Yeah, I have another question on this black box approach. I think one of the fancy things about our industry is of course that whenever we have difficult problems to model, the black box is nice because we can't spend time. And of course whenever it's too complicated we just use it as a black box. I have an issue with this because it's also from a scientific point of view. I think maybe you saw some talks of Condès is if you look at how science has been made until now is that usually you bring some experts in industry to develop a theory and use data to validate that theory. And now we're going to a realm where we use data to create theories basically. How do you see just change of paradigm in the way engineers and researchers are facing with science in the sense that we use data to create Maxwell equations instead of finding Maxwell equations and then using data to validate it in terms of how we are training our students and how also we're doing research at the moment. I think it's an excellent question. And maybe data is something that is putting us away from knowledge and putting us away from wisdom maybe. Yeah. That was short, yeah? Yeah, so I think I answered your similar question in my talk. So let me just follow his Geo's point and maybe it's just one step. If the problem is too complicated, we don't know how to model this in a mathematical model and maybe data-driven approach can help a lot. And so this also going back to the previous question, deep learning is being used in different fields in optimization, et cetera. And to me it's not a surprise because it's just a technique for function approximation with data. But we use different functions in different fields for different purposes. And if there is data and the problem is too difficult to model in mass and then maybe machine learning, deep learning can help a lot. This is true for telecommunication problems as well, right? But longer term, once for something, for some problems maybe if we have enough knowledge, it's also likely. We don't know, but we can build a new theory based on the experiences we have gained in machine learning, deep learning, right? It's likely and there is a kind of chasing game. So we're first to build something that works and then on top of that we can build a new theory. That's also likely to happen in the future, at least for some areas, some problems. So we don't know, but personally I'm a strong believer in math. So I think it's necessary to understand the details when necessary. And even it takes time or even it's very challenging but it's at least likely to happen for some areas. Maybe Jean, but he's tired. It's the end of the day. We probably all have the same kind of training. So of course we all want to model things with mathematics. I hate the idea of black box and again these drawings with those things and those arrows. I hate those, but these things are useful and I don't think I have anything intelligent to say. But I don't see those are new ways of doing science at all. I see these are useful tools for certain things and if scientists can use them, good. But to me science is about making theories about physical phenomena. And we are in a bit different realm as well for engineering and science are different. So in science you have natural phenomena, you observe, you want to explain. In engineering we build complex systems and we want to analyze their behavior and understand them. And the two I think are probably a bit different. Okay maybe one question because I think it's going to be time, it's already 45. So do we have other questions? Yeah, maybe Miriam and we'll finish by that. Why ask the question about why do we make machine learning into communication? It's the following. In communication I give you a message, you transmit it, you don't know where it is. In machine learning you understand the underlying structure of information. How can this help you? For example you do a test classification, you know the context, you know the keywords. How does this help you to compress the text inside it? This sort of interaction, how we change the paradigm of communication. Not just pipelines and bits and coding schemes and that's it. So I don't know if you have any insight about this one. I didn't get your question very well. So yeah, so clearly for instance you talk about compression and using unsupervised learning. In autoencoders for instance you can build up new compression systems that can then be used in communication but it's more sequential engineering than global. Do autoencoders work better than classical methods? I don't know, it depends. Maybe she was asking the question what do you think about the recent Elon Musk project about connecting the brain to AI. It calls this neural link. That was your question. What? Take the fifth. But I know I think in itself it's a good question to look at the limits of how AI can help improve networks and communications. And I think the whole idea of looking at how the message can be in plus the user behind and the whole point to point. But I think the biggest gain are from a network point of view rather than a point to point. So basically it's going from separate layers designed to more cross layers designed because of the correlation of information, because of the utility of information. Basically you have to know when in communication you have a separation theorem where basically you have a source, you compress it to hell and then you encode it to hell, things like that. But whenever you're not fully compressed, meaning because you don't have long bits, then doing the whole modeling, a lot of people work on something called cross layer. Meaning basically try to combine, but then it gets complicated to understand how layer one or layer two, layer three things. Why don't consider all this as a black box? Because we like black boxes now. And basically start learning the whole thing and basically to be able to build a system which takes into account what's happening. Meaning you have finite packets, you have everything finite. And maybe instead of trying to model each part like we're doing and trying to connect it into cross layer design, which is a bit complicated. You could do it as a black box stuff. But if you have prior knowledge, why don't you use it? Even if you aren't fine-tuned things, I mean me, naively I would build in whatever knowledge I have and then maybe I can tweak it by using one of these methods. But I've never understood why, when there are people, for example, in robotics, try to learn robot kinematics. I've never understood why you want to learn robot kinematics. You know the kinematics, you build the machine. Of course you know it imperfectly. And you may want to adjust it, but I think when you have good knowledge of things, unless the systems are so complicated that the knowledge gets confused when you pile it on top of each other. I think to me, I would use it. The black box ID, to me, is very strange. The question I'm asking here, that's a good question. If you have prior, it would just reduce, I would say, the learning space. Which is important for these things. But do you have enough data in any case? No, because you have a non-convex optimization thing that may work because the gods are happy that they are. It may not because it doesn't work. And my understanding of CNN is that, you know, you change the initial parameters, you get very different results. And I'm not especially, so I may be wrong. The fact that you can get more data doesn't count. If you get an infinity of data, I guess there are some consistency things that will tell you that it may work. But in general, you don't get an infinity of data. So to me, it's not clear at all. Okay, I think we'll stop here, except if there are other questions. Let's applaud the speakers.