 Let's look at the most or one of the most intelligent machine the human us and If you look at one of the important factor of development of human is communication Really early on we communicate or happiness or surprise Or even our sadness We do it through our gesture and this is a core aspect or really important aspect of a Child and her mother Communication becomes important from the beginning and over all of her life We develop this communication skills to the point of today being able to have this wonderful event Like the world economic forum where we communicate ideas Emotions and this is done through three main Modalities the first one and this is the rule of three V the first one is the verbal when we communicate are deciding on specific words and They're really important. They're meaning is important But every word can have a really subtle change to it even a word like okay if I say Okay Okay Okay, there I change and the subtlety of human communication is so powerful and Makes our communication so efficient that makes these meetings possible And so when when we want to think about a you intelligent machine We have to go in the verbal in the vocal and you're probably expecting the next V the visual So a lot of what we do is to my gestures to my facial expression Through the proxemics to my posture all of these three V's are essential to human communication So how are we communicating with computers with keyboard and mouse? We are also these days having much and we have newer technology like touchscreen and VR And we're getting closer But it's much further than what is human communication. So when we think about these Intrusion machine human communication being able to understand and communicate with human in a natural way becomes important And this is true for embodied Intelligent machine like a robot a virtual human, but it is also true for non embodied Intelligent machine your cell phone your computers that may not have a phase that may not have a direct embodiment But this is also really important So for all of these case one of the core component is what's called? multimodal AI multimodal AI is this capability of computers intelligent machine to understand the subtlety of human communication And what's exciting now is that we and I would say we the community scientific community in general the community Have found really interesting new technology to address this challenge One of the big one is we were able to go from a really shallow interpretation of human communication To a much deeper understanding going from understanding only the facial landmark Motion of eyebrows or simply the motion of of a mouth to a level where we also understand the Appearance of a face and the subtlety in the appearance to the point where we can in fact study all 48 muscle of the face and from this being able to infer the Perceive expression or emotion and I say perceive in this case because what really someone feel is also still a challenge But we're getting closer to a deeper interpretation a second aspect is that Computer but with all this power we have in these computer We can go from a micro to a multi-level Interpretation of human communication where we understand at the first level Something simple as a gaze That's the first level of looking at the dynamic of human communication But as I said the communication is a lot more than that We should look at the multimodal how a person is behaving and how all these What the 3v the verbal the visual the vocal? But I'm not just talking to myself I'm talking to you and so a lot of it is also when computers now these days are looking at Communication as an interpersonal dynamic to people talking together a patient Patient with their doctors So that's what one of the key aspect and this is going now these days to the next level of group and social Dynamic and looking at different cultural norms So this is a really important aspect of this new technology is that we're looking at at multiple level and the last one The last piece the ingredient was that we are able to have a lot more information Freely available and so we can go from small data to big data Where a lot of example of people communicating and not just Laboratory but in really different scenarios different culture and different individual the last piece is that the Algorithms are getting a lot smarter and we can handle a lot more of this data So multimodal AI is one of the key component I believe of intelligent machine and that's also what allows us to think of what do we want these? Intelligent machine to be or to do right now. There are a lot of command like I Command my Siri or command my cell phone to do something, but I believe that computers are a lot more about Colleagues and co-workers so I believe and that's in fact We already see some example of that that these technology will help for example a doctor To be able to help during diagnosis or assessment of mental health disorder and be able to look at the subtlety of Communication and be able to give this feedback to the doctor. There's a big change in how Education is happening in fact people are learning remotely But we know that remote interaction is really challenging when you compare to face to face interaction and Technology can help us for that and this is true for business interaction Also, we are lucky today to be all together But what if we have to interact remotely? So let me give you an example of a first step toward these intelligent machine and let me introduce Ellie In fact, Ellie is part of the SimSense a project and Ellie is there to help doctors during their assessments of Mental health. She is not a doctor. She is not a virtual Doctor, she is an assistant. She's an interviewer. She's gathering information and when we're looking at depression or distress in general This is a challenging thing for any clinician to successfully diagnose And so what if we could do what exists in other part of medicine? Example a doctor has the intuition about a disease They will do a blood sample a blood test and then look at the details for mental health We have now the possibility of doing an interaction sample How did that person change over time? So maybe we can reduce the treatment and change the medication So let me introduce Ellie with an interaction and showing how she can interact in real time and perceive the verbal The visual and the vocal. I'm Ellie. Thanks for coming in today I was created to talk to people in a safe and secure environment I'm not a therapist, but I'm here to learn about people and would love to learn about you I'll ask a few questions to get us started And please feel free to tell me anything your answers are totally confidential. Are you okay with this? Yes So How are you doing today? I'm doing well That's good Where are you from originally? I'm from Los Angeles Oh, I'm from Ellie myself When was the last time you felt really happy? um But I try to stay happy. Um, I'd rather be happy Uh, my kids keep me going What advice would you have given yourself 10 or 20 years ago? um to uh to not believe uh To to not be so gullible to not be so gullible So this is a short example. In fact, we had more than 150 people interacting with some sensei And this is just over a period of a few weeks And what was interesting is the interaction was supposed to be only 15 minutes People enjoyed talking with her. She talked for 30 or 40 minutes In fact, if you look at how much sadness or real behavior they show There's more sadness when they're interacting with a fully entomatic Intention machine than when they think that there's a human behind pressing the button And what is really interesting is because of all this data We can start looking at differences for example differences between man and woman for depression or for anxiety So these tools are one example and as you can imagine, there's a lot more of these example going on for both uh healthcare For education and business where intelligent machine can help as a co-worker And as a colleague. Thank you very much for your attention