 All right. So this day, and in large part this week, is really about bringing together researchers from industry and researchers from academia. And it's easy to see all the ways that we have common and shared interests and values. It's very easy for us to get excited about each other's research, and I think that's one of the wonderful things about this network and about the MIT IBM lab. But at the same time, I think there's an elephant in the room, which is that we're very different. We have industry and academia have different sets of interests and values. And while I think all of us want to have impact ultimately, I think it's hard sometimes for us to understand each other. So we thought that a really helpful way, perhaps, to end the day after all of these fantastic technical discussions would be to have a conversation about, what does it actually take? Because I think oftentimes we don't dialogue across that full spectrum. And what we have here today, I think, is a panel which is uniquely equipped to ask those questions and perhaps even to answer them. So we have Catherine Gorini, who's the Vice President of Industries Research at IBM. So IBM Research is a 5,000 person organization, perhaps 1,000 working on AI in various forms. A significant fraction of that is focused specifically on industries and industry applications. So she brings broad expertise across health care, financial services, and a whole bunch of different industries. We have Richard Puri, who is the CTO and Chief Architect Watson. Watson is the business unit of IBM, if you aren't familiar, that basically sells AI services, particularly in the cloud to enterprise customers. And actually, Richard started his career in IBM research and was recently drafted into the product division. So he's uniquely qualified to discuss what that's like, making that transition. He's also an IBM Fellow. He was named an IBM Fellow in 2006, which is basically the highest honor that IBM bestows upon scientists and engineers within the company. And he's worked across many different fields in the stack in research. And now he's been living in the product world. And finally, we have Antonio Toralba, who you've already met, who's my counterpart, the co-director on the MIT side of the MIT IBM Watson AI Lab, who's a professor in sea sail. And his lab has spawned several startups, so he's also played across the entire spectrum. So with that, I just want to start and just have everyone just spend a moment or two just to reflect on what are the biggest challenges and opportunities in taking a research idea and bringing it to industrial impact. So Catherine, do you want to start? OK. Hi, everybody. I'm Catherine. I think there's a tremendous opportunity ahead of us. In the marriage of industry research and impacts, we see innovation everywhere. We see opportunities because almost every single industry is transforming or on the verge of disruption. We see that in banking and insurance. We see that in health care and education and manufacturing and agriculture. And we could go on. There are real specific applications of AI and blockchain and automation and other technologies in every one of those. And I think the excitement ahead of us, why we are all so passionate around this field, is we can actually see and touch and begin to solve some problems that we couldn't in the past. That's great. I think one of the challenges is to have that impact at scale. And how do you go from a demonstration, a proof of technology, a proof of concept to something that is really transformative, something that will be adopted in mass across the industry? And that's not easy. So the reality is the real world's messy. And going from something that works in the lab or works on your laptop to something that we're really going to deploy to the entire banking industry or across the entire health care space. This is something that requires deep partnership with key players in the industry. It requires experimentation in market and validation. And that's an iterative process. And it requires deep partnerships that go beyond our academic or an industrial research lab. But we need the innovation with the ideas coming from this community. So to me, it's a partnership with a deep understanding of the industry problems, a deep understanding of the technology and how we marry that to have greater impact. Great. Richer, tell us what it looks like from the world of product. I think when I was in research, as Catherine put it very rightly, the real world is very messy. And you want to have your lot of us who come from PhDs, graduate schools. You like your hands clean. But there's a reason why I in particular and a lot of us chose to be the world we are in. We wanted to make an impact. And I'll give you quick examples on what it might mean. So first of all, when you look at messy problems, they have a major component of what I call shoveling, actually. It involves a lot of shoveling, which is there is a dirt pile. You try to shovel that dirt pile into here, actually. Which is you're coming from here. That's where you need to go. That's what will solve a big business problem. Interestingly, in that dirt pile are these nuggets of gold that we love to actually figure out and find and really solve as well. And I'll give you a quick example, actually. So let's take example of document understanding, actually. Kind of very unsexy problem. Why would you? Not very sure. Anybody wants to solve business document understanding. And you look at the world of AI where we are today. We are in this big nirvana of visual recognition, auto drive, sort of automatically driving cars, and so on. And you look at, then you start digging into this and say, OK, most of the interesting words documents are, as we look at across businesses, then they don't come in neat schemas, actually. They are scanned a lot of times. So last time when somebody did a mortgage, they have you print a lot of hundreds of papers and they have you sign it and then they scan it. Interestingly, when you scan it, it becomes pixelized. And when it becomes pixelized, now interestingly, in real world documents, they are not just text. There are tables. Table spans multiple pages. It's not just a neatly defined table with one row and multiple columns or so on. They span pages. There are graphs in there. There might be equations in there as well. And as most of us, or all of us who have gone through our elementary education know that second graders answer questions on graphs. And the real world academic problem is text understanding. That's an academic problem. The real world problem is text understanding in the presence of, which is, by the way, scanned documents which are pixelized. So I would rather be doing cats and dogs any day than understanding these messy documents. I can tell you that for sure, because a much easier problem. And taking that problem and to be able to, now I'm saying, to be able to answer questions on those. Most of the kids in elementary school can reason on graphs, can reason on bar graphs, multiple colored graphs, and relate it to table, relate it to text, and answer questions. And that's a real world problem versus a academic problem. And to me, that is, it is our job in industry to be able to formulate problems in a very neat way, these gold nuggets, to be able to hand it over to research community, which are very hard problems. And to be able to solve that and then integrate it back. But that, to me, solves that whole pile of dust from one place to another, including that gold. Great, Antonio. OK, so I have a very different background. I've been living always within the nurturing environment of the school and the university. I never went into industry. So I don't know what it is to build something that actually works. So most of the time, the research that, so one of the difficulties of connecting research with a product is that when you work at a university, research, and in particular, fundamental research, is something that you are not looking necessarily about solving a problem that you can deploy into a workable product that will solve a particular problem. You want to solve a problem. You want to have an impact in the world. But not now, maybe in 50 years. And not you, someone else. So that's mostly what fundamental research is about most of the time. That doesn't mean that sometimes we don't find things that have an application that will work in a year or two years. But it's not really necessarily what we have in mind. In fact, if you are working with students and you tell them to work on a problem that it seems like it can be solved very, very well within six months, it's quite likely that they will not want to work on that. Because it seems almost like engineering more than really actual research, fundamental research. So there is a big difference in how you are trying to solve problems, but the horizon in which you want to have an impact in the world. And fundamental research, many times, is about posing questions and showing that there is a sketch of a solution, that it can be done. Maybe not with the approach that you are proposing, but with something else that someone else will figure out later on. And eventually, someone like you will think of the actual solution that makes it work and be able to sell a product that people buy. They don't suit you because it doesn't work. So I think that also industry has changed a lot. So there is a lot of now interesting research labs within industry. So this fundamental research is not anymore something that only happens in academia. Industry also has fundamental research labs within industry, and IBM is an example. And just following up on that, so you've identified some ways in which fundamental research, the goals are very different than the goals of some kinds of industrial research. And yet industry is reshaping the landscape of academia today. So industry is hiring, for instance, not just the students, but also the faculty. And by the way, Antonio, I don't know how you are feeling about your job right now, but it's just part of this world. So how do we balance that? Because clearly, industry has identified that these problems are important and that they're valuable. But at the same time, academia has a different horizon. That horizon, I think we all agree, is valuable. How should we balance those two competing forces? Yes, that's a very complicated question to get a definitive answer. It's true that what I was saying that research is always looking at long horizons, but it happens from time to time that one of those long horizons arrives. And this is what has happened with machine learning. For instance, what we are seeing now is really a revolution that started 20 years ago. So these are the 20 years we have to wait. Well, people, you know, we're not making a lot of money and now they are getting paid like NBA basketball players now. So this is the year that it takes. And now what is happening is that they are getting these really nice offers and they actually get to continue doing fundamental research because part of this fundamental research has a direct application with the industry. So there is all this talent that has been taken away from universities and probably this is going to require, particularly within AI, other fields are not having this, but at some point quantum computers might work and all the physicists here in the room are going to make a ton of money being hired and all the AI people will fire. So, who knows? So I think that we need to rethink at some point also how universities wants to collaborate with industry. I think it's not something that industry needs necessarily to change, but also universities, for instance, having been able to have a professorships that can work in industry without leaving their teaching responsibilities and having like a more balanced connection between industry and universities. So that's something that will have to happen and there are a few universities starting to look in that direction, but so probably changes like that will be necessary. Good all. Good. You know, I think the challenge that you're raising around the time horizon, it's one that we even struggle with in the IBM research, right, because we have some fundamental science research that we're doing with that much longer time horizon and we have areas in which we are trying to build that pipeline of innovation for our product teams and the next new business for IBM and it's a challenge straight to span that. How do you prioritize? I think one of the important things that we recognize and I certainly recognize in the scope of responsibility I have now is that the innovation doesn't happen only at the long horizon science level. I mean, there's just as much innovation, invention, ah-has that are required in building the integrated solutions and some of those are long horizon challenges as well, right? If you're gonna go build new fraud detection capabilities or ability to do disease progression modeling and health or personalized education service, whatever, you know, pick your favorite AI application in industry, it's not enough to have a great natural language processing solution. It's not enough just to have a whatever, a video recognition. I mean, those are all good, those are good fundamental capabilities, but how do you apply the innovation to the particular hard problem in the industry is hard and it requires some collaborative innovation and I think both the academic and the industrial research communities can align around those industry hard problems. And maybe other aspect of adding something to what Catherine said is many times Antonio, but you find insights when observing things as well. I don't think any place will know better than this actually one. You really are observing and you get that ah-ha moment when I'm stuck, you are in the middle of it actually. You are living it every day, you never sleep actually. Even when you're sleeping, you are thinking the same damn problem actually. And I do believe when you are in the middle of it, only, I'm not saying only when, obviously, there are these classical pure problems that you can pursue for 50 years and God help those who solve those actually for sure, because they're gonna raise the humanity level, but there are a lot of other problems which are moving the ball forward and that you really solve when you live in that problem. We see that, a lot of us who are at that cross section see that where innovation has been done because you were living that problem. If you were in this so-called ivory tower, it is hard because you wouldn't be living the problem. Yeah, and one of the things I've noticed in the short time we've been running the MIT IBM Lab, we have a pillar around application to industries and this is actually one of the biggest points of confusion with the MIT IBM Lab when we interact with the MIT community because the first round of proposals we got, there were quite a few that were really about applying AI, existing AI techniques to industry problems and we said, hmm, that's actually what we want. We want you to find fundamental problems that you can only see through the lens of industry problems and I think that perspective can be really powerful. And Antonio said he hadn't solved any, made anything that really worked yet and I've only been at IBM for six months and I haven't made anything that works yet either, but I did wanna ask as we have Richier here, what do you wish we knew? So I mean, you're in some sense the catcher of these insights. So if we're building, even if we're looking through an industry lens and we're identifying problems that are interesting and we wouldn't see otherwise, we ultimately have to hand that off. This isn't a charity for IBM, we don't do this just because we're nice people, we are nice people, but we do this because we ultimately see this driving value for our customers and for that to actually be realized, we have to hand it off to you, you have to catch it. And some days I feel like we're just throwing crap at you. So are there things that you wish we were doing better in terms of that handoff? Is there something that academics could do that would make that transition more powerful or more easy because I think at the end of the day, academics also want impact, right? They aren't happy necessarily being just in an ivory tower. Yes sir, I would say it's not that I can just say in academia or in research, this is what you should be doing better because to me it is a joint responsibility. We should be, first of all, I do agree that you don't want to throw the crap we have, by the way, over as well regarding Solvis because a lot of people will not get excited by it and Antonio is right, that is engineering actually, it is a fact. We need to separate these problems out and also give it with the constraints it needs to be solved in with as well, actually, with the right kind of regressions and as all of us know, actually, five years back, one of the major things that happened in this area was, okay, imagine that data sets, data sets are a big part of it, that capture those boundary conditions under which that problem will be successful. So to me it is not just about, you know, either one side throwing the crap over or we throwing our crap over, actually, on the other side, but it's really defining those boundary conditions under which if solved, and it shouldn't be a small problem, actually, it should be well-defined, it should be a hard problem which will significantly move the ball forward, but it should come with the constraints, probably in the form of data sets, and not just one data set, actually, that captures the reality. And as all of us know in machine learning, the training data is a representation of reality, and if that representation is wrong somehow, you are really shooting in the wrong direction. So capturing that representation is significantly our responsibility because we know the reality, and on the other side, they should make the assumption that if they solve that, that will have an impact. So to me it is really joint as opposed to sort of one side or the other. Great. So we're getting the signal from the ticking bomb here that we're getting to the point where we're gonna take questions from the audience, so we're gonna take 15 minutes. What problem could possibly be more exciting than real document understanding? Well, really much understanding. Thank you, Michael. From Michael Weprin, does somebody wanna take that? I already answered it. What is it? Real image understanding. Okay, very good. Yeah, I'm sure we can think of many exciting things. Does IBM, AHN provide funding for early stage startups? If yes, how should one apply? Is anyone on this panel qualified to answer that question? Well, one thing I will say, so not today, but we're interested and I would say come and talk to us. I mean, not just AI-HN, but I can't speak for AI-HN, but for the MIT IVM lab. I mean, we're sitting here in the middle of Kendall Square, tremendously exciting environment for startups. The two director, you've had a couple startups come out of your lab. I'm a recovering academic. I came from Harvard. We had three startups before I came here. I think startups are a really important part of that innovation ecosystem. And I think it's not incompatible with big companies deriving value. In many ways, startups are a way that you can push risk into other entities, people who can tolerate the risk because they're young and they're energetic. And then companies can learn then from those startups. So we're exploring ways to work with the VC community. We're actually regularly having meetings with the VC community. And I'm sure there are other efforts like that within AI-HN. Anyone else? Well said, I hear here. We know how much innovation and excitement is happening within the startup community here and globally. And we need to continue to partner and tap into that, influence it, learn from. It's part of the innovation ecosystem, just as open source as well is happening. And many of you are participating. So thinking about nontraditional ways in which we can maybe more traditional today than it was in the past, ways in which innovation is happening, we as a corporate partner want to make sure that we're tapping into that. And as joint partners between IBM and MIT, I think we can do that together. Great. In the last year, which industry problems have you focused on? So I think Catherine has the broadest view. And maybe I'll just add one thing to that. I've been delighted to learn about all the interesting unexpected industry problems that exist. As an academic, I didn't get to see all the interesting, kind of weird and wonderful things that are happening. So maybe you could tell. There are so many. So let me go by industry. So in financial services, we certainly see AI helping with financial crimes, detection, AI driven machine learning, graph analytics, to look at entity resolution, identifying individuals that could be a suspicious activity for money laundering, for example. And AI application in that space is hot and exciting. And certainly, there's an opportunity to influence the direction in finance. We also see applications like microfinancing and loans in emerging markets and applying AI for credit scoring for those who don't have credit histories. I mean, the applications are so diverse. In the health care life sciences space, AI has applied to disease progression modeling as applied to patient similarity looking through electronic patient records and genomics and so many exciting spaces. We're certainly very invested in partnership here in that space. Education, personalized education solutions like tutoring and classroom solutions at every level in manufacturing, applying AI for automation, for improving worker safety, improving manufacturing yields, and on and on. There are so many. And I think the fact that there is such a breadth of application is what's so exciting. And every one of those problems, it's not about taking an existing AI model, applying it, we're good to go. Every one of those requires innovation to understand the unique types of data, unique security requirements, whether you're HIPAA or have financial regulatory challenges, must keep your data on-premise. Each of those issues poses some unique challenges and then some opportunity for innovation. Richard, do you want to add? Yeah, so I think one area that I see emerging quite a bit and I won't talk, I think Catherine addressed a broad range of problems already. One observation that we are making as we move forward is and Antonio referred to it as salaries of NBA players. Not many people can afford salary of NBA players. You might realize it. That automation will become a critical step as we move forward as well in addressing that bottlenecks emerge, automation of AI becomes a critical step as we need to take as we move forward as well. So some of the problems that we are solving and we need to solve as a community, as we move forward will be breakthroughs in that area, whether it is deep learning or machine learning or combination of those techniques or otherwise. That to me becomes a critical area whereby a broad range of industry problems can be solved actually. Great. What companies, IBM customers or otherwise, have been especially adept at applying AI in their businesses, exclude AI tech companies like Google and Amazon and you shouldn't assume that they aren't our customers. Just an interesting side point, but do you have some perspective on that? Which industries? So one thing that I've found myself starting to realize is for instance that banks are increasingly AI companies that happen to transact in money. And many, many industries are sort of turning out this way. What are some other examples? So banking would be the number one on our list and we see hiring and many of you probably see that as well, right? And the amount of talent, deep, deep AI skills that exist in many of the large banks of the world, they are doing a lot of innovation, engineering and even more what we call research, right on the research end of applications in the banking industry. I think that's the one I would call out first. I certainly think in the pharmaceutical space there's a lot of interest in applying AI, whether that's drug discovery and many other areas, there's tremendous opportunity there and lots of activity. I would say areas like manufacturing probably less technical investment on the manufacturing front and more interest in partnering with companies like IBM to help them apply the technology. I think the very, just from a broad, I think use case point of view, when you start interacting with customers that's a very broad use case. And interestingly, yes, financial services, companies are leading that. In fact, many in Europe are at the forefront of that as well, we see that on a regular basis. And the other ones I would really say will be in the area of media is another one where we are seeing a lot of traction as well. But I would really say any company that deals with interaction with a large investment in their customer interactions where their brand is at stake is trying to actually apply AI in interesting ways to lead those innovations and media and certainly manufacturing as well. And Antonio, so you're on the front seat. In Kendall Square, startups are everywhere. So you've had a few interesting ones come out of your own lab and I'm sure you see your colleagues also spawning startups. How do you think, which areas for startups do you think are especially ripe for AI from where you're sitting? Well, I agree with what you say about the banks, but it really is about all of the companies. As soon as they have to do anything, there is AI in between. And it's just amazing because artificial intelligence is having an impact in almost anything, especially small companies even, because as soon as they have to deal with massive amounts of data or complicated things and they don't have the workforce to deal with it, they are going to need AI. So there is really companies with the full spectrum of applications. I mean, of course, autonomous driving is one of the main ones and then healthcare is another one. In fact, the two that came out of students in my lab were on those two topics, but there are really things about many other aspects of technology that doesn't necessarily relate to just banking and it's very broad. Great. Since the MIT Watson AI lab does fundamental research, what all needs to happen before the breakthroughs in the lab will really make a difference. So maybe this comes because I say that it takes 20 years for fundamental research to do. So let me... Box ticking. This doesn't mean that nothing is going to happen from now on until 20 years, okay? There are many things that are starting like 18 years ago that might happen in two years. So every single year something new started. So there will be a few things that will start within the lab and we hope they will have impact in 20 years and everybody will remember that it came from here. But there is also many researchers working in the fullest stack of research areas with different levels of maturity and some of them might happen in a month or in a week or they might be already submitted to a paper, to a conference. It might be like yesterday that it happened. So we'll see all these things happening during these 10 years. It's not just in 20 years. And I think this is the partnership, right? How can IBM and even some of the IBM clients begin to leverage the innovation that's happening here together and figure out how do you apply this for solving some of those hard problems? And it's, again, it's not like falling out of your chair but I think it starts with we have innovation. We have something that's really special because we've collaborated on defining those right problems to solve. We've solved some of those in new ways. Now we can say, okay, how do we apply that, right? How do we take this and then in the real world with messy data, with real clients, go begin to see how to make that real and solve some of those real problems and then scale it. Mature? The last word? No. Let's go make it happen. I can't wait for 20 years actually. So I just need to make it happen actually. I don't have the luxury. All right, so we're all gonna make it happen now. So our time is actually up. This is a good time to end. Thank you all for staying and for this time.