 I don't know if you remember the Sierra text adventure games like King's Quest or Space Quest But I don't have firsthand experience So these are the text games where you'd be like get knife and then it'd be like I don't understand what knife is and then you have to be like get dagger And then it'd be like I got the dagger and you'd walk around with this very old bitmap to animation So these were in the early 80s. I don't like talking to a bank, you know Colin Yeah, like one of those bad space systems, right? So so these types of enter games I remember they just came out as a Right after I was born and I sit and I play them with my dad and And on the basis of playing these computer games. I actually learned to read and write before I entered school just because By playing and actually reading on the screen to having this interactive process That was very driven to not have my dad in the loop to be able to play the games by myself I I just ended up learning to read and write so that's now, okay, so so computers were always a part of my life I'm a hacker born and bred. I started coding in C around when I was around 10 years old just face glued to the computer screen all the time and and It turns out that I'm like I just really enjoyed doing it I like coding under pressure at coding things correctly. I represented the United States in the computer programming Olympiad in high school when I went to Harvard. I was on the computer programming team there We got eighth place in the world. I think that's pretty good and and then after after I am after I graduated Harvard, I was like, well, what do I want to do? What do I think is interesting and I've always been drawn to what I think is the most applied and for me at the time I took a class from a bright young professor or Dan Malomet and I was like and he talked about statistical and data-driven natural language processing and I was like this this is important this data-driven approach is I think going to be Very important. I want to learn about it This is the right way to do artificial intelligence to use a lot of data to use a lot of statistics This is what I want to learn and I study it under Dan and And I started focusing on natural language processing But I realized I was not so interested in the linguistics per se, but more the the the large-scale machine learning behind it My approach has always been you know You can always sit down and take a human and put them in the loop and start engineering do doing doing the manual engineering But that can only get you so far. What you really want is approaches that are very automatic and general purpose Because then they have the most widespread applicability So I've always been pushing in in my research to try and take a step back and do the most general thing possible When I submitted my thesis in 2006, I read about this I read this tech report. It hadn't been published yet about this wacky new thing called deep learning and deep learning It was a highly ambitious attempt to move us closer to artificial intelligence But not only that it was methodical and elegant and it worked and I was like I think deep learning is what I really want to focus my energy on I spoke to the At this time, there are only three labs in the world working on it I spoke to all three of them and I decided I wanted to go to Montreal and learn about it I spent the past three years in Montreal with Professor Yau Shou Ben-Gio studying deep learning with especially applications to language processing text analysis and And it turns out it turns out that these three guys were right deep learning Is now the next big thing in machine learning and artificial intelligence It's a 300 researchers showed up at a workshop on it Which is very big considering the size of the community the academic community DARPA and NSF are now funding initiatives in deep learning and and I think they're right I think that it's kind of like how support vector machines SVMs were very popular five or ten years ago This deep learning thing is going to have a huge impact coming out. So so AI is sort of coming back I mean when I was a child there was sort of this big AI push, but it just didn't work Right, so that's the thing. So we all got disillusioned. Is that how? Exactly. So so so what's oldest new again, right? So so so because yeah I hope I hope it's still the case so so In the 50s and 60s, there were a lot of wild claims being made. Oh, just give us 10 or 20 years We'll solve this whole AI thing. We'll have like talking robots and they'll drive for you yada yada yada, right? and when And they over-promised and under-delivered and when it didn't work Everyone really became disillusioned and AI became a dirty word and it and it was for a very long time and I think due to the efforts of certain researchers who said, okay, you know what like it's time to stop like like, but it's time to stop just looking at Just models and approaches that we know can't solve AI, right? Let's Like we know we're not going to solve AI in the next five or ten years But let's at least start trying things that can can plausibly move us forward towards AI instead of just Just refining things that we know will never move us closer to AI and and and that's what they did applying I think methodological and scientific practice To to actually moving closer to this goal and and it does actually it has a has a media applications in industry That's a that was that was the essence of my try to talk is what what what's on the horizon? That you could apply an industry right now, but you just haven't heard of yet So your talk actually was the buzz of today. So I mean it was a pack house There's a couple of problems with the AI one is that it's expensive, right? Yeah, to research and actually to maintain so where's that funding coming from is that going to be private? Or is it going to be government? So I think Well, you know, honestly, it's one of these things that early on I think it's it's it's a As with anything new You know only only Only people that are either very dedicated to it or have a lot of resources can take advantage of it and then get a head start but recently there's been a lot more interest and I think that with Honestly, I'm a big advocate of open notebook science and code sharing. I always share my code So when people start sharing code, I think that's when When it really starts making making more waves also just recognition More widely in industry. I mean that that was one of the reasons it was important to me to come and speak at Trout a Conference about about these things. So the economic driving forces When it comes to marketing, we've known who you were we know what you did, but we didn't know why you did it So as it comes down into that me aspect AI and natural language processing actually it's kind of moving forward of the intention base Actions. So that that's one approach there. They're also the the other thing is that these techniques like you could take a lot of existing problems You know say like risk analysis, which is something the machine learning is very commonly applied to And you could take these deep learning approaches and actually get higher accuracy So you can use it on not just to try and tackle new problems Which I think is more speculative risky But also perhaps more opportunistic if you want to really try and push the envelope You could also just take what you're working on right now and then try and improve it and make it more sophisticated No, I'm let me say one thing. I'm actually an advocate of use the simplest thing possible Okay, if what you're doing right now works, okay, don't change it, right? Only if You're seeing like diminishing returns from big data All your competitors are using big data and then you need to eke out more of a competitive advantage because big guys No longer an advantage That's when you'd start using more sophisticated algorithm So it you should only really do it if it's going to provide a qualitative Improvement to your product. I'm not an advocate of just use the new fangle thing so a lot of it is also threat assessment property Relief so let's say wildfire modeling earthquake modeling and one of the things that natural language is helping with is moving that forward Because we can take pre-existing data sets for days, but what's the randomness of data, right? So is that something that you're interested in is the randomness of data? Could you say the question sure so? data as has a Has laws that we abide to we can understand what's happened What we're trying to understand with AI is what might happen. So randomness of data. How do you enter that into it? Yeah, so It seems like they're two parts of this question one There's this question of what do you do with time series and this this is still a very active question So I can't I can't really comment too much on that But the interesting thing about these new approaches is that the really base on trying to find as much structure and pattern and signal and the data is possible and that's essentially What what has driven these algorithms forward trying to find structure and then using this structure and these patterns to actually Enhance the predictive ability. Yeah, so what do you what do you make of IBM Watson? I mean interesting project. You've seen it. Obviously. I mean I I think So I'm of two opinions. Um, so so on the one hand, um, I you know, as I said, my approach is Always do the most general thing and really try and take a step back and and take humans out of the loop and really push That as far as possible. So, you know, their their approach is different They they're actually using a lot of human engineering, right? But then on the other hand when you're talking about the real world What you do is you take a general-purpose approach and then you start adding humans into the loop to get that extra mile So in that sense, I think it's really interesting to see well How far they can can they push it because that's the that's the real question What is the upper bound right now? How far can they take it? And I think I think it's great to see that in industry So are you following any of the work at the Kurzweil Institute? I mean, what are your thoughts on? Obviously what he's he's put forth over and over is different than what I believe is a I Um, I'm not familiar with his work to comment on it in a serious way So obviously the idea of singularity, right the idea of singularity I think is much different than actual performance or Validity of AI so how do you know when it's ready as far as application? I guess I'm I'm very much of I'm very much user driven So my approach is really if it satisfies a user need if it actually creates value for people then it's ready and And a lot of the time that doesn't mean artificial intelligence, right? It just means maybe something that works good enough on a specific problem So when basically when the users say that they like it is is is my answer You're thinking five years right now. It's a kind of a goal Well, it depends what problem we're talking about. Okay, right? So sentiment analysis right now a lot of people are working on it. My impression is that it's not actually that good maybe for a Qualitatively like qualitative improvement in sentiment analysis a few years I'm also I'm also an optimist, you know, if you were to ask me when are we going to have like robot cars? my I would say you know Ten years, but I'm an optimist So that's that's for full for full automation for some automation. We're almost there. Yeah. Yeah So Joseph, where do you go from here? You're doing this all this research You've been you like sucking up all this great information. Now. What do you want to do right? So I was doing I was doing my postdoc for three years in Montreal as I said and from my perspective I always really wanted to build applications and and put things into practice So for me the postdoc was really just sharpening my knives and right now. I'm ready to eat some steak. I Recently quit my academic post. I've been doing a consultancy Essentially I consult on when there's a large-scale machine learning or natural language processing question Or for that matter you want to take your data and monetize it. That's my sweet spot In general I'll decline gigs unless they're of particular strategic importance to my consultancy because I've identified a certain class of problems They consider of particular importance in utility So I'm not consulting full-time There's actually one of the main problems I'm working on is if you have a very large unstructured collection of information text documents You know pattern something like that and you want to organize it you want to make it navigable You want to make it explorable? How do you do that? So that's one of the products that I'm licensing right now So if people want more information about you and and I mean how can I follow you? Where can they go? I mean sure. Um, so I mean you can always You can always just send me an email Just Google me and it'll be easy to figure out my email address Besides that I want to point out so so I started on a Q&A forum and people actually have questions about machine learning natural language processing big data I started Q&A forum called it's the meta-optim that's name one company meta-optimized Q&A forum and And so if you had a question like what clustering algorithm should I use? There's a lot of folk wisdom that hasn't been commonly shared and now there's actually a forum for people to come together It's actually incredibly popular both in academia and industry It's got a large mind share among people that do machine learning natural language processing It's in it's a tremendous resource So that's one thing I just want to put out to the audience if you have questions about this sort of thing That's the right form in fact I think it's the most popular forum for machine learning natural language processing big data on the web right now So we here with Joseph Turian Really interesting discussion. Let me tell you what I've learned here in addition to many things of it forget books fire up The laptop and start coding And then and then sorry, sorry, honey. I got four kids Oh, no, no, I mean, I think I think it is great, but I also love paper Okay, so you read as a child. Oh, yeah, I think I think you know, you can't be something tactile You can't be something you can agree. Okay. So okay good So so don't forget books but but definitely fire up the laptop and start coding support vector machines been there done that Let me make one more comment about coding. Okay, so I've always I've always I've been saying this to my friends recently that um That basically like it I really feel like if you can't code nowadays It's like being in the 18th century and not being able to fence, you know It's like it was one of these things that maybe you could get on with your life and not be able to fence But you really it's a skill you really should have you know, if you if you want to be able to like like Well, yeah, okay now support that your comment about SVM was really interesting to me I first heard first learned about them probably about three years ago And it basically that was old news by then for guys like you and I was hoping that it would solve a data classification problem But it really hasn't but anyway, and then AI John old is new. Okay, so there's there's hope Find ways to make qualitative improvements to products and presumably lives and when the user loves it, you're there Yeah, right and then Google Joseph Turian if you want more information and check out the meta optimized Q&A forum Yeah, sounds like a great resource for people. So great