 Hi everybody and welcome to the literally just launched Queensland AI Hub. There's the sprocken the hoodie. Queensland AI Hub is in Queensland, so I actually was only wearing this for the advertising. I actually don't need it. All right, so welcome to sunny Queensland. My name is Jeremy Howard. I'm originally from Australia. I grew up in Melbourne and then spent 10 years over in the San Francisco Bay area. What I always used to think of a Silicon Valley, but then I got there was staying in San Francisco when somebody said, let's meet up in Silicon Valley in an hour and a half later. I still hadn't got there and I thought, oh my God, okay, it's actually quite a long way, especially with the traffic. So San Francisco Bay area I was there for about a decade and returned back here to Australia two months ago and have made the move from Melbourne to Queensland, which I'm very, very happy about. So this is a really lovely place to be having said that overwhelmingly the reaction that Rachel, my wife and foster co-founder and I get when we tell somebody, you know, when they come up and they'll say, oh, welcome to Australia. Welcome to Queensland. How long are you here for? Oh, we've moved here. You've moved here. Why? And there's this kind of sense of like, why would anybody want to move to Australia? Why would anybody want to move to Queensland? You were there. You were in Silicon Valley. Not really San Francisco, but what are you doing? And, you know, to be fair, it is a reasonable question because so to be fair, this is not exactly the global hub of AI and AI investment. In fact, we're way down here in terms of investment in AI at a massive 0.29% of global investment. And this data is from Andrew Ly from BOAI. Thank you very much to Andrew, who's actually given me quite a lot of cool data that I'll be sharing. So, yeah, I definitely feel that I got to say it's 0.29% more than when I left. So that's good. But, you know, I want to kind of make the argument today that actually this is a really great place to start a tech start up and actually a really great place to do AI research or AI implementations despite the obvious issues. So let me tell you about this insight through the lens of kind of describing my journey, I guess, to get here. So my journey, as I said, kind of started in Australia, right? That's a bit of a thick one, isn't it? That's true. Making that a bit thinner. Okay, so I started out in Australia. And 25 or so years ago, I thought, you know, it'd be really cool to start a start up. I mean, I can only think of the start-up sentence. Start a company. You don't make a company. And then I thought, well, there's a problem, Jeremy. I don't know anything about business. So, you know, initially it was like, oh, let's do a start-up or a company. And it's like, nah, you don't know anything about business. You don't know what you're doing. So let's learn about that. So I actually went into consulting. So I thought, okay, let's go to McKinsey and Company. They know about business and spend a couple of years there. And I went to a couple of different consulting firms along that journey. And what I discovered along the way is there's no such thing as business. There's such a thing as like making things that people want and then selling it to them. And that's about the end of it. So I did certainly learn some valuable skills from my time in consulting, particularly the skills around how to influence people, how to influence organizations. But the actual explicit feedback I got about my ideas were on the whole terrible. For example, I was very proud of myself when one day I came in to work with a CD-ROM that I bought that contained this really cool thing. Somebody had got lots of data about what movies people like. And it's like, this person likes these movies and this person likes these movies. And through some kind of magic I didn't understand, which I now know is called collaborative filtering, you could type in some movies you like and it would tell you other movies that you might like. And so I went into and I talked to one of the directors at the consulting firm and I said, imagine building a company based on this. Like you could even have like a website that wasn't static, but you go to their homepage and it could tell you what things you might want to buy. And wouldn't that be awesome? And the consulting director was like, you have no idea how companies work. You know, this isn't a company. Companies are about competition, about market forces. This is nerdy technology. Similar reaction when somebody was talking about creating a new web search engine, which was going to be just like Yahoo, but as a Java applet. And so, and it would also have the power of these like big brands behind it. And I kind of said to them, I don't know. I don't know. I wondered about like, what if we, instead of having like lots of humans finding websites and putting them into a hierarchy, could we use like an algorithm that would automatically find interesting websites based on like what you typed in or something. Similar reaction. This, no, no, no, no, you don't understand. Humans need other humans to help them find things. You can't like get some computer to like do this very human job. And so overall, this was kind of my experience of learning business. And this is the first piece of advice I have for potential people doing tech startups here is don't listen to old people because we, us old people, you know, don't know what we're talking about unless it's explicitly about the actual thing that you want to do. And they actually have years of experience in that thing doing it in the new way that you're thinking of doing it. Because otherwise, all you get is, you know, these kind of biases about business as usual about the status quo. So somehow, you know, and I mean, in my 20s, I didn't know that. And I thought there's something wrong with me that I didn't understand business, that I didn't understand why these ideas were bad ideas. So I actually ended up doing consulting for 10 years, which was eight years longer than I had planned, still trying to figure out what's wrong with me. Eventually, I decided to do it anyway. So that was the end of consulting. And I thought, okay, I'll start a company. Now, the problem is that I had read that statistically speaking, new small businesses generally fail. So I actually had a genius move. I decided to start two new small businesses because I thought, you know, probabilistically speaking, better chance of success. So I started two companies. I started Fast Mail. And literally within like a month of each other, I started Optimal Decisions Group. Now, aren't you drawing Optimal Decisions Group? So Fast Mail was an interesting startup. It was basically the first one to provide synchronized email, whether email you got in your phone or on your laptop or in your workplace, you had to see the same email. It's something that actually everybody in business already had because they used MS Exchange or they used Lotus Notes, but normal people didn't. And I wanted to have that. So I built this company and it's still going great. And then Optimal Decisions was an insurance pricing algorithms company. So very, very different. Fast Mail sold to millions of customers around the world. And Optimal Decisions sold to huge insurance companies. There's basically only three or four insurance companies in Australia big enough to use our product and then, you know, a couple of dozen in America, some in South Africa and so forth. So very different kind of things. And I didn't know anything about, you know, the Australian startup scene. So I didn't get any government grants. I didn't get any funding because, like, for a consultant, you don't know about this stuff. You just build things and sell them to people. And so these were not Australian startups. They were startups that happened to be in Australia. For example, Fast Mail, at the time, this is really weird, I called up IBM and I ordered servers. And I had them shipped to somewhere in New York that I'd never been. And they plugged them in for me. And so my servers were in there. Because, like, why wouldn't you do that? The cost of bandwidth in America was about 100 times cheaper than Australia. And the number of customers I had access to in America was orders of magnitude higher. And so it never occurred to me to have my servers in Australia because Australia's far away and it's small and it's expensive. And kind of similar with ODG. The focus, I mean, I certainly had some Australian clients, but my focus was on American clients because there's a lot more big insurance companies in America. And so this turned out great because living in Australia, I didn't quite have a sense of how far away we are and how much no one gives a shit about us other than maybe like Cricket. But they don't. But the fact that then we were just companies, not Australian companies, it didn't matter. It didn't matter we were a long way away. It didn't matter we were somewhere with crappy, expensive internet. You know, we were competing on a global stage without any constraints caused by our location. And so that turned out to be great. We ended up selling Fast Mail to Opera, which is a Norwegian company. We sold ODG to Alexis Nexus, which eventually is a UK company. And you know, that turned out great. And so the kind of advice I guess I found, I feel like from that I got out of that was in Australia, don't try to be an Australian company. You know, yes, there's lots of agriculture. Yes, there's lots of mining. But that is tiny compared to all the world out there. And furthermore, Australian companies are very, very hard to sell to. They're very conservative. They're very slow moving. If you create something like Fast Mail, right, where anybody can go on the internet and give you money for your thing, that tends to work out great. So like for example, you come across this company called Octopus Deploy, which was a guy in Queensland who thought, oh, I could create a better kind of continuous integration system for .NET, create an open source software, check it up on GitHub, made a better version that you could buy if you wanted like 10 copies of it. Again, it's a similar idea. It wasn't an Australian company. It was a company that happened to be in Australia. And a few years later, now a few months ago, they got, I think it was $185 million of funding. And none of that funding was from Australian investors. It was all from American investors. So it kind of bypassed the whole Australian thing and just focused on saying like, you know what, I'm a pretty good .NET developer. I pretty much understand quite well deployment. You know, well, I don't know, make something that anybody can just come along and use. And so it's similar thing now for Rachel and I with Fast AI. We started Fast AI, which we'll come back to later in the US. We're now moving to Australia. It doesn't matter. Like no one thinks of Fast AI as being an American AI company and we can do it just as well here as there. And so, you know, we have access to the global marketplace. Having said that, the next startup, and some of these are co-founded, it's ODGA co-founded and obviously the next one, which is Kaggle co-founded, with Kaggle, we decided to try a different approach, which was to get VC funding. Now, a similar thing, you know, I said to Anthony who we're doing this with, let's not even try to get funding in Australia because Australia doesn't fund tech startups. Like it's basically so little as you could just ignore it, it's tiny. In fact, the amount of funding of startups in Australia in a year is less than the amount of funding of startups in the US in a day. So when I say it's different, it's very, very different. So we went to San Francisco to try and get funding. And we were pre-revenue and honestly we didn't tell this to the VCs. We were kind of pre-business model. We were pretty enamored with the idea, but didn't quite know how to make money out of it. And so we thought we were being very bold by asking for $500,000. Okay, that's crazy. But we did, you know, and I will never forget the time when we went into Andreessen Horowitz and Mark Andreessen said, how much money are you looking for? And we said, $500,000? And Mark was like, hmm, what would you do with $5 million? And we were like, make a better company. And this was actually the start of a theme in the Bay Area, which was every time we'd say we want to do X, people would say like, well, okay, that's great. What if you could make an even bigger X? Or like, what if you could make an even better X? So then the Node-Cosler came to our little co-working space in San Francisco. And this is the other thing to know, if you ever go fundraising in the Bay Area, everybody knows everybody. And they all know everything about what's going on. So if an old was like, oh, I heard Mark Andreessen is looking at giving you $5 million. I'm like, oh, yes. What would you do if Coastal Avengers gave you another $5 million? And we're like, wow, you know, it just kept pushing. And it was a very different experience because I found doing my little startups in Australia, it was always like, you know, oh, I'm trying to create an email company that does like synchronized email and I'm trying to sell it on the internet. And almost everybody would say like, why? Microsoft already has an email service. Yahoo already has an email service. They're bigger than you. They've got more developers than you. Honestly, is there any chance that, obviously there's no chance you can beat them. So why are you doing this? Is there something smaller you could do? Is there something more targeted you could do? Is there something focused on the Australian market you could do? That was like everybody, best friends, colleagues, acquaintances. And it's very difficult because you end up constantly doubting your sanity. And the truth is to be a tech founder requires, you know, a whole lot of arrogance. You know, you need the arrogance to believe that you can actually build something that other people are going to want to buy and that then other people who come along and try to compete with you won't do as well as you and you'll do better. You have to have the arrogance to believe you can win. You know, which is a lot of arrogance. But you also need the humility to recognize that other people come along and they actually have some better ideas than you. And so sometimes you should borrow those ideas or sometimes you should try and find ways to do it better. So it requires this weird combination of great humility and great arrogance. And in Australia, I found people mainly notice the arrogance. But yeah, in the Bay Area, there was, you know, everybody was just like, oh, this is really cool that you're trying to do this thing. You know, how can we help? Can we help you make it bigger? The other thing that I got a lot in Australia was this kind of sense of like, why are you trying to create that when they're already perfectly good things? You know, like what it's like, it's like you're a winger or a complainer. It's like things aren't good enough. You know, why aren't you just, why aren't you okay with what's there? There's this nice sense in the Bay Area of like, oh, it's really cool that you're trying to do something better. And so there are some cultural things that, you know, I felt Australia's kind of needs to get over to build a great tech entrepreneur ecosystem. Because, you know, it doesn't have to be Australia-wide, but you want people in your community who are cheering you on, you know, and who are believing in you. Anyway, we didn't actually end up taking money from Andreas and Horowitz, I can't quite remember. Oh, that's right, I remember why. They hadn't done any machine learning investments before. And so what actually happens with these VCs is the VCs you speak to don't do any of the tech stuff themselves. They hand it off to mainly to academics, right? Which is something we don't sort of have a great ecosystem for here either, is like you don't see this strong connection between investors and academics in Australia. In the US, you know, Bernard would ring up one of the professors at Stanford or Berkeley and say, can you please meet with Jeremy and Anthony? You know, this is what they're building. Can you check this, this and this? So with Andreas and Horowitz, I mean to their credit, they, through their DD, they kind of came to the point where they said, okay, we're just not convinced about the size of the machine learning marketplace. We haven't done machine learning before, we're not comfortable with this. So we ended up getting our $5 million from somebody else. And one of the really interesting things in the VC world over there is the whole thing is so driven by fear of missing out by FOMO. So then suddenly people that we hadn't heard from suddenly started emailing us with like, can you come here today? You know, we really want to see you guys. We're really excited about what you're doing. These are people who would not reply to emails for weeks. And I'll never forget one of them. I'm not going to say who. We went down to there. We're like, we kind of had a promise between Anthony and I had a promise between ourselves. We'd never say no, right? We'd take every opportunity. We're like, we were sick of talking to VCs. We're like, okay, we've said, we've said, we'll always say yes. And so glad we did. Otherwise we would have missed out on this amazing situation. The people who said they were dying to see us left us waiting, I can't remember, for like half an hour in their giant boardroom. And then this guy finally does come in. He charges in. No introduction. I hear you're going to take money from fucking Mark fucking Andreessen. Is that right? And I think Anthony was about to reply. And the guy doesn't let him. He goes, well, let me tell you something. If Mark fucking Andreessen was here right now, I'd throw him out the fucking window. I'd break his arm. I'd take him to Stanford hospital. It's just down the road, you know. And then I'd fucking break it again. This was his introduction. And Anthony goes, we're not taking money from Mark Andreessen. Well, that's fucking all right then, because I fucking hate Mark fucking Andreessen. It's like, it was so much like this over there. The place is crazy. If you've ever seen Silicon Valley, the TV show, it's all real, but it's crazier than that. But they couldn't put that in the real thing. Do you guys remember the hot dog detector in that show? Did you notice there was a real hot dog detector? They actually built for it on the app store. That was built by a fast AI student, by the way. He used to come in every week to class. And he had always asked these weird-asked questions. He'd be like, I can't tell you what I'm doing. But let's say somebody was trying to find microphones. And then they got lots of pictures of microphones. And then some of them weren't microphones, but they looked like microphones. And eventually the show comes out, and he's like, OK, that's what I was building. That was so great. That was definitely one of our star students. Anywho. OK, so with Kaggle, what happened was I actually didn't expect us to raise any money, honestly. So I just kind of was humoring Anthony. He was always the one with gumption. And I was like, yeah, OK, I'll pitch, and I'll build the financial models, and I'll build the deck, but don't have high expectations. So then we raised over $10 million. And, yeah, the node coaster kind of looked at us and was like, so when are you guys moving here? I was like, oh. And obviously at that point I can't not, because I've been in every pitch and whatever. So that's how I moved to San Francisco. And I got to cord my mom and was like, oh, this is what just happened. So, yeah, I mean, moving to San Francisco was interesting. It was like, all right, so let's do that. Australia, US. What is going on with this? Yes. There you go. It was interesting. I was really starstruck. I was like, oh, there's Google. There's Facebook. Meetups would be at Google or Facebook, and I'd be talking to a Google product manager. And I was definitely like, wow, this is very exciting. I felt quite starstruck. But the other thing I really noticed was I was talking to these legends, but then I was like, they're actually really normal. I kind of expected them to be on another level. I felt like as a little Australian nobody, I would just be dominated by these people. But no, when I compared them to my mates back in Australia, they weren't all that. They were fine. They were smart enough. They were passionate. But they weren't on another level at all. And I kind of realized that actually the Australian kind of talent pool is just fantastic. But there's this huge difference in opportunity and belief. Like everybody I spoke to in San Francisco, literally I'd stay in Airbnb for the first few months. And the people that ran the Airbnb, I was like, oh, you're here doing tech startup? Because everybody's there doing tech startup? Yeah, yeah. Oh, yeah, me too. I'm a photographer. And I've got this idea that's going to revolutionize how photography is done in product development settings. Like everybody you talk to has not just got an idea, but they want to tell you about it. They believe it's the best idea. They believe it's going to succeed. I don't get that, or at least at that time in Australia as I was kind of in Australia, I didn't get that nearly as much. So I think that was a really interesting difference. And it gave me a lot of confidence in myself as an Australian to see that actually Aussies are not way behind. We're actually pretty damn good. So that was kind of interesting to me. But there was other differences there. I guess it's part of this kind of, I call it boldness, right? So I feel like folks there were on the whole more bold. But interestingly, even though they were in the center of the world's biggest marketplace, they were still actually more global. None of them were trying to build American startups or American audiences, American companies. There was always this assumption that we're going to chuck up on the internet and everybody's going to go and buy it. And in terms of who really needs that attitude, it's us in Australia. Now one of the really cool things about being at Kaggle was that I got to see, I was the chief scientist there as well as the president. So I actually got to kind of validate and check out the winning solutions. And so I was always really seeing what are the actual best ways to do things right now. And around 2012, I started noticing deep learning, starting to win things or at least do pretty well. And I had last year's neural nets like 20 years earlier. They kind of put them aside as being like, probably going to change the world one day but not yet. And then 2012, it's like, oh, I think the day is coming. And that really became very clear during 2013. So one of my real concerns was, which I shared with my wife, Rachel, was that the people using these neural nets were like all the same person. They were from one of five universities that were all very exclusive. They were all white, they were all male, and they were all solving stupid problems, like trying to find their cats in their photos or whatever. Okay, it's nice to find your cats in your photos and people make a lot of money from that. But where were the people trying to deal with global water shortages or access to education or dealing with huge economic inequity? It wasn't on the radar. And we knew that that was because you only get a kind of a diversity of problems solved if you have a diversity of people solving them. So we actually started getting pretty concerned about that. But at the same time, I also felt like maybe there's some low-hanging fruit. There's something I could do right now that would make a really big difference. To give you a sense of this, I wonder if I've got any slides about this. Let me have a little look. So to give you a sense of how I feel about deep learning now and I felt the same way about it then, it's a fundamental technology that I think is as important as electricity in like it's literally like electricity and steam engine kind of said, okay, you don't really need to generally put human or animal energy inputs in anymore once it was eventually really sorted. And kind of deep learning is on the way to doing the same thing for like intellectual inputs. It's kind of this fast extraordinary thing. And there are people who kind of have this sense of like, oh, neural nets are some hypey, fatty thing. It's just another in a long line of AI and ML technologies. I just don't agree with that at all. If you just look at what it can do, right, so here's an example of Dali, which is an open AI algorithm. You type in an illustration of a baby daikon radish in a tutu walking a dog. And these are not cherry picked. These are the first things that it does. It's not finding these. It's drawing them from scratch because nobody's asked for that before, right? You type in an armchair in the shape of an avocado. It draws these for you. This is not something an SVM does. This is not something a random forest does. This is not something a logistic regression does. This is, you know, to somebody who doesn't know what's going on, it just feels magical, you know? DeepMind created this thing called AlphaFold, which blew away decades of research in protein folding from a bunch of people who basically never worked on protein folding before. I mean, the closest, you know, a really close example of this, kind of, that I've seen, is early in the days of my medical startup in Lydic, we were bringing in everybody we could to tell us from the pathology world, from the radiology world and so forth to tell us about their research. And so we had this guy come in and tell us about his PhD in histopathology segmentation. He spent 45 minutes telling us about his, you know, new approach involving a graph cut algorithm and waterfall and blah, blah, blah. And he was getting, like, news, data-the-art results on this particular kind of histopathology segmentation. And we were like, oh, that sounds pretty cool. He was like, yeah, I used to think that too yesterday. But I saw you guys doing some stuff with deep learning and I kind of got curious. So I thought I'd try this with deep learning yesterday and I ran a model overnight. And it beat my last five years of work. So now I'm not so sure. And, like, this is, like, a really common story. Like, every time I try just about anything with deep learning, I'm, like, beating everything I've done before, beating other people, what other people have done before. And the interesting thing about this is if you haven't done any deep learning yourself, you might not realize that there really is kind of just one algorithm. Like, there's very, very little changes that go between kind of one model and another. So for example, I looked at the source code for the AlphaGo Zero model, which was the thing which absolutely smashed all previous Go playing approaches. And the model was almost identical to the computer vision object recognition models that I used. You know, it's basically a bunch of residual layers with convolutions and relays and batch norms and stacked up. And, you know, it's just an extraordinarily powerful general approach. And so it's really cool kind of as a researcher because you can read papers from, you know, proteomics or chemoinformatics or natural language or game playing or whatever. And like 90% of it you get because it's just the same stuff re-jigged in a slightly different way. So that was kind of how I felt and how I feel about deep learning. And actually I realized that there really was some low-hanging fruit at that time in deep learning and specifically it was medicine. Low one literally was doing deep learning in medicine. And it turns out that there's such a shortage globally of medical specialists of doctors that according to the World Economic Forum it's going to take 300 years to fill in the gap to basically allow the developing world to have access to the same medical expertise as a developed world. But this is totally unacceptable. I wonder if we could help make doctors more productive by adding some deep learning stuff to what they're doing. Let's try and do a some kind of proof of concept. And so we spent four weeks, me and three other people spent four weeks just training a model on some lung CT scans. And again like literally none of us knew anything about radiology or whatever and we discovered much to our kind of shock that this thing we trained had much lower-false negatives and much lower-false positives at recognizing malignant lung tumors than a panel of four top-standard radiologists. So that turned into my next start-up which was called analytic. And again for analytic, I went the VC route, raised over $10 million. So this time this was actually started from the start in the US and it was kind of a lot easier because I knew people. And yeah, I mean this was both great and disappointing. It was great in the sense that I really hoped that this start-up would help put medical deep learning on the map and it absolutely did. I got a huge amount of publicity and within a couple of years, particularly in radiology, deep learning was everywhere. On the other hand, it always felt like I'm just doing this one little thing and there's so many great people around the world solving important problems and disaster resilience or access to food or whatever and they don't have a way to tap into this incredibly powerful tool. And so between this and this kind of concern about inequality and the kind of exclusivity and the homogenous group of people working on deep learning, Rachel and I actually decided to start something new, which was Fast.ai. And so Fast.ai is all about helping everybody do what Enlidic is doing but not having a bunch of deep learning people do it but to have disaster resilience built by disaster resilience people and to have ecology stuff built by ecology people because it's much easier this is our hypothesis, it would be much easier for a domain expert in ecology to become an effective deep learning practitioner than from a deep learning practitioner to actually fully immerse themselves in the world of ecology to the point that they would know what problems to solve and where to get the data from and what the constraints are and how to operationalize things and understand the legal frameworks and then we started Fast.ai this was quite at the extreme end of kind of ludicrous ideas because there was just this total knowledge that everybody said to do deep learning you need a PhD you probably need a postdoc it's something that only a few people in the world could ever be smart enough to do you need very, very deep math and you need increasingly you're going to need more computers than anybody can afford and it was really lots and lots of gatekeeping and thankfully it turned out our hypothesis was actually correct and in the intervening years we've trained through our courses hundreds of thousands of people and every few days we get lovely, lovely emails from people telling us how they've just published a paper in a top journal or they've got a new job or they've bought deep learning to their startup and increasingly they're using also the software that we're building the Fast.ai library to do this more quickly and better and so that's been that's been really great and you know one of the important things here which I guess is something I did learn from consulting is that the world's smartest people are not all at universities what universities do have are the people who stay in the same place their whole life you know if you're an academic at a university you've literally spent your whole life in educational institutions and so these are not generally you know not always but they're not generally the most bold and grounded group of people as you may have noticed and in fact in industry there's a lot of brilliant people doing brilliant research and so this has been one of the interesting things in Fast.ai is a lot of the really powerful examples we hear about are actually coming from from industry unfortunately the problem with America is well you know so we realized we couldn't stay there so we're bringing up our child there particularly after 2020 because you know so we tried really hard to get back and eventually the government here let us in and coming back to Australia was just amazing because having lived here my whole life I kind of had this vague sense that Australia had a really nice culture and kind of this like something about going to America that was a bit off but then coming back here it just really hit me that like Australia is such a bloody good country like and the people like there's this kind of like you know sense of this kind of fair go and this kind of sense of helping people out and this kind of informality and it's just after spending 10 years in America there's this huge breath of fresh air to be back here and that fresh air you know how when you're really hot and there's a cool breeze and you really that feels great it's like that you know it's like it felt like a stifling humidity for 10 years and I kind of came back to sanity so that was amazing but at the same time I was also shocked by how little have changed here yes a whole lot of endangering networks had sprung up none of which existed when I was here but when it actually came to the rubber hitting the road I was trying to find people like doing like really world-class deep learning research or building startups which had you know huge global impact or venture capitalist investing in the biggest boldest ideas and I can't really find it and actually Michael Evans was kind enough to let me share some some stuff that he has been working on kind of looking at this from a data point of view and you can kind of see it in the data right from an investing point of view seed an angel investment in Australia is like per capita is like an order of magnitude behind the US and this is like this is where things get going right if you've got 10 times less money per person going into like getting things going that's going to be really hard for entrepreneurs right investment activity Australia is not even on the chart so our investment activity in AI is ravaging around 20 million dollars a year and here's something that Michael told me that shocked me last year it decreased by 80% now you might think oh fair enough COVID guess what the rest of the world it grew by 20% so on the rest of the world investors went like oh this is creating new opportunities in Australia which is like not even hit that much by COVID investors but they went home so this is kind of lack of risk taking that's a real concern there's a lack of investment in research so you know this is a CD average not only are we worse but we're getting worse and again this is the fundamental stuff seed investment angels research so in general tech our share of the global value added to the amount of stuff value that we're adding to the economy this is the Australian tech share of that and it's near the very bottom of the OECD we're behind chili turkey so and I this is like data points that reflects something that I was already seeing so like I kind of caught up Michael I said this is something I'm seeing am I mad and it's like no you're not mad I've got the data just to show you what you're seeing this is actually the one that that was kind of resonated the most with me in terms of talking with enterprises this is a Deloitte study talking with big enterprises they asked okay why are you interested in AI half of Aussie enterprises said oh we want to catch up or you know keep up 22% said because we want to get ahead and this is a worse this is worse than every other country that they spoke to Aussie customers are so conservative you know they really I really noticed this like if you want to sell to enterprises in Australia you have to tell them that their competitors already bought it you know that if you want to say you could use this to power ahead of your field and become a global success story they don't care I don't exactly know why this is but it's true in the data and it's kind of absolutely true from from all of my experience having said that in the OECD it's like right at the top in terms of like our use of tech and this is what I was saying earlier like Aussies are awesome we're smart we're technical and yet we're nearly at the bottom in terms of our investment in tech so it's kind of this weird thing and this is actually why I think Australia is a great place to build a startup the reason I think this is because if you can get past all this stuff pulling you down all this like why bother you'll just get beaten can you take less money than you want blah blah blah you're in a place where you're surrounded by brilliant people they don't have other cool tech startups to go to on the whole not that there's none right but there's relatively very few one of the things that was fascinating in San Francisco was that people would say like we've got such an edge because our R&D hub is in Melbourne and so we're paying on average one quarter to one fifth of the salaries of being paying in San Francisco and they could actually get people straight out of university and in Liddick to get people straight out of undergrad I had to pay them at least 200 grand US which by the way if you're a student not working on deep learning right this is the technology where people who understand it and can wield it well can get paid 200 grand straight out of undergrad so it's not a bad thing to have in your toolbox even from a job market so it's actually sadly it's kind of like this hidden gem it's like this diamond in the rough and so I've often noticed when kind of VCs come and visit or top researchers come and visit they're often really surprised at how many brilliant people are here because let me tell you in San Francisco even though I'm Australian I'm looking out for it you don't hear about that it's like even looking at academic papers I'd always be looking out for really influential academic papers that help me with my work and deep learning do they have any Aussie authors and invariably if the answer was yes it's because they've moved to the Bay Area I think that's such a waste we have all these brilliant people we have this kind of fantastic system we've got technically competent people in the workplace I think there are big opportunities here but I'd say for building a tech startup and obviously for me I particularly think building an AI startup where deep learning is some key component why wouldn't you be like a steam age and trying to create a new kind of loom that doesn't use steam you know it doesn't make any sense to me anyway so you create startups here it's like do it in as un-Australian a way as possible right it's like you don't have to have Australian investors you don't have to have Australian customers just believe that you can put something up on the internet that people are going to buy you know don't worry about whether it's mining or whether it's agriculture or whether it's something your PhD advisor who's never built a deep learning model thinks is interesting or whatever you know to me that's kind of the secret to how you know we can have some great startups here and I will say as that happens things will change right and things are already starting to change so like something really interesting is what's happening in Adelaide right so Adelaide has this fantastic AI and machine learning center and they're doing something which is almost unheard of in universities which is that they're forging really great partnerships with the tech community to the point where Amazon is now there too right and so Amazon has gone and said okay we're going to partner with Adelaide University of Adelaide and so there's now kind of the two centers next door very closely related and of course what's now happening I can't tell you the details but it happened to know lots more big tech companies are now planning to head to Adelaide as well and so you can imagine what's going to happen right now lots of people are going to like go to those and then they'll leave and they'll create startups and then other startups will want to go there and then other big companies will want to go there and so and then of course what's going to happen in all the other capitals they'll be like oh my god what's happening in Adelaide we have to do that as well and this is very very different to how things are currently done because universities like here are in many ways incredibly anti entrepreneur anti tech entrepreneur so for example you know a lot of brilliant work gets done out of UQ and QUT there's sponsoring this AI hub that's fantastic but if an academic there wants to start a startup they have to give QUT 70% to start and let me tell you that's literally impossible so there's zero successes because that's no one will invest in that company and the founder can't even be invested in that company like and it's not just Queensland this is basically every university in Australia Adelaide made a huge step of going from 70% to 49% compare this to like Stanford or Berkeley where like every academic I know there in engineering or computer science has four or five startups that they have a 5% equity stake in you know half of their students go to those startups then those students find interesting research directions from the work that they're doing which they then go back and then they fund a new group of people at the university I mean if you look at the relationship for example between Stanford and Google it's like constant back and forth research huge amounts of funding from Google to Stanford lots of job opportunities for standard people at Google the idea that the way you leverage your academic talent is by forcing them to give you 70% of their company absolute insanity and it's totally not working and I personally know of many academics in Australia who have decided not to start startups because of this reason and also because most universities will tell you you're not allowed to keep working here if you're working at a startup which of course it should be the opposite it should be like oh wow you're getting industry experience you're learning about actual applied problems will pay you a bonus you know so there's a lot of kind of issues with with how the kind of tech sector is working here and how entrepreneurialism is working here but the most important thing is the kind of the raw foundation that we have which I think is one of the best in the world and so that's one of the reasons that you know we came here is because we want to help anyway we can change Australia from a diamond in the rough to a glowing diamond that everybody around the world knows so that's what we want to do, thank you that's awesome to get an insight into your experiences over the last well since you started your first startup from the beginning when you first started to when you went to US and you had your first couple of months back in Australia what's harder getting an idea getting money or getting good data to make it all happen I think if getting good data is the thing you find hard then you're doing the wrong thing right so the thing you're doing should be something which you're deeply in that field right so if you're somebody in the legal industry you should be doing a legal startup if you're in the HR industry do an HR startup if you're in the medical field do a medical startup because then getting data is easy because you're surrounded by it or your friends working companies with it you personally work in companies with it so I'd say start working on a problem that you're deep into and then coming up with an idea that shouldn't really be hard because like everything's broken you know if you notice like nothing quite works properly everything's like finicky and frustrating and has stupid bits so like just particularly like stuff at your workplace you know all the stuff that like takes longer than it should or problems that have never been solved properly so really the key thing is execution and tenacity like one thing I really noticed with fast mail was when we started fast mail it was actually pretty hard to start an email company because there was very little open source software around and you know very few examples of how to build this kind of thing but very quickly there was kind of like all kinds of open source software appeared and we got new competitors monthly and they'd stick around for like six months and then they'd disappear because they'd give up you know because it was hard and I will say like in most startups I've been involved in every month it feels like there's a problem so dire that we're definitely gonna die but you kind of have to keep going anyway so I think it's your execution and tenacity thank you Jeremy the Dolly model is very impressive when I was young it was obvious what a computer model didn't understand it couldn't recognize a car for example when you look at that model it's not clear to me what it does and doesn't understand anymore I wondered if you had a comment about that only to say I actually don't care about understanding or not like I'm kind of philosophically interested and I am a philosophy major but as a deep learning practitioner all I care about is like what it can do so yeah I mean it's a fascinating question I don't think there's any way to ever answer that I actually don't know what you understand you could tell me but I don't know if you're telling the truth it's just a fundamentally impossible question to answer I think but it's not one we need to answer we just need to know what can it do what kind of do any new courses planned for 2021 under some vague definition of planned yes we need to do a part two of our deep learning decoders course so that's planned in the sense of like yeah I should write that sometime and the other course which I'm really excited about is I'm planning to do a course which is a kind of full stack startup creation course involving everything from like creating a Linux server and system administration of Linux through to how the domain name system works through to investment, through to getting product market fit, through to collecting data and so forth there is a course a bit like that that 2021 it might be 2022 but those are a couple of courses I'm looking at okay so that's that one already are you going some track days since I had a five year old I'm suddenly less interested in rotor cycling I'm sad to say so yes those courses I described will probably be in person at whatever university feels like having us so that's what so yeah what's next you know keep doing what I'm doing but what I want to do is I want to do fast AI with awesome Australians it's from a really selfish point of view I'd like this to be like a real global hub of brilliance because I want people around me to be awesome you know so I would love it if people were flying here in order to be part of this amazing community and I actually think that's totally totally doable particularly because you're so beautiful like I think we've got a lot of benefits particularly in Queensland like who wouldn't want to come to Queensland yeah I'm thinking about in graph data sure that's a great question what's your recommended way of marketing okay so how to market an early stage company the first thing is make it very very easy to use your product and to buy it right so I don't want to see like okay so there's got to be a pricing section right I don't want to see a section that says like email us for sales inquiries that's insane like I'm not going to who does that right if it says it's five dollars a month it's like fine here's the credit card right I can need to be able to use the damn things like have an open source version or at least a you know a limited demo or something have screenshots like I want to be able to go to your site and immediately know what are you selling is it any good what does it look like can I give it a go and then pay you for it so that's kind of like the first is to avoid anti marketing you know where you make life difficult for your customers and then the best kind of marketing media right so like you will get far far far more awareness of what you're doing if you can get something written about it in Wired or the Washington Post or BBC then any amount of advertising and that is all about personal outreach from you the CEO to journalists who you have carefully researched and confirmed would definitely be interested in what you're doing and then telling them about it and that actually doesn't happen very often most people go through like PR firms who journalists can't stand dealing with and so like I've basically never paid for any advertising of any sort but if you do a Google news search you'll see that we've got a shitload of media right and last year in particular I wanted to like go take that to another level because I co-founded masks for all globally and so I literally wanted every single person in the world to know they should wear a mask and so this is like my media campaign so I just wrote to everybody I talked to everybody and ended up on everything from Laura Ingram on Fox News through to BBC News and wrote in the Washington Post in USA Today and you know nowadays thank god people actually wear masks so yeah, media is your magic marketing tool. Last one? Okay, last one. Thanks so much Jeremy and Rachel and your team for the fast AI course, it's amazing. Thanks. And accessible. In the era of global warming how concerned should we be with the energy usage of deep learning models and your thoughts or ideas on how we can master this challenge? So it's a great question. The way I think of it and I'm not an expert not this but the way I think of it is from a general resource constraint point of view we should not be using no more resources than necessary to solve the problem including energy. Unfortunately a lot of companies like Google to pick one out at random have huge research departments that are very explicitly in center to create research that shows the results of using huge amounts of energy, specifically huge amounts of Google compute hours and this is very very effective marketing because if you can like journalists love writing about big engineering solutions and they'll always say like this used 10,000 TPU hours or whatever. Now, you know, so the thing is this is what we focus on the vast majority of problems that we see solved in practice useful pragmatic solutions are solved on a single GPU in a few hours and you can buy a GPU for a few hundred bucks and you know there's all kinds of resources like this as the resource of just like the amount of education that you need or the amount of data that you need or whatever but like overall people dramatically overestimate the amount of resources you need to get good results out of deep learning this is very explicitly because that's what a lot of people want you to believe they want you to believe that you have to hire their consulting firm that you have to use their compute hours that you use their special software that you have to buy lots of their cards or whatever but yeah overall there's there's a massive over emphasis on you know using vast amounts of stuff in deep learning sure I have to mention so in fact I have a slide about Don Bench if I remember correctly because I kind of skipped over it. So this is something that Rachel and I are passionate about and we we went crazy when TPUs came out because Google was like oh these are these magic special things and the media was like okay everybody else is screwed now because they don't have TPUs so only Google could now do deep learning and so there was a competition at that time that had just come out to shortly after TPUs got marketed to Google called Don Bench which was basically who can train image net the fastest and at this time the fastest people were solving it in about 12 hours let's say solve it that means getting it to an accuracy like I remember top five accuracy of something percent and yeah not surprisingly Google put in their pitch and I think they got like three hours or something Intel put in a huge TPU part or whatever Intel competed and they of course put in an entry with 1024 Intel servers operating in parallel and we thought okay if these guys win we're so screwed because it's going to be like okay to be good at this you really do need to be Google or Intel so some of our students and me spent basically a week seeing if we could do better and we won and we did it in 18 minutes and it was just by using like common sense and just like yeah just keeping things simple and so like we've done similar things a few times because these big tech BMOS are always trying to convince you that you're not smart enough that your software is not good enough that your computer is not big enough but it's always been bullshit so far so I'll be. Thank you Jeremy I think we'll call it there if anyone else has any further questions feel free to try and have a chat to Jeremy depending on when he chooses to leave I think from everyone here at The Meetup we just want to say thank you for sharing the time Rachel as well we'll hopefully have you down here in the next few months and really looking forward to having you involved in the local community for everyone who is keen to be involved in the