 Pada pagi hari terdapat... ...dan untuk pembantu untuk mengorganiskan ini. Dan... ...tapi sebabnya... ...mungkin dia hanya memperkenalkan saya, Patrick Tay, Pemerintah Pemerintah Pembuan NTUC... ...pengurangan perkembangan yang anda ada di sini sekarang. Apabila anda berisik mengapa... ...pengurangan Unia dan Pembuan Ketuaan NTUC untuk mencuba... ...pengurangan perkembangan, komuniti perkembangan... ...kita berlaku untuk mengambilkan kepada orang lain... ...dan tidak hanya sepatutnya tetapi secara penuh... ...cukup dan pembantu. Jadi saya beritahu kawan-kawan saya, kita sebenarnya mempunyai UPME Centre di sini, di belakang N2C Building. Sebab itulah kita beri kamu satu kelas kelatan untuk kamu dapatkan idea tentang apa yang kita lakukan. Pembuatan N2C yang telah berlaku, telah mempunyai sebuah pelanggan dan mencuba untuk membantu pelanggan, pekerjaan profesional dan pekerjaan. Dan kita sangat berminat dalam komuniti sejumlah data. Untuk satu sebab, kerana kita tahu bahawa industri ini bergerak sangat-sangat. Kenapa kita tahu itu? Kerana pelanggan, pelanggan, pelanggan dengan beberapa industri dan pelanggan. Jadi kita tahu bahawa, termasuk sebuah pelanggan. Selain UPME Centre, pelanggan industri pelanggan dan pelanggan juga ditemui dengan UPME Centre. Kita tahu bahawa data, sains dan analitik data adalah perkara yang besar. Sebenarnya, beberapa bulan kemudian, kita telah mempunyai sains analitik data. Di sini di auditorium ini, apabila kita mempunyai nama yang terkenal untuk berbicara tentang analitik data. Mereka telah mempunyai sebuah pelanggan yang sangat bergerak. Mereka telah mempunyai sebuah pelanggan yang lebih daripada 200 orang. Dan kemudian, tidak mempunyai sebuah pelanggan. Dan kita tahu bahawa industri ini sangat besar dan kita mahu berjumpa dengan komuniti. Jadi, kita mempunyai, mari kita mempunyai dan melihat komuniti ini. Jadi, kita hanya sebuah sponsor yang berjaya. Kita bukan pelanggan dalam sains data, tetapi kita harap dapat melihat komuniti ini mencari dan melakukan baik. Kerana kita tahu bahawa komuniti ini adalah sebuah yang akan memberikan banyak kerja dan tempat yang menarik untuk bekerja. Kerana kita mencari kerja depan, kita mencari kerja depan untuk kerja depan. Jadi, kita gembira dapat berbicara dan melakukan perkara yang kecil untuk mengambil perkara ini. Dan kita harap anda menikmati sains data ini. Dan lagi, di bahagian perubahan dan perangganan, kami terima kasih banyak untuk anda untuk menikmati sains data dan nanti akan mendengar Adam Retolov dalam beberapa jam. Dengan itu, terima kasih banyak dan selamat pagi. Baiklah. Baiklah. Jadi, saya sedang beritahu untuk mengubah resolusi, saya rasa. Maafkan saya sekejap. Apa resolusi yang anda mahu? Apa resolusi? 720? Okey. Adakah itu pekerja? Baiklah. Saya Adam, CTO Co-Founder di SkyMind. SkyMind adalah salah satu start-ups yang terkenal pada hari pertama. Kita mulai pada tahun 2013 di San Francisco. Semua ini berlaku dengan Google, Facebook dan sebagainya. Jadi, masyarakat di Deporting Hight Machine telah dibuat pada masa ini. Dan apa yang kita memutuskan untuk membuat, anda tahu apa, saya akan mengambil Deporting Hight Machine dan saya akan pergi begitu. Jadi, apa yang kita lakukan adalah kita memutuskan, kita tidak akan membuat Python, kita akan membuat Java. Kerana pada akhirnya hari itu, itu yang paling terkenal pada sistem produksi. Sains data juga mempunyai sains data. Dan itu adalah orang-orang yang menikmati kawasan ke kawasan. Jadi, kita pergi ke kawasan itu. Jadi, apa yang anda akan dengar hari ini, adalah terutamanya mengenai, bagaimana saya membuat Deporting Hight Machine? Apa yang akan berlaku jika anda memutuskan untuk membuat ini? Anda tahu, anda mendengar cukup tentang kawasan, anda mendengar cukup tentang kawasan, anda mendengar tentang fungsi kehidupan. Ada sebuah sebuah tekstput yang saya hanya menghidupkan. Jadi, apa yang saya mahu lakukan hari ini, adalah mungkin bercakap tentang sesuatu yang tidak terdapat sepanjang banyak, yang adalah, bagaimana saya menggunakan hari ini? Dan jika begitu, apa yang saya suka? Dan apa yang saya lakukan dengan itu? Jadi, saya akan cuba menjaga ini sebagai sebuah kawasan terbaik. Sebenarnya, bagi saya hari ini, pilihan saya di Python bukan sebuah kawasan yang baik untuk produksi, tetapi sebuah kawasan yang baik untuk portotipan. Jadi, saya sangat memutuskan, saya sangat memutuskan dengan kawasan. Jadi, jika anda ada pertanyaan seperti, apa yang berlaku di Spark, kenapa java, saya dapat menjawab pertanyaan yang nanti. Tapi, apa yang saya akan bercakap tentang hari ini, adalah Deporting Hight Machine Acceleration Production Deployments dan Data Engineering. Jadi, salah satu perkara yang anda mungkin ingin tahu, adalah anda perlukan Data Engineers sebelum anda perlukan Data Scientist. Kerana anda perlukan seseorang yang menggunakan kawasan. Jadi, anda perlukan set-up. Jadi, apa yang saya lihat di kawasan ini adalah, pilihan pertama dari Data Scientist biasanya adalah Data Scientist dan Data Engineers. Jadi, saya rasa Singapura akan mempunyai kawasan kawasan. Jadi, beberapa anda akan perlukan java. Jadi, sebuah kawasan yang anda mungkin menggunakan, saya tidak tahu. Tapi, ya, seperti yang saya katakan, kebanyakan kawasan ini sepatutnya adalah kawasan jelas, kawasan data, C++ dan sebagainya. Jadi, jika anda mempunyai, tapi kita juga akan mempunyai pada akhirnya, kita akan mempunyai kebanyakan kawasan. Seperti apa yang anda mempunyai? Jadi, kita boleh bercakap tentang kawasan kawasan. Jadi, saya sangat mempunyai itu. Bagaimana saya memperkenalkan kawasan ini? Apa-apa yang anda mempunyai? Biar saya tahu. Jadi, dengan itu, kita akan mulakan. Jadi, hari ini, saya akan bercakap tentang kawasan saya yang saya membuat semasa 4J. Jadi, telah berjaya sepanjang beberapa tahun sekarang. Jadi, ia dibuat, ia dibuat untuk Hadoop dan Spark. Jadi, dalam kes ini, jadi, salah satu perkara yang saya akan mulakan adalah, kita memperkenalkan java, tapi java sederhana. Jadi, Hadoop, semua itu? Ya, ia bukan teruk. Tapi, java sangat bercakap tentang kawasan. Ia sangat bergabung dengan sistem file, dan berkawasan dengan kawasan kawasan. Tapi, ia tidak dapat membuat C++. Ia tidak dapat membuat kawasan hardware. Jadi, salah satu perkara yang mempunyai Python, adalah, ia adalah kawasan yang hebat untuk memperkenalkan kawasan kawasan. Jadi, mempunyai, semua orang menggunakan itu, Fiano, TensorFlow, C++, bindings, betul? Jadi, satu perkara yang, masalahnya adalah, anda tidak dapat memperkenalkan, anda tidak dapat memperkenalkan, anda mungkin memperkenalkan, anda mungkin memperkenalkan, anda mungkin memperkenalkan kawasan, itu banyak kepada anda, dan itu akan banyak kepada anda. Jadi, enthousiasis, ia cukup baik untuk, saya akan kata, 90%. Tapi, jika anda memperkenalkan kawasan dengan kawasan kawasan, itu akan sepatutnya, itu akan sepatutnya, java sederhana. Dan, anda tahu, dalam kawasan kawasan, sesuatu seperti Python tidak cukup baik. Jadi, jika anda menggunakan Spark, anda mungkin menggunakan Python Spark, anda mungkin menggunakan Jupyter, atau iPython Notebook, jika anda pun menggunakan Spark. Tapi, pada akhirnya, Python sebagai perlian, menghantar sesuatu yang dibuat javan. Jadi, perkara ini adalah, mengapa itu jadi botolnak? Kenapa tidak mempunyai javan, dan menggunakan yang sama dari Python? Jadi, itu, itu adalah pendapat C++. Jadi, alamak saya, dan saya akan membuatnya nanti, Saya akan kata Siphon untuk Java. Siphon adalah framework yang tinggi untuk menulis c-code pada Python. Apa yang ia lakukannya adalah ia menghasilkan c-code pada Python. Ia mempunyai stetik dan menghasilkan kompolisinya. Kemudian ia mempunyai stetik binari sebenarnya. Jadi ia menolakkan Siphon menjadi cepat pada beberapa masa. Sebuah framework yang anda mungkin ingin melihat adalah Numba. Dan Umba. Jadi ia mencoba melakukan perkara yang sama. Jadi anda perlu c++, anda perlu c-code. Siphon cukup baik untuk banyak perkara. Sebenarnya, salah satu perkara yang kita memilih java adalah... ...banyak sistem yang dibuat pada java yang kita menemukan. Paling kurang dalam industri. Mungkin anda membuat konsep google file sistem. Tetapi segalipun industri menggunakan sistem file Hadoop. Itu yang anda akan menemukan. Itu yang telah digunakan selama 10 tahun. Itu yang mereka menggunakan. Kita mempunyai teknologi yang lebih tinggi... ...dengan pengalaman yang boleh dibuat hari ini. Tetapi kita mengambil beberapa pakaian... ...yang menggunakan global interpreter pada Python. Anda tidak boleh mempunyai multithreading. Anda sebenarnya perlu menggunakan c++... ...untuk membuat apa-apa yang lebih cepat pada Python. Java sebenarnya 80% lebih cepat untuk menggunakan c++ pada sendiri. Jadi ia baik untuk menulis kode sistem stabil... ...yang menggunakan kode statis. Dan ada banyak infrastruktur sistem yang dikatakan. Kafka, Elasticsearch, Spark. Banyak perkara JVM yang digunakan. Itu adalah pelajaran yang penting... ...yang saya akan ambil daripada ini. Sebenarnya, kami masih memilih tuta. Kita juga memiliki CPU. Kita membuat hal dengan Intel. Kita menggunakan MKL. Anda tidak perlukan kuda... ...tapi anda dapat menggunakan MKL. Anda dapat menggunakan CPU. Sebenarnya, itu mungkin apa yang akan digunakan... ...pada aplikasi anda. Bos anda mungkin tidak tahu ke GPU... ...dan tidak tahu apapun yang perlukan. Tidak, itu benar. Jika anda fikir tentangnya... ...adalah bos anda akan berkata, ...atau saya, ini lebih cepat... ...untuk hal yang dikatakan. Kita tidak membuat hal yang dikatakan. Saya ingin membuat hal yang dikatakan. Kita perlukan GPU. Anda mempunyai masalah politik... ...kalau anda cuba membuat hal ini. Anda membuat hal yang dikatakan... ...an Amazon... ...tapi ia mengambil beberapa hari untuk dikatakan... ...dan anda masih tidak tahu apa yang berlaku. Dan itu sangat susah untuk mereka faham kenapa mereka perlukan hal ini... ...dan di atas hal itu, bagaimana hal yang dikatakan? Kenapa kita perlukan hal ini? Jadi anda tidak dapat membuat hal yang dikatakan... ...dan membuat hal yang dikatakan? Kerana ia membuat hal yang dikatakan. Apa untuk GPU? Kenapa saya perlukan hal ini? Sebab itu kita mencuba membuat hal yang dikatakan... ...dan untuk GPU dan GPU. Ketika mereka menghentikannya dan membuat hal yang dikatakan... ...dia dapat mengikuti hal ini. Saya akan membeli satu GPU. No, kita perlu empat. Saya akan membuat hal ini. Anda mempunyai hal ini yang dikatakan... ...dengan yang dikatakan... ...sebelum anda membuat hal yang dikatakan. So you have to pick the right problem as well but so one of the things we decided was well you know let's target the ops guys who actually run the clusters. Those guys know how to configure Hadoop clusters. I can go in and speak their language. So I'm not the Python data scientist. I'm the guy who actually touches the production code. So we go to those people. So what we assume is you've already prototyped something in AWS and Python. You know you use TensorFlow, you use Theano, whatever. And you probably actually the framework I would recommend for just getting started with deep learning is actually a framework called Keras, K-E-R-A-S. It's a higher level abstraction over TensorFlow and Theano. So if you're going to prototype something, use Keras. Franco Chollet, the creator of Keras actually has a manning book coming out later this year or next year I think. So there's a lot of getting started materials now. Most of them are in Python. And so you can do most of this on your own. That being said, you're going to have trouble deploying this though. Because your data engineers, when they go to run this stuff in Java and Scala, they're going like what's the production version of this? Unless you're just a small startup, you're probably going to have a data engineering team and they're going to want to know I want something in Java, where is it? So that's typically the audience we go for, if that makes sense. So that being said, you need C++, but there's problems with hardware acceleration and all sorts of other things that you might not be thinking about. So if you take any lessons from me today, it's figure out how to navigate the politics in your organization to get a small GPU cluster. That will take time. That being said, start prototyping now, start using the cloud, it's out there. You can do it today. It's not that bad. The biggest part of deep learning is tuning. At the end of the day, DL4J in this case uses a JVM, but we still follow the same constructs as Python, computation graphs, flow of matrices through a neural network. We still do the exact same thing. Python had such good ideas that I ported them to Java. So I ported them to Java and that's what we use at the core for our algorithms. So a lot of the concepts still apply. So the current landscape though, so right now, one of the problems is Spark is bad. So I'm going to call everything terrible tonight. Everything is bad and the world is going to end. So one of the things is, so deep learning mainly works on images. That's what everybody starts with. But Spark assumes this SQL abstraction. So it assumes Spark SQL, select star and you get an RDD back. Or a data frame. Oh, no, it's a dataset. So it would probably be data something. I don't know what it is. But they have this... Whatever the next abstraction is, they're going to have some representation of a distributed dataset. But right now, it typically requires SQL to get up and running. You can't do SQL on pixels. So pixels and images are just... It's 3x5x5 or whatever. It's a tensor. So it's a multi-dimensional data structure that has random numbers in them called pixels. So I take a picture of this room and there's red and there's all sorts of colors. I can't do SQL on that. What do you do? So we actually use... For our stuff, we use OpenCV. So we actually integrated OpenCV into Spark. So you can load image... So you can use the same stuff that other people do. But again, you can use them in Java. You can use them in Spark. So if that makes sense. So the current landscape is most mainstream tools just are not set up for deep learning. So I mentioned binary. So the other thing is actually... HDFS has a great byproduct. It's actually great at storing binary data. So what you'll find is that some random organizations have terabytes or petabytes of video footage and they're storing it in Hadoop. They don't know what to do with it though. So where they're storing the data they don't know how to use it. So they don't know how to analyze it. So you'll use neural nets for that. But then it's really hard to connect to where the data is actually stored. So what do you do there? You can't use Spark because again SQL and Pixels. So what do you left with here? So there was a lot of tools we had to write for ourselves in order to make this palatable for the Hadoop admin. That guy. The guy who holds the key to your cluster. So there's a lot of things we had to do to make this suitable for that audience. So that's actually a problem with Python and Spark is when you're trying to access data a lot of the image libraries and things like that assume data is local. So second-learn images it all assumes everything is local. So it's hard to connect to Hadoop. That being said, Wes McKinney at Cloudera he's now at Cloudera. So the creator of Panda works for a Hadoop company. Oh god. His sole job is actually kind of cool. Wes McKinney is a great Python advocate but he's actually a C guy. He does most of his coding in C. So he's actually a systems programmer whose sole job is to make Python and Java interop fast VSC plus plus. So he knows Siphon, the underlying C Python and he knows Java native interface. And he's making those two things talk to each other via C. So he's making interop with the Hadoop on Python easier. So I took the opposite approach and I said there's no hope for Python. Cloudera did the opposite thing and said well you know people use Python so let's just invest in making it easier to connect to Hadoop. So there's cool things happening that I think is going to make I think there's going to be a convergence. It probably won't matter what tool you're using in the next couple of years. If that's the case well there it is. But at the end of the day a lot of these underlying problems are still going to need to go away. What's a GPU? Why do I need to store that data? How do I load data? How do I load data from my cluster? How do I do that? You know those questions are usually roll it yourself and that's at the end of the day that's still time spent. You need to think about loading the data you need to think about how do we make it fast how do I make this palatable or tolerable to the guy who runs my cluster? So there's multiple things involved there so if you're going to get your organization involved in deep learning find a use case, prototype it yourself prove it works take that to your boss and then try to figure out how to deploy it. So at the very minimum you gave yourself an excuse but you can try to see if you can get into it. So what I would suggest is start with Python get in the cloud and then start from there. When you're ready to deploy to the production call me. But seriously, it depends on the scale though you might not need Java you might not need Java at scale. 1,000 Python servers might be fine. People tend to reorient things from Python to go or Java when it's an actual bottleneck. So that happens with larger companies but it varies enthusiasts aren't going to need to do that. So there's a very small subset of the audience that would likely do this. So the solution we kind of proposed was again this idea of Siphon for Java. So kind of like how West McKinney is making via C code is actually making the Python and Java interock faster we just did the reverse and said well we already have this Java CPP we already have the Java bindings for cafe, TensorFlow and all these other things already in Java CPP. So we have the tool we wrapped all the C code that Python people use we wrap that in Java so we have that too. So it's possible to run straight Java and Scala if you want to try it. So we have a lot of if you're going to learn so don't learn Java learn Scala it's a little newer and honestly it's closer to Python the Scala code we write is very close to Python but you get static typing for free which is kind of nice. So the other thing is we actually made image storage easier so 64 bit indexing what does that mean? It means we can store larger data structures than is normally possible in Java. So we actually for all of our stuff we have this concept of off-heat memory. So off-heat memory basically means we don't rely on Java's memory model we do our own garbage collection we have our own GC and everything built into the system we do this for CUDA as well so we actually manage memory on the GPU and you don't have to know we do that for you and this all happens passively so in this case the lesson here is think about storage think about the fundamentals of computer science loading data from disk is slow having things in RAM is fast that actually matters for training time so there's things especially for something like mini batch learning when you have a small when you have just a laptop and you're training on images you're not going to load all that in a RAM at once you might have 16 gigs of RAM in a laptop you're not going to load all that in at once so in this case having efficient data structures for storing these mini batches batches of images is actually good so Python can't quite do this C is low level and flexible enough however let me see and then from that though once we have more efficient storage then we can leverage what Java is good at so there's very fast databases written in Java, Presto, Cassandra Elasticsearch these are all things written in Java that are great at storing binary data so these things do exist sometimes it is C code as well but Java in practice has been fast enough for distributed systems actually distributed systems code you write the thing that connects your cluster together like that stuff is typically written in Java and is already fairly good at storing data so the thing we need to think about is just being able to access that data in some way so the other thing though is again Java is not Python the reason it took off is it's kind of close to MATLAB you have these ND array things you have this panis and it makes it's very concise and you have you have about a decade of code for scientific computing written in Python that connects to C, that's NumPy so Java didn't have that so we had to create it so we did that because again I believed in this giving low level access to like SIMD instructions and the latest hardware acceleration techniques I believed in giving people access to that in Java so that's something Python already had so we had to create that ourselves so that's called Live Indie 4J pure C++ code base that's all it is we're benchmarking because it's actually kind of cool we actually get to benchmark on DGXs so I have an NVIDIA supercomputer it's from Intel I actually have my hands on the stuff you read about because we do this C++ optimization ourselves so there's so many benefits so if you've even figured it out yet the bulk of SkyMind is actually we're actually a systems company most of our people are actually Java and C++ systems engineers who happen in no machine learning and actually this is actually true of a lot of the research labs a lot of the research labs until they hit scale it's not like data engineers are your first hires for your data science cluster it's actually very similar so GPU programmers are typically the first hires for research labs so actually Andrew Ng when he was starting the Bidu Research Lab was only hiring systems programmers because the thing he said was the machine learning is teachable distributed systems is not so distributed systems is kind of the hard part of this especially distributed linear algebra I mean that's esoteric you're only going to use that for deep learning it's definitely a very strange set of skills to have so if you have those things go get paid in the states you're probably going to make a lot of money I mean that being said governments are starting to use this kind of stuff but again it's a very specialized set of skills that you need to hire for let me see what time was I supposed to go till 8.30? okay alright that's about right I'm going to leave about 20 minutes for questions so so this is kind of what this looks weird this was not anticipated so this is what a production stack looks like this is at least our interpretation of this so what you'll see you'll have all this all these data sources under here you have relational databases so just think about your job you have MySQL you probably have some Postgres in there you probably have a Hidup cluster that probably uses Cassandra or HBase or one of the other databases you probably have about 10 databases you use you probably only know about one but you actually have 10 your whole company probably has 10 or 20 databases they use so there's a lot of fragmentation and all sorts of different data sources so one great example this is Twitter Twitter has lots of teams hello what happened okay so twitter has lots of teams i can broadcast too so twitter has lots of teams so think about that for a second so you have 20 teams each of them has an opinion about a technology right you guys you guys probably have your favorite stack all of you has a favorite web framework all of you has a favorite database you all know how to use those things and then those teams know word one thing but then when they go to work with another team they're like what's going on here you know that's actually why a lot of companies force everything to be API's no oh there it goes okay so that's why these companies i like this better companies force everything to be API's so this started with Amazon so Jeff Bezos Jeff Bezos said to every manager in Amazon i'm going to fire all of you unless you make everything API's that's what created Amazon web services i'm serious so that's how because they had a long lag on engineering time emigrating different teams is difficult so that kind of solve that kind of solve some problems but that doesn't the problem there is that doesn't happen in Hadoop so big data systems typically have one central data source and so the problem is moved from okay now we have all these web services talking to each other but now we have all the data in Hadoop what do we do there's a lot of benefits to having one Hadoop cluster some people have multiple but at the end of the day you're not usually going to connect to multiple Hadoop clusters you might connect two different HBase instances on the same Hadoop cluster but all of it is stored in the same data source so you still have this has caused a lot of fragmentation so you ended up with the same problem that people have so again just remember Amazon they had all these different web web product teams they tried using each other's libraries and people rewrote code and chaos everywhere so this is actually still happening in Hadoop the problem is there's dedicated teams called data engineers they take care of that so there's a big focus on data governance so what do I mean by that I mean say validating data input keeping an eye on storage and things of that nature so but the interesting thing about machine learning is you have all these different data sources and you're actually unifying things so for example let's just think about ad fact or actually no just a general problem of customer service if I want to know the state of a customer can your company answer that today can I say for this individual customer all logs like our interactions with calling this person their web activity on their login and the support chat where they're typing text the customer probably uses all those things depending on where they're at what time of day it is and everything else and they're just expecting the company behind the scenes to take care of that for them in reality internally you're like oh god you're the guy who has to go out collect all the data aggregate it in a way just pull the machine learning model that understands our customers and then you're like how do I do that so what you have to do is connect all these data sources together that's actually one of the things we're good at figuring out how to do that so that's actually why we did that in Java that's actually why we did it in an integrated way because a lot of the problem with Python library how many of you probably written scripts that are 200 lines long that collect data from different data sources and then maybe there's a CSV in there and then you have to filter then you map and then you look at it at the end of the day and it's a ball code and you're wondering where did this come from and it came from you incrementally saying run this go query get back data oh crap I made a mistake get data again then do that transform then you end up with this gigantic penascript and you're like what's going on you're confused because it's hard to track all these transforms in half the time actually you don't even document why you did that transform you forgot everybody does that so that's actually one of the things we wrote is something called data back it's sole job is just we cover this oh so I'm an O'Reilly author and I'm selling my book my book just dropped today so if you want to talk about end-to-end stuff this is all covered in my book 440 pages it's online but not being said one of the things that's great about static typing is it actually makes it easier to do it actually makes it tolerable to work with so when you have messy data at least it's static detected at least the compiler can tell you you're doing something stupid in Python you'll get a random error at one time how many times does that happen you don't get a compiler error can't parse to string to well why oops and then you run it again that's actually why a lot of that's actually why a lot of large scale systems were kind of written in Java because at the end of the day a lot of these things are messy job is verbose scale is not too bad but at the end of the day at least the compiler can scare some of this stuff for you so so this is an actual cluster this would be an actual cluster so one of the things that's not mentioned here is actually just GPUs though so I'll show a couple more kind of pictures that might help you navigate this a little bit but actually underneath all this is the chip itself so you have your des and you have all your data sources here but then in here you'll have a mix of CPUs and GPUs so that's what clusters will probably look like in a couple years you know you'll actually have to start thinking about what's at the chip level what chip am I using that's actually why there's a project you should know about called Apache Mesos Apache Mesos it's a resource manager Apache Mesos was actually Apache Spark was actually a sub project of Mesos Spark was actually a proof of concept for Mesos so Mesos now has billion dollar they're a billion dollar company and they have a significant portion of investment from Microsoft because they power a lot of Microsoft data centers now but Mesos added GPU support so this resource manager actually added the ability to say I'm going to submit a job to my cluster and I need at least one GPU so GPUs are starting to propagate I don't know you guys probably aren't on the container but there's a thing called Kubernetes and it basically controls Google's resources so if you're part of Docker you probably use Docker on your laptop and you just played around a little bit maybe you're cool, Linux on Docker it's a lightweight VM but it's not actually a VM that's another story though but you have this thing called Kubernetes and again they also added GPU support so a lot of people are actually looking at GPUs as a way of saying we know this deep running thing exists and we need better compute so we'll make it easier to do so it has real industry traction it's still very nascent though but it's starting to come out so you'll start thinking about you're going to start thinking about the chip soon maybe not now in a couple of years you'll be thinking about the chip that's my job now, as I think about the chip we help companies navigate that so no one thinks about how do I connect the data the storage to the computer how do I connect the new file system to a GPU that's like a very esoteric problem but that's something that a lot of companies have asked IBM being one of them IBM has asked that question I work with IBM Tokyo Switzerland and a couple of the US we work with Spark Technology Center we work with different parts of IBM because they ask that question how do I take an RDD and compile the bytecode representation the Java bytecode to CUDA code so they ask that question actually created something so this is starting to come up Nvidia has asked us that same question what do we do with Spark we keep running in this big data thing company Intel they've invested in data artisans that's the company behind Flink Flink is kind of a competitor to Spark mainly focused on real-time applications so similar API to Spark but better in all respects so Spark has a lot of traction but what you'll typically see you start seeing Flink for ad tech and other streaming use cases and you'll typically see the two side by side because they both use Hadoop so the takeaway is that the JVM people are continuing to innovate some of them are doing it in Scala but it's a thing the Java ecosystem is still evolving so you're still going to see data stored here so the best streaming infrastructure in the world is still JVM based for example Samsung's IoT platform that's actually all Scala so a lot of IoT platforms but the cloud side of that is all JVM based so they use the actor model they use streaming that's actually all Java based so you'll still see that so you're still going to have this problem deep learning seems to be more accurate for certain problems but again how do we connect this JVM thing to this GPU thing so that's actually what I predicted what happened a couple years ago when I started all this they're going to want to take this to production so what do we do about that and that's kind of where I'm at now because a lot of a lot of things start in Python and a lot of the frameworks are still in Python now so now the thing here is how do I use it on my job today so that's kind of the question I think a lot of people are still trying to answer for themselves so I'll just skip a lot of this I think I've already covered frameworks oh, one of the things I'll mention so I'm going to try to talk a little bit so Google put out this thing called TensorFlow Server written in Python these are real streaming frameworks people so this is what you're actually going to use when you're not on AWS on your small hobby instance you're going to use something like this so you're going to use Kafka, you're going to use Plank I mean even Spark Streaming Spark Streaming is kind of slow so if you don't have distributed systems or you care about this stuff feel free to ask, I can rant all day but most streaming frameworks so they're typically done in Java so what I'll say is there's something I like to call Python Island most people are going to stand Python Island it's a great location spot it's warm it's easy to prototype things but then when you want to go to Enterprise when you want to actually go in and enjoy this where your boss actually hits the approve button and you're in a large organization that deploys to do or Spark or what have you you're going to have to get off of Python Island a lot of people on Python Island stay there so these trees this pretty island those buildings are each universities so this is a lot of innovation happening on Python Island so Python Island it's a Bahamas it's a Bahamas of data science because you go there, everything's pretty there's libraries for everything but when you try to write imagine trying to write imagine trying to actually and then having something break on you at runtime rather than compile time because you forgot to add a string or something there's a reason static typing still kind of dominates most of the web and actually that's actually why I've moved to TypeScript or something similar to that because they don't want to deal with dynamic typing because it is error prone so prototype on Python Island take a vacation and then when you're ready when you're ready to start thinking about deploying this stuff you're going to have to think about the systems you're going to have to avoid how do I avoid running out of RAM on my laptop or on my cluster how do I avoid blowing up a machine and not having the ops guy call me like what are you doing you don't want to get that call you don't want to talk to your ops guy your ops guy is angry you don't want to talk to him because if he's calling you, you broke something just remember that it's just something to think about anything I'm missing here I guess one thing I'll mention at the bottom we imitated NumPy go ahead and take a picture of the buzzwords if you want but the bottom there is spark friendly so one of the things that a lot of people still haven't thought about is how do we run this on multiple threads so what does that mean exactly how do we do multi threading it doesn't actually do multi threading typically it's handled for you and C at some point but at the end of the day if you're going to one of the things we did was we actually took a lot of the existing C++ mechanics and made those manageable from Java so that includes multi GPU and a lot of the other fancy buzzwords if you want faster training we just give you all the knobs and you can do it straight from Java just kind of interesting actually so deployment so we just this is for training we run as a spark job we're just a spark jar you can build it locally we don't rely on spark but when you want to do kind of distributed training alright so so actually i gave this presentation a while back so one of the things we actually have now is reinforcement learning so we actually taught DL4J how to point doom so we actually have robot killing i can show you that too later if you want so it's called RL4J we actually over the summer we actually contributed to gem open AI gem, the reinforcement learning environment we contributed the Java bindings to gem so we're actually using by the way we're actually using reinforcement learning there's fintech use cases insurance so just get the word out there's actually practical applications for reinforcement learning it's not just games that being said i will go ahead and alright, right on time i will go ahead and just take any questions you have about deep learning i've pretty much touched everything at this point so general questions on deep learning if not i can talk for another 30 minutes that's not hard for me to do but i figured i'd open up to the beginners maybe go ahead so i'm not completely wrong with this and also it's an accessible way yeah it's all patchy it's all patchy license so hold on i'm just going to start dropping links now so it's called nd4j so this is the c++ this is the code so it looks very similar to numpy so we actually have a scalar wrapper called nd4s that wraps this very library and and it's actually all it actually looks like python so here we actually have some of the analogies so we have the exact same semicodic sugar so it looks exactly like python but it's statically tight and then we have and then in that and then one of the things we do is we also so as i mentioned we actually ported literally all the concepts from python so we actually even have the same storage mechanisms so we actually teach you what tensors are so we made a neat page this actually maps pretty well to numpy and matlab so if you just want to learn what an nd array is there's even a video there's a video play okay we explain it in 40 seconds what an nd array is so something to keep in mind so and then anyways the underlying the great thing we did was we didn't have to bundle the c++ with the java project so the c++ is actually pretty approachable so you can actually compile this on your own if you want one of the things we do though is we actually deploy these banners to maven central for you the pip of java so how we do it so you'll see this nd4j thing and one of the things you'll see is you'll see this linux power pc we even deploy on ibm servers you have x86 mac and windows so we actually compile we actually compile all these c++ things for all the different operating systems so this you just download this you don't have to know how to compile anything in c you just download it and include it you're done so that's one of the that's actually one of my frustrations the python ecosystem is compile it from source just do that every time so you have to you have to worry about compilers and all sorts of other crap but you just don't want to deal with especially when you're just coding in a higher level language so that's one of the things i'm glad we were actually able to fix can you show us an example sure so what industry do you want to know about manufacturing i've done banking okay so i will give so i did i did another talk earlier actually that talk literally i was presenting to business people and it was all use cases so i'll just pull that up real quick so this was let me see okay so this was this was today so i presented it to the sun tech the iot show and we had so we had a couple so we talked about kind of what deep warning is oh present okay so we kind of talked about what deep warning is so what would you use it for so these are different kinds of unstructured data so images self-driving cars, face detection voice search for example so one of the coolest things i think will happen in the next couple of years is we're going to start speaking to our phones and they'll actually understand what we're saying even with our accents so that's one of the neat areas i think Baidu kind of innovated on was they actually adapted to all the different dialects of chinese as well as all the different accents as well so i think they have the most accurate voice search system in the world right now just because the next thing was this was use case driven most people in china they treat their phone like their friend they talk to it like their dad can i get milk and then somehow milk will get delivered i don't know what that actually happens but it's pretty close what is the capital of california people ask those kinds of questions and that's how they interact with their phone so Baidu i think have the biggest reason to kind of make chinese really good so the best interface is no interface if that makes sense so if i want pizza i just pizza and then it should fall from the sky it's in my hand right so that's a smart it's not a smart home it's a smart presentation center so that's all going to be audio driven if that makes sense but the use cases are pretty broad though so time series this is what we mainly do time series data so banking, telco wad data basically anything if you've ever seen an apache log and it has a time stamp and an ip that's what we deal with typically and it's typically we also do machine logs predicting when your computer is going to break so data center predicting when something is going to break in your data center this is a lot of the hype stuff okay so it's all based on these data types but it all really depends on the use case that you're looking for so image search is a great example so this is e-commerce so one of the cool things about deep learning is since it knows how to kind of pick up its own hidden patterns it's actually able to come up with a compressed representation of images that you can use for search so you can treat like a basically a neural net activation as like a search query so that's called transfer learning so that's also actually used in real estate as well so Trulia does this Trulia and Zillow do this for their real estate matching so if you want to find a home let's just say you have an idea of what's pretty you can show it what's pretty and the machine will say oh you like that okay so you can speak to a machine better than through pictures rather than through text so Pinterest also did this so they made it so you can highlight like a portion of a picture and only get back images similar to what you highlighted not just a whole picture so image search is probably one of the cooler consumer applications that you could build so again e-commerce e-commerce uses this recommendation engines are big so deep learning it's used a lot in recommendation engines now so very similar to adtech adtech using deep learning for ad targeting like oh i know what you clicked on and i know your interests are i'm going to put all that into a neural net and it'll tell me what to click next it'll tell me what to add to serve you you can do something similar for recommendations okay we already mentioned voice search personal assistant so basically everything i'm talking to my phone use case is deep learning so i think every company is doing deep learning for this now see corny stuff information self-driving cars so one of the things about self-driving cars if you actually decompose the problem it's actually object recognition sensors as well as an aggregation of those things all to tell when to stop how to speed up that's all a combination of a lot of things so there's a lot of different neural nets in here actually handling the driving so despite the hype i know there's some self-driving cars stuff happening in Singapore actually let me ask about that is it only on restricted roads only one zone okay so what that company did was they literally map they probably took pictures of that whole zone they probably took pictures of that whole zone inch by inch so by news trying to do something similar they're trying to build special self-driving zones so that actually doesn't surprise me there was actually a there's a YC startup that launched out of my batch actually i was my commentator in winter 16 so they launched out of my batch and they were basically just driving around college campuses so restricted domain self-driving cars is it's at least a proof of concept but it's nowhere close to them picking us up from anywhere and driving us anywhere we're not there yet but it is mainly deep learning that is kind of driving the adoption of self-driving cars so sensors so also being able to predict oh you know at the current speed i'm going the likelihood i'm going to crash into something is x so neural nets and IOT are actually very big so again it's time series so environment stuff okay so automotive so NVIDIA one of NVIDIA's biggest markets i can't name their first one but their second one is automotive so they work with a lot of the car companies and they put GPUs in cars i think it's mainly the german car makers that they're working with but they're putting GPUs in cars and they're basically using it for embedded object recognition for collision detection among other things like that but it could also be sensor data it could also actually behavior is really good interaction with the rider in the car behavior is really big drones you know drones okay so we know what drones are used for surveying, surveillance annoying people mostly the latter smart homes walking up to your door and unlocking authentication and security is really big so that's faces all the people medical so detecting cancer you know finding certain parts of an x-ray and saying oh that looks strange you know so assisting doctors is pretty big automatic machine translation so Skype there's a lot of hype around it i haven't actually used it but the disclaimer but the idea is that you speak in a Skype and it it has a neural net for automatically translating the text, the voice to text the translating the text and then outputting it generating the appropriate audio that's all end to end deep learning so this stuff does exist another thing you can actually do for most languages you can actually take a picture with your phone and you can do automatic translation and OCR that's again one neural network that's actually a startup Google buy just to embed it into their app music generation so neural nets are now Beethoven so you can actually teach it to generate music which is kind of cool so there's no really lost function for pretty yet we might create one but you know at some point at some point there will be limited amount of creativity neural nets can can use so algorithmic music it's kind of a cool concept a lot of people, a lot of hobbyists are starting to take that on so there's actually a real application for this called musical information retrieval though searching songs or recommending things Spotify does a lot of this they datamin the raw audio in the music and then they use that to recommend new songs spam so a lot of machine learning use cases too so things you thought about, fraud detection, spam that's all pretty typical like anything you can use machine learning for you can use deep learning for because deep learning at the end of the day is just a subset of machine learning so it's a subset of machine learning so in this case like you have 3 or more layers on a neural network the old school 80s version of the neural net was a perceptron that was one layer or maybe one hidden layer this is about stacking 3 7 to 10 10 layers and then being able to actually tune a model tune a model and add so deep learning is just more fancy neural networks than 3 layers supervisor supervisor unsupervised so there's something called an auto encoder that you can basically on its own what it does is it does gradient descent on something called kaila divergence it's a similarity between probability distribution and what it can do is it can learn to reconstruct the data so it starts random and then it tries to reconstruct it what it does is it feeds its input back into itself and calculates a difference between the two and then what you can do is you can actually use that to kind of automatically learn features so that's being used and that's being used alongside like kamin's clustering to do automatic clustering of data so it's all gradient descent there's still loss functions involved but it's all supervised and unsupervised it's mainly so most things use backprop in supervised learning that's where most of the applications are still yeah and it has to be like the future like well so we're going to be so SkyMind is actually working directly with Francois Chollet creator Keras to create like a common file format so we actually ported we ported Keras to Scala so it literally it looks so I'm not joking when I say it looks exactly like Keras so if you just let me try to pull it up actually actually you guys can just it's not released yet but I'll show it to you anyways it's the we don't have a name for it yet we literally don't have a name so we're just calling it this for now it's not actually going to be this when it's released though so so we have an amnesty example so this is a confnat this is Scala so you'll see so again it looks exactly like Keras so this is built on top of d24j so so Francois released those pre-trained models recently we're going to hook into that for all that infrastructure and then that wraps Tiano and TensorFlow so that essentially gives me access to all that stuff so what we're going to do we're going to host it too I'm actually going to sponsor like a hosted server where all this stuff runs where people can just download like a model zoo we're also going to work on a large scale effort to Keras if it's not already because Keras already has a big enough community that we can just rely on that and then we're just going to take that and host it for people and then what we want to do is we want to encourage people that'll mainly be Francois because he's a researcher so they trust him to publish a neural network and they're a file format because the fragmentation is very large but Keras already has loaders and Torch already so it shouldn't be hard it shouldn't be hard to pour everything over it'll take a little bit of work but we have incentive to do so because we mainly want to target production systems so I have every incentive to do this actually so event, I'm working on it you know actually so to be fair there's also an attempt to create a PMML for neural networks coming out from the open CL standard body they're sort of working on something and I had actual implementation so I mean evening in these compiling included now that's all I'm going to say so Scala at this point you can just go straight to Scala I would recommend that mainly because it's similar enough to Python and honestly manipulating data in Scala is a lot more pleasurable than it is in Java Java I would say is for a lot of legacy systems and in our case we trust Java Java just has a bigger audience and frankly I trust I'm a Java programmer at heart that's what I've been doing most of my career so that's a big reason why this is in Java but we're mainly in Java for interrupt purposes with closure and Scala and whatever else wants to use it but if we're going to write applications Java it's not bad but you more or less can just go with Scala so a guy from Red Hat actually is going to contribute link integration soon which will help but actually DL4J in our proprietary stuff we actually mainly use Kafka so DL4J itself is actually standalone so this doesn't rely on Spark there's no Spark code in here so this will run on your desktop so even one of the DL4J in the wild so even like so we actually have a training GUI so so you can actually this is called flow where you can monitor progress of different layers which is kind of cool we also have the neural net diagrams this is all D3 based by the way we have the histograms so you can actually monitor neural network progress in real time and then this is all the deployment stuff and then okay, yeah, Mace House anyways, so I guess what I'm getting at is none of this relies on Spark this is all desktop so we only use Spark really for training and I highly advocate against using Spark's training you can if you want it depends on your use case so we can integrate with that now as well though I said we can integrate with that now as well though so we mainly use Kafka for a lot of our streaming stuff so if you see the I don't know if I have I don't have a diagram right now but basically for streaming models what we do is we just subscribe to a Kafka topic and just we can just update models like that sorry in memory occasionally to disk that's all you got to do so we stand up models with something called lagom so lagom or the equivalent in Java world another one is spring boot basically we embed it in a web stack and then we do state management in the wrapper I mean we have something called a model serializer you just got to do save done, load save so our models are serialized our mix of binary coefficients and JSON basically and then so we just occasionally save that to disk and we're good to go but a lot of it's in RAM that's why I made DL4J the way I did was because it doesn't assume anything about the environment it just knows about a matrix it's taking in and it needs to output something so we don't make any assumptions but we decoupled it from Spark streaming frameworks we just made it embedable the solution is just to make it embedable because everybody has their own esoteric requirements so so Aero Aero is interesting Aero is a great way to represent data so we want to use it for in memory columnar format so we want to use that as a way of passing data around kind of like protobuf so I don't think so I think a lot of people are still it might be better but it's just a it's an memory columnar format it's a very specialized use case I think it's so Clutter Clutter is obviously going to drive adoption of this somehow even if it's a brute force they're going to use it because it was written by one of their engineers so it'll happen at some point I don't know when but it's written by a lot of smart people Aero is a favorite project of mine I think it has a lot of potential especially when you think about compression and just data representation in memory if the idea of stored data in memory in a compressed format is in its own library that actually gives you a lot of options for using it in applications so I think the idea has potential and I think it'll catch steam but I think it'll take some time people didn't adopt protobuf immediately so some people they're starting to use protobuf because they need something faster so that's kind of what's happening now it takes it takes time to adopt that kind of stuff it's very use case driven good yeah