 Mae'r ddod, yn rhoi gwlad yr oedd yn y troio i chi i gyd ar y gyfodol o'r Oshug a'r Osch Camp. Rydyn ni'n gwneud o gwybod, y gwasanaeth ymlaen, a'r ffordd ymlaen yng Ngherniad Arfer, dwi'n gweithio i'r ysgynig yw'r ffordd yn amser mae'r amser yn ei gweithio'r ysgol. A ydych chi'n gwybod o'r gweithio'r ysgol yma yn y gweithio'r gweithio'r cyffin. Ond ydych chi gyd, yna bod ydych chi'n gweithio'r ysgol fydd, rwy'r bwysig y gwrth gynnig. Cynnyddio'r gweithio allan oedd ydi, ond rwy'n mynd i'n ffordd o dwi'n mynd o'r fath o'r 160, 151 cwm o'r cych. Rhywbwysig mae'r magoseneis, rhaid o gyllid, mae'r cyllid o gyllid yn y lektor. Rwy'n meddwl yw'r gweithio'r rhaid o'r gwrth gwrs o'r bobl yn hyn o'r pwysig yw'n mynd i gael'r magoseneis. Rwy'n meddwl o'r bwysig, rhaid o'r fath o'r gwrs is this, that's an AM radio among other things. Some of which work, some of which didn't. Then this kind of stuff came along, my very first computer, a ZX81, fantastic little machine from Sinclair. That led on to these kind of different processes, one of my favourites 6502, which was later used in the BBC 6809, something I used at college, which meant I had to understand things like this, the Neumann architecture for processing. There's the chap himself. That's how we used to do programming when I was at college. Occasionally we were let loose on our hex keyboard, which made it a little simpler, that kind of thing. Then later on we even had printouts and stuff like that, which was amazing. Then things like the Apple II and stuff came along, and we actually had screens and a keyboard, and we could type our stuff in and start programming and editing. Meanwhile, after two AI winters, the Perceptron was revisited. People like Marvin Mints did a lot of work on artificial intelligence, and I came across this in a bookshop when I was at university and fell in love with the idea of parallel distributed processing. This was really focused on artificial neural networks, among other things, heavy in learning, et cetera, et cetera. Same time, 16-bit processing occurred, 68,000 there, which made, this is the Apple Lisa, for those of you. This was the thing that was in between the Apple Macintosh, sorry, the Apple IIs and the Apple Macintosh. This hybrid thing with a graphical user interface. I ended up working with a lot of that, writing things like Pascal, for example, and Intel then came to dominate the market with things like this. That's the 55SX from IBM. I remember having to get something like 200 of those working, having a room full of them, trying to get them working with a graphics card that worked with some of them and didn't work with others, a British aerospace. Then I got back into doing a bit more of this, because I thought, oh, I could do a lot more with this now. I don't know if anyone recognises that particular editor, a DOS editor. Then you had things like Windows come along and graphics cards, which I spent an awful lot of time working with, American companies, UK companies, Japanese companies, building graphics cards, writing softwares and drivers and things for them. That led on to these kind of high-horsepower graphics cards. Meantime, CPUs were doubling in performance every 18 months, well, memory capacity was. The clock speed had reached its kind of climax at about three or four, but the capacity of silicon was still going up. You also saw some wilder things, such as these. That's a 64-core processor. I also got in back into hardware with people like Exmos, who do a multi-core microcontroller. You've also got a picture down there of the old transputer, from which Exmos kind of developed its background from. Then this made a real comeback. That's a neural network. In this case, it's a convolutional neural network. Suddenly, there was enough power, processing power for you to be able to do very complex networks. These then came back into the picture because they managed to accelerate it. They grew into things like the Maxwell and the Pascal and, more recently, the Volta from NVIDIA, which has literally thousands of thousands of little, minute floating-point units or numerical cores that can all work concurrently, which provide this enormous power to do all the matrix work, the matrix calculations, the multiplying ads that you need to do for any kind of neural network processing, which got me right back into this stuff. Because I've been working in electronics and robotics, then I started looking at ways that I could use that in these different areas, adding smartness to embedded systems and robotics, et cetera. Of course, you can't exactly have one of those to do your processing when it's an embedded robot or a car or a vehicle or an IoT device. There's just not enough power, let alone enough room to do it. So, what you need is a brain, preferably on the chip that doesn't use an awful lot of power, so you can run your stuff off batteries and things. So, this is a classic one I show every time I do one of these presentations to see where we've got to. It's the S-curve. The old technology in this case is really just the old van Neumann type of architecture being multiplied here towards the top. We'll only get you so far. And then there's another S-curve starting, which we still don't know exactly what that is. There's probably a good likelihood it might be something like quantum computing, but that's for another presentation. What I want to talk about right now is visual processing and in particular machine learning. Currently, video processing uses probably more compute resources than anything else on the planet right now. This has grown over the last couple of years enormously. So, this is like a magnitude more computation that's going on concurrently around the world than was before with the internet and things. So, what's going on here where there's lots of processing of images in big data, things like classification. You will see that Google, Amazon Web Services, Facebook, etc. are processing ginormous quantities of different images. Whenever you upload to Facebook, that is being processed. Faces are being recognised in there. People are being tagged, etc., etc. It's going on on a huge scale. It can also be used to actually enhance an image. So, you can take a relatively low resolution image and you can actually enhance it using neural processing type blocks and units and convolutional neural networks. You can also use it to actually animate videos and create these kind of fake videos and have politicians that you know or celebrities that you know actually saying things that they've never said by getting the GAN networks to actually improvise. Something else that we see more in our kind of space here for hardware, etc., is things like face tracking, analysis and recognition down from things like the, you know, using a camera, camera phone where it tries to track where the face is, etc., and get it centre and do the focusing and dynamic focusing across the images, etc. Also for security applications, your face can be your sign-in, for example, or from other security type applications, tracking, terrorists or criminals, etc., etc. And then the other thing is image analysis and identification. This is a huge growing market for embedded and real-time whereby you are actually using video systems in conjunction with hardware for actually doing analysis on and maintenance in terms of installed sites, etc. So I'm going to focus in on the last couple of these, but let's just put some scale into here. So if you're doing machine vision, what do you need? What sort of power do you need to actually get something useful done? So probably at the top of the embedded end are self-driving cars and trucks. They literally need to be processing tera-ops and tera-flops. These are vast amounts of information that needs to be concurrently processed. If you think about a self-driving car, it needs to process not only the video around it, but things like LiDAR, the point clouds, etc. It's also got radar. Some of these systems are layered on top of each other and it's processing them all concurrently. Not only does it need to work out where things are, it needs to identify what those things are. Are these safety issues? Are people walking into the street? Where are the lanes? Where does the car need to be relative to the other cars? Is this a junction? Is that a road sign? What does that road sign mean? Is this traffic lights? Should this car be stopping here? There is a huge amount of processing that needs to go on. So you can actually put this into cars now and certainly all the testing that's going on at places like Google, Tesla, etc. are using some very high-end Nvidia kit. Moving down from that, things like inspection drones and vehicles need probably less processing. There's less going on concurrently. The real-time nature of it is slightly less. Then we're talking about gigarops and gigaflops. Let me go down to your average mobile phone that has camera. This is getting into the area where this now has gigarop capability and gigaflop capabilities. I'm going to mention something about that. At the lower end, you've got these very simple robots, drones and IoT-type applications where you might just want to add a small amount of intelligence so that they can work without user intervention. If we look at the top one there, the self-driving cars, that board there is an Nvidia board that they're currently putting into many different self-driving cars. That has something like 320 tera ops per sec. It's just an incredible amount of processing power on a board that size. Obviously it's got lots of CAN bus controls and things to hook into the standard electronics that you find in cars these days. It's having to deal with, as I mentioned before, things like identification and interpretation of the objects around the car. People around the car has to deal with lane position and speed control. It has to be to drive itself within lanes, overtake, manipulate itself, follow a road, preferably not go over the pavement, not run people over except you. But it must also have some sort of perception and prediction. It needs to guess what's going to happen next. There's a lot of that going just in the way that we would do when we see things going on when we're driving or unconsciously making decisions all the time. It needs to have things like evasion strategies and things, very safest evasion modes where it doesn't actually run people or animals over or kill its occupants, preferably. This is massive. There is so much money in this market at the moment. It's incredible. Every vehicle manufacturer you can imagine is spending millions and millions of dollars in this area as well as the upstarts like Tesla and Google, et cetera. Moving down then, so what do we want to do if we perhaps don't want all of that capacity in terms of processing? What are we going to need? You know, several terra ops or hundreds of gigaflops. Then again, we're looking at an NVIDIA product here, which is a low-powered version of their Pascal chip or Volta. That might be the Volta version, which is their latest generation of chip. That has about 256 floating point cores on it, as well as what they call some neuro-processing cores or engines to speed, things like convolution up and addition multiplication vectors. The sort of applications you're seeing this stuff use here, you see a drone flying around a wind turbine. What it's doing is it's actually using an infrared camera to inspect the temperature for the various different parts of that wind turbine. Obviously, if there's some friction problems which you quite often get in the gears of those things, then you get heat buildup, so you will see a heat pattern. The drone is trained to be able to focus in on those and work out where those issues are and alert the operators that there's an issue with that particular device. Again, you get very similar things with the power lines there. It can actually map the power lines. It can measure the EMF force coming from the lines, et cetera. It can see if there's any issues there. Again, it can examine things like metal fatigue, et cetera, on the structure. Communications marks, again, the same sort of issues. It can go and monitor the outputs from those. It can also look at the physical structure. Then the next one with the drone down the bottom there is an agricultural use. It can actually go over crops, for example, and using a mixture of things like infrared, ultraviolet and regular cameras, it can map all sorts of information about growing crops, how well they're doing, what stage they're at, what sort of temperatures they're running at, that kind of thing. That can automate farming, in a great sense. On the right-hand side, you've got things like the submersible type robotic systems. Having something a bit more intelligent so it can find its way around and manoeuvre itself is much easier than you have to manually do that. The operator can then focus on the inspection tasks, maybe around an oil rig or something like that, for example. Then the one above there on the right-hand side, we've got a robot that actually crawls into long pipes and can actually find issues like fractures in the pipe leaks, things like that, et cetera. It can recognise those. Again, vision in all of these cases and machine learning, machine vision processing is essential to make all of these automotive. But again, these are still fairly expensive. That's Jetson TX2 NVIDIA card that you need to do the processing in that case. They're about 400 pounds each just for that actual processing card. Then you're going to need a board for that to sit into and all your sensors, et cetera. Not only that, because it's a GPU, it actually consumes quite a bit of power. So it might not be applicable for smaller type applications embedded or IoT type applications. So what are the options if we move down a bit further into a smaller power envelope and lower cost, hundreds of giga-ops and hundreds of gigaflops? What are our choices? This is an interesting example. The Kirin 970 systems on a chip was recently put in the Mate 10 mobile phone. Again, this is coming out of the mobile market from China. These are chips Chinese made phones. That particular chip, the Kirin 970, is probably one of the first of the new generation of chips that are finding cameras that claim AI potential. So not only do they have a quad-core processor that you see commonly in phones now, but you've also got the GPU in the ARM world that tends to be like a Mali chip or the next version of that, which is good for signal processing and graphics if you've got a screen. But they've also added something in here called an MPU. Now getting specifics about this is very difficult because they haven't actually revealed it. But having run tests doing inference object detection, for example, there are standard inference tests that will read a 320x240 image and detect hot dogs, for example, for one apps. They perform very, very well. So what you get here with the MPU, the critical thing here and the phone is you're getting kind of two times the performance for about half or a third of the amount of power use compared to running that on the ARM core and the floating point units in the ARM itself. So that's the way that the mobile market is going. There are equivalent chips, SOCs coming from Apple, Qualcomm, Samsung, et cetera, to put in their own phones that are going to take this further and further. ARM themselves have announced that they will be embedding their own numerical processing units inside their ARM cores directed directly at doing kind of matrix type calculations, just like a TPU does from Google. But obviously at very, very low power. So what you're getting is much more powerful matrix, much faster matrix calculations that use a fraction of the energy that the old ARM cores would use in order to achieve this. So you're going to see this breaking through certainly on most mobile phones. They will come with AI and machine vision included kind of thing. And that's good for us because that brings it down into the maker community in terms of powerful chips available at relatively low cost using low power. And we'll see that breakthrough. So what are our choices now? So if we're looking at doing some machine vision, what are the things that we have available to us at relatively low cost, low power Intel acquired a company, a covidious doubling based company about a year or a year and a half ago. And they've got this dongle called the compute stick, which has one of these tiny embedded processors in it, which has numerous what they call shark processing cores, which are very, very fast, as well as some hardware convolution type acceleration hardware. So you've got a very low energy package that's capable of actually processing tens, even hundreds of giga ops, but it's definitely operations, not floating point operations that this is aimed at. So it works well when you've got integer type operations, but not floating point type operation. So that's kind of, if you like, a very powerful kind of microcontroller, like our accelerated microcontroller. However, there isn't that many development boards available for that to make it accessible, have a camera card. Google do have one. I think that picture of the card there may be the Google version of this, which is the video development kit, but it's like getting hold of rocking horse shit, I'm afraid, because they only made so many of those to kind of see the market. Hopefully they'll come and produce some more of those, but that will literally sit on top of a Raspberry Pi Zero, you plug your camera into it, and then it then takes its camera output into the camera input of the Raspberry Pi and leverages that. So anything else that's possible when the microcontroller market is something called OpenMVcam. Anyone heard of that? Okay. This is a great little project. It's used a lot by 3D drones, for example, and DIY racing cars, if anyone's heard of that, where people try and build a little robot that follows a track that's been drawn on a warehouse floor and have these competitions. You're not allowed to aid it or drive it. You can start and see how quickly it gets around if it does indeed get round at all. Now, in order to do that, you need to do some basic machine vision. So the OpenMVcam is quite interesting because all it is is a PCB that's got a microcontroller on it. In this case, the current version is an ARM Cortex, it's an ST ARM Cortex F7, it's about 200 MHz, which has DSP extensions to make things a bit faster. It's got quite a bit of memory and flash in it. Also, there's a JPEG accelerator in some of the higher-end versions of it, which I think is included with the OpenMVcam. It's got a camera built into the top, which is just like one of these OVR-type cameras, which are very low-cost, although the OVR cameras that you buy vary in quality dramatically. There's a lot of work in selecting good providers of these cameras from Chinese manufacturers directly, and they put decent lenses on the top as well, because if you haven't got a decent lens quite often, the results you get are going to be really bad to start with. It's kind of rubbish in, rubbish out. It also accepts somewhat precariously some kind of daughter board, so if you want to add Wi-Fi to it, etc., or a motor controller board, people make some little boards that strap on to this. One of the best things about this is they've got their own little IDE, which is really useful, so you can see the video going on, and you can apply processes, and it will draw nice little graphs for you. For example, if you're detecting different colours, looking for blobs, that kind of thing, then you can see a kind of graphical representation of how well that's doing. That is a Python-based IDE as well. They're actually running MicroPython on the M7. The stuff that you write, you can actually control from Python, which I think is a fantastic idea, and we're going to see more and more of that. When it comes to things like machine vision, you need good high-level abstractions to make it accessible. So in a microcontroller world, these are two very good examples that are breaking into this area. There are more. And then one other area that you can use, which is perhaps even more efficient in some cases, is using an FPGA to do some of the machine vision. Now, machine vision tasks vary dramatically. I'm currently working with an organisation in Guildford. I'm redesigning their automatic number plate recognition systems, which are these mobile things that go into car parks and that kind of thing in surveillance systems. The algorithms for doing that are actually fairly simple, but it just needs to operate very quickly. So they translate very well into veralog, for example. I'm actually using some lattice chips to do this work. I'm using the lattice ECP-5. You can actually go down as far as what's called a nice 40 Ultra Plus, which is a little 48 QFN. It's kind of a $5 chip that has 5,000 logic gates in it. And you can run on what's called a binarised neural network on that, which is a very coarse, low-level representation so that you could possibly maybe track a face. You couldn't recognise a face, but you could track a face in a camera-type application. It's a bit like the ECP-5. We'll go up to kind of 85,000. And it has a lot more DSP units inside on modules inside that you can use to do your numerical calculations. Whether that be for a kind of convolutional neural network or whether it's just something like a face recognition, face detection or window tracking type application. And I'm finding that these produce very good, very flexible results. This is one area that has a lot of moving targets. There's so much development here. You need to be able to change quite quickly. As better algorithms come out, you need to be able to apply them. An FPGA is really good for that. However, it does take you quite a lot of work to actually get these optimised in ways that can then be repurposed. FPGA design is not quite as easy as software design, in that sense. So really, I think we're at a perfect storm for machine learning. We've got very low-cost cameras that are available, directly from China. Literally a few dollars. I remember when cameras used to be $500 a shot even for the simplest low-resolution one. We've got GPUs, NPUs and FPGAs to throw processing power at these. You've got good software libraries out there. Just take a look at OpenCL. That has most of the common machine image processing type algorithms built into it, as well as some simple vision stuff such as convolutional neural networks, et cetera. You've got things like CUDA and DNN from NVIDIA, which are the libraries that help you accelerate your processing for neural networks and parallel processing. You've got ARM neural network software developer kit, which is part of their Seamsys software set now, making that very easy. Again, they're adding in more and more hardware that will automatically be supported by that software. Something to come that isn't really out there a lot at the moment. Something I'm spending some time working on are very large libraries for machine learning. That's like machine learning and machine vision. I want to see more and more of that. More people will get involved in doing that. All in all, it's never been a better time to get into machine learning and machine vision. It's a really interesting time right now, and you'll be able to add this onto your robots, IoT devices, white goods, et cetera. It's going to become very, very commonplace.