 So, before I start, I'm just, before I get into details of MicroPython and executing it, I'm going to tell a story. So in 2006, I fell in love, so over 10 years ago, with Python. And when you fall in love, you just want to spend all your time with your loved one. And we did. We did command line scripting, web applications, graphic applications, networking, coroutines. We spent all our time together. But then on one fateful day, something horrible happened, conky. So who here knows what conky is or ever used it or heard of it? Raise your hand if you have. Good. It makes it even more exciting. So conky is kind of a silly piece of software, but very useful. And this is conky, what it looks like. What it lets you do is just monitor the resources on your computer. It's a very geeky thing to do, but you can see CPU, RAM, disk user, stuff like that. And this is my conky setup that I have on my computer. So I could do all this stuff with Python. I was very happy with it. And then when I encountered conky, I went to configure and customizing it. And as these 300 built-in widgets for CPU, RAMs, all that stuff. But then you always have something custom that's relating to your setup. And for me, I had these five things I wanted to watch for Debian packages or have any updates, my domain controller password, stuff like that. So the way conky works is if you want to extend it, and they have a simple hook-in where you just, in your config file, put some scripts, and then you specify how often you want it to be called. So it's like, cool. So I created five scripts in Python, of course, the love of my life. And I wanted it to update every second. So I was like, OK. So conky is going to spawn these processes every second. And I have five of these scripts. So I just did a calculation. I was like, oh, crap. It's going to run this 400,000 times every day. So I said, hey, maybe that's OK. Let's see what happens. So I did. And initially, I just made a very simple Python hello world and kicked it off and said, OK, what's this going to do? And conky itself is implemented in a very efficient fashion, because it would be really lame if your resource manager put a lot of load on your system. So when I did this, my CPU usage shot up to 15%. But then instead of Python scripts, when I just wrote hello world and bash, I'd use a lot less CPU. And so I ended up just not using Python for this and slapping together some batch scripts and not doing it every second, doing it every 10 seconds and whatnot. But over time, over the years, I've found stuff that Python hasn't worked out for me. And it's always frustrated me, because I hate programming in bash. I mean, if statements and for loops in bash are very ugly and painful to work with. So it frustrates me if I can't use Python. And these are two other examples, separate to the conky example, where there's issues sometimes, let's say. And maybe you want to measure that performance. So one example is in your text editor, you can hook in a lint checker whenever you save. And I use them. And whenever I save my source code, it runs Flake 8 on it. And if you just check this on a very simple file, like a single line print hello world, it takes over 300 milliseconds. Now, from a usability perspective, anything beyond 100 milliseconds, people will perceive. And it's very frustrating when you're editing code and then you save. And then you feel this kind of lag as you're saving the file. But it's very useful to have the Pepe checking syntax checking. So I accept it. And what I'm going to kind of present in this talk is an alternative to see Python that has a much faster startup time, which is using MicroPython, but we'll see that later. Another use case is bash completion. So whenever you're in command line doing tap, tap, tap, you're typing. And you really want a responsive experience. So you notice anything takes more than 100 milliseconds. You start thinking what's going on. And when I first started working with Pip, I was like, oh, cool. They have bash completion for Pip, so I set it up. And when I did Pip, tap, tap, tap, sometimes it can take up to a second to kind of do the bash completion. Because Pip is a little heavy to import and execute. And this is very frustrating. When I first started using it, I thought maybe there's something wrong with my terminal as a sage connection, maybe the network stuck or whatnot. And so in the end, I just disabled and stopped using it. Because when something gets so slow in terms of usability, then it impacts the functionality of it. So we're going to try and look at two things to measure in script execution. So we focused on, let's say you have a specific script to program, and you're going to execute it. It's going to start and it's going to end. And you want these two simple things. You want to know what was the elapsed time, and also the clock cycles. Now, why the clock cycles? And it's funny. End user perspective, all they care about is the elapsed time. But when I started running these benchmarks on this laptop, which is five years old, and then my other desktop, which is a newer computer, on the desktop was a lot faster, which is natural. You see pews, and that can be kind of frustrating when you're comparing results. But what's cool is if you compare the clock cycles, it'll be the same. Because if you have the same OS, the same software, generally it's going to go through the same number of instructions to execute. So clock cycles can be useful to kind of more fairly compare, and elapsed time is the real kind of experience of it. And then when you're looking at the case of concave, maybe you're planning to actually execute some sort of crazy job that every second is going to do some polling. And you want to kind of measure how heavy is this going to be on the CPU. So clock cycles will tell you that. So why measure one when you can measure five? So what I said is since I'm doing these benchmarks, and I know already that Bash Hello World can be faster than, say, Python, what I did is I said, let's throw into the mix five interpreted languages. So we have Python, Bash, Perl, Lua, and Ock. All of these languages are interpreted, like Python. And they have variables, and functions, and for loops. And so I said, OK, this is a fair comparison. Because it would be unfair to compare it to C or Go that is compiled. And so I created these five files. And essentially, they're all the same implementation adding one plus one. So all of them, if you execute them in their language, they should output two. And then we're going to run all of this through the performance and then see, how does Python compare? Is it unfair to expect a very fast performance time from an interpreted language? And the first one, add.py, will be used for the Python 3.5 test, as well as MicroPython. So the same script will run it through both interpreters and see the difference in performance. So how do we measure execution time? The easiest way is to just say time. And this is available in Bash on Unix systems, like Linux and Mac. And now, this time is actually built into Bash. And it's different than a time binary that you'll find on your computer. So it's just important to know that distinction, because some of the command line options are different when you look at the man page for a time and it's not going to be this time, it's a different time. But generally, it's a very easy way of just measuring, so you say time, but the command. And the only number you really care about is the first number, that's real, under real. And this shows that when I just run a script that prints 1 plus 1 in Python, it takes on this machine 48 milliseconds. And now, this is more like a thorough approach. And this is a very powerful command. It's very useful. So this is a Linux performance tool that comes on Linux machines. And you can use your package manager to install it. And the perf command has a bunch of subcommands. And this is stat, the stat subcommand. And I've just highlighted and read some things that are worth noting. One is the dash R10. So any time you do benchmarking, you don't want to just take one sample, right? Because it's not really going to be indicative. So what's nice about this in the same command, unlike time, you can say, hey, do 10 shots, a sample of 10, and give me the aggregated results. So here it's saying it's running for 10. Another thing to note is the context switches. So if you're running a benchmark, or you're trying to measure the performance of a piece of software, and you're running a lot of other programs on that computer, it's going to give you inconsistent results. Because the CPU is switching, context switching between those other processes. So when you get zero context switches, it means this is getting. You ran the benchmark on a machine that didn't have a lot of other stuff going on. Then now we can get the cycles number. So when you just, in Python, say print 1 plus 1, it goes through 50 million instructions CPU cycles, which is surprising. I don't know much about computer architecture and CPUs, but that's how much it goes through. Then the last thing is the elapsed time. And so you can see that this took 17.8 milliseconds. And the percentage sign, plus or minus, is the variance. So you just want to keep an eye out on this, because if it's plus or one percentage points, that's fine. But if you have 100% variance or 200% variance, it means that you're getting a lot of inconsistency between those different samples, and then you should step back and be like, well, I shouldn't trust this number. So now I jump back to my love affair with Python. And I bumped into this little beauty earlier this year, and I purchased this on January 15th, 10 years after first discovering Python. And it's a popular Internet of Things hardware, and I got it just to tinker with it. It's a microcontroller running at 80 megahertz. And amazingly, it can run Python through this implementation called MicroPython, which runs on bare metal. So when I got it, I just got it to play with Internet of Things, but then I started playing around more with MicroPython, and I discovered that you can install it on servers, on desktops. It's not only for microcontrollers, right? So then I started to compare the performance of MicroPython against regular Python, and you'll see the results. But just a brief on MicroPython, it's a lean implementation of Python 3. They had to be much more cutthroat about the way they implemented, because it's targeted for microcontrollers, which have limited CPU and RAM and the rest of it. And so what they've done is they've taken a selection of core libraries and implemented them, and they have this naming convention of putting the letter U like micro in front of them. So you have some of the ones that highlight is Usocket. It's just like the Python standard library socket, and that lets you open HTTP connections so you can do HTTP client server and TCP and UDP. UOS, that lets you spawn processes, retrieve output, and JSON, regular expressions, time is here. So it's got a nice mix of quite useful packages taken from the core library. So let's look at the numbers. So when we run those scripts and just add two numbers, one plus one, right? This is a single text file with a single line of code that's interpreted in a number of different languages. We see Python is like way out there. It's around 18, 15 times slower than all of these other interpreted language, Lua, Bash, Perlin, Ock, right? But what's really interesting is MicroPython beats them all, and it executes so fast just adding these two numbers, right? And I was really surprised that you can do this in under a millisecond, because once you get to that level, just the process of spawning a process, opening the text file, reading the contents, compiling it, right? And it's amazing that it does it in under one millisecond. So I really encourage people to explore MicroPython outside of microcontrollers, even on their desktops and their servers. It could be useful in a whole array of stuff. So I use that same tools for both set of graphs. And this is in terms of the CPU cycles. And so we see that Python needed 46 million CPU cycles, but MicroPython only needed one. And so it's a much kind of more lean implementation. And so now, you know, it's about like, I guess, 30 or so times less, which means that I could probably use this now for a conky, and I wouldn't get this high CPU usage. So cool, so the next part of the presentation is just looking at a real life implementation of using MicroPython outside of the microcontroller to do something useful. So what I've done is I've done PIP bash completion using MicroPython. And what's cool about that is PIP already has bash completion built in. So we can compare the two in terms of performance and in terms of features. And besides speed, there's also stuff that you can do in MicroPython that would be frustrating to do in other languages that have better startup, that have equivalent startup times. Like bash, if you had to go and connect to a web server and fetch a listing of the wheel packages, which is what we'll be doing here, you know, it'll be frustrating to do in bash. So cool, so let's see how the built in PIP performs. And if we just import PIP, because you know, when you do the auto completion press tab, it imports it and then calls it, just importing the package. Like I haven't called the auto completion, you know, on my laptop takes one second. And a lot of the machines that I've seen, it's heavy. And so you're really gonna feel that, you know, when you press tab. The general, you know, benchmark is you want it to be under 100 milliseconds usability. Generally anything under 50 milliseconds, people can't perceive, you know, they can't perceive more than 24, 25 frames per second. But stuff more than 100 milliseconds or 50 milliseconds, you're gonna feel that kind of jittery lag. And anything in the terminal typing will be like that. The other thing is the built in PIP completion, it only completes the subcommands. So if you do PIP it says install whatever, but it won't complete the names of packages or stuff like that. So we're gonna take it to the next level and we will also complete all of them. So now the code that I'm showing and everything, it's more kind of like a demonstration. It's not the perfect kind of implementation. So for the demonstration, I have a local PyPyMirror that I've created and what I've done is I've just grabbed a whole bunch of wheels, over 400 wheels, and just have them served on a static web server on Nginx. And to do that, you just set these environmental variables so that PIP looks in that location. I'm doing it as local host, but you can do it on any web server in your environment. And so we'll auto-complete the subcommands and the implementation and we'll also, every time you press tab, it will connect to HTTP, check on Nginx, what's the set of wheels. So the moment a new wheel is published or put on the web server and you press tab, it will auto-complete, which is nice. Because sometimes there's a trick when auto-completion is slow, people cache their results, but it's frustrating because all the time you have to log out and back into the terminal, this will be live for each call. It will also complete not just the package name, the versions as well, so you'll get all of that information in the terminal when you do tab completion. So I won't go into too much detail about this, but there's different ways of doing bash completion. Usually if you wanted to run live each time, then you have a bash function and use this complete in that fashion. I've set the command option file names because when we say pip install, click equals equals 1.0, equals is actually a special character and bash and it needs to be escaped. You'll see that. So this is how you say, allow special character as file names. And then you'll see that all the work is being done by this pip comp guy. And we'll look at his source code and he receives one argument, comp c-word. And comp c-word, if you're completing the first argument, then it will say one. If you're completing the second argument, then it will say two. And we'll need to keep track of that because if you do pip space tab, then we want to say install freeze, whatever. If you do pip install space and then tab, we want to give the package names. So yeah. So let's look at the code. So these are all the packages that we'll be using in the implementation. And at the top, that just says, hey, use the micropython interpreter instead of Python. Urequests is just like very similar to requests that we all know and love, except it's implemented to work with you sockets and some more kind of lean implementation. And then OS and Ansys. So this function does the bulk of the work. And the first line will go and connect using requests to the URL provided and fetch the HTML. The second line will go, parse all the HTML and get the list of wheel files that are hosted on the web server. And then we just loop through the wheels and parse by dash and split, sorry, by dash to show the name and version. The wheel file format's really cool because in the file name, it's encoded the name and the version so you can get all that information easily. So, and then that's the whole function. The lower bit is just the main and all he does is he checks if you gave argument one, then this is the list of subcommands that we'll complete. And if you gave argument two, then we don't want to hard code the URL so we'll get what the user has specified in his environmental variables by getting pip find links. We'll call get packages and then we'll print out this new line delimited. So to do batch completion, you just have to spit out a list of values that are spaced delimited, new lines delimited, whatnot. And so let's compare the performance. So we know that just importing pip for auto completion can take up to 100, 1000 milliseconds. So how does this fair? So we see that when he's just simply spitting out the commands, he does in 3.9 milliseconds, which is way under 100. And when he connects to the package names, to fetch the package names, he opens HTTP connection, gets that, parts the HTML and prints it out all in 11 milliseconds, which is fantastic performance, way more than what we need. So let's do a live demo. Okay, so the first thing that I will do is just set up a clean environment. So this shouldn't have anything installed on it. And we can start pipping. So let's just look at pip com for a second. So if I say one, he should tell me the commands, which he does. If I say two, he should connect to my web server. And we can see the engine ex web server here. And it just has this bunch of wheel files here and it's just opening this URL and fetching and then spitting out the package names equals the rest of it. Now let's just check how many packages we have in the demo, 409 wheels. And let's start doing some auto completion. So if we do pip, well, let's do pip tab. So this is working now, I can put the commands. I can say pip s gives me options, install. Now I can just do tab and it shows me 409 packages. So if you, I can go and say, okay, show me all the packages between the C and you'll see that it's showing a click and coloroma you do. And now, so I can go and say, okay, I wanna install click. Let's say version 1.0, do tab. So you see the equal signs escaped correctly and let's install coloroma too. We can do pip freeze to see what's installed and why not upgrade click to 6.0. And if we see it, pip freeze, we can see it's been upgraded to 6.0. So that's it, just, whoops. Yes. That's the whole presentation, thank you. First of all, thank you very much for your talk. It's actually a really interesting approach to use it for completions. But I'm wondering to which degree this micro Python is compatible or incompatible to regular Python and is there a realistical chance to port larger existing programs from Python into micro Python? Thanks for the questions, very good question. So it is actually challenging like if you just took a big code base like pip, for example, you know, or flake, you know, and you're trying so importing it. It's just a lot of stuff's not gonna work. But, you know, what would be really great, and Armin talked about this, you know, in the keynote a few days ago, is, you know, let's say you have the specification of Python. It would be really nice if you could go and then have a formalized subset of Python called the micro Python specification. And then people could come in and be like, hey, I'm going to write flake 8 so that it supports both, you know, the full-fledged Python and micro Python. And that's happened in the past with Jython, for example. You know, Django for a long time supported Jython. And when you'd run Django, it would make sure that this code would work equally well in Python as well as Jython. Even though the Jython join time environment is so different. So I think right now at this stage there will be issues, but I think there's just a lot of opportunity and potential. And micro Python is just, it's really stable. And it's used, so I think there's a lot of potential and for an opportunity for it, but yeah. Thank you, that was pretty informative. I just wanted to add that apparently Linux has a long tradition for solving these problems like you had with Kanki. It's like you just spun a long running process because it generally boils down to the startup time of Python interpreter because this is a little misleading that you say all the way the performance of Python because mainly this is bootstrapping time. If you put nothing instead of print, then you would approximately get the same time because this is all about loading libraries, et cetera. But yeah, like in case of awesome window manager when you are writing scripts that want to run fast, then mainly you just spun long running process and then use just some type of buffering. Like with Kanki you could just dump your data into file then cutting in Kanki and then you would get some kind of real time because that's what you want, yeah? Because 18 milliseconds is not so much, really. Yeah, you know, what is 18 milliseconds, right? Like when I first saw it, I thought it's nothing. But I think it's always good to look around at what's there. So when you can see an implementation of a subset of Python running it 30 times more, maybe there's these opportunities that can arise from it. So think of the presentation more as an eye-opener to possibilities, but what you said is very important and it's true. One could argue that the architecture of Kanki is messy. You're just gonna spawn a process every time, right? Even if it's fast, that can feel very kind of sloppy, right? Maybe I should have a long running process and then do some inter-process communication, right? Like D-Bus or something like that. But one of the reasons why calling other processes and just getting the output is such a popular method of hook-in is because it's so easy to implement and it's very widespread, right? Like Kanki uses it, Vim uses it, Emacs uses it. Like when Vim, I'm using Syntastic, when I press save, each time it's calling Flake, right? Even though maybe someone could have made a Flake server and then it could kind of call it more efficiently, right? The same thing is true of a lot of Jenkins as an example. Like when it wants to hook into Python, a lot of times it's spawning these processes. So it's kind of a very popular way of hooking in, executing programs and just capturing their output. But as you said, it's not necessarily a fair assessment of Python and I wouldn't put it as a criticism of Python because I've never had, at least in the systems I create or production environments, this sort of issue, if I have a startup issue, I create a long running process and then do some inter-process communication or whatnot like you said. But yeah, thank you. Hi, thanks for the presentation. I don't know anything about MicroPython apart from what you just showed us, but I was wondering if there's any split between MicroPython 2 and 3 or it's only 2 or it's only 3? Yeah, good question. So MicroPython, isn't that old? So when they created it, they said, we're not gonna support Python 2, so the whole thing is Python 3 from the start. And so, oh yeah, so that's a very important thing actually because you don't wanna go back at this stage, right? So it's just all completely, cleanly implemented in Python 3 and it's the same exact syntax as Python 3. There's some differences, some edge cases, but you know you have all the same data structures, functions, classes and the rest of it. The more I worked with it, the more I didn't encounter any surprises. Thank you for your talk. Quick question, do you think MicroPython will have an impact on Python 3 itself, main C Python? So you say the last bit again and just cut off. Do you think MicroPython, the good things in MicroPython will have an impact on the C Python implementation? Well, I hope so because before January 17th, if someone came to me and said, hey, Python should start in one millisecond, I'd be like buzz off, you know what I mean? 18 milliseconds is great. It's such a powerful language, right? Like in a way you could argue Bash, you know, or some of those other languages, they're very limited, you know, in a way they're not as powerful as Python. But I think now that MicroPython has been implemented and there's such a growth in Internet of Things and so much contribution in MicroPython and stability, I think it's gonna make us ask that question again, you know, that Armin again raised about like, you know, PyPy, does it have to follow the C Python thing? You know, maybe we can have this specification, maybe we can have a Python specification and then a subset, you know, or like I said, maybe some of these tricks that MicroPython has C Python could use, but I'm not sure about that because keep in mind MicroPython, there's drawbacks to it. There's no multi-threading, no multi-process and that's how I was able to get away with this kind of oversimplified look, but it's amazing that you can get that much functionality with that few CPU cycles, so I'm surprised at it. Hi, thank you for the great talk. Regarding the problems with the incompatibility with the code, is this in the syntax level or is the lack of the standard library? What kind of incompatibilities do you encounter? So good question, for me when I implemented this, so one good example is requests, right? Requests is such a popular library, right? Then everyone's like, ah, do I have to directly talk to Socket because they give you use Socket, but they don't give you the Python three libraries that you have for the HTTP client, right? So imagine like writing low-level Socket code connected to an HTTP app server, it's a pain, right? So, you know, the MicroPython guys made you requests and they made you JSON, regular expressions and the rest of it, so I think a lot of the popular libraries are there and popular functionality, but it's not necessarily that the Python code that you write, that the list is different or the string is different or the function is different because the code that you saw looks exactly like regular Python, which it is, it's just, you know, if you take any of these packages and you try and import, I don't know, collections, you know, or even RE, as a regular expression, you know, you don't have the full RE and so, you know, they've made you RE, which has a lot of the functionality but it's limited, so you'd have to change a lot of your import paths and you'd have to be thoughtful, which I think, I think if, you know, the code isn't so crazy and complicated, then it's definitely worth it because, like you saw the PIP autocompletion, it's not like 10,000 lines of code, you know, I think it was just 15 or 20 lines of code. So, you know, if you have this kind of small thing, if you're making something new, I think it's definitely worth trying out. All right, thank you very much, Marvin, for the talk and the Q&A. Thank you.