 Good morning, everyone. Before anything, I would like to thank Mario. I think you're doing an amazing job in Laszlo and all the rest of the team of the Panhandinium organization. You guys are doing an amazing job. We had great time last year. And this is really important. So I should also thank Grunberg, of course, and all of you for being here on time on a Saturday morning at 9 AM, because the room is packed. And I wasn't expecting that. So yes, as Mario said, so I'm the founder and CEO of Quantstack, which started in 2016. I'm a Jupiter contributor. And this may be one of the reasons. If you know me already, this probably is the reason why you know me. And I'm a steering committee member for the project. I'm also a member of the board of directors of non-focus. So if you have any questions about non-focus and if you would like to know more about the foundation, please come talk to me after the talk. You can find me on Twitter and GitHub with the same handle, which is Sylvain Corley. And so the title of my talk was Jupiter being about the redevelopment loop and humans in the redevelopment loop. And so what I mean by that is that scientists and engineers don't only use programming languages to build systems or build a software that they know in advance, where they know in advance what they want to build. They have interactive workflows where to explore a problem and reason about it, they load some data, they run some code, they look at the visualization and write some more code and loop into the saturation process where they try to figure out something. They are not in a software development process. They are using code and they are using the computational resources to figure out something. So what they're looking after is not a number necessarily. It's just insight. And where most of the progress in software and science is made from incrementalism, any obstacle into that loop, any obstacle to fast saturation hinders progress. But the reality is different. And I'd like to quote Fernando Perez for, I don't know if you guys know Fernando, but he's the founder of Jupiter and I Python. And in an earlier talk, he said, my advisor had a heavily customized oak-said bash workflow to manage job submissions and post-processing of C code for super coding runs. So I used her scripts to run my jobs and on top of that has had in my own layer of Perl plus a hefty amount of new plot, ideal, and Mathematica. So if he was to try today to reproduce that workflow with, by trying to install the versions that worked at the time, he would probably never succeed. So this is a real obstacle to reproducibility and to embody new contributors to software or to any project of significance where you would want to have more than one person, right? Even for himself 10 years later. So what is Jupiter? It's a tool that was built, or it's a collection of tools that were meant to solve that issue, which is to provide a consistent set of tools to improve the workflows of engineers and scientists from the exploratory phase of their work to the communication of their results. So including all the intermediary steps, visualization, loading data, dealing with large computational resources, the complexity of the problems that we're dealing with, and the size of the data and the size of the computational resource are just growing, but our brains are not, right? So humans are not getting better at any of this. We are not better than we were 30 years ago, right? Tools are getting better, and problems are getting more complex. And within the Jupiter ecosystem, the one star application from the start was the Jupiter notebook. So what is it? I just would like to make a quick poll in the audience. Who doesn't know Jupiter? And who doesn't know what the Jupiter notebook is? Okay, there are a few hands raised, but I don't believe you because I know some of you over here. So, well, it's an interactive browser-based computing environment, and I'm not saying an interactive development environment. It's really an interactive computing environment that's meant for exploratory data science, ML, visualization, stats, and it's also reproducible document format where you can put code, narrative text, equations, images, visualization, and this works in a language agnostic fashion with over 50 programming languages. So everything's open source, obviously, and it's PSD licensed just like most of the patent stack. So here's a screenshot of the classic notebook, the main application that made Jupiter what it is today and this widely-adapted software. Or you can see all of the components I listed earlier. So equations, markdown, visualizations. But more than just notebooks, actually, even in the very first classic notebook web app, we already had other components. We had a file browser, a text editor, a console so that you could actually interact with the terminal of the machine where you were executing the code, right? And what people were telling us about this is that they would really like to have all of these components into one single web application and one page application that would provide the same sort of experience as a desktop app, right? And so this is a reason why the Jupiter team got together and started rebooting all of the front-end work into a new project called Jupiter Lab which is this new front-end to Jupiter that talks to exactly the same server and speaks the same protocol as a classic notebook. And in that screenshot, you can already recognize some notebooks but you see that you have this doc area and other components that you usually find in a desktop application. Another area where Jupiter has shined is the Jupiter widgets. So Jupiter widgets are special objects that when displayed in a Jupiter notebook have a visual presentation and they have been used and expanded upon to make data visualization and to allow people to build grease in the Jupiter notebook. So that screencast here is a recreation of Mike Bostock's worth of nation visualization where the x-axis is the income per capita and the y-axis is the life expectancy and you can animate that plot for all the countries in the world over time and see the path of individual countries, right? I just also wanted to mention that none of this is actually Python specific. As I said earlier, Jupiter was really meant as a language agnostic platform and in this screencast here, I am showing how using X leaflet which is a C++ back end to the IPa leaflet Jupiter widgets library and the C++ Jupiter kernel. We were able to do exactly the same sort of things with the C++ programming language. And so this actually gives me sort of a transition to that. One of the key ideas for Jupiter was that a specification matters a great deal more than any implementation. So before actually writing the software, every single components of the ecosystem was built as well-specified communication protocols for kernels, widgets, MIME types, file formats, serialization schemas for multiple data types. And actually all of these components together sort of turn Jupiter into the HTTP of scientific computing for exploratory analysis data visualization and it's really language agnostic. So today, if you are coming up with a new programming language rather than having to build your own rapport, it's probably a better idea to actually implement one of these protocols and benefit from all the rest of the Jupiter ecosystem and the frontends and the notebook format and so that you can share easily code for your new language on GitHub, right? The fact that we have a specified format for the notebook files allowed GitHub to actually render these notebooks online, right? And it's not just a format defined by the implementation. It's truly something that has, you know, is checked and specified. And so I mentioned earlier the C++ kernel and I'm going to do a quick demo of that to really show you that, well, Jupiter is truly language agnostic. So here I'm launching JupyterLab. A quick poll here. Has anyone never seen JupyterLab before? Okay, that's about half of the crowd. I was not expecting so many people. So JupyterLab is like, as I said earlier, the new frontend to Jupiter. It's a web application, you know, or lives in the browser just like the classic notebook, except that as you see here, you can already recognize some components that you would not see in a classic notebook, such as a file explorer on the side where you can navigate the file system, open images, drag them to the main area and arrange the layout of your application. You have multiple activities that you can launch, such as a terminal and notebooks, obviously, right? But the one thing that I wanted to sort of emphasize here is that I don't just have the Python 3 kernel available for a notebook here. I have a bunch of other weird icons here that may be scary to you Python folks, but the C++ logo, right? So let's launch a C++ notebook, right? And let's start writing some C++ lives in a tote. Let's do a hash include, I will stream. And hello world, right? All right. So that works. I can do interactive C++ in the Jupyter notebook, right? So this Jupyter kernel was actually built upon an interpreter for C++ that comes out from CERN. And if any of you know actually what's going on at CERN, it's a research institution over in Switzerland where they consider C++ as a scripting language, and which I think is great. And so we put together a Jupyter kernel for it based on the C++ implementation of the Jupyter protocol. And there is no Python involved whatsoever, which is great for Python conference. But what I mean here is that by just taking one component of the ecosystem, we could replace it completely just by having it implement the communication protocol that is the same, that is the one implemented by PyKernel, right? Gonna show a bit more of the C++ kernel. Here, I'm gonna use this new library called extensor, which is too long in the read numpy for C++ and use the rich display mechanism of Jupyter to actually show some array, that end of array that I can reshape in place. Some of you may know of Simengin or SimPy. So Simengin is a C++ engine that powers a lot of the SimPy symbolic programming library. And so you can actually directly use Simengin in the C++ Jupyter notebook and have latex rendered expressions, expended and whatnot. Or you can start combining extensor and Simengin and do crazy symbolic programming by, you know, multiplying and doing some matrix operations and symbolic expressions. So just showing that pretty much anything you could do in the Jupyter notebook with a Python kernel, you can, is actually also the case for other programming languages as soon as they implement backends for Jupyter widgets, which might I'm rendering and whatnot. I'm gonna get back to that. So building upon Jupyter protocols. The other area in which I was very involved since the very beginning is the Jupyter interactive widgets. So as I said earlier, Jupyter widgets are objects that are meant, that are displayed and replaced by a visual components when displayed in the notebook. So basically they're trying to solve the following issue, right? Whenever I type some code like 12 by 12 in the Jupyter notebook, what happens is that I actually edit the document model, send an execute request using the Jupyter protocol to the kernel of the web circuits. The kernel like makes the computation returns, then computes the output and sends me a message over this protocol and through the web circuits to the notebook frontend that displays it to me, right? So there is a lot going on every time I type a bit of code here, right? The thing here is if I'm like just a high school student looking at the square function and I'm trying to figure out the value of the square function for multiple inputs, I am the bottleneck in that loop, right? So in this loop that I'm seeing here is the redevelopment loop. The same thing that you're using when you're using the Python interpreter, right? And I'm the bottleneck. So I could be smarter and actually use a function to have less to type, but really what widgets were meant for is to enable you to actually fasten that loop by scrolling through the data with a visual and interactive components. So the first use case for widgets was this magics called magic function called interact, which actually uses the introspection features of the Python programming language to figure out what type of widget to instantiate for each one of the inputs of the function. And now I get some idea of how the multiplication function works for multiple types and values in Python. So, but in order to enable that, Jupyter widgets were, so we put together the Jupyter widgets library, which are the special objects that when displayed in the Jupyter notebook are replaced with this visual presentation that you can interact with. So if you display it twice, obviously you are displaying the same object and they keep, they are synchronized. You can set the value from the Python side, you can change them over attributes and set the number of visible digits. You can link widgets together. So what's really happening here is whenever I create one of these widgets subject, a frontend components gets instantially that he's going to be synced, always synced with my Python object. And every time I display this, a view component is actually created that is always going to be reflecting the same data. So more than a collection of controls, Jupyter widgets are actually a framework upon which you can build and build pretty much anything. So I should probably show you this quick example of how, you know, so I could like copy paste this example from the D3JS documentation, which is a small application that generates some points randomly and where you can see the vulnerability solution of these points plus the vulnerability of the current position of the mouse, right? So this is a really nice display of what D3JS can do, so this great JavaScript visualization library. So I essentially copy pasted that code into a Jupyter notebook and made a widget out of it, maybe a hundred nights of code. And now I have this thing, which is exactly the same application with some CSS differences, right? That is actually synchronized with the data that comes from your kernel, right? And so this is the one thing that made Jupyter widgets so popular is that if there is a visualization library or any sort of components of the JavaScript ecosystem that you want to include into your Jupyter notebook and make available to people who don't know JavaScript and just want to feed data from the Python kernel in it, you can, right? And there are dozens of these Jupyter widget libraries that are there, two D plotting libraries, three D plotting molecular visualization tools, mapping and whatnot. I'm not going to go over all of them, but I just wanted to mention that. So back to Python. What did Jupyter change for the Python community? And one thing that I think it changed is that up until maybe 10 years ago, most of the Python developers were using classic ideas or developer tools. And their purpose was to do things like web programming, like classical software development workflows. They were doing software development with Python, right? And today, a large proportion of the Python users actually use it interactively. They actually make use of the interpreter by just giving it new code to run all the time, right? Using one of these functions such as Jupyter. So they are not spending the time in an IDE and they just make use of the wrapper. So my point here is that now the wrapper matters. So for a long time, the focus of the Python community and the C Python community seems to me was to improve the language for good reasons. I mean, we all love that language. It's a great language to learn and there are so many things and so many places where we could improve it, but they did not put so much importance into the actual interpreter. The one that is shipped with core Python is very minimal. It can be, it's slightly better if you have, if you're in Linux and you have rely installed, but for, in my opinion, that's sort of my point here. The wrapper could become a more central project. So for example, by improving C Python APIs to facilitate building frontend such as IPyton and Jupyter. Like the Jupyter team really had to jump through hoops to make all of what you've seen possible by overriding display hooks, getting into the AST of every single cell that we run so that we detect whether an expression was returned or not. We actually do crazy stuff to do everything that you already use in Jupyter and consider as like normal or something you expect from an interactive computing environment. But it's not, it doesn't come out of the box with the Python programming language. So, and another crazy thing with we to actually upstream some of Jupyter, right? What could, what is in Jupyter that could become part of C Python? One first candidate could be to have something along the lines of the rich MIME type display. Well, there is the built-in wrapper function in Python. Would it make sense for Python to have a MIME bundle wrapper just like the one that is used, function or built-in function just like the one that is used by Jupyter where we could have multiple MIME bundle representations of an object, right? Other candidate would be the Jupyter magics, right? So magics are commands for Jupyter kernels that are not part of the language. For example, you can use magics to time the execution of a cell or to load a file into a cell and do all kinds of other things such as annotating code and whatnot. They are like over a hundred magics. And so what I think is that when you are using a programming language interactively like this, you need another sort of meta level of interacting with the environment that is not in the language and having that sort of higher level model magic syntax where you can do other things such as access terminal commands and shell commands or change the current working directory access to files without having to do this in Python is invaluable and very popular in the Python ecosystem. So yeah, maybe C Python and the wrapper from the core Python should have something along these lines. Another example is the asynchronous wrapper. So currently in Python, if you try to make use of an asynchronous API using the await keyword, you're probably going to have some invalid syntax because you can't make use of asynchronous functions in the wrapper, right? Although, if you're trying to debug some asynchronous application, this is something you naturally would want to do, right? So we enabled that in IPython and Jupyter kernels recently. And I think this is something that would be extremely valuable for Python developers. So, and that's it for the first part of my presentation. Now I wanted to get to another subject because in the very beginning, I mentioned that Jupyter was really meant to improve the workflows of researchers from the exploratory phase of their work to the communications of their work, right? And so, well, they have already proved very useful for that thanks to the binder project to the ability to have these documents where you can work someone through the code, right? So they have already proven invaluable to provide a narrative, I don't think the code, and to work a reader through the logic of an algorithm. But it's not always ideal, right? For example, if you're trying to address a non-technical reader, they may be confused by code source or just having to run all the code to see the widgets and the visualization that are part of the notebook, right? Also, sometimes the actual literate programming approach, which is to follow the order of the code for the narrative, is not what you want. For example, if you run a notebook, typically the most interesting part of the notebook would be the very end, the thing that you manage to achieve last in the code, right? And if you are communicating with your manager, you may want to put that thing at the top, right? So how do you do that? How do you use notebooks to communicate to a non-technical audience? And so someone who is not interested in the dirty details of your implementation. And then sharing notebooks comes with other challenges, such as how do we deal with the fact that notebooks' main feature is arbitrary code execution that poses a security challenge. And so I'm really glad today to announce the release of Voila. It's the name of the software. So which is a tool to transform Jupyter notebooks to standard web applications and dashboards with just one comment. So we actually announced the release a couple of days ago on Medium and on the Jupyter blog. So you may have seen it already. So besides the name that what's really great about Voila is that it's really simple to use. So it's just another viewer on notebook format. So here, like I'm showing you a screencast where there is this complex notebook where I have some rich display and widgets. If I just type Voila basics.ipinb, it's gonna launch a browser, execute the code and show me exactly the same thing. Whether we're just alive and backed by a kernel. And what, so one aspect of this is that the execution model is actually completely secure. So I'm just gonna go a bit into the dirty details here. But when we launch Voila with a notebook file name, what you get is a new tornado application. And when the client connects to that tornado application with the URL of a notebook, the backend will actually open the notebook file, read which kernel it should start for this, start a kernel, execute all of the cells and populate the outputs and then serve you the converted HTML with a bit of JavaScript that is going to connect to the kernel and enable interactivity. What's really important here is that you never have the code in the front end and you never have the ability to send arbitrary code execution requests. So that scenario for running notebooks is actually completely secure but still enables round trips and communication for widgets interaction. So another cool features of Voila is that you may not just want to actually display all of the content of the notebook. You may want to have other formats towards, for other target formats or layouts for converting a notebook. And so these layouts or these alternate front ends to Voila are called templates. And you can enable a template called FUBAR by just typing Voila dash dash template equal FUBAR the name of the notebook. So what templates are from a technical standpoint is merely a directory that is going to be in a configuration or standard configuration directory for Voila where you can overwrite pretty much anything that is part of the Voila front end, the classic Voila front end. And since all of these are based essentially HTML, Jinja templates and JavaScript, you can just install them as part of, as data files in your Python worlds. So here's a quick example where I use the grid stack template which can be installed separately from Voila. And what I get is this web application where each cell lives in a different area in my grid stack layout, which I can rearrange to have a nicer layout. Voila templates can actually make use of any modern front end. They can also essentially be certain drop areas into your blog or whatnot so that you can have live applications running on the web page. And this one is actually another Voila template called Voila Vutify, which makes use of UJS to produce really nice dashboards. So I can't really imagine corporations putting together Voila templates with their branding or people making Voila templates for publishing and making it look more like a scientific paper and whatnot. So the question is gonna come, so I'm just going to make my pitch now. So how does it compare to other dashboarding systems like dash or panel? So I would say that the strength of Voila is that it's built upon Jupyter standard protocols and formats, so it's language agnostic. It's not just Python. Voila templates and Voila dashboards work with C++ kernels, Scala kernels, R kernels and whatnot. The main entry point is a notebook document and Voila is really another way to look at it. It's just another view on the same notebook file that you already know and love. And it's also really a low-separating content from the dashboarding layout information by allowing you to have your custom templates or rearrange your layout as part of the notebook metadata. So what is coming in the Voila ecosystem? First, a Voila template for slide shows so that I can make this presentation using Voila. It was not the case today, but I hope that next time it will. We also are working on a Voila template for the JupyterLab doc area, which is that rich layout we are using within JupyterLab to rearrange tabs, and we really like to have a template for Voila making use of that layout system. We're also working on progressive rendering of notebooks in Voila so that we don't have to wait for the entire notebook to be run before we display anything to the user, but we have some progressive rendering where the cells are rendered one by one. And we also want to support the legacy Jupyter dashboards format in Voila GridStack. There is more coming as well, especially with respect to hosting. With respect to hosting, so we're working on JupyterLab scenarios for Voila and also a Voila gallery website where anybody can actually post their Voila dashboards and have something similar to what you already know if you are our user with shiny apps. So quick example of a Voila application. I'm going to conclude with that. Voila, ipa.npl.ipa.nb. So ipa.npl is a Jupyter widgets package that renders matplotlib figures in a canvas and streams mouse events and the matplotlib figure PNG to the front and so that you can interact with it as with the matplotlib desktop application. And using Voila and that widget, you probably have the simplest mean to share matplotlib figures in an interactive web environment. And pretty much all of matplotlib just works. But now we have matplotlib web applications. So if you have any questions, I'm going to take questions now. And if you look for me on social media and GitHub, I'm still going to call it everywhere and my company is Quantstack. We've got one here. You briefly talked about hosting. Yeah. Can you tell us more? Is it possible to host on the web? Sorry, can you repeat this? Yeah, you briefly talked about hosting these converted notebooks. Obviously they're being hosted locally. To what extent is it possible to host that on the web because you need the kernel and everything else running underneath it? So are you aware of Binder? No, so Binder is a free service for hosting Jupyter notebooks. So you can find it at mybinder.org. So to learn and read mybinder.org is a free online service powered by and maintained by the Jupyter team to actually render any notebook that is in a GitHub repository in which you would also have provided a manifest of all the dependencies for your environment for you want to run that notebook. And so we have this local repo to Docker that transform that GitHub repository into Docker based on the requirement of TXT on environment.tml or even Docker file that you may have put into your environment. And then you can just render them. So as a quick demo here, if I go to the extensor repository, I have a batch here for LaunchBinder. It's going to first show me like a static rendering of that notebook and it's launching a Docker image for me on the public cloud that is hosting mybinder. So, and now I have like a runable version of that environment. And I hope that you appreciate here that this is a C++ notebook. So that environment actually comes with a C++ compiler and a C++ interpreter from CERN and whatnot. So it's an actually pretty complex environment to spawn, right? So it's very arbitrary. You can do anything you want. So binder is pretty popular today. We have over 100,000 launches per week. And it's built upon the Jupyter Hub infrastructure that is the multi-user notebook server that allows like small and large organizations to run Jupyter as a service publicly or within the organizations. Hello, very interesting talk, thank you very much. I have a couple of questions if I may. One is that I use a Python library and I'm investing a lot of effort in Jupyter widgets. And I wonder if you have an idea to provide some graceful degradation for non Jupyter environments because it turns out I have a lot of spider users and users that use like classic IDE and et cetera. And we just do not work there, which is a bit of a trouble. And the other question is, I know that Jupyter Lab 1.0 is coming very soon. Is the idea to replace Jupyter notebook or they're going to like stay together because they serve like different purposes? No, the idea is really to replace it, but the Jupyter notebook has millions of users. So we're going to keep maintaining the classic notebook for a while. And maybe one day there will be like a website for the notebook countdown or something, but for now we're going to keep maintaining the classic notebook. But the idea is really to have Jupyter Lab replace notebook eventually. We have, you know, Jupyter Lab has this plugin infrastructure where you can, which makes it just a collection of plugins and none of them actually have more privilege than the others. And there has been remixes of that application where we make just a single notebook in one page like the classic notebook using Jupyter Lab components. It's a project that UVPandak called the littlest notebook. And you can check it out on GitHub, yeah. And about the widgets, the other question? About the widgets? The graceful degradation for like... Yeah, Spider. Spider is really dear to my heart. Actually, it's the first project I contributed to as an open source developer. I really like widgets to work in Spider. I think the simplest way though to do this would be to actually have a QT WebKit into Spider, which is built with QT and we could handle widgets in that way. There are also efforts in the NPConvert project also actually using QT to render widgets in PDF, but merely with a static image and corresponding to their current state or whatever state is in the notebook when it's rendered. So yes, we are thinking a lot about this, yeah. Hi, thanks. Very interesting about X-Tensor. How's the performance of that? I just wondered how it compares with NumP and also maybe with just standard C++ with the normal libraries. And would you be able to get, would you have to rewrite your C++ code so that it didn't use, say, sort of lay-pack libraries if you needed to do matrix stuff? Yeah, so X-Tensor is, first, it's a library that is one of the main focus of QuantStack and we've been putting a lot of love into this over the past couple of years. And it's an entire ecosystem of multiple packages. Like, X-Tensor for the pure C++ thing. I would say that in terms of performance, we tend to be faster than NumPy, but not always. And when we don't, it's a bug, essentially. So the one way to put it is that we strive to detect when we can do a SIMD acceleration at compile time. And when we can't, it actually falls back to a more simple implementation, like more generic implementation, that can be slower sometimes. And when we, sometimes we actually fall back to this when we could actually do acceleration and that's when we have not as good performances. But most of the time, it's much faster than NumPy. That's really interesting. I'll try it. Thank you. Thank you. One of the constraints of Jupyter Notebooks is that they don't load balance well. Are there any good load balancing patterns in Voila? They don't load, sorry. Load, like if you want to scale, if you want to run the same instance, like what MyBinder does, like launches it in a container. Yes, so if you actually use the Notebook as an authoring environment for, say, Voila web application, right? One thing that we've been exploring is the ability to fork kernels so that you have a pool of available kernels already there so that when someone connects, it gets like, it doesn't need to wait for the entire Notebook to be executed and get something on the screen right away. So we are thinking of ways to work around that limitation. Martin Brodors, who also is the co-author of Voila, worked on also ways to use Jupyter widgets in Flask applications. And I mean, when you know about all of this, so yeah. Thank you very much, that was a lovely keynote. So I live inside Notebooks, I teach Notebooks to teams. Notebooks are brilliant and they encourage some really bad coding practices, particularly for novices. Does JupyterLab enable better coding practices, debug cycles, and ways of cleaning up code that go beyond what's available in Notebooks? So JupyterLab doesn't have an interactive debugger at the moment, but I can already say that QuantStack, I mean, our company is working on it and you can see poor requests over the place right now on us implementing the debugger. And hopefully by the end of this year, we'll have something already in there. In terms of whether Notebook encourages bad practices. Well, I'm sorry. Encourage them, it's just easy to do bad practices if you don't know software engineering and many people coming into the industry don't know software engineering. So they don't know to do any better and they end up with overly complicated Notebooks that are not easily debugged. Well, another way to put it is maybe people who would have been writing VBA in Excel and not writing Python in Notebook, and I think it's a net improvement over the previous situation. Okay. Thank you very much for your talk, Shovem. We're going to have to bring it to a close there. I'm sure you're sticking around for the rest of the weekend. It's today. To the rest of the day. So if you've got any more questions for them, I'm sure you'll be available over coffee and lunch. So thank you very much.