 Thank you everyone for coming. I'm really happy to be here today. So today I'll talk a little bit more about Bokeh. I'm Fabio. I work for Continuum Analytics and I'm Cardav of Bokeh Library. I'm involved in many things and open source community stuff. I like helping with the open Python world itself. So if you have anything to share or want to talk about later after the talk, I would love to hear stories about your Bokeh experience or your code experience, what you're doing. So come and talk to me. First, some warnings. This talk is actually highly inspired by other Bokeh talks. So you may see some duplicate content, especially Brian Tandevan talk at SciPy this year. Sarah Bird talks this morning and the one at PyCon, Christine Doug Tutorial. In fact, I'm stealing a lot of material here, mostly because of time issues and because there is great material and it's worth it to reuse it. And that's what open source is about, right? So I'm also sorry that you don't have the very welcoming and great speaker like Brian or Sarah, so you need to have me. This is very inspired by the Bokeh team work. The brainstorming is an idea we have. Actually, it turns out that this is very different from the talk I wanted to give when I applied it. I tried to stick with the main guidelines about visualization, nice visualizations and big data, but it changed quite a bit from the original idea. Also because user feedbacks and requests, there has been a lot of new things on the Bokeh side in the last months and I wanted to also give some real-world showcase about these things and things that people are asking for already, but apparently we need to do a better job of publishing those things. Yeah, and of course the recent evolution of Bokeh itself and what's going to be next. Who is he talking for? First, who here have heard about Bokeh before? Just heard. Okay, cool. Who here uses Bokeh or have used Bokeh? Okay, that's what I was expecting. Actually, the main idea was to show some talk for advanced or middle users. I actually changed it a little bit so I will try to show nice things that you can do, but also touch real code, show you real code so people can actually see what we are using and experienced users can actually have some inspiration. This is probably the most hacky talk, Bokeh talk ever. So we prepared for demos and probably things not working, especially with the Wi-Fi, but let's see what happens. So what is Bokeh? Another question, who here is more into the scientific or by data branch of things? Computing, okay, and who's here is more web geek? Okay, cool. The numbers are changing over time. So this is good. What is Bokeh? Bokeh is an interactive visualization library. Python, but not only Python, obviously more about that later, that lets you do novel and beautiful graphics, very data-driven that lets you do dynamic streaming, large data crunching on the browser and have deep roots in data science, but it's really available for everyone. The thing about roots in data science is that it started with some specific problems in mind, but it actually touches a lot the web world, so it's kind of an interesting balance. And we used to claim a lot about this. You don't need to write JavaScript to have your stuff written in the browser, but I want to push back a little bit saying that you may want to, and we'll see about that later. So back to what Bokeh is. Bokeh is by design separated in few parts that actually lets your Python code talks with your UI, your front-end code, or at least generate your stuff. So Bokeh basically, the path to render your plot on the browser, it's generated by a specific Bokeh JS library written in JavaScript, JavaScript, that basically consumes JSON blobs. So it's quite easy to have language bindings, because what the lower level of each specific language just need to do is implement the JSON interface to communicate to Bokeh JS. So right now we have the Bokeh Python binding, our Lua, Julia, we have a Scala binding, and I'm pretty sure we have new ones in the next year or so. So also, looking at the language specific side, Bokeh is composed by a series of, let's say, modules or stacks. One stack is on the library interface, we have the models or glyphs and plotting and the charts interfaces. Glyphs models are the lower interface, we have the middle interface plotting that basically builds a lot of shortcuts and nice patterns to help you create your stuff. And we have the new charts interface that basically just lets you use canned charts, preview canned charts to be like the main stream line charts like line or series or bar chart or whatever. Right now we can output to different outputs, we can output to both, I Python node, Bokeh file or server, we have the presentation layer, which we have a lot of modules to actually do some styling tools, add tools to your plot, layouts, interactions and embed results. Cool. So what's the difference between those API levels, we have the glyphs in API level, which is the lower level, and those three shows you how to create a line chart. As you can see, the glyphs interface is pretty much lower level, you need to create a lot of objects to compose your chart. It's not that it's difficult, but it lets you compose really, really custom plots, really novel plots with those. So if we have time, I would love to show more. Then we have the plotting interface that actually it just needs a few lines to create a line plot. All you need is just to specify the output you want, in this case we went to file, we create a figure, which is basically a plot object. We add a line glyph to that, and right now we just do some styling. This line is actually extra, we couldn't avoid it. Charts is even more shrinked. You just need to have your own data set of linear data, and you map those data, you just pass your data to a line builder, and it builds a new chart for you. Okay, so right now, new users can ask, how can I learn more? First place would be the documentation, as I will say later, documentation has been a big effort for us, and I will show a little bit more about what's next, what's new. There's the GitHub repo, Twitter, our Twitter account, our YouTube channel that actually shows some nice examples, and I think you can learn a lot from talks we give on conferences. You can see a lot of talks on our YouTube channel, or on YouTube, on conference channels. And of course you can watch the talks of Europe later. So Prolog, sitting from Brian presentation, I wanted to show a little bit about what happened this year on Bokeh. So we had a lot of nasements, adding sophisticated tools towards the plots. Now we have a lasso tool, poly selection tool, point selections. We have a new improved crosshair inspector and a hover tool inspector. We have finally hover above lines, we'll see that on an example later. We have really improved visual design, especially on tools, but also in small details. We have a really improved documentation set, as I will show later, showing many interactions. And actually we have reference guy 100 complete and auto-generated. So basically all our APIs are documented, and it has live plots that you can interact with directly from the documentation. So basically all glyphs, models, they have doc strings that have code on comments. Our documentation chain grabs that code, executes it, integrates into the Sphinx chain that is generated in the object. So we actually generate everything. We have a new callback mechanism to build very sophisticated interactions into static plots, static documents. So actually documents that you can build with static, with dynamic interactions, and you can just integrate into an e-mail or a send to someone over the chat, and they will see everything I will show that later. So, and actually, I wanted to mention our Bokeh. We had new improvements on many language bindings, but I think probably R is the most, the big news, big new one. You actually can integrate R with Bokeh plots. Ryan has been doing a lot of work on this, and there's actually really nice visualizations that you can do. So what can you do with Bokeh today? So first, just to show a little bit of difference between, one year ago, the project was those current numbers, version five, and those are the numbers today. Increasing, it's increasing with an incredible speed. It became quite popular and big project, and quite actually broad project that touches many different technologies. So many languages, many technologies, and it's really nice to see that users are using it and it's growing the user base. This is something that is really brought by a great team that is growing a lot. I don't have time to show and talk about what everybody is doing, but we have added a lot of new people to the team to work on specific problems, so on having better tools for styling and better nice looking plots, other for specific computation problems, so it's very exciting to work with all these different people. So one of the main important new things are dynamic interactions on static documents. With JavaScript callbacks, you have now the capability to create nice, nice interactions within static plots with just a few lines of JavaScript. So pushing back from what we claim, probably writing a few lines of JavaScript is a good thing because you're still in your Python environment, it's just that you're integrating your JavaScript code within your Python code. In the future, we want really to encapsulate a lot of those patterns, so we want actually to reduce JavaScript you need to do interactions. Another very powerful addition is the AJAX data source that opens really new scenarios. So in Bokeh, a data source is basically a bag of data that you tell your plot to use. So for instance, if you have a dictionary or a penless data frame with X and Y, X and Y are linear arrays, it can just pass that to a column data source and it's a bag of data that you can use. So what happens when you pass that to a plot, that data is serialized in the JSON object, it's sent to the JavaScript site, Bokeh JavaScript library, which will consume that and generate a new plot, that data is static on your UI side, on your front-end side. With an AJAX data source, you don't pass any data, you say, track this specific REST API, ping that, pull that with a specific frequency, and use fields X and Y from data you grab from that source and basically it will just interact live with a completely different service that it's not related to Bokeh itself. I will show some live code later. Another nice addition is the Bokeh embed components. So basically now you can create like one, two, four, ten, twenty plots and you can basically just call Bokeh components to have a mapping between keys and your components and you can use those keys to embed the plots in your page. I will show that later as well. And there's a lot of more info now with our new user guide that I would quickly show you. That didn't work? Okay. So this is the new documentation we have. As I said, we have live working demos. So those are generated from our docs. We are going to the interactions. So let's see. Okay, defining actions right now. So for instance, this is a static page and within a static document you can actually define actions. So for instance, here you define a new plot. I say use this source, draw this glyph and use the tab tool to open a new URL when I click on a glyph. And it just opens the new. And this is created as a static document in our documentation. This is the new concept of callback. So right here you have a line plot, a figure, a normal data source, and I create a new callback that have some JavaScript code that we will just take, serialize, send to the JavaScript side, and bokeh.js will take care of that and add this as a new interaction on your plot. So we are saying here add this callback object to a bokeh slider widget. And basically when I move the widget, I will call, I'm actually able to pass, so here I'm passing, I'm saying map the source object of the Python side to a source object in the JavaScript side. So I'm actually passing that Python object and bokeh knows that on the JavaScript side this is the same source object. And I will use that object from the source, the JavaScript side to get its data to create a new callback for the value and update the values of the line. So here you can see in action when I move the slider, it just changes the line. There's a lot of examples of callbacks, some are really nice, fancy stuff. You can see here, and you can see it can create a fairly large amount of complex interactions from the Python side that translates to the JavaScript side. And this is not communicating to any server, it's just on static documents. Okay. Cool. So this is another example of nice interactions in static documents. This is thanks to Sarah for building this. This is a creation of a famous tech talk that basically puts together mortality and birth rate over time. So I loaded this notebook here. And as you can see, it loads static notebook. But what I want to show here is that this is a very nice interaction, very nice example of a cool plot, a novel plot being updated on a static document. And this is being rendered on a Jupyter notebook just because we said output to notebook. If we say output to an HTML file, this is an HTML file, a self-contained that you can just email to anyone or send over chat or whatever. You can see that on any device supported by Bokeh. Cool. The other one is nice spectrogram example that those that already use Bokeh or heard about Bokeh may have seen that in the older version. Basically, it's an example that shows some streaming that collects information from the mic, computes the parse practice, and basically updates the Bokeh plot. So let me show you this example live. So the one of the big there are a lot of improvements on this example. One of the main ones actually is the customization of the layout before it was just a Bokeh layout with here you go. So as you can see as I update volume and it just tracks my voice and shows everything, I can move the sliders to update a little bit again. I can pause it. And this is everything is running streaming. It's not using Bokeh server at all. And this is a very interesting demo basically because it touches a lot of key Bokeh features and as the library itself evolves it's nice to get back to that and see how we can play with this example. Oh no, I wanted to show something. So this is a fairly large example, but of course I won't find that. So here you go. Okay, so I just wanted to show this part of the example. So we're basically building Bokeh widgets. This builds the widgets we are using for the example. What I told you before is that right now you can use a mapping to say map this key to this Bokeh object, this to gain sliders, back, back, et cetera. And when you pass to Bokeh components it returns a script that you can integrate in your template and the divs for each one. So divs is just basically a dictionary that have those keys and the values are just the HTML tags that you can use to place your objects within the page. And you can do it and use your own template script with whatever web thing you want to use in the Python world. Cool. Because we don't have enough web frameworks in Python. Spectrogram, Remix, okay. So as I said, you use components, performance enhancements, mostly within Python. We replace a lot of JavaScript code here. This one, I think it's a nice example of use of Bokeh in a real world app. This shows an app in use for our X data project team. It's something that I cannot show you code and it cannot show you entirely. It's a fairly large project. But it's a nice example of to show what you can do right now with Bokeh and putting a lot of things together on a platform. And this is not using Bokeh server at all. This is a static HTML page. Not so static because it communicates with a lot of services but it uses AJAX data source as well. So as you can see, you can interact with the plot to zoom in some areas. You have some nice hover tools. You can pop windows. You can select and select glyphs. This is a very nice addition that we have. So basically the possibility to those elements on the bottom, the legends are actually plain HTML objects that you can use to hide or show Bokeh objects. So there is a really high interactivity between plain vanilla JavaScript and the Bokeh objects that you have to use. So that's that. Cool. Yeah, as I said, this is an internal project we are using for the Xdata initiative to detect and help search for financial fraud. And again, no server is used for that. And another important thing is that there is no custom code there. Every feature, everything that is there is just using the Bokeh most recent release. I would like to show that demo as well but I don't have enough time I think. So I would just tell about that. This is another quite known demo that we had for a long time that basically shows a large data group of data. It should be like 5 giga data cube of ocean data that is basically being down sampled and in slides through time with a live plot service on the web. So it's a very cool example. The thing that is really cool in this new version is that you don't have to, it's not using Bokeh server at all, it's just using an Ajax data source and a small flash script that you can actually see here. This is the code that is used on the flash side to receive the data from Bokeh, down sample the big chunk of data and then resend back to the plot itself. Okay. This is another example of nice visualizations with Bokeh. And basically the previous example down sample was done, the previous example was done by Brian of the Bokeh team and Matt Rockling who wrote the basically down sampling code. It used Dask. Dask is basically a task scheduler for distributing arrays and tasks on those arrays. And it generates some dot files to show the current execution of the tasks. And this example shows some Bokeh plots rendered using those dot files. It's a pretty cool thing. And those are actually Bokeh glyphs. You can add interactivity to those and move them around or color them depending on the situation. Okay. So I talk a lot. I wanted to show those features put in place. So right now the first demo I wanted to show is the Stocks demo. And let me check here. This demo is basically showing the evolution of a Bokeh chart that's a plot that shows stock prices data. And usually we showed a lot of those examples with stocks data per day. So let me grab the code here. So this one shows a simple plot of stocks data over time. This is showing daily prices since 2000 until today. And the code is fairly simple. Here we go. I just say output to a file, create a figure with where the X is daytime. Draw a line taking from this data. This data is basically just a matrix, an array of prices. And it shows the line. Nothing really cool about that. The data itself is not so big. So the next example is something that we actually have on our examples folder. And it shows both K and O. It's why it's not showing. I'll skip that for now. I'm going to skip that for now. But basically you can see that example on our code base. It shows focus error being used to compute some live data from comparing two stocks. What is really new right now is this one. So let me take a look at this. Right now I will basically run a Flask server that is used to serve Ajax data. Of course it's not working. It's already running. Okay. So let me take this. This one shows the same example with an Ajax data source. Let's stop it, rerun it. Okay, we started. Okay. So this is basically the same thing just using an Ajax data source. The code is very simple. So as you can see, it's basically the same code saying use this Ajax data source as a source instead of my Pandas data frame. So what is really cool about Ajax data sources is that you can use them in a fancy way to build cool things. So that's called the second example. For this one, I'm actually using both two new things, the bokeh components to place the components here. So I have one plot on top, another one here. And this is basically the data that is being served for the Flask server. The Flask server have a rest API that I can use to say give me back a chunk, update this data source. So when I select a region here, I'm telling the server to resample that data for just this interval. I need to resample this in real time. As you saw, I don't know if you noticed when I zoom in the region itself, at first the plot was different and then it updates right now the same thing. What happens is that I update the ranges on the UI side, I say to the rest API, update the range, it resamples everything, down samples of the source and resends back. I will show more, as you can see I can add some nice tools here. I like that the default hover tool, but I can do better and build some more custom. Okay, so this one, an address that I need to use. Okay, so I want to use the... So this one shows a little bit more things. I'm using the bokeh components to actually place the components of the page. I added a plain HTML div to handle interactions and a theme combo. So right now, if I do the resampling, you can see the data is being resampled in real time. Right now, I set the resample to 30,000 points, so right now it has 13 and something points, which is a nice thing about bokeh because it's using Canvas instead of SVG. If you try to place as many points on SVG, you see it blows up very quick. On another range, if I zoom in quite high, I will down sample to less than hours, so I will see a factor of resampling that is pretty higher. Right now, on the server side, it just down sampled around 60,000 points to 12,000 points, and you can see that the data is really detailed here, so I can try and see here. It has a lot of points, and everything is done in real time, just using a few lines of code that I will show you in a bit. As a nice addition to this example, I've added a theme so it can actually theme your plots, so the same visualization is shown on a dark or light theme. Cool. So this example is a bit more complicated than the other, but here it is, and basically what's happening is I have a lot of code. This code is just style, style, style. Here, I create a data source that you will use later. This is just a function that creates the figure for the main plot. This is a selection, a smaller plot, and here is the code, the JavaScript code that I wrote to, say, both update the ranges on the main plot, tell the Ajax service that it needs to update, down-sample everything, and send back the results. The only thing, as you can see, the only thing that is doing on the callback, I could do most of this without most of this code. The thing that I wanted also needs to create that panel on the right, so this is done by this part of code. The other part is just the small flask example. This is all the code that I'm, for the flask rest application, which is fairly small. Okay, so I'm kind of running out of time, but I want to show another nice example, which is the Airports example. Here we go. This example is based on a site that's called OpenFlights, and it has an OpenFlights repo for open data about all the Airports in the world. Okay. So first I would show you, this is an example with bokeh server. So no line of JavaScript to build this. There you go. Sorry. No, it's not working for some reason, but it's not something that I would like to show. I really want to show the Ajax one, so that's not so bad. Okay, I will show you the complete, the full demo. This one shows basically all the Airports in the world and highlights a connection for an airport. The cool thing about this is that basically the architecture of this app is that both the app and the bokeh part, the service and the bokeh part are on the same flask service. So I can call a simple, a simple script that just sends requests to the service saying update airport. And this translates to the service changing the Airports on the app. So as you can see, you can stream your data to your bokeh plots using other patterns than what we usually show on the bokeh server. You can have your own API and it basically updates everything on your API without you having to write fancy stuff. This is something that is really useful for, for instance, we have many users that say they have sensors collecting data from somewhere. They want to, they push data to their own REST API. They don't know how to integrate their stuff on bokeh side. This is very simple and the code is not so big for this demo. I will show you the code right now because we are out of time. But we have the break just after it. So if you want, I can extend a little bit. Cool. Sorry for being so slow. So what's coming next on bokeh side? Just a brief small words about it. Some features that we have been asked a lot. First one is latex annotations. This is a really painful feature. We have been looking at this for a long time. There weren't many promising technologies for that without a huge amount of work. Right now we are very positive about something that seems K-Tec that seems pretty, pretty popular and really nice library. We'll try to integrate that soon. The other one is static output. So people would like to create static output images right from bokeh plots. Surprisingly, it's a very hard problem to stab. But it's one of our main priorities. Right now the patch that looks more promising is from MPL or Cairo back-ends. Let's see. New charts and festing. The charts interface is the most recent one. Evolving is not really stable as an API because we learned a lot from the first ones. I'm starting this week with Nick Roth to work on that more heavily. We have done a lot of conversations and we are very excited about the new charts. The new validation layer for bokeh, which is something that we wanted for a long time. So basically one of the problems we had with serialization is that it's easy when you hold a lot of serialization from the front end to the back end. It's easy to put stuff that is missing from the serialization side. Right now if you are validation system, if you forget to add something or you misspell some data spec, we'll warn you. Right now another couple of really cool things are bokeh, apple thawing, a new thing coming out. I want to show a video but I don't have time. But I really invite you guys to watch Brian van de Ven talk at sci-pi. It shows that stuff. New R, Scala and JS improvements. We really want to integrate the trial scope, use new Angular or React stuff. Really promising and really cool thing that many people have been asking of FGL back end. I'm really happy to say that we actually wanted to show an example here. We, Almer, have been working on that for the last month. We have now a branch where actually it's working and it's really promising. It's incredibly performant. We have an example of plotting 10k points on WebGL. But it probably can increase a lot. I tried to increase to 100,000. It performed quite well. So we are getting there. We really would like to reach a 1.1 version by the end of the year. Which means that this is going to be an awesome year. Okay. So, yeah. I want to show this one. It's really quick. We are working on new ways to work with Bokeh. One of this is a very experimental branch. Nothing really official of the moment. But it lets you sort of use this Bokeh develop command that tells Bokeh server to check a file that we are working on. And while we type and you change your stuff with your own editor, I don't know. I would not name any editor because then I will get inside of Flameworks. So, as you can see here, it's updating. This is a normal plot Bokeh code. As it updates on the right side of Bokeh, it changes on the left side. It reloads the plot and changes everything. It's a really cool thing for people that want to prototype because usually the main flow that I see is like using iPython notebooks. But if you have two views and you can see actually things get interacting while you type in your editor, it's a pretty cool thing. Well, as I said, we are working on, maybe I didn't say that as I wanted, we are working on changing a little bit the Bokeh server or the redimension. It's really, really cool piece of functionality. But it probably we would like to separate things a little bit more. So keep the Bokeh server a little bit smaller and simplify it a little bit and add functionality that you can plug in on top of Bokeh server. Cool. I wanted to show a WebGL demo live, but I didn't. Same episode, Tori. So I really would like to invite everyone that wants to use or try Bokeh to try it, to try to provide feedback. If you want to come talk to me later, come talk to us on the main list on social media, we would like to hear about your needs, try to help you and to build custom apps, dashboards or integrate new functionality to Bokeh. It's an open source project and we really invite people to help. And that's all. Thank you. I hope it was interesting. I have heard a lot about Ajax, but do you support WebSockets in a way to push data from the server directly? So you're asking if Bokeh provides ways to push data from a server? Yes. Well, Bokeh server is a way to push data directly. Using Ajax data source, you can only send Ajax requests to your server, so it's not the same as using WebSockets. But you can poll on whatever frequency you want, so if you expose your service, you can do that. One problem could be updating all the servers. You don't want to transfer big chunks of data every time. Right now, at the moment, you can use a trick for that. Instead of using Ajax data source, you can use a normal source. It's basically on the JavaScript side, it's basically just a bag of data. So if you have a callback and your callback just say the best chunk of data, you can just take that and push to your JavaScript arrays and it works well. It's not the same, but it's a possible solution. Hi. I have two questions. One is you said there were new charts, so I'm interested what kind of charts those are coming. And the second one is about the tech talk, I remember seeing it and I was wondering the talk, how it influenced you. So the first question was I already forgot the first question. Charts, new charts. So charts is an interesting API because we want it to be very high level and at the same time we want to give all the flexibility we have on the lower level chart of the plot interfaces. So we had a lot of discussions on this and we are being we have studied also the main user requests and user needs are usually linked to data grouping, data phasing, data aggregation and all those. So we want to I'm not saying we want to go in the same path but I don't know if you are aware of those but we want, Bokeh itself is very influenced by the grammar of graphics, I don't know if you know about this but we want to follow this path somehow and let user tell for instance I want to build a new scatterplot and I want those from this data set it's not normalized so you need to group it and you need to do this, this, this and we want to add a new way to spell this regarding new charts we right now have I think 10 new 10 charts already or 12 one that is there for ages is waiting for the new work is bubble charts but we are really open on to new type of charts we want to build like a vialing plot or a lot of new fancy we actually took into consideration a lot of fancy charts the other question was regarding a tech talk I would defer that to Sarah who is here luckily and she did the the example itself so I think she is a better source for that come to the end both later and we can talk thank you more questions it was mentioned before but how about supporting maps I have seen some examples with maps are you going on that yeah so we already have support for Google Maps I will show you a minute while I talk we also support building patches from files to generate maps so we have a fairly support geo special things are an area that we want to work a lot so for tasks for instance it's not really really well supported but it have some examples that you can already use so for instance if I run Python examples should work if Wi-Fi is working if we have other questions Wi-Fi is a little bit slow oh here you go so basically this is a plot uploading Google Maps and a scatterplot on top of that map and you can actually some interactivity this example here you go we are over on time I think so thank you very much for coming I'm talking