 Ben is the engineering director at Plotly. He works on Plotly Python, TV visualizations, Plotly sharing platform, and backend Django infrastructure. Alex is an engineer at Plotly. He does front-end work and a lot of design and hardware related work. I'm going to mirror the display so we can see it on here. Yeah. All right. Thank you for the introductions. My name is Ben, and up here with me is Alex Vados. He's also an engineer at Plotly, as well as one of our on our design team. And we're going to give you a short demonstration today about a tool that we've been working on for a couple of years now. We had financing last year, so in earnest for the last year. And as I'll explain in a little bit, Python is a really important part of Plotly. We have a lot of Python developers on our team. And so as we develop this tool, we've been thinking about the Python developer having them in mind. We think that our product and the visualizations that we do can offer a lot to the Python developers. So we're going to walk you through and see so you can see how it can be useful to yourselves and hopefully get you started using it right away. But first off, I'd just like to thank everyone for being here. We traveled quite a ways to be here. And so for myself and the Plotly team, thank you so much for this opportunity. We arrived in India. It's my first time in India. We arrived on Friday. And wow, to be honest, it was mind-boggling, the amount of energy and action in the city and also hospitality to people. It's mind-boggling, but also very humbling for myself. It's the same thing I found at PyCon India here. Last summer, I went to PyCon Montreal, which is the North American PyCon event. And it's a very large event, a lot of people there, but the enthusiasm and the willingness to learn of people here is 10 times what I saw there. And so I'm really happy to be part of this event. And I'm really happy to see it so successful here. It's a really great story. So as I was saying, Python's a really important part of Plotly, where the core team, the original founders were ex-scientists who moved over into programming. And so Python was part of our daily workflow doing research. And it was out of that that we got the idea for creating this tool. And not only that, I mean, because we had experienced Python, we wrote our back-end of Plotly is in Django. All the associated tooling is in Python. The very first library released was the Python library. And that definitely gets the most love out of all the rest of the libraries in the team. So I think that's good. We'll step the demo up here, and then we'll show you what that might look like. I'll reduce that a little bit. Or you can all see that? Yeah? Okay, so the library we have, it's free. It's open source. It's on GitHub. You can, if there's any questions or you wanna give us feedback on it, if you use it. We love contributions. It's part of the reason why we're here in Python India is we wanna give back to the Python community, but also reach outside of where we're from, come over and meet new people. You have, I know you have great ideas and there's a lot of talent here. So at the end, we can show you a little bit about how you can start to interact and check out the code. So the library itself, you import it like a standard Python module. You can install it with PIP. So here you're importing the library. And then, so say I'm working on some data analysis or doing some research, or perhaps this is part of a web app. Here we wanna create a data visualization. So we have some helper functions in Python that comes in the package. And so here we're gonna create a histogram. I'm pulling in some data. This is data about age of rocks. Doing a bit of analysis. I create a second histogram. This one's from a different rock type. You can set various options. The tool itself, the plotting, the graph is highly configurable. You can look online. We'll show you a little bit about that later. About how to find all the different options you have available to you. And here we're gonna do a bit of analysis. Gonna take a, so we have a histogram. I'm gonna make a fit, fit a curve to it, like a normal distribution. So we're gonna, we could take that and I'm gonna send out as a trace object to plotly. And here we have, I'll do the second fit for the other histogram. And then we put all the data together in a list. And we create a layered object. That's gonna give you some of the global properties to set things like the title and the color. And then right here we create the figure object and then send that figure object off to plotly. So this is a web, it's a web tool. So when you make a plot, it actually returns you a URL. And in our pipeline library, it'll actually automatically open up your browser and show you what that looks like. So, so I'm working on this project and I have a collaborator, collaborator, could be in a research department or someone I'm working with at a company. We wanna share these results. And the standard way, and a lot of the workflows that we have now is you get some results after analysis, you make a PNG or turn it into a PDF, you ship that to them. And then if they want to interact with that or they have some concerns, they wanna look at the data or they wanna change some things in the figure itself, they're unable to. So they gotta ask you again about can you send me the code or can you send me the data? And there's a lot of back and forth going on. So that's the problem domain that this tool is trying to solve. So let's run this. So the goal of this, I wanna send these results to my collaborator here, Alex. All right, so we can see that opens up a browser window. And so it gives you the URL, but also, like I said, it opened up the window. And so this opens up in Plotly and here's the figure. And so all of these are interactive. You can show you the data points. You have access to the URL up at the top. And so let's copy this. We're gonna ship this over to Alex here. Let's find him. We use HipChat at Plotly. So let's post him this. All right, so then we ship the URL to Alex and he can take over and then start iterating on this work. Go ahead. Can you guys hear me all right? Awesome. All right, so Ben's done all the hard work. I just come around now. He wants me to take a look at it, see if I have any concerns. So he sent this to me in whatever chat system you use. I can pull up this figure here in my own browser. As you can see, it's identified. Ben created this graph. His username is there. And his unique identifier for this graph as well. So if we wanna work on the same graph, we have a good starting point here. So I take a look at this. I'm a little bit more design oriented than Ben. I see it. The colors are a little bit weird. Maybe we should change that. So just head into the tunnel here. So this here, let's make it a little smaller. So this here, what I'll be doing, again I import the Plotly library. So on my own system, you pip install Plotly. And I'm gonna create a figure, but instead of actually creating the figure how Ben created it, I'm just gonna grab his. So I can do py.getfigure. And I'll instead replace this identifier with the one that he sent me, which is at 13. So it's actually already set up. So some things I wanna do, I mean the colors on this, not great, maybe a boss wants it to match the branding, whatever they want, or you wanna mess with the data. So here I just pulled down the figure, py.getfigure. And that's gonna return the entire graph object that Ben has created. Now if I wanna go and change the bar colors, I can select the data, one of the data traces, go right to the marker, change the color and assign whatever hex color I want or RGB, or even just a string like blue or red. And we look at this, we see the fits, they could be a little nicer, we could see them a little clearer. So let's change that as well, maybe change some of those line colors, the background color. So it's all pretty straightforward. It's very declarative. We have what's called a graph reference, so I'll give you a quick peek at that. So this, essentially all of our graphs, what we're trying to do is create a very generic object so people can collaborate. And our plotly object is fully described in this reference. So we can always go in and see, okay, we're making a scatter, what do we need to include? So everything is all here, and that's basically what I've done. I've gone and said, I want to change the line width, I want to change the color, the background color, it's no big deal. So we just quit there, and we can run that. So now this is grabbing all of Ben's stuff, running my changes on it, and pushing it back up to the same plot. So if we go here, you can see it's still Ben at 13, we're now working together. And now we have the Picon India colors, and we're all good to go. So from a design perspective, much better than before. So this can also be done for data as well, you can actually pull down the data traces, maybe do some analysis and send it back up, all working together in tandem. Now one of the nice things about this is let's say, for example, Ben works in Python. What if I use JavaScript, unfortunately? Well, I wouldn't necessarily have the time to learn Python syntax or a whole programming language. So what I can do is take this magic URL, put it .js, and here's that same plot in JavaScript. So now if I'm a JavaScript developer, Ben's made this in Python, I can pull down all the data, work on it, change it, using my own tool set, so something that I'm comfortable in, and push it back up. If you want to embed this in a blog, use our magic URL, hit .html, and this would be the iframe code you would use to embed it. Now if you can see the source as Ben at 13, which means that any changes that we make will automatically show up. There's no more regenerating PNGs, SFDPing that over, putting it on your website, making sure it's all updated, it's all done for you while we edit the graphs together. Just a quick show of hands, how many of you guys use Mapplotlib or have used Mapplotlib? All right, so it's like 90% of the people here, that's great. So we love tools like Mapplotlib, and we're not trying to replace Mapplotlib, we think it's great, we love ggplot too. There's so many great tools out there, but the problem really is when Ben is working in Mapplotlib, and I'm working in Python or JavaScript, how do we collaborate? So let's just get another demo going here. So here we're pulling in Mapplotlib, we're getting an evenly sampled line at 200 millisecond intervals, and we're gonna plot that. So if you guys are familiar with Mapplotlib, this is nothing new. This is Mapplotlib right up to the last line. Now what we're gonna do is just run the Mapplotlib code, so you see the output of that, and there's the output. It's pretty great, I mean it's a good chart, but the thing is, what do I do now? I send this to my boss, it's in an email, maybe he doesn't like the colors, the spacing, the data, he wants to do some more work on it. He doesn't have access to what's behind it. So we can just go back into a script, instead of doing plt.show, which is your Mapplotlib command, we're instead gonna call and pass our figure object, our Matlab figure object, directly to the Python, or rather the Plotlib module. We're gonna give it a name, so we'll run this. So now this is taking our Mapplotlib figure, changing it to a very generic graph object and sending it to Plotlib, hopefully. So we're given another magic URL, and there's our interactive graph, complete with the Mapplotlib styling, but fully interactive. So the fun doesn't really end there though, because like before, this was creating Mapplotlib, a very specific tool, we can still go in and see the underlying data. We can still manipulate this URL and turn a Mapplotlib figure into JS, or into Matlab, or into R, or ggplot2, whatever you use. So it allows us a great deal of flexibility to collaborate. So yeah, that pretty much sums up my part. Ben has some, he's been building some great systems for real-time streaming, which we think, I mean every time you speak about graphing nowadays, everyone's interested in real-time streaming, especially with the internet of things, and just the accessibility of internet all over, it's great. So without further ado, Ben will come back and show us more stuff. Thanks, Alex. Okay, so this is dependent of the internet connections working, so let me just give this a shot. So also, Plotly spends a lot of time about on the APIs for the different libraries and the cross-platform nature. So the underlying graph object is a JSON structure, and so it's more like a protocol. So that's how we can shift it around. It's not attached to any particular platform. So even if in the, let's go into the web app here. So if you have someone on the team that's not a programmer and just uses Excel, we have a core web app which has spreadsheet-like functionality, so you can get the data in the spreadsheet, make edits there. And here, and sometimes I've had the problem when I was doing research where I'd have some core code doing analysis, and it's 200 lines long, and then to get the right plots for publication, there are about 800 lines of code. And so often I find it's nice to get your base plot up there and then bring it into another utility that's more focused on styling. So we can bring what Alex and I have been working on into the web app itself, and there you have available to you a huge range of options to change any little feature about the plot that you want. But all of these features are also available through the API should you choose to do it all programmatically. So it's just a web interface for the same features. And let's take a look, see if we can get this going. Yeah, all right, there it is, okay. So one of the other things that we've been working on is expanding the core graphing library out to real-time data. It's a bit laggy. The Wi-Fi connection's a bit hit and miss right now. But so this reaches out to Arduino's, Raspberry Pi's, we have a Python, you can run Python on the Raspberry Pi, use that to stream data. The API for that's really easy, it's the same Python module or library, except you just call the stream API and then it passes you basically like a file object and you just call write. And every time you write, it'll send that data point right to plotly, which will add that, if you can embed this in your website and as you stream up points to it, it's real-time so anyone else that's viewing that plot will see the same data scrolling by at the same time. So that's one thing that we've been working on and another feature that we've just released recently, let me open up the local host here, is 3D, so underneath the, most of the plots we're using SVG, so that's rendered by D3. And then for 3D work, we switched over to WebGL. And so that allows, so here's a 3D element, this is done inside the WebGL canvas, it does require that the user has a video card to view it, but it is offered the same sort of styling features that you would expect from the rest of the plotly API. So it's this kind of tooling, the real-time streaming, the 3D API and also able to collaborate between platforms that we're excited to bring to Python workflows as well as other workflows. So it gives really nice, high quality, the results and data visualization to your workflow and it's, like I said, it's free to use. The graphing, or the Python library is open source, it's on GitHub, I would love you to come check it out and give us feedback. I know that there's better Python programmers than anyone on our team, so we'd really love you to take a look. And with that, are there any questions about, yes? Okay, while we wait for questions, I just wanna thank you again for you all being here, it's an honor to be in India, thank you so much, I'm looking forward to coming back. Thank you. Yeah, I think the mic is ready now. The thing is, exporting it to JSON is really amazing. So how is the support for Sage? What was the support for? Sage, so actually Sage, if you want to import the same graphing to Sage, so are there any provisions for that or should we have to again? Yeah, we don't have any direct tie-ins with Sage yet, so some of the, for the platforms like Sage, Mathematica, we do not, from that lab we do, so that's something that is in the future, we just, our first set, we have about six platforms that we've targeted, but we're expanding that slowly, so that will be coming. Hi, this is Ray, and how does this work in 3D? It doesn't work and doesn't render in, so it doesn't render in other browsers, right? It renders only in Chrome, so it doesn't render in. No, the WebGS spec is actually supported now in Safari, Firefox, and Chrome, the more recent additions. I think for the last, roughly about a year, it's been in Chrome, and Safari and Firefox have caught up, so the latest ones, it does work in those browsers. Tablet's still getting there for the WebGL, yeah. That, I mean, we're not in control of that, that's whenever they get that going, then it'll work. Hi, Plotli, Plotli, I have a question to ask. I have been using Matplotlib, and... Oh, sorry, yeah. Sorry for one second. Yeah, I have been using Matplotlib, and I have faced some problems using multiple access plots. Like, when you have a multiple access plot and you zoom in, the structure of the plot, it gets disturbed. Have you handled that kind of things in your library, Plotli? Yeah, so we have really strong support for multiple axes, as well as what we call subplots, you can tie different axes together so that if a user zooms in on one plot, it'll zoom in on the other axes, or you can unhinge them as well so that they'll be unaffected. You have full control over that. Do we have an example we can show you here? I'm not hitting local. So these ones, these are subplots. It's very similar to the way that we handle multiple axes. So you can tie the Y axes of one plot to the other so that as you pan on one, the same data range will pan on the other. And if it's zoomed in, it'll pan slower. So it works as you would expect. This is actually a subplot case. Yeah, so you'll have more than two axes on the same subplots. Yeah, we'll get that up here. I think this is not the Matplotlib docs, but the other Python docs. There's a few examples you can check out. Our documentation is pretty robust. So all of the examples on the right are the code that was used on the left. The code was used to create the example on the right. So you can see it's very declarative and the Plotlib graphing library just takes care of all the heavy lifting for you. That's good. I want to ask you one more thing. Okay. In some use cases, we have very large data sets, say around 20 lakh points or something. And we want to plot that, but plotting so many points on the screen doesn't make any sense. Right. So we need to decimate that. So do you have any decimation algorithms inbuilt in the library? Okay, it's an interesting question. So yes, on the web app, when you try and plot a million points and your screen's only so big, that's not going to give you any extra information. You're just, it's wasting points. It's also going to slow down the browser because it's making SVG elements. So in the browser itself, we do have a decimation algorithm that runs. And that's how we can get up to a million points. Otherwise it would bog down a lot lower than that. Past that, it becomes a problem of the network trying to send that much data into the browser. But what we're working on right now is we have a nice data visualization API. So what we're working on is doing the same for data. And so every user will have their own little database and we're going to unlink the plots from the data. And this should allow you to have arbitrarily large data sets. And then the plot will just be a view on that. And then you can specify your own decimation algorithm. So you can take a mean and only do like one in 10 points. So you can get a sample of a lot of data and just get the overview. And then as you zoom in, we'll provide more resolution, but it'll be a tighter range. So that's something that we are working on right now to address that problem. Thank you. So can we extend the visualization for dependency, you know, dependency graphs and all we have, you know, like Neo4j graph databases and all. So can we extend this or? So the core plot with JS library, which is similar to, there's a lot of other JS graphing libraries out there. That one uses D3 under the hood. And if you're using that, I think it has requirements on D3 and jQuery, that's it. But the rest of the ecosystem, that's all built into the tooling already. So those, but for just the library itself in the browser, it's just got the dependencies of D3 and jQuery. And also a video card if you're doing 3D plots. Is there a way to integrate it into something that is not a web browser or image page dependent? Like a simple QT interface, because if I'm developing a desktop software, I don't want it to be a browser dependent. Yeah, so there's a few projects out there, like Node WebKit, which have stripped out the WebKit from Chrome and then offer that as a desktop app platform. And so in that scenario- That is still browser-based dependency, just an interface to link it up. Yeah, so that, I mean, that's, and you can't really get out of the browser. I mean, the core plotting isn't JavaScript, we use the canvas for WebGL. So it's stuck in there, but if you want to send and like so- If it's stuck with the canvas, then we can probably integrate it into some PyQT or QT-based interface. Right. Could be possible with something you could talk to me about. QT, so- This is why we're here. Yeah. Great, all right, thank you. Hey, this is, this is, so how long do you, add the data up on the server? So how long, when you, when you push a graph up, and how long do people have access to the graph, and how do you store the data and all those stuff? How do you scale up to multiple people having that? Right, so we store it in S3 on Amazon, so as long as Amazon's S3 is in business, then your data will be there. And following up on that, so you have access control, right? So like, this can be shared and accessed with only this site kind of people, and not anyone with the URL can access it, or how do you manage the data security? Good point, I didn't cover that. We have public and private, so if you, you can mark at any time your graph is private, and then we, that URL will no longer be accessible, except for anyone that's logged in as you, just hopefully you. Thank you. And just to follow up on that, you can also specify collaborators. So similar to, let's say if you're on GitHub, and there's a repository you wanna work on, but they haven't specified you as a collaborator, you can always fork that repository and work on it on your own. You'd never be able to change that underlying code in the main repository. It's the same thing with Plawley. So if the graph is public, you can always pull it down, work on it on your own, but you can actually specify collaborators as well, which is like what Ben and I did, where you can actually work on the same data and push to the same graph. Any questions? All right, thank you again. Thanks so much. We'll be able to then booth and talk to us. Thank you. Thank you Ben and Alex. Thanks. Thanks for being with us. Yeah. The next session for the DL.