 Hello. Yeah. Okay. So I'm going to start at 12 o'clock now. And I'm going to be talking about JavaScript chatting using YUI Flot. Who here is a JavaScript programmer? Okay. What is your interest in this talk? Everyone? Who's here just because they're curious? Cool. Okay. This is who I am. My name is Philip Delis. That's a very dark picture of me. And that's how you can reach me. I work at Yahoo. I'm a geek. I've been an open source developer for over 10 years. And I'm Blue's Moon on Twitter and most social networks. So, chatting. What's this talk all about? Chatting options for web developers, primarily JavaScript-based chatting. Because that's what I do. JavaScript chatting with YUI Flot. What I learned while building this and how you can contribute to the project. Who here contributes to an open source project? What kind of projects? Who does it on the front end? Good ones. Okay. So why do you need chatting on the web? For me, that's the way I deliver content to my users. My users are actually all internal to Yahoo. But I deliver content to them using the web because so much of the infrastructure already built for me. I don't have to go and build a window and several widgets and stuff. It's all there. I just use HTML. The data I need is also on the web. It's easy for me to send users to a particular page. I just send them a URL. They get it. And if I find a bug that I can fix it without actually having to send them a new URL. So they get that data out. And it's a little more portable than my manager sending me an Excel spreadsheet that I can't open. That's why I get chats on the web. Now, what are the alternatives to doing chatting on the web? So initially, I used to use static images. So there are several libraries depending on what language you want to use. So PHP has a bunch of libraries. What programming languages do people use on the back end? JavaScript on the back end? Python? Java? Anybody does CJScripts in C and C++? Yes. So regardless of what language you use, there is a chatting library on the back end. And if there isn't, use Google Charts. Hit the REST API, pull down the PNG and serve your users so that Google doesn't figure out your referrer. That's what I do. There's flash based dynamic charts. So the thing with static images is they're static. The user sees the chart. They can download that image, email it to each other. But they can't really interact with it. They can't drill down into the data. They can't roll up and stuff like that. So you move to dynamic charts, which is one of the options is flash, which can pull data from the server in real time and change the chart. It can do funky things like animate your chart. And managers really like it. But I don't, for several reasons, one was I develop primarily on Linux and every time I use flash, it grabs the focus of my page. So I'm hitting tab and nothing is happening. So that was one reason. The bigger reason is that if I need to change something, I don't like, I want to add a feature to the chart. I can't actually go into the flash and change that because I don't know how to write code flash. But I know JavaScript. So that was the third option I looked at. And actually this is how I actually progressed and got into JavaScript charting. So JavaScript with canvas. So by default JavaScript can't really draw things, but once you add in the canvas tag, which all really cool browsers support, you can draw pretty lines and lines go on to give you charts. So you can do that. A couple of weeks ago a friend of mine, a colleague at Yahoo, Christian Heilman, developed CSS only charts. So he draws bar charts using CSS. They work well for him. I don't do that stuff. And many years ago, like in the early 2000s, I used to draw charts using HTML tables and set the background color of the cell and cell height and stuff like that. So don't do that anymore. Image-based charts. So many charting libraries kind of works in all browsers. If you have a text-based browser, you can still draw the image and view it offline or something. It's probably not accessible, but when we're dealing with charts, we need to make alternate arrangements for making the data available. You can't easily interact with the data. Things I've already mentioned. So flash is pretty. It works on most GUI-based browsers. The problem with the plug-in is also most instability with browsers is because of plug-ins. It doesn't work on the iPhone. Last I heard. Does anyone know if that's changed? There is that JavaScript-based flash runtime, which could make it possible. Yeah, got him. So it may work. I don't know. But last I heard it doesn't work on the iPhone, which for many people is a big downside. It grabs the focus, which for me is the big problem. Flash on its own is fast, but when you want to interact with it through JavaScript, there is that flash to JavaScript bridge, which can be slow, especially when you're doing two-way interaction. So for example, you want to attach a JavaScript-based event handler to a flash event. That's really slow. And another cool thing is you can progressively enhance a data table. So you have data in an HTML table. You can tell your flash to read the data out of that using some kind of tool kit. JavaScript-based charting libraries. So it's easy for any JavaScript developer to hack. I think that there are more JavaScript developers than flash developers. I don't have exact numbers on that, but based on whatever I've seen across various conferences and discussion lists, it seems to be the case. It's based on open standards. So the canvas element is now part of the HTML5 specification. JavaScript already has as part of the ECMAS specification. It doesn't natively work with Internet Explorer, which I'm not sure if that's a good thing or a bad thing. But you can get it to work with using the EX canvas library. So if you really need to support Internet Explorer, that's you have an option available to you. It's easy to integrate with other JavaScript libraries. So if you're using dojo or using jQuery or using YUI or Mootools, that's already JavaScript. You have your data and JavaScript in a JavaScript data structure. It's easy to dump it into a JavaScript-based charting library. And again, it can progressively enhance a data table using whatever kind of data accessor your library provides you. CSS-only charts. I don't have Internet access here. I'll show you that. But that's a URL you can go to if you want to see what it looks like. Icon.co.uk. CSS charts. Very simple code. It's a list item and CSS to enhance that into a chart. The thing is, your data needs to be encoded twice, once in the actual data that you display and the other in the style sheet. And currently, he only supports vertical bar charts. I think he could do horizontal bar charts too, but I don't think anything else. I'm sure creative people can figure out how to do polygons with dibs and borders and stuff. Then you may as well write doom, right? So what is Flot? Has anyone heard of Flot before? One, two, three, four. Does anyone here work on Flot? No? All right. So Flot is this charting library written for jQuery. It's really cool. It's pretty, which is what we all like Flash for, but it doesn't have any of those downsides of Flash. It was developed by a company named IOLA. I forget what IOLA stands for. And it's released on an MIT license, which is pretty free. It uses the HTML canvas tag for drawing charts. It's really smooth, very pretty, and uses EX canvas to work on Internet Explorer. If I remember correctly, it currently supports Firefox versions 2 and above, Internet Explorer versions 6 and above, Opera and Safari and Conqueror. Those are the five browsers that are listed on its homepage. So YUI Flot also supports all of those. So why did I like it? It looks good, and since my primary use of charts is to get other people, mainly managers in my company to look at performance-related data, I need a nice way to make them interested in that data so that they say, okay, here's some more money and go and improve your performance. So I need my charts to look pretty, otherwise they'll never look at them. It has a good event model, so when you do things interacting with the chart, it fires events that you can then attach your own handlers to and react to that. It's fast at drawing and redrawing, so something that I didn't see before, but you can draw a chart and then redraw the whole chart and it's almost instantaneous, except on Internet Explorer. Flot has this ability to zoom into sections, so it has a selection model. You can select a subsection of the graph and then zoom into that subsection. You need to do things through events, but there's example code that does it, and it's like three or four lines of code, and you can do that, which again is important if you want to drill down into the data, and that's something I could not do in any of the flash-based libraries that I looked at. So why did I port it to YUI? Now, I work at Yahoo. I've been using YUI before it was released to the public, so it's pretty much hardcoded into my brain. I know how to write code in YUI, and when I write a JavaScript, when I work on JavaScript, the first library I pick is YUI. So when I needed a charting widget, I was looking around, YUI charts is obviously available, but I had problems with that because of the flash. I used it for a few months and then decided to look at something else. So Flot was available. It had a very good community, so if you look at the, it's on Google code, and you can look at the forums, and there's a lot of discussions, many people sending in patches and stuff. So there was something I liked about it, and I realized even if Flot itself didn't have a feature I needed, somebody would have written a plugin or written a patch for it that I could then import. And since I was already working with JavaScript, and I knew JavaScript, I'm not a JavaScript expert, but I knew some amount of JavaScript, so I could take a JavaScript library, and if I needed to fix something, I could go into the code and figure it out and change things. So that's why I picked Flot and why I ported it to YUI. So what can we do? Section two. You can do a bunch of things. I'm not going to look at all of them. I can show you some examples that we have. Random scatter plots. Now, if you look at that data set there, it's not in order, so there's random points that are just in any kind of order, and it'll plot it on a chart. It doesn't draw lines between it, so you won't go zigzagging through the chart, but you will get nice points, and these are useful if you want to see where your data is clustered, so if you want to get a good idea of, like, 10,000, 30,000 points, it actually is fairly comfortable drawing that many points on the chart, and you get a good idea of where things are. Now, when you get to too many points, you have an obvious limitation. If your screen width is 1024 pixels, and you want to draw 2,000 points, each point is going to be half a pixel wide, and you can't plot half a pixel. So one of the things I did with YUI Flot was try to figure out what's the optimum number of points that can fit in and drop points in between those. So it works sort of, but when you're drawing scatter plots, since you don't actually have a linear progression, you don't always want to drop points, so there's an option in there to turn that off. You can draw a simple series. So the best example of a series is a time series where your x-axis is increasing in constant steps. So in this case, it's one day at a time, and your y-axis could be anything. The thing about what I added to YUI Flot is it doesn't just take in timestamps, it can also take in date objects and plot your x-axis based on that. Resolution is in seconds, unlike in Flot, which is in milliseconds. And the reason I did that was most back-end languages, if you use the time function, they'll give you out unix timestamp in seconds and not in milliseconds. So rather than have to iterate through the data and multiply everything by 1000 before I pass it to the library and just let the library handle it in seconds. And full strftime-based formatting. So you can actually plot something using any strftime-based format of. So I can show you an example. Time series. So this is the chart that I have. And you see the y-axis there. Can everyone see this code here? So that's a time format string. I can change that to anything else. So let's go with J. And if I refresh that, it doesn't work. I know why. I don't have internet access. Yeah, so unfortunately I need to connect to yahooapis.com. So we're not going to see examples. Sorry. But yeah, these examples are linked from the slides. So if you go to my slides and you click on that, you'll get it. Slides I'll put up on slideshare as soon as the talk is over. So once I get back to the main building, I'll put it up on slideshare. Let's go ahead. So you can also do multiple series using a common x-axis. So many times what I find is, so in the data that I look at, I'll have multiple dimensions and a single control point. So for example, the date is the same across all my data. But I have different types of data that I need to plot against that date. So when you select it out of the database, what you get is a single row with one date column and all your other columns that need to be plotted. So it makes it really helpful if you can just pass this data structure that you get out of SQL, pass it straight to the chatting library and get it to draw a chart. So that's exactly what this does. It lets you take a schema and say, okay, I've got this is my x-axis and the rest of these are my y-points. So go ahead and plot it and Flut will figure that out for you. So an example of what this data looks like. So we have a common x over there and y is different. It does some intelligent things. If you just pass it a data structure, it will try and figure out which is your x-axis and which is your y-axis. So it looks for common things like if you have a field name x, then it takes that as the x-axis. If you have a field name time or date, it takes that as the x-axis. If it can't figure that out, it will take the first point as the x-axis and the rest of them as the y-axis. Or it will let you specify which one is your x-axis and which one is your y-axis. What that means is you can pass 20 different dimensions and say only plot these four. And it would do that as well. So how do you use it? How am I doing on time? So you first create a div. You can create any HTML element to any block level element that you put on your page where you want the chart to go and give it an ID. The ID can be anything. It just needs to be unique on the page. So I've given something like div ID equals placeholder. And then include your JavaScript. So the first thing you do is include ex canvass if you're using Internet Explorer. So that's a conditional include that's supported only by Internet Explorer which means that code looks like a comment to all other browsers. So it looks like an include to IE. Then include the JavaScript. So you first include the YUI library. You need actually only so depending on what kind of chart you need, you may actually need less than this data. So the Yahoo DOM event part is all that you need to draw charts. If you want to do time series then you also need the data source which includes the STRF time based formatter. The two of these together pretty small. I did not actually check the byte size of it but it is fairly small. And once you've done that include YUI float. As of today you probably need to download this code and include it on your own server but probably going to push it into the YUI gallery soon so that should also be available through the combo handle. The question is this URL valid for EX canvas. So I know EX canvas is available on Google CDN. It's not as far as I know it's not available on Yahoo CDN. So you could link to it from Google CDN or put it on your own server. Yeah, I would not recommend combining EX canvas and YUI float because you don't want all your other users to have to download all of EX canvas when they don't need it. So that's pretty much all you need to initialize the data and to initialize the chart. Now you need to get some data. So you could get your data from a static array or you could get it using a data source. So in YUI we have a data source object that can pull data from various sources. It could be a URL, it could be a static array, it could be a function that returns the array, it could be an HTML table that gives you the array. If it's a URL that data could be in XML or in JSON or in CSV format. So there's really a whole bunch of formats that you can get your data in and pass it to the chart. So once you do that, you need to pass it to the chart. So you need to configure your chart. So not just YUI float but float on its own has really good results. So you could actually not configure it at all and have a really good looking chart. The default is to have lines on and points off. But sometimes I also like to have lines and points together. So use lines show true, points show true which means it will show you lines as well as points. You could also say bars true in which case it will turn off and draw a bar chart. And then initialize it. So yahoo widget float. You just call it as a function. Give it the id of the div that you want to draw it into. The data needs to be in an array and I'll explain that in a second. And the config object. The config object is optional so if you don't want to put that in, just leave it out and it'll do a default. Now why do we put the data in an array? What can take in multiple data arrays and plot all of them on the same chart? So you could have data one from your first data source. Like looking at performance data. I could get one data source which is reading out my hard disk performance and another one that's reading out my network performance and plot them both on the same chart and possibly even give them different axes because the units are different and the scales are different. Does anyone know what the problem with plotting multiple data dimensions on the same chart is? Probably done it in school with pen and paper. So you have two data maybe of different scales. That's one problem in that you might be measuring one thing in GBPS and another thing in KBPS. If you plot them on the same chart your KBPS will be along the x-axis. So you need to scale them differently. Secondly is if you have too many dimensions your chart just has too many lines and eventually you run out of colors that look nice. So colors is actually a pretty hard thing because you need to pick colors that are visible to everyone. People who can't see certain color or can't see the difference between certain color choices. So the cool thing about Flot is it again it picks good defaults. So if you just let Flot pick it it'll pick your first four first five colors will be pretty good. So what are the changes I made in YUI Flot? So apart from actually changing every call to jQuery to a call to YUI and sometimes I had to rewrite some of jQuery functionality in YUI because YUI did not have that functionality. So it was actually really simple to change it. There were maybe like four or five lines of code where Flot was actually referring to jQuery and I had to change those. Not lines, but four or five different types of calls that I had to change. So it was simple to just do a global search and replace on the code and then run through all the lines that were replaced to verify. Then it was based on Flot 0.5 which was released sometime last year. Flot 0.6 was released at the end of last year. So November, December I don't remember which they added a few new features that I find pretty cool. But they're not in YUI Flot. One thing I added was logarithmic scales. Flot itself only had linear scales both x and y axis were only linear. I added logarithmic scales to all axes which is actually more than what Flot has. So Flot has two axes on each side so you can have two x axes and two y axes. YUI Flot can have an unlimited number of axes on any direction. It doesn't draw all of them so you can only draw two but you can have an infinite number of scales. Again, infinite depending on the resources you have to actually draw it on to actually store that in memory. So and the reason I added that was because I've worked with four dimensional data which is of different scales and needs to be scaled differently on the same chart. So we have more than two axes in each direction all of which can support linear as well as logarithmic scales. So you can plot linear data and log data on the same chart. It's not advisable because it thoroughly confuses your users. Another cool thing I added was axis labels. I did not add this. I added the code myself but it's based on something I saw in someone else's implementation of Flot where now you're using your numbers below the axis and on the left of the y-axis. But you also have an axis label which says this axis is about this data right at the bottom. I'm not sure of the diagram. It's probably after. So if you look at this you'll see a number at the bottom there. That's the x-axis label in series on the left over there which is the y-axis label. So this is something that was added. Axis labels. Point dropping which I already told you about. So if you have too many points on plot it will try and drop them. Now if you're drawing a line chart this really works out because if you're drawing lines and points so it will still draw the lines where those points should have been but it just won't draw the circle around the point. So you don't have just a forest of points on the chart. That's a nice indentation to tab indentation. Some more things added. Time series. So the initial time series accepts milliseconds as the data value. I change that to accept date objects and also use seconds rather than milliseconds. Again I mention it because I write my back encoder in PHP and Perl. Both of which will spit out unix time stamp in seconds rather than have to iterate through my data and change that all. Send the data straight to the chart and not have anything in between. Full strftime based date formatter. So plot itself as an intelligent date formatter but it's not based on strftime, it's based on a subset of strftime. This was actually another pet project of mine so when I was working with the Flash based charting library I didn't like the way it was formatting dates so I went and implemented strftime script and released it under BST license which then YUI decided it was good enough to put into YUI so it's now part of the YUI library. So now since YUI had strftime that I had written a few months before I decided hey I'm going to use it in every charting library that I work on. So it's part of this as well. And the thing I did with the strftime library was it's fully internationalized so you can actually print a date in any locale in the world except I don't know every single language in the world so I did not implement the dates in every language in the world but it's three lines of code to add your own locale and then you can plot chart with your axis in French, German Korean, Bulgarian, Swedish anything you want. I don't know how it works in languages. So things I'm looking at in the future first of all move up to Flot 0.6 there's some new features 0.6 has a plugin based model so if jQuery itself has a plugin based model and Flot is one of those plugins for jQuery Flot itself designed a plugin model so now you can define plugins for a plugin for jQuery so I like that model I like it because it lets other people add code to it without having to modify the library itself which would make my job easier because I can then pull in some of the Flot plugins and use them directly in YUI Flot without having to change them YUI Flot has thresholds feature so for example think of a finance based application or chart where you want to plot everything above a certain level as green and everything below it as red or even in performance if everything goes above your SLA needs to be red and everything below your SLA needs to be green so that's where the threshold feature comes in where it says this is my threshold and everything above it needs to be in this color and below it needs to be in this color and it's a pretty useful feature that I'm adding in. I just can't figure out how I do thresholds for four-dimension, well I have four different time series on a chart because then I need to pick eight different colors I have a hard time picking two also as stacked bars which are useful, I don't have a need for it at the moment but there could be a need so stacked bars basically have so you have one bar and the next bar goes on top of that so your data can see the sum of the data as well as individual data points and it's useful when you're looking at an overall value it's kind of similar to an area-based chart where the values add up so if you're looking at requests per second across five different servers you're primarily interested in what's your total request per second so you look at the overall chart but you can also see the size of each band which tells you more data than just the two dimensions of the chart step line charts are useful so a step line chart is let's say you have two points one here and one here instead of just drawing a straight line between that it draws one line that way and one line that way so it's basically a discrete jump it's not a continuous chart and it's useful if your data is discrete rather than continuous again I don't have a current use for that but I can think of people who would like to know more things I'm looking at so these are things that are useful for me so error bars and candlesticks again when dealing with performance data you can't just say this is the response time of that page because it depends on how many measurements you've made and what's the error in that measurement so you can say okay this page has taken two seconds to load plus or minus 0.2 seconds and you need some way of plotting that plus or minus 0.2 seconds that's where error bars and candlesticks come in so it's basically you take you have a point and you draw a line going above it and a line going below it and say okay that's my range of data or you can do other cool things like shade the chart behind it in a color a sort of lighter color based on how much error there was on each reading adding smoothing plugins so again if you take a time series which has say 100 data points they might be pretty jagged and that doesn't you don't really need to have that level of information you just want to see the trend line is my performance going up or is it going down so by adding smoothing to that what it does is it takes a moving average so take the average of 3 points or 5 points and plot that instead of actually plotting the data point and that gives you a good indication of what's the trend of the data rather than what the actual data is finally upgrade to YUI 3 and once I do that move it into the YUI gallery which means then plot would directly be available through Yahoo CDN so you can directly access it from there without having to host it on your own server which will be one less HTTP request one less DNS lookup the things I learned I was no expert JavaScript programmer before this so the first time I was doing Flot I did something like what's on the left there basically wanted to normalize the data so I passed in the data structure did some manipulation on it and it worked perfectly I was doing stuff then I wanted to add more code which used that same data structure later and plotted a data table and the data was completely different from what I was getting from the server so it's because it was different and so when I was normalizing the data I was normalizing the actual original data structure that I had so internally now I've changed Flot to work on a copy of the data what that means is it needs to iterate through all the data once before it can plot it so that kind of slows it down a bit especially if you're going to redraw the chart multiple times but I need to do that anyway in order to check if all your data points are valid sometimes you might have things that are not numbers in there or you have too many points which need to be dropped so all those things need to be checked so I do it in the same loop there's more to charts than drawing lines so after I started doing this I started getting more interested in statistics so I've spent a lot of time on Wikipedia which is where I'm learning statistics from but there's also some pretty other interesting sites one of them I've listed here so different visualizations for different types of data and if you go to that white paper it's a really interesting one about this guy's got a paper of how you visualize different kinds of data and he's shown like if you have a population chart rather than just show a bar if you show it in a number of human figures it's a little understandable to the person who's reading the chart so it's once you start getting into this thing of actually designing the user interface to the data you need to think much more than just drawing lines to present it to the user basically looking at summarizing this data visually so when a user is looking at a chart he doesn't want to know the actual numbers if you wanted to know the numbers he just print out a table and figure that out he wants to know is it going up, is it going down, is it green right so and you can plot more than two dimensions on a two dimensional canvas so an example is if you take a scattered plot you can just plot all the points where the x and y coordinates are or you can plot bigger points at certain places and smaller points at certain places indicating that this data is more important than this data and similarly using error bars you can have three dimensions on that two dimensional data you can rotate html elements using canvas so if you go back there look at that series it's actually printing upwards I had no idea that was possible until I did this so this is how you do it to work in all browsers mouse transform rotate 270 degrees for gecko based browsers webkit transform rotate 270 degrees for webkit based browsers and then those two lines for internet explorer or you can do this one line for internet explorer so yeah it actually works in everything including internet explorer 6 and above one caveat though when you use a filter in internet explorer it tends to execute that filter every time you interact with the page so with mouse overs and scrolls and stuff it's going to run that every time which kind of makes it slower I don't I haven't measured the performance of this particular filter at all so I don't know sorry those things I learned when working on this project finally how do you contribute so you go to github the projects and github github.com slash bluesmoons slash yui-flot I have the url at the end you fork it make your own changes you'll need to put your changes under the bsd license and tell me about them through a pull request so I can pull them into my own repository now this code is available to everyone but you need to download it from my repository or your own repository when you can totally just run your own fork of it and have your own changes and not ever put it back into my main tree I have no problem with that and you can distribute it on your own if you want to get it into the yui gallery which is what I plan on doing then you need to sign yui-cla I have a few copies here anyone's interested in signing today it basically says that you are legally allowed to commit any code that you actually do commit so like you haven't stolen it from someone or seo doesn't come into us sometime and that makes the code available through the cdn so you can get your code changes into the yahoo cdn which makes it it's a greater chance of it being cached across the internet and better performance through users so that's the url and I have some stickers laptop stickers in case you were bored with my talk thank you for sitting through it but yeah there's a few people I'd like to thank jokin mace and flip paste how do you pronounce it? paps yeah they contacted me a couple of weeks ago and helped a lot with getting my visa and stuff I wouldn't have been here guys who built yahoo developed a network and the yui team for helping me out with all my development work and stuff so you want to pass these things around or do you want to just come down here and pick it up yeah just leave it here I don't know some people are lazy haha sit down that way and uh pass them down this way paps paps paps um yeah so if you need to contact me that's my details again yeah questions does it support dynamic charts kind of so through the event model you can do that um the problem is internet explorer is again really slow with doing the redrawing so I didn't build it into the library but it's possible and I don't have an example yet I'll publish an example showing how to do it right yeah do I support vertical bars I don't know haha horizontal bars I I know it's in Flot I don't remember if that's a feature that was added after I ported it up before but in either case it's going to be in soon others yeah okay so the question is have I been in touch with the original developers to abstract out the jQuery dependencies and maintain that as one plugin and the rest of Flot as its own library no I haven't been in touch with them I I might I haven't thought about it yet so far it was it was my pet project and I put it up on github because it was easy to do that but I have only one person who's ever forked the project so far so I know there's only one other person using it but yeah if there's more interest then I might get in touch with them and do that any other questions does anyone else need stickers you didn't get any sorry sorry I didn't get it say that again yeah I didn't look at the dojo chatting library I looked at GNUplot GNUplot so GNUplot has a yeah I didn't get it so GNUplot has it's it allows you to translate your GNUplot charts into JavaScript it's interesting that way but no Flot was the second one I saw and I liked it immediately so I kind of stopped my search anyone want to sign the CLA thank you