 How about these other speakers? Yeah, I I don't know how I'm gonna follow up Nigel's talk But here we are I want to appreciate everybody at OpenVizConf and pretty much everybody that I've spoken to This is my first time speaking publicly by myself So yeah, it's a huge honor that you guys want to have me here and happy to talk about stuff now and later So I'm gonna get started with maybe shooting myself in the foot Unlike everybody else. This is a Google doc presentation So you can actually go to that link and follow along with me on the slideshow if you want I've kind of annotated more stuff in the slides. There's more links and other goodies hidden in there if You happen to be on your laptop So yeah, the title of my talk sounds pretty high-fluten and it's gonna start pretty high-fluten, but it's it's a very Dear and and kind of normal thing to me that I'm trying to kind of work through and that's the human side to being a digital practitioner you know making stuff on a computer and Trying to find human qualities in that so I want to start actually by going back a little bit Does anybody know what this is? Right, it's not a pipe or maybe it is a pipe I'm not I'm not sure and actually the point that I want to introduce my talk with this slide is not so much What it is or what it isn't? But you know Despite the fact that we can argue about the meaning. I think we can all agree that a person made this That person is Renee McGreet. He made this when he was 30 years old 1928 1929 and I find it so fascinating that that an object an inanimate object can have kind of human qualities If you go and we take a step back and we look at the tools that McGreet used to make his work paints paint brushes specifically oil paints for that painting the treachery of images It's it's less clear that you know There's a story and an intent and an ideology behind, you know, what's what's being made And I would argue that that has translated to the digital as well. This is an amazing animated gift by this motion graphics artist J. R. Canis hopefully I'm pronouncing that right and You know, he he uses digital tools to make these animations and they're just so full of character and and Personality but if you look at the tools that he used to make his work I don't know if you guys are familiar with After Effects Maybe not this particular version But it it's such a pragmatic tool that it loses a lot of this these these intentions Especially when it crashes Which it happens often for me because I have a pretty crappy MacBook So why is this exactly? I'm actually not here to tell you what the answer to this question is But it is something that I am super interested in and have been kind of grappling with in my own kind of artistic explorations And it's this idea this dichotomy of expression and utility expression kind of the metaphor being you know McGreet's pipe and Utility being the tools that he used to make his pipe And I'm super excited to be an artist working today because we have the advent of software and Software allows us to make tools that are also art And I I'm arguing that kid picks here is you know, this is an early version of the software It in of itself as as a UI and as a tool is is incredibly expressive It has a lot of intention and in some ways could be considered art But it's still a tool and I'm using it to kind of riff and remix McGreet's treachery of images here So all of a sudden with the advent of software We have this kind of muddling of this initial dichotomy that we've seen in fine arts And I would also argue that this has some sense in data visualization as well This kind of just as an artistic piece the genealogy of pop rock and music, which I'm sure you guys are all familiar with as a Tufti has popularized You know as a as just a composition Aesthetically, it's like an interesting thing to look at but it's also a tool you can you know dive into the super graphic and find parts of Kind of like music musical eras that interest you and and find You know what's relevant to you and how that is situated in the rest of the world the world of music And in this way Expression and utility are muddled So I want to spend this next kind of half hour or so Talking about expression utility and software through a couple projects that I've been working on Really over the last ten years it's a space that I think is super unique and Not something that a lot of people are exploring and I think there's a lot of possibilities This is this is like the Wild West I think of today So to introduce myself my name is Johnno This is my website my handle is J O N O B R number one Yeah, I'm from San Francisco. It's really cold here Was not you know on the the temperature wise it's like only five degrees cooler than SF, but It really feels a lot cooler here But yeah, you can usually find me riding my bike on the weekends I like to not look at a computer screen when I'm not kind of making stuff And if you're ever in San Francisco hit me up and love to you know, go on a hike or something I grew up in the Bay Area, but I actually went to school for graphic design at UCLA Which is also where Lauren went. Maybe there's some other people here that went there The design media arts program is really really awesome program. It's in at least in the undergraduate program You kind of learn a lot of different disciplines In in how to communicate through design. So, you know, you learn graphic design and typography This is a project. I did one of the first projects. I did kind of trying to synthesize photography as typography and Kind of analyze it a lot of the references and stuff We also learned some data visualization. This is a public policy graph that I did that was kind of comparing a lot of Different publishing events In England kind of which ones were popular or not We also got to do branding and identity every summer the high school students have the opportunity to come to design media arts and try out a few different of the classes and in the undergrads and graduate students get to teach So I got to do the identity and then I was also a TA for the branding class We get to do some film. I thought this one was particularly Ripe for all the people that work on the web. I made a video in college called. We are the inter dot net It's still online. It's in flash. I'm sorry But it was it was coming up to the Internet's 37th birthday And so we made a documentary we went from the art department, which is a north campus down to the south department where the computer science People were in a Bolton Hall and looked at one of the first servers that ever existed and interviewed people on the way And it's like how did the Internet change your life? It's pretty ecology But yeah, and then there's there's a lot of other stuff that you kind of get to grapple with so You know if anybody of you are thinking about going to school or have kids That are thinking about going to school. I would highly recommend going here if for none of those other reasons I was introduced to this while I was there and Lauren McCarthy who spoke earlier she kind of Touched on this probably better than I could ever But Casey Reese who's one of the professors at UCLA he He teaches this class called interactivity and he uses processing to teach that class And so this is this is what it looks like It's just this kind of sandbox environment where you can write little instructions You hit the little play button in the upper left hand corner and then a window pops up with you know Your drawings and you can do animations and a lot of really crazy stuff the one class that really Kind of got me hooked on programming. I had done some stuff before but I hadn't really dug dug deep into it and really understood the potential of it was this class called visual music and The premise of the class was a 10 week course where we were given a song That was like not like a pop or rock song But something a little bit more avant-garde and we spent the 10 weeks kind of trying to emotionally distill what The song was about and then write a program that would run in real time that would represent that So to embarrass myself as this seems the nature of this conference Gonna show you my first programming project also So Yeah, over the course of the 10 weeks, you know, we learned about four statements and arrays and classes. I didn't use any of those Basically just a really long like maybe like two thousand line document of just if statements like if Middle East is greater than this Kind of draw this thing and like copy the function over So yeah, you know it kind of changes scenes and progresses I'm not gonna show the whole thing because There's a lot of stuff I want to cover but This this was really seminal and kind of like making me realize that I I was interested in animations growing up But I was kind of discouraged by hand cell-drawn animations because it's it's such a tedious process and This particular project was very tedious because I was handwriting everything But I realized the potential that you know, you could abstract a lot of this stuff To to create like systems and rules to kind of dictate your animations So after I graduated college After I graduated from college, I continued kind of messing around with this idea of visual music messing around with Processing these real-time kind of effects. So taking in audio data or using computer vision Exploring different techniques, but then also exploring different aesthetics different compositions different color palettes things that I thought You know were interesting or appropriate for the various songs so I want to show just a little montage of Stuff that I would work. I was working on for three years In my free time if it looks so these Google Docs slides. Oh my god I Have everything else loaded Okay, here we go so the format is that The label is the artist that I'm making a visualization for and the year that I made it Okay, so compression was kind of tough on that, but hopefully you get an idea of You know kind of the ideas and techniques that I was kind of grappling and during that time I did learn how to properly use arrays and classes and also leverage GL and it was it was a lot of fun and I kind of kept a blog about that and kind of Took took a break a little bit from these visual music projects because I had an opportunity to work at Google and What I thought would kind of really refine my programming chops So kind of fast forward a couple years and this nonprofit organization That's an art and technology kind of community center in San Francisco approached me and was like hey We've seen your blog. We've seen you make a lot of really cool music visualizations Would you be interested in making one for the show that we have coming up? they do events and workshops and Have exhibitions and stuff. It's a it's a really cool space and opportunity and I couldn't say no and Kind of in the discussion what they wanted to do was something really collaborative And kind of a lot of the previous work that I had was leveraging a lot of FFT Trying to do like beat sampling and all this like kind of algorithmic stuff And in in these discussions realizing the collaborative collaborative aspect We kind of realized that okay. Maybe I should should try to not use those Maybe there's just a way that you know, we could really put the person in the forefront of this so I'm gonna try to demo this two-year-old three-year-old software right now Hopefully I can exit this Slide show Okay, and so as you can see it's running in the browser This is to a song culprit the artist's name is culprit Here we go If it loads Yeah, that's the software, but actually the setup for it if I can go back to full screen It was actually set up on a monome Which is a MIDI controller with a bunch of buttons on it And so we put it in the center of the audience and let anybody mess around with it I wasn't the person that was in charge of making the visuals the audience was And this reception was like incredibly profound to me Because nobody had ever been like oh my god. That was like amazing To my work before so I kind of I started thinking like wait a second Maybe this project isn't done with this format so kind of I took a break from that for a year and I would you know I listened to a lot of music when I do work and I thought you know Maybe I could just hit an artist up and see if they would be interested So this is a band lulatone. They're from Nagoya, Japan and I didn't I didn't know them prior to Emailing them. This is the email that I sent on November 27 2013, which is basically like oh my god I love you guys so much. I Do have a job so I could pay you But hopefully you'd be interested in collaborating with me and making sounds to you know create Kind of a standalone application And they responded it was it was kind of the start of a collaboration Which was super exciting, but I didn't realize that this was really just the beginning of a collaboration So we made this app called pat a tap You can probably maybe you guys have seen it I won't take too long to demo it but basically every key on your keyboard when you go to the app is a Different sound and animation. So this is Q This is W And these are like all of them together Yeah and So we started working on this the process was you know He sent me a bunch of sounds and I was like oh my god these sounds are incredible And I was like these animations I made are not cool enough Like let me make more and we kind of went back and forth for about six months Making stuff and then editing stuff down and trying to reduce something to what we thought Would be an octopus no Would be something akin to an acoustic Instrument we wanted to make something that was easy to pick up and play but was not necessarily like a DAW that you would be That would be kind of your whole creative suite for making something. This was just a little piece You know kind of like Jeff's talk at the beginning This this is one piece in a very big ecosystem of creating music and we were hoping to help You know a small group of people make stuff What we didn't realize is that this thing would blow up This is this guy He's he's the most famous vlogger in France. His name is Cyprian. I think I'm pronouncing that right and like Yeah after after he sent this like the servers crashed and everything kind of went Hey wire. I was on the front page of Reddit and all this stuff and It was it was so great to see you know people really like it But it was also really interesting to see the different types of ways people make something that you know We had been basically collaborating over the internet for six months, and I had been kind of working on this thing for a year It's like holy crap people use stuff a lot So one way was that you know people just tried it out Another way was that people would try it and then they would share it and you know Individuals from our communities the music community the web community other people would share stuff But also entities in organizations would share stuff and I found this really I find this interesting in general about social media is that Entities specifically, you know They have a voice and usually one person is in charge of of their PR campaign or whatever so you get really interesting stuff like this like wired Published an article about pat a tap and it says get this rad portable animation whatever and it's really small You can't really see it but venture beat at replies like I am so unfollowing you for that gnarly animated gift And then wide response and oh Exclamation point which sounds like so out of tone for an entity like wired It was interesting to see kind of like oh, there's a human behind that, you know and likewise Unfortunately social media also kind of automates and roboticizes individuals sometimes this Jason David Unfortunately took the embed code and just stuck it into a tweet So that's that's our bad for making a bad UI that wasn't explicit like this is for a tumbler or a blog or not So yeah, we we also saw you know these entities right kind of long-form articles about the work There's this this was an awesome article from kill screen that was basically saying how these ungames There's a huge corpus of them pat a tap was one of the ones in this article are really great for kind of jogging your creative energy We also have a merch page if you go to the site pat a tap There's like this UI at the bottom if you hit merchandise You can actually buy artwork that we make Lulatone has a bunch of albums that they produce I make you know print posters And then you can also buy we have an iPhone app and an Android app Which I also want to promote we used this thing called mobile Chrome apps And I don't know if you guys are familiar with me in Chrome extension But basically you make a normal website you package it with a manifest Jason And that becomes a Chrome extension So this this group at Google has also created a command line tool that you can take that same manifest format And it creates an Xcode project and an APK for you And it was dead simple Well The the actual like coding part was dead simple the the whole other like Authoring and like key signing and stuff is like a huge pain in the butt, but So yeah, if any of you are interested in making apps and converting your HTML5 apps or whatever I highly recommend it. I'm just gonna try to skip through this this guy Robert, honey actually Use pat a tap with logic pro to kind of create his own music Anyways, you can wait for the single I think it's coming later this fall And then also in other communities We've seen tons of stuff on SoundCloud and Instagram if you just search for pat a tap or do a hashtag search It's crazy to see most of it is just people messing around But occasionally like this magazine Animal New York made like a really sick beat with it that then another rapper used As a sample to rap on top of it People also do really crazy stuff with hardware. This is this woman Camille Martin I think from France and She had her thesis project was there's an Arduino that had this box where you could touch it and kind of use that as some kind Of input and she hooked it up to pat a tap so you could make you know pat a tap beats with her I think it's called a mysterious sound box And then you have people actually augmenting our code. This is the craziest thing about the internet This guy basically wrote a little bit of JavaScript that I'm just gonna Get copy and then paste into the console of of the site and now we have a sequencer And now I can just you know make beats save them and play them BPM it's really crazy to see Like So yeah, and then finally we actually we've been invited for installation so Museums and art organizations ask us. Hey, was it okay if we use this thing and install it in X and we're like Yeah, we don't have to do anything. That's rad Yeah, if you could just send us a video. We're happy to share it online or whatever So this was in Slovenia in January. They had it set up at this music festival meant And these are the different places that it's been over the world physically installed You know, it's really crazy to think you make software and it's stuck in this screen But with the help of all these people and organizations, we've actually been a lot of places Like physically a lot like here. I'm standing in front of you probably because of this project So this is the this is the first time I'm actually sharing any of the analytics This is the time spent on pat-a-tap in the first month and you can see there's like a huge spike for the first 10 seconds There's basically three quarters of the traffic look at it for 10 seconds or less But if you look down at the bottom almost 10,000 sessions Use it for half an hour or more Which I think is a scarier number um But I wanted to juxtapose this with what I was just showing you because these numbers this data is actually less Relevant than the kind of interactions and discourse that I just showed you the people that try it They share it. They review it. They purchase it. They support us as artists. They make recordings They do music production. They augment it with hardware. They augment the code or the install it They do so much cool stuff and that is way more powerful than any You know million milestone number in my mind So this project kind of sits somewhere in this dichotomy of expression and utility and one other way that we've seen that is through a lot of email discourse literal discourse So I'm going to share some emails with you this one and this helps if you're actually following Along in the dock because I'm not actually going to read these but this is from Ukrainian electro acoustic musician who he and his friends have this like jam session every week and they use pat-a-tap once And it's amazing because this was around the same time that kind of really heavy stuff in Ukraine was hitting last year and it's it's cool To see that, you know, we could play a part in kind of helping Relieve at least a little about tension for people that are actually there People have different backgrounds. This was a filmmaker from New York City He was like, hey, I want to make a video of you using pat-a-tap. So we made a video. It was crazy Sure, why not? A lot of different interests. We got a lot of emails from students. They're like, hey I'm an illustration student. I'm a musician student. Is it cool if I use your samples? Is it cool? How do you do this thing? How do you get started? kind of doing a lot of just sharing knowledge sharing has been really really fun and then randomly we also get like occasional economic Backgrounds somebody like the day before my birthday last year just sent me $500 and it just says thanks for making pat-a-tap Sweet, thanks, dude That's really great because our server costs were actually kind of expensive because I'm a bad programmer or something So yeah, and then we've also been in contact with other industries a lot of record labels have been in touch with us because they want to sign us to do music or You know do some other kind of advertising based website. It's really opened up a lot of opportunities for collaboration So yeah, kind of going back to these hard numbers versus this qualitative stuff. Here's more. This is the last year in traffic It's normalized. So you don't know how many exactly but basically, you know, it doesn't to me It doesn't tell that much except that those are the two times we're at the front page of Reddit So red it really is the front page of the Internet, but other than that it's you know, it's really Kind of these other types of discourse interactions that we've had that have been really powerful But those those don't stop with that project actually I Use a lot of tools in order to make a project like this Some of those are off the shelf. I use bourbon and jQuery people I don't know why but they like really hate me for you still using jQuery I think it's a great library and actually the guy who did the sequencer wouldn't have been able to do it if we didn't have some arbitrary event binding system like the jQuery events that you could spoof events, so Shame on you. Who hate me for using jQuery. Um, I also augment libraries. There's this really cool library It's hard to see here, but it's it's called tween.js. There's a lot of tween.js This one is by github user sole and it's kind of Formatted from the flash days, but I've kind of forked it and augmented it a little bit because the syntax I like a particular set of syntax and Then I also make custom software and so I want to introduce 2.js It's a it's a rendering environment So all of the drawing the shapes that were drawn are using 2.js It's a scene graph, but you get to what's what's unique about it is that you can choose SVG canvas or WebGL As your rendering system and then also the API is something that I find is much more friendly than more performant libraries like Pixie or something like that So this is the website It kind of gives an overview and then kind of initial first steps This is how much the code is you know to make this thing and then it also has just documentation for all the methods and Properties and stuff and then we also have a page that shows the projects and examples and Yeah, if you have any questions go to the issues on github We have a stack overflow Hashtag, but I don't get an email when that happens, but I get an email when the issues are posted So feel free to post questions there These are the examples they're kind of based off of 3js I do Contribution on 3js. It's kind of how I learned a lot of the stuff that I know about rendering and Yeah, I won't go over these But I will go over. Oh at the bottom There's more like a list of more utility based examples also all these examples are less than 150 lines of code I made a big deal to like make it kind of bite-sized not like here's a thing and it's two thousand lines. It's massive So there's one this one down at the bottom blank. I actually use this a lot I don't know if any of you guys use Python I find the console with a loaded library very similar to ipython Basically, I have an instance of two that already exists so I can see what this is if I hit period I can see all the methods and properties that exist with it So I already know the library so I'm gonna make a rectangle. It has a bunch of make methods I can put it in the middle of the screen And I'll make it 50 so there it is and the rectangle now you get that return because it's a scene graph You have this object that persists from frame to frame So it has a bunch of properties, too. It has a translation Let's move it over 50 pixels. It also has like a fill kind of normal CSS stuff that you'd expect. I really like this pink right now Yeah, I don't like the stroke. I don't know why I have default stroke. I followed illustrator Anyways But What was I gonna do? Oh, yeah, so it also has a matrix so we can rotate this 45 degrees And if we kind of Bring that down if I kind of added addively do it we start to see animations And so we have a function that you can do that programmatically instead of having to do it Every every frame yourself. It's hard Programming when you're nervous So now it spins So that's kind of the basis of it and you can see that Patatap is heavily influenced by the kind of structure that this framework provides One thing that it is really good with that. I'm that I haven't showed that maybe I should show an example Is it's really easy to morph vertices? So I have a I have a notion of where your verts are all the time So if you move a vert I got you covered and I have a method that Keeps your curve, so you don't have to calculate curves, which is really cool and helpful for a lot of these more animated stuff So yeah, why would I make a tool when so many other people make tools and organizations really? And I'm just a dude One reason is for edification. I know the W3C spec on SVG canvas and WebGL so well right now It's it's not even funny The other is community participation, so this example is actually really cool this guy I had initially written the children as a map and he made it an array and wrote all the unit tests for it It's like what the awesome. I just have to hit okay. That's amazing But what I really mean by community participation is that I've learned I never took a computer science class I've learned everything that I know just kind of from the internet and also from Casey Reese and processing in that that community and It's one way to give back kind of the knowledge that I've learned over the last ten years really because I've seen so many people that have That they should have had the opportunity that I had but for whatever reason I'm here and they're not so part of it is for for those people and other strangers that are Interested to learn and I'm happy to help and answer questions in any way that I can So this is an example of good participation This is an example bad participation somebody's basically like hey, this is cool But it's broken and I'm like oh, okay. What's the operating system or browser you're running? No response Helpful, and then there's just the ugly. I got this from cam JG 71 This is really boring and then a few minutes later I get an email notification. Oh, she updated her post What did it say? It's no better And then the last thing is is Reusability so over the course of working at Google and then also making side projects for myself I've written kind of from scratch Rendering systems and also a lot of the other stuff that makes kind of certain things really really fluid and nice Particularly this one called fit to window takes a DOM element stretches it to the canvas make sure that there's no scroll bars and Reduced it to a full-screen parameter, and it's a lot of kind of like over the course of Using or writing something a lot. You're like, yeah, I can make this a lot easier Which is something you can't do in a lot of industries And it's it's pretty empowering to try to tighten tighten your loop So yeah, I think these kind of three properties have a have a really good Kind of symbiotic relationship that help you a understand the content and technology that you're working with and result in a in a more kind of Tackle approach in in how you make content and express yourself Through your work And then to kind of like cap off the section, you know I bring in this graphic again because it's I have a whole other set of dialogue and discourse that I'm interacting with people on You know we have this stuff from pat-a-tap, but then the engine that runs pat-a-tap people are actually making other stuff And if I had more time I could spend like an hour just showing off the stuff that people other people have made but you know we have we have different types of discourse and Kind of want to end with you know discourse is At least why I Work in this tech in this medium. It's a it's a way for me to talk to people It's not like I'm sitting here talking to you. I'm not a very social person, but I am deeply interested in humanity and You know at at in any crafts kind of Oh, I'm sorry go to the slide at any crafts Kind of like highest point I think it's it's an expression an idea that sparks discourse and and that's something that I'm super interested So whether it's rename agreed that paints a painting or my friend Camille who creates a digital version of the pipe or my friends fog these two dance a version of the pipe or My buddies echo the curiosity that do fashion design that kind of represents the pipe or my mom who writes a haiku about the pipe or My friend who does kind of high-level microscopic photography about the pipe Or my other friends who just likes coffee Or typography or me who uses code does anybody know what this will look like when it runs And with that I want to say thanks