 So that was a wonderful performance of dance we just saw, and my presentation will also be about something that's normally silent, I guess, rather than music. And this is about solving problems in some new ways. And I call some of these ways live-solving. Here's an outline for what I'm planning to present to you, a brief introduction with a video that I found on YouTube that just fit the theme perfectly, a brief introduction to the classical theory of problem-solving, I hope I have time for all this, and a brief review of a system we developed called COSOLVE, and then I want to talk about the actual subject matter of live-solving, of which I'm offering several different types, but I'm really going to focus on one of those types that's drawing the paths that represent solutions to a problem, and then some discussion and so forth. So here's the video to kind of introduce this topic. I met Mr. Rubik about three months ago in Budapest, and so I realized how much of an inspiration he has been to the world. So these young people are solving Rubik's cube problems as a performance in competition on this stage. So this is a type of problem-solving that is performance-oriented. OK, so the point of this is that we don't always solve problems simply because we want the solution. Sometimes the process of solving the problem is the end goal, and so I want to talk about that process and kind of show its similarities to live coding and also present a particular technique that I'm presenting here for the first time. Before I do that, though, I want to base the particular kind of problem-solving in a theory. I call it the classical theory of problem-solving. It's perhaps better known in AI as state-based, search-based problem-solving, and kind of goes back many years to some work done by Newland Simon on a system called the General Problem Solver. Once again, I'm showing you my age because I was actually alive when the system was created in the late 1950s. But here's a little bit of theory. I know perhaps this is not the right place to show mathematics, but there's not much of this. But if we describe a problem in terms of three components, a starting state, a set of operators which represent methods that can be applied to transform the state, and then an indication of what a goal represents, some kind of predicate that can be applied to a state to tell you if it's a goal, then we have a problem that we can solve using this particular methodology that some people call states-based search. Each of the operators or methods that can be applied to transform the state has two parts. It has a precondition, which is a condition under which it can be applied. So you can test a state and say, okay, yeah, we can apply this operator. We're allowed to make this move, et cetera. It has another part which actually transforms the state assuming that it's applicable. These implicitly define a set of possible states, the set of all reachable states from the initial state by applying over and over again members of the set of operators. So sigma represents this state space or problem space, as some people call it. So here's this sort of diagrammatic representation of the problem space with the initial state, sigma, so not up at the top, and then perhaps it's tree structured because at each state we can apply a finite set of operators. It might not actually be finite. It might have parameters where we can have floating point values and so forth for these parameters. In any case, we have this state space that's defined. For example, how many of you are familiar with the towers of annoy puzzle here? Okay, probably at least half of you. It's commonly taught in computer science classes. It's this puzzle where we have three pegs and a pile of disks on the first peg. And by moving these disks from left to right and sometimes right to left, we have to get all of the disks piled up on the third peg and you're not allowed to put a large disk on a small disk, etc. So there are some constraints. In this situation, the initial state is diagrammed like that, and then we have a set of six operators in this case, move from peg one to peg two, move from one to three, etc. And this set of goal states is just this single set sigma. So that's an example of how the definition of a problem plays out for one particular well-known puzzle. So here's the system co-solve that we built to kind of explore collaborative, deliberative problem solving with this theory. So instead of having an agent, a computer agent doing the solving, it's humans that do it and do it in a collaborative web-based environment. It was a successor to a system called TSTAR that was done in Python as a sort of single user system with the possibility to share trees and so forth. So co-solve kind of took that transparent state space search approach and put it on the web. So let me show you a quick video of some co-solve activity. This was done by my former PhD student, Sandra Fan, who's now at Google working on other web-based things. But in this particular situation, we see two solvers operating in the same solving session starting with their initial state for the Towers of Annoy problem with three, this is the version with three disks. And Sandra is moving some disks from left to right. As people typically find early on when they're trying to solve this puzzle, you get stuck if you just always try to move things from left to right. And so in the meantime, here she is annotating a state where she says, oh, I got stuck. This was not so great. But in the meantime, I've been editing this other branch of the tree. And collaboratively, we decide that this is a better approach and so forth. So this was co-solve, but I found that the amount of latency in applying operators in co-solve was frustratingly high. And I was looking for an alternative way to kind of create a problem solving experience for users that would be a little bit more like that sort of very live kind of solving we saw in the video. So let me show you this first form of live solving which is using sort of drawing gestures to move through the space of possible states, the state space in order to possibly generate a solution or perhaps just to explore the space. Normally one sort of looks at a state and figures out what the possible moves are and then makes the move and so forth. But in this environment, we're going to use the state space graph, the problem space graph as a representation for actually interacting with the puzzle. So in this example, we've got the towers of annoy with some number. This is not actually accurate because there would be many more states if we actually took all of them for this particular version. So the start state has been set up over here in the lower left corner. The goal state is the lower right and then there's this intermediate state which is sort of notable because it's very much like the other two, but it has all of the disks piled up in the middle. And these sort of states span this or sort of anchor this graph that's laid out in a particular way that I'll talk about later. And let me see if I can now bring up the demo here. Find it. I think it's in here. Sorry, that's not it. Must be in here. Okay, here it is. So in my live towers of annoy here, what I have is the state space graph on the left and the particular state we're looking at, and this is the initial state on the right, and if I just sort of move the mouse over the edges of the graph, I am tracing out a path on the graph and at the same time making the moves in the towers of annoy. So let's see. Is this going anywhere? Not clear, but it's all right. So this is what I'm calling the first form of live solving where through this process, and this particular graph has been pre-computed, but one could well imagine that the graph is being computed as I move through the space, so that in some sense it could be live at level five where it's anticipating possible continuations of the solution path and then offering these continuations. Oh, I have to back up here. I've got to a dead end, and so this is an example of an error in solving that becomes part of my performance just to carry on the theme that we mentioned this morning where if you make an error, it should be part of the narrative for the performance to be successful. So I am improvising here and making mistakes perhaps on purpose. Let's see, where should I really go? I probably should have gone this way, right? It might have been possible to get out of that, but okay. Actually, I don't need to go through the rest of it. Let me go back to Keynote. So one of the challenges of making this available is to come up with an appropriate graph layout that has the right sort of affordances for drawing on it. You don't want states piling up on each other, and you want them spread out, you want them arranged in some intuitive fashion, and so you've got to come up with some sort of mapping that takes a set of states and puts them into the plane or perhaps into a three-dimensional space. You might also want to render each state with a little graphic instead of just a circle so that there's a little bit more sense of orientation within the graph. So clearly one of the challenges of providing this sort of facility is coming up with those visualizations. They have a number of advantages besides supporting the live solving. They give you a shared representation. You could have multiple people looking at this representation, drawing different parts of the graph, and a basis for communication. I'd like to show you one other example. This is the missionaries and cannibals puzzle just to show that the methodology isn't completely limited to one problem. I need to go back again. Whoops, sorry, that's not it. MC. So in this particular puzzle, the goal is to get all of the missionaries and cannibals across the river subject to some constraints. One is that there has to be a missionary that steers the boat. At most three people in the boat, we must never allow the missionaries to be outnumbered by the cannibals either on the left bank, the right bank, or in the boat because disaster will strike. The missionaries get eaten. Or if you prefer the politically correct version the other way around, we should never allow the cannibals to be outnumbered by the missionaries for fear that they would be converted. But whatever your version is, this methodology should be applicable. So one of the challenges in this particular problem, again, is that you sometimes have to do counterintuitive things, sort of moving backwards in whatever space that you've used to lay out the states. And in this case where we have sort of progress for the missionaries on the x-axis and progress for the cannibals on the y-axis, sometimes we have to move people back in order to make it all the way across. So there's a solution. One of the challenges of the visualization is that using only those two attributes, numbers of missionaries and cannibals, there are states that really should be separate that would otherwise pile up on top of each other and make it difficult to specify the path. So we have to have other criteria or perturbations or something to avoid those collisions. Okay. So back to keynote. And the method used to generate this particular visualization was a bit different from that used in the Towers of Annoy. Here we had two independent heuristics used as axes, the values were placed on axes that gave us a way to span our problem space. And this is just an example of another way of doing things, whereas the Towers of Annoy approach was using landmark states plus a certain kind of distance metric that could allow you to find a distance between two states, especially an arbitrary state and the landmark states, and then use sort of generalized barycentric coordinates to place every state on the plane. And I won't go through all the details here of doing that, but this approach could be extended for other problems. Oh, here's a point which is if you change the distance function you're using, the solving experience can change quite a bit. So the golden path here is a solution path, but now instead of using a sort of weighted distance based on how many discs are different and how big they are, we're simply counting the number of moves from the goal and from the initial state as the distance, and then suddenly the graph that we get uses a straight line for the golden solution path. One more example, this is also using the landmark method of layout where we have this is the famous blocks world program, or problem sometimes used in artificial intelligence courses where there's a pile of blocks, CBA and with these basic moves we have to robot as opposed to get them piled up ABC and you're only allowed to pick up one block and put it down and so forth. And this is a graph you could get for that, but it also suffers from potential collisions in layout. Other ways of laying out spaces have been suggested by people like Poo and La Laan at Lausanne, Switzerland where they plot state spaces by doing partial breath first searches and so forth. There are a bunch of other forms of live solving that I won't talk about today, they're in the paper and anyone who's interested can take a look there. But I want to kind of close out by talking about the question that people often ask, well is this stuff any good for real problems instead of just puzzles? And I'd like to claim the answer is yes because it gives you ways of thinking about these real problems even if you don't find the solution that you want. But a good problem is I need to skip ahead here. Sorry. Could live solving contribute to the solution of a significant problem? What's a good example of a significant, say, wicked problem? One is climate change. There's a tough problem, right? Why is it difficult? It's multi-dimensional. There are lots of issues. People have incentives to impede the solution to the problem or deny that it exists. But there's plenty of evidence that it does exist and what can we do about it? Can we bring technologies like these to like live solving and so forth to such a problem? And I'd like to argue yes if you at least focus on key aspects of a problem like this, like the basic physics of heat. You can get something to work with and express it in a way that people can start to think about the solution to the problem in a systematic way. So here's the black-body radiation equilibrium equation and solve it for T and then this is the physics part of it. Then this is sort of a fictional part of it where we create these imaginary operators that might actually do something to change the situation. And there's lots of other literature about how you create those things related to game design and modeling and so forth. But in this particular example I did an implementation in t-star and then later did a visualization. I don't have an actual live version of this, one of the challenges is the layout. Here's the layout we get by using the second approach that I showed you to letting out where we take two particular heuristics or two particular variables in the model, in this case time on the x-axis and temperature on the y-axis is something I think we can all relate to. The problem by the way here is to start in 2015 and to make it to 2050 keeping the average temperature of the earth below 20 degrees C I think it is. And it's quite difficult but it's possible in this particular formulation of the problem. But live solving in this particular example would take more than simply the drawing affordances that I showed you because of the overlaps. We need ways of either navigating in three dimensions or higher dimensions in order just to specify the particular states that we'd like to. But that's future work. Conclusions. Interactive computing offers new ways to mediate problem solving. The classical theory offers one particular approach that can help ground some of these things. Live solving methods can reduce the latency between user problem solving intentions and the system responses and I hope or I'd like to believe that live solving has some potential as a performance activity as well as a way to help people improve their problem solving abilities. So I'll use QED. Not that I've solved the problem but okay. Thank you to the people. Thank you very much. Steve, do we have a question? Yeah, please. Just wondering what problem do you think we're trying to solve in these live coding concepts? Well one problem is you know like Sam was doing last night, he was trying to get people to dance. I don't know if he was, that wasn't a big problem perhaps you solve it by trying different things and you try them perhaps in some systematic way and that's certainly a performance related problem of trying to get the audience to respond in a particular way. Is that what you're looking for? Oh yeah, I'm being a bit facetious. Yes. Any other questions? What is there another way of looking at the live code that I might want to think of with the problem as something that they study beforehand and then the resultant solution is going to work? Is that one of your implications? Yeah, yeah. I had a start formulation for a musical composition problem which I don't have time to show you but certainly a composition problem can be thought of as an arrangement of components that you could use as a framework but it could be any problem presumably and still potentially as the sonification would be interesting. Okay. That's just crashed. Shit. Bugger. That's just totally crashed, I'm afraid. Oh god. Oh, no. No. Do you want to know the question? Yeah. One more question. I don't believe it. So because music doesn't give you a problem so you have to make constantly a problem like a goal to yourself for the next seconds of it but wouldn't you have problems so you have to make yourself Right, so one part of the problem is investigation. I know it isn't, but that's because I re-loaded it. What's left is an exploratory framework and so if you have a bunch of sonic affordances that you'd like to explore this framework to help you do that a lot of design problems don't have any clear goal except perhaps a set of criteria that you'd like to kind of satisfy as they say. Okay. So yeah, thanks for that question. Oh, my windows are on absolute crap. Oh, shit. Okay, thank you very much. I think we'll move on to some more practice in live coding. So we decided we're not going to do the dead mouse thing here where you just press play and everything goes smoothly. Sam said, no, don't use PowerPoint so I've coded up our presentation slides in my live visual coding language so it crashed as soon as we went on stage as you may or may not see the talk we planned. Fortunately, we don't have anything very complicated to say. We'll just deal with the software as it goes. So we're interested in practice based practice lead research and particularly interested in the question of craft. Fortunately, everything I'm about to say Jeff Cox said much more articulately in the previous paper session. So I'm just saying the same things as him but with simpler pictures. And I should say this is drawing on work from eight years ago with Sally Joan Norman which is amazing that she's here as the as our keynote for tomorrow morning. That's true. This is riskier than Jeff's talk. So what's special about doing practice based research and the problem is, as Jeff said it's a problem of epistemology, what goes inside of our head and this kind of practice based craft, there's a craft that's sculpture our claim here is that live coding is a kind of craft too and particularly the relationship that we have with tools is like the relationship that craft researchers have with their tools that's what we're really talking about. So this sculptor has got a chisel in their hand, the sort of knowledge that we produce and it's basically like we've got a hand in our head so how does I told him he should have looked at this before we gave the talk so so we're constructing knowledge that we gain through our hands and somehow we have to articulate it's in our heads there are other crafts that are far better discussed than live coding crafts like knitting for example so here we have knowledge that we're constructing in our hands and Alex has done some wonderful work both with weavers and with knitters so oh it's good he's arrived now because he hasn't so we're constructing we're building a structure in our hands and those things I said yesterday about experience they really they made physical in craft but the problem is turning this into knowledge because oops we crashed whoop okay so if you've just got a big tangle you look at what you've made and where did the knowledge go the stuff that's in your hands so this is the question for our paper really is how do you get knowledge out of craft based practice led research well we have an answer for you of course to the big question and I just press play that's the answer is that we are looking for some kind of rigor that's the difference between just doing craft and doing research in craft is that we have to have a process of structuring it and formalizing it a way that we can talk about it in a rigorous way that's why you would do it in a university rather than just doing it on stage or in your craft workshop so what does that look like I just have to get past a bunch of other stuff here that generated that amazing animation that's right okay so here is the rest of the talk the structure is basically this we have some rigor consists of four things we're claiming first an awareness of the previous work from which your own craft interventions are inspired secondly having critical questions in relation to that previous work so you just don't simply forge ahead you forge ahead with reasons thirdly we claim that the kind of toolmaking we do is an essential aspect of our craft and that exploratory implementation as an engineer might call it or hacking as many of us have called it during this workshop is essential and then finally being reflective and thinking using our making as a basis for thinking so with some more cunning plans we can move out of that mode and deal with the first one first previous exemplars and here's a chance for Sam to say something hi and here we're talking about overtone and Andrew Sorenson's extemporary so we're trying to find an example of these things is interesting so here's an example that I was quite pleased about because it's allowed me to express which is Steve Wright's canon everyone know what they are? a simple pattern of notes here's my notes here E up to 4 F sharp 4 B 4 etc if I play them one after another and then repeats that's one person and if the person plays the same note just slightly faster or just slightly slower they'll start standing like they're playing at the same time but they'll go out of phase and then back in phase one note off and then round and round and round to eventually come back to the same point again and so this implementation expresses exactly that in code because it used this idea of this temporal recursion so Andrew talks about this in previous papers but the idea was lovely but it's one of those ideas where you have to really sit like this for a while I wonder how that works how does it actually work so I've got a function and I'm taking parameters and I'm destructing the parameters and I'm doing an F statement here if some condition is true like if I have a note to play then schedule it to play in the future and this is the really weird bit so then the function asks itself to be called in the future when it needs to play the next note so it's in charge of scheduling itself so there's a bunch of machinery which happens, which does this and then here we say well that's actually, let's fire two of these guys off and fire them with a very just slightly different time between the notes, that's your tempo and off it goes, it's lovely so this is like the prior work for the sonic prior work but again itself taking off ideas from Andrew's work so I'm thinking how do I actually make this simpler is it possible to make this simpler mathematically that's quite nice and easy so the sonic prior solution great reveal is this so we have the same thing like the notes here but instead of doing all this recursion and scheduling and stuff, we just have like an intuitive solution and we have two live loops which are just playing with slightly different times and they're ticking through the notes I'm even able to actually specify it's going to be a short note so you don't even need these bits here that's just for aesthetic prettiness and so again like it's about trying to take these previous ideas understand them and figure out how we can rewrite them and rework them in ways that fit our challenges of what we're going to talk about next OK, next the critical questions so and here Sam is going to describe what critical questions arise in the making of sonic pie tell me before I talk about sonic pie actually you ought to use the where are we where has that web page gone I just closed it the thing about the wire is that you want to get it back doesn't matter the critical goal there was to ensure that every aspect in the second solution I can imagine teaching to a ten year old that was like a fundamental question I wasn't going to put any features I don't put any features in the sonic pie unless I can imagine teaching to a ten year old and that concept is something which I've got through a lot of experience going into schools so I'm not necessarily like the world's expert on ten year olds but have a general test that sort of works so all the concepts in there pass that test and the contemporary solution for me don't pass that test I still struggle to imagine how to teach those concepts to ten year olds and then learn myself so what am I talking about now I've got sonic pie I can make a tune so I think you're going to say something about how the challenge how you've architected this language yeah okay so you've got a weird keyboard there now okay so this system this system the whole thing about being really simple and about demonstrating the simplicity of these things, the critical challenges communicating these things operations at the top are simple, very bold, very obvious this preferences thing by default is hidden and then you've got two things you've got text to put in and words to put out and then the goal is again what's the simplest thing you can teach a ten year old and how do you actually also what's the simplest thing you can teach a teacher and so the first program you write in sonic pie is a word play and then a note and then you run and you hear a beep there's nothing loud but there we are and so again this fits those exact same criteria okay that was exciting so the goal here is what is a way to express the first program you could write that would make an output that you could actually engage with and understand and appreciate and also be able to tweak and play with and this is the first program you write a word play and then a note and then you don't even think about the notes talking about those numbers and you can talk about the fact that they can go up so 80 and down and off they go and then we'd also talk about the fact that if you want to play chords you can actually play these things at the same time so this is actually not obvious to a lot of people but it's there because we're actually changing the way the permanent language works a little bit here but we've got no time in between two things that's actually played at the same time note and chord and so we need one more bit of semantics to describe melodies and that's sleep so I'm going to sleep for half a second play that note and sleep for a second and play that note and again here we have this ability to express most of western notation just two commands which is pretty cool do you want me to say anything else like structurally or I think you have time to say something else yeah okay so the other thing the final thing was that this is also specifically for teaching computing curriculum but we soon realise it's much more than that it's much more fun to play and to make music and so one of the goals is to use the computing constructs to be able to do interesting things so loops are obviously quite useful because I can sort of get this thing and repeat it forever right and that's how we can make dance music that's all that is I just replace that with drums but the problem with that though is that once I've got a loop there's like black holes you know they don't allow you to modify them they just the control flow stays in there until you stop it and so again an interesting thing is to think about how can we make something which is as simple as a loop but also offers a really interesting sophisticated interaction sort of paradigm which is that notion of live coding that's why we're all here so how do I turn that into a live coding thing well I could use a temporal recursion kind of thing which is what I overturned and did but I invented this thing called the live loop and you just put the word live in front of it because that's what we're doing and give it a name Bob right or like Rachel and then and it's cool actually you can explain to really kids really easy say well I've got a band and I've got a drummer Rachel and I've got the guitarist Joe right so there's Rachel she's playing the she's playing the drums actually she's playing the guitar and then I've got a Joe playing the drums and off they go at the same time so two live loops this notion of live loop is not more complicated to a ten year old to look at than a loop but actually it's really sophisticated because I've got this notion of concurrency but for these things happening at the same time I've got this notion of being able to update them so whilst they're running I could ask Joe to play, you're not playing loud enough Joe could you play louder please so let's change him to be louder well you're too loud too loud to do that so quieter so just by changing the parameters pressing run again we have this predictable behaviour and so and again this is really easy to explain to ten year olds and I know this because I go to school and they're all doing this and I haven't been taught this it's just amazing to see great excellent next so our critical questions are about pedagogy about cognition and about curricula and things so we're going into these things with with an approach to previous systems like scratch logo and how do we go about this we build stuff to see what's going to work and so the palimpsest environment that I'm stuck in here is just to prove that I actually coded this stuff somewhere here so here's a chunk of palimpsest code I spent nearly a year creating it so it's implemented in about 60,000 lines of Java and all implemented in IntelliJ's idea environment which is actually like the kind of craft tool that sculptors have refined over centuries in order to make it easy to get what they want out of stone so this kind of environment, IntelliJ or Eclipse if you use that and these have been crafted by hundreds, thousands of crafts people showing the way that they think about software and the way their hands bring concepts out so when I was implementing palimpsest I let my hands lead me through the implementation of the system which is why it crashes so much but really making a lot of use of the tools like refactoring systems so I would say every class in this whole system changed its name at least once because it was only two or three weeks after playing it after writing it two or three months afterwards that adorned on me after using the system coding the system using the system I think actually now I understand what that class was, now that I've been working with it for three months now I really know what it was I was trying to do in the first place and the original name was completely wrong so this is kind of the experience that every hard working programmer has is that your concepts arise through working on the code and it's only over time that you really get a refinement of understanding what you're doing so this is really making knowledge through the work of your hands the final piece is about reflection on practice and Sam's going to say a little bit about the way he combines practice with reflection yeah so like there was a real cool challenge actually when I was working on the overtone for your system I was working on it with Jeff Rose I was imagining the most sophisticated system I could possibly build as a programmer because that would be obviously the key to the right thing to do what would give you the most power to write interesting music and to perform with it and of course let's say we're out of the room talking about using IntelliJ I used Emacs and then I modified Emacs myself because I was making my own tools so I'd get and also the tool I was writing to make the music overturned the tool itself and so I'd get very lost in the division between the tool I'm using to actually perform with and make music to actually express myself and the making and the honing and the polishing and the looking after and the cleaning and the tool itself I'm using to make the tool and so there was this real problem and so if I was trying to make some music I'd sit and start making some music and then I'd go wow it'd be really cool if the tool did that that would be much easier to do and then I'd go stop making music start working on that thing two hours later the whole practice session had gone and maybe I'd got this thing that I built often not and I'd waste my time and so what I really did very carefully with Sonic Pi is took a very many different practice which was first of all to work on the Raspberry Pi at the performance tool which meant that I was in a very constrained limited environment I really couldn't have didn't have all the nice editor environment that I would only have available that explicitly still to this day don't install those things even though I could and just make music with Sonic Pi in the practice sessions and then when I have ideas I have lots of ideas about I wish it was this so instead of stopping and working on those ideas I would just jot them down in a book I'd just have a little book next to me and actually I'd find if I start practicing and having got the book and feel anxious I need the book because the idea is going to come and I need to run it out very quickly and so here is actually a couple of days ago I've actually got a book before just using bits of papers to turn them away but I have a book and often the idea is to don't make much sense when I read them back and sometimes what these tend to do is these tend to feed into the development practice so the following day I'll take these ideas, read through them scrap most of them and then take the good ones and then put them into the do list to actually implement and so I would separate my activity between practice performance and play and experimentation which is specifically in using the tools to drive this asynchronous list of activity to then go into the actual development practice and that's made a huge impact really something I really recommend people to consider doing, it's been fantastic so that's our recipe for the way that we cope with the fact that we spend our days basically doing craft we don't have research questions we don't have research grants we just build stuff because it's fun and because we want other people to enjoy it and this is kind of how we think we turn it into knowledge that's that thank you very much one more question and maybe Diane would like to come and get set as well yes please practice based research is quite established lots of disciplines and there are obviously with models so I think most of me you might encounter a model of research in three practice or actually you might even encounter a model of practice as research so in a way there's nothing quite conservative about the practice based model but this you may even get the proposition of an epistemic object so not an object that is translating to a model but an object is itself a model so yeah I'm curious what you made of those ideas we are kind of situated within a specific tradition of practice based research which is practice based design research which is actually it's a real provocation in the field of human computer interaction at the moment and there are major debates in international conferences that are part of those debates we believe that what we are doing is not necessarily design research we think we are doing practice based find out research but we are in a technology department and it's a very strange experience to try and explain to computer scientists why we do the things we do which actually look incredibly undisciplined and we are trying to explain there's a place for us in computer science we have a discipline it's just a different discipline to the one you have so for example in the design environment the dominant model is a problem solving model of research that's correct yeah and in the find out concepts the dominant model is a question generator and how should life coding seem to oscillate between those two well our claim really is craft craft is it and this is the paper I wrote with Sally J Norman was trying to make a place for craft as the best model traditional model of accounting for what computer people do where they're in an arts context okay, thank you very much thank you very much we have Félien on copy paste tracking hi my name is Lina Ehrmans I'm assistant professor at Delft University of Technology and I'm presenting joint work here with Thijs van de Stormen and many people familiar with my work about spreadsheets were maybe somewhat confused to see me at the contents of life coding and I'm actually going to argue that spreadsheets are the most important or the biggest success of life programming so not life coding but life programming because I think this is a distinction and all the material we've seen yesterday was a lot of life coding and today life programming was a little bit more perfect and I actually went back to the call for papers to see if I was in the right place this is me on twitter right away if you're tweeting so it wasn't on the website anymore I like to put it back to look at it so if you look at the call for papers it has life programming in it but specifically in the context of programming languages so it says life programming languages here cultural things about life programming languages and here it's a little bit mixed between life coding and life programming it's called life coding before life coding historical perspective on life programming so these things seem to be sort of confused and who was actually at the workshop in San Francisco in 2013 a few people I was there too and their call for papers was also still online and they made this distinction really really clear so this is their website and they say this is life programming so life programming is the idea of abandoning edit compile run cycle in favor of immediate feedback so you type something in the computer and you immediately see the response that's life program life coding on the other hand are live audio visual performances that use computers as sort of musical instruments and I think it's important to reinstate this distinction between life coding which is a performance and life programming so I would say this is life coding and I actually really liked Alan's term of yesterday performance programming as well you're doing programming as an art form and this is life programming so here's a piece of code and if you change it immediately the result of the code will update without having to hit compile and those things are not very different in a sense they are related because suppose I would do life coding performance programming with a language like Java or C sharp it wouldn't be a very nice performance because every time I would change the code I would have to hit compile and wait for a very long time so the liveness of the programming language makes it more practical, useful or appealing as a means of life coding but those things are different and actually the reviewers don't even know this because one of the reviewers of my paper said the authors make a terrific point that spreadsheets are both form of life coding so well you shouldn't shame reviewers but I think it's not a form of life coding although it would be interesting to think about how a life coding performance with spreadsheets would look like maybe I should submit that next year but I'm arguing that spreadsheets are life programming and they are the original life programming I would say the most popular many people use spreadsheets for all sorts of things and they don't even realize they are programming so they probably don't realize at all that they are life programming but they are and the minute you type the formula in immediately you get the result without the added compile cycle so why do we focus on copy-pasting because that is what my paper is about helping people to cope with copy-pasting behavior in spreadsheets so if you look at spreadsheets this is a typical example where I'm calculating per student the scores for homework and classwork and an exam there's not really a way that I can repeat this calculation for all of the students the only way that spreadsheets offer me to do that is by copy-pasting the formula it's even in the interface so users really rely on copy-pasting to make up for the fact that there's not such thing in a spreadsheet as a student class with a field of homework score and a field of classwork score I can put my data everywhere so I really need the copy-pasting users can do that by dragging the formula down but it's even more prolific in the user interface because if you click a formula and you double-click it immediately your calculation is repeated so it's really one of the more ideas of spreadsheet calculation is copy-pasting and probably you've never thought about this but it's actually pretty magical what happens here because all the formula references are updated automatically and this is also the case if you take a formula and you copy-paste somewhere else in your worksheet all the references are still relative pointing to the same location so this copying is really one of the key ideas within spreadsheets and that's a way to reduce to make up for not having classes or fields or higher levels of abstraction but of course managing clones is really hard as all sci-fi fans know as soon as you start copying things you run the risk that one of your clones and whole body of sci-fi work on this one of the clones is always starting to have bad behavior and rouse up all your nice little plans so managing the clones is hard this exact same problem happens in spreadsheets as well and this is not a theoretical problem I did not just do this for fun because I'm an academic and I need something to work on but it's also a real problem here for instance a Canadian power company that lost 24 million dollars because a copy-paste error in Excel happens so this is something that actually happens people make tiny little mistakes to screw up to mess with the interface of copy-pasting so this is exactly the problem that we want to address with the solution that we have made called Xenagy that remembers what you have copied and updates the copies for you as soon as you edit one of course we could solve it in a different way we could ask users to define their data we could say do we do it to a spreadsheet anymore we ask you have to define this is a student and this is homework and this is classwork that will be a solution but then this starts to look a lot like access or even like real programming languages and that in a sense would break the spreadsheet if you introduce abstraction in a more formal way you take away the creativity and the flexibility of the spreadsheet and this is one of the things that people like about it they like to be able to put their data anywhere and make a calculation and then just drag it down and change it it doesn't look right let me do something else you want to keep the good things you want to keep the flexibility not take away the flexibility but you do want to improve and help people manage the clothes so that's exactly what our solution is about if you use our plugin copy your formula down and we remember what the origin of the formula is this whole origin tracking and this is quite a big field in computer science research so it also happens in normal programming languages where people try to understand what is the origin so we remember that all these cells were created not by typing but by dragging the formula down and then if you want to change one then automatically all clones will update so you don't have to define your function or your abstraction to the front but if you rely on the copying mechanism we will help you by updating all copies and of course that's not always what you want sometimes there's this one student that you made an agreement with they're a special a special rule for you because it's the reset from last year and you can still have the rules of last year so what you can do is take a copy oh wait, this first this also works if you have as I said if you copy a piece of data twice in a spreadsheet then it could be tracking also still works and near it's even more important because if you would change your formula here you can easily still drag it down but if you have the formula in multiple places the same one that's represented in the same calculation it will be even more cumbersome to change the formula and drag it down everywhere especially if you would have multiple worksheets if every year I do the same calculation and I want to update it I would have to go to many different places so the more clones you have the more you will benefit from having origin detection and as I said sometimes you have this one student where you want to do something different you can detach your clone from the copy so in the same fashion that you can take a formula and base it as values you can also take a formula and separate it from the clone in a new group of itself and then if you make a change to the clone it won't change with the other one so you don't always keep the origin but we say by default we want to keep the origin tracking because probably if all the formulas are the same you want to do the same updates to all of the clones so the system is improved and we help you make fewer errors by doing orange tracking but the life properties are not sacrificed we don't take away from your flexibility from your ability to just program whatever you want wherever you want and we don't add abstraction upfront but we help you maintain clones and in the context of some life coding languages what I saw yesterday is many people also rely on copy-pasting for a lot of things maybe something like Sam just showed where you have two pieces of code pieces of sound that you want to keep together if you would change one then maybe you want to change the other one as well so I see that some of the life properties of life coding languages could maybe also benefit from a similar origin detection where you have a piece of music and you want to slow everything down so you want to remember the same from the same line or the same thought and you want to change everything oh this was actually my final slide already that's it so time for several questions if there are I would like to challenge the initial assumption of the difference between life coding and life coding okay that's fine you put a picture of me forming using a system hybrid which is not a job it uses exactly the same thing that the pre-definition of life programming was so I don't see that distinction that was like what Alan and I were showing was you take an artistic practice it's a professional research and professional work there's actually benefits as well so I think the distinction between artistic practice and professional where artistic we're saying is like coding for performance programming as a business activity just in terms of swapping the functions around yeah yeah all of my coding systems I've seen are live programming systems let's see that distinction doesn't feel like a substantially optimal so this is not really my definition but from the organizers of the workshop but I don't think it says that life programming is supposed to be meant for business it's just the idea that you don't hit compile also we hit compile this has nothing about where does the term come from sorry I just want to show in terms of origin tracking where does the term come from live programming because usually that was called interactive programming in the old days and we always thought doing live coding was doing interactive programming with all these jokes and then adding a bit of we have the notion of dynamic programming and performative programming we've already introduced this idea of a spectrum of life today and yesterday as well so I would propose that we not make this distinction of terms between programming and coding but instead focus on the spectrum of how live are you and so maybe there are some systems which are more dynamic that would all be I would say in the context of life life can be an aspect of a language so the language can be more or less life and I know this doesn't have to be Boolean but then there's also still the difference between performing with code and doing it by yourself in the computer and those I think are too I agree but I would have like terms of synonymous I would just take Alex's call yeah I agree with that I think that there is I think we're talking about live in terms of feedback loops there's feedback loops between the programmer and the language and there's feedback loops between the programmer and the output and also in collaborations or performances with the around and it's all different kind of into a lot of feedback loops and I thought with the email that it organizes this and they were very accommodating in sort of this negotiation of how to live coding and live programming together and they ended up writing quite a lot of back-to-back but in the end it seems that it probably links back to the previous talk really how you start forward as serious in your own domain if you're trying to get grant into computer science and say you're doing live coding I'll do a google search and get all these people dancing so I think the difference between live programming and live coding is social or it's about community communities but once we have these different terms it's for the live people so it's been very accommodating in connecting with the live community because this workshop at IXE was very much about trying to bring these two communities together to help people that are live coding thinking about hey how does your interface as a programming language look like and what can you learn from programming language design for live performance and the other way around that made it interesting I was kind of trying to present myself to different people it was quite deliberate using live programming languages because programming and coding might do it's almost a class structure like programs have been encoded but to me that's anonymous but once you say live programming language it doesn't really make sense to say live coding language so that's when you focus more on the feedback between the programming and the coding and the aspect of the language or is it what you can do with it which is a performance I really like performance programming I agree with both of you I'm a son but I still need to work out a terminology that works for professional lives the other thing is all of our literature has confused terms every second paper I write I'll do live programming and live programming we're not being against I was one of the reviewers were you a reviewer or three and you drifted because you had a picture of me on the slide so actually we know each other pretty well so I was being intentionally provocative I think this is a pretty serious issue for us and it does relate strongly to the paper that I wrote yesterday Alex's point is exactly right it's about live feedback with the machine and it's also about live feedback with other people so in the work that Sophie Jane and I did we interviewed some prominent arts programmers including Rosswell and these people made the point is that they are artists but they are also serious engineers and obviously in this community we understand the challenges of engineering this stuff and we also know what it means to give a performance and we understand that we are performing when we stand up and show people our code that other community which some of them have been quite critical of us actually in a way that Alex is too polite to mention but there's been some slightly nasty exchanges that's community of people who understand engineering and don't understand that they're giving performances and you have to ask for example when someone like Brett Becta had an American of life programming languages so in his videos he is giving performances but people speak as though he's just an engineer and those are performances just as much as anything we do and they ought to talk about that in knowledge and think about what the implications are for their research yeah so in a way I'm quite sad that this nice mix that we had in San Francisco we so far engineers are I've known for a while and by the music people and it's sort of sad that we aren't here to learn from what this exact mix that you may have been about in San Francisco so you've come here you've given a great performance which is a good thing and in my review comments what I want to say is you know please talk about it I'm pleased you did okay so thank you very much on this live if you let's