 Can I start? Yeah, let's start. Okay. Oh no, well. We did just long enough for the laptop to work. Yeah, so please welcome Tobi. All right. So, hi everybody. For those who weren't here for the last five minutes of random rambling, I'm Tobias from the GNOME project. I'm a designer there, and I'm not really going to talk about that in detail. I'm going to talk about a more specific topic which is animation. Specifically, the ramifications of adding animation to interfaces. But before we get to that, let's briefly make the case for animation in interfaces. So, I think especially among the more technical, maybe people in our community, I think there's this notion that animation is this really frivolous thing that you can't do it, but it's like, it's so much work, and it doesn't really deliver any actual value, and it's kind of eye candy. And it can be that, right? You can just make your interface and then be like, oh, let's have it bloop-de-bloop somewhere. And that is one way to treat it, but I don't think it's a very useful way to think about it. I think the more productive way to think about it is to just think about it as adding the dimension of time to your interface. So, a regular interface is just a 2D surface, and then you do some fake 3D depth with shadows. But if you add time, you have a whole 60 frames per second between every state that people see, where you can add more information through animation that you use to tell people things. And that is not really something that is inherently functional or just visual, but you just sort of see it as an additional way to communicate things to users. And here's a very simple example from Gnome, where this is when you switch between two workspaces with no animation. What you see here is just this hard cut, and then you kind of try to look around, like figure out what happened, and then you look at the little thumbnail and you're like, oh, yeah, now I moved up to the other thing because if you do all that matching yourself, the same thing with an animation looks like this. So, immediately, this animation tells you that there is a spatial model here, things are above and below each other, and later, when you're not even in the overview, you can just use the shortcut keys to go back up and down because you learned this from the animation. And this is one of many ways that you can use animation functionally, but that's not really what this talk is about. What I want to talk to you about is sort of, if we do animation, first of all, what are the things that we want to avoid? And second of all, what follows from that? So, a couple of pitfalls that we're all familiar with, you know, like animation is great, but most of the time, like somebody gets really enamored with an animation, they're like, oh, how about we make it three seconds and make the user wait? And this is a great example, MacOS, where if you add a workspace, there's this button that you click and the thing moves all the way to the other side of the screen and while it's doing that, you can't actually click the button again. So, for these kinds of things that happen a lot in your interface that people do a lot, it really doesn't make sense to have very slow animations because people get it after a while. Whereas maybe if it's an animation that you only see once in onboarding, it's fine if it's a little slower. But so, yeah, generally make your animations fast. And then maybe a second common thing is like interface that go way overboard and like the problem with animation is that, or like one of the things about animation that you have to really consider is the fact that it's really good at grabbing your attention, but if you overdo it, like you're gonna get crazy fast. So you need to like make sure that you communicate the things that are important and you don't overdo it with your animations because that just like distracts from the content. So these are kind of the obvious cases. Let's get to the more interesting ones. This is the iOS home screen up until iOS 11 and this is the animation when you open an application. So what happens is you click the icon, it zooms in like the whole home screen, zooms in and all the icons get bigger, right? The whole home screen is now huge and the icon has transformed into the app. And so this is great so far, right? We know now that we are inside this huge space and like this is our app, right? The problem is if you go to multitasking, you get kind of an opposite animation, but now you're somewhere else and all that stuff that it just told you, like those icons being around you, that's now not true anymore. And if you go all the way to the side, there's another home screen inside the home screen. So, you know, like there are some inconsistencies there from a spatial perspective and things like don't really line up. And I think this example is indicative of a way of thinking that we've had an interface design for ever, really, because up until very recently, like doing lots of animations was not really feasible from a hardware perspective. And so it's fine if you're not doing any animations to just like design all your screens, right? And then think, well, maybe we'll add an animation here or there or something because at the end of the day, like if you just bloop, bloop, bloop between screens, there, you don't have to think about these kinds of things. And so like this screen-based kind of way of designing sort of works like this. You design all your separate screens, right? There's like our home screen, that's this grid of icons and then we have the full screen application and then the switcher is its own separate thing. And then at some point later, we think like, oh, how are we gonna connect these two screens? And then how are we gonna connect those two screens? But if you do that, you don't think about like how all those things exist inside one larger system. And that's, I think, how you get this inconsistency. I mean, I don't know how that example happened in practice, I can only speculate. But I think if you do animate all your transitions, at the end of the day, you'll always have to contend with space because in the user's mind, they don't think about screens. They think about the stuff they see on the screens. And so as a user, I see a thing and then I see another version of that thing. I expect them to just be the same thing, just transform. I don't expect there to be multiple versions of stuff. And sort of there's a whole bunch of implications that come from this. And so I think we need a new way of approaching, like a new process for designing interfaces that takes us to an end account if we're going to do animation. And this new approach is semantic animation. The difference with semantic animation compared to the process we've just seen is this. Instead of thinking about the screens as units first, you think about what's on the screens first. You think about semantic components that have a meaning and that have states. And so in this case, we have this app component that's like an icon or an open app or a card. And those are the three states that we have. And then we have a bunch of transitions. Those five transitions are all the sort of transitions that we will have to handle. And we think through that entire system. And then when we actually get to the physical sort of rectangles that are visible on the user's device, it's kind of obvious what the animations are gonna be. The icon is the same thing as the full screen app. So it just expands. The whole home screen doesn't have to zoom. That's not really necessary for this to work. And then when we go to the multitasking thing, it just gets a little bit smaller, right? But it doesn't affect everything else because that's just not necessary for this component to change the state. And interestingly, this is exactly what iOS 11 did. So now it's a little fast. But if you look on the part here, you can see that the home screen doesn't actually zoom. The home screen just fades a little bit into the background but it doesn't do anything crazy. And so on the multitasking animation, what happens is it's still there. So we have not broken the promise of, like in the other example, we are sort of keeping all of that stuff behind the application around. And all that happens is our sort of application component changes its state. And I really love this example because it's been like under all of our noses for the past five years. And they just recently fixed it and I'm so happy. So yeah, some examples. One very common pattern is this kind of interface that has like a list of things and then each of those things you can click it and then you see a little more of it, right? So like this is an email client, the default iOS email client. And you see that the email and the sort of detail view of the email are totally separate, right? But for the user, that doesn't make a ton of sense because for them the email is like a semantic idea, right? Like this is my email. So why would there be two of them? And why would there be like, I don't know, a totally separate thing over there on the side? And this is especially interesting in the context of this email application that does the exact same thing, like functionally, but it doesn't duplicate the object, right? See that? It actually comes from within the preview. And this is the kind of thing I mean. Now, there's a lot of examples of this and sort of I think you're getting the idea. You basically think of what all the states are that your components have and you think of all the transitions and then you will not have these contradicting situations because you've already thought of them. So, but I also know what you're thinking sort of on a larger level, right? Because those of us here who are designers and developers look at this and they're like, shit dog, I cannot implement this. Because like all of our toolkits, all of our layout technologies are just fighting us on this left and right. And so I would like for the second part of this talk, I'd like to talk a little bit about how we're gonna do this. And so I think it's important to keep in mind in that regard that there are some things that maybe we will not be able to do for a couple of years or maybe like even many years because the technologies we need to catch up. But there are also a few things that we can do right now. And so like if we sort of look at the timeline of that, I'd like to talk about a few points on that timeline. First of all, what can we do today, right? With the technology that we have. This is an example from GNOME. This is the GNOME to-do app in its version like that came out last year, 326. And you can like drag and drop these tasks. And when you click one, what happens is you get this like panel coming from the side with the details. So when you wanna edit the thing, like you have to click it and then there's like a totally separate thing that shows you the details of the thing, which is exactly the pattern we just talked about with the email, right? And what we did in 328 is we did this instead. You can click it and from within that preview comes the actual detail stuff, which functionally the exact same thing, but like semantically it makes so much more sense. And this is something that we were able to do relatively easily because GTK already has this thing called I think GTK Revealer that just does this thing. It like expands the thing sort of like an accordion. And there are things like this that you can do in a lot of technologies where you have an idea of sort of how the components should behave and maybe you can't get the perfect animations, but thinking through all of the component stuff, I think it's useful even if you maybe don't do any animation because you have like a clearer picture of what's going on. So that's the present. Let's talk a little bit about the future. So over the past few years, I've built a bunch of pretty hacky prototypes trying to explore some of these ideas. This one is one of the most interesting ones, I think. It's like I built it for a large touch table. It was like a university kind of research project where you would essentially like order food by dragging these plates towards you. So you could like drag these towards you and then the plates are sort of semantic components. So you see like the label when it goes away, it like goes into the component and the like detailed view for it comes from behind the plate. And so I was exploring some of these ideas there. And it looks and also like there's this spatial navigation where you have like these categories on the horizontal axis and the sort of checkout flow on the vertical. And it looks really simple, right? But the implementation is just a garbage fire. Because like this is actually this super simple component actually has five different states. We was like when it's not spawned yet, it's invisible there in the middle and then you sort of spawn it to the grid and it has the label and then you start dragging it and you need to get rid of the label and then you order it and you need to change the constraints and needs like be fixed to your viewport and like there's the dialogue. And doing all of that stuff, I mean I built this in web technologies but I'm not sure it would be any different in like most other stacks because most technologies just expected to like give them a bunch of XML files. Those are the states. And now click okay and go to that other XML file. And doing this kind of stuff just requires hacks on every level. At the moment, which is why it's not really feasible in real world product I think. Unless you have insane engineering resources. This is a second prototype that exposed a little bit of a different idea. Where it's kind of like this chat email sort of thing that actually sort of goes directly from the preview to the full view without ever sort of having duplicate items or breaking special consistency. And you can actually like interactively kind of drag and drop the thing to expand and open it. I mean like this is also very small and hacky prototype but this is I think the kind of thing that we should enable in layout technologies and sort of toolkits to make easier. So yeah that's a little bit about the future. Now how do we get there? And I think I've already alluded to this. We need tools that sort of just have this baked into them and that makes this really easy. And this will have to happen both on the design side and on sort of the toolkit development side. A little bit about like from my experience, from my experience what would be good to have on the development side. In the web ecosystem there's a lot of like talk about React these days and sort of their whole unidirectional data flow. And you have like this one hierarchy of components where like this component renders these sub-components and I think that really goes in the right direction. What they don't have or like what is much harder there is to like have multiple states and actually do animations between that stuff. But I think like that ecosystem is going I think in an interesting direction. So probably like if we were thinking of like the future API for awesome interfaces that I think would be an interesting part of it. Then yeah like the whole component idea where each component is essentially a state machine but not only that like each of those sort of state machines is made up of smaller components that are also state machines. So like essentially like you have your list that has like three states and then the list is made up of list items that also have like two states each or something like that. And I think like that is also an idea that goes in the right direction. But yeah I mean take all of this with a grain of salt I'm like a designer and sometimes web developer. And then the other thing that's also really hard to do right now is like actually sort of transitioning the properties without just manually putting lots of values like hard coded values all across your code. Like just declaratively defining all your components and then like these are the properties in state A, these are the properties in state B just automatically like tween them. And then yeah for interactive animations super, super hard to do right now I think in almost everything. Because what do you and that even goes kind of against like the component idea in a way because then you if you wanna do that declaratively like you have to sort of map the input to like the property changes. So like that introduced a whole bunch of in between states and I think like that is maybe a whole separate discussion on its own. So a couple of things that I'm excited about as I said the React ecosystem there's a bunch of libraries for React there's something called React Flight that kind of allows you to like have a bunch of states and then we will automatically tween them but it's a little hacky and like I haven't actually personally tried using it. And then in React Native they have this thing called like animated values where you will essentially like assign a value to like your sign does like value variable to a property and then you change it somewhere else and that kind of like allows you to nest these structures like it's really complicated and I barely understand it but it seems promising. And then I think another ecosystem that's very interesting is like the kind of prototyping tools they're mostly Mac apps for the most part but like this is also a JavaScript library. That kind of ecosystem has a lot of interesting stuff. This is a tool called Framer that's a JavaScript library that you can use to sort of prototype interface. It's not actually for production use but they have like this idea of components with states and they actually do most of that like animation stuff that I talked about where like you can animate between those states. So that's definitely I think a very interesting thing to look at. And then finally I'm very excited about GTK4 coming out soonish maybe end of the year or something which will make animations much easier from what I understand which also take you to the grain of salt and there's a lot of like really cool work happening there about sort of redoing the way we do layout with constraints and stuff like that. And we've actually been talking about potentially sort of like having graphical tools that allow you to kind of like have these nested component structures and edit them in a graphical way which ultimately is probably what we want. Probably we want some kind of interface builder that just allows you to visually build these graphs of states. So yeah I mean as you can see this is all like very hand wavy and like maybe at some point in the future but I think yeah this is the direction that I feel like it's most promising to work in. But that said if any of you have any ideas in this direction or like are involved with something or like oh yeah this is the thing that we've already solved wait just a second that we've already solved I would love to hear about it and like come talk to me or come to IRC and know design, find me. So yeah that was like a brief look at semantic animation and sort of where I think we are right now in the timeline towards the future. And with that I'd like to thank you and if there's any questions let me know. All right that was one there I think. Yeah I actually. Hmm okay. And these things are quite easier than web technologies or. Okay. Because everything is nested and everything is declarative and everything has changes when the body changes so you can plug the position of the mouse to the width of the component so move the mouse and then it changes. Wait I think I need to repeat the question right. Okay so you're going. Yeah. Sorry I guess I need to remember. Right. Okay so the question or remark was that QML or right? Yeah QML. Already kind of does a lot of this stuff with nested components and like mapped values if I understood this correctly. Right in the declarative. Right in the declarative. And yeah that sounds very interesting. What I would be interesting is like does it have states? Like so okay. Exactly what you described. Interesting we should talk more. Any more questions, remarks, discoveries as we've just found? Yes. Do you have a principle? I do, I haven't used it personally. I mean I know how it works. I was going to use it directly to do UI innovation. Right. Because I've also started a lot with like the state-based animations but the UI event, different animations I've found quite easy to implement and also very useful. We'll have like zero design users and quite a lot more expanding and collapsing area. So that's like an animation category that's sort of relatively accessible and easy to implement. Yeah absolutely. So okay so the remark was the principle for the Mac does some of the stuff with what is called drivers. Yeah that's what they call it. Right. The term they use for it. Yeah yeah I think like a lot of these sort of experimental Mac prototyping apps have some really cool ideas. What I would really like to see is like those ideas going more into the base technologies where you just like can use it and don't have to, because then at the end like what I've heard a lot of times is like the designers make these cool sort of prototypes and then the developers like kill me now. But yeah, thanks. It's very easy to implement usually. Like in our case we want to mix in with Emory I think it works really well. Okay that's great to hear. There was another question there. So the question is do I think that animation improves productivity in graphical user interfaces? So I think that's maybe the wrong question to ask. That's like asking does color improve productivity in user interface? You can, it's another variable that you use to design with. You can use it in ways that might impact productivity but like that's a very sort of difficult thing to measure in the first place, right? Like doesn't, you know, if we use more like, I don't know color or contrast in our icons is that gonna improve the productivity? Maybe, but it's something that we can use and that we sort of have the, the academic sort of prove for lack of a better word that like humans can sort of be, be, be taught about things with and that can be useful for anything. And I think also yeah, in general like any kind of productivity applications. Yes. The idea we talked about and some teams I've worked about on is that the idea that, I don't know if you agree with this but the animations can help T-Code user but perhaps considering whether as the user learns, the software also learns that the user has learned and maybe increases efficiency by phasing out some of the animations or speeding them up. What do you think of that idea? So the quest or like the remark was that there's sort of the idea of like animations being something that you use to teach people and then maybe later they're more in your way, if I'm understanding correctly, rather than helping. And I think that's definitely a fair point. I think for the most part, like that's what I sort of alluded to before. If it's something in an onboarding process that you only see once, it can be slow. It doesn't really matter. If it's something that you see every day, it should be like 200 milliseconds and that's it. I think like at that point, like if they're not faster than sort of like the really sort of the limit of perception, I think there's not really a huge gain that you can get by like shaving off maybe 20 more because humans can't really react in that time anyway, right? So at that point, I'm not sure, like maybe you could speed them up a little bit, but I think that's a larger sort of discussion about should we change interfaces as your users learn them? I think there's also like sort of, that's another point of contention that, I think it's maybe separate from this discussion entirely. Last question. Sorry? Sure. If you are, instead of, right. To implement it. Yeah. So the question was like, yeah, we just talked about Mac OS prototyping tools. What do we have on Google Linux? I'm sorry to say nothing. I mean, there's JavaScript libraries and that's what I do. I just like basically figure out the thing on paper and in Inkscape and then write a little bit of very bad and hacky code. It's not very efficient and I'm sure it could be better like if I had any graphical tools, but if anyone's interested in working on tools for that, I would be interested to talk. Yes. Suggestion. It's not a resource. Yes. The suggestion was using Figma, which personally as somebody who cares about software freedom cannot endorse. I'm sorry.