 Physical, which is a Berlin-based intermediate dance collective and we've been making work since 2012. Over this period, we did several collaborative efforts to build interactive performance experiences between mediums, including 40 sound animated and creative coded visuals and choreographies with sensors on the body to affect sound and visuals. If you want to know more about those works, you can visit our website in the link I will paste into the telegram chat, but we're here tonight to talk about the current work we're building and researching titled Human ID, which is coming from questions about human identity and technologies like AR, AI and the ability to deep fake a personality about what is that thing that makes us human and why is it we feel so threatened if it could be overwritten. From my side, these questions began a couple of years ago when I started to teach myself coding and wondering if I could take some of the syntax and put it on bodies and choreographies and at that time, Daria and I, we were making a lot of loops of choreographies, so I thought it would be fun to do something simple like dot pop or dot push and just play around with the phrasings there. So it started out in a playful way, but soon bigger questions emerged. I became fascinated with this idea of being able to look at the code behind a project like a website and you can clearly see like what is functioning, how it looks and so on. And I started to wonder, well, what is the code behind human, which is not as easy an answer to discover or to even peek in on. But we started with a simple outline. So thinking of the human as a shell that runs scripts or programs. And then this bigger question of could we take one performer or shell and exchange the script from the other and erase the original copy or erase the personality of the other before our very eyes. In other words, is it possible to produce a live deep fake in performance? Since those questions emerge, I've been lucky to research and and try to test these research from these questions through the support of the combustible residency that we're currently being hosted by Counterpulse in San Francisco until April of next year. And in collaboration with Alessandra Leone, who is a co-founding member of Serato Physica and does the visual content, the video and lighting, as well as collaborations and sensor building, but unfortunately could not join us tonight. The two collaborators that are present is Daria Kaufman, who is a longtime partner in crime in making thoughtful intricate choreographies dance and is kind enough to have shared a stage with me over the last 10 years or so or more. And I would say the one asking a lot of relevant probing questions to to help mine the meaning of the works, as well as Ian Heisters, who is a new media artist working with dance and installation and is the AI researcher for this project. And I will also post links of their works, their websites in the chat. But without further ado, I'm going to let them talk about their parts of the research for this work. Daria. Well, actually, before I talk, I just wanted to show some movement studies that I've created in as part of Human ID. Just take a couple of minutes to show these short choreographies that I call glitch studies. And there's no sound for this, just so you know. Those are just a few short choreographies that I've been developing that I call glitch studies and talk about their relevance to the project now. I mean, I feel like in general, looking at how individuals' appearance, their body language, their mannerisms are mediated through video technology. And like so many of us in the world right now, I'm spending so much time on video calls, interacting with people so much that way. And over the past months, there are all these moments when I'm on a video call with someone where it just like freezes on their face. And it's like this strangely intimate and voyeuristic moment. I don't know that they know that they're frozen. I also don't know if I'm frozen or not. And I'm just like staring at them in this kind of punishingly naked moment. And it's like simultaneously painful, but I can't really look away. And I've just been thinking about that and that this is how so many of us are interacting now that this is just our new normal. So those are like the freeze moments that I find is actually really powerful and kind of poetic in of themselves. And then also the glitches where, you know, there's this kind of stuck in place back and forth. Um, I've always been interested in glitch in video art and sound art. And I feel like there's this, there's some sort of opportunity for transformation in the glitch for what the thing that is glitching. The glitch is like a vehicle for it to transcend whatever it is. Um, so I've always been interested in that. And right now I feel like getting through video calls and being confronted by this glitch phenomenon quite a bit. And I also feel like it just has greater relevance in the world right now because, I mean, the whole world is sort of glitching right now. We're all stuck in place. We're not able to move forward or back and we're just but we need to move, you know, so this is all that's been inspiring these studies and that's related that in terms of human idea, I feel like something we're we're creating this entire project through video, which none of us have ever done before. I mean, we're working remotely at it, like never have all four of us been in the same place. And it's just going to continue to be like this for a while. And it's like a whole other way to create. And it's interesting because it's become like the process of this remoteness and this everything being mediated mediated by video has become the subject of the piece, I feel is like, how do we actually how are identities mediated by video technologies filtered by them? So, yeah, that's something that's a little bit of my personal thread. And I guess there, I'll just pass it on to Ian. Hi, I'm Ian. Yeah, so picking up from what there is, like the running this entire project through various video networking devices. One of the things I've been starting to think about is how the because we've been so we've been working with these neural networks and doing some research around how we can build it into the piece to create certain deep fakes and thinking about how a neural network can learn an identity of a person. And if there's any valence of who that person actually is learned by the network. And for me, there's this question of whether the non neural networks, but the networks that are connecting us now, you know, the Google and the Facebook and the zoom are while working on this project are learning something about the identity of the project itself. And it's like kind of lower tech, less obvious way. Some of the deep learning stuff we've been working on. Let me share my screen here. Okay, y'all see this. One of the initial things that we started talking about was so that the piece is very much about identity and how identity is expressed through movement and how a neural network can be interposed into that relationship to start fucking with that relationship. So we we researched this deep learning technique called motion transfer. This is some of the early research. This is based on a paper out of Berkeley called UC Berkeley here in California called Everybody Dance Now. But this applied to obviously Daria is here along the top and has along here on the bottom. So what we do is we take a bunch of frames of video of say Daria dancing and we run pose estimation on that that like reduces her body position to a set of labels about where her different body parts are. We train a neural network on performance of Hens that learns the visual style of her what she looks like visually. And then we can apply the structure that we pulled out of Daria's performance and apply it to Hens visual representation and we end up with down here on the bottom. We end up with this synthesized image of Hens image performing Daria's movement. This one's a little bit buggy. You see like her shadows get picked up by the neural network as part of her body because the shadows look so much like a body. But it provides this opportunity for like the kind of this weird puppeteering and this weird amalgamation of their identities as expressed through through their bodies and then through the camera like the camera becomes a really important player in this and it brings up a lot about how you're filming it and both technically and in terms of cinema. So let me show this. So so then based on that outline we can take a video like this of Hens performing a movement and then we train a model on Daria's visual representation and we can get Daria. This is very buggy still we're testing experimenting performing the same movement. Even though she never performed this stance. Let's put these side by side and we'll restart them just so you can see the synchronicities. So obviously like there's errors in what the neural network learns about Daria's identity and her visual representation and it seems to get especially screwed up around the face. I think it mixes up. It's learned a lot about her hair but it's a little bit confused about what's hair and what's face. So kind of like you if you look closely you can see some upside down down eyeballs in her face and like it's just like all this contorted that's something we might address or it's something we might leave in the piece. But it's also it's strange watching these videos when you're making them and seeing yourself like what you recognize as yourself. Oh that's me that's Daria and I but there's a video of me doing something that I never actually did. It creates this weird. Distance so coming back to just want to share some of the research we've done around this the original paper that we're working off of does the skeleton based structural analysis of the body. I start playing around with doing there's another technique that was developed by Facebook actually called dense pose. And what you see here so under this is a there's a video of hen just from the shoulders up and dense pose instead of just marking out. The basic structure like here's the head like just these two little dots in a line. It actually creates like a silhouette or a mask. And then we can also lay over eyebrows nose and this becomes really helpful for different kinds of video like the skeleton based analysis is really. It has to have the full body in view and like when it loses the limit starts to freak out and so it creates a lot of limitations in terms of what you can do cinematically. And when we start working with dense pose it opens up the some of the frames we can do it still has a lot of limitations. But for instance we can do a shoulders upshot medium close up like this. And then we can do. Sort of a portrait transfer. I'm standing in here because I couldn't get a video from Daria. We can do a portrait transfer. This has other shortcomings for instance because so the dense pose instead of creating a skeleton it creates like a full outline of the body you end up with this again this amalgamation of bodies. We're like ends up with my face facial shape including my beard because it doesn't understand how to segment a beard from a body. But then it's like hands appearance on. My. Outline or my body type. Obviously there's some strange glitches around the mouth. Probably also due to my beard. It always throws off these kinds of computer vision analyses. There's something about a portrait that I've been interested in on this project because like the portrait is this photographic analog of a lot of the things we're talking about in terms of capturing an identity or finding an identity photographically or cinematically. But I've also been playing around with some like unintended mappings. Kind of leaning into the space where things start to screw up and you can start to see the edges instead of going to this like hyper realistic deep fake that you could use for propaganda you know like oh here's Vladimir Putin saying things you would never say or Barack Obama. How can we actually go the other direction and find the edges of the algorithms and where they screw up and because then you can kind of get a perspective into what's actually happening and what it's actually doing. So here's some experiments I did with mapping my hand to my face. And so the neural network is learning. You're basically telling it like this hand equals this face and you do that. Tens of thousands of times and it's able to resolve. And then when I show a different hand in a similar position it's able to say OK that's this face and the face you see on the right. Well the face you see I hope it's on your right. Is completely synthesized. But it maintains a lot of the the details of the original photography or cinematography. But then you can also start to see these weird artifacts of the algorithm. Both in terms of like this cross hatching that you see on the beard. Sometimes it resolves really clearly and you're like oh that's a photograph of a person. And then sometimes it almost you see more of the hand. And so in working on this like one of the things I'm interested in exploring is like what is the algorithm actually learning about me. Is there is it learning anything about my identity. Hold on one second. Just go away. Go away. Is it actually learning anything about my identity. Is there some kind of vectorized representation of my identity in the same way that a photograph. We can say captures some dead moment of somebody some little slice of their identity. You can see it shine through and that's legible to you. Is there some sort of you know. And dimensional representation of that person in the network that's left behind. Is this a way of maintaining a memory of a person. Yeah, I think that wraps it up for what I have to say. Thanks guys. So we have with us today Chris Harris and Miriam Simon and I will love to invite them to introduce themselves. Sure I'm happy to go first. Thanks and thanks Ian and Daria that was interesting to see what you've been working on there. Yeah my name is Chris. I have had historically quite a technical background. I studied AI quite a long time ago and long story short became quite although although I started studying AI because I was fascinated by human data and human computer interaction biomimicry. I became quite disillusioned with a lot of the uses of the technology and indeed there's a lot of critique about the way we currently approach AI from you know the direction of which we approach it mimicking or rather taking the human intelligence as the apex of intelligence and trying to mimic that or or using it to subvert our own intelligence. So I appreciate the investigation that you have done into where things break down and you know using this as a as a critical investigation into the scope of our relationship with identity and how much a machine can or cannot learn. Yeah so yeah I'll let Miriam also say a few words about herself who I also know is a very interesting HCI practitioner. Miriam. Thanks. My name is Miriam Simone. I'm an artist. I also actually yeah I do do user experience design so I think a lot about how humans interact with machines and just broadly and as an artist interested in the social and poetic and political and weird implications of new technologies and bodies. Yeah really excited actually about all of you. Everything you guys said. Cool. Daria I had a couple of comments on your rather observations on the glitch studies that you showed. I really liked what I mean you mentioned the time we spent on Zoom and the Zoom fatigue we get and I feel like a lot of that is because of the lack of feedback we get in a virtual capacity as opposed to in a physical capacity and you mentioned the the role of technology in mediating and indeed in reducing that bandwidth we have between each other. And what I felt what I really saw in your glitch performances was this asymmetry of perspective. I think we always to some degree have an asymmetry of perspective even when we're one human talking to another human or be in our context very much grounds us in some kind of mimicking of where we are but when we're so distant I feel like the the symmetry of perspective is so yeah there's so much dissonance there between what I see and what the other person sees and especially when things break down and start to glitch out and you become this voyeur in this slice of reality. Like you like you mentioned this is almost almost punishing voyeurism you can't quite look away from. I think we're constantly fascinated by what is quite an unconventional viewpoint of a normally conventional scenario. So yeah I appreciated your your studies there at some point. Thank you. I have a question. Sorry, I just interrupted. Okay, part of the zoom beauties and I'm just so I recently I was just like well okay there's a lot. But I'm going to ask like the last question I had first because it's kind of a really big one. So one of the last things Ian said but also kind of all of you guys were talking about like what is the algorithm learning about me. Is it really learning about my identity. Then there's this question I would pose but I'm not about what Hen was starting with around like personality getting put in another shell but then like personality here is equated to movement, which I get but also that's a big thing to talk about. But then I just recently was listening to like Susan Sontag talking right and the whole thing of like taking a photo and what the photographer takes from reality takes from that moment takes from the subject. And this is just a still a still photo that you're printing right and so then it really like kind of exploded my mind of everything you guys are doing because what is yeah like what are you taking. So that's my first question. It's kind of a huge one but because also I guess because you guys have started doing this and I actually see it a bit more like you guys are finding really the beautiful part and where like where the machine gives you more rather than takes away than like a human to human. But I do wonder like having done this process like and yeah like what is it taking from you or what are you taking from each other or yeah. You can speak to that a little bit. There's a lot in there. Well, I guess I'll say that I'll focus on two different threads. One is like I wanted to mention that like earlier on in our process Ian introduced us to Miriam Boyn. Is that her name Ian? Petlana Boyn. Petlana Boyn. Thank you. Miriam in person. Who writes about nostalgic technology and finding the humanity in the flaws and the errors going towards the errors. And I feel like that's kind of become a touchstone like that was really useful for me to read and it keeps returning this wanting to look at the flaws and the errors. If the error is human then you want to find the human in the machine you go towards the mistakes. And I think that's kind of and I yeah so that's been a kind of touchstone for the project. And then something else you were saying Miriam another thread towards the end I've lost it though about taking. Who is taking like what are you doing? Yeah, I think it's like what is being captured. It doesn't even belong to me. The thing that's being captured. Where does it lie? I think that's a really interesting question. And I understand people feeling so threatened by these technologies and the notion that someone can take your image and make it do anything you want. And then that gets described as the tree and this violation. But the thing is the thing that's being taken it's not you. It's the representation of you. So I think it's just the right territory right now. And I'll leave it at that. I'm kind of surprised to hear you say that you like that we're creating all this beauty. Because I think like what it's taking is so obvious to me and these like these strange like neural pastiches of different body parts and this like strange like I mean it's like it's grotesque. And I think I mean where the beauty does come through is it was something in the solar punk like manifesto for this event of like imagining new possibilities for our relationships to technology. And that is in there and that's like also from the Svetlana Boehm she talks about it in terms of nostalgia for like a future or even a present that isn't not like is a parallel present to the current. She calls it the off modern. So we have our modern the dominant modern ideology that AI is going to save us and it's also all going to put put us all out of jobs and Elon Musk is you know creating this this modern notion of AI I mean and I'm using him to the standard for a lot of people. But then there's an imagination or a nostalgia for this other possibility we had where technology was built for for these kinds of interrogations that we're doing for creativity and for thoughtfulness and for beauty. And part of what we're doing is is I mean the tools that I'm using are we're built for like automated driving like a lot of it was built to train cars on how to drive. And so you were we're taking these like high grade industrial tools and and hacking them so that we can turn we can create this other possibility for an application of technology that is a little more introspective. I don't think that answers any questions. Are you trading like this is my question. Are you trading the or what because you are the thing I find beautiful is like all these yeah like the beast Daria made and and all of the weird glitches and you the hand inside your face like all this crazy beautiful to me stuff that is in the error blah blah blah. But like what are you trading for that. That's I guess that's a good question. I wanted to just maybe follow on from that a little bit and extend that into the question that I had that from especially when Ian you're showing the puppeted movement and the fact that the machine was seeing into different parts and projecting more human there perhaps what I was really what really hit me there is the bias of identity that we have because we know we as humans have a bias of what constitutes as our identity and the machine now has a bias towards what constitutes as identity based on on what is trained it and I think bias is right there because no two are necessarily correct. I mean what what is our identity and what is and what isn't what is the environment what is the machine right to project the human into the side of the wall. So the question that I wanted to let up to for me was asking you as artists and practitioners doing this work whether you've had any shifts of identity or whether you've grappled with this or if there's been any anything that's kind of come to mind in that way that's that's changed or that you have new considerations after doing this personally this project like it's been this very extended because of covid and just working this way working through technology so much um it's like if you're interacting with someone there's you and there's them and then there's the thing that is emerging between you and I feel like in some ways the technology that's based between is bigger so it has there's like because there's more of a disconnect there's also more potential for something to arise in between and I'm just like in our process for instance for a while I was like really frustrated I was like wanting to hold it and know what it is and like be able to frame it neatly and I kept coming up with a frame like it's this and then we would get on a call together and never mind it's not that so then okay now it's this like this constant back and forth between being with them virtually and then being on my own and all these the storytelling that goes on and I have my own storytelling that I do on my own and then I have to try and reconcile it with them on these virtual calls and it was just getting exhausting so eventually I just kind of let go and was like okay the story is just it's emerging and I need to not try to have so much control over it stop trying to hold it just like let it be what it is and let it ping pong is something that we've a structure that we've landed on let it ping pong and each time the ball comes back it's a little different and that's okay and I don't have to hold it so neatly didn't that make any sense yeah that was beautiful yeah thank you very much Daria the identity is created in the spaces in between and just to let it be and to not be afraid of that and to know that that can happen and I can still be me that I'm the chorus and that's okay brilliant words to end on I think that's about all we have time for so thank you very much Daria you and Hanny and Miriam pleasure to speak with you all thank you Chris and Miriam thanks