 Welcome to Cooper Hewitt! I'm so delighted to have you all here. I'm Ellen Lupton. I'm curator of contemporary design. And tonight's event is held in conjunction with an exhibition that just opened two weeks ago called Face Values Exploring Artificial Intelligence. And that's what we're going to do tonight. We have some amazing explorers here to show you the way. And they are all featured in the exhibition. So if you haven't seen it yet, it's open until the end of April and you'll have a chance to experience for yourself AI and facial recognition technology. The exhibition actually originated last year at the London Design Biennale. When Cooper Hewitt was proud to represent the United States of America in the 2018 London Design Biennale and we commissioned the original work by Luke Dubois, Zach Lieberman, and Jessica Hellfand to explore facial detection and more specifically emotion detection and facial measurement as ideas and technologies that are being incorporated into all kinds of products and systems that we use from your iPhone to surveillance and security equipment. And it's often working there in the background. We don't always know how well it works or why it's there or whose faces it's based on. So these pieces that were created by these amazing artists and historians really get inside of that and allow you to experience it. So tonight we're going to have short presentations by each of our three creative guests and they're going to talk about their work in relation to these themes and what's in the exhibition but also a bit more broadly. They are amazing people and amazing contributors to the future of design. October is a crazy month at Cooper Hewitt. It's like design month. It's always design month at Cooper Hewitt, but like on speed in October. We have the whole week of October 12th through 19th. It's called design week. I think we invented that. Lots of other cities do it too, but we started it. We have a design career fair. We have a design fest for all ages. We are celebrating the 20th anniversary of the National Design Awards with an amazing gala and you can buy a table or five tables and make us survive the century. There's a winners salon in relation to that event with 20 minute programs featuring some of the most important designers of our time who all got prizes from Cooper Hewitt over the last 20 years. So I hope you'll join us for that. There's another event planned in relation to face values. Karen Palmer is an interactive filmmaker who we commissioned to create a brand new piece for face values and she will be back here on October 29th talking about her piece and bias and artificial intelligence and she'll be joined by Emily Balcitis who's a behavioral psychologist from NYU. It's gonna be amazing. Karen is a force of nature. All the work in face values is like created by these amazing people for this exhibition and I just feel so lucky to have been able to collaborate and be part of it. So with no further ado I'm going to introduce Luke Dubois and I want to tell you a little bit about Luke. He's amazing. He's an award-winning composer. He's a coder. He's a software guy. He's a teacher. He's an advocate for inclusive design and an inclusive society. He's the director of the Brooklyn Experimental Media Center at the NYU Tandon School of Engineering and much of his visual work explores the limits of portraiture in the digital age by linking human identity to data and social networks. The guy has a brain and a skull on the shoulders so Luke come do it. Hey so I don't I don't I don't use PowerPoint so you'll have to forgive me. I'm bad at this. Quick show of hands. How many of you have ever seen the film desperately seeking Susan? Amazing. How many of you have never seen this film? Oh wow. Okay you need to see this movie. This movie, listen, this movie explains everything. Okay so quick recap 1985 directed by Susan Sodleman. Most people think it's starring Madonna. It's not starring Madonna. It's starring Rosanne Arquette. It's featuring Madonna. It is a movie about a woman who lives in Fort Lee, New Jersey who's a little bit too much time on her hands and gets obsessed with the personal ads in the back pages of The Village Voice where upon she discovers a real live couple who are communicating in a long distance relationship through the personal ads. The man rolls into town. He puts out an ad saying desperately seeking Susan. Meet me at Pier 40 on Thursday at five o'clock. She decides she wants to observe this couple because she thinks it's incredibly romantic. Somebody gets hits on the hit on the head. There's mistaken identity shenanigans ensue. This movie to me is is is is is incredibly relevant to our cultural moment because it's it's what it's ultimately about is it's about someone who crosses a Rubicon from observer to actor in a social situation in which normally they would not be included right. So if you think about how we participate in social media these days and click likes on things how we perform in social media and how we sort of kind of constructed our lives in the last few years it's just you should watch this movie. As Ellen mentioned I make portraits. Most people when they think of portraits they think of this. This is Gilbert Stewart's portrait of George Washington the so-called so-called lands down portrait which is in a Smithsonian Museum in Washington DC called the National Portrait Gallery. There's a lot of symbolism in this portrait right. He's got a sword but he's in his left hand because he's a man of peace. There's a rainbow out the window that like you know my six-month-old could have drawn. It's got like you know the kind of stuff. I make portraits based on data so let me find you one. So like this is so here this is my portrait of George Washington. This was commissioned for the 2008 Democratic National Convention and these are the 66 words in his State of the Union addresses that he uses more than any other president. So the word he uses more than anybody else most commonly is gentleman. George W. Bush who is president at the time I made this piece his number one word is terror and the way you get from gentleman to terror in 43 easy steps is a history lesson of the political rhetoric of the United States right. Bill Clinton spent most of his presidency talking about the century in which he would no longer be president right. Ronald Reagan his deficits Richard Nixon interestingly is truly you have to keep in mind that one of Richard Nixon's speech writers was an amateur linguist by the name of Bill Sapphire who also counted words. A sequel to this project would be I tried to make a sort of sequel to this it was more about how Americans describe themselves and so I did this project in 2010 where I made a census in the United States. I joined 21 different online dating services as a straight man a gay man a straight woman a gay woman in every zip code in America and downloaded 19 million people's dating profiles and made maps so what this is showing you is this is where all the lonely people are lonely people tend to be in the you know Appalachia this is where all the shy people are shy people tend to be in the upper Midwest. This is showing you that Nebraska is not particularly funny and this is showing you where all the kinky people are so this is telling you that men in southern New Mexico and women in West Virginia need to get together and have a good time. Then what I did was I went through and replaced the name of every city in the United States with the word people use more in that city than anywhere else in the country. So if you've ever dated anyone from Seattle this makes perfect sense. I grew up somewhere this sort of will make sense to you. I grew up somewhere between annoying and cynical and New Jersey and and my favorite one is you know New York City's number one word overall is now which which solves the sentence right now now I work as a waiter but actually I'm an actor right that's something people say in New York but my favorite would be you know if you go upstate right so um so the only good place to eat in Syracuse New York is a Hell's Angels barbecue joint called dinosaur barbecue right so this is where you would take someone on a date right so this is this is sort of what I do um what I make portraits um not all of them are quite fun and game so for the last three years I've been touring a piece called take a bullet for this city it's a live fire in Walter ppk 9 millimeter semi-automatic that fires a blank whenever someone's shot in the city it's in by tapping a civic open data feed the vitrine it's in fills up with bullet casings this is called data visualization data visualization is a problem in our society not a solution we can talk about that some other time um and so I um I do stuff like that I also make a lot of films and I thought I'd show I thought I'd since we're talking about machine learning I thought I'd show you tell you two quick stories a fun one in a fun one about how I've used machine learning at work and maybe I'm something more serious one the fun one is um is um I did a soundtrack for a film once about Star Wars about 10 years ago um where um the filmmaker it was an it was an artist commentary of Star Wars you can imagine a director commentary with only with a split screen and instead of it being you know George Lucas talking about whatever it was an art history lecture the artist was named John Powers he's a sculptor and and he made this really interesting kind of tour de force redo of the first Star Wars movie with the following conspiracy what if instead of it being a real pop culture man 1000 faces kind of film what if it was a really smart intertextual critique of post-war modern art so every time you see a lightsaber you talk about dance every time you see the Death Star you talk about the period I go housing complex and see this right um right and Luke Skywalker is Robert Smith's right so he asked me to do the music right and so what I did so it turns out that Philip Glass and John Williams went to school together right they were both students at the Juilliard School and so I imagined a long lost collaboration where they wrote music together and so it took all the all the 70s keyboard pieces of Philip Glass and threw them in the computer and then I took all the harmonies and melodies of Star Wars and threw them in the computer and ran a thing called a neural net which is a term a thing that you do I guess when you're doing AI um and this is what came out and this is what came out so you end up with a thing that strangely sounds just like Steve Reich but that's not the point um that has the harmonies and melodies of Star Wars with the keyboard and rhythmic performance practice of 1970s era Philip Glass so that's kind of an upshot that's a that's a that's an upside of machine learning story the downside of machine learning story is is you know so the the Defense Department of the United States does a does a design challenge over here where they put several million dollars of funding on the table mostly universities and ask them to solve them solve a problem for them right and I and I teach at an engineering school so I'm sort of sort of intimate knowledge of this challenge um and uh in January 2002 they put out the following challenge right the Defense Department of the United States put out the following challenge they said if we give you a bunch of videotape and don't tell you anything else about it can you help us find one specific person um and so if you rewind your mind back to January 2002 you can sort of guess who they were looking for they were looking for a specific person they were looking for him very hard and they were looking everywhere they could and they needed a little bit of help this is actually in some ways a good STEM advocacy story because the winning team was not from NYU certainly was not from MIT was not from Caltech it was actually from Cal Poly San Luis Obispo which is the largest public engineering school in the United States uh and the student design team that hacked it figured out how to modify a standard algorithm for recognizing faces I think called the Harcascade to take into account cultural variations in hairstyle and facial hair right um and so I decided to implement that exact algorithm on the one person on the planet who would never need to be found that way which is Britney Spears um and so I took um 5000 paparazzi photos of Britney Spears and trained my computer to find her and I can find her in any footage in the world so I can put any live footage music video footage whatever Britney Spears through this and it will detect her and lock her eyes um in place and this is a portrait in the mpg right so this is and this is another Smithsonian commission and the the the sort of conceit of this piece is that we have a we have a really um disturbing cultural double standard in the United States around surveillance and observation where we are very um risk adverse to the idea of being watched but then there's people like this who we could look her up on TMZ and find out what what she had for breakfast this morning and that's maybe not not super fair um so that's what I have to say to you um and we'll show you some other stuff a little bit thanks so uh we have a little tech change here um I'm really excited to introduce Jessica Hellfand who contributed a beautiful piece in our show upstairs about the history of of facial measurement so we like to think about you know AI is something that just got invented yesterday and the robots are coming and no one will ever have to work again and people give us all a thousand dollars a week or whatever that's the other guy um but actually the the kind of thinking behind facial detection technology goes back hundreds of years um and there's a long history to us wanting to for artists and scientists wanting to measure people's heads in order to determine whether they're good people bad people beautiful people ugly people criminals whatever um Jessica Hellfand uh is publishing a book called Face a Visual Odyssey with MIT Press it's coming out next month um and I was excited to be able to work with her as a historian for this exhibition to look at the history of how we measure faces um so she's going to talk a little bit about her book um and some of the philosophy behind faces she's the founding editor of Design Observer she is the author of numerous books on design and cultural criticism including screen essays on graphic design new media and visual culture scrapbooks in American history um and design the invention of desire so thank you uh Jessica it really sucks to go after Luke I thought I talked fast I now know I don't talk fast but I'm not measuring no measuring will happen in the next time I'm here to talk about measuring this is Sylvia Sydney she was said to have the saddest eyes in the world in 1933 it was very sad and we probably all had sad eyes then but what I want to talk about as I show the next few images is the fact that um you know I was thinking about this as I was driving down here yesterday that uh in business school there's a magazine called Poets and Quants for people go to business school you're either a poet or a quant suggesting you are a visual person of creative means or you count things well I'm here to say we're all quants and that quant goes back really far in everyone's DNA it goes across cultures and where it lands on the face I think is really interesting and and and something we need to pay attention to so this is beauty this is what beauty looked like in 1933 this is max factor and that contraption on top of that woman's head was actually developed for Hollywood to be able to calculate very specifically how makeup was put on the face so that the camera would catch it the right way there's nothing pernicious or particularly difficult about that except that it's a big ass thing to put on another one it's called a psychograph right the psychograph this is 1931 1934 but this idea that we measure beauty and that beauty is has an absolute value uh is a really I think a very curious thing and Luke mentioned a moment ago this line between surveillance and observation I think there's another line that's very thin which is the line between measuring as something that is quantifiable and demonstrable and maybe even scientific and something that's actually really pernicious so here on the left you've got a man who's measuring a woman in Tibet in the 19 early 1930s I think 1938 actually he later went on to work with the nazi ss party to help identify jews and on the right for those of you who are old enough to remember is Marlo Tom's the exact same image she was a tv star in the 1960s and doesn't she look happy to have her face measured it's the same image calipers are calipers and the idea that you can measure someone's face as a social determinant as a moral determinant as a cultural determinant as a racial determinant is what I'm here to talk about so this is where it gets really messy this is Sir Francis Galton in the late 1880s and he develops something he calls composite portraiture so instead of measuring side by side he's measuring something on top of something else he's literally doing what something Luke mentioned a moment ago he's putting the eyes together and he's locking eyes between people of similar social strata jews gypsies homosexuals white people black people what is he looking for he's looking for a kind of pictorial averaging and he did this in the 1880s after a suggestion by the Victorian philosopher and social prominent liberal theorist um Herbert Spencer that this was the best way to create an average of a group so the idea that you would average to quantify you would melt them you would morph them together by by putting them on a plate this is what they were going for this is called the fitter family so in the 1920s as eugenics is becoming taking hold of America this idea of measuring is about this cultural ideal it's always white it's always based on some beauty standard that's completely ridiculous so the fitter family was only made more absurd by similar contest across america in fairs and in churches and across different kinds of community organizations called the perfect baby so there's fitter families and there's the perfect baby and everybody wants to look like this so where does this start scientists believe for a long time throughout the 19th century that you could determine intelligence human ability and even criminality by measuring the skull skull so i'm going to make a distinction in a moment between the skull and the face but basically it begins with the cranium so we have this thing called phrenology now now in the interest of full disclosure i grew up with a crazy father who collected phrenology hence so my whole life my father who was the world expert on quackery loved this stuff because it was crack for him right it was like the idea this is a prostitute right the idea that we could tell that she was a woman of ill virtue because of the way her skull was shaped but it's not just phrenologists and crazy crack seekers like my father and people that practice phrenology this idea that you could decide who was intelligent based on not just the skull but measuring the features and the relative position on the face the trajectory of the nose as a determinant of intelligence for example so you can laugh at the fact that a nose shows crafty treachery or pessimistic gloom or scientific truth i'm sure i've dated all of those men who have those noses by the way and they all really are those people it's a joke it's when you look at the fact that the white nose is a superior species to a person who is of a race that doesn't have a Caucasian features nose with that line so it starts to get a little scary and then you get into this idea of the ideal type okay so we talk about typecasting it really has a long and difficult and pernicious history but on the right you probably can't tell from there but this is a chart that actually compares the circumference of a black child's skull and the circumference of a white child's skull so the idea that measuring is comparison has its roots in all of this work so this is the image that got me just excited about writing a book about this stuff first of all i love this guy that his lungs are on his cheek i mean you can't make this stuff up that his heart's in his chin his stomach is on his jowls i mean like what's not to love the haircut alone he's just beautiful so this is a book from 1903 it was called vaude's practical character reader and this is the beginning of something called physiognomy which is the art of character reading from the face it was funny it was highly illustrated it was very whimsical it was really messed up and it persisted throughout a good four year 50 years in the last century but lest you think that this is racially specific to white people you have to look at this same year 1903 on the left children of the poor and uneducated on the right children of pure and intelligent parents so you were meant to actually study this stuff they were textbooks given to schools they were entire curricula based around the idea that you should learn about reading in the face and therefore pick a better partner have a better job have a better job interview be a better citizen really strange that becomes a game 1930s okay but this game hides something that's very specific there's a guy named Jacques Penry who was a consultant to the police in the US Canada and Britain for more than 40 years and he was the guy that made this thing completely of use to not only people who were buying games and reading books about intelligent if your parents but the police so if you go upstairs and see the exhibit there's many examples of this in the timeline that Ellen invited me to contribute on the left is from a guy named Alphonse Bertillon he's the guy that gave us the mug shot this is the late 1880s calipers in use he decided that we needed to understand everything about criminal activity by measuring people's eyebrows and noses and shins but on the right that's work he's doing for the the British police force in the 1970s so Bertillon in the 1880s this is almost a hundred years later and we're still looking at adivism recidivism the fact that we can actually understand that criminals are born not made made that born doesn't matter if you've got a good set of calipers you can figure it out mug shots are the great dividend that was paid by this but this is again measurement into comparison this is the beginning of facebook i'm sorry it is right we measure people we may not measure their noses with calipers but we look at what they look like we read that we make judgments and this is what i started to think about the fact that we're all biased we all make judgments based on what we see and we may not be getting out measurements and trying to actually create new adivistic principles or or be genesis but we do compare things so that's the scary side of a facial comparison and measurement but there's also i think a little more lighthearted side which is beauty so again facebook we measure ourselves in in in terms of beauty it's basically the same photograph one is from a side show at the time of pt barnum and one is a movie star we measure ourselves in terms of social aspirations this beautiful picture from the wpa of a girl looking at a window a black girl looking at this this thing that she's hung on the wall there probably were no black movie star pictures she could hang on the wall and then we look at things like skin colors so does anybody know what that is on the left it's called a surely card so the surely card was goes back to the mid 1950s kodak developed us to test skin color in the printing of film it was always a woman she was always Caucasian she always looked that stupid she had blonde hair and red lips because you were looking for that color balance and there's a wonderful scholar in canada who if you're interested has written such amazing things about this it wasn't they didn't actually retire the surely card until the 1970s or 80s and in fact you can still find them on ebay and people collect them they're really hard to find but the fact that we were actually measuring skin color until recent recently is i think just shocking uh and on the other side is polaroid polaroid did all sorts of studies in the 50s and 60s and all of the polaroid records are the baker library at harvard uh really trying to look at black skin they don't get as much press as kodak gets for the bachelor cards so what is it about these pictures and the fact that they become documents by which we measure ourselves and again way before we get to facebook there is this whole history that is my crack which is what i call indexical portraiture so indexical portraiture is these graphic notations this visual language of that confers an incredible authority that tells us who we are and that to which is a fixed a headshot that's anime wang but when you look at this beautiful redolent in history thing that's at the hunting the library these are her papers when she landed during the chinese exclusion act when she landed and became an actor she's 22 years old 19 years old passports are indexical portraits id cards are indexical portraits the idea that each of these things bespeaks the language of its era the country it's from the typography i mean i love this stuff because i'm trained as a graphic designer but more importantly the picture is the perfunctory thing we stick on it to give it this final legitimacy and it gets lost we all become kind of anonymous because what was really interesting are these other forms and how we fit into them i think report cards are this right so i write in my book about the series of report cards that i got from the collector paul lucas every single thing on these cards is wrong i went into answer three dot com i looked at these people their names are wrong everything's crossed out and we look at this as having a kind of authority a visual authority about who we are as human beings so if you think about this as the cultural backdrop to surveillance and to the fact that we're all indexed by databases all over the world we're already losing the identity that is our god given gift as sentient people with a pulse and it is that gap between the human and what robot is this now called the humanoid that i think is really terrifying and so all of my work as an artist and a scholar and a writer is is always working with historical documents to try to reclaim these stories and understand where they went off the rails it's more measurement you know it's the school the annual school picture i'm really interested in this sort of visual language of how we compare ourselves to each other so before i close i want to just shift gears quickly this book does anybody know what these are you can't make this shit up so there i went i did a deep dive into sex dolls and sex robots and why they have the faces they do and who makes the decisions about why a woman who has a pelvic thrust monitor that you're paying 50 000 for because you don't really want a real woman don't get me started this is the baby equivalent and this is an artist named jamie diamond who actually studied with the makers of these things and there are people who actually want want a baby that's not a real baby and so this idea that we are living in a culture where not only are we afraid of being looked at but there's also a subset of people who are looking to provide and make an engineer and create these things that are just remarkably human they have individual hair follicles they ship smelling like baby powder there's this whole culture of simulation that you know we're all worried about being looked at as surveillance objects but i think there's a whole artistry to paying it forward in this way that is equally worthy of our scrutiny i'll just say i'm taking a shift with this book and what i want to end with is i'm going next week to europe for a month to work on turning this book which is 27 chapters about this crazy material which measurement is only to turn it into a television series because the stories are deeply impactful deeply human heartbreaking amazing face transplants gender fluidity othering there's a lot to do and a lot to say so i made a trailer for the book and the beauty of making the trailer was collaborating with a young designer filmmaker and we made this trailer which we shot live action because when she read the manuscript she said it's not a book it's a story about who we are as people and let's reclaim those pulses and that humanism and uh we cast 30 actors and shot for two days in brooklyn last month and shot on 16 millimeter film and it was a real game changer for me i'm i'm older than maybe everybody in this room and i didn't know i could do this and i'm really excited to see that this material that was all about my measuring what i found i mean the scholarship i'm not scholarship but it would a lot of work it's a lot of work to pull up together i mean we all have a face enormous numbers of stories but the idea that i had to go back and find these stories and she objectively took it aside and and realized that this could be something else the next chapter for this book is going to be something i think quite exciting and here it is i hope it works can we dim these lights actually for one second you can okay here we go it's my business to detect and analyze facial characteristics every year i make my observations on approximately 20 000 people and the majority of these are boys and girls i would like you to observe for yourselves the vast difference between two young girls of opposite types those on the floor have those represent opposite natures november um thank you very much i just want to say that baby casting a baby was a non-believable experience and in order to get the baby to look up and have that expression that is on the cover of the book which is a collage which is only half a baby flipped by the way because it's all about the fact that we complete the gesture and the cognitive dissonance of what we think we see is never what we see so it's a little uncanny still we wanted to a baby that looked like it and in order to get the baby to do that um my friend maggie who may be here tonight his photographer had to stand on a chair with her phone like this and play the soundtrack from frozen for you it worked thank you okay and now zag Lieberman is going to speak um zag also has an amazing piece upstairs in our exhibition i hope you'll come and try it and experience it he is using technology to augment the body's ability to communicate so he's exploring um physical interfaces with computers which is really exciting um he's the creator of open frameworks a tool for creative coding he's co-founder of the school for poetic computation um he helped create eye writer which is an eye tracking interface designed for people with paralysis and thinking about how we can control a computer not just with a mouse but actually with your eyes is really cool he's actively exploring the face as a controller and interface for software and he does in a way that's like really fun and beautiful to look at and i kind of wanted to end with beauty and zag um so i'm super honored to be here i will tell you please never agree to speak after luke and jessica um very intimidating um i want to give you a few um like interesting facts about this exhibit so what you see upstairs was presented before as ellen mentioned at the abundant design biennial and there's this beautiful part of the exhibit which is these like insane sculpture and the kind of architecture so just want to give a shout out to matter design who created this and there's a beautiful thing that happened which is we built and prototyped all of this um this this whole design in new york and then shipped it to london and actually gravity is different in london than it is in new york so when you put these rod there's this sort of insane rods in the space and they did in a different way in london and there was a bit of a sort of gravity crisis um the second thing that i'm going to tell you which is that these are sort of hints or clues is you have to ask um jessica and ellen about their scarfs so they're both wearing incredible scarf so please like yeah take advantage of that um so i'm an artist i work with uh animation and try to create extremely strange things with uh animated form love the face and exploring the face oftentimes thinking about how you know we can manipulate the face and you know explore um kind of expressive space this is an animation i made after trump was elected and i felt like we were living in a cartoon universe and i wanted to show sort of happiness for the new year and it's my deep unhappiness about the political situation um i'm gonna show up sort of precursor project and i'll talk about the project that's upstairs this project's called muskela cara created together with my partner momo we're inspired by the artist and designer bruno minari and he has this amazing spread in his book called design as art and it's um a collection of faces but what's so lovely is he's showing you all these kind of in a sort of economy how you can use such small amount of things to represent a face that you only need a few dots and because our brains are so wired to see faces that you don't need a lot of elements um there's also a concept of paradelia where you can see faces in things this is a museum of rocks that look like faces in japan i really want to go here this is a rock that looks like Elvis um and uh and when we were working on this project we were really thinking about masks and that kind of culture of mass and where you know how how different cultures use mass to tell stories and express you know what's important to them also kind of mass as as play um and this project one of the things that i loved about it's a public art project we did in houston every time we would go to houston we work with school children we build crazy mass and and then we um focused really specifically on kind of graphical representation of the face so how could you sort of show a face with a minimal amount of elements and the way this installation works is that it's um it's a camera and when you approach it it finds your face and it kind of shows you your face as a living backdrop to a poster so kind of zooms in on your face and your face becomes the kind of background to a poster that sort of attached on top and this project was installed in downtown houston and you know it's really delightful i was this i filmed this in the back i was sort of working in the back and i'll watch people on the streets kind of come up and and play and start to jam i love this person was using it out these tattoos on his face and the software is adding a kind of additional layer of tattoo um sometimes we did kind of playful things with the face most people are used to snapchat filters but we were thinking about more of a kind of at a at a very primitive graphical level like how could we what's the simplest way we could show a face and one of the things i loved about this project is you know you if you hide your face you know the software disappears and and then when you reveal your face you know this augmentation pops up and i could log in remotely from new york and watch people use this project and one time i logged in and i saw somebody on a segue and he would slowly come close and then the software would find his face and then he would back up and then software loses face and i was like i freaked out it was so lovely um the project that's upstairs is called expression mirror and the um the basic idea is when we're thinking you know how could you um like mirror expression and so this is some early test where we're trying to figure out you know if i smile or make a weird expression could i find that in a database could i find could i have a database of faces and when i make an expression with my face use that to search for an expression in in a database um and so this is what it looks like please come upstairs and give it a try when you come down it's it it zooms in on your face and it shows you your face with other people's faces so as you smile it will find a smile that looks like yours and for this exhibit in uh in new york we built also a kind of visualizer so this is showing a bit of what's happening under the hood there's a second part where you can see both um there's a part of this project which uses um this thing which which i really love called face action coding system which is essentially the muscle movements of your face so how the different parts of your face you know these are the the actual muscles these are you know nose wrinkler or dimpler chin razor um i really like nose wrinklers very sensitive so if you do this kind of and these you can think about these white lines as almost a kind of fingerprint so as you you know emote with your face that these lines represent um you know this is how the computer is seeing your face and uh i love all the sort of literature about f a c s is amazing i found this video i was telling alan i was quite excited to show this video um this is like this insane training video so actors and actresses study f a c s like you know to to sort of experiment with how yeah i could watch this stuff for days um and the other part of this project so in addition to looking at the um muscle activation the software is also looking at you know how do we perceive um emotional states so these are trained on a bunch of photographs so it's showing you you know as you use it it'll say okay i'm i'm detecting this as happiness anger fear sadness um one of the things that i really loved is this project um people always sit down and then they take out their phone to take a photograph of it and when they look down because this software has been trained on a bunch of photographs um the the computer thinks you're sad when you look down because all of the photographs of people pretending to be sad they're all like so when you when you look down at your phone it's like you get these tears and i think it's also a sort of commentary about like grabbing your phone when you're in that interactive work um and uh and so you can go upstairs give this a try um and the other thing i was thinking about as i was watching these two presentations i was reminded of a few projects that i think are really beautiful so i'm just going to show them um so i help uh run and teach at a school called the school for poetic computation and i was thinking of a few student projects that um i was reminded of in the talks one is called the average face mirror by sarah huarca and this project's quite beautiful it's using kind of similar to what luke was showing what um jessica was showing with this um alignment on the eyes this is a mirror and as you stand in front of this project it just averages all the faces it finds so the longer you stand in front of this mirror it kind of builds a composite face and it's actually quite amazing because if you come right up right at the beginning you know you have a very big impact on the image right if you're right there in the first few frames you know your face has a really big impact on the final result but after thousands of frames you have to stand there longer and so when this project is exhibited the longer this project is exhibited the the longer people have to stand and by the end it's like slowing down time it's such a beautiful experience and the images are very kind of yeah quite quite stunning um the other thing that i really love robby craft is also was a former student um he he uh there's this hashtag called faces and things which is a cool hashtag anyway because it's all of these um peridilia objects that look like faces and he ran it through the face detector so the the way i explain it so the faces and things are like you know images that look like that and he downloaded thousands of them um and then he ran them through a face detector so he said you know can the computer find a face in this and then when you average them you get this kind of like haunting face so it's finding faces in things that are not really faces and then when you average them together you wind up with this kind of eerie composite which is it's almost like a ghost of what the computer thinks a face is the last thing i will say so sort of back to the project upstairs so i had this experience yesterday i i was asked to come speak to some trustees and you know it's quite nice for the most part you know meeting these um these people from the museum but one of the trustees was like you know what what is the use of this like what is the what is how how is this useful like what how could you use it and i i couldn't really like respond i was really like flustered and so i tweeted it and it's actually quite i will just say this thread is really beautiful like i tweeted about you know i don't i wish i could come up with a good line um and people came up with like you know clever things to say but my favorite thing is this this line you know my favorite art makes me feel an idea and i think about i you know i was thinking about this in the last 24 hours like what we're trying to do or at least you know what i feel like this exhibit is about is trying to feel this idea so thank you we're gonna have uh 15 minutes to talk we're gonna clear the room at eight o'clock that's what we do like television we're in we're out okay um and i i'm just so moved and excited by hearing what everybody has to say and i'm gonna ask a couple questions but i really want to hear from this amazing audience and i know many of them came to see what what this is all about i luke um since you didn't discuss your piece in your remarks so i wanted to say something about luke's piece that he he made a piece where um the computer figures out who you are and it's usually wrong um and i thought you could say why the hell did you do that luke and what you know tell us what you know explain to these people so i i made a just sort of dystopian photo booth so you you hit a button and you sit down and then orders you to emote in a certain way so it'll pick a random emotion and be like your emotion is sad be sad for 30 seconds and then it'll judge you and tell you how well you did it and then along the way i will also um take a crack at guessing your age and race and gender and point out that that classification system is inactive use all over the place in america all the time without your consent right and so the reason it's often wrong is the the the way i made the piece was i put myself in the shoes of a mid-level software engineer at a startup who's boss called him and was like you have to use ai and so i did all the stuff that you would do if you were a person who's boss so i so i googled it i downloaded some stuff i i called my dear friend kailyn sakura is in the audience who who actually works at google and knows how this shit works and asked her to help me um i uh you know so i did all those sort of stuff that you would do if you were you know sort of a machine learning newbie and the really upsetting part of that is that there are quite a large number of off-the-shelf free open source datasets that purport to allow for this kind of recognition and they're all deeply deeply flawed either technically ethically usually both and i implemented all of them um and so it will screw up yeah you can feel the idea of ai failing and that is really part of what we want to do with this show is not do like a wiz bang technology show but actually show that there's a history behind it that this is not something invented today by some asshole like google you know that it people long ago have had these bad ideas but then also you can do beautiful things with them and i feel like zack that's part of what you do like when we first started working on the show zack would come to the meetings and go i don't want to address surveillance like i'm not doing another surveillance art piece so can you tell us about that about why you felt that way and you made the microphone i mean yeah sure sure sure um yeah i um yeah i i was quite interested in yeah trying to explore you know this this space of how the computer is sensing our expression and you know emotional state but not i not that i have anything against surveillance art but i didn't want it to be about surveillance i wanted it you know i wanted to make something which was weird and you know strange and playful and and organic and for me there was something so beautiful in london you know because i was like huh maybe my piece is not political enough like i always second guessing myself so i'm making this like you know it's blobs and minds and fun and and i'm sitting there you know sort of tweaking the software and then they turned on jessica they turned on your your videos and there were these amazing images that were just it was actually so lovely to be in this kind of room where these pieces were really in in dialogue with each other so might might you know you're using this like kind of playful blob thing but you look over to the side and you see this these diagrams from your book and for me that was the joy of being in an exhibit like this great and i have just one question for jessica and then we'll open it up to the to the floor it seems like a lot of what is interesting to you about the face is somehow the face made artificial or strange or uncanny in this capturing it through numbers or making these rubber babies what what is it about the quantified face it's all about the rubber beans well i i think i began this project in the work for the book because i had this great antipathy about selfie culture and it may be because of my generation but i don't understand with all the problems we have in the world why we spend so much time photographing ourselves and not other more important things and yet at the same time there's the very real fact that not you're an identical twin so this is an interesting question so even i had a creepy copy you have a copy but and you can probably open each other's iphone 10s absolutely but a child uh under the age i think 12 cannot because the musculature of the face has not been set yet so you have to have the maturity of a face that matches the other person then of course you know one of you doesn't gain 400 pounds or something but be that as it may the idea that really got me was that one thing you have that is yours is your face so here is this highly individualized thing which may explain why we try to measure it but it's also the fact that we're all capitulating to systems like facebook and snapchat and i'm i also have an antipathy about the forms right the fact that all of our facebook pages look alike and all of our twitter feeds look alike and we are willingly participating in something that doesn't differentiate us because it can't differentiate us because our picture has to be a certain size and that i started to look back at things like id cards and and and passports which of course have other things in them that say something really more about the country and the politics but that are this very uniform bug shot they are there has to be there's this perfunctory quality and and one of the things that to me was the most poignant in the pictures that i came across because i'm a big as a as a person who loves history and as an ephemera collector i'm really interested in found photographs and vernacular photographs and i was on sabbatical in paris a couple of years ago and there's a man who has a shop really nasty guy but he has great stuff and i would go visit him because the best thing he had was this giant box in front of the store of id pictures that had fallen off id cards so there were these like orphan photographs that had just seceded from the families into which they were born they had seceded from the forms that i collect and all they were were these photographs and most of them were taken during the war and many of them were children and i knew how many children have been deported from paris during the war and so i just i mean i spent hours coming through these photographs because there's almost a theatrical quality of trying to resurrect some history that you're imagine they are an index of someone and so then you're applying you may not be applying a facebook form you're applying your own contextual obligatory sense of how you make sense of the world around that photograph and so my book is all about the fact that we're all biased the fact that you can't actually get away from judgment that all context forces us obliges us to think about judgment in some way and i and i just love portraits and faces and photographs from children and yeah i'm stuck okay and you love it and it's a scary future okay so we have a few minutes left for some questions from the audience what do you want to know or share or needle us about hello out there yes i see someone in the back thank you so one of the things is that you mainly hear this fear-based focused talk when you talk about ai and in all of the applications that we're talking about you're exploring it so it's continuing the science down that path it's continuing the applications in daily life how what applications do you see in which because we're not going to take technology away right like it's not going anywhere we are continuing down this path right how do you use it in a way that is not about surveillance and more about observation or engagement so there's so there's a bunch there's a so there's a so my response to that is always that ai is not full is not captured entirely by things like surveillance capitalism facial recognition whatever so like i you know so i you know i i i teach with this guy named guido garak who uses machine learning to detect early you know early incidents of brain cancer right so he's using fmri imagery right and so he's got so it would be the same kind of idea is what goes into to the technology behind zax piece or my piece right he's got a million photographs of people who have had brain cancer he has sets the computer loose on it says what the hell is a pattern here and then when you take a live fmri you can start to see it right um and so that is you know we can all probably get behind that that's okay right that's a good use of machine learning um there's there's lots of other ones there's lots of creative uses of machine learning that's why i played you this ridiculous filled glass mashup right like that's a that's that's using that same technology it's called style modeling and it's in the visual arts shows up in a panoply of media artwork where you sort of like make a david bowie video look like a Picasso or whatever but it's an exact same technique right it's called style transfer um and in music it just has a different kind of vibe to it um yeah i mean but there is lots of stuff i think that the the big thing with the facial with the facial recognition and the AI stuff that we're talking about is that in the in karen will have like a really good riff on this if you see her program i assume but it's karen's hallmark because that's over 29 practice is about amazing it's just it's just that you know the it's all it's very it's very situational and contextual and this bias very much does exist so it's not so much about rewinding the clock on the technology but it's more about situating the people who develop that technology and then deploy that technology in an ethical framework where they don't use systems that discriminate right um and so i think a big problem going on now is that there's too much trust in the technology so when law enforcement local police you know uh agencies buy amazon recognition tools they all know that it's not accurate and they're like oh it's the computer trust the computer and then these things are used to falsely accuse people and um amplify bias and get people in trouble forever yeah it is a big mistake to think computers are always right a more correct formulation of that as computers will always do what they're told right and so that's an important piece of like media literacy or digital literacy that everybody needs to kind of like get behind in the 21st century because that's that's otherwise we're all going to die i think if you come we all will anyway but if you come to the exhibition you can really experience this and i i think what's really intriguing to me about zacks pieces in the show are how they kind of reveal the primitive quality of it there's almost like a back to the bow house ai experience where you see your face reduced to this line that is tracing what your nose wrinkler is doing in your eyebrow razor and it's like a new way of looking at your face that's also extremely familiar and primitive and i think seeing the kind of um at bottom simplicity of it is sort of demystifying and that's what makes it a design exhibition that we're trying to look at the how and the why and the what behind stuff that's very mysterious and um treated as kind of a magical force right tech it's magic it's not really and i think what all of you have done is to really kind of let us look inside and i have one minute yes and yes you were here first and you're going to get the last word big six i i want to hear a little bit about emotion detection i wonder if given that we do have more happiness pictures at least it's like selfie culture cosmetic happiness data is there likely to be there's eventually it's confusing confusion by these algorithms like the sadness being like there aren't that many pictures of people being sad or performing sadness but there are pictures of people performing happiness way more so i want to hear about what what do you think it's going to be the outcome of i just want to perform in sadness is my band name i mean i i think you can you can see that in you know the um you know if you go upstairs and you look at the sort of diagnostic screen you know it'll tell you how the computer is reading sadness happiness etc and it's really good at detecting a kind of expressed happiness right as soon as you smile as soon as it sees teeth you know it it's like a it's like i just call it a smile detector because it's trained the the number of images that you know the database is trained on there's so many more images that are labeled as happy and you know it's um it's like it's kind of poor in a way like it's it's it's worthwhile to try it because you can see you know it it does detect anger but it's very subtle or fear surprise they're they're quite subtle in a way and you know in a way the project invites you to perform and just you know to kind of train your face just how the computer will will perceive it not how you know natural so i really love watching people use it because they start to emote and and get a little crazy to get a response from from the machine feeling trying to get it to recognize disgust yeah yeah and but i think that is what the last slide i said about trying to feel an idea like i think these projects are really about trying trying to help you feel that's the space between these algorithms how we think about these algorithms and how they actually work that's a beautiful way to end thank you so much and thanks everyone for coming out the first night