 Currently on the faculty of Runcillia Polytechnic, Daniel Horowitz studied art history and design of the environment as an undergraduate and holds a master of architecture from the degree of University of Pennsylvania, where he received the Henry Adams Medal. He recently completed two postgraduate degrees in technology and pedagogy at the Southern California Institute of Architecture earlier in his career. Dan works closely with Robert Venturi and Denise Scott Brown, where he worked on several high profile international projects, including a science building for the University of Pennsylvania. Today, Dan will present his students work, which was developed using an artificial intelligence application called Style Transfer, which uses machine learning and machine vision to create novel new forms. Please welcome Daniel Horowitz. Hi everybody. Thank you for a nice introduction and thanks to Wendy Cox for inviting me to speak here this afternoon. Also, thank you to Matt Lutz and Kenny Onway for lunch on the tour of the campus earlier. It's my pleasure to speak to you today at the University, at Norwich University. Wendy had asked me to speak on AI or artificial intelligence. And as you all know, AI is a very big subject and obviously it's been around for quite a while. And most people are familiar with the AI applications, even if they're not aware of it. I want to say right off the bat that I'm no way an AI expert. I don't consider myself one. I do however like to think about technology and I like to use AI applications because I find them to be easy and useful. I like to operate in the technological space that Heidegger defined famously in his seminal work called The Question Concerning Technology, where he says the problem concerning technology is essentially not about technology. What this means is that the narrative we build around technology has always and will always drive new technologies. Technology itself does not drive a narrative. The best example of this is Elon Musk's Mars rocket project. He didn't build a rocket and then ask himself, hey, what can we do with this rocket? Rather he built a narrative around going to Mars. And then once the goal of going to Mars had been successfully sold, resources could be deployed to develop the appropriate rocket technology. For me, this is the way I approach technology. I like to talk about it, think about it and then deploy it. For me, it's a cultural project and in that sense, it's an architectural project. I'm less interested in the labor investment needed to develop the actual technology, if that makes sense. I will leave that to other non-architects. The application I've been thinking about and working with over the last two or three years is called style transfer. And it's based on a technology known as a convolutional neural network. I'll get into it a little bit more later in the lecture, but I'm not gonna go into great detail. This presentation is primarily focused on some of the style transfer work I'm doing right now with my second year students in the BRC program at Ranslow Polytechnic Institute in Troy, New York. I'll also be showing some of the work my students did at Pratt and CyArk, as well as some new research that I'm just beginning to work on. Now, the title of the lecture is Two Algorithms and the 436 Mandolins. They may sound like a strange title, but hopefully by the end of the lecture, it'll make a little bit more sense. First, I'd like to quickly try to find what an algorithm is. An algorithm is a set of predetermined rules that are applied to an input or group of inputs, which then consequently produce a new output. The best example of an algorithm is the baking instructions on the back of a brownie box. If you follow the five or six instructions, you should end up with a predictable, in this case, hopefully delicious result. The two algorithmic procedures I'm referring to in the title Two Algorithms and the 436 Mandolins are those performed on the one hand by the artist Pablo Picasso, and on the other hand, a set of operations performed by the computer artificial intelligence tool known as style transfer. It is the possible applications of this style transfer tool for architects that I'm primarily interested in. The image you see here was painted by Pablo Picasso and is called The Girl with the Mandolin. It was painted in analytical Cuba style in Paris in 1910, and surprisingly, it has some haunting similarities in structure and style to some of the AI-generated images that we've been working on. The images you see here in the slide were produced by my students this semester through an iterative or step-by-step process known as generative design. We are using a program that generates outputs based on the images that we feed it. This is how algorithmic design works. As we fine-tune the variables of the inputs, we try to produce novel outcomes, hopefully with some measure of control. In other words, we try to improve the ingredients in the recipe as we go. It is this measure of control that makes us architects, not necessarily the labor that went into generating the output. This is an important concept, and I would like to stress it. Iterative design is not new, obviously, as any artist will tell you, but today, powerful computers can process large amounts of digital data rather rapidly. And as a result, AI programs can do an enormous amount of iterative work very quickly and can be very useful to architects and designers. Again, the reason for the 436 mandolins. Well, when I first became interested in the painting, I googled girl with the mandolin, and I was surprised to find 436 different images by different artists with the same title. This near-instantaneous Google search result is a reflection of the way we see and process images today, as opposed to perhaps in old days, when one wouldn't have to spend the good part of the afternoon at the library, probably just to find one or two relevant images. We live in a time now where images, both relevant and perhaps irrelevant, are easily accessible and are delivered quickly and in great quantity. As a result, the gravitas or the aura of the original image has somehow been reduced, as Walter Benjamin famously argued in the work of art in the age of mechanical reproduction. This loss of aura creates a kind of flatness or equality of all images regardless of the source. But somehow this way of working doesn't seem that bad, right? It has some advantages too. It affords some novel new ways of looking at things. In a way, it frees us from the yoke of history. Yet on the other hand, by respecting the image and engaging in deep analysis of one image rather than cursory analysis of say 436 images, perhaps we look deeper and find more information. Nevertheless, we have this wonderful new tool and it's our job to leverage it to its greatest effect. A machine does not quote unquote see the way a human does and as a result, they cannot make value judgments the way a human does. But it can produce great things nonetheless and we then try to make our own judgments about them as humans. Style Transfer Technology employs machine vision and machine learning. This is how a machine learning tool uses machine vision. It reduces information to data. We might see Abraham Lincoln but an AI sees something completely different. This software, this convolutional neural network or CNN uses machine vision and machine learning to collect data from these digital images and is then trained to sort the data according to a set of algorithms and then it synthesizes everything into a new digital image or data set. These machine learning and machine vision programs are producing images that have a distinct style, a kind of AI style. In some ways it's a new style but in other ways it's not so new if we again look back at the work of say the Cubis for example. These are two freely available websites where you can experiment and create masterpieces of your own using Style Transfer AI technology. The first is called Ostrogram and the second is called DeepArt. They both work in much the same way. A computer is trained to recognize two kinds of data embedded in digital images and then to separate them into two categories. One category is quote-unquote form content and the other category is quote-unquote style content. In this case the form content on the left side is the spaghetti and the style content is Chewbacca. The result is a kind of pasta alla Chewbacca. The more images the computers are fed during the machine learning stage the better they get at recognizing the content. The computer software then processes the two images selecting pixels and embedded data based on style and form from the library of images that has been fed and has learned to decipher. It then blends the images together in a very synthetic and sophisticated way. The result in image represents a clever blend of form and style and creates a unique new aesthetic based simply on data from the two images. Now again I want to be very clear that by my using Style Transfer technology I'm in no way an expert in AI. No more say than I would be an expert in wireless communication technology simply because I was using a smartphone. For example, when Alexander Graham Bell invented telephone in 1876 his first words were essentially Mr. Watson come here. Obviously being a great technologist doesn't necessarily make you a great conversationalist. In other words you can do great things with AI technologies without being an AI engineer necessarily. That's an important point that I'd like to make. So this new AI style is not surprisingly taken hold of quite a few architecture schools. Mostly because it's easy to use and it requires little human labor to produce such intricate and seemingly laborious results of enormously high artistic merit. Here are three Instagram feeds from three well-known architecture schools. As you can tell these images are quite seductive and that's the point. We live in a time of limited attention spans and a huge appetite for new images. One might lament that the desire for fresh images has replaced the desire to truly understand or learn new things. Image alone does not equate with mastery or knowledge or expertise. So for me the images we create are no more than idea machines to spark our imaginations, to create real things in the real world of true architectural merit. It's important to remember that these images represent data only. They do not represent anything other than what we read into them. It simply is a tool for generating aesthetic data content. We as architects and designers act as curators or to use the recipe analogy again as cooks or chefs. We select the input images and control the variables until we find one that is aesthetically satisfying to us, perhaps surprises us or tastes good to us and hopefully sparks an excitement in our imagination. The tool is a kind of collaborator yet these digital images are still extraordinary originals and seem to have an authorship wholly not our own. But they are also like Rorschach prints in that anyone looking at them may see completely different things. And this is the beauty of it. The Rorschach test or the ink block test is a psychodiagnostics tool. It's used to assess emotional problems in patients by identifying the objects that the patient projects into or onto the image. This is precisely the kind of projective operation that is happening when we look at and try to interpret these style transfer images. It seems to me that the hardest part of being an architect has always been the problem of inventing a formal language of aesthetics. Typically, new styles emerge out of new politics, new economics and new methods of fabrication. In this case, the aesthetics are produced through an automated process and are void of any cultural regime. In my view, this means the images have limited cultural value in and of themselves and this makes them very different. These two projects up the screen were done by two students of mine. Avchi Aksap and Colin Tapont at Pratt. In a three-day workshop I did there with Carol Klein, who with her husband, David Rue, have been early adopters of style transfer technology in their studios at Syark, Penn and Pratt. The innovation here is that for the first time we tried to use handmade drawings and models as a way to embed intuitive content into the final results. We're trying to suggest that precedent here is important, but so is intuitive intelligence. We then use a software called ZBrush to create height fields and volumize some of these images into three-dimensional forms. The next step would be to turn these into actual buildings. In some ways, these images, although novel, are quite traditional in the sense that they have a clear structure, organization and hierarchy and aim for a kind of synthetic seamlessness that has a distinctly dogmatic classical ethos to it. The four principles, or the four principle issues that this technology makes us confront are, one, the role of labor in design. Two, the role of the image in culture and the loss of aura of the original versus the copy. Three, the question of precedent in architecture. And four, the question of authorship and influence in our work. Are we okay with the idea that we are collaborative authors and that the idea of the genius architect no longer or never did really exist? If we don't labor over our work, do we feel we are not the true authors of our work? If we see enough images of a place, do we feel we don't have to visit that place? Do we already know it? And in what way do we know it? Has the aura of that place been lost somehow by the repetition of images and copies of the place? In my current studio, we're designing a museum. In order to develop a formal and aesthetic language and to imbue or embed the projects with the human intelligence, we decided to study the work of Baroque architect and sculptor Gian Lorenzo Bernini. We did analytical studies of the pieces and then combined them with photos of the pieces to generate our style transfers. Early on, we realized that a very important structural part of these pieces are the folded pleated drapery elements. All the final images were embedded with the DNA of these folds of these drapes. Here, one of my students, Josie Moran, began with Bernini's masterpiece Apollo and Daphne from 1622. The piece is really interesting to us because it is an incredibly rich cultural artifact that embodies the highest levels of human intelligence. It is a very delicate piece set at the moment that Daphne is pierced by Apollo's arrow in the Greek story from mythology. The student was most interested in the relationship here between the bodies and nature. The way the hands become leaves and the way the ground swells into legs of the characters. Her interpretation of the piece captures the moment and movement and relationship of the parts and create fertile ground for her style transfer iterations. Here, another one of my students, Abby George, was inspired by the same piece and she developed these style transfer results. While these images are very suggestive, they are once again simply digital images. I then asked my students to imagine the spaces within them. Does the piece represent the site? Where might your building be? Where is the entry to the piece? What are the clues to the structure of the piece? To the skin of the piece? In this case, I see a skeleton structure just below the surface with a thin almost vacuum sealed skin that folds itself into layers. What kind of architecture might this become? This is the question and this is what we are currently working on. In this case, another student, Catherine Betts, did several iterations based on Bernini's ecstasy of St. Teresa. Here, you can see her quote-unquote form content piece very clearly in the bottom left corner and the style content piece very clearly as well. In order to focus on the form, she has used a filter and Photoshop that highlights the edges and planes and reduces the painterly effect of the image. This is a clever device, I think, and as a result, you can see her final piece is quite interesting. There's a unique spatial organization of two principally different parts. She was particularly interested in the golden rays of light portrayed in the piece and was able to curate that element into her final image. And I'm looking forward to seeing the next steps as she tries to interpret and volumize the parts. One technique we employ is to carefully trace the line work of the image in Rhino and then extrude the volumes and other ways to import the image into ZBrush and give different values to different colors and then create a height field accordingly. It's a challenging process and at the end of the day, cannot be solved by procedures and algorithms alone. It requires a lot of imagination on the part of the student and at the end of the day, it is this imagination that we nurture. This ability to imagine is still the hardest part for students to grasp for some reason. It reminds us that as powerful as AI is, it is still nothing but a tool at the end of the day. Students still struggle to be authors, I find. Often they become used to prescriptive studios where they are given a set of procedures to generate a predetermined result. This is not the intention of this work. Students need to be critical thinkers and trust their intuitions and be able to take a stake or a position in their own work. No AI can help them do this. Here's an interesting piece by another student, Brianda Valerio. She made a hand model out of lined paper, deforming the paper to the shape of the cups and then removing the cups. She then photographed the results to put these images back into the style transfer process. I think you will agree the final piece is a very evocative landscape perhaps with an interesting arrangement of parts. Again, fertile ground to develop a palette and aesthetic regimen for the rest of her project. Here's another striking set of iterations by Andy Martin. He was very interested in color and as you can see, he used a photo of Bernini's David in its gallery setting and curated the colors successfully into the final piece. Here, there's a fascinating kind of basket weaving language developing which may suggest a kind of structural system for his project. Here's a nice group of images produced by Jason Lindquist. Again, here we see a very evocative and mysterious kind of synthetic space of solids and voids and volumes. In the end, it is this very synthetic quality that becomes most appealing in the final images. You cannot remove any one part without disturbing the whole. This is the basic ingredient of any great work of art and this is the quality that I ask that we incorporate into the final projects. Now, I would like to touch on some of the research that I've been doing with algorithmic design and AI. Here on the left, we have Picasso's analytical Cuba's piece titled Still Life with Bottle of Rum. On the right, we have some of the possible elements or inputs or data that is in this painting and this painting is based on. Form and style elements possible. At the top right, we have a French antique journal and below we have typical cane chairs and tables of a European cafe. Clearly these elements have been synthesized using some kind of algorithm to create another kind of alternative representation. Captured in the final image is the essence of these objects or forms within their contexts or styles perhaps. The resemblance between the mode in which Picasso was operating and the mode in which we are operating is quite evident. Picasso would have simply replaced form with figure and style with context. In this case, Picasso was attempting to deconstruct reality and then reconstruct our experience and understanding of it. Analytical Cubism represented a dramatic shift in art representation as a proposed a new way of presenting the way we see, experience and feel in the world. Cubism's principle revelation was that a simple one perspective view of a thing is a very limited way of knowing that thing. The thing has a context and it changes over time. And we also know the thing as we move around it and as the objects in the setting interact with each other. Style transfer could be just another way of knowing a thing. Another way of representing our understanding of an object and its context. Figure and ground become form and style. In the end, style transfer is really a figure ground ambiguity problem. And in this way it is a kind of cultural production very similar to the kind that Brock, Picasso, Leger and others were exploring. This new representation is a statement about the ontology of things and the notion that there still remain other ways of being and feeling in the world. For Picasso, the world itself represented sensory data. His still lives provided a collection of input data and his paintings were the synthetic output images. So I thought it might be interesting to try and imagine a way of reverse engineering these images and then using style transfer technology to obscure the preconceived figure ground relationship and see what kind of similarities and differences we get. You can see the very similar kind of synthetic quality of both these images and the caning of the chair for instance translates very clearly and a very similar way to the final image as it does to Picasso painting. I'm trying to produce new synthetic digital images that provide an extension to the work Picasso and others did. And in so doing I'm attempting to stand on their shoulders in order to open up perhaps another view of the world or of reality. The painting you're looking at on the left still live with a bottle of rum was painted by Picasso in 1911 in a small town in the foothills of the French Pyrenees called Sarray. The painting depicts a round tabletop, a small stemmed glass, a bottle of rum, a pipe and a newspaper. I am hoping for my research to take me to Sarray this summer to do some detective work. Using the clues in his paintings I will try to locate the original pieces for data he drew from. My goal is to create a library of digital data. I want to draw from this data by developing workflows and digital images using style transfer technology. I hope to produce new synthetic digital images that provide an extension to the algorithmic work Picasso and others have already done. And I hope to expand our understanding of the work of Picasso and the analytic cubists while providing a roadmap forward for architects as they attempt to leverage new artificial intelligence tools. I noticed that in this painting Picasso was making a connection in time and place through the subject matter of the painting. So many artists of the past generations have taken on the same subject matter. And here we are in 2019 doing it again, albeit with more powerful AI tools. Perhaps Picasso didn't want to produce yet one more standalone final image. He didn't want to produce simply his single view at one single moment in time. Rather, I imagine he wanted to create a painting that included all previous paintings as well. I found this to be a kind of revelation that this painting is a kind of arc in time, not only connected to all previous paintings of girls with mandolins, but in fact to all future paintings of girls with mandolins. The castle himself painted quite a few paintings with similar titles as well, which is an interesting notion. He documented himself at various stages of his own life and career in terms of how he might paint the girl with the mandolin. Here we have on the right his girl with a guitar, which clearly represents a maturity of his work and perhaps even of his subject. And one could say even his viewer perhaps. As his work becomes more and more complex over time, the politics of the single view and the single viewer mapping out a single reality from which all other viewers become, quote unquote, the other is certainly being questioned today. And perhaps this view that reality is always changing and there is no single view that Trump's all other views can be quite liberating. In this way, perhaps AI can help us uncover even more human and perhaps non-human ways to see, experience and understand the world. After all, I think we are still most impressed by and perhaps even more comfortable with human intelligence. So I leave you with this final image, which I suggest presents the highest level of human achievement and imagination, or what can be called HI or human intelligence without which AI or artificial intelligence means nothing, that means nothing. This image is of course, the martyrdom of Saint Sebastian by Bernini from 1650. Thank you. For a few questions. Sure, try. Thank you. I'll see you at the end. Second year. So we introduced technology really very well in the program. So that will be like digital technologies on personal. Yeah, so a lot of like the more known schools or dinner schools are getting very addicted to these flashing images that they create really is some of these digital technologies of AI technologies. And so there's to you that become very restrictive where you follow the recipe and everybody gets a very interesting result. But if you stop and say, well, hey, you're not gonna, that's not the point here. I want you to tell me what you're seeing in your project as being important to you. What do you see in these images if you then create something on your own? How does that, I've looked at it, you can all look at this image and see all kinds of different things. But I find that for some reason the students don't have to do that on their own. They seem to struggle with this kind. Maybe the burden of their own authorship. That even though these images are very rich, they're just same. The great bio-schema doesn't seem nice to me. So whatever we see, it's highly personal. And you might see one thing in the image I want to say. So the challenge is it doesn't make it any easier to justify. It fends up just as difficult. So I have to learn how to make it. You found that students have to struggle with creativity. Is that how you think that it's time about, you know, these new flashing images and that's all the time. I mean, some people might argue that the new digital technologies design process has become, like, frustrating in many ways. And they've created more more into it. And some people also argue that the education, high school education has changed a lot. It's not different to excessive rather than the critical thinking that has. So some people do argue that I've asked some critical thinking skills that are questionable, especially not architects that actually make those are architects. You know, working on that model all night doesn't make you an architect. You get a machine to do it, right? So it's that brainpower that we're thinking that imagination, that intuitive knowledge, your sense of humanity that is an architect. Sometimes it's easy to forget that or to be disguised by the adherence of sophisticated work that we've just had in the city. So I mentioned a couple of programs in the lecture when I wrote down ZBrush and Photoshop, but what programs do you use to mix the images? Yeah, so those are the two that I was showing with the Chewbacca image. So one is called Ostagram, and one is called Deepark. Really easy to use. You guys can go on there yourself to start playing around with it. But they're still, so I have to curate the images and figure out how to manipulate and have a measure of control, right? Because it's not like you're just having an accidental result and it's not like architects design the ability to have a measure of control of the world. Which is something that's totally different.