 Welcome back to our closing plenary session. I hope that people have had a good conference. I've certainly seen some amazing things in some of the breakouts. Some of the breakouts have been every bit as good as I expected from the abstracts and from what I knew about the people. And some have been even better. And I mean that in the most positive way. There have been, you know, just some absolute unexpected gems, at least from my point of view. And I hope you all feel that your time here has been worthwhile. You will find in the program somewhat modest but factually accurate bio of Bernie Frischer. I'm not going to read you his bio. I'm going to just tell you a little about him from my point of view and about the importance of his work. He's one of the true pioneers of digital humanities. But he has staked out in his work an area of digital humanities that's a bit different than the folks who compiled textual databases or did text analysis or things of that nature. His work makes integral use of high performance computing, of visualization, of image capture and manipulation, of 3D graphics technology. It combines a very, very rich field of computer science and electrical engineering with a set of profoundly important and difficult questions both in the humanities and also in the documentation and ongoing preservation and stewardship of cultural heritage broadly. He's been involved in an enormous number of activities and he's going to tell you about some of them. I think that one of the things that's so wonderful about his work is the way it really confronts and takes advantage of questions about what do illustrations, what do 3D models and things like that mean to the humanities, to the tradition of evidence and argument? What do they also mean as parts of the scholarly communication stream? One of the most fascinating things that's happened recently is ancient Rome on Google Maps as a layer. And Bernie, of course, played a central role in making that happen. And, you know, you hear about the insularity of the humanities about how it's disconnected from the broader public. You hear about how difficult it is for one humanist to build on the work of the other. And yet we now see in something like that just a really mind-boggling new pathway being opened in the way that humanistic research can be shared and built upon and made available to inspire and attract huge numbers of people in every walk of life. I'm just thrilled to be able to introduce Bernard Frischer and I'm just delighted you're here. Welcome. Thank you for that lovely introduction. I'm glad my wife is here in the front row to have her. She doesn't usually travel with me. I told Cliff what to say, Jane, so you could hear that. Well, but really seriously, thank you. I'm honored to be here. Edward Gibbon explained the inspiration for his masterpiece, The Decline and Fall of the Roman Empire, in the following famous words. It was at Rome on the 15th of October 1764 as I sat musing amidst the ruins of the capital while the barefooted friars were singing vespers in the Temple of Jupiter that the idea of writing The Decline and Fall of the City of Rome first started to my mind. In the slide you are seeing John Battista Piranesi's view of what Gibbon will have seen, looking down from the capital line that day in the direction of the Roman Forum below. I begin with Gibbon not only because I will be talking quite a bit about what was born, our digital 3D model recreating ancient Rome, but also because his words are a typical, if unusually, eloquent expression of what initiates so much research in the humanities, a fascination with human creations in relation to time. And so with due apologies to any string theorists in the room, I begin with the observation that we live or we seem to live in a four-dimensional world and this fact has not escaped humanists. For us the goal of humanistic research is, and long before Gibbon had already been, documentation, interpretation, preservation, and transmission of the human record. This is true whether we are concerned with texts, two-dimensional images, or three or four-dimensional artifacts, monuments, or even entire settlements and cities. Today I would like to talk to you about the subset of humanists who deal with three and four-dimensional objects. They are mainly art historians, archaeologists, and architectural historians. My thesis is that the humanists who deal with 4D have benefited from the digital revolution in a special way. For us, digital technologies have blurred the fine line, separating the act of documentation from that of interpretation. I do not think that this blurring has occurred in quite the same way or to quite the same extent in the other branches of the humanities. All branches of the humanities have developed their craft rules about how objects of study entrusted to them should be documented. Here I may remind you of the complexities in editing a text whose transmission is as long and tortuous as that of the works of a Homer, Virgil, or Heraclitus. But those of us documenting three-dimensional objects have a special problem. The philologists can document a text with a text, and art historians can document a painting with a photograph. But apart from physical models, of which we have never had enough, doubtless because of their expense, and which don't travel very easily, by the way, 4D structures such as a building have not usually been documented with 4D representations. Rather, they are as a rule denoted by drawings that are abstractions which, as Ernst Gombrich reminded us, are not a faithful record of a visual experience, but the faithful construction of a relational model. To the extent that they are objective, they have traditionally been orthographic projections, that is, two-dimensional reductions to a plan, section, and elevation. The one concession made to non-reduction was the axonometric view, an intentional skewing of perspective to allow a simultaneous impression of the section and elevation. But all such projections are only possible in the case of a structure, such as the pantheon, which we've been looking at, that still survives more or less intact. For monuments and buildings that have been damaged or destroyed, something else was needed, what might be called a reconstructive vision. This is a characteristic that we especially associate with the École des Beaux-Arts in Paris, which, since the 17th century, has sent its best students to Rome to complete their education by studying and reconstructing on paper an ancient monument. The results were the en voie, dispatches sent back to Paris where the younger students, few of whom would ever be able to travel to Italy, could study the classical monuments. As a distinguished contemporary archaeologist, Paltzanka has noted, as we study these en voie, we admire their boldness, but at the same time recoil from it. This duality runs through the history of documenting three and four de monuments and arises from a mediatic feature of that history, which only a martial McLuhan could have done justice to. The almost complete separation between, on the one hand, the philological approach through ancient texts and descriptions written up in modern texts, and on the other hand, the graphic approach drawn up in modern projections and reconstructed in architectural exercises like an en voie. Thus, the founding work of European archaeology, Papal Secretary Flavio, beyond those de Roma in Starata on Rome restored, written in the 1440s, was a systematic account of the ruins of ancient Rome, written without a single map or image. In the next century, papal architect Pirro Legorio composed his ancient, his image of ancient Rome as an elaborate 2D bird's eye view, but none of the features in the city is identified with an inscription and no written explanation for the thinking behind the reconstruction is provided. So we don't really know what we're looking at. Indeed, by the time of Piranesi in the late 18th century, there was even an explicit battle between the textual and the graphical approaches. Piranesi, the great architect and engraver, was, of course, on the side of the graphicists. His enemies were scholars such as Mariette and Dishopi, whose works on the monuments were all texts and no images. The most amusing expression of this quarrel is perhaps the work of Piranesi, which you see illustrated on the screen. In it, Piranesi sends up a three-volume publication of the archaeological site of Horace's villa, Nertivoli, published in the late 1750s by the French Abbey Bertrand Capmartin Dishopi. Chopi's three-volume work obviously has many words, but not one illustration. And as far as those words are concerned, by my estimate, no more than 5% have anything at all to do with its ostensible subjective study, Horace's famous Sabine villa. On the spine of the books, you can see that Piranesi calls Capmartin cappoconfuso, or muddlehead. He also gives the site plan missing from Capmartin's book. The scatological shape of the structure on the plan explains why Piranesi scholar John Wilton Ely gave the engraving what's become its technical name in the Piranesi literature of the turd engraving. Yes, I think I'll turn that one off. Of course, since the 19th century, which brought us the idea of Altertum's Wissenshaft, or the interdisciplinary study of antiquity, the textual and graphical traditions of archeological documentation have been successfully merged, at least in the Gutenbergian medium of the printed report. But given the 2D nature of the printed page, the graphical expression was always the reductive solution of the plan, the section, and the elevation, with an occasional axonometric drawing thrown in for good measure. Unfortunately, the odd and indefensible separation of text from image survived in the best, if rarest form of 3D illustration, the 3D physical reconstruction model. Here I am thinking of such great 20th century examples as the Holy Land Hotel model of Jerusalem just prior to the Roman destruction, or the great Plástico di Roma Antica in the Museum of Roman Civilization in Rome. The latter was made over a 40-year period under the direction of architect-archaeologist Italo Gismondi at the enormous scale of 1 to 250. So it fills a room that is 20 meters square in a suburb of Rome. It has the reputation for being very accurate, but it is difficult to check the validity of this claim since Gismondi published nothing at all about his sources and his methods. So when we look at Gismondi's model, especially the thousands of structures that are not obvious landmarks such as the Colosseum, we simply do not know what we are looking at and why Gismondi's team reconstructed them as they did. So to sum up my main point thus far, humanists who work with 3D and 4D objects have not had an easy time of fulfilling the first of their four basic charges as a humanist to document, interpret, preserve, and transmit the human record. Has the introduction of digital technology changed anything? I think the answer is obviously yes, but it took us 20 years to reach the point where we could begin to make that claim. Until then, digital tools in the humanities, especially the 4D humanities and 3D humanities, were used more to improve our efficiency in doing the same old things in a different way, not to transform our methods and our disciplines. I can cite myself as a case in point. I first publicly proposed making a 4D digital model of ancient Rome at a small conference held at Apple Computer in 1986. The idea went back to discussions I had had with UC Berkeley urban planner, Donald Appleyard in Rome in 1974 when we both first saw that Gismondi physical model of ancient Rome. The theme of the Apple conference was the CD-ROM, a new storage medium that could hold 750 megabytes of data. Imagine that. Leading lights including Doug Engelbart and Alan Kay were invited to discuss what one could possibly do with all that storage capacity. I think there's some younger people in the room, so I will say that it may seem odd that way back then this was even an issue, but I recall buying my first PC just five years earlier in 1981. It was a K-Pro and cost $2,000. Computers always seemed to cost $2,000. That hasn't changed. But those were 1981 and $2,000, so I guess they'd be equal to 6 or 7 today. For storage, it had two 128K floppy disk drives. I asked the salesman if there was a Winchester available. Winchester is what we called a hard disk back then. He said, yes. The bad news is that it cost $10,000. The good news is that it has 10 megabytes, so it'll last you the rest of your life. That's a true story. You can see why in 1986, 750 megabytes seemed like a lot of storage. At Apple, I proposed to use the CD to hold the data needed for a digital representation of Gismundi's model. My motivation was to promote interest in my field of classics by making it possible for students to do virtual time travel to ancient Rome, where they would be met by a guide and taken on tours of the city. I was inspired by the Aspen project, as Nick Negroponte stated, used video disks to allow the driver, the user, to drive down corridors or streets as if the vehicle were located in those corridors or on those streets. Every street of Aspen, Colorado was filmed in each direction by taking a frame every three feet. Similarly, every turn was filmed in both directions. I thought we could run a miniature robotic camera down the streets of the Gismundi model and do the same thing for ancient Rome as the MIT group had done for Aspen. However, when we studied the Gismundi model, we discovered that this was not, in fact, a good idea. Only about 15% of the surfaces of the buildings are detailed with doors, windows, balconies, etc. The rest, those that cannot be seen by the museum visitor from the sides of the room, were left as blank white plaster. So a video disk of ancient Rome made in this way would have been quite uninformative, not to say boring. Instead of working on a 3D simulation of Rome, I made a hypermedia interactive tour of the Villa of the Papyrite in Herculaneum using the J. Paul Getty Museum in Malibu, a physical reconstruction of that villa as my source of data. I tied shots of the Getty Museum to a corresponding plan showing the rooms of the villa, and I provided a library of Roman texts in which the ancient authors addressed topics somehow relevant to a particular space in the villa. It is strange to see these screenshots today and to recall how cool these rather expensive black and white scanned images seemed to us in the late 1980s. But visual quality aside, the idea of tying text to spaces in order to create places, places invested with meaning, was to have a bright future. Today it has led to the famous spatial turn in the humanities. This is a widespread movement across our disciplines to show how humans project meaning onto their physical environment. My interest in 4D digital reconstruction proper revived when I saw an IBM-sponsored show about Pompeii in London in 1990. In the slide on the screen, you see one of the 3D models shown at the exhibition, a model reconstructing the stabian baths. In retrospect, the Pompeii exhibition was less important for its inspirational displays of such reconstructions than it was for the essays in the catalogue presenting the research results of IBM's CMEA lab on the potential of IT for cultural heritage. In one of these, entitled Fan Neumann meets Pliny, Stefano Bruscini explained how digital technology could help represent the complex knowledge that archaeologists assemble about the archaeological artifact and site. He described a digital implementation of Vannevar Bush's Memex in which the computer screen is a window open onto texts, codes, archaeological and technical maps, images of documents, frescoes and iconographic sources. He recognized that creation of this digital memex could greatly improve the efficiency with which archaeologists, conservators and cultural resource managers could do their jobs. But he also threw out the tantalizing idea that the memex could also be what he called a knowledge multiplier where the computer is no longer a mere organizer of knowledge but a methodological operator that provides not only information but proposes precise actions and research. But it has to be said that for understandable reasons knowledge multiplication had to wait a long time to emerge. First, those graphics I just showed you had to be greatly improved, not only mine but also IBM's I have to say. By the mid-1990s, it clearly happened as research advances in the then-hot field of computer graphics were applied in new software packages such as Maya and 3D Studio Max. A compendium of work from this period was published in 1996 by Maurizio Forte and Alberto Siliotti. As you can see from this shot of another reconstructed building in Pompeii, the Temple of Apollo, photorealism had by now reared its beautiful head. At this point, three things started to happen. First of all, the scholars started to review the work done by the companies and the result was a grade of C-minus. In a thoughtful review of the Pompeii model, archaeologist Daniel Brighman wrote, the design team had fashioned their temple complex out of mural vignettes excised from different archaeological sites and contexts. Painted panels along the periphery of the reconstructed sanctuary were unsettling to many precisely because they had been filched from other Roman cities. Thus, the reconstruction took the form of a pastiche. Not the kind of thing scholars of antiquity are bound to love. On the bright side, at least we scholars had started to sit up and take notice. The Temple of Apollo model, like almost all the models illustrated in this book, were made by commercial companies, typically the very companies that made the hardware and software needed to produce digital creations like this. By the second half of the decade, some humanists had started to produce models themselves. A milestone event was the meeting of the leading professional organization for digital archaeology, Computer Applications to Archaeology, or CAA for short, which doesn't stand in the rest of this talk for the College Art Association. It stands for Computer Applications to Archaeology. This meeting took place in Barcelona in 1998, and its title was Virtual Reality in Archaeology. In the introduction to the conference proceeding, CAA President Nick Ryan expressed the view that digital modeling was useful because it was a better form of illustration, to quote Ryan. The advantage of virtual computer models in comparison to traditional analysis is evident. Models, actually I should do this with an English accent, but I'll spare you that. Models permit spatial queries such as what is next to what, what surrounds what, what is above, below, and to the side of what. Another advantage is the ability to generate sections, views and elevations, and to visualize physical properties. In other words, according to Nick Ryan, with digital modeling, we can have our plan section and elevation without resorting to the trick of the axonometric view. My group from UCLA presented our first model of Rome at CAA Barcelona, a reconstruction of how the Pilgrimage Church of Santa Maria Maggiore in Rome looked when it was new in the year 435 AD. In the related publication, we stressed that the model had been made as a collaboration of model makers working with subject experts. In fact, the members of our scientific advisory committee were quite distinguished and qualified with the exception of that chap on the right. The curator of antiquities of the Vatican Museums, the superintendent of the church itself, and a Dutch scholar who had written, written the most recent monograph on how the building looked 1,500 years ago. We also pioneered the idea of having a software switch that could be thrown to visualize alternative reconstructions. In this slide you see our two versions of the nave. So we were not only making our models look photorealistic, we were also learning how to go about the task in a traditional scholarly way. The next big problems were how to publish online models like our reconstruction of the Roman Forum, and how to provide a built-in digital link between features in the models and the underlying archaeological documentation. In 2002, we received a grant from the National Science Foundation to experiment with how these things might be done in the days when good bandwidth was not something we could count on. We took our model of the Roman Forum and converted it to QuickTime VR format. We tied panoramic views of the digital reconstruction to a time-sensitive map of the Forum, and we synchronized the views found at the various spots with views of how the Forum looks today from the same vantage point. Finally, we provided documentation from the foundations to the roof tiles explaining what our evidence, or in the absence of physical evidence, our reasoning was behind each major element of a monument. Others elsewhere were working along the same lines, and the climax of our independent efforts to tame the new technology and make it a legitimate medium of scholarly expression came in the early years of this decade with an important initiative, the London Charter for the computer-based visualization of cultural heritage. The preamble says it all. A set of principles is needed that will ensure that digital heritage visualization is, and is seen to be, at least as intellectually and technically rigorous as longer-established cultural heritage research and communication methods. One is almost tempted to add harumph. Now, in the second half of this decade, things are shifting again. The community of 4D humanists has come around to the view that we understand fairly well how to use this technology as a form of scholarly expression and communication. Thus, at the annual meeting of CAA held this March in Williamsburg, it was no longer possible to have a paper accepted that merely presented the results of creating a 3D reconstruction model of a cultural heritage site. That was considered old hat in the absence of some additional innovative feature. What are the challenges that require innovation today? I want to talk to you in some detail about the three most important because I'm sure that there are many present in today's audience who can help us over hurdles that seem at times so daunting. In brief, they are providing real-time use on the internet with secure remote rendering to protect intellectual property, collecting, preserving, and disseminating the models on the internet, and figuring out how to use a model, not simply as an illustration of what we knew before we made it, but as an instrument of new discoveries like a telescope or a microscope. Let's start with the first problem. So far in this talk, I've shown you static screenshots of the models I've been discussing, like this shot of Rome reborn in the slide. But of course, the models were not primarily created to be used that way, but as virtual spaces that can be explored interactively, which means that we can see them at a frame rate of 20 or more FPS and have complete freedom to move up-down sideways and to change our pitch and orientation as we do so. Achieving this frame rate can require great computational power and the geometry being manipulated is complex. And transmitting the frames over the internet requires good bandwidth. You're watching a screen capture of a real-time fly-through of Rome reborn, our digital model of Rome in the year 320 AD. The model represents over 7,000 buildings covering 25 square kilometers. It consists of over 400 million polygons, which makes it a very complex piece of geometry indeed to try to run in real-time. This video, which you've been watching, was made for us by IBM on one of their BladeCenter QS21 servers, which cost well over $100,000. At the moment, we cannot get this kind of performance on standard PCs, nor can we pipe it over the internet to the screen of the end user. Even when the average PC can run a model like this, it is unlikely that the creators of cultural heritage models will in general permit or will be permitted to allow their products to be downloaded from the internet for local use on a PC. I say this because of an experience I have repeatedly encountered when obtaining permission to scan an important cultural monument. The authorities in charge are happy to grant the permission but only if we can guarantee that their intellectual property rights are protected. So we have had to develop a system that simultaneously offers the end user a high-resolution photo-realistic view of the digital replica while not letting him simply download it to his local machine where it could be subject to cyber theft. The solution was scan view. It was developed by Dr. David Kohler, a Professor of Mark Lavoie's graphics group at Stanford in computer science in the late 1990s when Stanford was getting permission to scan Michelangelo's David and other works in Florence. In the slide you are seeing not the real David Lavoie's digital version of the David. Scan view implements the idea of secure remote rendering. Using this approach the end user downloads a very simplified and commercially worthless digital model of the monument onto which is draped a high-resolution view from the full model which is securely stored on a remote server. With decent bandwidth the whole process seems to be happening in real-time as you can see in the screen capture I've been showing you of a typical session viewing the digital David. We can even change the lighting as you see. What about a much bigger digital model such as Romery Bourne? We are working with mental images a subsidiary of NVIDIA and the Italian company ANEA to find an analogous solution to scan view which will take advantage of ANEA's 2800 core IBM server cluster to do the remote rendering. You are watching the screen capture of a recent real-time session. We started south of the Arch of Constantine in Rome and then proceeded north toward the colossal bronze statue of the sun god next to the Flavian amphitheater home of the gladiators. That's not a football team by the way. By the way we call this facility the Colosseum because it is next to that now vanished 100 foot tall bronze statue. You can see how quickly we can move through the model despite its 400 million polygons. This is because mental software progressively renders each frame and the longer we tarry the more pixels come in. Ultimately when we find a location of special interest as you can see in the video we can stop and set up a high resolution rendering. The mental ANEA system will be implemented in the first six months of 2010. It should allow us to greatly improve the frame rate and also to support the simultaneous login of dozens of end users around the world. Stay tuned to this channel next year to see our first results. Here you can see the rendering which we're zooming in on. Once we have figured out how to securely serve even very complex models we can confront the second challenge collecting preserving and disseminating the models on the internet. Ironically 4D humanists who devote their careers to preserving the world's cultural heritage have not spent much time thinking about how their own contribution to that heritage can be saved for future generations. Remarkable as it may seem to date there is no online scholarly repository in which 4D humanists can deposit their digital models and in which student scholars in the general public can find a scientifically produced model. The only thing we have along these lines at all is Google's KML format which of course only collects the very simple kind of models you can make in Google's KML format and which offers no peer review so there is no quality control. Thanks to two generous grants from the National Science Foundation our research group has been working on what we think is a better solution. We call it SAVE which stands for Serving and Archiving Virtual Environments. Our vision is a peer reviewed online journal in which scholars can publish their 3D and 4D models with links to the documentation along with related monographs. Through a needs assessment survey which over 300 scholars took we found that over 90% consider creation of something like SAVE to be a top priority in our field. We have also had a strong expression of interest from one major commercial European publisher, not to be mentioned, whose name is not to be mentioned in this company I fear, to support the journal and we would be happy to have other publishers especially in this country. Finally, let me talk about the new hot thing in the 4D world. How to use a model to make new discoveries, in other words for what Stefano Breschini called Knowledge and Multiplication way back in 1990. A year ago we published a book with a dozen examples of pioneering work along these lines. Let me just cite one example, the research of David Kohler First a word about the remarkable monument itself. The marble plan is a highly detailed map of Rome in the early 3rd century AD. It was made at a scale of 1 to 240 and was exhibited on the wall of a room in the Temple of Peace in the city center. The map was in size on 150 blocks of marble and measured 17 by 20 meters so it was very big. It showed not only all the streets of the city as you would expect in a map of the ground plans of all the buildings. In this slide you see a close up of the map showing the ground plan of the Porticus of the Empress Livia. A site that offered a place for relaxation and some green space and shade in the heart of the city and it's never been excavated. On the basis of the map we can see all the columns which I've now highlighted in red and we can observe that the structure was surrounded by shops highlighted in blue we can see a square step structure highlighted in yellow in the middle of the courtyard which we know was a garden so this square structure was probably a fountain. We can see something else too. The fact that the map comes down to us in fragments. In fact we have over 1900 fragments constituting about 10% we think of the original map. This slide shows what remains highlighted in orange. The other 90% was melted down for lime The name of the game for scholars since the marble plan was discovered in the 16th century has been defined joints between the fragments so that the map or what is left of it can be reconstructed. By 1995 about half of the fragments had been joined but that still leaves hundreds in cartographic limbo. In the late 1990s Stanford computer scientist Mark Levoix had the idea of using 3D data capture and modeling of all the fragments in Rome while he was in Italy scanning the sculpture of Michelangelo. His doctoral student David Kohler eventually took over the task of making the models and making sense of the data. He developed algorithms for puzzle solving automatically using the computer and in his dissertation he presented these and his application of his work to the insanely difficult case of the marble plan. A jigsaw puzzle which comes with no picture on the box. In this series of slides I'll show you how Kohler matched up two fragments. First the computer noted a similarity in the lines incised on two pieces catalogued as 351 and FN9. This made it likely that the two fragments belong to the same general part of the map but it didn't tell us how close or far apart they were. Then in manipulating the 3D models on his computer screen in Stanford Kohler noticed that there was a concavity on a side of FN9 that seemed to fit nicely with a convexity on the side of FN9 that seemed to fit nicely with a concavity on a side of fragment 351. Of course to be sure that the two fragments actually fit together there was no substitute for going to Rome and we're always looking for excuses to do that and opening the cases were all 1900 remaining pieces of the map are stored. So Kohler went and opened the cases with the fragments were opened and lo and behold the fit was a snug as the computer models suggested it would be. Kohler has demonstrated over 20 such new joins and this was an amazing achievement coming after 400 years of archaeologists working on these fragments with traditional methods and my wife and I saw them at work doing this in the 1980s which basically moving these very heavy things one cubic meter weighs a ton so moving these fragments around on long trestle tables and hoping to find joins very tedious and work that relies very much on chance and good luck. Kohler's software works offline what we need to do next is to make it possible for end-users to make such discoveries online. A new group of 4D humanists was formed this year to do just that. This is called the Humanities Working Group of the Virtual World Alliance we were brought together by SRI International and the Federation of American Scientists through a Mellon funded project called Harnessing Virtual Worlds for Arts and Humanities Scholarship. So what is this all about? Potentially it is a way to convert our 3D worlds from being places that the end user can only passively view to places she can explore like a scholarly time traveler with a bunch of new insights and discoveries. Another way to think about our Alliance is that its goal is to tame Second Life, that beast. Don't get me wrong Second Life is wonderful and I use it all the time. As the Linden Lab website claims it's a place where we can unleash our imagination socialize with people from all over the planet, buy things visit museums, have serious meetings and even get educated. But can you do humanity scholarship in a virtual world? I mean, of course, can you do anything besides use the virtual world as a place to meet and discuss scholarly matters with colleagues? The Humanities Working Group of the Virtual World Alliance has the goal of answering this question. We are studying how research environments can be created using virtual world technologies such as Second Life and especially the open source equivalent which is called Open Simulator. We like OpenSim not only because it's open source but also because it has loaders making it easy to import an existing 3D model into the virtual world, something promised by Second Life for several years but also something Linden Labs never seems to get around to. Since the digital humanities community now has several thousand scientific models the ability to port over all this good work to the new technology platform is a major advantage of OpenSim. The technology has been defined as follows by SRI and FAS. A virtual world is a computer program that generates a dynamic representation of a real or imagined world and embodies the essential qualities required to support higher-order cognitive functions. The key characteristic of this dynamic representation is that it takes the form of a simulation of the real world by offering literal replicas of three-dimensional objects whether animate or inanimate and the passage of time. The behaviors and interactions of the 3D objects over time can be constrained by their physical properties. Using virtual world technology we can see a building the people and the furnishings in it. We can listen to what the people are saying and to the music that they are hearing. Moreover, we are not limited to passive observation but can actively participate in the scene through a self-representation called an avatar. In short, virtual world technology supports the creations of realistic literal representations of 4D information. Once accurately represented, the lost historical world can become the subject of empirical and experimental research in a way unthinkable or impossible in the past. We can seemingly turn back the clock and return to the past. Let me give you a couple of fairly simple examples of what has already been done and then conclude with a project proposal we are developing that we take this kind of research into a new level of sophistication. In the examples, our lab has used Second Life for prototyping encounters between me and Amanda Levin, CEO of Pleiades our partner for virtual world development. In Second Life, my name is Hermarkis Afterthought. Afterthought is my last name. Hermarkis, my first name. Amanda is a name that can't be pronounced but is spelled M3DUSA which I guess is sort of like Medusa but I'm going to call her Amanda. The first encounter I had with Amanda concerned 3D modeling. We met in Rome reborn's reconstruction of the Curia Giulia, the Roman Senate House on the Roman Forum in the heart of ancient Rome. The meeting was staged as a discussion of Pleiades' work in correcting an error I had spotted in reviewing a first draft of their conversion of our model to Second Life format. I think you will quickly appreciate the way in which this technology supports real-time communication between people at a distance and be in the same virtual space see the same things and interact with each other and the space in a way that is otherwise impossible and best of all it doesn't cost anything. So Amanda, hello. You're working in Montreal. I'm working in Virginia and you're really an excellent 3D modeler. I really love what you've done with the Roman Senate House, the Curia Giulia. Thank you very much. When this was created to discuss the ceiling Yes, the last time I was here a couple days ago I pointed out to you there was a gap where the ceiling hits the top of the wall just behind me to the right I could see the sky. Have you been able to fix that? Okay, I'm going to go in and take a look. All right. It's right in that corner there. I was able to see some daylight when I was here but I don't see that anymore. So thank you very much for fixing that. The second video simulates a scholarly encounter. This time I wear my scholarly hat and Amanda pretends to be a colleague at another institution. We are discussing the philosophy of restoration. How do you deal with the all too common problem in humanistic research of uncertainty owing to the loss of data? To set this up let me draw your attention to the upper left now shot of the interior of the Senate House. You can see that the building is preserved to the ceiling. We have the ancient floor, the three deep steps on either side of the hall where the senators put their chairs and the speakers platform at the end of the hall. We have some fragments of the marble wall decoration but mostly what you see is the unadorned brick since the medieval strip most of the marble away for reuse elsewhere. Okay, with that background in mind I'll run the second little video. Welcome to our building reconstructing the Roman Senate House. How are you? Thank you very much. I know you're here for the first time and I wanted to show you the building especially the interior and discuss it with you. We have a lot of as you know the building was studied in the 1920s and 30s by Bartoli who published the remains and there are a lot of remains in this building in Rome. I mean it still survives up to the roof but the interior space had only spotty evidence. The floor was well preserved the steps were well preserved and we talk about registers of wall decoration. Each one is bounded by a molding so there's a low first register here and then above that there's a second register up to this big thick molding and then a third register and up to that point we have evidence of one kind or another but when we get to the fourth and the fifth registers our evidence fails us. So I wonder what your opinion is what we did was we kept the fourth and the fifth registers very simple. Yeah I know what you mean I mean the Romans who said you know America is a less is more culture ancient Rome was a more is more culture. Alright that's a simulation of a scholarly exchange in a virtual world but virtual worlds are not only useful for putting contemporary scholars into past spaces and allowing them to talk about them they also allow us to populate the reconstructed ancient spaces with furnishings people and activities to test out hypotheses about how things worked. As an example I would cite our partnership with the School of Engineering of the University of Zaragoza to study how many spectators could fit into the Colosseum in Rome. This is a hori scholarly problem and the estimates on record by reputable archaeologists range almost absurdly from 35,000 to 78,000. Why is this even a problem? As you can see in the lower right now shot of the Colosseum none of the interior scene survives so you have to apply a lot of guesswork and hypotheses about how the space was articulated with reference to the seating. The engineers in Zaragoza use structural engineering software to populate our Rome reborn model of the Colosseum with artificial intelligence figures who may not be handsome but they are smart. This video clip shows a screen capture of how the figures entered the seating area and managed to find their seats without falling down or bumping into one another. Everyone always laughs when they see this. By the way the correct answer turned out to be 48,000 to 50,000. Besides running experiments in the past and making the study of history and archaeology more precise and empirical we can also use this technology to re-experience the past. The point in doing so is to test out hypotheses of reconstruction about how things happened and once we get that right to create the conditions whereby we can try to interpret the world as someone in the past might have done. In archaeology this is actually a hot area called landscape phenomenology and it was pioneered by Christopher Tilley in the book you see on the screen. In literary and cultural studies the study of how people make spaces into places by the projection of meaning derives from the notion of the chronotope developed by Russian theorist Mikhail Bakhtin. For example what if we could use virtual world technology to figure out how the gladiatorial combats in the Coliseum actually worked. Up to now we have been limited in understanding these highly dynamic and fluid events by the worst possible kind of evidence static images such as those seen on the screen from relief lamps and mosaics. The new technology allows us to construct the various kinds of gladiators the murmillos with their oblong shields and high crested helmets or the hoplomachy with small round shields. It allows us to drape motion captured from living people over these figures and we captured the motion from the gladiator club on the Via Appia in Rome which reconstructs the the combats using rubber weapons. And finally it allows us to give control of the animated figures or the avatars to scholars who can then fight it out amongst themselves. Through these tests they're very serious of course conducted in a very sober atmosphere the strategies used by the gladiators can be better understood. Once reconstructed they can then be observed to help us understand things like the audience reaction or the interaction between the crowd and the athletes. Well in fact all the things I've just said are possible and have been done. I show you now a screen capture of the final result reconstructing the fight of a murmillo and hoplomachus in the Roman Coliseum. Well these examples that I've shown you so far are off I would say fairly simple conceptually but it is easy to predict that virtual world technology will be used for more complex experiments and experiences. Our lab is developing a virtual world of the famous villa of the Emperor Hadrian near Tivoli. This World Heritage site is well documented and studied. It covered almost 300 acres of land and was filled with enormous pavilions, palaces, porticoed gardens and so on. It is estimated that it took over 3,000 people just to run the place which was essentially a mini city that served as a retreat of the Roman Imperial Government where the top 1-200 officials could work in splendid isolation. As that en voie that I showed you at the beginning of my talk attests it has even been reconstructed in 2D many times and we now have a nice 3D physical model. On the screen you've been seeing a recent artist rendering of a bird's eye view of the reconstructed villa. So we have chosen to focus on this because at this point scholarship has progressed from excavation documentation and reconstruction to analysis and interpretation. A book published in 2008 by Federica Chiappetta stands out for advancing research in a new direction. The investigation of who used the various rooms and structures in the villa and at what time of day they tended to do so. Chiappetta's thesis is that Adrian's villa was designed to minimize the contact between the service staff and the privileged elite. She furthermore claims that the spaces of the villa were structured to avoid exposing the emperor and empress to environmental irritants such as inclement weather, smoke from everyday activities, noise from the work of slaves tending the villa and from the animals and vehicles used to transport supplies to the villa. In support of this thesis Chiappetta develops the first analysis of the functionality of the villa studying almost every room in each building from the point of view of who used it for what purpose, at what times of the day and in what way. She divides users into six distinct groups the emperor Hadrian, the emperor Sabina high court officials invited high-ranking guests petitioners, normal citizens in other words and servants and staff. She presents architectural plans that show the typical paths of circulation of each of these groups through the individual building complexes. There are over 130 such plans in the book. The slide you've been looking at shows you one example, the so-called imperial palace. Chiappetta supports her thesis by argumentation that is both logical and cultural. Architectural plans of the individual components of the villa are enhanced with lines showing the itineraries of the groups that use them. But these plans are simply illustrations of the thesis. They do not permit the thesis itself to be tested. Moreover, Chiappetta's form of illustration is 2D, but the problem she addresses is 4D. The users of the villa are 3D as is the villa spaces themselves. The movement of the users through the villa adds the important fourth dimension of time. For example, two groups who ought not to be in the same place at the same time perhaps appear to do so in Chiappetta's 2D illustrations because the element of time, of course, has to be ignored. Finally, each of Chiappetta's 2D views is devoted to a single complex in the villa. There are over 20 of these complexes. There is no composite diagram showing all the itineraries of everyone. This is not surprising because such an illustration would be extremely large, would require many different colors and in the end would doubtless be so complex as to be undecipherable. By the way, I hasten to add that I don't think any of these comments to be critical of Chiappetta who is in fact on our team and who would be the first to agree with what I've just said. To test and refine Chiappetta's thesis we need to have a virtual world of Hadrian's villa. A 4D architectural reconstruction of the villa with avatars representing the dynamic motions of the people in the six groups and including dynamically-spatialized environmental features such as smoke, sound, and light. The resource will allow for experimental history of a kind otherwise impossible short of actually rebuilding the villa and then finding 3,200 people to show up to simulate a day in the life of the emperor Hadrian at his villa. Having created a highly detailed CAD model of the villa over the past two years we are now seeking funding to convert it to a virtual world format so that we can test out Chiappetta's ideas. Last summer David Kohler who had in the meantime moved from Stanford to our university joined with me to start the virtual world heritage lab to support research that takes 3D technology and moves it beyond illustration. The scope of our interest as is implied by the phrase world heritage includes the entire human record. The focus of our investigations as is suggested by the phrase virtual world is this new technology and how it can make possible experiences and experiments that otherwise would not be possible. We are working with scholars such as Federica Chiappetta to use this technology to go beyond research undertaken with traditional tools and methods. We are cooperating with our partners in the humanities working group of the virtual world alliance to create the cyber infrastructure needed to link multiple historic virtual worlds so that each fits into the right cell of the space time matrix. We don't make the wild claim that virtual world technology to transform all of scholarship in the humanities any more than we would claim that the GUI of today's internet will be replaced anytime soon by a virtual world interface. But we do believe that there are some and probably many concrete problems in traditional humanities research that have been graphically reduced from their normal four dimensions to an abnormal and abstract two owing to the limits of Gutenbergian technology. And we think that by using the new technology to restore the two missing dimensions we can more accurately and interactively test historical hypotheses about how culturally significant spaces functioned and were symbolically invested with meaning. And we think that this is a good thing. If you know of any humanists at your institutions who might be interested in a collaboration with our lab, please do not hesitate to put them in touch with us. Thank you very much. I'd be happy to answer questions if there are any. And I know there's a microphone in the middle of the room. Fabulous. It was just that stunning all the work you've done, Bernie. Quick question, have you gotten any funding or have you thought about applying to funding to gaming companies? You think of the success of games like Rome Total War that bring that kind of historical perspective in a gaming setting and how they'd be interested in this kind of work, I would think. The short answer is no. But if you know of any that might be interested, I'd certainly like to contact them. Well, thank you again. Speechless. Questionless, at least. Thank you to the echo, Martin. That was absolutely tremendous. And there's so much in there to think about. I really appreciate you sharing this kind of really deep look at the effects of this with us. So thanks so much. And with that, we are adjourned. Let me wish you happy holidays, safe travels, and I will see I hope all of you sometime early in the new year. And if not in April. Thanks again.