 Welcome everybody and thanks for joining us. I'm Cliff Lynch I'm the director of the coalition for networked information. You have reached one of the project briefing sessions on the beginning of the third week of the CNI fall 2020 virtual member meeting week three is dedicated to technology infrastructure standards and related issues. And I'm delighted you're here with us. I think you're really going to enjoy this. A couple of logistical things we're recording this session. The recording will be available to meeting attendees quite quickly, and shortly thereafter to the general public. There is closed captioning that you can make use of if you wish. In the chat box please feel free to use that as we go along. And there is a Q&A tool at the bottom of your screen. We will take Q&A at the end and Diane Goldenberg Hart from CNI will moderate the Q&A session. I do want to note that with week three we have released a bunch of pre-recorded videos as well as the live sessions that we'll be doing throughout the course of the week. With that, let me turn to the topic at hand. We have with us two speakers today, both from the University of California at San Diego, Roger Smith and Scott McAvoy. And they are going to talk about a project which I think is just marvelous. I saw it and just felt we had to get it to CNI on a couple of bases. This is a very technically interesting project and it sort of stands in a series of efforts to reconstruct based on photographic evidence. I'm thinking of things like Microsoft researches work with photosynth over the years. It also raises a lot of really strategic questions about the roles of special collections, the role of libraries, about how we document place and historic place, especially in an era where we are losing historic places, because of wars, climate change and other disasters. And so I just think it's one of these things that's absolutely what I hope to bring to CNI. So with that, let me shut up and turn it over to Roger, I believe you're going to start. I am. Thanks for being here. Thank you for having us Cliff and thank you everyone for joining today. Scott and I are so pleased to be able to present our work on the UC San Diego effort to reconstruct digitally the ancient Temple of Bell and Palmyra Syria, which is an effort to preserve the overall cultural heritage of the region of Palmyra, along with other sites around the globe for future generations. So just briefly, I will outline the presentation today and talk a little bit about what we'll be sharing. I'll provide a little context and framework around the work of the digital media lab and also its home program within the library the scholarship tools and methods program. So we'll do a deep dive into the Temple of Bell project. And then I'll wrap up with a little bit broader context around the future of our endeavors in this space our repository work and how we're connecting to our campus endeavors. And then of course we'll leave time at the end for questions for both of us. All right. So a little bit of background, just sharing with you some context for where projects like this sit within the library. The digital media lab or DML is a unit within scholarship tools or methods or STM we love our abbreviations and STM or scholarship tools of methods scholarship tools and methods includes a variety of activities or overseas and supports a variety of activities associated with digital asset management and the development of content management of content in our main repository along with other UCSD and UC managed digital asset platforms. STM also engages in work around emerging formats, including reformatting of content and the application of format types to new modes of teaching and research. Within that context, the digital media lab or DML has evolved from its origins that were largely kind of framed around the concept of maker space. We still maintain a physical space if the library were open that provides services such as 3D printing high level digital media workstations. But the DML has evolved into a strong virtual component. It's really grounded in facilitating project based exploration centered in the use of new emerging formats for teaching research. That includes it's not limited to point clouds, 3D virtual reality and the application of AI to enhance services around these formats. The DML is increasingly connected to other services and activities both in the library and across, I'm sorry, both in the STM program across the library. These include geospatial and data services and data visualization, research data, especially the curation of research data involving unique format needs. The DML has a close relationship with our technology and digital experience unit or TDX, providing both support, where they provide support both for immediate technological needs, as well as development of existing and future platforms. In addition to exploration of new formats and their use informs our overall understanding of a development trajectory for the core repository in terms of its capacity and capability. Very briefly, I'll share that our repository is a locally maintained San Vera based dams with a black light based front end on a custom storage layer. Support for the dams draws from expertise across a range of library programs, including STM, metadata services, our technology unit and more. One feature of the repository ecosystem is a strong connection or direct connection rather to our chronopolis digital preservation repository, adding that aspect of digital preservation to the content that we develop and manage. And finally I'll note that we're increasingly developing features and functionality and partnership with other UC campuses, especially UC Santa Barbara through a project we call Project Surfliner. The intention there is to maximize the efficiency of development through shared goals and through pool development resources. So with that, as at least a brief ecosystem context, I'll kick it over to Scott to take it away and do a deep dive into the Temple of El project. Thanks Roger. Yes, I'd like to start by introducing Basil Cartabil, and Basil is a really interesting character and all of this he's an open source software developer or it was an open source software developer with close ties to the creative Commons and Mozilla foundations. He was an advocate for a free and open internet and in Syria in 2012 that was sort of a dangerous thing to be the Arab spring and the the beginnings of the Syrian uprising had just sort of happened and abated, and a lot of the protests and the armed insurrections had been coordinated via social media and the the open internet and it was sort of seen as the enemy of the state. So Basil is imprisoned in 2012 by the Syrian government. And I don't think they actually ever charged him with any crime, but one of the last projects that Basil had worked on before he was imprisoned. We see here a 3D reconstruction of the city of Paul Myra, as it would have been in the third century AD at its height. And Paul Myra had been a very important Silk Road city in Oasis in the desert sort of a gateway between the Roman Empire and the Parthian Empire. He's been, you know, important and prosperous for thousands of years. One of the, you know, is well known for its monumental architectures if you're a merchant and you have a successful caravan. And some of the money on on, you know, a tomb or a temple or or some other giant public works and you can see all the the tombs and temples out on the edges here all done by these wealthy merchants. So this is the part I struggle with this changing this slide. There we go. I want to show you some examples of the existing side of Paul Myra. This is its theater, and you can see the remains of decoration up here is just wildly ornate every single square inch would have had some sort of decoration over it back at its height. You can see this site right here that the monumental arch. Here it is on the Syrian 50 pound bill, along with a character is a no be a here who had been. She had been sort of the Empress of the independent Paul Marine Empire. For a while it threatened the Roman Empire, it was such a wealthy center of power. And you can see so the association with the regime is this, you know, points to Syria as once being very prosperous and very powerful on the world stage as part of the Roman Empire. And then here is the Temple of Bell, which we've reconstructed for this project. And this is an important religious building for thousands of years. It kept changing and adapting it had been in a Syrian temple to the God ball, and it kept taking on different flavors it became sort of a Greek pagan temple a Roman pagan temple, a Greek Orthodox church and eventually a mosque to all the while, you know, serving different communities and, you know, taking on all these different flavors but still being central to the community. I want to show. Palmyra has this amazing tradition of these funerary bus, and there's still about 3000 of these extant, and these are carvings of real individuals with their, their names and their familial connections and scribe them next to their faces so you know as as ancient sites it's a very personal one where you see individual people still sort of dotting this, this, you know, ancient base desert landscape. Basil is in prison in March 2012 he's kept in solitary for a while. Eventually he's moved to a more open prison camp where he's allowed to contact the outside world. There, you know he hasn't been charged there aren't a lot of options to defend himself locally within within this Syrian legal system or whatever. But he has all these, you know, ties to international open information organizations he's, you know, well known and important, and his friends internationally begin this sort of social media campaign which is very successful. They end up going, you know, posing his case to the European Parliament and the United Nations, both of which, I think I've got the quote here. Yeah they, they call his imprisonment arbitrary and call for his immediate release. He still hasn't been, you know, formally charged with anything in 2015 he's been in prison for over three years now. He's transferred to a new location he goes silent and and people start to fear the worst. So, so the media campaign ramps up they're trying to boost his profile. MIT Media Lab offers him a research position, anything that they can do to, you know, to try to dissuade the Syrian government from from executing what they potentially do, which, you know isn't actually confirmed until 2017. So there's, you know, this whole time from September where, where nobody quite knows what's going on. So at the same time, ISIS is rolling into Palmyra. This is a picture of the the Temple of Bell before and after. This is a picture of, you know what they did to the museum. You know, they're, they're going around smashing faces this. It's a symbol of the current regime that are using at the point to, you know, Syrian power, but it's also a symbol of other religions and paganism and iconography and everything so it's a real important target for ISIS. And so, through this, the same people that had run the free basil social media campaign. This huge group internationally with all of these high level organizations they sort of, you know, they feel that they've, they failed, but they want to keep this momentum going and they funnel their energies and their support into creating a new Palmyra project and new Palmyra ends up getting about 3000 high resolution images of Palmyra over the course of a decade and a half made to be public domain. And these are very, this is a very important site because, you know, not only are they super high resolution. They maintain their exit data, which is really important for 3D reconstruction. So I need to emphasize this is a real feat in that there's thousands of high resolution images all in one place they're made totally free public domain, the exit data is intact there is, you know, like no better sandbox out there for 3D reconstruction. And for some perspective on that, you know, there are a bunch of gaps in our 3D model. And if we were to go about buying, you know, commercially available photographs to fill in these gaps these these places in the model that aren't showing any of the photographs that we have. A single photograph can can cost $500 to license by a traditional media standards. So if, if you gave me a grant for, you know, half a million dollars, I would still run short of acquiring a full collection of the commercially available media that I could apply to this project and make it more complete. So I want to mention Reuters is actually starting to license their media in bulk for machine learning data sets. So hopefully that sort of license will eventually transfer over if we're only using the data to try to, you know, extract points and measurements and create a larger whole. I'm very interested to see where that's going. So I want to talk about the actual reconstruction process using structure from motion. And this is this. Here we go. Okay. It's this process by which individual photos have clusters of high contrast tagged. And then those clusters are compared to each other. So you're able to sort of triangulate their position in 3D as one cluster moves over here relative to another cluster. So you can see what's going on here at each one of these pyramids is is like pointing up to the angle, the individual photo involved. And here you can see one of the individual photos one of the enemies of this process is things that change. So you see I've had to manually mask out things like the tourists themselves the sky, the gravel vegetation anything that can change over time. I want to show to. So through this process I placed about 90 points manually here going through all the photos and and saying, this is that this is that. So I did 90 of those but this algorithm is choosing about 5000 of these per image. And honestly, you know we'd started building public services around photogrammetry in the digital media lab. You know people would bring in an object and we use it to 3D scan and 3D print it. And it was a big surprise that it worked as well as it did in this use case we have enough trouble scanning a student space or an apple or something like that. You know we got anything out of this was just incredible. And then we just kept refining it and got something better and better each time. So, Okay. So we're left with this model and it's a challenging data set. You know the numbers have lost all meaning at this point is 1.2 billion points using over 800 different photos for different parts of the model. Altogether is about 80 gigabytes of data. And it's a big challenge to figure out how to share that data set. And how do we how do we place this within the archive to how do we connect the model back to the images and you know allow the user to explore these things and in a way that's more or less continuous. And we stumbled upon. There we go. This viewer right here the poetry point cloud viewer. You can see, this is the entire full resolution being rendered in browser, anybody can pull this up in in Chrome or Firefox and just explore it manually I'm manipulating it manually right now. I can click around go to these different regions of interest. And you see it's filling in detail, as we go based on my cameras position. So I'm on zoom over Wi-Fi, and it's still doing a pretty good job here. You can see this, you know, this faint painting from the fourth century on the wall, all these crazy stone carvings. I'll go around a little bit here. I want to show you can zoom into read individual inscriptions. And it's really this incredible new ability to, you know, scale our view in a streamlined way between details which are sub millimeter and details which are, you know, many tens of meters long. So, yeah, this this means a lot of different things this viewer is, you know, built on existing JavaScript and HTML libraries is totally open. This resolution agnostic is the simplicity of its implementation with HTML and JavaScript means that we can customize it like crazy to we can, you know, we've got it linked back to our digital collections here. I forgot to show all the other features, we can do some basic analysis in here, we can, you know, do do measurements and annotations as we please. And the clipping boxes to, you know, separate out particular regions here. And here, you can you can walk around this thing in VR. I want to show, you know, the custom interactions that we can do I have this incredibly high resolution 1.2 billion point model, and I can just click another button to pull up another high resolution model, which is the same site. This is the 2016 after six exploded, and we're able to make this one from some commercially available drone footage. So that I mean all these capabilities to to communicate super high resolution data and the ability to sort of link it back to everything else. It shows a lot of promise and the way, you know, not only the way we use the archives, but the way we integrate different data sets with each other. I'd like to turn it back to Roger to take us home on that point. Well, thanks Scott for the deep dive into the project. I will try and be respectful of time and wrap up fairly concisely here. This type of project. As I mentioned earlier, as a kind of a core component of exploring new formats within the digital media lab and scholarship tools and methods. Kind of this work allows us to collaboratively explore areas of need on campus work with partners to ascertain what a sustainable service model looks like between campus and library partners working on these farm these formats these transformative technologies. In addition, the library itself uses these efforts to gain a more well defined sense of direction with respect to our repository infrastructure what our capacity should look like to ingest make available and preserve these new and emerging formats. So endeavors like these exemplified by DML and STM there's always challenges and opportunities. These challenges certainly are reflected in resources in terms of staff and budget sustainability of the service and the capacity to engage this work at scale what that scale might look like, and also alignment with competing priorities. And one way to kind of engage these challenges is to answer and answer questions around sustainability is to work backwards I like to call it from from outcome. So let me ask a series of questions what's the impact of the work to the future of teaching and research on campus. How does this support a varied set of activities across in the library but across disciplines. How does exploration in this space and form and craft. This is kind of a model for flexible impactful and sustainable repository ecosystems. And these are questions we're constantly revisiting as we track new use cases and look at the shifting landscape of resource challenges and opportunities. So in terms of compliance, you know limited resources might indicate competition for priorities but the challenge is to engage in a conversation or series of conversations that builds consensus and articulate articulates both short medium and long term goals that are kind of commonly understood and provide a kind of uniform direction. I mentioned that another goal the library and scholarship tools and methods has when we return to full operation post pandemic is to instantiate a replacement for a workstation we've maintained called a cave kiosk. And the replacement and the concept will take the form of a higher resolution media display intended to showcase projects such as the temple of bell, provide an interactive workspace allowing the promotion and engagement with these types of objects and services. So there's both a physical instantiation and the virtual web based instantiation of a lot of these projects to both spur interest engagement and use. And that's our hope for our exploration in this space. And with that, and just a few minutes left I'd love to entertain any questions you may have for Scott or I Thanks Roger. And thank you Scott. Just a fascinating project. Amazing. We really appreciate your bringing it to see an eye to share with our community. I dropped the URL and the chat there, which I believe is the access point for the general public to explore this tools. So if you see that Scott I just want to make sure that's the right link there. Yep, I even clicked it to make sure. Okay, yeah. Great. So the floor is open for questions and I hope our attendees will share your questions with us. I was just curious, could you just chat a little bit about who some of your partners were on this project to what who was involved how long did it take how many, how many hands in the kettle. This this was largely a passion project for myself. I'd begun working with some some people from Beats College in Maine, and we've been working on constructing the site during your opus and it all sort of snowballed at some point I became involved with the new Walmart project itself. But but really I just became a personal project. It was mostly me just sifting through image databases and, you know, clipping out. What's it called clipping out all that all the tourists. That is quite extraordinary. If I could also jump in Scott just to add to your question to your answer rather into Diane's question. This this project does build on a history of the library's work with our archaeology department and faculty members here, doing similar data on around LiDAR data and around other archaeological sites that have been imaged in one form or another. And yet what Scott has done here on this project is obviously explore a new level of transformation from still imagery to a three dimensional model so it builds on a history and has good connection to archaeology endeavors on campus and in the library that this this served as a use case that we've already applied to a lot of other archaeological projects at UCSD. So it's nice to have the project that we can control and do whatever we want with an applied and whatever way without constraints of, you know, academic partners to publish and have all these other concerns over how their data is used. Sure. Okay, thank you very much. I see that Cliff has a question. Go ahead, Cliff. So I actually have two questions. Some of this reconstruction work around Palmyra is looking really familiar like I believe I saw a museum exhibition on it someplace maybe a year, 18 months ago. Does that ring any bells. Yeah, there there have been. Well, first of all, there are a lot of different sites at Palmyra a lot of different by different means. Yeah, there. There have been some parallel projects that have been going through done by the Ark Foundation. And there's another one. And they're a really interesting group there a bunch of the FX artists who turn their eye on this. And then there's another group out of Italy and Switzerland and by Dr. The song Bob a and Gabriella thank you. And they, they actually had some really nice professional photographs that they were able to use for it. Neither of these have been made totally open along with their source material. This version of it is the most complete. I guess the only one to have such high resolution. You know, and to be so, so open. But yeah, they're the same challenges been tackled in parallel by by a number of different places. And the other thing I was wondering about was, you mentioned the difficulties of getting commercial satellite imagery, for example, to help with these reconstructions. I'm wondering if there's some way to mobilize organizations like UNESCO around that I mean we're talking. I'm pretty sure Palmyra is a World Heritage site. And I would think there are probably several of these that are, you know, in need of this kind of documentation reconstruction. I had been, I'd been looking into a grant through digital globe, who often often does stuff like this. They funded a few different UCSD projects to, I think there was a documentary about finding Genghis cons to where they provided all this imagery and project like that otherwise would have been cost prohibitive. Just the, just the satellite photography that I'd been looking at to get adequate coverage of that region when I looked it was going to be like $12,000 or something for a very small piece of this site. Interesting. Thank you for that. And I want to shut up and let other folks ask questions. Actually, we are, we are past time at this point so I will thank our speakers once again, Roger and Scott thank you so much for sharing this with us here at CNI and to our attendees thank you for making time out of your day to join us. Once again for more of CNI's fall 2020 meeting. And at this point I'm going to turn off the recording and just invite any attendees who'd like to stick around and join the conversation, ask our speakers a question. Please raise your hand and I'll be happy to turn on your microphone. And with that, thank you everyone and have a great rest of your day. Bye bye. Thank you.