 At this time, it is my pleasure to turn it over to your host, Martha Curilidu, ma'am. The floor is yours. Thank you. Thank you very much. I'm Martha Curilidu, and it is my pleasure to welcome you to the last of our six webcasts on the LibValue IMLS grant. Today's webcast is on the Digitize Special Collections, and I do want to thank you for joining us and cover with you a couple of logistical things. Everyone will be muted to cut down on background noise. We do welcome questions. Please type your questions. There's a Q&A button there, and we stand ready to answer all of them and questions and answers that we do not answer. As well as those we address during the webcast, we'll be distributed to attendees after the webcast along with the recording that will be available on the ARL YouTube channel. I would like to take a minute to introduce the colleagues that are with me here today and that will be featured in today's webcast. We have the pleasure of having with us today Gail Baker, Professor and Electronic Resources Coordinator at the University of Tennessee Libraries and Ken Wise, Associate Professor at the University of Tennessee Libraries. Actually, Ken will speak first and then Gail. What we are going to try to cover today is the following goals. We want to identify how we can articulate and measure the value of digitized special collections. We want to demonstrate how contingent valuation can be used for this purpose and also identify how Google Analytics can supplement value studies for digitized special collections. We did mention that this is the last IMLS grant. And a word about the grant. It is an effort by Carol Tenopier at the University of Tennessee and Paula Kaufman at the University of Illinois at Urbana-Champaign. They have pulled together an amazing team of research. A project involves multiple institutions using multiple methods to measure multiple types of values for different stakeholders. And our partners include researchers from the University of Syracuse. We've included in this work some of the GISC Tenopier is working with and none of this would have been possible without the support of IMLS. Now, while the library operates, we also realize that there are at least three key distinct in the library's role in serving the University. One related to teaching and learning, one related to research, and a third one related to social and professional. As the project, we have engaged people of these three elements. Developing tools, the community. We have branded the stats called Gateway as a platform where we collect a lot of data. We have a data collection that's going back more than a century, even before ARL got established. The ARL statistics, this is the annual descriptive statistics. And they started back in 1908 when Gerald was a university librarian at the University of Minnesota. And these statistics served a valuable purpose for a good part of the previous importance is being displaced. And over the last, they're engaged in exploring different protocols. And some of these protocols actually have become sustained operations that libraries use extensively. LiveQual, a tool for measuring library service quality, for example, has been used by more than 1,300 libraries nowadays on a global basis. A climate call and internal staffing service by many libraries to identify the organizational culture. And we have two protocols, the DigiQual protocol was a research project by the National Science Foundation to explore aspects of electronic of digital library service quality. And the Minds for Libraries protocol, which is operational currently and we are engaging with a number of libraries implementing it, focuses on measuring the impact of networked electronic services. And both DigiQual and Minds relate to some respect to the digitized special collections because both of those protocols are developed for the digital environment. Many libraries in the field and many colleagues have articulated of the increasing importance of special collections. I want to highlight here, recently came out as the inaugural piece of a NIFECA SNR briefing series. It is written by Rick Anderson at the University of Utah and it's entitled, the title of it is, Can't Buy a Slap, the Declining Importance of Library Books and the Rising Importance of Special Collections. Now in there, it's that the distinction that will shape our future is the distinction between commodity documents and non-commodity documents. Commodity documents are the everyday books that everybody can buy from the bookstores from Amazon. Commodity documents will bear heavily on the utility and health of an academic research library. And acquiring and digitizing and making discoverable special collections enriches the scholarly environment. A lot of what Rick highlights in this interesting briefing piece has also been articulated through the ARL profiles that we collected a few years ago and we analyzed. And right here you see on this slide the cover page of the report, ARL Profiles Research Fibers 2010. In there, the narratives each library provided to us showed that in the current century they do serve the public good. They have a global role. They do work on setting standards and exploring best practices and they have a key role in establishing non-visibility. Now through those narratives, we did do a further analysis primarily on the digital developments. Libraries highlighted in those narratives and we presented that at a conference a couple of years ago. And the three elements that were identified in those narratives was the acquisition of digital content, the development of digital services, but as you can see there, the top circle there is about the digitizing. Now in those narratives we have seen that the focus on digitizing special collections is on the distinctive materials and the signature collections. And libraries mention a variety of media, photographs, newspapers, film, audio, manuscripts, postcards, multimedia. And there are digitization models that vary from the large-scale digitization that has resulted in things like the happy-teller efforts like curated online exhibits. Now we also, while libraries are digitizing special collections outlined here, they support undergraduate learning, which is something, I think a few years it wasn't something, is to provide remote access to collections, to further the library's mission, to enable long-term preservation, to respond to user preferences, increase the discovery and use of these resources, and of course attract new donations. We go into the piece of the presentation. We do want to, and I'm going to go here and in your screen. And while you see it in your screen and you vote on it, I'm going to read the value you think is created by your digitized special collections. And these of course are not mutually exclusive, but please try to choose one that you think is the most important. So these can and can identify in this poll question. The number one is the user value, the value to the user in terms of time and money spent. Number two is the element of prestige, from high visibility digital special collections. Number three, value that accrues to the development effort of the institution. Number four, environmental value, value of the environmental savings from limited physical access to unique and fragile material. And number five is the value to school from the role of special collections in attracting, for example, graduate students and retaining faculty, of course. And number six of digital collection in attracting additional. I want to see whether you are voting. Yes, you are voting in some of these, so I can show you the results and I'm pushing here the results that the majority of you, the most important, 40% of you voted as the most important value to the user in terms of time and money spent. Then the second, most frequently voted category was Prista, the third one was scholars. And without further ado, I would like to move us on and ask Ken to join us and talk about some of these. What we're going to talk about today is a method for measuring what we might call the value of the return on investment in online digital collections. I'd like to begin by pointing out that with the rapid pace of digitization, where all but the most obscure items will be universally available to all of us, anywhere, anytime, some observers of the academic scene have suggested that the academic research libraries increasingly become an anachronism, kept alive by nostalgic sentiments and for the prominence of big buildings in the middle of campus. This sense of increasing irrelevant has generated a demand for libraries to show proof of worth. And research by Stephen Town appears to demonstrate that academic libraries measure of its contribution to the academic community has demonstrated that there are only two bottom line measures of worth. One is financial or related measures of value. And the second one is impact on research and ultimately on research reputation and to a somewhat lesser extent on teaching and learning. Now these dimensions reflect two lines which have been developing recently in library assessment. One is the quest for impact and the second for value measures. Now in 2009, as Martin pointed out, consortium of the University of Tennessee, University of Illinois and Syracuse University received a grant from IMLS to demonstrate possible impact and value measures by investigating return on investment in academic libraries. A live ball, as this project is known, entails seven separate research projects assessing the return on investment of various functions within the academic enterprise. Now three years prior to starting the lab file project, John Lombardi, while he was addressing the library assessment conference, raised the following question. If everything is digitized, then perhaps the relevant measure of distinction is whether we in our university library capture, maintain, and contribute digitized copies of unique materials. Now the operative word here is the relevant measure of distinction. As a response to Lombardi's question, one of the key components of the live ball project became that of identifying what John Lombardi called the relevant measure of distinction. In particular, we are attempting to assess the beneficial returns on resources invested in the digitization and online access of special collections and here at the University of Tennessee Libraries. In our project, the basic data for estimating the return on investment valuation was gathered through an economic methodology called contingent valuation. These contingent valuation findings were then collaborated with data that was guarded from Google Analytics. In 2012, Carol Kennedy published an article of beyond usage measuring library outcomes and values. Now in and of itself, the title beyond usage suggests that until recently much of the discussion of the benefits of library collections and services had been restricted to use value. In our search, we were attempting to look beyond use value to a value measure that economists called option value. I'll get into that in just a minute. Use value correctly measured, flip to the next slide if you will. Use value correctly measured indicates the value of collections to patrons who actually make use of their collections and services. Generally, it is a measure of an individual's maximum willingness to pay for access to these collections. However, this measurement alone significantly underestimates the actual value paid for the place on these collections and services, and that's the distinction we're trying to make here. Use measurements ignore the potential, the value to potential users who value the option of accessing library collections in the event that they require access. Such users place a value, which economists call the option price on the right to access these collections in the event that you see needs them. I stated more simply, though a patron may not use a collection for many years, he or she may still place significant value on the right to access these collections. To look at it even in a more simplified manner, many of you may have a toolbox at home when you've got a collection of tools in it, and there are probably some tools in that toolbox that you never use, but you still value those tools in the sense that they're there if and when you need to use them, and it's something similar to that we're getting at. David Hartness and Frank Allen were the first two to demonstrate that significant value resides in reference services. They made a reference department. Even for patrons who may not have used the services, and furthermore, that this value, the option value, is measurable. The idea of option value was initially introduced by an economist named Boaton Weisbord, and he used it in reference to economic amenities that can't be valued through the marketplace. Weisbord points out that the option value concept arises when an individual is uncertain about whether he or she would ever make use of a specific amenity. In the case of a library's collection, a special collection, the patron may be uncertain about when or how often they might access the collection. When this uncertainty exists, the appropriate measure of the total value of special collections is the value of the individual's maximum willingness to pay for access before the uncertainty about use is resolved. This value under uncertainty is called the option price. All right, a use value in the traditional sense measures the maximum willingness to pay for use of the collection by individuals after the uncertainty is resolved. That is, they know they want to use it. Option value of any collection is calculated as the difference between the option price and the use value, a measure that estimates the value to non-uses. That is, people who do not use the collection. The fact that some individuals have a positive option price but never exercise it is totally irrelevant. In other words, even for those who rarely or have ever accessed the digital special collections, the collection may nonetheless be intrinsically valuable to them. Now, it's this option price that is the appropriate measure of the benefit to users whether they are potential or whether they are active. So, how do we measure the option value? There are no markets for the buying and selling of online digital collections in which we can directly determine the value in terms of a fail purchase price. That's usually how we determine value is the arms length transaction between a buyer and a seller. There is no market in special collection. A special collection, like any other library service, may be considered a public or a quasi-private good for which there are collective property rights. For example, the citizens of the United States own Grand Canyon and Yosemite National Park, and thus they have a right to hike and to camp in its wilderness. Unlike pure private goods where the cost of purchase explicitly indicates to monetary benefit of the good to the purchaser, public and quasi-private goods like Grand Canyon and Yosemite are not extinct in the marketplace, and thus the benefit to an individual, consumer, user, or patron is not readily revealed. Similarly, a library-special collection is a quasi-private good in the sense that members of the university community own the right to research a collection, but the property rights cannot be sold. Now, in some circumstances, the value of quasi-private goods not explicitly traded may be directly enfold from market transactions. What I'm trying to get into here now is exactly how the contingent valuation method works. I'll consider, for example, the value individuals place on the privilege of, let's say, living in a safer neighborhood. Since there are no explicit markets in safer neighborhoods, there is no means for directly determining an individual's willingness to pay for living in a safer neighborhood. Nevertheless, willingness to pay can be indirectly enfold by examining the sell prices of all houses and extracting the premium that people are willing to pay for those houses that are in areas with safer neighborhoods versus those in less safe neighborhoods. And looking at the distinction, the difference between the two prices. In many instances, however, it may be difficult to infer willingness to pay values from transactions in explicit markets, even in the case of inferring it. Under these circumstances, we have determined that the contingent valuation method affords a useful means for estimating or determining the needed values. A contingent valuation method incorporates carefully designed surveys to elicit willingness to pay for improvements or to avoid degradation in public or quasi-private goods. Because the theoretically ideal method for managing benefits would be based upon individuals' preferences revealed in those direct-iron market transactions. Contingent value methodology establishes a hypothetical market in which the survey participants act as though they are purchasing goods. They pretend they are purchasing goods under consideration, thus revealing his or her valuation of the goods. What we are getting at here is that contingent valuation method is really a hypothetical market that we set up to see how people will respond when given a choice to buy something they really can't purchase. Without getting into the theoretical justification for contingent valuation methods, let it be sufficient here to say that contingent valuation uses survey questionnaires to discuss the individual's preferences for public goods by finding out what they would be willing to pay in dollars amount for suitable alternatives, that is, contingent situations. Now, the LabVal study that Gail Baker and I completed was done here at the University of Tennessee Libraries and were a state-supported institution in Knoxville, Tennessee. And the University's Hodges Libraries contains the online digital professional collections selected for the evaluation assessment. And we picked four of them. One of them was the William Cox-Cockman photograph collection. And the other one was the Albert Dutzeroff digital photograph collection. We also used the from Pi Beta Phi to Aramot collection and also the Thompson brothers digital photograph collections. And all of these, you can flip one more slide. All of these are part of the University of Tennessee's Great Smoke Mountain Regional Project. And at the bottom there, you can see a link or URL that will take you essentially to all of the digital projects that we have, particularly before I just outlined. Those are the four that we included in the questionnaire or the survey for trying to assess the value. Now, our survey model was modeled after a questionnaire used by Halice and Allen in their study of reference services. We constricted that as a baseline. And looking at the example right here, it's good. You can see that we began the survey with kind of a brief description of the methodology and how we're going to proceed. And then we went into questions concerning the user's familiarity with the specific online collection being measured. Here is the question, and this is just an example. Did you know that the Albert Dutch Roth digital collection is hosted by the University of Tennessee? And we would substitute one of the four, two of the four, whatever was relevant for that particular person into this question so we could have the Droth collection or the Thompson collection or the Pi-Fi collection of those. Once we had asked that question, we proceeded with another set of queries. And one was, first we wanted to ask about the level of satisfaction with the content in these online websites. The level of satisfaction with access and through the frequency of access through the collections. Now jumping back up, thank you, too far ahead. Oh yeah, that's all right, you're in the right place, okay. Jumping ahead a bit here, let me mention that all but two of the survey participants were very satisfied with the content of the collection. The remaining two were satisfied. And with the exception of one dissatisfied responder the response to the quality of online access was fairly even across the very satisfied and satisfied spectrum. The frequency of access range from once a week to twice a year. Now, this information was very helpful but it's all ancillary to the real question. To create the hypothetical market and directly connect responses to willingness to pay we moved to the next page of the questionnaire. And question six where we asked the patients to specify how much they would be willing to pay for online access to the collection if they were not freely available. Question six is the critical point of the survey. This is the question that we were trying to get an answer to. Now in contingent valuation studies some participants initially refused to answer the question or respond if they're maximum willing to pay either zero as a protest response. In this circumstance such respondents were taken to a separate set of questions. Now you can see here if they answered B, C or D it directs them to an X one, a Y one or a Y two to another part of the survey. They were taken to another set of questions and statements in order to persuade them to pay if they had initially refused to answer. Now you can see those individuals who responded with zero or nothing or refused to offer willingness to pay to ask the series of questions to identify the reasons for their responses. Now affirmative response did you say zero or nothing because that is what access to the online question it worked to use it is counted as a willingness of zero. Now in response to the other question did you say zero or nothing because you thought we were asking you to begin paying for access to the collection or did you give this answer because you think the University of Tennessee should be able to provide access or spend too much money on online collection. Pops the interviewer to read a short statement addressing the concerns reexamining the willingness to pay question and as you can see from this page of the questionnaire inviting the respondents to answer the willingness to pay question again. The entire questionnaire is very long and in fact it's too long to show in its entirety here but I hope from the next couple of pages to illustrate the sifting process but with the contingent valuations survey ask patients to make comparisons look at contingencies and ultimately reveal to us their maximum willingness to pay for access to these collections. First if the respondents honestly think that they would be asked to pay for access the questionnaire prompts the survey to X2 which is the screen you see right in front of you. Here the survey explains that the collections are supported by state funding and the state and student tuition but that they are being asked for their willingness to pay worthy access not free in other words it's a hypothetical market. What would you pay if you could not get to it earlier we're not asking to pay for? Similarly if a respondent indicates that they do not know what they're willing to pay the questionnaire prompts the Y1 where as you can see it addresses a set of the most likely objections before being directed elsewhere in the question and so the survey would work through each of the objects and trying to find out where the sticking point is for the participant and work them back into ultimately getting back to something to question 6 or something similar to question 6 their willingness to pay. For any given response a well-constructed contingent valuation survey should continue having the respondent making comparisons making and considering contingencies until arriving at some level of willingness to pay for access to digital collections. As you can understand it's a hypothetical market so we're trying to get these people to compare and contrast and consider contingencies in order to come with some price that they are comfortable with. In the end half of the respondent indicated that users are willing to pay at least $10 per month for access to any of the four online collections identified in the survey. Two respondents valued access to the collection at zero and another refused to cite a figure. Some examples here one responded indicated a willingness to pay as much as $20 per image if they were permitted to copy the images. So to even to request copies of the images. Three respondents indicated they were willing to pay $5 per visit some $25 per year and another $10 to $20 per year. So taking all these figures and all the data and using even the most conservative averaging of the responses users of the four online collections are willing to pay in the neighborhood for access to the collection. Now that sort of gets to the value part. Now we've got to look at the cost. So the estimates of cost of creating and maintaining access to the content of the four online collections referenced in the survey were approximated by measuring the cost of creating a small but similar collection of great smoking mountain photographs known as all Adams photographs collection. It only has 25 images in it. And the 25 images were scanned and correlated with the metadata by student workers in less than two hours and the online access were created by technician in less than one week's works. While these images may be easier to digitize than certain other special collections material the Adams project nevertheless indicates that the cost of optimizing and creating online access is not excessive. Upon release the Paul Adams online collections he flowed through favorably in a blog. This is sort of an accidental thing but it was on a blog on the great Smoky Mountains. And the following day the site registered over 700 hits extrapolating the estimated $5 willingness to pay over the 700 hits over the cost. Now if one assumes the value of an online special collection is limited only to that value turned by actual usage it follows that the total willingness to pay would not be higher for individuals who do not use the collection. The concept of option value however suggests that the willingness to pay value is more meaningful than the actual usage Infrequent or non-frequent or non-users of the online collections know that the access exists and the willingness to pay to ensure that the service is available when the need arises. It can also be surmised that infrequent researchers are likely to be the more unsophisticated users of online access and who need and therefore willingness to pay is greater. Now in our study Gale and I focused our survey on youth value trying to measure actually the non-users but there are other important values of special collection that cannot be measured in the marketplace and thus lend themselves to contingent valuation methodologies. We saw all of these in the poll question. As I pointed out earlier one of the two bottom line measures of work is impact on research or more explicitly research reputation. Reputation or as I call it prestige is in our case might be the question what is the prestige to the institution for high visibility digital special collections. Similarly, there is a measurable value in the worth of a library's special collection to a university's development efforts. What value accrues to the development effort of the institution? Again we complained questions for estimating the value from an environmental perspective to that of attracting scholars to the institution and for building a reputation will help attract each of the collections. In the case of the environment you can see the question might be what is the value to the environmental savings from limited physical access to unique and often fragile materials. For scholars the question might be what value accrues from the role of special collections in attracting graduate students. And from a standpoint of collections the relevant question would be what is the value of digital collections in attracting additional special collections. This latter question invokes what is known as the Matthew effect. It's a sociological victim that predicts how advantages beget further advantage or how the rich get riches or in our case how the prestigious collections become even bigger and more prestigious. If you look back on with these six values that I've listed and remember we're talking in contingent evaluation to measure beyond the user that is trying to measure the non-user. You can see how people who are interested in development who are interested in building collections who are interested in the prestige element may not necessarily be users of the collection. The collection is valued to the development office for example in the sense that they can promote the university and promote the library and raise money even though it's highly unlikely the development office is ever going to use a collection. Similarly people who are going to build collections knowing that they've got a good collection already there a prestigious collection are going to try to go out and acquire more collections but they may not ever be users of the collections there but the current collection has great value to those people. As many of you indicated in your survey, prestige with the number one vote in which of these values is most important. Prestige to a university people who market the prestige and who advertise university like having the collections there for value purposes but again they may not be users. So regardless of which of these values others that are important to the library's operation Gale and I feel that continued valuation represents a promising approach to measuring return on investment over the range of values that do not in the sense that they do not include active users. Now Gale is going to pick up at this point and talk about how we applied this data to Google Analytics and how we went further with that. And just there are a couple of questions if we can get them and can either both you and Gale can respond to these. One coming from the University of Michigan Helen Luke is asking how does your option value allow for people who wish to promote and support the common good? That is the needs of others. How does our collection do it? How the option value is the option value taking into account in any way? That's an interesting question. What the question is trying to do is how could we value the common good and the survey would have to be designed in such a way that you can sort of elicit whatever that how are you going to define the common good and how do you place the value on that? What most likely would have to happen is that the survey would have comparisons in there. For example, you would have to say this is a common good and this is a common good and this is a common good and this is a common good. Which one do you have a greater preference for and get some sense where the hierarchy is and then try to narrow it down? The survey does. It gives people these choices and they start to rank until you find out where the common good ranks within all the other options that they have or all the other contingencies. That's the methodology you would have to use. It would be a tricky one. Yeah. It's a very, very interesting question. Now another one, it may not be as hard but it says I did mention that in the beginning one of the findings we found regarding why libraries make special collections available is the ability to provide remote access to some of these resources. The question is how are you providing that access vis-à-vis copyright protections and intellectual property law? So how are some of these of the copyright protections and intellectual property law relate to some of the special? Okay. In some cases we have all the copyrights to do whatever we want. We can reproduce. We can sell. We have the rights and others the rights are somewhat limited. We cannot to deal with a commercial enterprise. For example, we get hotels and investments and we like this big smoky mount scene to put up on our wall. In those cases we refer to who has given us the collection or who holds the copyright. But we do secure the rights up to where we can give them to nonprofits the scholarly use for publications and so forth. And it's just an agreement that we have to get with each of the donors what they're comfortable with rights they're willing to give up. I think in all cases that we have except maybe one we can distribute them in every case except for commercial use. Very good. Gail, do you want to... You only get it when you get the collection. Yeah. Thank you. Gail, do you want to respond to either of these? Anything more? Well, I think I might have in my area something about mental savings with the online use. I mean it's very similar to the types of savings and values that people look at when they look at online courses and all we're saving in time money as far as travel impacting the environment that sort of thing and I will talk a little bit more about that in a few slides. And we do have a poll question for the audience. Does your institution value your efforts to digitize special collections? I just pushed that question out there and hopefully the answers are coming back. Let me take a quick look. Oh, yes. Everybody is coming back. I'm going to stop the question and I'm going to show the results to our audience and we see that, yes, resounding, yes, 76% indicated that the digitization of special collections is being valued. So, Gail. Okay. I would go to, I guess, a couple of slides beyond where you are now. The slide 41. For image collection data gathering, our plan was to use three different methods. Survey, log analysis, and interviews. Our survey did not generate a very good response rate. So, we moved on to focus on log analysis and we're hoping that the log files which serve as a history of events on a server would give us more information and then we were going to use the log files to help us identify perspective subjects for KEN's contingent value work. Deep log analysis, a lot of data elements, IP addresses, date and time stamps, IP address of the user, which I found very interesting and used. Referring URL, which gave us some idea of where the user got the link or the direction to get to the digital image site. On the next page, on this page that you can see a search referring URL and then you see news readers and metadata. When I started asking for more information about the IP range are technical folks that you need to look at Google Analytics. So, I promptly started reading about it and got some help, some additional programming help just to add the JavaScript that needs to be put in the header on the various web pages used for interacting with digital image collections. When that code was executed, a log record was stored at Google and a cookie was set on the user's browser to be checked later to see if that user returned. When I wanted to see analytical reports, I logged into Google Analytics to the URL that you see there and I did not have to depend on the programmers after that which was helpful because I could interact with the Google Analytics just fine. There are pros and cons for using Google Analytics. One of the best ones is that it's free and I guess for now the reports provide some very useful information about visitors. A number of page views you get. Most of the information is aggregated for certain slices of the user's sessions and also we've got average page views, average time viewing pages in a session, new return visits, bounce rate. Now bounce rate is something I had to learn about. The percentage of visits where only one page is viewed before the visitor exits the site. Google Analytics does have some flexibility. Dates can be changed to cover specific time periods and visitor information can be displayed by day, week, or month. One can also add another metric to do to look at perhaps visitors by country and then add a second dimension with the type of browser they're using or you can exclude records that meet certain conditions like I would want to exclude all visits whose bounce rate is greater than 50% something like that and makes for very interesting reporting going on. If we want to go to the next slide, the cons it's not a weblog. You can't get to individual user visitor information. It only records when JavaScript is encountered if a browser does not allow cookies, you don't know. There could be many people like me who I clear my cache cookies all the time so the visitor category may not be correct. We ended up putting up a privacy policy on our website to state that we used Google analytics to improve our web pages and that cookies are used for Google analytics and directing them to disallow cookies if they don't want their usage tracked. The last thing is Google has the data and given what's been on the news some may wonder who else might have that data too. If we can move on and start looking at the reports. Here's the dashboard report. I'm sorry it's kind of blurry. The colors are pretty but I will tell you that we are looking at the dashboard report for the Roth collection for June of 2012 through the end of May 2013. 3,491 visits by 2,653 people. This is the largest collection with over a thousand images so this has is our highest use collection in the Smoky Mountains project. Let's see. At the lower right of the pie chart you can see that the percentage of returning visitors is about 25%. And that the average bounce rate was about, was 53 .62%. So people stayed in for at least 5 minutes in 13 seconds or at least that's the average visit duration for that. Next slide please. There are some nice graphics for location here. Probably it's not surprising that Tennessee had the most visits and that was 1,422 visits with Kentucky next. With 266 North Carolina 201 Illinois that surprised me. Ohio next with around 150. Georgia, Florida in the low 100s, Indiana with an 8th, Alabama 9th, Michigan 10th and California 11th so that was pretty interesting. If you look at the next slide we've got some international reach more than 3,000 visits were in the U.S. There was, there were 29 IP addresses that did not have a location. They were not in the tables we used. That was the next highest. 17 visits from Canada 16 from Australia 15 from Germany and we had Ecuador as the 6 with 13 I believe. On the next slide I have some information. Oh, I'm sorry that you can't see that. These are top referrals by source and the one the site that was the largest source of visits was togeton.net with 850 visits then there's several from UTK that were sources of visits. One of the sources, the 3rd highest source of visits was a website containing geospatial data of points of interest in East Tennessee including the Great Smoky Mountains. Many other sites were associated with hiking or the Great Smoky Mountains or history. There's one site ENTS. BBS.org that's a bulletin board site for the Native Tree Society which is kind of interesting and then there was a real estate firm doing business in and around the Smokies that's listed on this site. So that's an interesting report to look at and slice and undicing with extra dimensions a little later. If you look at the next slide right here we've got top search terms. There wasn't much searching during the time we're looking at now but the search term with the highest number of pages per visit was for Dutch Roth about the 4th line down there were 3 visits but the pages per visit were over 100 and the average duration was 25 minutes. On to the next slide this was generated by getting the data by content page listed in order by page number of page views. I then added a second dimension the source. Where did the request to get onto this page come from? The first line the page is a digital image and the source is a site togeton.net. Let's see what this image is that has so many different page views. If you go to the next slide this is an image of Cumberland Falls in Kentucky. It's about 88 miles north of Knoxville and not really related to the Smoky Mountains but was in this collection and togeton.net hosted a website with something called a moonbow and I if you're interested send me an email and I can let you know what a moonbow is. Let's go on to the next slide Martha please. Usage metrics and user satisfaction. We have visit information listed by metropolitan area and again excuse me for not being able to see the detail information on this slide. In the column on the far right what I did was found the miles from Knoxville. So the first right with no miles is Knoxville, Tennessee then Nashville, Chattanooga, the Tri-Cities, Virginia. Nashville is 180 miles from Knoxville and a round trip would be 360 miles. That could be a numeric representation of energy saved based on mileage for not having to drive to Knoxville for each visit. So that's an environmental values it could be looked at. If you look at the next page we've added additional data or additional data with returning users. We've selected our data to only look at returning users and with page reviews greater than four. Again listed by metropolitan area and we have computed or found the mileage from Knoxville and we have computed a round-trip cost for a trip to Knoxville by doubling the mileage and then multiplying it by 55 and a half cents per mile that the government allows for mileage deduction. For example, Chicago, Illinois, there were 23 visits. That's $300.51 in miles per visit there. So one if you needed to have a dollar value associated with these visits this might be something useful. If you go to my last slide Martha I'm not going to read the fall vision value creation but the University of Tennessee Knoxville has a goal of being in the list of top 25 public universities like UCLA University of Michigan University of North Carolina Chapel Hill, etc. And this is an excerpt of the vision about outreach activities and I believe we've shown that the Roth Image Collection has benefited several communities and contributes to this vision and providing free online access to these images is definitely in the realm of the public good. And thank you Martha. Thank you. We have a couple more questions. I don't know we are a little bit over time but it's okay. We'll probably go about 5 more minutes. Let's see Jeremy Bueller from the University of British Columbia he's asking who was invited at the minister. The survey was a pop-up survey came up when one entered the main entry page to the collection and it was a pop-up survey and if you're like me I don't look at pop-ups I disable them so we don't know how many people went right through it. We do know we had several visits several visits from Google Analytics through that home home page and we got maybe 30 people who said go ahead and I'll look at the survey when we said do you want to continue once they got into the survey about half of them dropped off. We did talk about doing an intercept survey as I remember Gay Linda it was technical infrastructure that I did not have available to me. Yeah, but it is possible to do it as an intercept survey. We do have another question coming back to the option value that's actually interesting to consider. The option value seems similar to just in case it says. A justification library is using it different. It probably isn't any different it's probably the same thing but for those who are the non-users it's like the tool box example you have that tool in your box just in case you ever want to use it and from an economics standpoint it's precisely the same thing as a just in case. What we're attempting to do of course is to be able to develop this contingent valuation where we can reach out and try to capture things like the prestige and like the math effects on building a collection on how it helps a development officer bring in money. So we started our survey with what we thought might have been the lower hanging fruit is the non-user so we just identified the non-user as the person who wants it there for whatever reason and just in case. So I think your question is a good one in the sense that the just in case is really the same thing as the option value. Thank you. Thank you both of you. I do want to breeze through a couple of elements on the ARL agenda that Special Collections is a priority for ARL. We have a working group that looks into these issues a working group that tries to identify opportunities and recommend actions that will encourage concerted action and coordinated planning for collecting and exposing old materials that group also identifies criteria and strategies for collecting digital and other new media material. The focused on born digital special collections and there is a spec kit we've done in this area in 2012 managing born digital special collections and we have a series of workshops that are offered in collaboration with the Society of American Archivists that are happening throughout the rest of 2013 and 2014. In relation to digital in 2010 the ARL board endorsed a set of nine principles to guide vendors and publisher relations in large scale digitization projects of special collections materials and in 2012 we published a third of articles in our newsletter the research library issues newsletter on legal issues in digitizing special collections. There are collaborative research projects with Ithaca SNR on sustainability of digitized special collections and we have all done a couple of evidence based approaches that did collect qualitative profiles as part of the celebrating research volume that we put together a few years ago celebrating the 75th anniversary of ARL and starting last year we started collecting some basic data quantitative data on expenditures and stuff. There seems to be a lot of interest for doing more systematic work in this area. If you are interested in pilot work please let me know we will be putting a toolkit together and soliciting libraries to participate in different pilot activities come next year. I would like to thank you for staying with us. Have a great day. Thank you Martha, appreciate it. Thank you. Thank you Martha. Thank you for your attention. You may disconnect your lines this time and have a great day.