 Good day, ladies and gentlemen. Welcome to today's Live Value Undergraduate Student Success Webcast. At this time, all lines have been placed on a listen-only mode and the full review for your questions is comments found in the presentation. If you'd like to ask a question during a webinar, you may do so by clicking on the Ask a Question tab. Simply type your question and then hit submit. At this time, it is my pleasure to turn it over to Martha Kitty Lidoo. Ma'am, the floor is yours. Happy Valentine's Day, everybody. I'm Martha Kitty Lidoo and it is my pleasure to welcome you to the first Live Value Workshop on Undergraduate Student Success. We have six Webcasts planned this year featuring the work accomplished through this IMLS grant, the Live Value Grant, which is led by Carol Tenopir at the University of Tennessee and Paula Kaufman at the University of Illinois at Urbana-Champaign, our two copy eyes on this work. Now, a couple of logistics. Everyone will be muted to cut down on background noise. We welcome questions. Please type your questions and ARL staff stands ready to answer all of them. Questions and answers that we do not address during the Webcast as well as the ones we address during the Webcast will be distributed to attendees after the Webcast along with the recording. I would like to take a minute to introduce the colleagues that are with me in today's Webcast. We have here with us today Regina Mays, Assistant Professor and Assessment Librarian at the University of Tennessee Libraries. Rachel Fleming May, Assistant Professor at the School of Information Sciences at the University of Tennessee and Teresa Walker, Head of Integrated User Services at the University of Tennessee Library. Now our goal for today is to help you become familiar a little bit with the Live Value Project to consider the library's role in undergraduate student success, to look at a methodology for assessing the library's value in teaching and learning and examine a methodology for assessing the library's value in providing learning spaces and then discuss related assessments to student success. Now we do have a poll question for you and I'm going to pose the question here and please take your clickers and start clicking. The question reads, our campus administrators want the following evidence and you have four choices. Traditional input-output measures such as items added and circulated is the first choice. User satisfaction measures from local surveys and oral live quality standardized instruments is the second choice. Impact measures that demonstrate what the library enables users to do. That's your third choice and the fourth choice is all of the above. So we're going to take a minute to see your answers coming in and I hope you are all clicking and answer to this. So I'm going to stop the question here and we have 44 answers that have come in and you can see the results here. As expected, the majority of you answered that all of the above are evidence data that are important for our administrators. And in particular with the LibValue work, we are focusing on the impact we are having on our users. So we will, as part of our agenda, we'll do an overview of LibValue about assessing the library's impact on undergraduate instruction and student learning. We'll talk about assessing the impact of new academic uses of library space such as the learning commons on undergraduate student success and about sharing additional assessment methods. And we will look at your questions and share them with the speakers as we go along. So what about LibValue? LibValue is a three-year IMLS grant that tests new methodologies in multiple institutions using multiple methods to measure multiple values for multiple stakeholders. University of Tennessee and University of Illinois-Dubana-Champaign are the key institutions that are leading this effort and ARL is supporting this effort. We also have researchers from the University of Syracuse and we are bringing into this project the experience that Carol Tannapier has built through collaborations in the UK environment with the GISC collections. And of course, our work would not have been possible without the grant funds from the IMLS. The LibValue work takes a specific view on what return on investment is and it's worth pointing out how that view is a bit broad. In a strict sense, return on investment is just a quantitative measure expressed as a ratio of the value returned to the institution for each monetary unit invested in the library. However, as part of the LibValue work, we see that value is a broader concept. It is return on investment covers values of all types that come to stakeholders and to the institution on the library's collections, services and contribution to its communities. And as part of that work, we have focused in a variety of stakeholders, undergraduates is what we will focus today and actually our next March work webcast will also focus even more deeply on the common spaces value but we're looking into the value of books and e-books in April and in May we will look into comprehensive approaches to defining library value for the whole library system. In June we will look more specifically on the value in the teaching and research areas and our last August webcast will be on the value of digitized special collections and all of that has happened through the IMLS grant. At IMLS work supplements the work we have done over the years in developing the status quo suite of services, a gateway for library assessment tools that describe the role, character and impact of physical and digital libraries. The historical input-output measures as reflected in the ARL Statistics Survey are part of that picture. Standardized user survey measures like live call is another part of that picture. Climate call focusing on internal climate and diversity issues and staff perceptions is another tool. DigiQual and Minds for Libraries are works specifically focused on digital library service quality and effectiveness. The lead value work is a complement to the status quo work. Now my colleagues at the University of Tennessee starting with Regina will tell us a little bit more about the lead value work in relation to undergraduate student success. Regina. Thank you Martha. Hello everyone, this is Regina. I think to start out with we have a poll that we're going to ask you to respond to and that is do you formally assess student learning outcomes of library instruction at your library? And you can answer almost always, often, sometimes rarely or almost never. So we'll take just a few moments. Yeah we'll give them a minute. We're going to stop the question and show the results. We got a bunch of answers coming in. 22% said almost always Regina, 18% often. 29% sometimes, 15% rarely, almost never, 13%. Okay, go ahead. Okay, so that's probably not uncommon. I think we're all in somewhat different places in terms of assessing the outcomes of our library instruction. We've done it in different ways over the years. So today I want to share with you a little bit about our efforts here at the University of Tennessee to demonstrate the value of library instruction to undergraduate student success. And perhaps the most important guiding principle of the lead value project is to start by aligning the mission and goals of the library with the mission and goals of the greater institution that your library serves and to focus your assessment efforts accordingly when attempting to demonstrate your value to that institution. So although there are some things that will be somewhat universal every institution is unique and will focus more or less on different measures at different times. So to give you some background about our environment here at UT and what we're focusing on and why, about three years ago the then governor of Tennessee challenged our university administration to become a top 25 university. Now currently we hover around the mid to low 40s in that ranking. So this is a pretty ambitious goal, but our university administration has taken it very seriously and committed to it. And as a first step in what they are calling our top 25 journey, they've identified 12 metrics to focus on improving including first to second year student retention rates and six year graduation rates. And those are the two metrics that are particularly relevant to us in demonstrating our contribution to undergraduate student success. Now also currently shaping our environment in Tennessee, we have the Complete College Tennessee Act that has changed our funding model to an outcome-based funding model. Our accrediting body FACTS is now requiring that student learning outcomes be defined and assessed for all programs. And like many of you folks out there, we also have faced some rather sweeping budget cuts a few years back that we still haven't really recovered from yet. And in particular these budget cuts resulted in some pretty big cuts, some pretty big losses in our staff positions. So these are all factors that we're mindful of in our efforts to demonstrate our value and that we wanted to kind of touch on when we thought about how can we demonstrate the library's value to the university. And also in our minds as we designed this experimental study on library instruction was previous research that showed that the typical one-off variety of library instruction that most libraries tend to do where you have about 50 minutes or so with students once a semester or maybe once a year. That type of instruction has been shown to be somewhat limited in what you can realistically expect students to retain with such a short time with them. So we conceived a series of three workshops to test whether more time with students would make a difference. And I'm going to tell you a little bit more about the specifics of each workshop in a moment, but in addition to teaching the information literacy and research skills that you would normally expect, we also reasoned that even if the retention of those skills might be somewhat limited, what we did feel we could accomplish even in such a short time with students was to make students aware of the help and services available to them and to address any library anxiety that students might feel and acclimate them to the library environment. So we had in mind that we wanted to target students for this study who were in some way at risk of not being retained. And on the principle that this was both a way that we could make a real contribution to student retention and that it also might be easier to measure that contribution with at-risk students. And we also had in mind that we wanted, if we could at some point, to target students in the STEM field since attracting and retaining students in those fields is a priority for our university. And originally, when we started thinking about these studies and we thought about who we wanted to work with, we approached some instructions in our English 104 courses here and that's a course that's required for students who face some challenges in writing and we'd worked with some of these instructors before at the library and so we worked with them to tailor some workshops for those students. And so this is kind of a pilot effort for us and much like Edison and the light bulb, we learned some things that didn't work and so the major problem with this effort was that students attended the workshops on a voluntary basis and only about half the students who started the workshops ended up attending all three so that was kind of a problem there. But then we were fortunate to find two programs to work with that seemed kind of tailor made for our purposes so I guess maybe what they say in the secret is right if you set your intentions things will happen for you but regardless we did end up looking out with these and the first of the programs that we ended up working with is the UT Pellissippi Bridge program and this is the program I'm going to actually share some of our results with you today from. This is a program that's targeted at students who don't quite qualify for admittance to UT but who are very close and show great potential and it's an invitation-only program where the students attend classes at Pellissippi State Community College which is a local community college here for their first two years but they live on campus at UT and attend classes here in the summers and participate in programs to acclimate them to UT and just kind of help them with their academic as well as social success with the goal of their becoming full-time UT students in their junior year. So this is a perfect group of students for us to work with for many reasons and even though I'm not going to actually talk about our work with the other program mentioned here the peer program I do just want to mention to you that we also did the program with these group of students. This is a program that's mentors underrepresented students, graduate students in the sciences and it was just interesting for us to see how we could actually adapt this three workshop model to such disparate groups of students. So I do want to come clean from the beginning that I am not an instruction librarian so I'll just admit that but I was very fortunate to work on the design of the study with some folks who are very experienced and skilled in instruction including Rachel Fleming-May and Theresa Walker who are with us today but I also want to acknowledge someone who's not here and that's Rachel Radham who is an instruction librarian here and she really taught probably the majority of these workshops as well as participating in the design and she really did an excellent job so we're lucky to have her. So about the workshops so we did have these designed in three modules to address these three broad objectives of the orientation to the physical library that was in the first workshop. We wanted to provide an introduction for the students to library staff and to instruct them in identifying and accessing physical information sources such as finding a book and a stacks and here at Hodges Library that's pretty challenging. We have a kind of crazy library that's not laid out in the most intuitive way and also a major goal with this again was to acclimate students to the library environment and to the university environment and to make them feel at home and to feel that they can approach library staff and our librarians and ask for help. So we also were mindful throughout all of these workshops that we wanted to start breaking down library anxiety. So the second workshop was designed to introduce the students to the research community and what it means to be a part of a research university so we were dealing with some basic information literacy goals understanding the difference between scholarly and popular sources and the difference between primary and secondary sources and also understanding how and when and why to cite sources and recognizing and avoiding journalism. And then in the third workshop we focused on the development of some research skills so navigating databases search strategies using Google Scholar and again as with all the workshops we really did try to make this very interactive and to stress the approachability of our library staff. So we did pre and post assessments in each of the three workshops and the final post assessment in the third workshop was a cumulative assessment so it assessed items from all three workshops so we could look at the retention of the information at least over the three weeks of these workshops and I should mention that we did them in three consecutive weeks. So we did assess the students for learning and retention of all the skills that we covered in the workshops so understanding call numbers, plagiarism, citing sources, etc. But in the interest of time today because I don't want to hog all our time I've pulled out just a few results and so this first set of results here you can see this was from our preliminary assessments in the first workshop and one thing that we wanted to do was assess not just their skills but to understand students attitudes toward and their previous experiences with libraries and doing research and particularly how they approached doing research and this was to help inform us and help us to kind of tailor some of what we were doing as well so a little bit of it was just kind of a learning experience for us or gathering some data to see what happened. So as you can see from the first two items here the students did report they seem to have better experiences with their public libraries and with their school libraries that was kind of interesting and also as you might expect about 77% feel they're successful or you know pretty successful at locating information on Google and none of the students feel that they are never successful with Google so we know from other studies that people do tend to overestimate their search skills and kind of don't know what they don't know so we kind of would expect them to rate themselves as being pretty successful with Google but it was also interesting to see that about 86% of the students did report needing materials from sources other than Google at least sometimes and then the last item you can see about 35% of students reported seldom or never asking a librarian for assistance or other 35% sometimes do so about 70% of students do not regularly ask a librarian for help so we saw some opportunities there. So then when we go to the post assessments we did the post assessment information I'll share with you was from our last assessment after the three workshops and what students reported after participating in these three library workshops how they felt and in the next slide there you can see over 80% of participants report being more likely to ask a librarian for research assistance after participating in these workshops about 85% feel they will save time on their research as a result of the workshops about 90% feel they're better at finding information using sources other than Google and almost 95% feel they have a better understanding of how they apply to site sources and in the comments that we got from the students we did find that the students in this group seem to particularly value the citation instruction so that was good to learn and about 95% report feeling more comfortable in our library which of course was one of our main goals of the workshop because we reasoned that perhaps the most valuable research skill that anyone can have is knowing who and how to ask for help and feeling comfortable doing so so in the comments that we got from students as you can see we've got a few representative samples here we saw a lot of words like comfortable, confident, helpful and in general the responses that we got from both students and instructors with this group tended to be pretty positive so one of the great things about working with the bridge program is that this is a fairly new program that's still fighting for funding and so they have assessments built in they basically assess these students within an inch of their lives really but they were doing a lot of assessments and building those in and so they were happy to let us add in a few library questions to the assessments they were already giving the students so on the next slide there you can see about a month after the workshops after the students had finished the classes in questions and we were able to survey them on the following three questions and pardon me I've got a little confused today I thought I was going to make it three without I but I didn't quite make it I apologize so we did with this one you know this is at least a month later so we could kind of see what their experience had been with a little time to reflect and as you can see here 45 of the students are about 88% report feeling comfortable using the library's web page and databases library sessions about 94% or 48 of them that we had feel comfortable visiting the library and 42 of the students are about 82% report using having used UTK library resources for their class assignments so that was pretty interesting now the study is still a work in progress and we're continuing to analyze our data and we also plan some next steps and this is where I think we'll really get into some pretty interesting stuff we're going to compare the final papers of the first bridge cohort who did not attend the series of three workshops with the work of the students who did and we also plan to survey the first cohort and ask them the same questions that we've already asked the students who did receive the workshops and compare and look for impacts in that way so that is a quick nutshell overview of the work that we've been doing on assessing the value of library instruction to undergraduate student success and now Rachel Fleming May is going to enlighten you on a few other things that we've been doing here but first there will be a poll Hi everybody this is Rachel the poll we're going to kind of switch gears here and look at the other side of the coin that is the role of the library in quality teaching and success in teaching which of course we know contributes to student learning and student success so the poll that we would like to ask you to participate in now is basically inquiring if your library has engaged in any efforts to assess how well collection services and facilities support instructors teaching that is not their research needs but their teaching needs and options are yes no not sure our results are 55 percent saying yes 21 percent saying no and 23 percent not sure okay interesting well just a little bit about me and how I became involved in this and I let's see there I am I arrived at UT in fall of 2009 just right before we found out that the application to IMLS had been successful and I was immediately interested in participating in this project and I asked Carol Teneper if I could get involved it was kind of a natural fit for me to work on the teaching and learning side of things so I am and I was especially interested in looking into the ways in which libraries support academic library support teaching typically when we look at library support for faculty and instructors we're really focusing more in our research on supporting their research supporting their needs for research so I wanted to really kind of look at that other side the way in which we support teaching rather than you know the ways in which librarians help students learn through library instruction so what did we decide to do in order to start to look at this question we decided that we would launch a survey of all faculty staff and graduate students at UT that is basically everyone with instructional responsibilities. We're going to be following up on the survey with real time conversations focus groups probably with graduate teaching assistants and adjunct or contingent instructors we're going to be doing that this spring probably in the next month or so we're going to get started with that the idea being that those are instructors that really probably need the facilities at least of the library than regular full time faculty do who have private offices and everything so we will be following up with that so we conducted our survey we asked our respondents about first of all what they perceive to be the benefits of using the library's resources, services, and facilities to support their teaching. Overwhelmingly they reported that they had significant time savings as a result of using the library to support their teaching. Of course time is money so you can demonstrate a legitimate return on investment there. Some of them also expressed the belief that using the library to support their teaching resulted in improvements to their teaching as well as to their students performance. We decided to focus on what we did in the survey because as Regina already mentioned we have a very specific climate here at the University of Tennessee that's a combination results from a combination of factors both internally and externally. She mentioned the vol vision, our strategic plan, our endeavor to raise the status of UT as a public institution in the next several years. We also looked at external factors that is Regina mentioned the Complete College Tennessee Act which restructured financing to public institutions in the state to reflect not inputs that is you're not rewarded if you enroll a lot of students but outcomes. Rewards financially are based on the number of students that graduate and how quickly they graduate as well as retention. We were also interested in looking at issues at the federal level state and federal level rather. There is in the state of Tennessee our legislature is very involved in concerns regarding affordability of college education for students and one of the ways in which they look at that is trying to control the cost of textbooks for students. A number of states have state level legislation related to textbook cost and there is actually federal legislation as well. So we were interested in exploring the ways in which libraries might be contributing to that effort and I have to credit Megan Oakleaf with first putting that bug in my ear about this particular issue and I know as an instructor myself much of what I require my students to read comes from our subscription databases which I of course have them access legitimately and legally but there is cost savings there to students as well. So I'm going to talk a little bit about the nuts and bolts of distributing the survey at the end of my segment of this presentation and I also want to put in a little plug for the June webcast which I'll be presenting there again with Carol and we'll be talking about this particular survey in a little bit more detail and talking about different issues. But I'm going to really focus on the segments of this survey that relate to student success. I mentioned earlier that we asked instructors about time savings as a result of using library resources and services and facilities the majority of our respondents said that they saved 16 or fewer hours per semester. Some of them did deny that they save any time as a result of using library resources and facilities to support their teaching but most of them did acknowledge some time savings. We also asked about improvements to their teaching itself, their pedagogy and the course materials that they use both in preparing for teaching and in providing to their students and we just need to go forward one more slide. So our respondents reported that they believed that the readings that they assigned to their students have improved over the past several years as a result of using library resources and services they're consulting a broader range of materials to prepare for their teaching and more at their ready to access for preparation and although not a majority of respondents, a significant number of them did say that they believed their assignments were more creative as a result of using the library to support their teaching. We were a little bit surprised to find that about half of our respondents are no less likely to consult print resources in preparation for their teaching than they were three to five years ago. So that's something to maybe bear in mind as so many of our print collections and academic libraries are moved off site in order to accommodate other facilities and services. I'm not sure exactly what the answer is to that but I think it bears further exploration. As far as collecting and distributing readings to their students and this gets into a little bit of that cost of text materials. A fairly significant percentage of them said that they are less likely to require their students to purchase printed textbooks or course packets. I don't know if any of you remember course packets. 44% of our instructors said I didn't require them to purchase course packets to begin with but those are resources that are still assigned as course texts in some courses so we wanted to investigate that. We also asked about the instructor's perception of improvements to their student performance which is a little bit more difficult to get a handle on but we were really looking at their perceptions here. We're half of our respondents said that their students are using sources that are more appropriate for academic work i.e. not using Wikipedia. Merely half said that they're writing citations that are more complete and or correct. Not a majority but I still think 40% is pretty respectable and 66%. I think that this is also a significant number. Reported that their students are accessing information from a wider variety of sources. One of the goals of this project was to create an instrument that could be adapted for distribution at other institutions and other types of institutions and we had the opportunity to explore this just recently when Peter Fritzler and Ann Pemberton from University of North Carolina at Wilmington approached us about implementing this survey at UNCW. We helped them edit the survey to deploy at UNCW. There are some notable differences between UNCW and UTK that I just want to point out right now. While it's still a public institution, it is a regional institution, much smaller enrollment and smaller number of colleges, smaller number of academic programs, specifically graduate academic programs. So we wanted to look at some of the differences here. In some of the areas that we explored the differences were not really significant. In terms of looking at services about as many of our respondents and UT is orange here of course and UNCW is teal. Although I shouldn't say of course many of you I'm sure have not been in Knoxville and aren't aware of the ubiquity of the power orange color around town. But teal is UNCW orange's UTK. So in some areas there was not a whole lot of difference but in other areas the discrepancy between responses at the different institutions was much more dramatic. For example UNCW instructors were much more likely to engage with the librarians in the teaching process in various ways by asking them to lead instruction by asking them to help identify teaching materials or creating assignments. You can see that the UNCW instructors were much more likely to work with librarians in that way. We were expecting that because UT's collection is so much larger than UNCW's but we would see a much higher response rate in terms of UTK instructors usage of our collection to prepare for teaching or in support of their teaching. That just wasn't the case. This was probably the most surprising aspect of the whole survey. I suspect that this is because UNCW as a smaller institution that doesn't have a stronger research focus as UT that that accounts for this difference. What we really like to do is deploy the survey at other institutions and other types of institutions in order to explore this further. We offered respondents the opportunity to provide feedback throughout the survey and as is often the case, what they had to say to us in their own words was more interesting and telling than their answers to the questions that we pose them and the answers that we provided to them. There were some benefits to their teaching and students' learning in terms of time in terms of the information to which students had access to. This is specifically as a result of using the library. There were also some opportunities that came up and this is what's always great about doing this kind of assessment. You really identify some misconceptions maybe or opportunities for new services to provide or new approaches to take. In case you're interested in conducting a similar survey at your institution I do have a couple of caveats to learn from our mistake. I do not work in the library here so it was essential that I have the librarians here at UT vet the survey to make sure that the services I was asking putting in the survey to ask instructors about were appropriate and comprehensive. Because instructors are also researchers. They love to pick apart research instruments and project design. I do that whenever I get a survey in the mail. It's just kind of second nature. So be prepared for that. Some people are going to have unsolicited advice for your instrument. Frequently you'll be able to learn quite a bit from what they have to say because they have expertise in the field. It's very, very important to identify a distribution strategy for your survey that is effective and as comprehensive as possible. You may need to kind of cobble together a system we were not able to send here at UT a single email to all instructional staff. We had to figure out kind of a workaround. And that gets to my next point. How important it is to secure support. It can be tremendously helpful if your library dean or director is willing to go to bat for distribution of your survey. Securing that effort or a provost maybe. And again that gets back to the whole idea of designing your instrument to mesh with the larger goals of the institution. But we did learn quite a bit from this and again we're still looking at our data and working with other institutions to see about deploying the survey for them. And Rachel we had a question about whether the faculty survey is something you can share. Do you want to say how you can do that? I would encourage any of you that would be interested in pursuing adapting the survey to get in touch with me. My email address is I believe at the end of the presentation. I'm easy to track down anyway. And we can talk about how best to do that. Thank you Rachel. Sure. Teresa? Hi everyone I'm Teresa Walker head of integrated user services at the University of Tennessee Libraries. And that really means many of our public services including the commons and the studio, our media production lab. And then also our lower division general education instruction in the libraries. And student engagement and outreach efforts as well. So we're kind of into a lot of different things here. I'm going to discuss the value tool dedicated to assessing the contribution of commons spaces to student success. And I want to acknowledge our partner in the study Gail Baker. She's a professor in electronic resources coordinator here and she has been the person really driving the data piece of this. So first I want to say that when we started developing the commons surveys our goal was to create assessments that would help us correlate commons usage with student academic success. And the first thing we needed to do was to find success. And to define it in terms of our greater institutional definition. And you've already heard about our strategic plan the vol vision and that's the pursuit of the top 25. So at UT success has had everything to do with retention rates. Retention rates, increased emphasis on graduation rates and then more recently a focus on blended and online education. Which I think a lot of people are focusing on. So it wasn't just about impact. It wasn't about did the student get the help or software she needed. It was more about correlating student use of commons resources with academic success over a period of time. And towards their graduation. So we designed it with that in mind. We also knew we had some useful data that was either automatically collected or regularly aggregated and we wanted to make the best use of that. After determining what we had we figured out what we still needed to gather. And we needed student reported usage of the commons as well as some perceptions of the value. So our existing data I think you can see both sets of data right now on this slide. But our existing data included that automatically collected data like computer logins, statistics and things like that. You can also see LibQual the national survey of student engagement. But we knew that we needed data that was also going to be about that student reported usage of the commons and the value that they felt that it had. We also really needed an augmented university data set to help us to be able to get the progress toward degree data. And on this next slide we have a little poll about that. And the poll is do you perceive barriers at your institution to obtaining progress toward degree data on students? And the choices are yes, no and not sure. We'll give them a couple of minutes to get to their clickers. And we're going to stop the question and let's take a look at the results. And we have 58% saying yes, 41% saying no. Okay, well I'm not surprised that 58% say yes. I think my institution has always been pretty good about providing certain kinds of data sets. The next slide here that says a new data set shows the data set that we were after. Admissions and demographic data had always been available to us, but this set includes progress toward degree data that's tied to individuals and over a period of time we will have this data. And I think there are many reasons why getting data like this can be a challenge. It could be that you have a legacy system at the university or college that is really at the legacy system that's out putting data a certain way that's not transferable to another system or maybe it doesn't put out reports easily. Maybe you have incompatible systems that have many different kinds of data that you need to have together. We ran into all of these problems and this part really took the most planning and patience. But our Office of Institutional Research and Assessment worked pretty hard with us to make this happen. And we're very happy with what we have now. Okay, now we're onto the surveys. And we created two surveys for this project. We created an in-person survey that was distributed in the commons on paper for one week in the summer of 2011 and one week in the fall. And we had nearly a thousand respondents. The in-class survey, a little more complicated, was made available to a large general education course. That's Communication Studies 210 and 240. And that was in the spring of 2012. This survey was administered via participant management system and the students in the class are required to participate in research studies. And this was one of the studies available to them. For this one we had 146 respondents, which was about 20% of the entire class. The in-class survey contained detailed usage questions with information on time spent on activities. It also included a section on student feelings and perceptions about the value of the commons to their success. So the next slide, how does your college or university define success? We'll go back to our university's definition of success. And that was we actually had a task force appointed by our provost to look at retention. We had a serious issue with retention. It had, for over a decade, hovered right around 75 to 80%, which was well below our peer institutions at the time. Luckily our Dean, our Associate Dean Rita Smith, was on that task force and that really kept us in the loop on what was happening. Also our Student Success Center had been conducting, it still conducts student exit surveys determining why students leave UT. And the top three reasons were that UT is too large and impersonal. Students had trouble adjusting personally to UT. And three, they didn't feel like they were part of the university. I mean that's a pretty serious thing to try to deal with at a large university. And I think we've always believed in the library that the commons was a space that sort of uniquely addressed some of those issues. Because it's very student centered space. We care about the students. During finals we have the habit dogs come to be with them and things like that. And so we take it very seriously. And so we were very, very pleased to see the next slide student responses to this question. And that was to what extent does the commons do several things? Over 70% said it helps them do better in class. My favorite one was that just over 70% said the commons makes them feel more involved in the university. And keep in mind one of the top reasons for students leaving UT was they didn't feel involved in the university. So an overwhelming percentage also said the commons promoted learning to help them do better in class. So we were very pleased with this. So on to the next slide how does your college or university define success? We fast forward a few years and our retention rates have indeed improved at UT. And our focus has turned more now to graduation rates. And toward this goal the university's strategy is really about engagement in the university community and assimilation into the scholarly community. And again the commons environment is sort of uniquely situated to provide students with the tools for success in a non intimidating environment. Most commons environments have a combination of academic support, technology support, research support and that sort of thing. Okay so the next slide shows acclimation to the scholarly community. And this one is great. It represents a combination of our survey data paired with progress toward degree data for the individual students. Showing that students with a GPA greater than 3.5 make more use of research assistance, computer support and just slightly lesser degree tutoring services available in the commons. Obviously I think every librarian here and probably anywhere is thrilled to see that students who are using research assistance are doing better in their classes. The next slide conversely shows that students with a GPA lower than 3.5 make less use of research assistance and computer support than the top students. So in other words students who know how to and regularly use the tools of scholarship, research and computing do better academically. And it remains to be seen if these students get into their major program sooner or if they graduate more quickly. We're obviously delighted with the results and to see the role that the commons plays in a culturation and we want to track this over time to see how they're doing. Okay so I'm going to talk now a little bit about a few surprises we had with the survey. I think we'd always assumed we were serving freshmen in the commons. It really surprises to see the distribution of students and it's quite even by class standing. It's a big surprise also I think to see seniors representing such a large user base. We were happy to see we were serving the needs of all class ranks and very happy to suppose that our use of common spaces and resources were increasingly important to students as they neared graduation. The next slide shows our advocates. This was another big surprise. 80% of students in the survey reported that their instructors tell them to use the commons. And our students have always thought of the commons but as professors started buying into the idea of the commons as a learning environment the transformation from the information commons to the learning commons really began. I think we've long known that our commons was well regarded on campus but more and more instructors are incorporating the space into the learning process itself. And that's both formally and informally. We also had some results that students were commenting that instructors and professors were telling them to use the commons. That they were hearing a lot about the commons through orientation. And one of the ones that I really liked is that other campus units, academic support units like Student Success, the Honors Program, International Education Program were referring students to the commons. Okay, and then the impact on learning. These results show student reported beliefs about the commons and its impact on learning. And you can see 90% said the commons provide resources they need for class. 74% said commons help them do better in class. 85% said the commons is a place to get help with assignments. And 95% said the commons help this facility group work and collaboration, which we're seeing more of a mandate for that in our curriculum as well. So we were happy to see these results. You always hope you're going to find what you want to find. And we've been glad that we have this amount of data to put together to see that the students believe it's helping. And that we can actually look at the respondents and their individual results to see how they're doing in their classes and progressing toward their degree. And finally, this is my Why We Do What We Do slide comment. My Dean always says that assessments about numbers and stories and this story I really like. A student said, I want to thank the library for the commons. I feel like I would be lagging behind as a student without it. It's my home away from home. I almost couldn't hear this without bursting into tears or something. But I was like, yes, finally, that's what we've been working toward. But it was good to hear that the students actually were feeling this way about it. Because I really do think that's what we were trying to achieve. And that's what I have. There was a question that asks, are you looked into other factors that may have resulted in a higher GPA beyond the library use? Actually, yes. There was a companion. And I think there's probably some different ways we could do this in the future. There's a companion piece to this survey for students who said that they did not make much use of the commons. They got to take another survey, which looked at other campus services. And we did get some results on that about maybe the same kinds of questions for other campus spaces. But honestly, what happened was the results, most students reported that they made major use of the commons for these activities and not the other spaces. So the results we got for the students who chose to take the other spaces survey, there were so few results that we couldn't do a lot with them. Combining the two surveys might do a little more because it would show which students are using all types of spaces, including the commons, but most of them chose to take the common survey instead. So we really didn't get good results for the other spaces. The numbers were very low. I have a related question from Ohio University. Do students actually refer to the commons as the commons, or do they just refer to it and view it as how to use the library? And, you know, what their perception affects the way they respond to them? We've actually had a lot of success with branding our commons. I mean, they all know it's in the library. They may all say they love the library and they do. But the commons are branded here. Students call it the commons. Faculty on campus, other units call it the commons. And I think it's pretty well known as such. Okay, more questions are coming in. One from NCSU, North Carolina State University. How do you correlate the progress towards degree data with your survey data? In a few different ways. And honestly, most of the data, most of the data is still pretty new to us. And it took a long time to get the progress towards degree data. And what we've looked at, I don't think we have any direct correlations, just like we don't have any, you don't have learning outcomes in a strategic plan. We don't have any direct correlations. What we've been able to do since the survey data is attached to individual students, this other data is also attached to the individual students. So what we've been able to do is look at the student reported usage, individual student reported usage of a service, the reported value that they place on the service. And then we've been able to pair that with their progress toward degree data, current GPA, other information like that. And so we've really had to pair it on an individual level and then look at it in that way, aggregated after we've done that. Great. And the question coming from Toronto asks whether the survey instruments are available. Yes, and actually we're working with, as I mentioned earlier, Gail Baker working right now to tweak those just a little bit based on what we found. You know, when you run a survey a time or two, you realize that there are better ways to say some things. And some questions that no matter what you do don't make sense to people. And so we are tweaking that a little bit and the surveys will be available and I would encourage you just to get in touch with me about that. We can discuss that. Yeah, and we've talked about if there is enough interest doing a small pilot across a group of libraries like we have done with other work here at ARL. So thank you very much, Rachel, Nina and Teresa. I do have a few concluding thoughts. Clearly we're interested in outcomes and it's good to remind ourselves that ultimately it's not about graduation rates and test scores. And though these are important indicators and correlate with outcomes of success, this comes from Khan. It's in his book The One World Schoolhouse Education Reimagined. It's the person who established the Khan Academy, a series of online courses for high school and incoming undergraduates. And he goes ahead to say it's about what those things mean to the outcome of human life. It's about potential realized or squandered dignity enhanced or denied. So keeping that in mind as we look into our numbers into our correlations and understanding the larger set of concepts that underlie this line of work. Now there is other work happening in this space so it covers actually a number of continents. We are familiar with the work that Brian Cox and Margie Gentie have accomplished in Australia. It was featured in the Educational Review Online and this slide here has a URL with an article of theirs that I hope many of you will go and read and get inspired more. The other article from the UK worked done by Stone and Ramston and looks into the relationship between library usage and student attainment as well. And the last work brought to your attention here is from the University of Minnesota. They presented an award-winning poster at the Library Assessment Conference in October. There are two articles that are to be published out of that work. Keep an eye for them. They are coming in portal libraries in the academy in the April 2013 issue. Thank you very much. Happy Valentine's. Bye. We conclude today's webcast. We thank you for your participation. You may now disconnect your lines and have a great day.