 Hello everybody. Welcome to Libraries that Learn, Using Evidence to Transform Library Services. This is a webcast that will feature how live call has been used at Texas A&M University. And this is part of a series of webcasts. This is the second one. The first one has already been taped and it's available on the ARL YouTube channel. And it was with Celina Kilic from Cranfield University. In this one we have Michael Maciel from Texas A&M University. This is the institution where live call was created many, many years ago. And the next one, the May 5th one is going to feature Lori Clauda from McGill University. One of the key people who developed live call was Colleen Cook who was AUL at Texas A&M University when she originally developed live call. And then she became the dean of libraries there. And more recently now she is the dean of libraries at McGill University. So there is a little bit of a link between these two institutions there. But Michael Maciel is here to tell us about the way live call is used at Texas A&M University. Michael, the floor is yours. Hello Michael, can you hear me? I can hear you. Does that mean everyone else can hear me too? Yes, the floor is yours. Go ahead. All right, well thank you everyone for taking time out today to listen to me speak about live call at the Texas A&M University Libraries. The title of this presentation is Libraries That Learn and Using Evidence to Transform Library Services. I do want to start out by talking a little bit about what the assessment person in this case myself has learned over the years. And primarily I have three goals to talk about today. And the first of that is to talk about how I go about presenting the live call findings to the libraries and also to our library users. I've learned over the years that to be successful in this you have to learn to speak the language of your audiences. If your audience is more interested in expenditures then that's where you need to begin the conversation. If your audience is more interested in collections, once again, that's where you're going to begin this discourse of what live call has revealed to libraries, to its users and what we plan to do with that information. And last what I've learned is that over years and years I've just rushed in to get out all the live call data from a completed survey at one time. I've learned that the better approach is to parse out the live call findings by section and we'll talk about the various sections that I've developed over the years. But develop and deliver those findings by section over the course of time and not just in one large sum. It generates interest and it also keeps live call at the forefront of what people are thinking about when they are talking about user needs and how to identify those and meet those. A little bit about before we begin I want to talk about the university itself. We are a Lansing Grant University. As of 2013-14 we had over 62,000 students with 3,200 full time instructional faculty. The library itself is composed of 14 different libraries. These are located mainly throughout the state of Texas. We do have a library located in Qatar and the Middle East. We possess over 5.2 million volumes. As a matter of fact we just got through celebrating the addition of our official 5 million volume which was a first edition of the Hobbit. We have 121,000 unique serial titles, 94% of which are available electronic format, and 93 librarians. When I go about looking at the live call core questions what I've done is actually split them into smaller bite size pieces that people can understand. Within effective service there's nine questions. Five which address customer treatment. Four which talk about the job expertise of our librarians and the people that serve our users. Information controls broken into the resources themselves whether the print, electronic, or the distinction of being journal. Five questions of the aid of information control address accessibility of that information. What are we doing as a library to provide access to these resources that we collect in a format and in the delivery method that our users are comfortable with using and that is in many cases very self-intuitive to them to use. Third within libraries place there's three questions I think deal with the library environment as a whole while one question addresses individual study and another question addresses community space. If you can see from the slide how I've broken these out. So first of all at midwinter ALA what Martha and myself presented was setting up registering for setting up and starting the survey. So for those of you that are interested in that I believe that the slides in presentation that was presented at ALA midwinter and that will probably be presented at ALA coming up can help you with that part of the process. But just keep in mind that where I'm going from this point is as if you've already registered and set up your survey and you just hit the button to begin the survey. One of the first things I look at is or I do while the survey is running is I do look at the summaries page. I've included a snapshot of some of the graphs that are presented by user group of the data repository survey page that is on live call. I particularly look at the discipline survey and I look at the survey by position or by user group. What I'm looking for in particular within the discipline is to see if there's areas where our marketing efforts have not penetrated as well as we would have liked. For example, I'll go down to nursing. What I'm looking at more is that 21 rather than that 0.64%. And in particular, I know that's 21 out of a total population of 63. So we're actually reaching about a... we're actually obtaining about a 33% response rate. So even though that number looks low and looks less impactful on this graph, what that 21 is to the total population tells me we're doing a pretty good job. On the other hand, I can look at dentistry and tell you that that 28 is not as representative of the population. In cases like that, what I'll do is I'll contact the subject librarian and mid-survey and talk about other ways that we can market the survey to that particular group. I also want to point out that there's another undecided category and if that number gets too large, again, it makes me wonder about who these individuals are that aren't identifying colleges. And generally in those cases, I'll start going to the comments to see if I can figure out if there's a user group that we've mislabeled or a user group that we just haven't represented in the survey. And the same thing can be said for the user groups themselves, undergraduate, graduate, and faculty. It's no surprise that undergraduates represent the largest of that, but again, looking at those numbers in relation to what the total population is by user group tells me about where we're successful and where we need to kind of amp up the marketing of our survey to get a better response rate. Martha, any comments or questions at this point? Michael, what about Kay? I was going to ask you what year it is. I see on the slide it's 2013. This is good. Thank you. Yeah, that is correct. Now, when I start presenting results, what happened was we ran part of the survey in 2013 and we ran the survey again in 2014 for some user groups that we had missed. Our Health Science Center had just merged with the university and as a result, we weren't able to get an accurate list of e-mail addresses until 2014 to specifically invite people from that center to participate. As I start presenting the data, it may say 2012-2013, but in fact it represents two surveys combined over the course of 2012 through 2014. Okay, so next slide is, as I mentioned, while the surveys is going on, what I do is I typically start reviewing the comments and start coding them. A lot of times as the comments come in, you'll see trends that you can begin discussions with various people within the library on, but also you'll be able to see issues arising. For example, a user may comment on a specific journal title that they're saying that we don't have as a resource. It's just been my observation over the course of the years is that when we do have that happen, more often than not we do have a subscription to that journal and what the problem is is that we've neglected to put in alternative titles that enable our users to better access and better discover those journal titles when they're searching. It's a really bad example, but I use JAMA. A lot of people still look for that under Journal of American Medicine. And as a result of that, if we don't have that 246 line in there, they're unaware of the fact that we do keep a current subscription with that resource. As I get the comments, I do begin distributing them. I don't wait until the end of the survey to do that. Again, part of this is just to make sure that people stay involved in the live call survey process as it's going on as opposed to after it's done deep. I will send out comments, particularly significant comments that may address a particular user group, a discipline, or a library, or often find that comments that address specific libraries have to do with not only the library's place feature within live call, but also address some of the needs of that particular campus. We do have several libraries that are located on campuses that address a specific discipline. For example, we have a Kingsville library that primarily addresses pharmacy research and resources. I look for those comments coming out of Kingsville and do forward those to the Kingsville librarian and as well as the Pharmacy Subject Librarian to review. I send out library department comments by library department, library functions, for example, our ILL department, as well as begin the coding process for the various live call categories, information control, libraries place, and effective service. The one thing that's not on this list is I also, if a specific person or a specific department is mentioned, I make sure that I forward those comments not only to the person or group being addressed, but also to their supervisor as well as to our dean just to make sure that those kudos that come from our users are being recognized. Now, as I mentioned, I do break up the live call comments into the core questions three categories with the exception that I've actually broken out the information control into two separate. These are effective service information. Control has been broken into the resources, information resources, as well as information accessibility. In my mind, those are sometimes two separate issues. And then fourth, libraries place. You can see under each of these, I've assigned additional codes or subcodes for particular areas that we're interested in for effective service, any comments to deal with job expertise or our marketing efforts. Library and place, libraries place, we do look at food. It's become actually an important feature or factor when discussing the libraries place. I'd look to see if there's any comments whether it's great vending machines you have or if it's something like, gee, the library's beginning to look a little dirty with all the food wrappers around. And again, within information resources, the hot button these days particularly is textbooks and what the library does to provide access to them. So I keep a special eye out for that. Here's a question that came through when you mentioned that you forward the compliments to the people who receive complimentary mentions in the comments. Someone is wondering, Leona Jacobs is wondering, what do you do with the negative feedback you may be getting? Well, typically what I'll do is I'm going to route those primarily to our associate deans. And the reason for that is those are very sensitive comments and one, we may decide not to include them. We do present all the comments out to the entire library to review, but we may not include those type comments. And the other thing is what do we do about this issue? Is it necessarily the employee that is at fault here or is this an issue that we particularly drop the ball and the consistency of the customer treatment training that we want to provide? And that's become a real issue because A&M is in that transition of changing our customer service model. But those comments get directed to the associate dean first. In one case, and again just kind of unusual circumstance, but in one case the individual that was mentioned was retiring that year. So we just opted not to do anything about that comment at that level, you know, to that person. But just, you know, make sure we re-emphasize that particular point of customer service to the entire group. Yeah, this is actually an interesting practice because Leon also followed up and says that she herself depersonalizes the comments. So the focus is on the customer service issue and not on the person. Thank you. Thank you, Michael. And Leon, that's actually an excellent point. Where I can or where we can, we do that too because there may be a bigger point than just that individual person that needs to be addressed. Now one thing that I want to point out that when you do comments analysis is using the broad, you know, core question categories, you really don't see any surprises in what our user groups consider priorities. If you look at the undergraduate comments, 60% of them addressed the library's place. We expect that. Graduates, 30% addressed library's place, 35% addressed information accessibility, and then another 17% addressed information resources. Again, the expectation there is that, you know, graduate students are going to be using the library as a place to study as well. It's been my experience that we have to be very sensitive to this as a user group because they're not looking for the group facilities that we offered undergraduates. They really want to see those, you know, quiet individual study spaces available. And information accessibility is a big issue both with graduates and undergraduates but kind of more of a hot button with graduates in that they want to be able to discover materials on their own. This chart representing that 35% of comments addressed information accessibility from graduates again tells us something that we sort of already expected which, you know, self-discovery is a very important part of the library service that we need to offer to our graduates. Which, if you move on to faculty again, is something that we do know already that information accessibility for faculty is important as well as the information resources themselves. What's interesting about this, which is also borne out on the live call of questions is that treatment for how we treat faculty is very important and to be very specific it's being able to demonstrate to our faculty that we're able to speak to them about their subject in an intelligent fashion. We really do kind of have to prove to our faculty more than any other user group that, you know, we speak their language, we understand their research and what they're focusing on. Again, going back to the slide about the comments and how I provide subcodes where you may find, you know, self-evident discoveries within the comments, it's when you really start to break down the comments into these smaller groups where you get some pretty important revelations. One thing that we found out over the course of time was that we thought we had set a very reasonable policy on fines. In fact, the small changes that we thought were good changes turned out not to be well received by our user groups and we had to go back and revisit that issue. The devil really is in the details when it comes to looking at comments. You can definitely categorize them by the larger groups but here's a chance for you to really sit down and look at each comment and see if it's telling you something that either confirms the suspicion or introduces a new line of thought into how you perceive user needs. One of the first things I do after the survey is, again, as the survey is being conducted, I'll do the comments analysis. One of the first presentations that I do once the survey is closed is I do a presentation on comments. After that, the next review I do is what I call the top five list is where I look by user group, what are the top five desired scores, which I translate to mean priorities. What are the top five, again, by user group successes, which are perceptions. And then I address the areas where we're falling short of meeting user needs. I use a review of the adequacy gap within the live call zone of tolerance and it's a ratio that I use that turns this into percentage where it basically says where is your perception located within that zone of tolerance. The closer your perception is to the minimal score, we consider that a concern and the closer your perceived score is to the desired score, we consider that a success. But you can see here that you can do two different types of reviews. One of them is, let me see if I can get the marker to work here. One of them is I will look by user group across the board about what the priorities, successes and concerns are. You'll see that from the priorities, the biggest, again, no surprise, the priority is going to be the library space component. What surprised us this last year was that quiet space was very important to undergraduates. In previous years that hadn't been the case, it was group space. But this caused us to re-evaluate what our needs were for space redesign. We're currently going through, re-imagine the library's project, and looking at our upper floor study areas that historically have been group study areas that we're now becoming a little bit more particular in terms of defining quiet space areas as well as group study areas. We fortunately have a great model for that. I believe Leslie Reynolds is part of this discussion, but Leslie many years ago used to be the head of our West Campus libraries, which was de facto social center of the campus for many years. It quickly reached capacity each and every single night. So we had to address both group study needs, which would be associated with noise that comes out of putting a whole bunch of people in a large room as well as finding and defining areas where we could provide quiet space, but also in an area where the groups themselves disciplined themselves to maintain that quiet area. And if you look from the concerns, you'll see again that quiet space is a concern and that we needed at the time of this survey to have done something about that for undergraduates. In fact, as we're beginning our renovation, we are including a lot more on quiet space areas. If you look at our successes, what's really nice about this is that they're all talking about customer treatment and customer care, job knowledge. But once you start looking into more of the details you find that these successes are also matched by the fact that many of our undergraduates just don't want help. They really want a universe where they can discover materials on their own, be able to research materials on their own. So part of being able to be a success in customer service is understanding that when these undergraduates do come to us, they're coming to us very frustrated. They've already tried stuff on their own and not only do we need to be able to meet their research needs, but we also need to be able to understand their tone and their, well, for lack of a better word, mood when they approach us. We need a little bit of resilience training, if you will, and what we've incorporated into our customer training, in fact. So anyway, one way of looking at this is by the user group themselves. The other is to look at seeing what priorities, successes, or concerns translate across user groups. And again, if you look at priorities, you'll see that library space for undergraduates was a big priority. Information access and information resources was a big issue for graduates, as well as for faculty. But I do want to point out here, and this is where we talked earlier about the fact that with faculty, we found that the ability or at least being able to convince our faculty that we know what they're speaking about, and we do have the skills necessary to understand what they're looking for as part of our training matrix. It's not just enough to say, we're a librarian, we can find anything, but we really do have to be able to speak their language in order to help their needs. Questions so far? We're doing good, Martha? Yeah, I think we're doing good. People, feel free to type your questions, and yeah, let's move on. All right, so after I've done the comments analysis presentation and I've done the top five presentations, the next thing I start looking at is I start looking at comments by trend. In this case, you'll see that what I did is I picked the printed library materials I need for work, information control question, and I've tracked that from 2001 through 2012. Now, unfortunately, this is one of those charts that has maybe a little too much to look at at once. So for right now what I'm going to ask is that you ignore the ARL desired scores and the ARL perceived scores and just look at my blocks here, which are zones of tolerance, and then my diamonds, which are the perceived scores. And you can see over the course of the year that for our faculty, the perceived scores have risen to the point that in 2012, not only were we scoring very high with regard to the perception of the library print materials that we provide, but in the cases of our faculty, we actually exceeded their needs. The faculty were beginning to reach that point where format wasn't important or critical to this issue as it had been in previous years. I do want to point out 2005 for you here because it's just kind of interesting. This was a year that we made a very concerted effort to move as much of our materials as we could to electronic resources. It is a result of that where in the previous two years we were within the zone of tolerance. In 2005, we fell out of the zone of tolerance as well as the zone of tolerance increasing a bit. And again, this was affected by, this was the result of us moving more materials to electronic resources and our users reacting to that. But if you look at 2006 and then previous years, skipping 2007, we'll pretend that doesn't exist for right now. But looking at 2006, you can see we begin falling within the zone of tolerance. And over the course of the years, we steadily increased as the priority of print materials decreased. So that is the trend analysis that we're doing. The benchmarking analysis I do is, I do like to time-to-time compare ourselves to ARL just to see are we consistent with the trend or is there a trend out there where we need to be doing something different and there's evidence to indicate that there's libraries out there that are doing it better than us. If you look at the perceived scores, you'll see that over the course of time, ARL scores have also increased not as much, but keep in mind that may just be a factor, the fact that we're averaging scores here as opposed to looking at a particular institution. And desired scores, again, are doing the same thing that's happening in Texas A&M. They started out high and are slowly decreasing as our population gets used to the fact that more and more information is going to be delivered to them in an electronic format. One of the biggest hurdles that we're facing right now is that we've accomplished the print journal to e-journal transition and we're now currently working on the print book to e-book model. And many of our professors in particular are having to go through the transition process themselves and getting used to that change in format. So, again, I will also look at questions by department or by college to see if there's a particular need where we need to address a particular college's concerns. And you can see that desired scores vary depending upon department for electronic resources. The good news is that in all cases, we are within the zone of tolerance. The issue that raises concerns is that some colleges, for example, liberal arts, while we are in the zone of tolerance, we're not particularly in the higher part of that zone of tolerance. You can see we're the lower 50% of the zone of tolerance here. And so the question goes out, you know, exactly what's going on with the liberal arts and why they don't feel we're successfully meeting their electronic resource needs as we are with other colleges. And I guess this is a good time to point out that one thing that live call does is live call will present answers for you, but in many cases, live call will also identify an issue that needs further research. And in the case of liberal arts, what we had to do is speak specifically some subject librarians. In fact, went out there and began having a discussion about, you know, what e-resources that you know about that are out there that we're not providing and what can we do to improve that? Next slide. So, getting near the end here, we're talking about presentation of survey results, internal presentations. Again, the comments and analysis that I talked about, internal presentations, the data analyses. At this point, we are, even though our funding model has changed slightly, we're still beholding upon the student's blessing for support when it comes to funding for the library. So it's important that we go out in a perfect world annually, sometimes bi-annually, but we'll go out to the various student governments, the undergraduate student government, the graduate council, and definitely the faculty senate, where we'll present findings that are germane to their missions. For example, the faculty will certainly talk about what the faculty needs are, but to address instructional faculty needs, I'll make sure to include some of the findings from both the graduates and the graduates when we present to them. I have, upon invitation of subject librarians, given presentations to various colleges. When you talk about learning the language of the audience, this is probably the worst-case scenario, because going in there as an assessment person, you may not have the same dialogue or relationships built that our subject librarian does. So it's important that you partner with your subject librarian when you do present to various colleges on campus. And the big thing, especially now, in 2017, Texas A&M's getting ready to go through another reaffirmation of accreditation with our regional accrediting agency, which is SACS, and live call has become a very important part, both in our yearly assessment reports that we're preparing for the assessment report in 2017, but also for the particular guidelines and regulations for libraries within our regional reporting group. Now, we talked a little about what we've learned, what I've learned at the libraries, and I just want to kind of address the specifics in terms of the subtitle here, Using Evidence to Transform Library Services. What we found with customer treatment over the years was that it was very inconsistent. We were getting very good scores on the live call core questions themselves, but we found that within our comments, we kept every single year getting those comments that were Joe So and So, or the woman at the service desk was rude to me, and we really couldn't shake those. I'm not talking about maybe one or two within a set of 200 comments. I'm talking about a larger percentage of those comments with respect to effective service. So we really felt that our issue here was that, one, in many cases we already had a good model for how to treat our customers and how to talk to our customers, but that we weren't being very consistent with that training. So I want to say about five years ago, we went out and enlisted the services of the Disney Leadership Institute and have over these years developed a customer service model that's based upon what we've learned at that institute and in particular on how to deliver a consistent, underline network consistent, positive, another word to underline, customer service interaction. Job expertise was another issue and what we found in particularly for faculty and in particular for our subject librarians was that it was important that at the very least our subject librarians have some understanding of what courses our faculty was teaching or what research and in what fields they were teaching so that they develop that kind of bridge between the library and the faculty member specifically in order to initiate meaningful conversations. A little bit of knowledge about who your customer is can go a long way. Instruction came up. In addition to the live call questions that are provided, we had also conducted surveys within many of the instructional sessions that were provided and both live call and the surveys from the sessions all indicated that we needed a more consistent approach. And over the course of years we broke our approach into two perspectives. One is that instead of trying to teach what library services we offer and what lifelong learning skills we could provide for our students, we broke that into two groups. We provide a very substantial open health orientation model for our first year students that's irrespective of whether they're transfer students, first year undergraduate or graduate. We do provide a similar service for our faculty members where we talk just about what the library has to offer. It's hours, it's ability, what equipment is available for checkout, what facilities are available for use within the libraries and stuff of that nature. And then later on, especially in the case of undergraduates, more often in their junior and senior year when they're taking many of their core requirements and their writing specific classes is where we begin the discussion about lifelong learning skills and information literacy. We were very fortunate to hire a very talented person out of New York to handle our marketing in the libraries and we're delivering a more consistent message and when people see that certain icon that represent the libraries, they recognize that as an information resource in itself. We've learned over the years through our live call findings and in particular our live call comments that there was a need and as the comments are coming out we're finding that we're being successful in meeting that marketing need. Information control, information resources in many cases has sent us delving deeper into issues. I talked about the e-book versus print book matter. One of the biggest issues that I have is that I can, through circulation statistics, measure print transactions and print trends but have been unable to do that with electronic resources. I definitely want to talk about the success we've had with an ARL survey called Minds for Libraries where it actually helps us identify not only what resources are being used but by what user groups, faculty, undergraduate, graduate and also by what colleges and departments. I would recommend that you have the resources in the time that you look into the Minds for Libraries if you want to get a better understanding of who your electronic resource users are. Information accessibility was also a big concern that we've addressed over the years in terms of more than anything else, website design. As I mentioned at Midwinter, I don't think we're ever going to get ahead of the curve on designing the perfect web site. Technology will always be challenging us to keep up but we have learned things over the years and as a result have been a lot more effective in enabling our users to self-discover resources and services at the library. Finally, I talked about the library's place and library environment. The comments that have come out have helped us support arguments for funding for library renovations as well as support funding for ancillary projects such as we recently got approval to add an additional module to our off-site storage and the reason for that was that the library environment comments and findings all indicated that we needed to start devoting more room for user study as opposed to collection housing. Taking that argument to the powers that be, we were able to get the money to build an extra module which in return will enable us to get more space available within our main libraries to provide better studying facilities. Do you have an estimate of the percent of the collection that's now in very mode storage? No, I don't. The last time we checked we were just gearing up last year and within 60, I'm sorry, within six months of opening up the facility we had already reached our goal which I want to say was in the hundreds of thousands of books but as far as the percentage of the collection I will know that by the end of this year because it's one thing that we've built into our accreditation effort and again it's going to be one of those cases where we have the live call comments and live call findings to indicate why this is a priority and why we needed funding and then to be able to demonstrate what we did with that funding and how we alleviated those user concerns. Do you want to do one more comment? I want to talk about the climate call survey. I know we're talking about live call but climate call was an incredible survey done to talk about work climate within the libraries. We've conducted that twice and have used those findings extensively to create a friendlier and hopefully more positive work environment for employees which in turn is hopefully translating into better customer treatment. And with that I'm not sure if I hit my 15 minutes or have exceeded it but that's it. Thank you very much. Any comments or questions? Thank you, Michael. Just a closing question on the accreditation. What are some of the trends you are seeing highlighting in the accrediting reports beyond the fact that you show evidence that you're acting on feedback and improving? I don't quite understand the question. Could you... I was wondering if there are certain trends that you are seeing being highlighted in the accreditation reports from whether it's in relation to for example the print materials that faculty are meeting their desired expectations at this point. Are there any other trends like that that you see being highlighted in the accrediting report? One of the biggest in this next round of accreditation that we're going to be doing is going to be talking about facilities usage. We've talked in particular about per-square footage cost and what we're doing is translating by moving X amount of volumes from our main libraries into our off-site storage facilities. We freed up this and this has become a study area and compare the cost of this renovation project versus if we had to build a new facility. I believe as I said at mid-winter, someone said that they will never build a new library facility in their professional lifetime but they do need to prepare for the next dean or the dean after that that will have to start building new buildings and I'm seeing that trend especially with a couple of other universities where now we're talking not only about the administration of the existing leadership but what plans do we have out there beyond five years in terms of facilities. Customer service is always a big issue but it's the job issue, the job expertise issue that's also another trend. There's a couple of accrediting agencies both regional and programmatic accrediting agencies that have actually dropped the library requirements and so we're finding ourselves in a situation where if we're going to want to make sure that our voice is heard in the accreditation process and therefore heard by our administration as a priority then we have to go beyond just looking for those specific mentions of the library and take what we do and demonstrate how it supports training outcomes and other administrative priorities at the time. Wonderful. Thank you very much for sharing all this rich experience that you have built over the years at Texas A&M. Thank you to our audience. The presentation will be captured and will be available on YouTube and Michael is available at the email you see on your slide maciel.edu Thank you Michael. Thank you.