What We Learned
When conducting an annual assessment, an organization should learn about two things: 1) its processes for collecting and analyzing assessment data and 2) how well it is meeting it goals.
In the actual collecting and analyzing of assessment data, we were largely successful. The UCM Library Annual Assessment Plan 2011-2012 consists of six principal goals (A through F), twenty-one outcomes, and thirty-nine measures by which to gauge how well the library achieved its goals and outcomes. The library was able to collect useful data for thirty-four (87.82%) of its measures. The cases in which we failed to collect data (B12, B13, C3, E2, and F4) where the result of either a lack of staff resources or because obtaining the data proved to be impossible. As explained in detail below, we learned that in some cases we need to adjust the kind data we collect, while in at least one case (D6), the data proved to be meaningless and thus not worth collecting. Both our successes and failures have guided our plans for collecting and analyzing assessment data during 2012-2013 and should improve our processes.
The data show that the UC Merced Library is, by and large, meeting its goals. The details of what we learned and changes that we made or intend to make in reaction to what we learned are provided below.
Of course the hard-number data that comes out such efforts as the annual UC Libraries Systemwide reporting, while easy to collect and quantify, tells us much less about the true impact of the UC Merced Library on student learning outcomes and success than does the hard-to-collect, difficult-to-analyze data that comes out of such efforts as local focus groups and analysis of student writing portfolios. While it would be ideal to learn more about how well the UC Merced Library directly supports student success and how to do a better job of providing such support, there are not enough resources to assess everything we would like assess to the extent that we would like to assess it. As with many units within the university, the library must be conservative when it comes to how much of its human resources it puts into assessing itself versus how much it puts into carrying out its essential functions, especially in a time of fiscal restraint that already hinders our ability to provide the essential information resources and library services necessary to support research and student success.
As a service organization in a non-profit setting, it is important that the UC Merced Library hear what our end users are saying so that we can respond to their needs. At the same time, it is important to not be overly swayed by end-user comments, be they flattering or derogatory. While most of the comments by end users were positive, we need to use care when considering to what extent happy users equals supporting research and student success. Students and faculty are capable of asking the library for information resources and services that, in a world of unlimited demands and limited means to meet those demands, would not lead to successful outcomes if the library were to provide them. The reality is that the UC Merced Library must balance two roles: stewardship and leadership. Stewardship consist of providing the UC Merced community with the information resources, library spaces, and library services it needs to conduct research, teach, and learn. To successfully carry out its stewardship role, it is important that we in the library listen to our user community; however, it is even more important that we pay careful attention to how students and faculty actually use the library resources, services, and spaces. For one notable example, even though students and faculty have, quite justifiably, complained about the clumsiness of the end-user interface of EBL electronic-book database, the usage statistics for EBL show heavy, and steadily increasing, use of this resource by the UC Merced community in spite of its shortcomings.
Besides practicing stewardship, the UC Merced Library must show leadership by keeping an eye on the future of information and positioning itself to provide emerging information resources and services, the bulk of which its user community may not even be aware. Fulfilling only a stewardship role ensures a library with its feet firmly planted in the past and one which will, ultimately, fail to keep up with the evolving needs of its users. Balancing leadership and stewardship is a difficult, often sensitive, challenge which the library must face with both tact and rigorous professionalism.
Goal A
The library will leverage its physical resources (space, furniture, fixtures, equipment, and IT infrastructure) to provide for the common good of the UC Merced Library user community, with a non-exclusive bias towards meeting the needs of undergraduate students.
Outcome A1
Provide accessible, functional, comfortable, and productive spaces that support a variety of academic needs, including quiet study, group study, social interaction, and campus meetings and events.
Relevant measures: A1, A2, A3, A4
Outcome A2
Reduce physical barriers to information by providing easily navigated spaces, logically organized and maintained book stacks, and clear options for library services.
Relevant measures: A2, A3
Outcome A3
Make all decisions about library spaces based on both user assessment and the professional judgment of library staff.
Relevant measures: A1, A2, A3
Measure A1
The UC Merced Library will annually compare its opening hours with those of other UC Libraries with the goal of being at or above the UC Systemwide median for total hours open.
A1 Findings:
The UC Merced Library was open 95.4 hours per week during 2011-2012. For the same FY, the median opening hours for the combined UC Libraries was 92.75 hours per week. (The average was 93.2 hours per week).
A1 Actions/Next Steps:
UC Merced Library will continue to monitor the opening hours of our peer UC Libraries, as well as other academic libraries, to ensure that UCM Library hours are equitable.
Measure A2
The UC Merced Library will measure building occupancy rates, tying measured occupancies to day of the week, time of day, and library locations. The Library Access Services Department will conduct daily occupancy counts on nights and weekends; it will conduct weekday counts from 2 to 4 times per semester. The library will analyze this data and consider it in planning the scheduling of operating hours, staffing levels, and furniture acquisition.
A2 Findings:
A2 Actions/Next Steps:
The library will consider headcount data as a factor in determining hours and staffing levels. We will continue to track this data over time to see if any strong trends develop or building use changes.
A2 Supplemental Data:
Fall 2011-Spring 2012_Headcounts.xlxs
Measure A3
Seventy-five percent of students in focus groups will respond that they are “satisfied” or “very satisfied” with the library’s physical resources. Findings inconsistent with this goal will be examined for space planning implications.
A3 Findings:
Fifty-five percent of focus-group respondents indicated that they were very satisfied or satisfied with the UC Merced Library’s physical resources.
Individuals from the focus group referred to many library locations when asked what areas they used the most. Many liked the fourth floor for a variety of reasons, including “desks for two people,” “that’s where my favorite books are,” quiet rooms for study, and areas to study with “my sorority”. The third floor was called out for classroom space, the Green Room (KL 355), and meeting rooms. A few individuals liked the new partitions and white boards on the second floor (installed in January 2012), while another individual liked the ease of getting in and out of the second floor, especially when in a hurry. Students mentioned the collaborative work rooms and rooms for meetings as useful. In the comments, an individual mentioned using the Social Science Management Building for studying when the library was busy. The food and drink policy was viewed positively. Students mentioned liking the computer labs in the West Wing, though these labs are not in library-managed space and are under the management of Campus IT.
A3 Actions/Next Steps:
The surprisingly low satisfaction with library physical resources is possibly due to the previous questions asking students to convey their experience with public printing in the library, long a sore spot with UC Merced students. Although the public printers in the library are not under library management, students often direct their unhappiness over poor printer performance at the library. In 2012 the library moved all public printers located in library-managed spaces to KL 255 so that library staff at the second-floor desk could assist with printing problems. This change, along with technical and service improvements implemented by the campus units that manage public printers, should increase satisfaction with public printers.
With future focus groups, the question about satisfaction with library physical resources will be asked before the printing question to see if this results in a more positive response.
The library will continue to assess student needs and satisfaction in order to provide the best possible physical resources for the campus, though this will become increasingly difficult for two reasons: 1) the number of students on campus is growing and 2) the square footage of public space in the library has decreased as library spaces have been commandeered for other uses. It is inevitable that a crowded library building will be noisier and feel less welcoming than an uncrowded building.
A3 Supplemental Data:
Details of Focus Group 2012
Goal B
The library will sustain—in the face of a growing community of students and faculty—current levels and quality of user communication and instruction services by optimizing our resources, collaborating with partners, leveraging technology, and diversifying our instruction models.
Outcome B1
Increase undergraduate student research skills in a sustainable manner through consistent and effective information literacy instruction and the use of existing and emerging technologies.
Relevant measures: B1, B2, B3, B4, B5, B6, B7, B8, B9, B10, B11, B12, B13
Outcome B2
Use the iPod Touch tour to provide effective and independent library instruction to incoming freshmen students.
Relevant measures: B3, B5, B6
Outcome B3
Provide library student assistants with the knowledge and skills to provide excellent customer service, including the ability to answer and refer questions.
Relevant measures: E3
Outcome B4
Increase faculty and graduate student awareness of the benefits of the open- access movement and the unsustainable nature of the current scholarly publishing model for academic institutions.
Relevant measures: B11
Measure B1
Provide information-literacy instruction to over 50% of WRI 10 sections in Fall Semester 2011 and Spring Semester 2012.
B1 Findings:
The library provided information-literacy instruction to 56% of the WRI 10 sections offered during the Fall Semester 2011 and Spring Semester 2012.
B1 Actions/Next Steps:
The library will continue to meet this goal, though doing so will become more difficult as the number of WRI 10 sections increase while the number of UC Merced librarians remains static.
B1 Supplemental Data:
Information Literacy Instruction
Measure B2
Fulfill 100% of requests for instruction.
B2 Findings:
The library fulfilled 100% of requests for instruction.
B2 Actions/Next Steps:
The library will continue to meet this goal, though doing so will become more difficult as the number of requests increase while the number of UC Merced librarians remains static.
Measure B3
At least 70% of incoming freshman students will have taken the iPod Touch tour between the start of Fall Semester 2011 and the middle of Spring Semester 2012.
B3 Findings:
From August 2011 through March of 2012, 44.8% of all incoming freshmen took the iPod Touch tour of the library. (Out of 1,767 incoming freshmen, 798 took the tour.)
B3 Actions/Next Steps:
The library will work at increasing participation. The key to participation hinges on convincing WRI 1 instructors to require students to take the tour and complete the accompanying quiz.
B3 Supplemental Data:
iPod Touch Tour Data
Measure B4
Eighty percent of students in WRI 10 who receive library instruction will indicate through their portfolios that library instruction helped them successfully find information for their research papers.
B4 Findings:
Based on the analysis of portfolio cover letters from four sections of WRI 10, UC Merced librarians coded 20 of 30 students (66.7%) as explicitly referring to the library as assisting them in completing one or more Writing 10 assignments. The librarians coded another 9 of the 30 students (30.0%) as implying that that the library had been helpful in this regard.
B4 Actions/Next Steps:
Although the analysis of portfolio cover letters is time consuming, it appears to be a worthwhile measure for the library to pursue. Sharing this finding with WRI 10 instructors, and possibly WRI 10 students, may help raise awareness of the importance of library instruction in successfully completing research papers.
B4 Supplemental Data:
WRI 10 Cover Letters, Portfolios
Measure B5
Eighty percent of students who take the iPod Touch tour will indicate on the assessment that their knowledge of the library’s space, services, and resources is “good” or “excellent.”
B5 Findings:
In Fall Semester 2011 almost 87% of the participants indicated after the iPod Touch tour that their knowledge of the library’s space, services, and resources was “good” or “excellent.” In Spring Semester 2012, over 86% of the participants reported the same.
B5 Actions/Next Steps:
The library will continue to administer the tour and accompanying quiz and to assess the tour's effectiveness.
B5 Supplemental Data:
iPod Touch Tour Self Assessment
Measure B6
Seventy-five percent of the students who complete the iPod Touch tour follow-up assignment will receive scores of “high” or “perfect.”
B6 Findings:
In Fall Semester 2011 almost 90% of those who participated scored “high” or “perfect,” while in the Spring 2012 semester 78.3% scored high or perfect.
B6 Actions/Next Steps:
The library will continue to administer the tour and the accompanying quiz and to assess the results.
B6 Supplemental Data:
iPod Touch Scores
Measure B7
The library will provide information-literacy instruction for a group of undergraduates selected for a summer research program in the School of Natural Sciences. Students will show improvement between a pre- and post-test assessment.
B7 Findings:
During Summer 2012 the library provided information-literacy instruction to a small group of students from the Center of Excellence on Health Disparities (COEHD) undergraduate research program and conducted a pre- and post-assessment with these students. Based on a very small sample (8 students), the findings of a pre and post-test with COEHD students suggested that their library research skills increased with the library research session and resulting internship.
B7 Actions/Next Steps:
The library will continue seeking out opportunities to provide information-literacy instruction to groups of students involved in research and other academic activities that take place outside of the normal structure of graded courses.
B7 Supplemental Data:
Summer COE Results
Measure B8
Students in focus groups who received instruction from the library or used library guides in their course work will report that these were helpful.
B8 Findings:
Based on findings from the Spring 2012 focus groups, only 25% of the participants had used a library research guide, and of that 25%, only 15% found it useful. One respondent expressed a preference for going directly to the library’s website, while a second respondent reported that he did not find a library research guide for the topic he was researching. A third respondent noted that a library research guide was helpful in narrowing down what resources to explore.
Participants access library research guides in a variety of ways ranging from learning how to access guides in a workshop, to UCMCROPS, to a professor providing a link, to stumbling upon one. Overall, there was not much experience with library guides from the focus group participants. On the other hand, the statistics on the library’s Libguides show that many of them are heavily used. For example in 2012 the library’s Libguides had a total of 18053 hits; e.g. WRI 101: Writing in the Discipline, Psychology (Young) 1294 hits, Writing 10 (Walker) 590 hits, Writing 10 (Ruiz) 519 hits, CORE 001 – The World at Home 4498 hits.
B8 Actions/Next Steps:
The library will continue to create and maintain Libguides as the data shows a high rate of use. At the same time, the library will seek out creative, effective ways to bring Libguides to the attention of students.
B8 Supplemental Data:
Focus Groups 2012 Course Guides
Measure B9
Eighty percent of students who respond to UCCUES question on library research skill will report that their current ability level is “good,” “very good,” or “excellent.”
B9 Findings:
Of the UC Merced students who responded to UCUES 2010, 81% reported that their current library research skill proficiency was “good,” “very good,” or “excellent.” Only 43.9% of the same respondents reported that their library research skill ability was “good,” “very good,” or “excellent” when they started at UC. Only 1.2% of respondents reported a loss in proficiency, while 35.2% reported no change in proficiency. A total of 37.4% reported a step of improvement, while another 18.2% reported two steps of improvement. Eight percent reported improving three or more steps.
B9 Actions/Next Steps:
While the validity of self-reported proficiency in any area is questionable, the library will continue to participate in UCUES and look for any trends that could suggest a major change in proficiency.
B9 Supplemental Data:
Library Results UCCUES 2010
Measure B10
Seventy percent of students taking the graduating senior campus survey will “agree” or “strongly agree” with the statement “The UC Merced Library has supported my academic progress . . . .”
B10 Findings:
The library did not use this question in the 2011-2012 Graduating Senior Survey; however, in the 2011-2012 Graduating Senior Survey, the library asked a related, more specific, question regarding information resources provided by the UC Merced Library. In the 2011-2012 Graduating Senior Survey, 85% of the respondents “agreed” or “strongly agreed” that the information resources provided by the library supported their academic progress. Addtionally, the library has data relating to this question from the 2010-2011 Graduating Student Survey (administered in spring 2011); when we asked students if the library had supported their academic progress, 74.6% of the respondents "agreed" or "strongly agreed."
B10 Actions/Next Steps:
The library will continue to ask this type of question as part of the Graduating Student Survey and keep an eye on any trends revealed by student responses.
B10 Supplemental Data:
2011-2012 Graduating Senior Survey
Measure B11
The library will sponsor at least one Open Access Week activity. Eighty percent of those attending an Open Access Week activity will report that their knowledge of open-access publishing alternatives and of the problems facing traditional scholarly publishing have increased.
B11 Findings:
The library sponsored two Open Access Week events during the week of October 24-30, 2011.
- Thursday October 27, 2011, 10 - 11 am (KL362)
- Webinar: Alternative Metrics for Impact and the Future of Open Access
- Thursday October 27, 2011, 6 - 7 pm (Bobcat Lair)
- Film Screening: Copyright Criminals, a 53-minute documentary about the hip hop music sampling industry and the surrounding copyright issues.
The webinar was targeted for faculty and graduate students and was attended by three non-librarians as well as several librarians. Mendeley, a free reference manager and academic social network, was the focus of the webinar.
The film screening of Copyright Criminals was targeted at students and was attended by 34 non-library members of the campus community including students and staff. This documentary raises interesting questions about copyright in the digital age, and how society is grappling with these issues.
B11 Actions/Next Steps:
The library will continue to promote awareness of open access and, more generally, the economics of academic publishing. The library will continue to sponsor Open Access Week activities.
Measure B12
The library will ask questions about customer service and reference service on the UC Merced Graduating Student Survey. Eighty percent of students will indicate that they received good or excellent customer service. Eighty percent will indicate that their research questions were appropriately answered.
B12 Findings:
The library was not able to answer this measure in the way we had anticipated. The 2010-2011 graduating survey (administered in Spring Semester 2011) did not include specific questions regarding customer service and reference service. However when comments were coded from the 2010-2011 survey, respondents indicated that they valued staff/librarians as friendly and helpful. This was the fifth most valued aspect of the library.
B12 Actions/Next Steps:
The library will develop other measures to assess the quality of our customer service.
B12 Supplemental Data:
2011-2012 Graduating Senior Survey 2
Measure B13
Develop a means to automatically gather and report data on use of the UC Libraries Systemwide Tutorial by UC Merced students.
B13 Findings:
The library was unable to complete work on this project.
B13 Actions/Next Steps:
The library will complete this project and begin gathering data.
Goal C
The library will actively pursue innovative solutions and worthwhile collaborations that increase the usability and accessibility of information resources.
Outcome C1
Place the highest value on improving the usability and accessibility of information resources, recognizing that achieving this involves both collaboration (with vendors, knowledge institutions, other UC libraries, etc.) as well as work that can be completed locally (e.g. improvements to our web site).
Relevant measures: C1, C2, C3, C4, C5, C6, C7
Outcome C2
Focus on changes to resource access that improve the search and retrieval experience for the largest percentage of our users while offering them extremely relevant and comprehensive results.
Relevant measures: C3, C4, C5
Outcome C3
Develop or acquire solutions that allow users to independently discover and access information resources as part of their course work, research, and service.
Relevant measures: B5, B6 C1, C2, C3, C4, C5
Outcome C4
Provide, when necessary, personal assistance via a triage system that relies heavily on well-trained student employees and, as necessary, career staff.
Relevant measures: C6, C7
Measure C1
The per-capita use of 24/7 digital reference service by UC Merced students will be above the mean for all UC campuses.
C1 Findings:
Looking at all UC patron digital reference requests as a percentage of Fall Semester 2011 enrollment, Merced is 4.3% above the UC Systemwide average, the highest of all the UCs. Then next-highest campus is UC Irvine, at 3.3% above the average.
C1 Actions/Next Steps:
The library will continue to participate in the 24/7 digital reference services, to promote the use of this services to the campus community, and assess the effectiveness of this service.
C1 Supplemental Data:
UC Digital Reference Statistics
Measure C2
Eighty percent of UC Merced users who complete the 24/7 digital reference service feedback form will report that they will use the service again.
C2 Findings:
Eight-five percent of UC Merced affiliates who filled out the 24/7 digital reference feedback form selected "very likely" when asked if they would use the service again. Another 12% selected “maybe”.
C2 Actions/Next Steps:
The library will continue providing a high level of training to, and proficiency among, library staff who provide digital reference service.
C2 Supplemental Data:
UC Digital Reference Statistics
Measure C3
The library will analyze verbatim transcripts of 24/7 sessions initiated by UC Merced students. The goal is to use this data for continuous improvement of the library web site and user interfaces based on actual questions from library users.
C3 Findings:
The library did not do a formal analysis of local transcripts in 2011-2012. The last such analysis was done in 2010.
C3 Actions/Next Steps:
The library will attempt to do a formal analysis of local transcripts 2012-2013. Success will depend on the availability of staff resources to undertake such a project.
Measure C4
The library will conduct usability testing of the library website with new library student assistants looking for any patterns that indicate either success or failure on the part of information seekers. The goal is to use this data for continuous improvement of the web site and user interfaces based on actual use.
C4 Findings:
The library conducted website usability testing during Fall Semester 2011, recruiting student volunteers through Facebook and fliers on campus. Five participants (all new to UC Merced) took part in testing during the first two weeks of September.
C4 Actions/Next Steps:
Based on our findings from these usability testers, the following changes were made to the website. The library:
- created a new page with information on printing (instead of just linking to the campus IT page), and the link to this page was added to the Information for Students page (in addition to its original location under the Tech Help menu).
- added a link to the New Books & DVDs page under About the Library (in addition to its original location under On Display).
- updated its FAQs with new or improved information on requesting books through ILL and locating information on overdue items and library replacement fees.
- modified/enlarged the link to My Library Account to make it more visible.
- modified/enlarged the link to FAQs to make it more visible.
Measure C5
The library will analyze data gathered by the LibAnswers software to identify repeat questions or any pattern of problems that student encounter when searching for information or using the library. The goal is continuous improvement of the UC Merced Library web site, user interfaces, services, and instruction.
C5 Findings:
In August 2011 and June 2012, UC Merced librarians analyzed questions asked through LibAnswers looking for questions to add to the library’s FAQ (frequently asked questions) knowledge base tool. Librarians identified fourteen questions to add to the FAQ.
C5 Actions/Next Steps:
The library staff added the fourteen questions to the FAQ. Librarians will continue to analyze questions asked through LibAnswers looking for questions to add to the FAQ.
C5 Supplemental Data:
UCM Library FAQ
Measure C6
The library will gather a random-sample of data concerning the number and complexity of questions asked at the Library’s Help Desk and Service Desk. The library will analyze this data to insure that staffing levels are appropriate based on day of week and time of day.
C6 Findings:
Library staff collected and analyzed this data.
C6 Actions/Next Steps:
In general, the data confirmed that staffing of the library service desks is in line with customer demand. The library will continue to collect this data and adjust staffing as needed.
C6 Supplemental Data:
Desk Surveys 2011-2012
Measure C7
The library’s Roving Reference student employees collect data on the number of questions they are asked, the locations where the questions are asked, the time and day on which the questions are asked, and the complexity of the questions asked. The library analyzes this data to ensure that levels of Roving Reference staff are adequate.
C7 Findings:
From 1 July 2011 through 12 May 2012 (end of Spring Semester), Roving Reference students were on duty Mondays through Fridays for approximately 12 hours per week and answered 72 questions at the Consult Desk (79% of questions answered) or in the stacks (12% of questions answered). However, the Roving Reference manager suspects there was some under reporting by Roving Reference students.
The busiest times for questions were from 1-3pm, with 4-6pm being the next busiest time. This may be more of a reflection of when Roving Reference students were on duty rather than an accurate indication of what times have the greatest demand.
Roving Reference students coded 29% of questions as a “Piece of Cake” or “Pretty Easy,” 8% as “Medium,” and 3% as “Rather Difficult.” Seventy-three percent of the questions were answered in less than 5 minutes, 21% were answered in 6 to 15 minutes, and 6% took over 16 minutes to answer. This suggests that the Roving Reference students feel equipped to answer the bulk of questions they are asked. Forty percent of the questions answered by Roving Reference students relate to printing. The most commonly asked questions after printing were finding information on a topic (17%), locating materials in the stacks (8%), and finding known items (8%).
C7 Actions/Next Steps:
The library will continue to employ Roving Reference students and to collection data about the service they provide. The statistics on printing-related questions will be worth watching to see if technological and service improvements instituted by the campus units who manage printing will reduce the number of such questions. On the other hand, the proximity of the Consult Desk to the printers may work against such an outcome.
C7 Supplemental Data:
Roving Reference Statistics
Goal D
The library will provide relevant information resources in support of instruction and research in an environment of rapidly increasing numbers of users and growth of programs.
Outcome D1
Provide information resources–both owned and leased–that directly support the current research and instructional needs of the students, staff, and faculty of the university.
Relevant measures: D1, D2, D4
Outcome D2
Acquire information resources in any format as appropriate, though with a preference for resources in electronic format.
Relevant measures: D4, D5, D6
Outcome D3
Maintain current collection funding priorities:
1. Materials purchased or licensed at faculty request
2. All-campus licenses for e-journals, e-books and databases (Tier 1)
3. Selected less-than-all-campus licenses for e-journals, e-books and databases (Tier 2)
4. Locally licensed e-books available through patron-driven acquisition plans
5. Strategic investments in response to changes in research and publication practices (e.g. open-access publishing, digital preservation)
6. Purchase of print monographs through an approval plan
7. Coverage of areas of emerging interest that may be included in future research and instructional programs.
Relevant measures: D1, D2, D3, D4, D5
Outcome D4
Maintain commitment to, and investment in, non-ownership-based information-access strategies, such as:
1. Interlibrary loan
2. Supplemental course resources
3. UC Systemwide strategies (e.g. UC Shared Print in Place)
4. Regional/national/international strategies (e.g. WEST, HathiTrust)
5. Transformative strategies (e.g. open-access publication)
Relevant measures: D3, D7, D8
Measure D1
Seventy-five percent of students in focus groups will respond that they are "satisfied" or "very satisfied" with the library's information resources.
D1 Findings:
Seventy-five percent of the participants (n = 20) in the focus groups were “satisfied” or “very satisfied” with the library’s information resources. The break down of all focus group participants is:
Very satisfied: 35%
Satisfied: 40%
Somewhat satisfied: 20%
Not very satisfied: 0%
I have never used the resources: 5%
The follow-up comments by participants called out the ability to access and locate information more than the information resources available. The comments highlight the importance of providing transparent and consistent access to information resources.
The library also asked students in the Graduating Senior Survey 2011-2012 (administered during Spring Semester 2012) if the information resources provided by the library supported their academic progress. Eighty-five percent of the respondents (n = 290) reported that the information resource provided by the library supported their academic progress.
D1 Actions/Next Steps:
The library will continue to survey students regarding their satisfaction or dissatisfaction with the library’s information resources and to pay careful attention to their follow-up comments. The results of this focus group confirms that the library’s strong emphasis on making its information resources easily accessible is the right course.
Measure D2
In collaboration with the Academic Senate, the library will form a faculty advisory committee in order to get additional faculty input on information resources.
D2 Findings:
The Academic Senate Graduate Research Council appointed three faculty members to the Subcommittee on the Library and Information. The Interim University Librarian as well as various members of the Library Executive Team met with members of the Subcommittee on the Library and Information three time during Spring Semester 2012 to exchange ideas and information on topics ranging from interlibrary loan, to e-books, to information-literacy instruction, to the economics of academic publishing. During the course of the year the Interim University Librarian sent email messages to members of the Subcommittee on the Library and Information and the Academic Senate Chair to inform them about, and get their opinions on, matters relating to academic libraries and information writ large as well as about matters relating specifically to UC Merced Library.
D2 Actions/Next Steps:
The Interim University Librarian and members of the Library Executive Team will continue to meet with the Subcommittee on the Library and Information and to communicate with them as appropriate.
Measure D3
Participate in UC Systemwide Interlibrary Loan (ILL) Customer Satisfaction Survey to establish a baseline for future assessment.
D3 Findings:
The responses where generally positive. A number of respondents expressed a desire for longer and/or more consistent loan periods.
D3 Actions/Next Steps:
The library will continue to assess user satisfaction with ILL services. Loan periods are beyond the control of the lending library. The UC Libraries are working on standardizing loan periods among the UC Libraries. UC Merced has always been an advocate of long and standardized ILL loan period among the UC Libraries, but getting agreement among all ten campuses has been a slow process.
D3 Supplemental Data:
UCM ILL User Satisfaction Survey
Measure D4
In 2009-2010, the library filled 96% direct requests from faculty to order books or DVDs. (The failures (n=6) were DVDs unavailable from any source.) For 2011-2012 the library will achieve a similar rate of success in filling direct requests from faculty.
D4 Findings:
In 2011-2012 the library received 120 direct requests from faculty to purchase books or DVDs. Of these, 116 or 97% were filled. The four requests that could not be filled were for items that could not be acquired even by utilizing the services of specialized international vendors.
D4 Actions/Next Steps:
The library will work to maintain a fill rate above 95% for direct requests from faculty.
Measure D5
In 2009-2010, the UC Merced Library’s information resources budget ($1,174,706) represented 39% of the library’s total budget ($2,970,364). For 2011-2012, the library’s information resources budget will equal or exceed 39% of its total budget. In addition, UC Merced Library will be at or above the median for percentage of total budget expended on information resources as compared the other UC Libraries.
D5 Findings:
In 2011-2012, the UC Merced Library’s information resources budget ($1,365,700) represented 44% of the library’s total budget ($3, 103,885). The data for other UC Libraries was not reported in the UC annual library statistics.
D5 Actions and Next Steps:
The UC Library will continue to track information-resource spending as a percentage of its total budget and work to achieve a favorable ratio in this area.
Measure D6
In 2009-2010 the library’s technical services staff catalogued and processed 11,850 physical items and catalogued 50,000 electronic books and journals. For 2011-2012 technical services staff will sustain the same levels of throughput in cataloguing and processing as in 2009-2010.
D6 Findings:
Technical services staff succeeded in putting all physical materials on the shelf and loading records into WorldCat within five working days of receipt. The one exception was the period April-June 2012 when records could not be loaded due to WMS implementation. The resulting backlog has been eliminated and the service standard is once again being met.
D6 Actions and Next Steps:
After further consideration, the library decided this is not a meaningful measure as the number of items cataloged and processed is entirely dependent on factors beyond the control of technical services staff; notably, the library collection budget. The library will attempt to develop a better measure of the effectiveness of our technical services operations.
Measure D7
During the Fall 2010 and Spring 2011 semesters, the library’s Supplemental Course Resources (SCR) loaded 1274 documents to CROPS at the request of UCM faculty and instructors. For 2011-2012 SCR will load a similar number of documents to CROPS.
D7 Findings:
During 2011-2012, the library’s SCR service received 677 request for new documents and 490 requests for documents that had been used in previous semesters for a total of 1,167 documents. All of these requests were loaded into CROPS.
D7 Actions and Next Steps:
The library will endeavor to fulfill all SCR requests submitted during 2012-2013.
D7 Supplemental Data:
SCR Data Measure D7
Measure D8
Assess the amount of open-access publishing by UC Merced researchers in order to create a baseline against which to measure future growth or shrinkage of open-access publishing by UC Merced researchers.
D8 Findings:
Because of the widely distributed options for open-access publishing, identifying the number of UCM researchers publishing in open-access sources is difficult. The best the library was able to do was to look at the number of UCM researcher who have published in PLoS Biology (Public Library of Science, Biology) and BMC (BioMed Central).
UC Merced librarians were able to identify twenty-four UC Merced researchers who have published a total of thirty-four articles in PLoS Biology.
UC Merced librarians were able to identify twenty-four UC Merced researchers who have published a total of thirty articles in BMC.
D8 Actions and Next Steps:
The UC Merced Library will continue to assesses the number of open-access publications UCM researchers produce. The combined UC Libraries are currently considering a UC Systemwide subscription to an online tool that will allow the libraries to better track the publishing output of UC faculty, and the acquisition of tools of this sort will allow the library to do a better job of tracking open-access publication.
D8 Supplemental Data:
PLOS-BMC Publications.xlsx
Goal E
The library will maintain a positive work environment for all employees at all levels, providing regular opportunities for both individual and organizational development.
Outcome E1
Recognize staff recruitment, training, and development as key elements in the success of the library.
Relevant measures: E1, E2, E3
Outcome E2
Continue supporting appropriate training and professional development opportunities and funding staff participation in these opportunities.
Relevant measures: E1
Outcome E3
Codify and further develop the student employment "work-life cycle" to include all aspects of the student employee experience from the initial interview to the exit interview and beyond.
Relevant measures: E2, E3
Measure E1
A) UC Merced Library annual spending on staff training and professional development will be maintained at the same level as previous year
B) Annual total hours of participation in staff training and professional development activities will be maintained at least at the level of the previous fiscal year.
E1 Findings:
A) In FY 2010-11 the library spent $46,297 on staff training and professional development. In FY 2011-2012 the library spent $57,725 on staff development.
B) The library was unable to track the total hours of staff training and professional development.
E1 Actions/Next Steps:
The library will continue to provide the highest possible level of support for staff training and professional development. In times of tight budgets, spending on staff training and professional development is often one of the first victims. While it is inappropriate to be extravagant in the face of hard times, failing to invest in staff is a penny wise and pound foolish direction in which UC Merced Library will not go.
Measure E2
When surveyed about their satisfaction with the training and professional development opportunities they receive, at least 75% of library staff will report that they are “satisfied” or “highly satisfied.”
Findings:
The library did not conduct a survey of its staff regarding training and professional development opportunities.
Actions/Next Steps:
During 2012-2012, the library will conduct a survey of its staff regarding training and professional development opportunities.
Measure E3
For 2011-2012, 100% of student employees will receive the following minimum amount of training which will be tracked.
And,
A 90% attendance and completion rate will be considered successful.
E3 Findings:
The library was able to achieve all of the targets it set for Measure E3.
E3 Actions/Next Steps:
The library will continue to meet all of the targets it set for Measure E3.
Measure E4
For 2011-2012, the UC Merced Library will achieve a retention rates among student employees of 85% (excluding students who separate due to graduation, transfer, or by otherwise separating from the university).
E4 Findings:
The current retention rate among library student employees is 88.2%
E4 Actions/Next Step:
The library will endeavor to maintain a retention rate above 85%.
Goal F
The library will investigate digital curation, assess the data management needs of researchers at UC Merced, establish collaborative networks, and build expertise in order to define and develop library services.
Outcome F1
Define policies for archiving, delivering access to, and disseminating locally produced research data that are determined to be university assets or of value for future use.
Relevant measures: F1, F2, F3, F4
Outcome F2
Identify and evaluate service offerings available within the UC system (e.g. UC3) or external organizations that may address local needs.
Relevant measures: F1
Outcome F3
Communicate to campus stakeholders regarding identified data curation services such as EZID and pilot/assess usage.
Relevant measures: F4
Measure F1
The library will examine and compare collection policies and guidelines for research data from peer institutions in order to develop baselines for future assessment.
F1 Findings:
The library has developed a white paper for the Vice Chancellor for Research and Interim Provost that includes a summary of current initiatives and recommendations for data curation from leading research institutions across the country. One such report, issued by the University of North Carolina at Chapel Hill includes a comparison of research data ownership and management policies across a range of academic institutions: see "Research Data Stewardship at UNC" (February 15, 2012), Appendix 3.
F1 Actions/Next Steps:
The library will continue to compare its data management policies with those of other institutions and to ensure that its policies are aligned the with best and latest thinking on this subject.
F1 Supplemental Data:
The Responsible Stewardship of Research Data: A Roadmap for the University of California, Merced
Measure F2
Of the electronic theses and dissertations (ETDs) submitted by the Graduate Division, 100% of those submitted with signed author agreements will be archived and made accessible according to rights agreement terms.
F2 Findings:
In addition to working with the Graduate Division to establish guidelines and policies regarding ETDs, the library established the workflow and technical processes that would enable ETDs to be submitted to the CDL Merritt repository and published according to author terms in the eScholarship repository. The Library received 42 ETDS from the Graduate Division in 2011-12 and has archived 41 ETDs in the CDL Merritt repository. Thirty-three ETDS have been published and are publicly available through eScholarship. (One of the 33 was received by the library in 2010, before the workflow was developed.) The remaining documents are in queue at the CDL or dependent on expiration of their embargo period.
F2 Actions/Next Steps:
The library will continue to meet its 100% goal for archiving ETDs.
Measure F3
Analyze data from 2011 survey of faculty data curation needs in order to develop a baseline for future assessment.
F3 Findings:
The data has been analyzed and summarized in the white paper developed for the Vice Chancellor for Research. The survey findings were also presented to the Graduate Research Council of the Academic Senate.
F3 Actions/Next Steps:
The library is in the process of sharing the findings with the campus community and developing next steps based on the data.
F3 Supplemental Data:
The Responsible Stewardship of Research Data: A Roadmap for the University of California, Merced
Measure F4
Conduct follow-up with respondents from 2011 survey who requested more information on data curation services to evaluate current and propose initiatives.
F3 Actions/Next Steps:
Respondents have been added to a UC Merced data curation listserv and an introductory message was sent out. A digital curation librarian was hired and began employment in July 2012. One of the responsibilities of the librarian will be to conduct individual follow-up with those requesting more information on services.
Copyright @ The Regents of the University of California. All rights reserved.
