UC Merced Library Assessment

The library will provide quality information resources to support users’ teaching, learning, research and service.

Outcome 1: Provide information resources in any format as appropriate with a preference for resources in electronic form.

Relevant Strategic Goals for Outcome 1:

• Maintain quality collections to support academic programs.
• Maintain responsive, convenient discovery and access to information resources.
• Create partnerships with faculty to build collections supporting new and growing programs.

Outcome 2: Maintain commitment to, and investment in, non-ownership based information-access strategies, such as

• Interlibrary loan
• Supplemental course resources
• UC systemwide strategies (e.g. UC Shared Print in Place)
• Regional/national/international strategies (e.g. WEST, HathiTrust)
• Transformative strategies (e.g. open access publication)

Relevant Strategic Goals for Outcome 2:

• Maintain quality collections to support academic programs.
• Maintain responsive, convenient discovery and access to information resources.

Measure 1: UC Merced Library will be at or above the median for percentage of total budget expended on information resources as compared to the other UC libraries.

Result 1:

Median = 37%. 

 

Measure 2: Technical services staff will put all purchased physical materials (books, DVDs, etc.) on the shelf and load records into OCLC WorldCat within five working days of receipt.

Result 2: This measure was met. all purchased physical materials were on the shelf and records loaded into WorldCat within five working days of receipt.

Outcome 1: Digital Assets Projects

Dunya Ramicova Costume Design: Complete the digitization and archiving of over 2,000 renderings. 

Wilma McDaniel Collection: Complete digitization and adding selected digital content to Calisphere and Online Archive of California. 

Stories of the San Joaquin: Complete pilot project with Japanese-American Citizens League materials. 

UC Agriculture & Natural Resources Project: Hire project archivist and complete inititial work (dependent on final approval of funding). 

Proceedings of the Cognitive Science Society: Ingest proceedings into eScholarship. 

Digitization of the University Archives: Begin digitization of University Archives. 

Special Collections Material: Begin digitization of UC Merced's public-domain special collections materials that are not already available in digital format. 

Data Curation Videos: Complete pilot video and, depending on approval and commitment, complete the series which will be shared across the entire UC System. 

Identify a system to replace Filemaker for management of Supplemental Course Resources (SCR)  with the overall goal of revising and streamlining procedures for SCR. 

Initial investigation of collaboration among UC Merced, UC Riverside, and CSU San Bernardino to digitize water information resources. 

 

Outcome 2: Support a clear and streamlined workflow for the archiving and open-access publication of electronic theses and dissertations (ETDs).

Relevant Strategic Goals for Outcome 2:

• Maintain local and institutional digital assets (Clark Center collection, University Archives, ETDs, etc.).

Outcome 3: Continue to provide short-term digital access to supplemental materials for instruction.

Relevant Strategic Goals for Outcome 3:

• Maintain current digitization capabilities in support of instruction and research.

Outcome 4: Establish agreements and begin to gather materials on San Joaquin Valley.

Relevant Strategic Goals for Outcome 4:

• Create “Stories of the San Joaquin” digital museum.

Measure 1: 100% of ETDs approved and submitted by Graduate Division will have been accurately processed.

Result 1: 100% of ETDs approved and submitted by Graduate Division have been accurately processed.

 

Measure 2: 100% of SCR requests will have been processed in a timely manner.

Result 2: 100% of SCR requests have been processed in a timely manner.

 

Measure 3: Access to digitized collections, including Dunya Ramicova Costume Designs.

Result 3: 

• Completed digitization and have made available on Calisphere 1,094 designs from Dunya Ramicova collection
• Completed JACL pilot project
• 2000-2014 (volumes 22-36) of Proceedings of the Annual Meeting of the Cognitive Science Society are available in eScholarship repository
• Hired project archivist and begun pilot project for UC Cooperative Extension archives
• Completed pilot Data Curation Awareness video
• Identified system to replace Filemaker for SCR management
• Submitted proposal to NEH for planning grant to digitize water information resources

1.     Advance and support scholars’ ability to find, use, and manage information for their academic needs through offering intuitive digital pathways and in-person contact.
 
Outcome: Ensure library users can easily find library resources and services on the redesigned website.

Measure 1: Library users will be able to successfully complete 90% of the assigned tasks. (one-on-one talk aloud)

Result 1:

Members of the library’s web team conducted usability testing with seven students (undergraduate and graduate) in spring 2016.  Two of the seven participants were able to successfully complete 90% of the assigned tasks.  We did not reach our target with all participants.  However, we were able to determine areas for enhancement and improvement on the website.  The full report of the usability testing is available along with appendices that include the questions.  We sent all participants a copy of the report.  Since testing, the web team has completed the following:

• Created two video tutorials with instructions for using the Campus Reservation System (CRS) to reserve collaborative and meeting rooms
• Reorganized room information to more clearly highlight basic information
• Changed wording found in the Melvyl tabbed search box for consistency with other search options
• Modified Search box functionality to allow search terms to remain even when the tab option is changed
• Re-designed the Hours page to call out hours for specific library areas and services e.g. 155 Lantern
• Included Hours information on the home page for mobile users
• Changed some menu language for consistency
• Changed the default on database links so databases open in a new tab rather than overriding an existing page
• Changed hover actions to make text more easily readable
• Proceeding with slide-out menu to address challenges with drop-down menus particularly on mobile devices (in progress)

Outcome: Provide quality library instruction that supports students’ academic needs.

Measure 2: Over 80% of students will agree or strongly agree, through exit slips, that the resources in the session were relevant to their research and that they were confident in their ability to find and use library resources. (instruction exit slip data)

Result 2: 

For the majority of library instruction sessions, especially those offered in library instruction rooms, library instructors asked students to complete an exit slip. 

• In 2015-2016 (fall and spring semesters) 99.3% (1516/1527) of students strongly agreed or agreed that the resources described in the session were relevant to their assignment or research. And 97.6% (1473/1509) strongly agreed or agreed that they felt confident in their ability to find and use library resources as a result of the session. 
• An overwhelming percentage of students (99.0%, 1504/1519) strongly agreed or agreed that the information presented to them in library sessions was presented in a way they could understand. The remainder (00.5% 7/1519) strongly disagreed or disagreed while others (00.5% 8/1519) marked this as not applicable.

See full reports outlining exit slips findings in fall 2015 and spring 2016. 

Outcome: Support the growth of teaching capabilities in library staff.

Measure 3:  Library staff from access services will participate in providing workshops and library sessions for classes.  (instruction stats)

Result 3: In 2015-2016 we organized library instructor brown bag sessions to discuss literature around the topics of teaching and learning and connected that literature to our own library instruction practices.  Library staff from Access Services have participated and taken on teaching responsibilities.  In fall 2015 and spring 2015, three members of Access Services (Joe Ameen, Victoria Haindel, and Elizabeth Salmon) taught sessions in the library.  In fall 2015 they taught 13 of 78 (16.7%) sessions offered and in spring 2016 14 of 69 (20.3%) sessions.  They have been enthusiastic and valuable contributors to the library’s instruction team for course instructions, workshops, and internal training.

Measure 4:  Library staff will indicate their own growth in classroom teaching through written reflection and/or in conversation.   (brown bag discussion)

Result 4: N/A. We did not formally gather this information.

Measure 5:  Over 80% of students will agree or strongly agree, through exit slips, that their information presented to them was done so in a way they could understand.   (instruction exit slip data)

Result 5: See the response above for this data. 

2.     Increase faculty, student, and staff awareness of and engagement with library resources and services through creating and disseminating quality targeted communications (includes website).

• Did not measure in 2015-2016.  

3.     Lead in the development of information literate students through effective, sustainable instruction and collaborations with faculty and academic programs.
 
Outcome: Offer learning communities in conjunction with the Merritt Writing Program to integrate information literacy skills and knowledge into Writing 10 sections.

Measure 6: The majority of Merritt Writing Program (MWP) faculty participating in the learning community will note its value in the teaching and learning of students. (Learning community debrief)

Result 6:

All eight participants in the TRAIL learning communities in fall 2015 and spring 2016 expressed appreciation for the curriculum and observed improvements in student work.  Some additional information was gleaned from meetings notes and submitted written observations though we did not conduct a formal debrief survey in 2015-2016.

• Three of the four participants, from the fall 2015 learning community, discussed the benefits of the Teaching Research and Information Literacy (TRAIL) curriculum to their students in a debrief conversation on January 22, 2016.  In the discussion, they noted the benefits to students, challenges in implementation, and generated ideas for adapting and extending the curriculum.  All participants found value in the curriculum and observed that students were supporting arguments with evidence and were better prepared to use counter-evidence in writing than students they had taught in the past.   See additional notes for details.
• A second learning community with five participants implemented the TRAIL curriculum in spring 2016 Writing 10 classes.  Two individuals gave written feedback on the 1) quality of students’ arguments in their final papers and 2) use of sources.
• These instructors found that most students were able to made high quality arguments.  One instructor noted that those who were not willing to invest time into research were the ones who did not present a high quality argument.  There was value in the Problem/Solution essay to address argumentation; successful arguments included counter-arguments.
• When asked if students were using sources in appropriate ways, both instructors noted that students were integrating materials correctly and/or more students, than in previous semesters, had improved in their use of sources.  For one instructor, there were still areas where students needed to make improvements (e.g. signal phrases, paraphrasing vs. quoting, in-text citation mechanics).
• The spring learning community wanted to make the TRAIL curriculum more visible to their colleagues and suggested that a template syllabus for Writing 10 include TRAIL materials.  Based on feedback from the learning community and sharing of their TRAIL syllabi, a librarian drafted a syllabus with TRAIL materials included and obtained feedback from learning community members.  Merritt Writing Program (MWP) leadership adopted the updated template syllabus, with TRAIL materials referenced, and distributed it to MWP lecturers in August 2016.

Outcome: Students in TRAIL sections will demonstrate that they are thinking like researchers (evaluating sources, overcoming challenges, become more savvy/confident researchers, avoiding bias, asking questions, looking at multiple viewpoints, revisiting research).

Measure 7: The majority of students from TRAIL sections (sample) will demonstrate that they are thinking like researchers as assessed through their cover letters in the Research Ethics area of their portfolios. (WRI 10 cover letters from portfolios)

Result 7:

Librarians reviewed Research Ethics cover letters written by eleven Writing 10 students who participated in a spring 2016 Teaching with Research and Information Literacy (TRAIL) section.  All these students came from one section and the instructor received written permission to share their work with UC Merced librarians.  Three librarians double read each research ethics cover letter and scored reflections based on a single rubric criterion.  Librarians were looking at whether students were expressing the higher level thinking required of a beginning researcher.  Were these students thinking like researchers?  For example, did students identify an information need, ask questions, express curiosity, look for multiple viewpoints, support writing with evidence, or view research as a process.  Students could score between 4 and 0.  

4 Advanced
Illustrates a strong understanding of the researcher mind-set. Refers to a minimum of three higher-level thinking characteristics required of researchers.

3 Developing
Illustrates a solid understanding of the researcher mind-set. Refers to a minimum of two higher-level thinking characteristics required of researchers.

2 Emerging
Illustrates some understanding of the researcher mind-set. Refers to a minimum of one higher-level thinking characteristic required of researchers.

1 Marginal
Illustrates a limited understanding of the researcher mind-set.  May tangentially refer to higher-level thinking characteristics required of researchers.

Zero (does not meet criteria above)

Of the eleven students, 45.5% (5/11) scored at Advanced, 36.4% (4/11) scored at Developing, and 18.1% (2/11) scored at Emerging. Overall, most of the students 81.8% (9/11) scored at the upper two levels of the rubric.  We also took a closer look at some of the themes that emerged from their research ethics cover letters.  The majority of students (9/11, 81.8%) clearly wrote about research as a process rather than a one-time event.  They often reflected on the stages of research and noted that it took time, effort, and involved finding information to fill gaps as they worked on various portion of their papers. Students were often candid about the challenges they had during this process (8/11, 72.7%). Those challenges ranged from misunderstanding a prompt to working on unfamiliar writing assignments such as an annotated bibliography.  Other challenges included forming a topic, finding academic/scholarly articles, and locating credible information, the “right” information or “enough” information to sufficiently support a thesis.   Many students (8/11, 72.7%) discussed source selection and their need for unbiased, credible/reliable, informational, relevant, unique (not repetitive), and appropriate resources to answer their research questions.  This selection process involved analysis and many noted that they changed or added sources as they went through the research process. Over half of the students (6/11, 54.5%) indicated that their confidence levels around the research process grew during the course and the same number anticipated using what they learned in future classes or in professional life.  Students (5/11, 45.5%) also pointed to the value of assistance from librarians as they worked on their research projects.

4.     Contribute to the academic development and success of students (scholars) through providing informal and formal learning environments.
 
Outcome: Support student learning through providing Writing Center Pilot services in KL 260 and providing supervision for Instructional Assistants.

Measure 8: Students who use Writing Center Pilot services identify a minimum of two areas in which they received assistance and identify that they have received value from this service.  (survey to students using Writing Center Pilot services)

Result 8:

During the last two weeks of Writing Center services in spring 2016, instructional assistants who consulted in the center, asked their peers to complete a paper exit slip when they finished a consulting session. See Exit Slip questions. We received 83 exit slips.  It is possible that a single student could have completed more than one slip since we did not ask for any identifying information.  On the exit slip, students were asked to identify what they found most valuable about their consultation.  Students were most likely to value the opportunity to get more individualized feedback.  (Note: Though they were asked to select one answer, many students selected more than one. Due to this, results in Q1 add up to more than 100%.)

Students were also asked if the Writing Center provided them a unique service and why or why not. The majority (97.4%) of students responded that the Writing Center provided a unique serve that they were unable to find elsewhere on campus. 

Follow-up comments from students reiterated that they valued the opportunity for one-on-feedback and revealed that they valued the insider knowledge that their peer consultants brought.  In addition, peer writing consultants gave them a different perspective and helped them understand assignments, draft & develop ideas, transition between paragraphs, work on thesis statements, address grammar challenges, and build their confidence and motivation.  

 

Outcome 1:  Manage and utilize physical resources to benefit the UC Merced Library user community by developing physical resources to meet needs based on campus goals and priorities (areas of excellence).

Relevant Strategic Goals for Outcome 1:

• Maintain innovative instructional rooms.
• Maintain collaborative work rooms.
• Maintain meeting rooms.
• Maintain exhibition and event space
• Create 24/7 study space.

Outcome 2:  Enhance library spaces to provide productive spaces that support a variety of academic needs, including quiet study, group study, social interaction, and campus meetings and events.

Relevant Strategic Goals for Outcome 2:

• Maintain innovative instructional rooms.
• Maintain collaborative work rooms.
• Maintain meeting rooms.
• Maintain exhibition and event space
• Create 24/7 study space.
• Create library common spaces in new academic buildings.

 

Measure 1: Collect and analyze facility use and transaction data to enhance and improve facility use customer service.

Result 1: 

• Extended Friday hours from 6:00PM to 10:00PM based on room-use requests and headcount.
• Limited events in KL 355 to provide more study space for students.

 

Measure 2: Expend 10% of Library Operations budget on enhancements to public spaces.

Result 2: More than 10% of Library Operations budget was expended on enhancements to public spaces. 

 

These documents are also link in the narrative above for Research and Learning Services.