Evaluating the recommender

While our techies, Dave and Neeta, are working away to crunch the data we’ve pulled in from all of our partners, the other half of the Copac AD team are looking at the methods we’ll be using to evaluate the recommender.

We had some interesting user evaluations from the previous SALT project, where we spoke to postgraduates at The University of Manchester to find out if the recommender would support their research by surfacing valuable materials that might, otherwise, have been unknown to them.  There was overwhelming support for the concept from the groups we spoke to, so we’re interested to find out how other users will respond to the recommender.

This time, we’re planning focus groups with undergraduate students to find out if the recommendations we generate will help them to find course materials.  We think that students, already familiar with the concept from the likes of Amazon and Spotify, will welcome the Copac AD recommender so we’ll be showing the groups our prototype and asking what they think.

We’ll also be interviewing academics and teachers to find out if recommendations could support the development of course reading lists.  Some of the postgraduate researchers from the SALT focus groups were also graduate teaching assistants, and they told us that they could see how recommendations from the library catalogue could help their students to read more material and move beyond reading lists – we’d like to explore this more widely with academics from a range of institutions.

And finally, we’ll be conducting interviews with academic liaison librarians to ask if the recommender could support their work with collection development.  I was a liaison librarian in a previous life, and I can definitely see the potential of the tool – especially for gathering recommendations for stock purchases – but again we’ll be speaking to librarians from our project partner institutions to get their views.

We’ll be publishing our evaluation reports on this blog along with our testing instruments over the next couple of months.