Reading Lists: finding out what users really want

By Elizabeth Simpson and Kirsty Whitehead

In summer 2016 it was decided that the Library would look to upgrade its reading list system. The in-house system we had developed was at the end of its life and we wanted to switch to a product provided by a commercial provider. Members of the Academic Liaison team were on the project group and their primary responsibility was to engage our university community in the process of selecting the new system.

Gathering user requirements

We were fortunate to be able to pull feedback from a variety of sources. In addition to the knowledge that we already have through our work with academic departments, we were able to draw on existing feedback from our subject emails, CRM, and most importantly, the Understanding Academics project that had recently started. Rather than going back out and asking people to comment again, we started by synthesising the rich commentary we already had. The aim was to be able to clearly articulate what features people would like to see in the new reading list system. 

What did we do?

Initial work had already been done to code up the Understanding Academics interviews. During the summer of 2016 we looked at the 30+ pages of comments related to reading lists, and the first step we took was to analyse the comments in more detail and identify the main themes that emerged. We also made a note of our initial thoughts and ideas about how we could take this forward, and created a summary of them. Below is a snapshot of the document:

1.

Theme	Number of comments	Summary of comments	Take forward....
Alternatives to EARL	1	Currently using paperpile to generate lists
Clunky	20	Needs to be more user friendly for staff and student users. Issues raised: multiple screens, slowness, too many clicks, it’s ugly, difficulty moving items within lists and between lists (esp if scans are attached), only works properly with IE. rolling over could be smoother or automatic should be able to hide specific items for future use. difficulty finding items which are already in stock	Invite staff/student reps to the supplier demos. make it easier for staff to input the details of the item(s) they want to include (importing/bookmarklet) esp. for items on the catalogue. provide more flexibility for editing lists make it easier to find items already in the catalogue? improve rollover (automatic if possible)

However, this didn’t provide clear enough criteria that we could use further on in the project in terms of assessing potential systems. We developed a second document that really drilled down and listed each requirement separately, stating the needs and goals of our academic staff and students. Below is a snapshot of the document, in total there were 39 requirements listed. This was also useful to share with other colleagues in the project team who came from Collections and the Electronic Learning Development Team (ELDT) team.

2.

Id	Epic Category	As	I Want....(to do)	So That (Goal)...	Would Like to Have	Type	HL Acceptance Criteria
REQ-001	EPIC-001 Create a new reading list	Academic staff	Import references from a list I have already created e.g. word, paperpile, google doc, Endnote or other reference management software i.e. batch import.	I don't have to spend hours/days to create the list in a new piece of software. And it is easy to include things that I know we already have in stock.	Must Have	Interface	Must be able to import files from a variety of locations in a variety of formats (eg RIS). Must be able to identify resources which are in stock from imported list, and automatically create links to these on reading list.
REQ-002	EPIC-001 Create a new reading list	Academic staff	Import a list of references from our library catalogue e.g. list of starred items, rather than having to type everything into EARL individually and hope I've included the right information to find the item I want.	I don't have to spend hours/days to create the list in a new piece of software. And it is easy to include things that I know we already have in stock.	Must Have	Interface	Must be able to import files from a variety of locations in a variety of formats eg RIS. Must be able to identify resources which are in stock from imported list, and automatically create links to these on reading list.

How did this feed into the selection process? We decided that we needed to involve academic staff and also students in the process of selecting a reading list system. Using the feedback we had gathered via the process above and also from our conversations with other institutions, we devised 3 scenarios for the invited suppliers. These 3 scenarios covered all the points from the user requirements spreadsheet.
Five people from the project team were the official scorers but the staff and students who attended were also given a scorecard with which to score the scenarios, and prompts were provided. There was also space for them to write any other further comments. The feedback from the university community was to act as a benchmark should we all have wildly different scores. As it happened, we all scored the suppliers in a very similar manner.

"What does EARL* even mean?” This was one of the comments about our old system (called EARL) from one of the Understanding Academics interviews. We used switching to the new system as a chance to update the name and simply call the system Reading Lists. This has elicited lots of positive feedback from academic staff and students, who refer to lists as ‘reading lists’ and were often confused by our use of ‘resource’ lists. The new name reflects this user preference, regardless of the fact that lists contain much more than just reading.
What’s next? 18 months later, as part of reflecting on how successful the project has been, we’ve been able to return to the user requirements spreadsheet and evaluate the system that we have now. Having looked at each requirement individually, 91% of them have been met either partially, fully or, in some cases, requirements have been exceeded. The few requirements which haven’t been met relate to referencing styles, which is work in progress, and restrictions in the library catalogue system. After a really busy few months combining the start of term and the switch to the new system, being able to go back to the user requirements spreadsheet and use this to evaluate the work that’s been done has been invaluable, especially in helping us to plan our next steps.

Indeed, the work that we’re doing around this shouldn’t stop now that the system is fully up and running. We need to sustain a dialogue with users of the system in order to remain aware of their experiences of using Reading Lists: what they like and don’t like about it, and how this should shape proposed developments in the system.

For instance, in November we ran a focus group to which we invited staff and students from all departments. We felt it was timely to do this since we had reached the end of Reading Lists’ first term, and it seemed a good way to keep the conversation open with the academic community who had been so engaged with the process right from the start. The session allowed to us to gather some useful feedback about how people are using the system, and most of it was very positive, and it also gave us the chance to talk to people about the proposed changes that we want to make to Reading Lists. We will continue to run regular focus groups so that we can continue this conversation, and we are also continually collecting feedback that we receive during Boards of Studies in our departments, anecdotally from staff and students and via the training sessions that we provide. One of the great things about having a new system means that we can finally be more responsive to the concerns that our user community have and we are keen to ensure that we continue to actively engage with them.
*Easy Access to Resource Lists

“They retweeted me once - it was quite exciting!”

Using Twitter in Information Services

By Joanne Casey, Senior Content Producer

In September 2009, I was leading a small communications team for IT Services, and I'd had my own Twitter account for 9 months. As a team we were reviewing our approach to communication and saw that other university services were starting to use Twitter, so we decided to create an account for IT Services. We saw it as a new way to keep people informed of new services, alert them to any downtime, and raise awareness of security issues - phishing attacks and the like. We seethed inwardly when an occasional colleague gleefully dismissed Twitter to our customers, even as we tried to promote it (“I'd never use that”), and eventually we seethed outwardly too, which seemed to put a stop to it…

Managing Twitter

In 2011, the Library, Archives, and IT Services converged to become Information Services, and my communications team gained an extra member and became the Communications and Marketing team for all three services. The Library also had a Twitter feed, which had been operating since September 2010. At this point the Library had limited resource available for marketing activities, so the account was run by a group of interested people from different teams. This didn't always work well - with no defined responsibility for the account, some queries went unanswered, and the posting rates on the account varied hugely from day to day. As part of a review of relationship management activities, responsibility for the Library twitter feed was passed to us in the Communications and Marketing team, leaving us with two Twitter feeds to play with.

(As an aside, it's worth noting that this semi-outsourced style of working may not have been the best choice for our Twitter feed, but can be very effective in other contexts, so shouldn't be dismissed; our Instagram account is populated by eager photographers from different Library groups, and as such offers a broader picture - pun intended - of our resources.)

Fairly naturally, we fell into different voices for the two accounts; what we had to say on the IT account (downtimes, phishing alerts, new services) wasn't as easy to have fun with as the more varied topics on the Library account. Inevitably, we often retweeted from the 'other' account, and yes, we started talking to ourselves…

In the early days, we took a fairly concerted approach to building followers; we announced the Twitter feeds in newsletters, news items on our websites, and via the student union newsletter - the latter was especially successful. We added our Twitter name to printed materials, and included it in our email signatures, and colleagues were encouraged to do the same. Nowadays, people expect us to have Twitter so we don't have to work so hard at promoting it. Our focus is very much on user engagement, and we try to respond to all tweets, including negative ones; it's an opportunity to improve someone's experience, and turn around their perception of our service. When we engage with our users, whether it's with a well-chosen gif, a light-hearted reply, or supplying exactly the right information, we get more interaction which in turn helps to build our following. Any tweets with a feedback element are favourited and pulled into our monthly feedback report.

In 2014, the Library Twitter feed was nominated for an award from our Student Union, and were awarded Highly Commended in the Unsung Heroes of Non-Academic Staff category. Nominations were made by two of our student followers, who commented on the speed and humour of our responses, with one of them adding “they retweeted me once and it was quite exciting”. It was a very proud moment, with the added joy of champagne at the award ceremony.

Just an extension of the help desk? 

We were clear from the offset, that we wouldn't be the digital wing of the IT Support Office or the Library help Desk. Our Twitter bios explain that we offer 'news and updates'. However, while it's easy to set your own parameters and guidelines for how the Twitter account will be used, but not so easy to persuade your followers to work within them. And that's understandable - if my supermarket make an error with my delivery, then tweeting them is quick and easy, and the same is true for anyone with a question or comment for us. In addition, Twitter acts as an early warning system; this is easiest to see with the IT feed where users alerting us to issues with wifi can be as quick as our own alerts, but it's there on the Library account too, letting us gauge the thermostat, both actual (“It's too hot in Fairhurst”) and metaphorical (“Why are there so many A level students in the Library? We need seats!”).

Knowing its value

One way and another, we spend quite a bit of time working on our social media, and it's a key part of our communication approach, so it's important to evaluate its success. We use Twitter analytics, to assess engagements and impressions, and we use Google analytics to examine the traffic to our websites. But on a day to day basis, just getting a feel for it works pretty well too.

Learning from others

We didn't know when we set up these accounts how they would develop.Twitter is all about sharing and learning; we've learnt from those around us, and grown in response to our followers. We follow and retweet other accounts in the University and the city, and we also follow lots of library accounts - they're often funny and interesting (and we can steal their good ideas). Particular favourites are Liverpool Uni Library and Warwick Uni Library and, like everyone else, we have just a hint of a crush on the Orkney Library Twitter account.

The CRM at York: a technical overview

By Stephanie Jesper, Teaching & Learning Advisor

Last month, Michelle Blake wrote about our Customer Relationship Management system which we use in the Relationship Management Team at the University of York. She explained our reasons for creating our own system, and the benefits that system has given us. What she left to me to explain was the mechanics that lurk beneath its shiny surface. If you're the sort to quake at the sight of spreadsheets, you may wish to look away now.

It's fair to say that I'm obsessed with spreadsheets, so it was inevitable that my proffered solution to our CRM Question involved one. But it's not all spreadsheets. It is, though, heavily reliant on the Google suite of applications that we use here at York. The image below shows the fundamental components:

Our CRM system, in bits.

Let's take each component one-by-one.

CRM Form

The front end of the system is based upon an old-version Google Form. The problem with Google Forms is that they are very vertical, especially when you have a list of options. We wanted something a little more compact, so I basically copied the script from the Google Form and tinkered with it. This was a lot easier to do with the old-style forms than it is with the new-style forms, so if you want to do something like this, you might be better finding an old-style form that you can copy and modify. 

Filling in a CRM Form

The first modification I made was to run the options in columns, thereby saving considerable vertical space (you can imagine how long the page would have been with every option on its own line). There are also a considerable number of hidden questions that only display if certain options are selected. For instance, there's an option in the "Department" dropdown for "Multiple departments" that reveals a checkbox list of all departments. What's more, if you select a department for which we have an action plan, a checkbox list of actions is revealed so that progress on those actions can be logged. In total there are forty questions on that form, but only eleven are displayed by default. 

This toggled content is controlled using jQuery and style sheets. We also have some jQuery controlled autocompletion in the freetext "Other department" field. This searches against the default department list to mitigate against alternate formulations of existing department names.

The problem with hacking and modifying a Google Form is that you then need to host it somewhere. Initially we just put the CRM Form on a local drive, but now we have some password-protected webspace which means we can access it wherever we have a web connection. 

CRM Form (Responses)

When a CRM Form entry is submitted, it is fired off as the Google Form entry that it is, and gets added to a Google Sheet (as is the traditional output of a Google Form). The imaginatively titled CRM Form (Responses) spreadsheet is the heart, lungs, liver, stomach, and brain of the CRM (it's basically a giant spreadsheet-y haggis). It's a messy thing, reflecting the evolution of the system, and I hope to go in and simplify things at some point next year. 

The first thing that happens in this spreadsheet is that the data from the form gets tidied up using various arrayformulae. Comma-separated multiple-choice fields get pulled apart, mixed in with values from elsewhere, and are reconstituted as appropriate. The data then gets transformed into a layout that can be read by an Awesome Table (we'll come to that later). A second sheet counts up the number of matches for each form item to give a quantitative overview of our engagement. This is then broken down by department.

CRM Query

More in-depth analysis is carried out in our CRM Query spreadsheet which uses a Google Sheets query function to interrogate the data held in the CRM Form (Responses) spreadsheet. The query output can then be used to populate visualisations of our engagement like some of those we showed in the previous post. For example, here's some of our transactions by type:

Transaction types visualised via the CRM Query spreadsheet

CRM Summary

I mentioned Awesome Tables earlier, and this is where they come in: the CRM Summary is a user-restricted Google Site with an embedded Awesome Table from which we can see the transactions that have taken place. It looks something like this:

A couple of recent transactions in the CRM Summary

Like the query function in CRM Query, the Awesome Table lets us interrogate the CRM Form (Responses) spreadsheet as a database. But it does it in a way which looks a lot prettier and which can be used by more than one person at once. In the above example, I've limited to only those transactions with which I've had some involvement. There's even an edit button which allows me to revise any of the information I submitted.

CRM Alert

Going to a website or a spreadsheet to see what people have been doing is all very well, but it's a bit of a chore. If only we could set the spreadsheet up to notify us by email when something interesting happens... Well with Google Sheets we can. Each member of the team has their own CRM Alert spreadsheet which queries the CRM Form (Responses) sheet. Individuals can tailor the query to match search terms such as their department or area of interest. A Google Script keeps an eye on the spreadsheet and if a new entry gets returned by the query, an email gets fired off containing all the relevant details. 

Here's the code we use - if you're using a device with a pointer you can hover for a few bonus annotations:

function runAlert() {

  SpreadsheetApp.flush();

  var sheet = SpreadsheetApp.getActiveSheet();

  var from = sheet.getRange("Database Query!I1").getValue();

  var tot = sheet.getRange("Database Query!J1").getValue();

  var to = sheet.getRange("Database Query!J1");

  if(from!=tot) sendEmail(from); 

  if(from!=tot) to.setValue(from); 

}

function sendEmail(from) {

  var sheet = SpreadsheetApp.getActiveSheet();

  var line1 = sheet.getRange("Database Query!A"+from).getValue();

  var line2 = sheet.getRange("Database Query!B"+from).getValue();

  var line3 = sheet.getRange("Database Query!C"+from).getValue();

  var line4 = sheet.getRange("Database Query!D"+from).getValue();

  var line5 = sheet.getRange("Database Query!E"+from).getValue();

  var line6 = sheet.getRange("Database Query!F"+from).getValue();

  var line7 = sheet.getRange("Database Query!G"+from).getValue();

  var line8 = sheet.getRange("Database Query!H"+from).getValue();

  var line9 = sheet.getRange("Database Query!I"+from).getValue();

  

  var email = sheet.getRange("Database Query!B1").getValue()

  var subject = "CRM Alert";

  var body = "An entry matching your search terms has been added to the CRM:\n\nDATE:\n"+line1+"\n\nNAME:\n"+line2+"\n\nDEPT:\n"+line4+"\n\nCONTACTS:\n"+line5+"\n\nTRANSACTION:\n"+line6+"\n\nAIMS:\n"+line7+"\n\nCATEGORY:\n"+line8+"\n\nDETAILS:\n"+line9 ;

  

  MailApp.sendEmail(email,subject,body); 

}

The first function checks to see if there are any new transactions, and the second generates the email before firing it off. Obviously the cell references are rather tied to our specific setup, but the basics are there should you want to try something similar.

The result is an email that looks something like this:

An entry matching your search terms has been added to the CRM:

DATE:

Mon Dec 12 2016 00:00:00 GMT-0000 (GMT)

NAME:

Steph Jesper

DEPT:

External Relations

CONTACTS:

Father Christmas

TRANSACTION:

Other: Christmas List

AIMS:

CATEGORY:

Collections,Liaison and Relationships: General

DETAILS:
I sent Father Christmas my Christmas List. This year for Christmas I have asked for a shiny spreadsheet, a CRM system, and some gin.

If this transaction matched any of the search terms my colleagues have set up, they will be receiving an email now, just like this one. Perhaps they're writing a Christmas List right now too.

He knows when you are sleeping. He knows when you're awake. He knows if you've been bad or good, so be good for goodness sake...

Unlike Father Christmas, the CRM doesn't know any of this. It only knows what we put in it. And what we put in it may vary depending on the precise nature of our role. But hopefully what we get out of it is a good insight into what the rest of the team are up to and how it relates to our own activity.

Understanding Academics UX project

By Vanya Gallimore, Acting Head of Relationship Management

We wrote about our PGRUX project all the way back in July and mentioned that we hoped to post soon about our most recent project, Understanding Academics.

It’s rather an ambitious sounding title and indeed lots of academics laughed when we mentioned what we’d called it and wished us good luck! What we wanted to achieve in this project was to really gain a better understanding about how our academic colleagues approach their teaching and their research and where we, the library and the services we provide, fit in with that. It would also give us the opportunity to look for new services that we could potentially be offering and/or problems with our current services. We also hoped it would give us a greater understanding of how each academic department works within the University so we can start to provide tailored support for departments. Finally we hoped it would allow us to nuance our communications and think about how best we could talk to departments.

Background to the Project

The creation of our academic liaison team in 2014 strengthened our relationships between the Library and the departments but we wanted to build on that and go deeper into understanding the needs of the academic community and ensure that those needs were reflected into our service developments.

The work was intended to inform a variety of projects and pieces of work which are due to be undertaken over the next year in Information Services. This includes but is not limited to the following:

• Understanding textbooks
• Replacement for our inhouse reading list system and associated processes and procedures (Key text collection)
• York Pedagogy (and associated digital literacy teaching)
• Our lending system
• Student and staff inductions
• Communications to departments
• Engagement with those not based at our main campus, Campus West
• Buying habits of departments and individuals of resources
• Research Support services
• Understanding, managing and promoting our collections

We agreed, after much discussion and an earlier ambitious target, the aim of interviewing 4 members of staff in each academic department, ideally two focused on teaching and two on research (note: we didn’t mind if people had joint roles - rarely do people only do one of these activities - but we wanted them to concentrate on just one area for the purposes of the study as we had limited time with each individual and a lot to cover during that time).

We trialled the technique with three friendly academics to ensure that it worked, we had allowed enough time and to get their feedback on the process. These initial interviews worked really well and allowed the process to be refined before starting to undertake them across all departments.

Ethnographic methods

The project involved two UX techniques: a cognitive map, followed by a semi structured interview. While we had used love/break up letters in the PGRUX project we decided against using these for this project. We did find that some academics were very sceptical about the cognitive maps and didn’t fully understand what the point of it was but in nearly all cases they were pleasantly surprised by how useful they found it afterwards.

We did not offer any incentive for taking part and in some departments we had academics actively volunteering to be interviewed. I think this was due in part to how we pitched it about wanting to know more about them, rather than asking a series of questions about the library. The two questions we started with were very broad (can you draw a map of your research process or how you approach a new or existing module for teaching) to understand how the academics worked and where we, the Library, could see ourselves fitting into their process (were there missed opportunities, things we were doing they didn’t need/want etc).

The semi-structured interview always started with asking the academic to talk through their cognitive map with us and taking things from there. As much as possible we were led by the academic as we wanted to understand what was important to them.  We did have some prompt areas for the semi-structured interviews if the academics didn't mention them at all e.g. reading lists, student skills, resources, dissemination of their research that we would ask about if there was time. Our aim was to ask open questions and not lead anyone (this was one of the big challenges, particularly in the first couple of interviews) and to ask "why" a lot.

Staffing the project

In order to ensure that we could undertake the volume of interviews we were expecting to have (about 110 - each just over an hour long, plus time to write them up etc.) we needed a large enough pool of staff.

For the two previous UX projects we had used an intern but we were really keen this time that we used our own staff for this. There were a couple of reasons: our staff have the contextual knowledge about the library and a student often won’t understand the library context e.g. things like metrics would be very confusing to them - we had researchers talk about their "H index" for instance. I also felt strongly that this kind of activity would give staff a better understanding of their own academic departments (and indeed a couple of the liaison librarians had already undertaken similar conversations with some departments that they could then build on). Finally this is the kind of activity that we want to do more of and if we don't develop our staff and are always getting in others to do this, we don't build up expertise in this area.

We decided that our academic liaison librarians were best placed to undertake this work and in order to ensure that they had the necessary skills and confidence to do this, we put together a four week training programme for them. This consisted of:

• An overview of the UX techniques to be used and time to practice these on each other
• Neuro linguistic programming (NLP) training
• Business analysis tools (including techniques like the 5 whys)
• Active listening and questioning techniques

All except NLP were run by members of Information Services.

Due to the timing of the project we also had the opportunity to discuss the project during the Action Plan meetings with the Library Rep and Head of Department (HoD). This was really useful and we received a lot of support for the project at these meetings. HoDs were also useful in helping to identify individuals who would be good to take part, for example, new or relatively junior staff, research or teaching focused staff or someone they thought would be interesting for another reason.

Processing the interviews

We didn't have time (or money) to do full transcriptions of the interviews for this project so each interview was written up either in full notes or part transcription (where the academic talked through their cognitive map). This was then all coded into NVivo by a member of staff who and then produced reports on each "theme". These themes are now being analysed by a member of the team who will decide the best way to present the information e.g. the data about reading lists is helping to produce a list of requirements for a new reading list system but data around how researchers collaborate will help us to build personas about what researchers do/look like (or at least we hope it will).

One of the big reasons for undertaking this work in the first place was really to gain a better understanding about how academics work - both in their teaching and their research and to understand where we fit (or can fit) into that. We think librarians are often guilty of thinking we're the most important people and can't understand why someone hasn't got back to us. Understanding the other pressures that academics are often under is really useful (and helpful to articulate this to colleagues in other areas e.g. Collections) about why the deadline for reading lists is not working etc.

We now have a huge dataset and we need to continue to analyse each themed area. We will be updating the blog periodically as each area is reported on and we decide what to do with it over the coming year.

Library Action Plans Part 1: An Introduction

By Ned Potter, Academic Liaison Librarian 

The next post on Lib-Innovation will be by Michelle Blake and will go into detail on our Annual Action Plan process. This has proved an incredibly valuable way of engaging with, and getting engagement from, all the academic departments at York.

Ahead of Michelle's post, the presentation below gives a brief introduction to the process, its evolution over the last few years, and shows some examples from a 2015 Action Plan.

Lights, Camera, Action Plans! from Ned Potter