Reading Lists: finding out what users really want

By Elizabeth Simpson and Kirsty Whitehead

In summer 2016 it was decided that the Library would look to upgrade its reading list system. The in-house system we had developed was at the end of its life and we wanted to switch to a product provided by a commercial provider. Members of the Academic Liaison team were on the project group and their primary responsibility was to engage our university community in the process of selecting the new system.

Gathering user requirements

We were fortunate to be able to pull feedback from a variety of sources. In addition to the knowledge that we already have through our work with academic departments, we were able to draw on existing feedback from our subject emails, CRM, and most importantly, the Understanding Academics project that had recently started. Rather than going back out and asking people to comment again, we started by synthesising the rich commentary we already had. The aim was to be able to clearly articulate what features people would like to see in the new reading list system. 

What did we do?

Initial work had already been done to code up the Understanding Academics interviews. During the summer of 2016 we looked at the 30+ pages of comments related to reading lists, and the first step we took was to analyse the comments in more detail and identify the main themes that emerged. We also made a note of our initial thoughts and ideas about how we could take this forward, and created a summary of them. Below is a snapshot of the document:

1.

Theme	Number of comments	Summary of comments	Take forward....
Alternatives to EARL	1	Currently using paperpile to generate lists
Clunky	20	Needs to be more user friendly for staff and student users. Issues raised: multiple screens, slowness, too many clicks, it’s ugly, difficulty moving items within lists and between lists (esp if scans are attached), only works properly with IE. rolling over could be smoother or automatic should be able to hide specific items for future use. difficulty finding items which are already in stock	Invite staff/student reps to the supplier demos. make it easier for staff to input the details of the item(s) they want to include (importing/bookmarklet) esp. for items on the catalogue. provide more flexibility for editing lists make it easier to find items already in the catalogue? improve rollover (automatic if possible)

However, this didn’t provide clear enough criteria that we could use further on in the project in terms of assessing potential systems. We developed a second document that really drilled down and listed each requirement separately, stating the needs and goals of our academic staff and students. Below is a snapshot of the document, in total there were 39 requirements listed. This was also useful to share with other colleagues in the project team who came from Collections and the Electronic Learning Development Team (ELDT) team.

2.

Id	Epic Category	As	I Want....(to do)	So That (Goal)...	Would Like to Have	Type	HL Acceptance Criteria
REQ-001	EPIC-001 Create a new reading list	Academic staff	Import references from a list I have already created e.g. word, paperpile, google doc, Endnote or other reference management software i.e. batch import.	I don't have to spend hours/days to create the list in a new piece of software. And it is easy to include things that I know we already have in stock.	Must Have	Interface	Must be able to import files from a variety of locations in a variety of formats (eg RIS). Must be able to identify resources which are in stock from imported list, and automatically create links to these on reading list.
REQ-002	EPIC-001 Create a new reading list	Academic staff	Import a list of references from our library catalogue e.g. list of starred items, rather than having to type everything into EARL individually and hope I've included the right information to find the item I want.	I don't have to spend hours/days to create the list in a new piece of software. And it is easy to include things that I know we already have in stock.	Must Have	Interface	Must be able to import files from a variety of locations in a variety of formats eg RIS. Must be able to identify resources which are in stock from imported list, and automatically create links to these on reading list.

How did this feed into the selection process? We decided that we needed to involve academic staff and also students in the process of selecting a reading list system. Using the feedback we had gathered via the process above and also from our conversations with other institutions, we devised 3 scenarios for the invited suppliers. These 3 scenarios covered all the points from the user requirements spreadsheet.
Five people from the project team were the official scorers but the staff and students who attended were also given a scorecard with which to score the scenarios, and prompts were provided. There was also space for them to write any other further comments. The feedback from the university community was to act as a benchmark should we all have wildly different scores. As it happened, we all scored the suppliers in a very similar manner.

"What does EARL* even mean?” This was one of the comments about our old system (called EARL) from one of the Understanding Academics interviews. We used switching to the new system as a chance to update the name and simply call the system Reading Lists. This has elicited lots of positive feedback from academic staff and students, who refer to lists as ‘reading lists’ and were often confused by our use of ‘resource’ lists. The new name reflects this user preference, regardless of the fact that lists contain much more than just reading.
What’s next? 18 months later, as part of reflecting on how successful the project has been, we’ve been able to return to the user requirements spreadsheet and evaluate the system that we have now. Having looked at each requirement individually, 91% of them have been met either partially, fully or, in some cases, requirements have been exceeded. The few requirements which haven’t been met relate to referencing styles, which is work in progress, and restrictions in the library catalogue system. After a really busy few months combining the start of term and the switch to the new system, being able to go back to the user requirements spreadsheet and use this to evaluate the work that’s been done has been invaluable, especially in helping us to plan our next steps.

Indeed, the work that we’re doing around this shouldn’t stop now that the system is fully up and running. We need to sustain a dialogue with users of the system in order to remain aware of their experiences of using Reading Lists: what they like and don’t like about it, and how this should shape proposed developments in the system.

For instance, in November we ran a focus group to which we invited staff and students from all departments. We felt it was timely to do this since we had reached the end of Reading Lists’ first term, and it seemed a good way to keep the conversation open with the academic community who had been so engaged with the process right from the start. The session allowed to us to gather some useful feedback about how people are using the system, and most of it was very positive, and it also gave us the chance to talk to people about the proposed changes that we want to make to Reading Lists. We will continue to run regular focus groups so that we can continue this conversation, and we are also continually collecting feedback that we receive during Boards of Studies in our departments, anecdotally from staff and students and via the training sessions that we provide. One of the great things about having a new system means that we can finally be more responsive to the concerns that our user community have and we are keen to ensure that we continue to actively engage with them.
*Easy Access to Resource Lists

“They retweeted me once - it was quite exciting!”

Using Twitter in Information Services

By Joanne Casey, Senior Content Producer

In September 2009, I was leading a small communications team for IT Services, and I'd had my own Twitter account for 9 months. As a team we were reviewing our approach to communication and saw that other university services were starting to use Twitter, so we decided to create an account for IT Services. We saw it as a new way to keep people informed of new services, alert them to any downtime, and raise awareness of security issues - phishing attacks and the like. We seethed inwardly when an occasional colleague gleefully dismissed Twitter to our customers, even as we tried to promote it (“I'd never use that”), and eventually we seethed outwardly too, which seemed to put a stop to it…

Managing Twitter

In 2011, the Library, Archives, and IT Services converged to become Information Services, and my communications team gained an extra member and became the Communications and Marketing team for all three services. The Library also had a Twitter feed, which had been operating since September 2010. At this point the Library had limited resource available for marketing activities, so the account was run by a group of interested people from different teams. This didn't always work well - with no defined responsibility for the account, some queries went unanswered, and the posting rates on the account varied hugely from day to day. As part of a review of relationship management activities, responsibility for the Library twitter feed was passed to us in the Communications and Marketing team, leaving us with two Twitter feeds to play with.

(As an aside, it's worth noting that this semi-outsourced style of working may not have been the best choice for our Twitter feed, but can be very effective in other contexts, so shouldn't be dismissed; our Instagram account is populated by eager photographers from different Library groups, and as such offers a broader picture - pun intended - of our resources.)

Fairly naturally, we fell into different voices for the two accounts; what we had to say on the IT account (downtimes, phishing alerts, new services) wasn't as easy to have fun with as the more varied topics on the Library account. Inevitably, we often retweeted from the 'other' account, and yes, we started talking to ourselves…

In the early days, we took a fairly concerted approach to building followers; we announced the Twitter feeds in newsletters, news items on our websites, and via the student union newsletter - the latter was especially successful. We added our Twitter name to printed materials, and included it in our email signatures, and colleagues were encouraged to do the same. Nowadays, people expect us to have Twitter so we don't have to work so hard at promoting it. Our focus is very much on user engagement, and we try to respond to all tweets, including negative ones; it's an opportunity to improve someone's experience, and turn around their perception of our service. When we engage with our users, whether it's with a well-chosen gif, a light-hearted reply, or supplying exactly the right information, we get more interaction which in turn helps to build our following. Any tweets with a feedback element are favourited and pulled into our monthly feedback report.

In 2014, the Library Twitter feed was nominated for an award from our Student Union, and were awarded Highly Commended in the Unsung Heroes of Non-Academic Staff category. Nominations were made by two of our student followers, who commented on the speed and humour of our responses, with one of them adding “they retweeted me once and it was quite exciting”. It was a very proud moment, with the added joy of champagne at the award ceremony.

Just an extension of the help desk? 

We were clear from the offset, that we wouldn't be the digital wing of the IT Support Office or the Library help Desk. Our Twitter bios explain that we offer 'news and updates'. However, while it's easy to set your own parameters and guidelines for how the Twitter account will be used, but not so easy to persuade your followers to work within them. And that's understandable - if my supermarket make an error with my delivery, then tweeting them is quick and easy, and the same is true for anyone with a question or comment for us. In addition, Twitter acts as an early warning system; this is easiest to see with the IT feed where users alerting us to issues with wifi can be as quick as our own alerts, but it's there on the Library account too, letting us gauge the thermostat, both actual (“It's too hot in Fairhurst”) and metaphorical (“Why are there so many A level students in the Library? We need seats!”).

Knowing its value

One way and another, we spend quite a bit of time working on our social media, and it's a key part of our communication approach, so it's important to evaluate its success. We use Twitter analytics, to assess engagements and impressions, and we use Google analytics to examine the traffic to our websites. But on a day to day basis, just getting a feel for it works pretty well too.

Learning from others

We didn't know when we set up these accounts how they would develop.Twitter is all about sharing and learning; we've learnt from those around us, and grown in response to our followers. We follow and retweet other accounts in the University and the city, and we also follow lots of library accounts - they're often funny and interesting (and we can steal their good ideas). Particular favourites are Liverpool Uni Library and Warwick Uni Library and, like everyone else, we have just a hint of a crush on the Orkney Library Twitter account.

UX-led changes at York and beyond

Tweet 

By Ned Potter, Academic Liaison Librarian 

As anyone who has embarked upon User Experience work will have learned, ethnography is actually the easy part. For all its messy, complicated, time-consuming complexity, getting the go-ahead for fieldwork and undertaking it is relatively straightforward compared to designing (and getting approval to put into place) changes to our services. It is vital to have a cut off point where we as UX practitioners stop collecting data, bite the bullet, and move on to phase 2 of the process. After all, it's the design and service tweaks that make this UX - otherwise all we're really doing is ethnography.

I think it's really important to a) push as many small tweaks through as possible, and then learn from them and assess their impact, and b) make details of the changes publicly available so others can get not just inspiration but a use-case to push through their own change.

Last month I talked to an audience from the libraries, museums and archives sectors at the LAM Marketing Awards put on by the Welsh Government. The image above is from the presentation, which was on UX, how it relates to marketing, and potential application for ethnographic techniques outside the library sector. If you're interested I wrote more about the day on my own blog here, and I'll embed the presentation below, but in this post I want to focus on UX-led changes, which was one section of my talk. What have institutions been doing as a result of what they've learned from ethnography? I have several examples from the University of York and some from further afield too.

I thought it might be useful to group the examples of UX-led improvements into categories. In all these instances ethnographic fieldwork has either instigated the change or supported the change - it's interesting that often UX can be the trigger to get something done which library staff and users have been considering and / or suggesting for a while. Often the fieldwork is one source of feedback alongside a couple of others in the examples below, which combined to be a strong enough argument to make a change.

Catalogue improvements

At York we've made several small changes to Yorsearch, the (Primo-based) library catalogue, in addition to the full user-interface change which will arrive shortly. 

Classmarks visible in results screen 

• The classmark for books now appear in the search results screen, rather than the user needing to click on a title to reveal its location. It's only a small change but we get around 25,000 views a day for Yorsearch - that's a lot of people now having to make one less click to get what they need. This particular change came from our first UX project with Postgrads, along with work from the Discoverability Group, and from seeing that that Imperial had successfully achieved the same thing with their Primo interface already, following their own UX work...
• Talking of Imperial, they've made the full report of their 2016 UX work available for anyone to download [*applauds*] - have a read, it's fascinating and useful material.
• We changed the terminology in the catalogue on the buttons you press to access books and ebooks - from Get It and View It, to Find in Library and View Online. Again this came out of several sources of feedback, including the Discoverability Group, and front-desk staff reporting that users simply didn't seem to get it when it came to View It and Get It.

Library space and environment improvements

• We made a hot-water tap available 24/7. Our UX work revealed that particularly in winter  students from Asia like to drink hot water in the way that in the West a lot us like to drink chilled water; this gave more context to previous requests for a hot drinking water tap. One has now been installed alongside the chilled water fountain.
• We made the Burton Library accessible 24hrs a day. Our library is open 24 hours, but previously only the main Morrell Building (the one with the books) and the Fairhurst (lots of study space) stayed open all the time; the silent reading room in the Burton closed at 10pm. Our UX work constantly demonstrated that the Burton was not as highly valued as we imagined it was - for example several students left it out of their congnitive map of the building, almost no students included it in their touchstone tours, and in our behavioural mapping we even observed students wandering up to the entrance, peering in to the stairwell that led up to the reading room, then just turning around and coming back, apparently not feeling like they wanted to cross the threshold. As part of the UX unstructured interviews we discovered that even some students who knew about the Burton didn't like using it because even if they had no intention of working past 10pm, they loathed the idea of setting up all their work and devices etc and then having to move them to another building at 10pm if they were still there at that time after all.
  
  So we upped our promotion of the Burton, it had a very nice re-design (although that wasn't directly related to anything we'd done with UX, it was happening anyway) and we made it accessible 24 hours a day. We're now monitoring the space as part of a new UX Project and the initial impressions are that it's already busier.
• We've given the students blankets. A pile of blankets in a basket near the entrance - people can help themselves and deposit the blankets back there when they leave. I cannot tell you how popular this has been... There are examples of effusive tweets and feedback on our graffiti wall in the presentation below - it's so nice to do something simple but effective! Temperature is always a problem in libraries, and there's often a more or less even split between people who are too hot and too cold. My History of Art students came to me to say they found working in King's Manor (our City-centre site which is nearly 500 years old so not overly warm) really hard when it was so cold. So we managed to get Estates to get some more heaters, and we bought blankets - this idea came from some UX work undertaken at Cambridge in 2015. We also bought blankets for our main library and the Minster library too.
  
  (Top tip: buy really drab and unexciting looking blankets. They keep people just as warm but are much less likely to go missing...)
• Thanks to Ingela Wahlgren and Andy Priestner who gave me examples of their (current or former) institutions having changed the locations of digital screens as a result of behavioural mapping, in order to put the screens somewhere people will actually look at them. This could be displaying key info in areas where people have to queue, or it could be as simple as putting them in the direct line of site as students move forward through a space, rather than off to the side in people's peripheral vision.
• Sometimes students describe an area as noisy even though it's ostensibly a silent study zone. Truly observing the space can often solve the mystery of why this is happening - Jenny Foster gave me an example of her institution realising the beep of the self-issue machine could be heard four floors up! So they found the volume and turned it down. At Cambridge they discovered there were loud hinges on office doors so they oiled them...
• Like with the noise examples above, small changes really do add up. Carl Barrow told me some of the changes his HEI had made based on their fieldwork: additional signage (both analogue and digital), more printers, phone charge stations (why aren't we all doing these?) and a new coffee cart. Together all those minor tweaks will have a significant impact on the user experience, which is the name of the game after all.
• UPDATE: At the #NCLXUX event I've just heard Carl say they also re-positioned digital screens, having noticed no one looked at most of them. One, which was positioned in the entrance as people came through the turnstiles, DID get looked at - so they used that exclusively to promote the Skills Team's workshops, and as a result saw a much bigger uptake for those sessions... I love this - a great example of the impact UX can have in unexpected ways...

Library service improvements 

• At York we've moved academic staff onto our part-time package for borrowing books, giving them a little longer to return items without impacting too heavily on the rate of circulation overall
• We've changed the way we run our annual review of subscriptions to allow for more time and stop it clashing with other key things in the academic calendar
• We've changed the way we communicate key information to academics
• We've used academics' detailed views on our current reading list system to inform the choosing and customisation of the new one
• At Cambridge the FutureLib developed a whole app for finding study space

UX and Impact

I'm excited to hear a load more examples of UX-led change at UXLibs III (the third annual User Experience in Libraries Conference). The paper submissions we've had this year are fantastic, and the emphasis of the conference this year is on the impact of UX.

Finally, here are the slides from my talk which mention a lot of the examples above, along with some next steps if you want to try ethnography at your own institution, and introductions to ethnography and design:

UX, ethnography and possibilities: for Libraries, Museums and Archives from Ned Potter

If you've got more examples of UX-led changes at your organisation, leave them in a comment (or leave a link to where we can read about them) to make this post more useful! Thanks.

PGR UX outcomes and reflections

By Michelle Blake, Head of Relationship Management

This blog has already covered some of the original work we did looking at PG use of the Library when we had Emma do an internship with us. We followed this work up with another intern, Oliver, in January this year.

This post will discuss the results from that work and what we’re now doing - as well as what we learnt from the process more generally.

Some of what we learnt came as no surprise but there were areas I don’t think we could have predicated.

Sense of community and space

PGR students really valued the sense of community that they get in their departments (but not all students have dedicated space in their department) and didn’t want to miss out on the water-cooler type conversations that can take place. They like a variety of spaces to work in with no one size fits all. Being able to choose the type of space they wanted depending on how they were feeling at the time was really important to them, but they did note they often wanted to be close to those who were working on similar things to have quick, informal chats.

Some of the study spaces in the Library at York

Just in time access to help/support

PGRs told us that they only wanted to know about services, training etc when they needed them but often they weren’t then sure where to find them. It was also clear from the interviews that many students were unaware of some of the services that are on offer to them. One of the things they did tell us was some of our services weren’t working for them, interlibrary loans was one of these where the way the students are working mean that the delay in waiting for resources to arrive was seen as being too long and hindering their work. This was particularly the case for Arts and Humanities students.

PGR journey

Our research found that regardless of which faculty a student was from all PGR students use the Library in the first year of their PhD, particularly in order to undertake their literature review. However, use of the Library alters after this point dependent on their faculty, with Arts and Humanities students being much heavier users of the Library for the duration of their PhD while Social Sciences and Sciences, in particular, start to use it less as they conduct experiments and field work. While this isn't ground breaking I think highlighting how the Library is used during the course of their study was really interesting.

We completed this study nearly four months ago so what’s happened? Well I’ve broken down what we’ve done or are doing into the following four categories - confirmed findings from the original work which Emma did, simple things, medium term changes and longer term things.

Confirmed findings from our original study

• The need to display the classmark of the book in the catalogue search results screen, which we'd been looking to do for a while and have now done

An example of the classmark of the book appearing in the results screen, rather than the user needing to click on the specific item to find where it's located 

• The need to have a hot water tap for food and drink prep which we were looking at and have now done
• The Burton (part of the Library building) being under-explored by PGs at peak times so needing more promotion which we already knew about but have now done
• It discovered part of the reason people were reluctant to study in the Burton was that it closed at 10pm so you knew you'd have to leave eventually; it's now open 24hrs as part of a redevelopment of the space (which added in additional seating and made the whole Reading Room space look a lot nicer too)

Part of the redeveloped Burton Library 

Simple things

• We’re adding white boards to the PG lounge in the Fairhurst (they should be arriving soon)
• We’ve found a more appropriate space for PGs in the silent area of the Library (it’s a room) that we hope will help them to create the kind of space they want. We realise we can’t manufacture this sense of community for them but hopefully by providing them with a nice space this will help them to shape it to what they want it to be
• We’re talking more to the students themselves about what they want. We’ve always had a good relationship with the GSA (Graduate Student’s Association) but this is also about talking to students on the ground as well
• All new PGR students in 2016/17 will be sent an individual welcome email from their Academic Liaison Librarian with option to come and see them (hopefully thereby making them more aware of everything they can access)

Medium term

• Reviewing interlibrary loans (ILLs), how they’re used and really understanding why they aren’t working for our Arts & Humanities students in particular. To do this we’ll be looking at the data we already have about ILL use across departments as well as reviewing data from a GSA intern study. We hope to be able to be able to understand more about how the students are really working and then ensure our services fit these needs
• We are establishing PG Wednesdays with the Graduate Research School and other support services. These are afternoons dedicated to PGs which will involve a series of training, talks, and continue the Shut up and Writes events we organised last term (we hope to have a blog post separately about this once they’ve been established)
• We want to review the rest of the PG space in Fairhurst once the white boards are in place. It may be that after installing these nothing else needs to change or that we want to make further changes in this area
• We are organising supervisor training with our Graduate Research school so that supervisors know what we offer and can signpost their students appropriately
• We will be reviewing all the training we offer - is it the right stuff at the right time

Longer term

• We are looking at how we might re-purpose some of our space in the longer term - areas that we know don’t work and which might result in another UX project addressing space more generally (hopefully more on this later this year or early next year)
• We have begun some work on a digital humanities project to better understand needs of humanities researchers, this may well take into account some of the findings from our Arts & Humanities PGRs

Conclusions

So finally I wanted to finish on what we’ve learnt from the project and what we might do differently next time.

• While having an intern is great, one of the things we came to realise is that it can be useful to have someone who has more intimate knowledge about your services or someone with a library background. It was this thinking that led us to the approach we’re taking in our Understanding Academics project (we’ll blog about this project separately soon)
• You really need to do things quickly - I don’t think we did them quickly enough with this project but it’s been a good learning curve
• It’s really important to agree timescales and ownership and be clear what the outcomes are from any work you do
• People will “challenge” relatively small sample sizes. You need to consider how will you back up your findings e.g. what other data do you have that help to triangulate your findings
• Getting people to take part was hard work - we tried a lot of different approaches with this project and none of them were very successful. This has been in stark contrast to our Understanding Academics project where attracting participants has been much easier
• Originally we had wanted to attract students from particular departments who had lower satisfaction rates in the PRES (PostGraduate Research Experience Survey) but were unsuccessful in doing this. In total we had 11 participants for the PGRUX project and while this might sound like a small number the data that we got from these interviews was incredibly rich.

So has it all been worth it? Absolutely, without a doubt the PGR UX project has really given us a solid foundation on which to continue to build our UX work. I hope to post on here soon about our latest project, Understanding Academics.

A second UX Intern writes... Oliver Ramirez on User Experience

This is another guest post in the Embedding Ethnography series, this time written by Oliver Ramirez. He completed some hugely detailed work for us at the start of 2016, on what we called the PGRUX Project, and these are his reflections on the whole process.

Oliver is now based in London and enjoyed UX work so much he wants to pursue it further, perhaps with another internship - if you're reading this and you are potentially interested in having him come and work with you, send me an email and I'll put you in touch!

---

My UX research internship: More than ticking boxes | by Oliver Ramirez

I was interested in seeing how my user experience knowledge from my Computer Science degree could apply to physical services. So, when I was offered the chance to conduct a UX research project for the library team at the University of York, I took my chance to see how library UX works.

The motivations behind the project were straightforward; among postgraduate research students (PGRs), there was a large disparity in satisfaction scores when it came to certain areas of research life. In conducting the project, I was to identify the reasons for this.

For this, the UX research approach made sense. I was to report on how PGRs conducted research, in particular their habits and reasons behind study space choices and resource choices, their interactions with others during research, and their use of study services. Through this, I would identify the pain points for less happy demographics, try to understand what worked for happier demographics, and suggest ways to try and bridge the gaps between those experiences.

As part of this research, I ran 1 on 1 sessions with PGRs across different departments. The team also gave me a couple of UX techniques to test out, so as well as a “non-directed interview”, I ran Cognitive Mapping and Love & Break Up Letter exercises with each participant.

Cognitive Mapping

The first exercise that I ran with participants was cognitive mapping. Participants were asked to draw a “map” of the things that they interacted with while conducting research (limited to the areas I outlined earlier), mostly adopting the methodology outlined by Donna Lanclos. I say “map” because, in reality, few participants drew anything resembling a conventional map – participants mostly favoured writing down concepts (for example, ‘their office’) and linking those to other related concepts (‘their office’ may be near their ‘supervisor’).

It was a great way of getting the participants to think about the areas I would end up interviewing them on, and the temporal and relational information captured in the map made it easier to pick up on each participant’s thoughts. One good example of this is how a participant placed importance on their desk: they drew it as their map’s central element early on, and branched everything off of that central element. This was reflected in their interview, where they emphasised the importance of that desk to them.

A Cognitive Map from the PGRUX Project

“Non-directed interviews”

Using the participant’s cognitive map as a ‘guide’, I would then conduct a non-directed interview. This involved taking an almost passive, neutral stance in everything I asked about, primarily allowing the concepts brought up on the participant’s map to direct the conversation – then, after those points had been exhausted, I would consult my own discussion guide to cover the rest of the areas of interest.

Conducting the interview in this way was initially difficult for me – it was sometimes hard to probe without being ‘aggressive’ (asking weighted questions or changing the topic), and I sometimes struggled to facilitate the conversation without suggesting topics to talk about.

There was immense value from conducting the session in this way, however. By focusing the interview on the topics participants brought up, gathered information more closely reflected the participant’s “perspective” – their habits, their opinions and their choices, mostly on what they were aware of in the discussed areas. Gathering the information in this way allowed for me to more effectively deliver insight on issues of awareness.

Love/Break Up Letters

Finally, participants were asked to write a ‘love’ or ‘break up’ letter. By asking participants to address this letter to a personified IT or library service, we hoped to draw out the emotions of participants towards those services, and easily establish positive/pain points.

My participants seemed to be very polarised by the exercise; people either really got into it, or they really didn’t. Upon reflection, the abstract nature of the exercise may have made some participants uncomfortable, especially knowing that their letters would be scrutinised. However, while I feel that while this technique didn’t work in a 1 on 1 session, there is merit to trying it out in a pop-up-desk context, or a ‘prize raffle’ format – this would allow for many responses, and for the easy identification of pain points across services.

All in all, I felt that the techniques allowed me to attain some real insight into PGRs, and despite the initial nerves, I really enjoyed conducting these sessions with participants. But while I’m singing the praises of these techniques now, back before I started my internship, my mentality for designing around users (or stakeholders in general) was one of appeasement – design a website that does what stakeholders need it to do, and fix any issues preventing its smooth use. A real ‘checklist’ oriented approach. Historically, I had followed this approach in my degree through a type of observation called ‘usability testing’, where I noted any issues users had when doing tasks that I had set.

So, heading into this internship, I had expected to do just that: more observation, make a list of issues to fix, and suggest some solutions - tick those boxes off, one by one, on the way to a “good UX”. But, throughout my internship, I realised this approach just yields a ‘passable’ user experience – you end up with something that works, but not necessarily something that’s good.

Example of Findings: Lonely Researchers

For example, one of my participants told me something that really struck me: they said that when they were based at a general desk, that they felt disconnected from their department. It was always possible to contact or visit their supervisors, or use the department testing rooms, or go out of their way to interact with their peers, but not being based alongside all of that meant that they felt ‘distant’ with their department. This changed when they were offered a desk inside their department. Besides improvements on all of those fronts, they reported feeling ‘valued’ as a member of the university because of it.

The importance of ‘department community’ - being alongside your researcher peers and supervisor so that interaction is readily possible – was prominent in my discussion with some participants. During my research, I found that while non-department PGR study spaces covered various noise levels (something participants valued), those spaces did not facilitate this kind of ‘natural interaction’ that only happens when PGRs and supervisors are all based together – and so PGRs based outside their department missed out on this.

My approach of “observation to find issues, fix issues” would not have yielded this type of insight – I would have thought along the lines of “they don’t really like the silence in this building”, suggested to change the noise level policy and called it a day. It wouldn’t have made much headway in creating a better UX for the people based outside their departments.

But, it finally dawned on me during my time with the library team at York: good UX necessitates understanding what your user values, what is important to them, and actively working with that in mind. Which worked out for me, in the end: UX is a more satisfying when it isn’t just making something that works and ticking boxes.

Planning and delivering an Intern-led UX Library Project

By Ned Potter, Academic Liaison Librarian 

Tweet

Emma talking to one of our Liaison Librarians

Last time out, as part of the Embedding Ethnography series, Emma Gray wrote about what it was like to be a UX Intern here at York, and the techniques she employed while she worked with us. 

If it's in any way possible to get an intern to help out with your ethnographic project I'd highly recommend it, so this post is about our process for setting the project up and working with Emma.

(For an overview of UX, ethnography and design, please see our What is UX? page.)

Recruiting an intern

This was the part of the whole project we struggled most with, and were fortunate in how it worked out. 

Five of our staff went to the first UXLibs Conference in 2015, and came back wanting to immediately implement some of the things we'd learned. But we all had not nearly enough day-to-day time in our roles to do any serious amount of ethnographic observation and interaction. So I submitted a proposal to a University-wide Intern scheme - but despite making it attractive as a I could, all the applicants chose to go for other Internships on offer from the University. If anyone has any tips on writing a great UX Intern job spec and advert, I'd love to hear them in a comment below... 

We then got an email from the Head of HR in the Library saying a student at Durham University who lived locally wanted to work for the library over the summer, and did anyone have any suitable work? Naturally I jumped at this and sent Emma the existing job spec, she agreed it looked interesting, and she came in for an interview.

It was a very informal interview, just me and my manager and Emma without a huge list of pre-prepared questions. Emma didn't have any UX knowledge prior to coming in, but that didn't matter. As it happened she did have experience of working in a public library but that wasn't essential either. For us, the essential qualities were to show some initiative (Emma ticked this box, having found my website and read my reviews of the UXLibs conference...) and above all to be a good communicator. UX work involves a LOT of dialogue with users, so if that isn't something you enjoy it's going to be a slog... Emma was naturally communicatory so we had no doubts about offering her the role.

Pre-arrival set up

As Emma's manager I set about doing several things before she started at the Library:

1) Putting together a resource list on UX in Libraries to get her up to speed with an area she was unfamiliar with - I made that publicly available here

2) Putting together a document that outlined the aims of the internship so Emma would know exactly what she was working towards - I've put this on Google Drive here for anyone interested. I've not edited this from the original so there's some York-centric language - also I said 'emoji' when I meant 'emoticons' so you'll have to forgive me. Here's a preview:

Part of the Aims of the Internship document we put together

(It's worth noting that we didn't achieve some of the aims - for example visiting Cambridge and Sheffield Hallam, or trying out group interviews.) 

Essentially the thing that made this project different to future UX projects we'd undertake is this one was at least partly about understanding UX processes as well as our actual users - so Emma was tasked with setting up a UX Toolkit for our future use 

3) Sort out all the admin stuff associated with a new member of staff - entry card, username and password, where Emma would sit, PC she'd use, access to Google Drive folders etc etc 

4) Put together a timetable for the first week or so of her employment, after which she would become more self-directed. This included inviting Emma to a number of meetings and a couple of teaching sessions, so she could go away with a more rounded impression of what life in an academic library, and particular in the Academic Liaison Team, was like. We wanted it to be as rewarding and CV-enhancing as possible for her, as well as focusing on our project.

All of this took AGES. Any work you can put in beforehand is worth it though, otherwise it quickly takes up most of your job generating work and things to do for the intern. (This is something I've heard echoed across other sectors too.) 

Feel free to re-purpose the reading list or the aims document if they help at your own organisation. 

Planning the project

As mentioned part of the aim was to build a UX toolkit - a suite of information and resources to call upon for future projects. As such as we decided Emma would use, as far as possible, all four of the interactive ethnographic techniques we'd learned (cognitive mapping, unstructured interviews, touchtone tours, love/break-up letters) with each participant, as well as doing behavioural mapping. My explanations of how to do these are in the 'Aims of the Internship' document, or see Emma's own post her description of each of these. 

This meant that a) Emma could start on the behavioural mapping and general observation while we recruited participants, and b) we'd need at least an hour of each participant's time. This would in turn mean a large amount of time spend interpreting and analysing the results; as a rule of thumb UX work takes 4 hours of analysis and reporting for every 1 hour of ethnographic fieldwork - a 4:1 ratio. 

The UX Team (the five conference attendees) met to discuss what sort of thing we should focus on in the project - I found this tricky because you want to provide a framework and guidance for an intern, but also part of the spirit of UX is to let the data tell you the story and not to go in with preconceptions to, or even seeking specific answers to questions. In the end we settled on using the project to better understand Postgraduate students simply because, during the summer holidays as this was, there were many more of them around. There were various things we hoped to learn - or various aspects we hoped to learn about - but we didn't put these into the project documentation or ask Emma to focus on them (or even tell her about them); we wanted the process to be as neutral as possible. 

We agreed that the five of us would meet during Emma's 6 weeks with us to discuss progress, look at the results, steer the further direction and so on.

During the project

Once Emma arrived and worked her way through the reading list, we started with observation and behavioural mapping. Observation is a great way for an intern to settle in because it's a relatively low pressure environment - it's a break from ingesting huge chunks of written information and a chance to be in your own head-space, and actually DOING ethnography where the stakes are much lower if you're not familiar with it yet. Not being sure about how to label a map of someone's path through the lobby is less intimidating than not being sure how to ask someone to write love-letter to a library service! 

The biggest problem we had was recruitment. We put requests for participants on social media, e.g.

If you're a postgrad, join us for cake & help us to design better services! Pls RT @GSAYork @WentworthColl @UniOfYork pic.twitter.com/W1AdP6XH4L

— Uni of York Library (@UoYLibrary) August 5, 2015

... and we put similar info on a giant whiteboard in the Postgraduate Lounge area. We also approached people face to face and left them with info about the project and Emma's contact details. All in all these approaches yielded just three participants.

So all the Academic Liaison Librarians emailed their PostGrad cohorts via Departmental administrators: this was much more successful and yielded lots of emails to Emma, most of whom went on to book appointments with her. 23 people in total were recruited this way. The students were a mixture of PGTs and PGRs, from a variety of Departments and a variety of nationalities. 

As it happened this project would conform to the 4:1 analysis to field work ratio almost EXACTLY: Emma was with us for 125 hours in total, and engaged with 26 participants in that time for around an hour each, spending the other 99 hours doing everything else: analysing, interpreting, transcribing, and writing up (and getting to grips with UX in the first place). It must be said that Emma was an incredibly proficient transcriber, having done this kind of work before: for mere mortals (me, for instance) the 4:1 ratio would not be remotely possible with transcription included, and in fact transcription itself often comes with a 4:1 ratio of its own, before you even get as far as analysis.

In general we consider ourselves incredibly lucky to have got Emma as our first ever UX intern: she was extremely bright and showed a great deal of initiative and confidence, as well as working extremely hard. She produced a brilliant report detailing her experiences across the 26 participants, with the findings clustered around the areas of: study space, noise levels, the catalogue, the library building, and facilities. We learned more about those 26 students than we'd ever learned about any students before.

Working with an intern is a brilliant way to free up enough time to actually start the process of UX and ethnography, although it still takes existing staff time to manage the project.

Michelle Blake, Head of Relationship Management at York, will write a later post about the findings of this and our second PG UX project in more detail.