Processing and ACTING ON 100+ hours of ethnography: a 5 stage approach

By Ned Potter, Academic Liaison Librarian 

Understanding Academics, introduced in the last blog post, is far and away the biggest UX project we’ve attempted at York, and the processing and analysis of the data has been very different to our previous ethnographic studies. This is due to a number of factors: primarily the sheer size of the study (over 100 hours’ worth of interviews), the subject matter (in depth and open ended conversations with academics with far ranging implications for our library services), and actually the results themselves (we suspected they’d be interesting, but initial analysis showed they were SO insightful we needed to absolutely make the most of the opportunity).  

Whereas for example the first UX project we ran conformed almost exactly to the expected 4:1 ratio of processing to study – in other words for every 1 hour of ethnography it took four hours to analyse and process – the total time spent on Understanding Academics will comfortably be in excess of 400 hours, and in fact has probably exceeded that already.

UX is an umbrella term which has come to mean a multi-stage process – first the ethnography to understand the users, then the design to change how the library works based on what you learned. In order to ensure we don’t drown in the ethnographic data from this project and never get as far as turning it into ‘proper’ UX with recommendations and changes, Michelle Blake and Vanya Gallimore came up with a 5 stage method of delivering the project.

Two particular aspects of this I think are really useful, and not things we’ve done in our two previous UX projects: one is assigning themes to specific teams or individuals to create recommendations from, and the other is producing and publicising recommendations as soon as possible rather than waiting until the end of the whole project.

As you can imagine the 5 stage method is very detailed but here’s a summary:

Coloured pens used in cognitive mapping
(in this case with the interviewer's reminder about the order
in which to use them)

      1)  Conduct and write up the ethnography. Academic Liaison Librarians (ALLs) spoke to around 4 academics from each of ‘their’ Departments, usually asking the subject to draw a cognitive map relating to their working practice,
and then conducting a semi-structured interview based on the results.

The ALLs then wrote up their notes from the interviews, if necessary referring to the audio (all interviews were recorded) to transcribe sections where the notes written during the process didn’t adequately capture what was said. The interviews happened over a 2 month period, with a further month to complete the writing up.

      2)   Initial coding and analysis. A member of the Teaching and Learning Team (also based in the library) who has a PhD and experience of large research projects then conducted initial analysis of the entire body of 100+ interviews, using NVIVO software. The idea here was to look for trends and themes within the interviews. The theming was done based on the data, rather than pre-existing categories – a template was refined based on an initial body of analysis. In the end, 23 over-arching themes emerged – for example Teaching, Digital Tools and Social Media Use, Collaborations, Research, Working Spaces. This process took around 2 months.

      3)   Assigning of themes for further analysis and recommendations. Vanya then took all of the themes and assigned them (and their related data) to members of the Relationship Management Team – this consists of the Academic Liaison and Teaching and Learning teams already mentioned, and the Research Support team. This is the stage we are at now with the project – each of us in the team have been assigned one or more theme and will be doing further analysis at various times over the next 8 to 10 months based on our other commitments. A Gantt chart has been produced of who is analysing what, and when. The preparation and assigning of themes took around 2 weeks.

      4)   Outcomes and recommendations. There are three primary aims here. To come up with a set of practical recommendations for each of the themes of the project, which are then taken forward and implemented across the library. To come up with an evidence-base synthesis of what it means to be an academic at the University of York: a summary of how academics go about research and teaching, and what their key motivations, frustrations and aspirations are. (From this we’ll also aim to create personas to help articulate life for academics at York.) And finally to provide Information Services staff with access to data and comments on several areas in order to help inform their work – for example members of the Research Support team will have access to wealth of views on how academics think about Open Access or the repository.

These aims will be achieved with a combination of devolved analysis assigned to different groups, and top-down analysis of the everything by one individual. Due to other projects happening with the teams involved, this stage will take up to 7 months, although results will emerge sooner than that, which leads us neatly to...

      5)  Distribution and Dissemination. Although this is last on the list, we’re aiming to do it as swiftly as possible and where appropriate we’ll publicise results before the end of the project, so stages 4 and 5 will run simultaneously at times. The total duration from the first interview to the final report will be around 18 months, but we don’t want to wait that long to start making changes and to start telling people what we’ve learned. So, once an evidence-based recommendation has been fully realised, we’ll attempt to design the change and make it happen, and tell people what we’re doing - and in fact the hope is to have a lot of this work completed by Christmas (half a year or so before the Summer 2017 intended end date for the final report). 

The full methods of dissemination are yet to decided, because it’s such a massive project and has (at a minimum) three interested audiences: York’s academic community, the rest of Information Services here, and the UX Community in Libraries more widely. We know there will be a final report of some sort, but are trying to ensure people aren’t left wading through a giant tome in order to learn about what we’ve changed. We do know that we want to use face to face briefings where possible (for example to the central University Learning and Teaching Forum), and that we’ll feedback to the 100 or so academics involved in the study before we feedback to the community more widely.

As we go through this project we’ll try and keep writing on this blog to update on where we are, what’s working, and what the challenges are. Above all, Understanding Academics has been one of the most exciting and insightful projects any of us have ever attempted in a library context. 

Understanding Academics UX project

By Vanya Gallimore, Acting Head of Relationship Management

We wrote about our PGRUX project all the way back in July and mentioned that we hoped to post soon about our most recent project, Understanding Academics.

It’s rather an ambitious sounding title and indeed lots of academics laughed when we mentioned what we’d called it and wished us good luck! What we wanted to achieve in this project was to really gain a better understanding about how our academic colleagues approach their teaching and their research and where we, the library and the services we provide, fit in with that. It would also give us the opportunity to look for new services that we could potentially be offering and/or problems with our current services. We also hoped it would give us a greater understanding of how each academic department works within the University so we can start to provide tailored support for departments. Finally we hoped it would allow us to nuance our communications and think about how best we could talk to departments.

Background to the Project

The creation of our academic liaison team in 2014 strengthened our relationships between the Library and the departments but we wanted to build on that and go deeper into understanding the needs of the academic community and ensure that those needs were reflected into our service developments.

The work was intended to inform a variety of projects and pieces of work which are due to be undertaken over the next year in Information Services. This includes but is not limited to the following:

• Understanding textbooks
• Replacement for our inhouse reading list system and associated processes and procedures (Key text collection)
• York Pedagogy (and associated digital literacy teaching)
• Our lending system
• Student and staff inductions
• Communications to departments
• Engagement with those not based at our main campus, Campus West
• Buying habits of departments and individuals of resources
• Research Support services
• Understanding, managing and promoting our collections

We agreed, after much discussion and an earlier ambitious target, the aim of interviewing 4 members of staff in each academic department, ideally two focused on teaching and two on research (note: we didn’t mind if people had joint roles - rarely do people only do one of these activities - but we wanted them to concentrate on just one area for the purposes of the study as we had limited time with each individual and a lot to cover during that time).

We trialled the technique with three friendly academics to ensure that it worked, we had allowed enough time and to get their feedback on the process. These initial interviews worked really well and allowed the process to be refined before starting to undertake them across all departments.

Ethnographic methods

The project involved two UX techniques: a cognitive map, followed by a semi structured interview. While we had used love/break up letters in the PGRUX project we decided against using these for this project. We did find that some academics were very sceptical about the cognitive maps and didn’t fully understand what the point of it was but in nearly all cases they were pleasantly surprised by how useful they found it afterwards.

We did not offer any incentive for taking part and in some departments we had academics actively volunteering to be interviewed. I think this was due in part to how we pitched it about wanting to know more about them, rather than asking a series of questions about the library. The two questions we started with were very broad (can you draw a map of your research process or how you approach a new or existing module for teaching) to understand how the academics worked and where we, the Library, could see ourselves fitting into their process (were there missed opportunities, things we were doing they didn’t need/want etc).

The semi-structured interview always started with asking the academic to talk through their cognitive map with us and taking things from there. As much as possible we were led by the academic as we wanted to understand what was important to them.  We did have some prompt areas for the semi-structured interviews if the academics didn't mention them at all e.g. reading lists, student skills, resources, dissemination of their research that we would ask about if there was time. Our aim was to ask open questions and not lead anyone (this was one of the big challenges, particularly in the first couple of interviews) and to ask "why" a lot.

Staffing the project

In order to ensure that we could undertake the volume of interviews we were expecting to have (about 110 - each just over an hour long, plus time to write them up etc.) we needed a large enough pool of staff.

For the two previous UX projects we had used an intern but we were really keen this time that we used our own staff for this. There were a couple of reasons: our staff have the contextual knowledge about the library and a student often won’t understand the library context e.g. things like metrics would be very confusing to them - we had researchers talk about their "H index" for instance. I also felt strongly that this kind of activity would give staff a better understanding of their own academic departments (and indeed a couple of the liaison librarians had already undertaken similar conversations with some departments that they could then build on). Finally this is the kind of activity that we want to do more of and if we don't develop our staff and are always getting in others to do this, we don't build up expertise in this area.

We decided that our academic liaison librarians were best placed to undertake this work and in order to ensure that they had the necessary skills and confidence to do this, we put together a four week training programme for them. This consisted of:

• An overview of the UX techniques to be used and time to practice these on each other
• Neuro linguistic programming (NLP) training
• Business analysis tools (including techniques like the 5 whys)
• Active listening and questioning techniques

All except NLP were run by members of Information Services.

Due to the timing of the project we also had the opportunity to discuss the project during the Action Plan meetings with the Library Rep and Head of Department (HoD). This was really useful and we received a lot of support for the project at these meetings. HoDs were also useful in helping to identify individuals who would be good to take part, for example, new or relatively junior staff, research or teaching focused staff or someone they thought would be interesting for another reason.

Processing the interviews

We didn't have time (or money) to do full transcriptions of the interviews for this project so each interview was written up either in full notes or part transcription (where the academic talked through their cognitive map). This was then all coded into NVivo by a member of staff who and then produced reports on each "theme". These themes are now being analysed by a member of the team who will decide the best way to present the information e.g. the data about reading lists is helping to produce a list of requirements for a new reading list system but data around how researchers collaborate will help us to build personas about what researchers do/look like (or at least we hope it will).

One of the big reasons for undertaking this work in the first place was really to gain a better understanding about how academics work - both in their teaching and their research and to understand where we fit (or can fit) into that. We think librarians are often guilty of thinking we're the most important people and can't understand why someone hasn't got back to us. Understanding the other pressures that academics are often under is really useful (and helpful to articulate this to colleagues in other areas e.g. Collections) about why the deadline for reading lists is not working etc.

We now have a huge dataset and we need to continue to analyse each themed area. We will be updating the blog periodically as each area is reported on and we decide what to do with it over the coming year.

Building a new university press

By Michael Fake & Tom Grady, White Rose University Press


White Rose University Press (WRUP) is an open access digital university press, run jointly by the Universities of Leeds, Sheffield and York. It was launched in January 2016 as one of only a handful of new university presses, and the only jointly-run academic press, in the UK. WRUP is first and foremost, an open access digital publisher, intended to support the production of high quality academic works, including books, journals and textbooks - and to make them free to read online. As of today the Press has had a stream of exciting, high-quality submissions, and we’re looking forward to the launch of our first books and journals at the start of 2017.

In this post I want to consider how the Press came about, and why we’ve adopted the approach that we have.

White Rose Libraries

The Press was set up and is operated by the libraries of the three partner institutions. These work together as the White Rose Libraries, collaborating on projects and services in order to achieve things that each library would struggle to do on their own. The partnership has a long history, dating back to the launch of their jointly-run institutional repositories (White Rose Research Online and White Rose Etheses Online), and including early involvement in what is now the Copac Collection Management Tools project, as well as mutual support agreements, shared training activities, and various other regional initiatives designed to pool resources and share expertise.

The limitations of scholarly communication

In 2014 the partnership started investigating the idea of setting up a digital press, with the intention of publishing open access digital journals, monographs, textbooks, and other scholarly outputs. The initial driver for this was strategic. The libraries, of course, were very conscious of the limitations of many of the existing models of scholarly communication: high subscription charges for journals; high APCs for open access articles deposited in hybrid journals; high costs for books; and limited or cumbersome functionality in many commercial ebook platforms - all while commercial publishers put pressure on academics to hand over copyright and agree to restrictions in the dissemination of their work. At the same time the consortium was watching with interest some of the moves that HEFCE and other funders were making around open access publishing - the rules on open access to REF articles was due to come into effect in 2015, and HEFCE were also making noises about possible future requirements for open access monograph publishing, with the Crossick Report being seen as an important weathervane.

In this environment the White Rose libraries were looking to find ways to support alternative models that could enhance the broader open access agenda; that would help scholars publish academic material that was free to readers; and that could reach as wide an audience as possible - all while ensuring that academics kept control over the material they were producing. Developing an open access university press, focused on publishing high quality material free to a global audience, was a way of doing something very practical and concrete in this space, and in late 2014 this vision, perhaps for the first time, started to seem feasible due to developments in publishing technology and workflows.

Changes in digital publishing

Digital-first publishing now means that books can be produced without needing huge investment in paper stock and warehousing of expensive print runs. Online books can be distributed through a variety of discovery channels without building a complex distribution chain from scratch. Even where print production is still desired (and there is still plenty of evidence that print is the preferred medium for academic monographs) the growth of print-on-demand vendors means that a digital press can create ebooks, without needing to second-guess the print market, and without requiring an expensive up-front investment in stock they might never sell. At the same time platforms for managing the process of book and journal editorial workflows have become available as open source software (e.g OJS, OMP), significantly reducing the potential overheads in establishing and running a publishing operation.

Alongside these technical developments, in 2014/15 a new generation of publishing service suppliers were entering the market - most prominently Ubiquity Press - offering a package of production services to new presses, allowing institutions to set up a publishing operation much more quickly, with a full production infrastructure ready from the start. This offered the tempting prospect that we could launch a new press not just with editorial software, a logo, and optimism - but with a whole network of relationships with typesetters, graphic designers, copyeditors, and indexers already established, as well as publishing expertise to guide us through our faltering first steps.

The rise of the new university press

Leeds, Sheffield and York were not quite the first to take the leap into publishing: at the time that we were making the decision in 2015 there were a handful of other libraries/institutions that had moved into the same space: UCL Press is the most prominent example, but also Westminster University Press, Goldsmiths Press, and University of Cardiff Press - along with other experimental publishing initiatives like Knowledge Unlatched and the Open Library of Humanities, which were similarly trying to develop new (mostly open access) models for scholarly publishing. That surge of activity was encouraging, suggesting as it did that we would be joining the vanguard of a broader movement in the development of scholarly communication. Our own research, surveying academics within the White Rose institutions, confirmed that a press would be meeting a perceived desire - with a significant proportion of academics saying that they would prefer to publish through an institutional or consortial press over a commercial press.

Implementation and governance

From idea to implementation took a significant amount of work throughout 2015. In the first instance it meant persuading a lot of stakeholders in academic departments across the three universities: we wanted and needed buy-in from academics, so there was a lot of work in taking the business case to various academic committees and boards, alongside persuading key influencers. However we found there was a real chicken and egg problem. In order to persuade people that the idea was solid and could succeed we needed to point to scholars who had already signed up to publishing with us - but academics weren’t willing to sign up to such an unknown entity without clear institutional backing. In the end the institutions had to take a leap of faith, trusting in the evidence we’d seen of successful initiatives with other new university presses.

The model we adopted was to work with Ubiquity Press as our production partner. This allowed us to tap into an existing infrastructure at relatively low cost, and to prioritise our limited resources into the front-end operations of attracting and working with academic authors and editors. The financial model underpinning the Press is “gold” open access publishing - that is, we levy Article and Book Processing Charges to pay for production - supported by an initial waiver fund put forward by the libraries. Of course, because the Press is not driven by profit, and because we only charge for the services we provide, those APCs and BPCs are as low as we can make them - just £360 for an article, and c.£4 - £5k for a medium-sized monograph.

The final key component was governance - ensuring we had an engaged and committed editorial board willing to hold us to the rigorous academic standards we had set for the Press, and to oversee our quality control processes. We were incredibly lucky to recruit such a distinguished set of academics from across the partnership who have been working with the Press’s operational staff and libraries to ensure that WRUP remains an academic-led and academic-focused initiative. Chaired by Professor Wyn Morgan (PVC for Learning and Teaching at Sheffield), the editorial board is evidence of the serious engagement with the Press that we’ve seen from senior academics across the consortium.

The Press opened for submissions on 4th January and we've had a really inspiring level of interest from a range of scholars and academics. Our first journals and books have been commissioned and are in preparation, and we're looking forward to launching our first publications at the start of 2017!