June 19, 2019

What's in a citation? Does using high quality resources lead to better marks?

I’ve been telling students for years that using high quality resources in their assignments leads to them getting better marks. But is that actually true, or have I just been repeating that mantra in the hope that students pay attention?! In this blog post I’ll tell you a little bit about a piece of research I carried out with our Nursing programme to test that claim. I’ll focus on the practicalities of how I worked with departmental staff and some of the outcomes; you can read more about the process of the research in my article in the Journal of Academic Writing.

What was the research?

Inspired by something I’d seen at a conference - there are no new ideas after all! - I carried out citation analysis on some randomly chosen examples of students’ essays, from across all three years of the BSc Nursing programme. The analysis was in two stages: first I coded the bibliographies to determine the type of references used, then I gave each bibliography a rating depending on the quality of the selected references. I then compared the marks given to each essay against the quality of the bibliography. This showed that there was a relationship: the students who had used better quality resources generally received a better mark. Hooray, I hadn’t been misleading students after all!

Some books, because how else do you visually represent referencing?!

Why did it work?

As much as this research was important for my own teaching, the main reason it worked at all was that it was also important to the department. It wasn’t just a vanity project! I’d heard anecdotal feedback from academics that students were often using poor quality or otherwise inappropriate references in their work, so this was an opportunity to explore that feedback in more detail. If you’re thinking of doing something similar I believe that buy-in from the department would be essential; there has to be a benefit to them as well as to you.

It was also hugely beneficial that I maintain a strong working relationship with the department, which otherwise would have made this project impossible. I needed the department’s overall backing for the research, but also specifically needed the help of their Assessments Office to provide the example scripts and the students’ marks. I also went through the department’s ethics procedures for the research. If you’re thinking of doing something similar then that strength of relationship is vital, both to make the research happen practically but also for it to have a lasting impact. In my case I was also later able to co-author the resulting journal article with academic staff from the department, so there was a really useful product for everyone at the end of the process.

Was it worth it?

A resounding yes! Although that’s maybe not what I would have said when I was in the middle of analysing all the scripts! Whilst it was undoubtedly a lot of work to undertake, it’s been hugely beneficial to have an empirical answer to my assertion that using better resources was a good thing. I’ve been able to use this information since in my digital literacy teaching, so it’s directly informing the support that I give to students. This was especially important given the target audience of student nurses; I teach them about evidence-based practice, so can now say with certainty that I practice what I preach!

On a personal note I also found it a refreshing change to carry out a research-based project. It was a valuable opportunity to demonstrate the academic rigour of information literacy work and a useful learning curve around the research and publication process. It did highlight, however, that a future in statistics is very likely not on the cards for me!

What next?

I wanted to make sure that the research didn’t stop there, and that it brought about meaningful change. I reflect in the journal article on the need to address poor referencing in the early stages of a student’s degree, as it was very noticeable that bibliographies in the first year in particular were quite poor (and not entirely surprising given that referencing will be a new skills for the majority of students). I therefore transformed the teaching I offer in the first year to focus much more overtly on choosing high-quality references and creating a fully-formed and credible bibliography of resources. This has been well received by students, many of whom really struggle with this skill, in particular knowing what ‘good’ is supposed to look like in their essays.

Here are my redesigned slides:



I’ve also moved to a new model in my teaching on the BSc Nursing programme, moving away from a workshop-style delivery to a more integrated approach. I now provide an introductory lecture in certain key modules in the programme, each of which is immediately followed by seminars facilitated by academic staff. This means that staff and students are more engaged in skills development, and that my content is seen as a core part of the module rather than an optional extra. Students also have follow-up activities to complete in advance of their next seminar, setting an expectation that they need to take an active role in their own learning. Thinking back to referencing in particular, this has foregrounded the importance of students being able to select appropriate, scholarly resources and has encouraged academic staff to consider this as part of wider curriculum redesign in the programme. Not bad for what started as a random train of thought at a conference!

Have you carried out any similar research? Let me know your comments and questions below.

December 18, 2018

Branding the academic library

By Ned Potter, Academic Liaison Librarian

Earlier in the year I got to head over to Denmark, to deliver a keynote at the Danish Academic and Research Library group's annual conference. The subject of the conference was marketing, and I was specifically asked to talk about the academic library brand.

Here are my slides:

 
Branding the Academic Library from Ned Potter

There were a couple of key points I wanted to get across - one is that the brand is everything your users experience! Colours and logos are branding, and they are part of a wider attempt to positively influence the brand - but we can't control the brand. The voice we use in our communications, the experiences people have at the front desk, the collections and services we offer: all of this makes the brand.

We also can't influence the brand on a national or global level - at least not without a great deal of difficulty. It's a lot more realistic for each specific University library to try and build a better brand; rather than to try and change the overall perception of the academic library as a concept.

I also stressed, as I always do, that the key to marketing libraries well is to do so in campaigns. Any questions or comments, let me know below! 

November 19, 2018

[Video] UX keynote all about ethnography and design at York

By Ned Potter, Academic Liaison

This month I had the honour of giving the opening Keynote at the Stellenbosch University Library Symposium, the theme for which this year was User Experience.

I've travelled to South Africa for a conference before and it was one of the best professional experiences I've ever had, but this time I couldn't quite make the huge journey time work, with it being Autumn term, so we decided to do it virtually. We chose a mixture of recorded and live - I created a video, using Screencast-o-matic which I love, and then we did the Q&A live via Skype. I thought this was safer than trying to do the whole thing live and risk technology failure...

Although my colleagues and I have talked about UX at various events over the last three years, this was the first time we'd had the length of time afforded by a keynote, and my hosts had specifically asked to hear about the projects we've done at York - so essentially I pulled together all threads we often write about on this blog into one big presentation. I've embedded the video below and of course you're more than welcome to watch the whole thing if you'd like to! But given that people often have specific areas of interest around UX, here are some shortcuts to the various parts of the presentation:
  1. After the preamble there's an introduction to UX and ethnography. What it is (and isn't), the way it has spread across the industry, why it's so powerful. 
  2. Part 2 is a A timeline of UX at York. I discuss the various major UX-led projects we've done, our use and non-use of interns to help us, and the various ethnographic techniques we've used
  3. The massive up-scaling involved for the Understanding Academics project, plus some of our findings and the changes we made, are discussed next
  4. How we try to disseminate our findings and experiences is covered here
  5. Next I cover steering and guidance for UX. How it fits strategically, our UX Group which includes our UX book club, and so on
  6. Training and Support for UX is discussed here: our internal training sessions and the way we've been trying to bring this techniques to other parts of the University
  7. Finally, some tips for embedding UX at your own institution

There were some great questions afterwards not captured by this video. One of them was around the use of interns and the pluses and minuses of this. At York our first two UX projects were intern-led and we found this hugely beneficial for a number of reasons: someone having the time and space to focus on the fieldwork and analysis without other distractions was part of it, but so was having a completely fresh pair of eyes involved. There are advantages to non-library staff speaking to users as part of the fieldwork and bringing an outside perspective to the design recommendations.

That said, we did all the fieldwork for our two biggest UX projects to date ourselves, partly because it was such a good opportunity for as as staff to be learning about our users first-hand. It's also a fascinating process to be part of and a good opportunity for staff to learn new skills.

So without wanting to sit on the fence, there are good arguments both for using interns and for doing everything yourself...

______________________

Thank you so much to everyone at the Conference for inviting me and allowing me to present virtually. It was fun to put together.

November 06, 2018

Building the Fabrik Wall

By Ned Potter, Academic Liaison Librarian 

At the start of the academic year we reached a significant milestone in our use of UX at York (about which you can read many, many more posts elsewhere on this blog). We made our first major purchase as part of a re-design that came directly from an ethnographic project - woohoo!

We bought a sound-absorbent wall made of 'Fabriks' - it works like giant Lego, looks pretty cool, and controls the acoustics of the environment around it.

Here's what it looks like:

The Fabrik Wall as seen from the front

The Fabrik Wall as viewed from 'inside' it
To get the context to all of this we need to go back to 2017.

The UX Space Project

During 2017 we undertook the UX Space Project: ethnographic User Experience research into how the Morrell Lounge is used and how it might be improved. This is the area you see as you enter the Library at York - on average around 3,000 people pass through this every single day, making it perhaps the busiest area on either campus. At peak times there's around 7,000 entries.

The fieldwork consisted of more than 40 sessions of behavioural mapping. We actually had to modify the 'standard' way we'd done observation like this before, because the area was simply too busy to track people's routes through the space. That element was removed from the list of responsibilities, but we made other tweaks too: for the first time we used a template map of the space (based on our floorplans), we put a definition of the AEIOU framework on the back along with other guidance to remind people what they were looking for, and left space for notes on the map itself.

Here's an example of the map and notes from one observation session:


A behavioural map with notes 
The fieldwork was undertaken by Customer Services staff (each of whom had individual training before their first session) and they were absolutely brilliant at capturing key info, and then feeding into the design process to find solutions too. All CS stuff kept notes of their thoughts on the space when working at the desk too, so we built up a picture of how the area was used even on days when we didn't have a Behavioural Mapping session planned. This turned into a massive 19-page Google Doc, which also included feedback from students and staff as it came in.

The initial phase of fieldwork lasted 6 months. The project identified problems including the area being too loud for those wishing to seek guidance at the Help Desk, it being unsightly to visitors as students are sprawled on the floor, and fundamentally the space not being used as we intended.


Design Prototyping

We ruled out trying to change the way the space was used, early on in the process. Instead we wanted to change the space to better suit the way it has been adopted by students and staff: this included swapping furniture with other areas of the Library to suit the way people congregate in groups, which was our first proper attempt at design prototyping. The changes we made went down so well, getting so much positive feedback (we had people emailing the library just to tell us how much they liked it!), that following another round of observational fieldwork we ended up leaving the temporary furniture in situ. It's still there now, although long term we'd like to improve the space further - we have a full design ready to go, based on all the research and suggested changes we've come up with, but it will cost a lot of money to realise.

Most costly of all we really wanted some kind of acoustic baffling to both contain the sound and make the entrance lobby a visually arresting sight with which to greet our regular users and visitors. We went through several rounds of a bidding process to try and win some Alumni funding to make big changes to the space - in the end the 'YuFund' scheme provided us with the funds to purchase what we're calling the Alumni Wall that you saw at the start of this post, made of special sound-absorbent ‘fabriks’ to change the acoustics of the space. And the acoustic difference is absolutely remarkable, significantly reducing the noise spilling from the Lounge, and in particular better enabling staff and students to converse at the Help Desk. Many staff have commented on the effect this has had - it is now noticeably easier to help people with queries - and the students like it too: ’Wow I really like it!’ one returning student exclaimed, coming to the desk especially to tell us… It also looks fantastic, and gives the whole area a more cohesive feel.

The whole project felt really exciting to be part of. It involved people from different teams in the Library working together, which is always good, and the Alumni Office (and the Students' Union) were really enthusiastic about what we were doing. Ultimately we were able to make a significant change to the space, and overall it works much better for both students and staff than it ever has before.

Finally, here's a short time-lapse video of its assembly: