What's in a citation? Does using high quality resources lead to better marks?

I’ve been telling students for years that using high quality resources in their assignments leads to them getting better marks. But is that actually true, or have I just been repeating that mantra in the hope that students pay attention?! In this blog post I’ll tell you a little bit about a piece of research I carried out with our Nursing programme to test that claim. I’ll focus on the practicalities of how I worked with departmental staff and some of the outcomes; you can read more about the process of the research in my article in the Journal of Academic Writing.

What was the research?

Inspired by something I’d seen at a conference - there are no new ideas after all! - I carried out citation analysis on some randomly chosen examples of students’ essays, from across all three years of the BSc Nursing programme. The analysis was in two stages: first I coded the bibliographies to determine the type of references used, then I gave each bibliography a rating depending on the quality of the selected references. I then compared the marks given to each essay against the quality of the bibliography. This showed that there was a relationship: the students who had used better quality resources generally received a better mark. Hooray, I hadn’t been misleading students after all!

Some books, because how else do you visually represent referencing?!

Why did it work?

As much as this research was important for my own teaching, the main reason it worked at all was that it was also important to the department. It wasn’t just a vanity project! I’d heard anecdotal feedback from academics that students were often using poor quality or otherwise inappropriate references in their work, so this was an opportunity to explore that feedback in more detail. If you’re thinking of doing something similar I believe that buy-in from the department would be essential; there has to be a benefit to them as well as to you.

It was also hugely beneficial that I maintain a strong working relationship with the department, which otherwise would have made this project impossible. I needed the department’s overall backing for the research, but also specifically needed the help of their Assessments Office to provide the example scripts and the students’ marks. I also went through the department’s ethics procedures for the research. If you’re thinking of doing something similar then that strength of relationship is vital, both to make the research happen practically but also for it to have a lasting impact. In my case I was also later able to co-author the resulting journal article with academic staff from the department, so there was a really useful product for everyone at the end of the process.

Was it worth it?

A resounding yes! Although that’s maybe not what I would have said when I was in the middle of analysing all the scripts! Whilst it was undoubtedly a lot of work to undertake, it’s been hugely beneficial to have an empirical answer to my assertion that using better resources was a good thing. I’ve been able to use this information since in my digital literacy teaching, so it’s directly informing the support that I give to students. This was especially important given the target audience of student nurses; I teach them about evidence-based practice, so can now say with certainty that I practice what I preach!

On a personal note I also found it a refreshing change to carry out a research-based project. It was a valuable opportunity to demonstrate the academic rigour of information literacy work and a useful learning curve around the research and publication process. It did highlight, however, that a future in statistics is very likely not on the cards for me!

What next?

I wanted to make sure that the research didn’t stop there, and that it brought about meaningful change. I reflect in the journal article on the need to address poor referencing in the early stages of a student’s degree, as it was very noticeable that bibliographies in the first year in particular were quite poor (and not entirely surprising given that referencing will be a new skills for the majority of students). I therefore transformed the teaching I offer in the first year to focus much more overtly on choosing high-quality references and creating a fully-formed and credible bibliography of resources. This has been well received by students, many of whom really struggle with this skill, in particular knowing what ‘good’ is supposed to look like in their essays.

Here are my redesigned slides:

CLG2 The Good Nurse in the Literature May 2019 from University of York Library

I’ve also moved to a new model in my teaching on the BSc Nursing programme, moving away from a workshop-style delivery to a more integrated approach. I now provide an introductory lecture in certain key modules in the programme, each of which is immediately followed by seminars facilitated by academic staff. This means that staff and students are more engaged in skills development, and that my content is seen as a core part of the module rather than an optional extra. Students also have follow-up activities to complete in advance of their next seminar, setting an expectation that they need to take an active role in their own learning. Thinking back to referencing in particular, this has foregrounded the importance of students being able to select appropriate, scholarly resources and has encouraged academic staff to consider this as part of wider curriculum redesign in the programme. Not bad for what started as a random train of thought at a conference!

Have you carried out any similar research? Let me know your comments and questions below.