Elsevier

Journal of Surgical Education

Volume 76, Issue 1, January–February 2019, Pages 294-300
Journal of Surgical Education

ORIGINAL REPORTS
A Structured Review Instrument Improves the Quality of Orthopaedic Journal Club

https://doi.org/10.1016/j.jsurg.2018.06.017Get rights and content

Objective

We asked the following questions: 1. Does the use of an structured review instrument (SRI) at journal club increase presentation quality, as measured objectively by a standardized evaluation rubric? 2. Does SRI use increase the time required to prepare for journal club? 3. Does SRI use positively impact presenter perceptions about confidence while presenting, satisfaction, and journal club effectiveness, as measured by postparticipation surveys?

Design

A prospective study was designed in which a grading rubric was developed to evaluate journal club presentations. The rubric was applied to 24 presentations at journal clubs prior to introduction of the SRI. An SRI was developed and distributed to journal club participants, who were instructed to use it to prepare for journal club. The grading rubric was then used to assess 25 post-SRI presentations and scores were compared between the pre- and post-SRI groups. Presentations occurred at either trauma, pediatrics, or spine subspecialty journal clubs. Participants were also surveyed regarding time requirements for preparation, perceptions of confidence while presenting, satisfaction, and perceptions of overall club effectiveness.

Setting

A single academic center with an orthopaedic surgery residency program.

Participants

Resident physicians in the department of orthopaedic surgery.

Results

Mean presentation scores increased from 14.0 ± 5.9 (mean ± standard deviation) to 24.4 ± 5.2 after introduction of the SRI (p < 0.001). Preparation time decreased from a mean of 47 minutes to 40 minutes after SRI introduction (p = 0.22). Perceptions of confidence, satisfaction, and club effectiveness among trainees trended toward more positive responses after SRI introduction (confidence: 63% positive responses pre-SRI vs 72% post-SRI, p = 0.73; satisfaction: 64% vs 91%, p = 0.18; effectiveness: 64% vs 91%, p = 0.19).

Conclusions

The use of a structured review instrument to guide presentations at orthopaedic journal club increased presentation quality, and there was no difference in preparation time. There were trends toward improved presenter confidence, satisfaction, and perception of journal club effectiveness. SRI utilization at orthopaedic journal club may be an effective method for increasing the quality of journal club presentations. Future work should examine the relationship between presentation quality and overall club effectiveness.

Introduction

Journal clubs have a long history in medicine and orthopaedic surgery, and are common in orthopaedic departments around the country.1, 5,6 Important goals of these clubs include teaching residents to critically evaluate scientific articles, instilling formal habits for reading scientific articles, and facilitating resident learning about current research.4, 6 There are multiple possible formats for journal clubs.5 In orthopaedics, the most common is trainee led discussions of assigned articles.5, 6 At our institution, a similar format is followed: trainees are assigned a particular article in advance by a faculty leader and subsequently must read, analyze, and present it.

One important factor in presentation-driven journal clubs is the quality of the presentations. Poorly structured presentations can fail to highlight important points of the study being discussed, which may lead to less informative discussion overall. In our experience, this can subsequently lead to poor attendance, and journal clubs that are of limited educational value. The use of a structured review instrument (SRI) that can be used to critique and present articles has been described.3, 5 In general, SRI take the form of standardized checklists that guide the presenter toward important items when reviewing and presenting articles. In the Emergency Medicine literature, the use of a review instrument was shown to significantly increase satisfaction with journal club without increasing workload,3 but the authors did not comment on whether presentation quality was affected. In the orthopaedic literature, orthopaedic specific structured review instruments have been described,5 however, no data regarding the effectiveness of these instruments has been presented.

Therefore, we asked: 1. Does the use of an SRI at journal club increase presentation quality, as measured objectively by a standardized evaluation rubric? 2. Does SRI use increase the time required to prepare for journal club? 3. Does SRI use positively impact presenter perceptions about confidence while presenting, satisfaction, and journal club effectiveness, as measured by postparticipation surveys?

Section snippets

Materials and Methods

A prospective interventional study was designed to assess presentation quality at journal club before and after introduction of a structured review instrument. The study was reviewed by our Institutional Review Board and found to be exempt. At our institution, journal clubs are held on a routine basis and are subspecialty specific; the Trauma, Pediatrics, and Spine services were selected to participate in the study. These journal clubs are all faculty-led, with articles assigned to residents

Results

Does the use of an SRI at journal club increase presentation quality, as measured objectively by a standardized evaluation rubric?

There were 24 participants in the pre-SRI group and 25 in the post-SRI group. Prior to introduction of the SRI, mean presentations scores were 14.0 ± 5.9 for all presenters, 15.8 ± 5.2 for senior trainees (PGY 4-6), and 12.3 ± 6.3 for junior trainees (PGY 1-3). After introduction of the SRI, mean scores increased to 24.4 ± 5.2, 24.6 ± 5.1, and 23.9 ± 5.9,

Discussion

Structured review instruments have been proposed for use at orthopaedic journal club,5 but their effectiveness has not been objectively assessed. At our journal clubs, the introduction of an SRI increased the quality of the presentations delivered by orthopaedic residents. There was no difference in preparation time identified, and there was a trend toward improved presenter confidence, satisfaction, and perception of journal club effectiveness.

There is little existing data objectively

References (6)

There are more references available in the full text version of this article.

Cited by (4)

  • Redesigning Journal Club to Improve Participant Satisfaction and Education

    2022, Journal of Surgical Education
    Citation Excerpt :

    Even fewer studies have analyzed participant satisfaction and perceived educational benefits, and those that have analyzed those variables have been unable to consistently demonstrate improvements thus far. e.g. support for the use of a structured review instrument has been inconsistent with one study in 1995 demonstrating increased satisfaction and perceived educational value while another study in 2018 failed to demonstrate a statistically significant improvement in those same variables.4-5 Thus, the purpose of this study was to describe programmatic efforts to assess and revamp the format of journal club as well as to evaluate participant satisfaction and perceived educational benefits.

  • A pilot psychedelic psychopharmacology elective

    2022, Currents in Pharmacy Teaching and Learning
    Citation Excerpt :

    After each presentation, the students and the instructor graded the presenter using the same rubric. The rubric was adapted from Campbell et al15 (see eFigure), with presentation grades split 50:50 between the average student-assigned grade and the grade assigned by the instructor. This grade represented 10% of students' final grade, with another 10% allotted to participation in journal clubs they were not leading.

Conflict of interest statement: Each author certifies that he or she has no commercial associations that might pose a conflict of interest in connection with the submitted article.

Disclosures: MJG, KBW, JAB hold stock ownership, consultancies, or patent/licensing agreements unrelated to this study. One or more of the authors (MJG) has received funding from SMV Medical, SmartDevices, Medtronic, Synthes, but not for this study.

This study was reviewed by our institution's IRB and found to be exempt.

View full text