Do questions help? The impact of audience response systems on medical student learning: A randomized controlled trial

Background Audience response systems (ARSs) are electronic devices that allow educators to pose questions during lectures and receive immediate feedback on student knowledge. The current literature on the effectiveness of ARSs is contradictory, and their impact on student learning remains unclear. Objectives This randomised controlled trial was designed to isolate the impact of ARSs on student learning and students’ perception of ARSs during a lecture. Methods First-year medical student volunteers at Johns Hopkins were randomly assigned to either (i) watch a recorded lecture on an unfamiliar topic in which three ARS questions were embedded or (ii) watch the same lecture without the ARS questions. Immediately after the lecture on 5 June 2012, and again 2 weeks later, both groups were asked to complete a questionnaire to assess their knowledge of the lecture content and satisfaction with the learning experience. Results 92 students participated. The mean (95% CI) initial knowledge assessment score was 7.63 (7.17 to 8.09) for the ARS group (N=45) and 6.39 (5.81 to 6.97) for the control group (N=47), p=0.001. Similarly, the second knowledge assessment mean score was 6.95 (6.38 to 7.52) for the ARS group and 5.88 (5.29 to 6.47) for the control group, p=0.001. The ARS group also reported higher levels of engagement and enjoyment. Conclusions Embedding three ARS questions within a 30 min lecture increased students’ knowledge immediately after the lecture and 2 weeks later. We hypothesise that this increase was due to forced information retrieval by students during the learning process, a form of the testing effect.