Test-enhanced learning in a middle school science classroom: the effects of quiz frequency and placement. These effects should be far less pronounced in laboratory studies investigating open-book versus closed-book practice tests, in which participants were presented with learning content that was not important to reach a goal of personal relevance for them (Agarwal etal., 2008; Agarwal and Roediger, 2011; Roelle and Berthold, 2017). In addition, eight of the 59 students missed the exam due to illness. Schwieren J., Barenberg J., Dutke S. (2017). A potential methodological issue is that most studies compare testing only to a passive restudy control condition, as in the typical procedure described above (Kornell etal., 2012). (2018). Students who performed the practice tests in the closed-book condition performed significantly better (M=45.09% correct answers, SD=14.71) than those in the open-book condition (M=36.73% correct answers, SD=9.30). The results of Gharib etal. Agarwal P. K., Roediger H. L., III (2011). There is also one field experiment investigating the benefits of open-book versus closed-book tests, which was conducted with students in an introductory psychology course (Gharib etal., 2012). One week after the last practice test, a closed-book surprise quiz took place, of which students were not informed beforehand and which was introduced as a practice quiz for the final module exam. Different degrees of motivational effects might therefore explain the difference between our findings and earlier laboratory studies. In this case, potential differences due to the actual exam might have been obscured because students prepare differently for closed-book and for open-book exams (e.g., Theophilides and Koutselini, 2000). Different rates of forgetting following study versus test trials, ber Lesen und Rezitieren in ihren Beziehungen zum Gedchtnis. The https:// ensures that you are connecting to the
The critical manipulation concerned seven lessons.
Examining the testing effect with open-and closed-book tests. The fact that the proportion of students attending the surprise quiz was lower in the closed-book group than in the open-book group points to the possibility that the students in the two groups differed in some respect, for instance with respect to motivation. Further 6weeks later, the final module exam took place. At the final module exam, which took place 8weeks after the surprise quiz, 51 participants showed up (25 from the open-book group and 26 from the closed-book group)1. 8600 Rockville Pike Yet, the study reported here was conducted in line with the ethical standards of the American Psychological Association (APA) and the German Association of Psychology (DGPs), which are in accordance with the Declaration of Helsinki. In this case, the higher scores for the closed-book group in the surprise quiz might also be due to selectivity of the sample because only the more motivated students in the closed-book group showed up for the session in which the surprise quiz took place. Test-enhanced learning in the classroom: long-term improvements from quizzing. In some of the experiments, participants do not have the opportunity to reread the text after the testing phase and no feedback is provided (e.g., Roediger and Karpicke, 2006; Rummer etal., 2017, Exp. Forty-six participants showed up for the surprise quiz, 27 from the open-book group and 19 from the closed-book group. Many (laboratory) studies concerned with the testing effect follow a typical procedure: First, participants are presented with the learning material, regularly a brief science text. At the beginning of each lesson, students could ask questions regarding the previous lesson.
Even though students did not receive direct feedback after each practice test, performance in the closed-book tests should provide more diagnostic and more negative feedback than performance in the open-book tests in terms of noticing failures to retrieve the information from memory or less confidence in ones answers. Nonetheless, the module exam data should be taken with a pinch of salt. Surprisingly, however, previous studies comparing open-book and closed-book tests (with feedback), have not found any differences. Given the small number of participants, the restriction to two seminars in cognitive psychology, and the limitations outlined above, we are cautious here. The testing effect is one of the most prominent instances of difficulties during learning, being desirable for long-term learning (Bjork, 1994; Roediger and Karpicke, 2006). Thus, the test format of the course exam should not have influenced students behavior between the manipulation and the final test. Such potential differences either might have existed beforehand or might even be a consequence of the experimental manipulation, if the students who had to perform the harder versions of the tests enjoyed the courses less. Here, the manipulation was slightly different in that actual exam types (closed-book, open-book, and cheat sheet) were manipulated rather than no-stakes or low-stakes practice tests. Department of Psychology, University of Kassel, Kassel, Germany, 2 Given that our study took place in a field setting and covered learning matter that was relevant for passing an exam, what happened in the practice tests could also have influenced students behavior both in and outside the classroom. The quiz group scored significantly higher than the control group on the same questions, on similar questions, and even on new questions. This is, however, not the case for the module exam, for which participation was obligatory and for which group sizes did not differ. Loftus E. F., Miller G., Burns H. J. The .gov means its official. More than a century ago, first studies have demonstrated that active retrieval of previously studied information is a more useful strategy to prevent forgetting than passive restudying (e.g., Witasek, 1907; Abott, 1909; Khn, 1914; Gates, 1917). Remarkably, though, these beneficial effects did not always show up at the typical final test delay of 1week, but only after a 2weeks delay (Rummer etal., 2017, Exp. What is needed is more research comparing open-book and closed-book testing in field experiments using real learning materials. Consequently, the delay between practice tests and surprise quiz ranged from 1 to 7weeks.
The first and the last lessons of the experiment covered one psychological journal paper each, the remaining five lessons covered two papers each (see Table 1). 2). It is, however, noteworthy that the pattern looked slightly different when only the critical delayed final test is considered (Roelle, personal communication).
Under these conditions, we expect a retrieval practice effect, that is, better final test performance for the closed-book group than for the open-book group. They compared reading assignments and reading assignments plus quizzes on textbook chapters that were otherwise not covered in class. Semantic integration of verbal information into a visual memory, Retrieving essential material at the end of lectures improves performance on statistics exams, A split-attention effect in multimedia learning: evidence for dual processing systems in working memory. Meyersburg C. A., Bogdan R., Gallo D. A., McNally R. J. The critical manipulation concerned the final 10min of seven lessons in each group. Test-enhanced learning: taking memory tests improves long-term retention, Creating false memories: remembering words not presented in lists, Effects of incorporating retrieval into learning tasks: the complexity of the tasks matters, The effect of testing versus restudy on retention: a meta-analytic review of the testing effect. (2011). The related and unrelated questions were not matched with respect to difficulty and could not be matched with respect to the delay between coverage in class and the exam. One reason for the lack of a consistent effect might be the combination of an elaborative control condition and a (rather) short retention interval. The main dependent variable was performance on a surprise quiz in week 8. Therefore, we opted against trying to persuade an uninformed teacher who might have put less effort into following the script and supervising the student presentations. In particular, we think that the two versions of the practice tests could have had indirect effects on learning performance via students motivation. The lessons followed the exact same predetermined script in both groups and student presentations and printouts were strictly supervised to make the seminars as similar as possible. In the first 5min of each lesson, students could ask questions regarding the previous lesson (including regarding the practice questions). The site is secure. This text has to be learned under two different conditions: In the control condition, participants are instructed to read the text several times; in the critical testing condition, however, they have to read the text once and then recall its content (and sometimes for several times, e.g., Roediger and Karpicke, 2006, Exp. A comparison of the effects of exam types on performance, retention, and anxiety, Retrieval practice produces more learning than elaborative studying with concept mapping, Presentation modality and mode of recall in verbal false memory. The two seminars (Cognition) served to accompany the lecture Introduction to cognitive psychology, with which they formed the module Cognitive psychology. Right from the beginning of the lecture, students knew about the final exam at the end of the lecture. Based on these considerations, it is not justified to interpret our findings solely in terms of a direct retrieval practice effect. Also in the module exam in which different, but related questions had to be answered, a closed-book advantage was found. Bethesda, MD 20894, Web Policies Roediger H. L., III, Karpicke J. D. (2006). Overall, these studies indicate that the testing effect translates to naturalistic learning settings. Another indirect effect of the closed-book test concerns the preparation and repetition of the learning matter at home. 1); in other studies, the opportunity to reread the text after the testing phase provides indirect feedback (e.g., McDaniel etal., 2009; Karpicke and Blunt, 2011; Rummer etal., 2017, Exp. Camerer C. F., Dreber A., Holzmeister F., Ho T.-H., Huber J., Johannesson M., et al. This resulted in 30 participating students in the open-book group and 29in the closed-book group, 46 of which showed up for the surprise quiz, 27 from the open-book group (mean age=20.96years, 3 males plus 1 person who did not provide gender information) and 19 from the closed-book group (mean age=20.53years, 1 male). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. The The testing effect was not solely investigated in lab studies but also in schools (e.g., Carpenter etal., 2009; McDaniel etal., 2011; Roediger etal., 2011) and universities (e.g., Lyle and Crawford, 2011; Batsell etal., 2017). The two seminars were taught by the same instructor (AS, the third author) and the lessons followed the exact same predetermined script in both groups and used the same literature. Rummer R., Schweppe J., Gerst K., Wagner S. (2017).
(2008), p.872, speculate that any positive effects of closed-book tests may be even more powerful in a repeated testing design such that repeated closed-book tests with feedback may result in better long-term retention than repeated open-book tests. The present field study compared open-book testing and closed-book testing in two (parallel) introductory university courses in cognitive psychology. Adesope O. O., Trevisan D. A., Sundararajan N. (2017).
Here, participants who practiced with summarization questions showed a closed-book advantage for all question types in the final test, while the open-book advantage for practicing with inference questions was restricted to the inference questions and the transfer questions. Unfortunately, we do not have any data to test this assumption empirically. None of the participants received direct feedback regarding test performance but all of them kept the learning materials and were encouraged to restudy the materials at home. Seven lessons of each course ended with a practice test on the current lesson. This is probably the case because participation in the seminar lessons (including the one with the surprise test) was voluntary, while taking the exam was obligatory. Thus, final test delays of up to 1week might have been too short to detect a retrieval practice effect compared to open-book testing. Agarwal and Roediger (2011) replicated these findings for comprehension questions (identical with the practice questions) and transfer questions that required participants to indicate why a certain detail from the passage was true (after a 2-day final test delay). Accessibility
The answers were scored by the first author of the manuscript (RR) who was blind to the experimental condition. (2011). For the question of a retrieval practice effect, students performance in a surprise retention quiz 2weeks after the exams constituted the critical dependent variable.
Schwieren etal., 2017). testing effect, retrieval practice, open-book test, closed-book test, classroom study, On the analysis of the factor of recall in the learning process. Therefore, it is plausible that the advantage for the closed-book condition over the open-book condition is (at least partly) due to longer and/or more intensive repetition of the learning matters at home. In addition, we examined performance on the multiple choice questions of the official final module exam, which took place 8weeks after the surprise quiz. (2017). No use, distribution or reproduction is permitted which does not comply with these terms. But is it justified to give a strict recommendation to use closed-book tests instead of open-book tests based on our experiment? The ANOVA revealed a significant main effect for practice test type [F(1,44)=5.15, p=0.028, p2=0.11] whereas the main effect for final test question type and the interaction did not reach significance (both Fs<1). Maximally, participants could reach 32 points for short-answer questions (same format questions) and 32 points for true-false statements (changed format questions). Even with this covariate included, there was a significant advantage for the closed-book group over the open-book group, F(1,48)=9.40, p=0.004, p2=0.16. Federal government websites often end in .gov or .mil. In addition, these experiments should track the time learners spend at home with the provided learning materials. Carpenter S. K., Pashler H., Cepeda N. J. The total score (in percent) served as the main dependent variable. To control for differences in general ability, we also analyzed the effect of practice test type on performance for the related questions including performance on the unrelated questions as a covariate. In this setting, it is not very probable that the students relearned the materials following the course exam, passing which was their main purpose. The final test, which took place 1week later, did not reveal any systematic differences between open-book tests and closed-book tests with feedback. Performance on these quizzes accounted for 8% of students overall course grade. 3As stated above, the final module exam data could not be related to the surprise quiz data because participants did not provide their codes for the exam (which would have compromised anonymity). also Camerer etal., 2018) or note-taking (McDaniel etal., 2009; Dobson and Linderholm, 2015; Nguyen and McDaniel, 2016; Rummer etal., 2017). The surprise quiz data were scored by a student assistant who was blind to the experimental condition.
What is more, the different attendance rates for the surprise quiz should have inverse effects on the exam data: As more students in the open-book group participated in the surprise quiz, this group should have benefited disproportionately high from the additional retrieval opportunity and the feedback in the surprise quiz. As it was not possible to present the same questions at the second exam date, only the data for those students who attended the first exam were included in our study. Here, students received between two and three short-answer questions on a sheet of paper that addressed central aspects of the papers that had been covered in the current lesson (16 questions in total). (2012). ; see Dunlosky etal., 2013).
Ecological validity of the testing effect: the use of daily quizzes in introductory psychology, Retrieval practice and the maintenance of knowledge, Memory and metamemory considerations in the training of human beings, Learning with retrieval-based concept mapping, Why do some children benefit more from testing than others? 2) different conditions in a within-subject design, four of which included practice tests: study plus closed-book test, study plus closed-book test with feedback, study plus open-book test, and study and take an open-book test simultaneously. Most researchers assume that the testing effect is due to more than one mechanism and that the role of these mechanisms depends on the characteristics of the specific testing procedure as well as on the learning materials and setting (Rowland, 2014). From a retrieval practice perspective, the lack of such a closed-book advantage effect is surprising. To our knowledge, all previous studies contrasting open-book and closed-book tests have applied only a single testing phase with a single text. This, however, raises the question why we found a retrieval practice effect but other researchers did not (Agarwal etal., 2008; Agarwal and Roediger, 2011). The z-transformed raw values were submitted to a 22 mixed ANOVA with the between-subjects factor practice test type (closed-book vs. open-book) and the within-subject factor final test question type (short-answer vs. true/false). Performance in a final module exam, which took place 8weeks after the surprise quiz and included a number of questions addressing the learning matter taught in the seven critical seminar lessons, served as an additional dependent variable. (2003). Morris C. D., Bransford J. D., Franks J. J. As suggested above, this final test delay might be too short to detect an effect of retrieval practice compared to a strong control condition (see also Rummer etal., 2017).