- Intervention: Annual book fairs to provide summer reading to students in high-poverty elementary schools over three years, starting at the end of first or second grade.
- Evaluation Methods: A well-conducted randomized controlled trial.
- Key Findings: Increase in students’ reading achievement by 35-40% of a grade level, three years after random assignment.
- Other: A study limitation is that its sample was geographically concentrated in two Florida school districts. Replication of these findings in a second trial, in another setting, would be desirable to confirm the initial findings and establish that they generalize to other settings where the intervention might be implemented.
Download a printable version of this evidence summary (pdf, 4 pages)
The Top Tier initiative’s Expert Panel has identified this intervention as Near Top Tier
The Panel finds that this intervention meets the “Near Top Tier” evidence standard, defined as: Interventions shown to meet almost all elements of the Top Tier standard (i.e., well-conducted randomized controlled trials… showing sizable, sustained effects), and which only need one additional step to qualify. This category includes, for example, interventions that meet all elements of the standard in a single site, and just need a replication trial to confirm the initial findings and establish that they generalize to other sites.
Description of the Intervention
The Annual Book Fairs intervention provides students in high-poverty elementary schools with books to read over the summer, for three consecutive summers starting at the end of first or second grade. The goal is to prevent summer learning loss – specifically, the well-established tendency for low-income children’s reading achievement to fall relative to their more advantaged peers during the summer break. A number of studies have found that the loss is sizable, and may help explain the substantial and persistent reading achievement gap between more and less economically-advantaged students in the United States [e.g., Cooper et. al, 1996, Alexander et. al, 2007].
In the spring of each school year, students attend the fair, located in their school building, where they can order from among 400-600 books in a variety of genres (e.g., pop culture, series books, science). At each fair, students pick 12 books to keep as their own, which are delivered to them on the final day of school.
The study does not report the exact cost of the intervention, but indicates it was low – the main cost being that of supplying the students with 12 free books per year (which suggests a total three-year cost of $175-$225 per student).
EVIDENCE OF EFFECTIVENESS
This summary of the evidence is based on a systematic search of the literature, and correspondence with leading researchers, to identify all well-conducted randomized controlled trials of this Annual Book Fair intervention. Our search identified one such trial.1 What follows is a summary of the study design and the program’s effects on the main outcomes measured in the study, including any such outcomes for which no or adverse effects were found. All effects shown are statistically significant at the 0.05 level unless stated otherwise.
Overview of the Study Design: Randomized controlled trial of the Annual Book Fairs intervention in 17 high-poverty Florida elementary schools.
This was a randomized controlled trial of 1,713 first and second graders from 17 high-poverty elementary schools in two large school districts in Florida. Students were randomly assigned to (i) a group that received the Annual Book Fairs intervention, or (ii) a control group that did not.
Approximately 89% of the students in the sample were either African American or Hispanic, and more than 65% were eligible for free or reduced-price lunch.
Effects on the intervention approximately three years after random assignment (i.e., when most students were in fourth or fifth grade)
Compared to the control group, students in the Book Fair group –
- Scored higher on Florida’s state-mandated test of reading achievement by 0.14 standard deviations, which equates to about 35-40% of a grade level.2
- Reported reading more often during their summer breaks (the effect size is unclear, because the study used an index of reading frequency that does not lend itself to ready interpretation).
Discussion of Study Quality
- The study had low-to-moderate sample attrition and a reasonably long-term follow-up: Outcome data from the state reading assessment were collected for 79% of the Book Fair group and 76% of the control group at the three-year follow-up.
- The Book Fair and control group students in the three-year follow-up sample were similar in their observable characteristics (i.e., demographics and pre-program reading ability).
- The study evaluated the Annual Book Fairs intervention as delivered in 17 high-poverty public schools, thus providing evidence of its effectiveness under real-world implementation conditions.
- The study measured outcomes using Florida’s state-mandated reading assessment – the Florida Comprehensive Assessment Test (FCAT) – whose reliability and validity are well-established. The test primarily measures passage and word comprehension.
- The study appropriately obtained parental consent for their children to participate in the study prior to random assignment.
- A limitation of this study is that students in the Book Fairs group were dropped from the study sample at the three-year follow-up if they were no longer enrolled at one of the 17 schools conducting the book fairs (on the rationale that they probably did not receive the full intervention). By contrast, control group students were dropped only if they moved out of the school district entirely. This problem – an “intention-to-treat” violation – has the potential to undermine the equivalence of the Book Fairs and control groups. However, in this case, it appears to be at most a limited problem since students in the Book Fairs group who left their school often transferred to one of the other 17 schools providing the book fairs, and so were included in the final sample. As a result, the Book Fairs and control groups had similar rates of sample retention at the three-year follow-up (79% and 76% respectively) and remained similar in observable pre-program characteristics, as noted above.
- A second study limitation is that its sample was geographically concentrated in two Florida school districts. The Top Tier initiative’s Expert Panel believes that replication of the above findings in a second trial, conducted in another setting by the same or other researchers, would be desirable to confirm that the program is effective in other settings where it would normally be implemented.
SUMMARY OF THE INTERVENTION’S BENEFITS AND COSTS
If taxpayers fund implementation, what benefits to society can they expect to result, and what would be their net cost? The following table provides a summary.
Benefits To Society
Net Cost To Taxpayers
(Click on linked authors’ names for their contact information)
- Allington, Richard L., Anne McGill-Franzen, Gregory Camilli, Lunetta Williams, Jennifer Graff, Jacqueline Zeig, Courtney Zmach, and Rhonda Nowak. “Addressing Summer Reading Setback Among Economically Disadvantaged Elementary Students.” Reading Psychology, 2010, vol. 31, no. 5, pp. 411-427.
- Alexander, Karl L., Doris Entwisle, and Linda Olson. “Lasting Consequences of the Summer Learning Gap.” American Sociological Review, 2007, vol. 72, no. 2, pp. 167–180.
- Bloom, Howard S., Carolyn Hill, Alison Rebeck Black, and Mark Lipsey. “Performance Trajectories and Performance Gaps as Achievement Effect-Size Benchmarks for Educational Interventions.” MDRC Working Paper on Research Methodology, October 2008.
- Cooper, Harris, Barbara Nye, Kelly Charlton, James Lindsay, and Scott Greathouse. “The Effects of Summer Vacation on Achievement Test Scores: A Narrative and Meta-analytic Review.” Review of Educational Research, Fall 1996, vol. 66, no. 3, pp. 227–268.
2 Specifically, the average annual gain in reading achievement for U.S. students during fourth and fifth grades on seven nationally normed tests is 0.36 and 0.40 standard deviations respectively (see Bloom, Hill, Black, and Lipsey, 2008, referenced at the end of this summary). The difference in achievement between Book Fair and control group students, shown above, is 35-40% of these annual gains.