Tag Archives | Replicability

Findings on Summer Learning Loss Often Fail to Replicate, Even in Recent Data

Joseph Workman, Paul T. von Hippel, Joseph Merry

Sociological Science March 30, 2023
10.15195/v10.a8


It is widely believed that (1) children lose months of reading and math skills over summer vacation and that (2) inequality in skills grows much faster during summer than during school. Concerns have been raised about the replicability of evidence for these claims, but an impression may exist that nonreplicable findings are limited to older studies. After reviewing the 100-year history of nonreplicable results on summer learning, we compared three recent data sources (ECLS- K:2011, NWEA, and Renaissance) that tracked U.S. elementary students’ skills through school years and summers in the 2010s. Most patterns did not generalize beyond a single test. Summer losses looked substantial on some tests but not on others. Score gaps—between schools and students of different income levels, ethnicities, and genders—grew on some tests but not on others. The total variance of scores grew on some tests but not on others. On tests where gaps and variance grew, they did not consistently grow faster during summer than during school. Future research should demonstrate that a summer learning pattern replicates before drawing broad conclusions about learning or inequality.
Creative Commons LicenseThis work is licensed under a Creative Commons Attribution 4.0 International License.

Joseph Workman: University of Missouri, Kansas City
E-mail: workmanj@umkc.edu

Paul T. von Hippel: University of Texas, Austin
E-mail: paulvonhippel@utexas.edu

Joseph Merry: Furman University
E-mail: joseph.merry@furman.edu

Acknowledgments: Replication code is available at https://osf.io/f4jrb/. Unfortunately, we cannot share vendor data from NWEA and Renaissance Learning, but the ECLS-K:2011 is available to any researcher with a restricted-data license.

  • Citation: Workman, Joseph, Paul T. von Hippel, and Joseph Merry. 2023. “Findings on Summer Learning Loss Often Fail to Replicate, Even in Recent Data.” Sociological Science 10: 251-285.
  • Received: January 6, 2023
  • Accepted: February 6, 2023
  • Editors: Arnout van de Rijt, Richard Breen
  • DOI: 10.15195/v10.a8


0
SiteLock