Tag Archives | Summer Learning Loss

Findings on Summer Learning Loss Often Fail to Replicate, Even in Recent Data

Joseph Workman, Paul T. von Hippel, Joseph Merry

Sociological Science March 30, 2023
10.15195/v10.a8


It is widely believed that (1) children lose months of reading and math skills over summer vacation and that (2) inequality in skills grows much faster during summer than during school. Concerns have been raised about the replicability of evidence for these claims, but an impression may exist that nonreplicable findings are limited to older studies. After reviewing the 100-year history of nonreplicable results on summer learning, we compared three recent data sources (ECLS- K:2011, NWEA, and Renaissance) that tracked U.S. elementary students’ skills through school years and summers in the 2010s. Most patterns did not generalize beyond a single test. Summer losses looked substantial on some tests but not on others. Score gaps—between schools and students of different income levels, ethnicities, and genders—grew on some tests but not on others. The total variance of scores grew on some tests but not on others. On tests where gaps and variance grew, they did not consistently grow faster during summer than during school. Future research should demonstrate that a summer learning pattern replicates before drawing broad conclusions about learning or inequality.
Creative Commons LicenseThis work is licensed under a Creative Commons Attribution 4.0 International License.

Joseph Workman: University of Missouri, Kansas City
E-mail: workmanj@umkc.edu

Paul T. von Hippel: University of Texas, Austin
E-mail: paulvonhippel@utexas.edu

Joseph Merry: Furman University
E-mail: joseph.merry@furman.edu

Acknowledgments: Replication code is available at https://osf.io/f4jrb/. Unfortunately, we cannot share vendor data from NWEA and Renaissance Learning, but the ECLS-K:2011 is available to any researcher with a restricted-data license.

  • Citation: Workman, Joseph, Paul T. von Hippel, and Joseph Merry. 2023. “Findings on Summer Learning Loss Often Fail to Replicate, Even in Recent Data.” Sociological Science 10: 251-285.
  • Received: January 6, 2023
  • Accepted: February 6, 2023
  • Editors: Arnout van de Rijt, Richard Breen
  • DOI: 10.15195/v10.a8


0

Do Test Score Gaps Grow Before, During, or Between the School Years? Measurement Artifacts and What We Can Know in Spite of Them

Paul T. von Hippel, Caitlin Hamrock

Sociological Science, January 24, 2019
10.15195/v6.a3


Do test score gaps between advantaged and disadvantaged children originate inside or outside schools? One approach to this classic question is to ask (1) How large are gaps when children enter school? (2) How much do gaps grow later on? (3) Do gaps grow faster during school or during summer? Confusingly, past research has given discrepant answers to these basic questions.

We show that many results about gap growth have been distorted by measurement artifacts. One artifact relates to scaling: Gaps appear to grow faster if measurement scales spread with age. Another artifact relates to changes in test form: Summer gap growth is hard to estimate if children take different tests in spring than in fall.

Net of artifacts, the most replicable finding is that gaps form mainly in early childhood, before schooling begins. After school begins, most gaps grow little, and some gaps shrink. Evidence is inconsistent regarding whether gaps grow faster during school or during summer. We substantiate these conclusions using new data from the Growth Research Database and two data sets used in previous studies of gap growth: the Beginning School Study and the Early Childhood Longitudinal Study, Kindergarten Cohort of 1998–1999.

Creative Commons LicenseThis work is licensed under a Creative Commons Attribution 4.0 International License.

Paul T. von Hippel: LBJ School of Public Affairs, University of Texas at Austin
E-mail: paulvonhippel.utaustin@gmail.com

Caitlin Hamrock: E3 Alliance
E-mail: chamrock@e3alliance.org

Acknowledgements: We thank Mina Kumar for research assistance. We thank the William T. Grant Foundation and the Institute for Urban Policy Research and Analysis for grants supporting this work.

  • Citation: von Hippel, Paul T., and Caitlin Hamrock. 2019. “Do Test Score Gaps Grow Before, During, or Between the School Years? Measurement Artifacts and What We Can Know in Spite of Them.” Sociological Science 6: 43-80.
  • Received: February 20, 2018
  • Accepted: July 23, 2018
  • Editors: Jesper Sørensen, Stephen Morgan
  • DOI: 10.15195/v6.a3


0
SiteLock