Blog Viewer

ASA provides National Academies Panel statistical perspectives on furthering education research at Department of Education

By Steve Pierson posted 10-11-2021 10:58

After consulting with several education research experts in its membership, the ASA provided the National Academies panel considering the future of education research at the Institute of Education Sciences in the U.S. Department of Education several recommendations from a statistical perspective.  The text of the letter is below; a link to the pdf will be provided when the ASA website is updated. 

October 5, 2021

Adam Gamoran
Chair, Committee on The Future of Education Research at the Institute of Education Sciences in the US Department of Education
National Academies of Sciences, Engineering, and Medicine
[Transmitted electronically]

Dear Dr. Gamoran,

We are pleased to have the opportunity to provide input to your panel considering the future of education research at the Institute of Education Sciences (IES) in the US Department of Education. As the science of learning from data, statistics is fundamental to IES’s mission to “provide scientific evidence on which to ground education practice and policy.” The role of statistics in education research starts with framing the problem and designing the study and continues through analyzing and interpreting the data and communicating the findings. We believe emphatically that engagement of statisticians and the statistical perspective results in better science.

The tremendous strides in education research over the past 25 years underscore the important role of statistics both through the Statistical and Research Methodology in Education (SRME) program and more broadly. One manifestation of this success is the What Works Clearinghouse (WWC), which provides decision-makers with information about effective interventions in reading, math, science, dropout prevention, and more. Many of these advances, and the confidence in the studies reported in the WWC, would not be possible without strong statistical methods underpinning the study designs and analyses and a solid research base for understanding which designs and analyses yield accurate results.

Through the SRME program, we appreciate that IES has recognized—and indeed, fostered—the importance of statistical methodology grounded in and disciplined by the context of education research. Recognizing the need for statistical advances that respond to the specific challenges faced by the field, SRMA-funded projects have ensured the following:
• Principled analyses of primary data collected in empirical studies
• More informative use of large-scale survey data routinely collected by IES
• Advances in methods for characterizing findings and synthesizing bodies of evidence from multiple studies
• Advanced power-analysis methodologies, with assumptions informed by empirical data, to ensure the money spent on research is put to good use
• Robust methods to determine what interventions work best for whom—again, a particularly important topic in times of limited resources

For IES to continue furthering education research, we recommend thoughtful implementation of the following statistical perspectives:
• More strategic use of existing administrative data, and new modalities for collecting and processing data, to provide practitioners and decision-makers with up-to-date information on student progress
• Study designs representing in more detail the heterogeneity of student and school characteristics to better inform local decisions
• Improved systems for archiving, accessing, and reanalyzing data collected from completed primary studies to better address emerging policy questions and improve the relevance of available evidence
• Continued development and improvement of methods for evaluating systemic and structural-level reforms that may not be easily randomized or evaluated using traditional quasi-experimental approaches currently examined by the WWC
• Further use of statistical methods and strategies for helping identify study design and analysis approaches most likely to yield accurate results, as has been done for the WWC to this point
• Development of methods that monitor or measure systems of discrimination
• Increased support of programs, workshops, and training initiatives in statistical and methodological research in education settings both generally and to increase the diversity of researchers engaged in statistical and methodological research in education settings

The following experts provided input and time to craft these recommendations: Vivian Wong, University of Virginia; Tracy Sweet, University of Maryland; Elizabeth Stuart, The Johns Hopkins University; James Pustejovsky, University of Wisconsin, Madison; and Luke Miratrix, Harvard University. My comments here echo the comments of some of those who presented to this committee over the summer.

Thank you for your consideration.

Ron Wasserstein
Executive Director, American Statistical Association