What Works in Social Policy?
Programs with Credible Evidence of Important Effects on People’s Lives
Most social programs are unfortunately found not to produce the hoped-for effects when rigorously evaluated. When evaluated in scientifically-rigorous studies, programs in K-12 education, employment and training, crime prevention, and other areas are too often found ineffective or marginally effective. Programs that produce sizable effects on important life outcomes exist, but tend to be the exception – a pattern that occurs not just in social spending, but in other fields where rigorous evaluations are conducted, such as medicine and business.1 Meanwhile, respected government measures show that the United States has made little progress since the late 1970s in key areas such as reducing poverty2 and increasing K-12 educational achievement.3
To Help Address This Problem:
This site seeks to identify those social programs shown in rigorous studies to produce sizable, sustained benefits to participants and/or society. The purpose is to enable policy officials and other readers to readily distinguish these programs from the many others that claim to have such evidence. Although we support many types of research to develop and identify promising programs, this site focuses on the results of well-conducted randomized controlled trials (RCTs), which are widely regarded as the strongest method of evaluating program effectiveness.4 The site also focuses on whether such studies show sizable, sustained effects on outcomes of clear policy importance (e.g., high school graduation, workforce earnings, teen pregnancies) and not just intermediate outcomes (e.g., children’s ability to recognize letters or numbers, positive parenting practices, take-up of services) that may or may not lead to important outcomes. Our focus on important, sustained effects distinguishes this site from many other clearinghouses of evidence-based programs.
The Programs We’ve Identified Are Linked Below
We have identified these programs through our systematic monitoring of all rigorous evaluations published or posted on-line across all areas of social policy. Programs identified as having the most credible evidence of effectiveness, based on careful review in consultation with outside experts, are categorized as “Top Tier” or “Near Top Tier” (see sidebar for definitions). The other listed programs have been found promising but not yet Top Tier or Near Top Tier due to study limitations (such as only short-term follow-up) that underscore the need for additional testing prior to larger-scale implementation. As a nonprofit philanthropic foundation, we receive no financial benefit from any program, enabling us to serve as an impartial reviewer of the evidence.5
Click Here for the Full List of Programs, with Brief Abstracts
Interventions Organized by Policy Area:
1Medicine: John P. A. Ioannidis, “Contradicted and Initially Stronger Effects in Highly Cited Clinical Research,” Journal of the American Medical Association, vol. 294, no. 2, July 13, 2005, pp. 218-228. Mohammad I. Zia, Lillian L. Siu, Greg R. Pond, and Eric X. Chen, “Comparison of Outcomes of Phase II Studies and Subsequent Randomized Control Studies Using Identical Chemotherapeutic Regimens,” Journal of Clinical Oncology, vol. 23, no. 28, October 1, 2005, pp. 6982-6991. John K. Chan et. al., “Analysis of Phase II Studies on Targeted Agents and Subsequent Phase III Trials: What Are the Predictors for Success,” Journal of Clinical Oncology, vol. 26, no. 9, March 20, 2008. Michael L. Maitland, Christine Hudoba, Kelly L. Snider, and Mark J. Ratain, “Analysis of the Yield of Phase II Combination Therapy Trials in Medical Oncology,” Clinical Cancer Research, vol. 16, no. 21, November 2010, pp. 5296-5302. Jens Minnerup, Heike Wersching, Matthias Schilling, and Wolf Rüdiger Schäbitz, “Analysis of early phase and subsequent phase III stroke studies of neuroprotectants: outcomes and predictors for success,” Experimental & Translational Stroke Medicine, vol. 6, no. 2, 2014.
Social Spending: Coalition for Evidence-Based Policy, Randomized Controlled Trials Commissioned by the Institute of Education Sciences Since 2002: How Many Found Positive Versus Weak or No Effects, July 2013, linked here. Laura and John Arnold Foundation, Request for Proposals: Low-Cost Randomized Controlled Trials to Drive Effective Social Spending, December 2015, appendix p. 3, linked here. Isabel Sawhill and Jon Baron, “We Need A New Start for Head Start,” Education Week, March 3, 2010, linked here.
Business: Jim Manzi, Uncontrolled: The Surprising Payoff of Trial-and-Error for Business, Politics, and Society, Perseus Books Group, New York, 2012, pp. 128 and 142. Jim Manzi, Science, Knowledge, and Freedom, presentation at Harvard University’s Program on Constitutional Government, December 2012, linked here.
2Proctor, Bernadette D., Jessica L. Semega, and Melissa A. Kollar, U.S. Census Bureau, Current Population Reports, P60-256(RV), Income and Poverty in the United States: 2015, U.S. Government Printing Office, Washington, DC, 2016. U.S. Census Bureau, Official and National Academy of Sciences (NAS) Based Poverty Rates: 1999 to 2011, 2012. Kathleen Short, U.S. Census Bureau, HHES Division, Estimating Resources for Poverty Measurement, 1993 – 2003, 2005. Panel on Poverty and Family Assistance, National Academy of Sciences, Measuring Poverty: A New Approach, 1995, pp. 31-36. Christopher Wimer, Liana Fox, Irv Garfinkel, Neeraj Kaushal, and Jane Waldfogel, Trends in Poverty with an Anchored Supplemental Poverty Measure, December 2013.
3Cornman, S.Q., and Zhou, L. Revenues and Expenditures for Public Elementary and Secondary Education: School Year 2013-14 (Fiscal Year 2014) (NCES 2016-301). U.S. Department of Education. Washington, DC: National Center for Education Statistics, 2016. Richard H. Barr, Revenues and Expenditures for Public Elementary and Secondary Education, 1973-74 (NCES-76-140). U.S. Department of Health, Education & Welfare, National Institute of Education. Washington, DC: National Center for Education Statistics, 1976.
4Institute of Education Sciences (of the U.S. Department of Education) and National Science Foundation, Common Guidelines for Education Research and Development, August 2013, linked here. National Research Council and Institute of Medicine, Preventing Mental, Emotional, and Behavioral Disorders Among Young People: Progress and Possibilities, Mary Ellen O’Connell, Thomas Boat, and Kenneth E. Warner, Editors (Washington DC: National Academies Press, 2009), recommendation 12-4, p. 371, linked here. CBO’s Use of Evidence in Analysis of Budget and Economic Policies, Jeffrey R. Kling, Associate Director for Economic Analysis, November 3, 2011, page 31, linked here. U.S. Preventive Services Task Force, “Current Methods of the U.S. Preventive Services Task Force: A Review of the Process,” American Journal of Preventive Medicine, vol. 20, no. 3 (supplement), April 2001, pp. 21-35. The Food and Drug Administration’s standard for assessing the effectiveness of pharmaceutical drugs and medical devices, at 21 C.F.R. §314.126, linked here.
5While LJAF receives no financial benefit from any intervention, in some cases we may be funders or co-funders of an RCT to evaluate an intervention listed on this website. In such cases, we disclose our role as evaluation funder within our report on that intervention.