- Intervention: Streamlined personal assistance in completing a college financial aid application, provided to low and moderate income families with a dependent child at or near college age.
- Evaluation Methods: A large, multi-site randomized controlled trial.
- Key Findings: A sizable increase in college attendance and persistence over the 3½-4 years following the intervention – e.g., 29% greater likelihood of attending college for two consecutive years.
Download a printable version of this evidence summary (pdf, 4 pages)
The Top Tier initiative’s Expert Panel has identified this intervention as Top Tier.
The Panel finds that this intervention meets the Congressional Top Tier Evidence standard, defined as: Interventions shown in well-designed and implemented randomized controlled trials, preferably conducted in typical community settings, to produce sizable, sustained benefits to participants and/or society.
Description of the Intervention
To be eligible for federal financial aid for college, applicants must normally complete a detailed form – the Free Application for Federal Student Aid, or “FAFSA” – which in 2008 was eight pages in length with over 100 questions, and included three additional worksheets with nearly 40 questions. In this intervention, H&R Block – a national tax preparation company – provided streamlined personal assistance in completing the FAFSA to customers at their tax preparation offices who were low or moderate income and had a family member at or near college age without a bachelor’s degree.
Key elements of the assistance are summarized as follows: (i) specially-designed software used the family’s tax return data to answer most items on the FAFSA; (ii) the H&R Block tax professional conducted a brief (10 minute) interview with the family to answer the remaining questions; (iii) the tax professional provided immediate estimates of the amount of financial aid the targeted family member was eligible for, and the net tuition cost of four nearby public colleges; (iv) if all FAFSA information was complete, the tax professional offered to submit the FAFSA electronically to the U.S. Department of Education; and (v) if not all information was complete, a call center phoned the family after the interview to collect answers to the remaining questions and then facilitated submission of the FAFSA to the Department of Education.
As delivered in the study described below, this intervention cost approximately $90 per program participant, in 2012 dollars. It also increased federal financial aid payments by an average of $375 per participant, as described below.
The intervention was delivered as part of a time-limited demonstration project; resources to foster program dissemination (e.g., a website) have not yet been developed.
EVIDENCE OF EFFECTIVENESS
This summary of the evidence is based on a systematic search of the literature, and correspondence with leading researchers, to identify all well-conducted randomized controlled trials of this intervention. Our search identified one such trial. What follows is a summary of the study design and the program’s effects on the main outcomes measured in the study, including any such outcomes for which no or adverse effects were found.
Overview of the Study Design: A multi-site randomized controlled trial launched in 2008 at 156 H&R Block tax preparation offices across most of Ohio and the Charlotte, North Carolina area.
In the trial, families that had just completed their taxes in the H&R Block office were screened by computer software to identify those with incomes less than $45,000 per year, and a family member at or near college age without a bachelor’s degree (the “targeted participant”). If the targeted participant was interested in learning more about college and/or participating in the study, he or she provided verbal informed consent, and then was randomly assigned to one of two groups:
- A FAFSA treatment group that received H&R Block’s FAFSA assistance intervention described above; or
- A control group, for whom the tax professional provided only a brochure on the importance of higher education and general information on college cost and financial aid.1
The overall sample contained the following subsamples of interest, which were specified prior to study implementation: (i) a “dependent” subsample, comprised of high school seniors and recent graduates who were financially dependent on their parents; and (ii) an “independent” subsample, comprised of adults age 24-30 who had not completed college.2 The Top Tier Panel identified only the effects for the dependent subsample as meeting the Top Tier evidence standard, and our write-up therefore focuses on this subsample. The effects found for the independent subsample were substantially smaller, and are not summarized here.
In the dependent subsample, approximately 1,045 targeted youth averaging 17.7 years of age were randomly assigned. Approximately 57% were female, 55% were white, and 39% were African American. Their family’s average income was about $23,000.
Effects of the intervention during the 3½ -4 years after random assignment:
These are the effects on the dependent subsample, described above. All effects shown are statistically significant at the 0.05 level unless stated otherwise.
Compared to the control group, youth in the FAFSA treatment group:
- Were 24% more likely to attend college in the first year following random assignment (42.3% of the FAFSA group attended college vs. 34.2% of the control group).
- Were 29% more likely to attend college for two consecutive years (36.0% of the FAFSA group vs. 28.0% of the control group).
- Were 10% more likely to attend college in the first, second, or third year after random assignment (53.3% of the FAFSA group vs. 48.5% of the control group). However, this effect was not statistically significant (p=0.18).
- Spent 20% more time in college over the follow-up period (an average of 13.7 months for the FAFSA group vs. 11.4 months for the control group).
- Were 36% more likely to receive a federal need-based (“Pell”) grant in the first year following random assignment (40.2% of the FAFSA group vs. 29.6% of the control group).
- Received an average of $375 more in federal Pell grants over the follow-up period. The statistical significance of this effect is not reported.
Discussion of Study Quality
- This was a large, multi-site study that evaluated the intervention as delivered at scale in 156 H&R Block tax preparation offices in two states, thus providing evidence of the intervention’s effectiveness under real-world implementation conditions.
- The study had a reasonably long-term follow-up (3½-4 years after random assignment) and moderate sample attrition. In the dependent subsample described above, outcomes were obtained for approximately 75% of the individuals randomly assigned, and follow-up rates were virtually the same for the FAFSA treatment group and control group.3,4
- The FAFSA treatment group and control group members in the follow-up sample were highly similar in their observable pre-program characteristics (e.g., demographics, educational expectations). This is true of both the overall sample and the dependent subsample described above.
- The study appropriately sought to measure outcomes for all individuals assigned to the FAFSA treatment group, regardless of whether they completed or submitted a FAFSA form (i.e., the study used an “intention-to-treat” analysis).
- All outcomes were measured using administrative data collected by independent organizations – namely, the U.S. Department of Education (for data on financial aid awarded) and the National Student Clearinghouse (for data on college attendance).5
SUMMARY OF THE INTERVENTION’S BENEFITS AND COSTS
If taxpayers fund implementation, what benefits to society can they expect to result, and what would be their net cost? The following table provides a summary.
Benefits To Society
Net Cost To Taxpayers
* For example, the study found that the intervention caused a modest increase in participants’ receipt of federal student loans (as opposed to grants), the dollar amount of which is not reported. Because such loans are subsidized, they likely generated some additional cost to the government. In addition, the intervention may have caused an increase in participants’ receipt of financial aid from state government (e.g., the Ohio College Opportunity Grants). The study did not measure the intervention’s effect on such aid.
(Click on linked authors’ names for their contact information)
- Bettinger, Eric P., Bridget Terry Long, Philip Oreopoulos, and Lisa Sanbonmatsu. “The Role of Application Assistance and Information in College Decisions: Results from the H&R Block FAFSA Experiment.” Forthcoming in the Quarterly Journal of Economics, August 2012, vol. 127, no. 3.
- Bettinger, Eric P., Bridget Terry Long, Philip Oreopoulos, and Lisa Sanbonmatsu. “The Role of Application Assistance and Information in College Decisions: Results from the H&R Block FAFSA Experiment,” July 2011, linked here. (This paper reports on the intervention’s short-term effects, in the fall semester following the intervention.)
2 The study also had another subsample of interest – high school sophomores and juniors age 15-17 not yet eligible to apply for the FAFSA, which the study included to examine the effects of early information on college attendance. The researchers plan to report findings for this subsample in future work, as its members reach college age.
3 Sample attrition in this study was caused by the informed consent process, as follows. While the study appropriately obtained families’ verbal consent to study participation prior to random assignment, families were not asked to formally sign the written consent form until the end of their interview with the tax professional. In some cases the tax professional mistakenly sent the consent form home with the families, preventing the study from obtaining their outcome data. This produced equivalent rates of sample loss in the treatment and control groups.
4 The follow-up rate for the entire dependent subsample – including the high school seniors and recent graduates that we focus on in this write-up, as well as the younger students age 15-17 described in footnote 2 – was 75.4%, and was equivalent for the treatment and control groups (75.6% and 75.1% respectively). H&R Block did not provide the study researchers with the follow-up rate specifically for the high school seniors and recent graduates, but there is strong reason to believe it closely mirrored the above rates, as follows: (i) the follow-up rates were nearly identical across all samples that the study reported on – including the full sample, the subsample of independent adults, the entire dependent subsample, and the treatment and control groups within these subsamples (rates range from 75.1% to 77.7%) – providing reasonable confidence that similar rates apply to the subsample of dependent high school seniors and recent graduates; and (ii) for this subsample, follow-up data were obtained for virtually the same number of treatment and control group members (390 and 398 students, respectively), mirroring the study’s one-to-one random assignment ratio and confirming that sample loss was likely equivalent for the two groups.
5 The National Student Clearinghouse (NSC) data encompass more than 3,300 colleges, enrolling 93% of U.S. college students. This includes 92% of all enrollments in Ohio and North Carolina. For one of the study outcomes we summarize in this write-up – college attendance in the first year following random assignment – data from the Ohio Board of Regents are used to complement the NSC data. (The Board of Regents data cover some Ohio vocational colleges not included in the NSC.)