HIGHLIGHTS
  • Intervention: A program-delivery system in which universities partner with community teams to implement evidence-based programs for preventing youth substance abuse and other problem behaviors.
  • Evaluation Methods: A well-conducted randomized controlled trial.
  • Key Findings: At the 6.5-year follow-up (end of 12th grade) – (i) community-wide reductions of 10-35% in illicit drug use initiation by youth who were non-users in 6th grade (prior to program delivery). (ii) Moderate reductions in substance use for the full sample – non-users and users (e.g., 14% lower likelihood of past-month cigarette use).
  • Other: (i) These findings are based on the use of the PROSPER system to deliver a specific set of evidence-based programs, and may not generalize to the system’s delivery of a different program set; and (ii) a study limitation is its relatively small, homogeneous sample – 28 rural towns and small cities in two states. Thus, replication of these findings in a second trial, in another setting, would be desirable to confirm the initial results and establish that they generalize to other settings where PROSPER might be implemented.

Download a printable version of this evidence summary (pdf, 6 pages)

The Top Tier initiative’s Expert Panel has identified this intervention as Near Top Tier

The Panel finds that this intervention meets the “Near Top Tier” evidence standard, defined as:

Interventions shown to meet almost all elements of the Top Tier standard (i.e., well-conducted randomized controlled trials… showing sizable, sustained effects), and which only need one additional step to qualify. This category includes, for example, interventions that meet all elements of the standard in a single site, and just need a replication trial to confirm the initial findings and establish that they generalize to other sites.

Description of the Intervention

Promoting School-community-university Partnerships to Enhance Resilience (PROSPER) is a program-delivery system in which universities partner with community teams to implement evidence-based programs for preventing youth substance abuse and other problem behaviors. Each participating community (i.e., town or small city) forms a community team of 8-12 people, which is led by a Cooperative Extension System representative1 and a public school representative, and includes social service and health agency officials as well as community parents and youth. A state-level team of university researchers then provides the community teams with a menu of evidence-based programs,2 from which the community teams select (i) one family-focused program to deliver in 6th grade, and (ii) one school-based program to deliver in 7th grade. A prevention coordinator team, based in the university Cooperative Extension System, serves as a liaison between the community and state-level teams, providing ongoing, proactive technical assistance to community teams to optimize program delivery and sustainability.

All PROSPER communities in the study described below selected the Strengthening Families Program: 10-14 for the family-focused program. This program, delivered to parents and youth in seven sessions, focuses on enhancing parenting skills as well as youth substance refusal and other pro-social skills. For the school-based program, six PROSPER communities selected All Stars, four communities selected LifeSkills Training, and four communities selected Project Alert. All three are substance-abuse prevention programs delivered to students in classrooms, generally by a regular classroom teacher trained in that program.3

The university research team updates PROSPER’s menu of evidence-based programs over time based on new research findings regarding both the listed programs and potential alternative programs.4

The cost of implementing PROSPER in a community of 25,000 people – roughly the average population in the study described below – ranges from $168,000 to $182,000 per year, and the cost per participating youth totals about $390-$420 over 6th and 7th grade (depending on which family and school-based programs the community selects). For a larger community of 50,000 people, the cost is $197,000 to $225,000 per year, and the cost per youth is about $230-$260.5

Link to PROSPER’s website.

EVIDENCE OF EFFECTIVENESS

This summary of the evidence is based on a systematic search of the literature, and correspondence with leading researchers, to identify all well-conducted randomized controlled trials of PROSPER. Our search identified one such trial. Importantly, this trial evaluated the use of the PROSPER system to deliver the specific set of evidence-based programs described above, and its findings may not generalize to PROSPER’s delivery a different program set.

What follows is a summary of the study design and the effects found on the main outcomes measured in the study, including any such outcomes for which no or adverse effects were found. All effects shown are statistically significant at the 0.05 level (two-tailed test) unless stated otherwise.

Overview of the Study Design: Randomized controlled trial of of PROSPER in a sample of 28 rural towns and small cities in Iowa and Pennsylvania, conducted 2002-2009.

This trial had a sample of 28 rural towns and small cities (“communities”) located in Iowa and Pennsylvania, each of which contained one school district with one high school and one or more middle schools.6 The communities were grouped into matched pairs based on school district size and geographic location, and then one member of each pair was randomly assigned to participate in PROSPER and the other to continue school and community services as usual (i.e., the control group). The school districts contained a total of approximately 12,000 sixth-grade students across the 2002-2003 and 2003-2004 school years, who comprised the student sample in the trial. Approximately 50% of these students were male, 85% were white, and 77% came from two-parent families; and their families’ average annual income was approximately $67,800 (in 2014 dollars).Effects of PROSPER 6.5 years after random assignment (spring of 12th grade) for the full student sample:

The study found statistically-significant effects on three substance use outcomes, as follows –

  • An 18% reduction in the number of different types of illicit substances ever used (1.7 for the control group versus 1.4 for the PROSPER group);8
  • A 14% reduction in the likelihood of having smoked cigarettes in the past month (36.2% of the control group smoked versus 31.1% of the PROSPER group); and
  • A reduction in the frequency of marijuana use (2.1 for the control group versus 1.8 for the PROSPER group, on a seven-point scale ranging from 1 (never) to 7 (more than weekly)).9

In addition to the 3 effects above, the study found suggestive evidence of a reduction in the 10 other substance use outcomes that were measured, but these other effects did not reach statistical significance at conventional (0.05) levels and may therefore be chance findings.

The following table summarizes the effects found on all 13 substance use outcomes that were measured for the full sample.

Outcome PROSPER Control
Number of different types of illicit substances ever used7 1.4** 1.7
Number of different types of gateway substances ever used10 2.1 2.2
Past month drunkenness 40.9% 42.7%
Past month cigarette use 31.1%** 36.2%
Past year marijuana use 35.9% 38.7%
Past year driving after drinking 23.3% 25.3%
Past year inhalant use 4.2% 5.6%
Past year methamphetamine use 3.0%* 4.3%
Lifetime prescription drug misuse 27.3%* 32.3%
Lifetime prescription opioid misuse 30.1%* 33.8%
Frequency of drunkenness at 12th grade follow-up11 2.8 3.0
Frequency of drinking and driving at 12th grade follow-up10 1.1 1.3
Frequency of marijuana use at 12th grade follow-up10 1.8** 2.1
**Statistically-significant at the 0.05 level *Significant at the 0.10 level.

Effects of PROSPER 6.5 years after random assignment for the non-user subgroup – i.e., students who had not used a given substance at the time the study began (fall of 6th grade):12

Compared to the control group, PROSPER group students were –

  • 35% less likely to have ever used methamphetamine (8.0% of the PROSPER group used methamphetamine versus 12.3% of the control group);
  • 35% less likely to have ever used Ecstasy (11.6% of the PROSPER group versus 17.8% of the control group);
  • 20% less likely to have ever used inhalants (21.3% of the PROSPER group versus 26.5% of the control group); and
  • 10% less likely to have ever used marijuana (51.2% of the PROSPER group versus 56.7% of the control group);

All of the above effects were statistically significant at the 0.05 level.13 The study did not find significant effects on three other outcomes measured – the likelihood of ever having used cigarettes, of ever having used alcohol, or of ever having been drunk.

The following table summarizes the effects found on all substance use outcomes that were measured for the non-user subgroup.

Outcome PROSPER Control
Ever used alcohol 85.5% 85.5%
Ever drunk 73.9% 74.3%
Ever used cigarettes 60.2% 62.3%
Ever used marijuana 51.2%** 56.7%
Ever used inhalants 21.3%** 26.5%
Ever used methamphetamine 8.0%** 12.3%
Ever used Ecstasy 11.6%** 17.8%
**Statistically-significant at the 0.05 level *Significant at the 0.10 level.

Effects of PROSPER earlier in high school – 4.5 and 5.5 years after random assignment (i.e., 10th and 11th grade):

At the 4.5 and 5.5 year follow-ups, the study found effects on substance use that were generally similar in magnitude and statistical significance to those described above at the 6.5-year follow-up. At the 4.5-year follow-up, the study also found a statistically-significant reduction in an index of conduct problems (e.g., stealing, truancy). While encouraging, we believe the 4.5- and 5.5-year findings constitute suggestive, rather than strong, evidence because of a study limitation at these earlier follow-ups – namely, a sizable difference in student attrition between the PROSPER group and control group that could potentially have undermined the equivalence of the two groups and led to inaccurate findings.

Discussion of Study Quality

  • The study had a long-term follow-up (6.5 years after random assignment), and low attrition of the communities in the sample: Outcome data for the analyses described above were obtained from 86% of the PROSPER communities and 86% of the control communities.
  • At the start of the study, students in the PROSPER and control groups were highly similar in their observable characteristics (e.g., demographics, and rates of substance use).
  • The study appropriately sought outcome data for all communities and students assigned to the PROSPER group, regardless of whether or how long they participated in PROSPER’s family or school-based programs (i.e., the study used an “intention-to-treat” analysis).
  • The study’s statistical analyses appropriately accounted for the fact that communities, rather than schools or individual students, were randomly assigned to the PROSPER versus control group.
  • The study evaluated PROSPER as implemented in 14 rural communities in two states, thus providing evidence of the system’s effectiveness under real-world implementation conditions.
  • Study limitations:

    • Although there was low attrition of the communities from the sample (as described above), there was a moderate level of student attrition in the final sample of communities, due to transfers, absenteeism, and dropouts: 27% at the 6.5-year follow-up. The attrition rates were similar in the PROSPER versus control communities (29% and 26% respectively), suggesting that attrition was not influenced by group assignment; and statistical tests show the attrition did not create differences between the two groups in their observable characteristics. However, the attrition conceivably could have caused unobservable differences between the two groups, possibly leading to inaccurate estimates of PROSPER’s effects.
    • A second limitation is that the study relied exclusively on student self-reports to measure substance use outcomes, obtained through a confidential written questionnaire administered in the classroom. Preferably, the study would have corroborated at least some self-reported outcomes with more objective measures (e.g., saliva tests for smoking).
    • A third limitation is that the sample of communities in the study was relatively small and homogeneous – 28 rural towns and small cities in two states. The Top Tier initiative’s Expert Panel believes that replication of the above findings in a second trial, conducted in another setting, would be desirable to confirm the initial results and establish that they generalize to other settings where PROSPER might be implemented.

SUMMARY OF THE INTERVENTION’S BENEFITS AND COSTS

If taxpayers fund implementation, what benefits to society can they expect to result, and what would be their net cost? The following table provides a summary. This is intended to be a general overview of social benefits in relation to taxpayer cost, rather than a comprehensive benefit-cost analysis. It assigns monetary value to particular benefits and costs only when doing so requires minimal assumptions.

Benefits To Society

At the 6.5-year follow-up (end of 12th grade) –

  • Community-wide reductions of 10-35% in illicit drug use initiation for youth who were non-users in 6th grade (prior to program delivery).
  • Moderate reductions in substance use for the full sample – non-users and users (e.g., 14% lower likelihood of past-month cigarette use).

Cost To Taxpayers 

  • Approximately $230-$420 per youth (depending on community size and other factors), in 2014 dollars.

REFERENCES

1 The Cooperative Extension System serves an outreach function for land grant universities in every state, disseminating scientific information to the community.

2 PROSPER’s website defines “evidence-based programs” as programs that have been rigorously evaluated in a randomized controlled trial and found to be effective.

3 Trained observers found that both the family and school-based programs achieved close adherence (approximately 90%) to the program’s content. About 17% of families in the PROSPER group participated in the family-focused program offered in sixth grade.

4 For example, after PROSPER’s delivery of programs to the students in this study, Project Alert was replaced on the menu by Lions Quest Skills for Adolescence.

5 This includes both the cost of the university-community partnership (i.e., PROSPER’s “infrastructure” cost) and the cost of delivering the family and school-based programs, as described above. Further detail on PROSPER’s infrastructure cost is shown here.

6 The population of the communities ranged from 7,000 to 45,000.

7 In a departure from random assignment, two communities assigned to the PROSPER group withdrew from the study during its first year and were replaced by two other communities that year – a substitution that could potentially have undermined the equivalence of the PROSPER and control communities and led to inaccurate findings. Thus, to help demonstrate that the findings are valid, the study authors – in addition to estimating PROSPER’s effects for the final sample of 28 communities – conducted a second analysis that omitted the two replacement communities in the PROSPER group along with the two control communities with the highest overall rates of substance use at 6.5-year follow-up, so as to provide a conservative (lower-bound) estimate of PROSPER’s effects. In this summary, we report the effects found in this more conservative analysis (which are marginally smaller than the effects found for all 28 communities).

8 This measure was obtained by assigning one point to reported use in each of the following five categories: (i) methamphetamine, (ii) ecstasy, (iii) marijuana, (iv) drugs or medications prescribed to someone else, and (v) Vicodin, Percocet, or Oxycontin without a prescription.

9 The effects on two of the three outcomes – number of different types of illicit substances used, and frequency of marijuana use – remained statistically significant at the 0.05 level after applying the Benjamini-Hochberg adjustment for multiple comparisons. The effect on cigarette use remained significant at the 0.10 level after such adjustment, but not at the 0.05 level.

10 This measure was obtained by assigning one point to reported use in each of the following three categories: (i) alcohol, (ii) cigarettes, and (iii) marijuana.

11 Frequency of drunkenness, driving after drinking, and marijuana use were each scored on a seven-point scale, ranging from 1 (never) to 7 (more than weekly).

12 Such non-users comprised at least 90% of the full sample for each measured substance.

13 All four effects remained statistically significant at the 0.05 level after the Benjamini-Hochberg adjustment for multiple comparisons.