Attention Placebo: An ineffectual but harmless treatment provided to control group members, in order to ensure that both the control group and intervention group believe they are receiving treatment. The placebo protects against the possibility that intervention group members improve relative to controls simply because they believe they are receiving treatment, rather than because the intervention is truly effective. Attention placebos are appropriate in studies where there’s a reasonable possibility that this might occur — e.g., in a study of a counseling program to treat depression, where participants’ beliefs that they are receiving treatment may actually alleviate their depressive symptoms.

Attrition: The percentage of the individuals (or groups) originally randomized for whom outcome data could not be obtained. A general guideline is that attrition should be less than 25 percent – the lower the better. Furthermore, the amount of attrition should be roughly the same for the intervention and control groups. Low attrition is important in ensuring that the intervention and control groups remain equivalent over the course of the study – i.e., have no systematic differences other than those caused by the intervention.

Blinding of researchers: This means that the researchers collecting outcome data were kept unaware of who was in the intervention versus control group. Blinding is particularly important when study outcomes are measured through methods such as interviews or observations, where researchers’ biases (e.g. as proponents of the intervention) could influence their outcome measurements.

Effect Size: The standardized “effect size” is a measure of the magnitude of the intervention’s effect. Specifically, it is the difference in the mean outcome between the treatment and control groups, divided by the pooled standard deviation. The following table helps illustrate the meaning of standardized effect sizes in intuitively accessible terms:

An effect size of…

 

Is considered…

 

It means that…

 

0.2

small-modest

The average member of the intervention group had a better outcome than 58 percent of the members of the control group.

0.5

modest-large

The average member of the intervention group had a better outcome than 69 percent of the members of the control group.

0.8

large

The average member of the intervention group had a better outcome than 79 percent of the members of the control group.

A full discussion of effect sizes is beyond the scope of this site. We merely note that standardized effect sizes may not accurately convey the practical or policy importance of an intervention in some cases, and, when used, should preferably be translated into more easily accessible terms (e.g., an improvement in reading skill by two grade levels, a 20 percent reduction in weekly use of illicit drugs).

Intention-to-treat analysis: This means that the study sought outcome data for all of the individuals randomly assigned to the intervention group, even those who did not complete the intervention, and used such data in calculating the intervention’s effect. Such analysis is important in ensuring that the intervention and control groups remain equivalent over the course of the study – i.e., have no systematic differences other than those caused by the intervention.

Sample size: This is the total number of individuals (or groups) randomly assigned to the intervention and control groups. In general, larger sample sizes are better than smaller sample sizes because they provide greater confidence that any difference in outcomes between the intervention and control groups is due to the intervention rather than chance.

Statistically Significant: If a study finds that the difference in outcomes between the intervention and control groups is “statistically significant,” it means it is highly unlikely that the difference is the result of chance. For example, statistical significance at the .05 level (the most commonly-used level) means that there is only a 1 in 20 probability that the difference could have occurred by chance if the intervention’s true effect is zero.