Serious faults identified in design and reporting of abstinence-only studies

Studies exploring the effectiveness of “abstinence only” programmes to prevent HIV are limited by severe methodological weaknesses, investigators from Oxford University assert in the January 11^th edition of AIDS.

The investigators found that abstinence studies did not include enough participants to justify the initiation of large-scale public health initiatives; did not specify procedures used for randomisation; did not report clinically meaningful outcomes (such as incident HIV infections); did not use intent-to-treat analyses; did not provide enough details to demonstrate if interventions were delivered as planned; and did not compare the effectiveness of abstinence programmes with health education initiatives that provided wider information on sexual health, including condom use.

Abstinence-only HIV prevention programmes have been the source of fierce debate in recent years. Often acrimonious discussions have focused on the public health, political and moral aspects of these programmes.

Despite these heated debates, investigators at the Centre for Evidence-Based Medicine at Oxford University were concerned that evidence from clinical trials regarding the effectiveness or otherwise of abstinence programmes were being overlooked. They expressed particular concern that a forthcoming Cochrane review of the clinical effectiveness of abstinence-only education in richer countries, which identified policy and practice implications, is in fact, limited by reporting shortcomings from the trials that informed it.

The Oxford scientists found “recurring reporting deficiencies that may limit the extent to which programme trials can make specific recommendations for policy and practice.

Despite the use of randomised controlled trial designs, no abstinence study met the CONSORT reporting standards and missing data was an obstacle to meta-analysis.

Deficiencies in trial design and reporting in five broad areas were highlighted by the Oxford investigators:

• Randomisation – no study properly described its randomisation procedure. The authors emphasise that such are “critical for appraising trials”. The investigators add “despite searching 30 databases, 16,765 abstracts and unpublished literature, the review encompasses only 13,191 participants from randomised controlled trials (a small figure for a widespread public health intervention).”
• Outcome measures: Abstinence studies avoided reporting “clinically meaningful outcomes” of behavioural change such as HIV incidence. Rather, they focused on self-reported sexually transmitted infections. In addition, it was often unclear whether oral or anal sex were measured, and terms such as “virginity” and “intercourse” were often poorly defined.
• Conducting appropriate analyses: No study conducted an intent-to-treat analysis to counter reporting bias by accounting for dropouts.
• Could not determine if programmes were delivered as planned: Incomplete reporting made it difficult for the investigators to tell exactly what programme activities, settings, facilities and contexts were used.
• No comparison data: Only one study compared abstinence programmes with more comprehensive sexual health education.

The authors conclude, “abstinence-only programme evaluations…must report key data more completely.”