Microbicides and the trouble with good news

As we report in this issue, the 16^th Retrovirus (CROI) Conference in Montreal featured the first ever non-negative result in a trial of a microbicide.

We say ‘non-negative’ rather than ‘positive’ because the 30% reduction in HIV infections seen in women who used the microbicide PRO2000 wasn’t ‘statistically significant’. It could have been a chance finding.

Despite this, the first thing to do is to celebrate. The idea for a microbicide has been around since 1990 when the South African epidemiologist Zena Stein published a piece called "HIV Prevention: The Need for Methods Women Can Use", pointing out that whether to use a condom was essentially the man’s choice and women were often in a position where it wasn’t possible to “choose safer sex”.

Nearly 20 years later we at last have a hint that microbicides could work. This is a testament to the scientific belief and persistence that has got us this far – often driven by feisty female and gay male advocates - and the success of some of the most difficult research studies ever attempted in the history of medicine.

But what about statistical significance? This is a complex area and misunderstanding it was the reason why the headlines about the trial ranged from the confident “Vaginal gel effective in preventing HIV infection” (Hindu Times) to the disappointed “Microbicide Gel Falls Short of Showing Significant Efficacy” (Doctor’s Guide).

Statistical significance is an arbitrary limit that scientists impose on the data they get from experiments, in order to decide what means something and what doesn’t. Odd things can happen that are purely due to chance: if you toss a coin for long enough, eventually you’ll get 20 heads turning up in a row.

Researchers work out the probability that the findings from their study are due to chance. If there’s a less than one-in-20 chance that the result is just ‘noise’, then that is ‘statistically significant’; if there’s a more than one-in-20 chance, it is ‘not significant’. In the microbicide study, the probability that the 30% reduction in HIV infection seen was not real was one-in-ten: so it was ‘not significant’…

…which, to put it another way, means that there was a 90% chance that it was real.

Still, you may say, 30% isn’t very impressive. Would you trust a condom that was 30% effective? PRO2000 is, nonetheless, promising rather than disappointing because:

·         For some people, 30% may be better than nothing. As Professor Abdool Karim, the principal investigator in this study said, “This may be a niche product for women with no other choices”.

·         If the still ongoing three-times-bigger UK-supported study produces similar results at the end of this year, we’ll really be on to something.

·         The more gel women used, the more protected they were. In women with above-average adherence to PRO2000, HIV infections were reduced by 44%.

It may work even better than this. You can’t ethically test a prevention product without offering women the best in existing methods, and condoms and safer sex advice were freely available. A woman who is a regular condom user has already protected herself against nearly nine in ten possible HIV infections. Adding a microbicide won’t reduce her risk much further.

But for a woman who doesn’t use condoms at all, a microbicide may add a considerable amount of protection. In the group of women who used condoms rarely but PRO2000 frequently, it stopped 68% of infections. This could be closer to its ‘real’ efficacy, but we don’t as yet have the figures to support that conclusion with confidence.

Right now the result of the trial poses more questions than answers.

Do we start using PRO2000 as the comparator drug in forthcoming microbicide trials because it will be unethical to use an inert placebo? Or can we say “It wasn’t proved to work” with a clear conscience? If we use it as the comparator drug, it means new studies will have to be larger to prove an effect.

If PRO2000’s efficacy is confirmed to be in the 30 to 40% region, is it worth developing and how do we ethically market it? Will there be pressure to market it anyway?

What about gay men and others who have anal sex? A rectal safety study is starting this year, but isn’t that a bit late if they decide to license it?

Would you use it…?