Because he was in Paris attending the AIDS Vaccine 2009 meeting, and because I asked him to, Dorian McIlroy from the University of Nantes has written an account of the presentation of the recent Thailand HIV vaccine trial results. Thanks Dorian!
Ed Rybicki.
Here in Paris, the initial results from the Thai ALVAC/AIDSVAX vaccine trial have just been presented. The first presentation was by Dr Supachai Rerks-Ngarm, who was followed by Colonel Nelson Michael (who gave his presentation in uniform). This was a big double-blinded RCT, with more than 16000 participants, about 8000 people in each arm of the study. I am not a methodologist, but this trial does appear to me to have been very well-designed, carried-out, and analyzed. So I think one should unreservedly treat the results as high-quality.
The headline result – a 31% reduction in HIV transmission in vaccine recipients was reported in the press in September, but the difference between the vaccine and placebo recipients was only just statistically significant. So the big question was, are the data convincing enough to reject the null hypothesis? That is, could the difference in the number of HIV infections in the two groups just be down to chance, rather than vaccine efficacy?
Both presenting scientists involved in the study gave talks that were very scientifically rigorous, explaining the why the data was analyzed the way it was, and what conclusions can and cannot be drawn from the trial.
With regards to the first question, it was pointed out that the statistical analysis of the primary endpoint (new HIV infections in the two groups) was decided before the data were unblinded. That is, the statisticians who analyzed the data did not choose their technique to manipulate the interpretation in any way.
The main statistical approach applied was Kaplan-Meier analysis, looking at the number of people infected in each group over time. Differences between vaccine and placebo arms were tested by the log-rank test. However, there were three different ways of determining exactly which of the people enrolled in the trial were included in the analysis. These were intention-to-treat (ITT), modified ITT, and per protocol (PP).
The ITT definition was everyone who was HIV seronegative at study entry, and received at least one injection. The modified ITT excluded 7 individuals who were found to be positive for HIV infection by PCR at study entry. The PP definition was, everyone who received all of the vaccinations at the allotted times. Now this was a rather strict definition, because a person who got a vaccination one day later than the schedule was excluded from the analysis, leaving only about 6000 people per group in the PP analysis.
Kaplan-Meier curves for all three analyses (ITT, mITT and PP) looked pretty good, and showed more infections in the placebo arm than in the vaccine arm, although the difference was only statistically significant (p=0.04) in the mITT analysis. The reason why the ITT analysis did not show a statistically significative difference was because 5 of the 7 people who were infected (PCR postitive, but not seropositive) at entry into the trial were in the vaccine arm. So a net increase of just three more infections (5 in vaccine arm – 2 in the placebo arm) in the vaccine group changed the p-value from 0.04 to 0.08. However, excluding people who were infected before the beginning of the trial is entirely justified, and it is clear that the mITT analysis was preferable to the raw ITT.
The comparison of the mITT and PP results was more interesting. Although the same tendency was observed (more infections in the placebo arm) the Kaplan-Meier curves looked much more similar. There may be two explanations for this. Firstly, since the number of people in each group was decreased, the statistical power of the test also went down – so the same effect would not be statistically significant. Another factor, that was pointed out by Col. Michael, was that the PP analysis automatically ruled out patients who became infected during the vaccine protocol. That is, over the first six months of the trial. Looking back at the Kaplan-Meier curves from the mITT analysis, the main difference between the vaccine and placebo groups accrued during the first year of the trial. Afterwards, new infections occurred pretty much at the same rate in the two groups. Most of these infections were excluded from the PP analysis, resulting in a non-significant difference between the two groups.
This for me, is the key to the interpretation of the trial. In my opinion, there was a protective effect of vaccination in this study (so yes, the data are convincing enough to reject the null hypothesis) – but it seems to have been short lived. Indeed, Col. Michael also mentioned that innate immune responses (presumably induced by the viral ALVAC vector that was injected four times during the 6 months of the vaccination protocol) could be involved in protection. No empty virus vector was used in the placebo arm, (described here : http://www.fda.gov/OHRMS/DOCKETS/AC/04/briefing/4072B2_2.doc) only “a mixture of virus stabilizer, and freeze drying medium”. So more short-lived, non-specific innate immune responses could have been induced in the vaccine arm compared to the placebo arm. This is also consistent with the higher frequency of adverse reactions in vaccine recipients compared to placebo recipients that was also reported in Dr Rerks-Ngarm’s talk.
If the partial protection that was observed in the Thai trial does turn out to have been due to a transient induction of innate immune responses due to the ALVAC vector, then I’m afraid we won’t be able to say that the ALVAC/AIDSVAX candidate vaccine induced an adaptive immune response that is able to protect people from HIV infection.
Dorian McILROY
ViroBlogy 