Accuracy of Neural Network Models in Predicting HIV Treatment Response from Genotype May Depend on Diversity as well as Size of Data Sets

Poster presentation at:

The 11th Conference on Retroviruses and Opportunistic Infections

8th February 2004 - 11th February 2004

San Francisco, USA


Larder BA1, Wang D1, Revell AD2, Harrigan R3, Montaner J3 and Lane C4.

1: The HIV Resistance Response Database Initiative (RDI), Cambridge, UK; 2: RDI Ltd, London, UK; 3: The BC Centre for Excellence in HIV/AIDS Vancouver, Canada; 4: National Institute of Allergy and Infectious Diseases (NIAID), Bethesda, USA

Background The RDI has trained Artificial Neural Network (ANN) models using clinical data which, when tested with independent data from the same clinical setting, have demonstrated accuracy in predicting virological response to antiretroviral therapy from genotype. Here we examine the performance of ANN models developed using datasets of increasing size pooled from several clinical settings.

Methods 228 'treatment change episodes' (TCEs) were obtained from 153 patients in the NIAID cohort: episodes with a genotype 12 weeks and a VL 8 before treatment change and follow-up VL within 4-40 weeks. Four ANN models were developed; each trained using 171 TCE's and tested with the remaining 57. Results were compared to those obtained with 13 ANN models using the NIAID data plus data from a second cohort (BC), yielding 747 TCE's. The data were finally added to data from multiple sources in the RDI database, yielding 1,581 TCEs and 10 ANN models trained, and tested. All the training and testing sub-datasets were independent, randomly partitioned and normalised.

Results Correlation between the predicted and actual VL change for the four ANN models developed from the NIAID cohort gave a mean value of 0.71 and mean correct trajectory prediction rate of 76%. The 13 ANN models from the combined NIAID and BC cohorts gave a mean value of 0.65 and mean correct trajectory prediction rate of 78%. The ten models developed from the largest composite dataset gave a mean of 0.55 and a mean correct trajectory prediction rate of 74%. The value for the composite dataset was significantly lower than for the other two datasets.

Conclusions These results indicate that ANN models trained using limited data from one clinical setting can be surprisingly accurate in predicting responses to treatment from genotype for other patients from the same cohort. Models developed using considerably more data from a variety of settings are not automatically more accurate, when tested with data from multiple settings. Preliminary analysis suggests that the accuracy of ANN models may depend on the training and test data being similarly diverse, although this does not necessarily imply similarity in terms of specific mutations or drugs. These results confirm the potential of ANN models as an aid to treatment decision-making but underline the need to collect sufficient diversity of data to develop models that are able to predict treatment response for disparate test cases.

PDF View Poster

RDI

Join the RDI mailing list

Enter your email address below to start receiving the RDI newsletter.

You can unsubscribe at any time. View the RDI privacy policy.

Email