Frequently Asked Questions

1. Predictions made from data including a genotype

What models does the system use to make its predictions?

The predictions of treatment response are made by a committee of 10 random forest models, the outputs of which are averaged to give the final prediction.

How reliable are the predictions made by the system when the genotype is included?

During cross-validation, the 10 random forest models that provide the predictions of treatment response for the system achieved an area under the receiver-operator characteristic curve (AUC) of 0.87.   When the models were tested with an independent test set of 600 treatment change episodes the AUC was 0.84 and overall accuracy was 76%.

How do you classify the system’s outputs as response or failure?

The output of the models is an estimate of the probability of the HIV viral load going below 50 copies HIV RNA/following a change of antiretroviral treatment. During the development and cross validation of the models we identified the optimum operating point (OOP) – a cut-off value for response and failure that provides the best overall accuracy of the system. In this case the OOP was 38% - any value below is classified as a prediction of failure and any above as a prediction of success. In order to optimise the sensitivity and specificity for regimens including the either raltegravir or elvitegravir, the OOP for those cases including each of these drugs were used to develop a drug-specific OOP of 0.60 for raltegravir and 0.66 for elvitegravir.

How much data was used to train the models used by the system?

The RDI database contains data from over 160,000 patients.  29,574 treatment change episodes that are representative of contemporary HIV clinical practice and meet all our stringent quality criteria were used in the training of the models that are currently being used by the system to make predictions without the use of a genotype.

What data are the predictions based on?

The system makes its predictions of response to a new antiretroviral treatment based on the individual patient’s treatment history, viral load, CD4 count and the time to follow-up, as entered by the healthcare professional.

 

2. Predictions made without a genotype

What models does the system use to make its predictions?

The predictions of treatment response are made by 5 random forest models, the outputs of which are combined to give the final prediction.

How reliable are the predictions made by the system when a genotype is not available?

During cross-validation, the random forest models that provide the predictions of treatment response for the system achieved an area under the ROC curve (AUC) of 0.83 and overall accuracy of 76%. When the models were tested with an independent test set of 3,000 treatment change episodes, the AUC was 0.81 and overall accuracy was 75%.

When the models were tested with an independent test set of 461 treatment change episodes from Sub-Saharan Africa, the AUC was 0.76 and overall accuracy was 70%.

How do you classify the system’s outputs as response or failure?

The output of the models is an estimate of the probability of the HIV viral load going below 50 copies HIV RNA/following a change of antiretroviral treatment. During cross validation of the models the optimum operating point (OOP) - a cut-off value for response and failure that provides the best overall accuracy of the system, was identified. In this case the OOP was 42% - any value below is classified as a prediction of failure and any above as a prediction of success. In order to optimise the sensitivity and specificity for regimens including two of the newest drugs to be added to the system, elvitegravir and maraviroc, the OOP for those cases including this drug were used to develop drug-specific OOPs of 0.63 and 0.52 respectively.  When presenting the results on the report, the system standardises the predictions around a fail/respond cut-off of 0.5

How much data was used to train the models used by the system?

The RDI database contains data from over 190,000 patients. 50,270 treatment change episodes that are representative of contemporary HIV clinical practice and meet all our stringent quality criteria were used in the training of the models that are currently being used by the system to make predictions without the use of a genotype.

What data are the predictions based on?

The system makes its predictions of response to a new antiretroviral treatment based on the individual patient’s treatment history, viral load, CD4 count and the time to follow-up, as entered by the healthcare professional.

 

3. General questions about HIV-TRePS

Why can’t I get predictions for rilpivirine or dolutegraivir from your system?

When we developed the models that we currently use we had not received adequate data from clinical practice involving these drugs to train the models to make reliable predictions. We intend to include them in a future version of the system, using new models, once we have collected sufficient data.

How can I turn on/off TRePS reports via email?

Log-in to TRePS and then in the top right hand corner of the home screen select 'My Account' on this screen and then 'Account Details' you will then see an option at the bottom of the screen entitled 'Email Reports' here you can change it to 'Yes' or 'No' and then Save your preference.

 

The RDI

What is the RDI?

The HIV Resistance Response Database Initiative is a not-for-profit group set up in 2002 as a wholly independent international body to:

  1. Be a global repository for HIV resistance and outcome data
  2. Use these data to develop computational models to predict patients response to antiretroviral treatment
  3. Make such models available free-of-charge over the internet

More information

Who are the RDI?

The RDI consists of a small research team based in the UK and a large global network of advisors, research partners and data donors.

Where does your data come from?

The data is donated to the RDI by hospitals, clinics, research programmes, pharmaceutical companies and other institutions and groups around the world. More information.