Version History

V14.8.8.0 - Release Date: 31/08/2021

New models installed:

CNG(-VL)-50: Classification models that do not require a baseline genotype or viral load and predict the probability of a follow-up plasma viral load of <50 copies HIV RNA/mL.

• 5 x RF models
• 156,090 training & 8,157 test TCEs
• Developed Sep-Dec 2020
• In independent testing AUC = 0.82, sensitivity = 74%, specificity = 74%, overall accuracy = 74%
• Unpublished

CNG(-VL)-400: Classification models that do not require a baseline genotype or viral load and predict the probability of a follow-up plasma viral load of <400 copies HIV RNA/mL.

• 5 x RF models
• 156,090 training & 8,157 test TCEs
• Developed Sep-Dec 2020
• In independent testing AUC = 0.85, sensitivity = 78%, specificity = 77%, overall accuracy = 78%
• Unpublished

CNG(-VL)-1,000: Classification models that do not require a baseline genotype or viral load and predict the probability of a follow-up plasma viral load of <1,000 copies HIV RNA/mL.

• 5 x RF models
• 156,090 training & 8,157 test TCEs
• Developed Sep-Dec 2020
• In independent testing AUC = 0.85, sensitivity = 77%, specificity = 77%, overall accuracy = 77%
• Unpublished

CNG(-ToT-VL)-50: Classification models that do not require a baseline genotype, time on therapy or viral load, and predict the probability of a follow-up plasma viral load of <50 copies HIV RNA/mL.

• 5 x RF models
• 156,090 training & 8,157 test TCEs
• Developed Nov 2020 - February 2021
• In independent testing AUC = 0.83, sensitivity = 73%, specificity = 77%, overall accuracy = 75%
• Unpublished

CNG(-ToT-VL)-400: Classification models that do not require a baseline genotype, time on therapy or viral load, and predict the probability of a follow-up plasma viral load of <400 copies HIV RNA/mL.

• 5 x RF models
• 156,090 training & 8,157 test TCEs
• Developed Nov 2020 - February 2021
• In independent testing AUC = 0.85, sensitivity = 78%, specificity = 77%, overall accuracy = 78%
• Unpublished

CNG(-ToT-VL)-1,000: Classification models that do not require a baseline genotype, time on therapy or viral load, and predict the probability of a follow-up plasma viral load of <1,000 copies HIV RNA/mL.

• 5 x RF models
• 156,090 training & 8,157 test TCEs
• Developed Nov 2020 - February 2021
• In independent testing AUC = 0.85, sensitivity = 75%, specificity = 78%, overall accuracy = 76%
• Unpublished

CNG(-CD4-VL)-50: Classification models that do not require a baseline genotype, CD4 cell count or viral load, and predict the probability of a follow-up plasma viral load of <50 copies HIV RNA/mL.

• 5 x RF models
• 156,090 training & 8,157 test TCEs
• Developed March - May 2021
• In independent testing AUC = 0.80, sensitivity = 74%, specificity = 73%, overall accuracy = 73%
• Unpublished

CNG(-CD4-VL)-400: Classification models that do not require a baseline genotype, CD4 cell count or viral load, and predict the probability of a follow-up plasma viral load of <400 copies HIV RNA/mL.

• 5 x RF models
• 156,090 training & 8,157 test TCEs
• Developed March - May 2021
• In independent testing AUC = 0.84, sensitivity = 76%, specificity = 75%, overall accuracy = 76%
• Unpublished

CNG(-CD4-VL)-1,000: Classification models that do not require a baseline genotype, CD4 cell count or viral load, and predict the probability of a follow-up plasma viral load of <1,000 copies HIV RNA/mL.

• 5 x RF models
• 156,090 training & 8,157 test TCEs
• Developed March - May 2021
• In independent testing AUC = 0.84, sensitivity = 76%, specificity = 76%, overall accuracy = 76%
• Unpublished

CNG(-CD4-ToT-VL)-50: Classification models that do not require a baseline genotype, CD4 cell count, time on therapy or viral load, and predict the probability of a follow-up plasma viral load of <50 copies HIV RNA/mL.

• 5 x RF models
• 156,090 training & 8,157 test TCEs
• Developed June-August 2021
• In independent testing AUC = 0.80, sensitivity = 72%, specificity = 73%, overall accuracy = 72%
• Unpublished

CNG(-CD4-ToT-VL)-400: Classification models that do not require a baseline genotype, CD4 cell count, time on therapy or viral load, and predict the probability of a follow-up plasma viral load of <400 copies HIV RNA/mL.

• 5 x RF models
• 156,090 training & 8,157 test TCEs
• Developed June-August 2021
• In independent testing AUC = 0.83, sensitivity = 77%, specificity = 75%, overall accuracy = 76%
• Unpublished

CNG(-CD4-ToT-VL)-1,000: Classification models that do not require a baseline genotype, CD4 cell count, time on therapy or viral load, and predict the probability of a follow-up plasma viral load of <1,000 copies HIV RNA/mL.

• 5 x RF models
• 156,090 training & 8,157 test TCEs
• Developed June-August 2021
• In independent testing AUC = 0.83, sensitivity = 77%, specificity = 74%, overall accuracy = 77%
• Unpublished

V13.8.7.0 - Release Date: 18/06/2020

New models installed:

ANG(noCD4)

• 5 x RF models
• New drugs: Historical EVG and DTG
• 62,940 training & 3,260 test TCEs
• Developed Feb-Apr 2020
• In independent testing r= 0.69, r2 = 0.48, MAE = 0.75 log¹⁰ HIV RNA
• Unpublished

CNG(noCD4)

• 5 x RF models
• New drugs: Historical EVG and DTG
• 62,940 training & 3,260 test TCEs
• Developed Feb-Apr 2020
• In independent testing AUC = 0.82, OA = 75%
• Unpublished

V12.8.6.0 - Release Date: 18/12/2019

New models installed:

AG: Absolute models that use a genotype and all other input variables

• 5 x RF models
• 13,936 training & 696 test TCEs
• Developed Jul-Sep 2019
• In independent testing r2 = 0.55, AE = 0.69 log¹⁰ HIV RNA
• Unpublished

AG (noToT): Absolute models that use a genotype and all other input variables except time on therapy

• 5 x RF models
• 19,080 training & 969 test TCEs
• Developed Jul-Sep 2019
• In independent testing r2 = 0.52, AE = 0.70 log¹⁰ HIV RNA
• Unpublished

CG: Categorical models that use a genotype and all other input variables

• 5 x RF models
• 13,936 training & 696 test TCEs
• Developed Sep-Oct Oct 2019
• In independent testing AUC = 0.89
• Unpublished

CG (noToT): Categorical models that use a genotype and all other input variables except time on therapy

• 5 x RF models
• 19,080 training & 969 test TCEs
• Developed Sep-Oct 2019
• In independent testing AUC = 0.90
• Unpublished

V11.7.6.0 - Release Date: 29/05/2019

New models installed:

ANG:  Absolute models that do not use a genotype but with all other input variables

• 5 x RF models
• 56,717 training & 2,856 test TCEs
• Developed February – April 2019

ANG(-ToT-CD4): Absolute, no-geno models without therapy experience or baseline CD4 counts

• 5 x RF models
• 56,717 training & 2,856 test TCEs
• Developed February – April 2019

V10.7.5.0 - Release Date: 29/03/2019

New models installed:

AG(-ToT):  Absolute models using a genotype but without therapy experience.

• 5 x RF models
• 18,242 training & 1,000 test TCEs
• Using new non-adherence filter based on concordant TRePS and GSS
• Developed Nov 2017 - Jan 2018
• Published In JAIDS 2018

ANG(-ToT): Absolute, no-geno models without therapy experience

• 10 x RF models
• 50,270 training & 3,000 test TCEs
• Developed Jan-Apr 2018
• Published In JAIDS 2018

CNG: Classification models that do not use a genotype and include therapy experience

• 5 x RF models
• 56,717 training & 2,856 test TCEs
• Developed April – September 2018
• Compared no time on therapy vs time on therapy (ToT) vs line of therapy (LoT)
• Independent testing AUC for CNG(-ToT) = 0.82, for CNG(+ToT) = 0.84
• Results indicated that ToT would be taken forward as part of standard variable set

CNG(-ToT): Classification models that do not use a genotype or therapy experience

• 5 x RF models
• 56,717 training & 2,856 test TCEs
• Developed April – September 2018
• Compared no time on therapy vs time on therapy (ToT) vs line of therapy (LoT)
• Results indicated that ToT would be taken forward as part of standard variable set
• These were the control models from the study used to develop the CNG models above

CNG(-ToT-CD4): Classification models that do not use a genotype, therapy experience or CD4 counts

• 5 x RF models
• Developed April – May 2018
• 50,270 training and 3,000 test TCEs
• Compared models trained with baseline CD4 counts with models using an estimated CD4 count
• Results indicated that the models that simply omitted the CD4 count were as accurate as those using an estimated value

V9.6.4.0 - Release Date: 01/12/2017

CG(-ToT): One set of five new RF models for cases of virological failure with a genotype

• A committee of 5 RF models trained with 17,378 TCEs with genotypes and tested with an independent global test set of 940 TCEs
• These models were developed using a new non-adherence filter:  All TCEs were put through the current with geno’ models.  Any TCEs that were predicted by the models to respond but failed in the clinic were identified. These cases had their genotypes entered into the Stanford HIVdb genotype interpretation system.  Any of these cases that were predicted by Stanford to respond, defined as a genotypic sensitivity score of ≥2, were then removed as presumed non-adherents
• These models were compared to five RF models developed using data that was censored using the existing non-adherence filter: All TCEs with a baseline viral load of ≤3 logs and an increase in viral load at follow-up  (following a change to their antiretroviral regimen) of ≥2 logs were excluded for presumed non-adherence.  
• Output variable was probability of follow-up viral load <50 copies
• The new filter was more stringent and removed 4.7% of the original pool of TCEs vs. 0.3% removed by the old filter.
• The average AUC during cross validation was 0.88 for the new models vs 0.86 for the old.  Overall accuracy was 79% vs 76%
• The OOP for the models developed during training was 0.42.  As before, separate OOPs were developed to optimise performance for regimens containing EVG (0.56) or RAL (0.68)

V8.5.4.1 - Release Date: February 2017

HIV-TRePS optimised for use on mobile devices and running on new server with increased speed

• Same models as version 8 below

V8.5.4.0 - Release Date: 07/02/2017

CNG(-ToT): One set of five new RF models for cases of virological failure without a genotype

• A committee of 5 RF models trained with 50,2750 TCEs without genotypes and tested with an independent global test set of 3,000 TCEs and a subset of 461 cases from Southern Africa
• These models were developed using data with expanded baseline windows (viral load increased form 8 to 16 weeks and CD4 counts in creased from 12 to 24 weeks
• These models are able to predict responses to combinations including tipranavir, maraviroc and elvitegravir (the first time for RDI models that do not use a genotype for their predictions)
• Output variable was probability of follow-up viral load <50 copies
• The average AUC during cross validation was 0.83, sensitivity 71%, specificity 80% and overall accuracy was 76%
• When tested with the global 3,000 test set, the committee average AUC was 0.81, sensitivity 73%, specificity 76% and overall accuracy was 75%. For the Southern African cases the AUC was 0.76, sensitivity 72%, specificity 69% and overall accuracy was 70%

 

ALL THE MODELSL FROM THIS POINT ON HAVE BEEN SUPERCEDED ON THE SYSTEM

V7.5.3.0 - Release Date: 22/01/2016

CG(-ToT) & CNG(-ToT): Two sets of new models for cases of virological failure with or without a genotype

• A committee of 10 RF models trained with 29,574 TCEs without genotypes and tested with an independent global test set of 1,700 TCEs and a subset of 222 cases from South Africa
• A second committee of 10 RF models were trained with 15,130 TCEs including genotypes and tested with an independent global test set of 750 TCEs
• Output variable was probability of follow-up viral load <50 copies
• AUC during cross validation for the models without genotypes was 0.83, sensitivity 71%, specificity 80% and overall accuracy was 76%
• When tested with the global 1,700 test set, the committee average AUC was 0.82, sensitivity 69%, specificity 77% and overall accuracy was 74%.  For the South African cases the AUC was 0.79, sensitivity 68%, specificity 76% and overall accuracy was 73%
• AUC during cross validation for the models including genotypes was 0.87, sensitivity 79%, specificity 79% and overall accuracy was 79%
• When tested with the global 750 test set, the committee average AUC was 0.84, sensitivity 79%, specificity 74% and overall accuracy was 76%

V6.4.2.0 - Release Date: 05/11/2013

New models for cases of virological failure with a genotype

• A committee of 10 RF models trained with 11,417 TCEs and tested with an independent global test set of 600 TCEs
• 105 input variables including 20 individual treatment history variables
• Output variable was probability of follow-up viral load <50 copies
• AUC during cross validation = 0.86, sensitivity = 73%, specificity = 83%, with an optimum operating point of 0.43
• When tested with the global 600 test set, the committee average AUC was 0.83, sensitivity was 63%, specificity 83%, with the OOP of 0.43

V5.3.2.0 - Release Date: 11/07/2013

New models for cases of virological failure without a genotype

• A committee of 10 RF models trained with 22,567 TCEs and tested with an independent global test set of 1,000 TCEs and an independent test subset of 100 TCEs from Southern Africa
• 42 input variables including 20 individual treatment history variables
• Output variable was probability of follow-up viral load <50 copies
• AUC during cross validation = 0.82, sensitivity = 66%, specificity = 81%, with an optimum operating point of 0.42
• When tested with the global 1,000 test set, the committee average AUC was 0.80, sensitivity was 66%, specificity 79%, with an OOP of 0.42
• When tested with the southern African 100 test set, the committee average AUC was 0.78, sensitivity was 81%, specificity 60%

V4.3.1.0 - Release Date: 13/11/2012

New therapy cost function added

• New section to baseline page and home page enables input of annual costs in any currency for all available drugs
• User can include costs in 4 ways:
1. Simply add the annual cost of therapy to the existing modelling options
2. The user can define an annual therapy cap and only those regimens that are below that cap will appear on the report
3. The user can use the system to model alternative regimens that are no more expensive than the most costly user-defined regimen
4. Instruct the system to present regimens that are predicted to be effective in increasing order of cost, with the least expensive in first position
• Separate input field are available for 3TC and FTC, and APV and F-APV costs

V3.3.1.0 - Release Date: 24/05/2012

New ‘With Geno’ models added

• A committee of 10 models trained with 7,638 TCEs, including 375 from Stanford
• 100 input variables include 18 individual drug treatment history variables
• Output variable is probability of follow-up viral load <50 copies
• AUC during cross validation was 0.84, sensitivity = 67% and specificity = 83%
• OOP is 0.41 for non-RAL regimens and 0.58 for RAL regimens
• Output to TRePS is simple committee average prediction for each regimen
• Updates to the process files and database table changes were required

v3.2.1.0 - Release Date: 22/02/2012

New ‘With Geno’ models added and sequence upload function introduced

• Two committees each of five models, trained with 7,263 TCEs and tested with 375 independent TCEs form Stanford
• Committee 1 with 88 input variables including 6 simple treatment history variables
• Committee 2 with 100 input variables including 18 individual drug treatment history variables
• Output variable was probability of follow-up viral load <50 copies
• AUC during cross validation = 0.82 (both), sensitivity = 64% and 62%, specificity = 81% and 84%
• Committee average AUC during testing was 0.87 and 0.86, sensitivity was 67% and 61%, specificity 90% and 89%
• Output to TRePS was simple average of the two committee average predictions for each regimen
• OOP was 0.44 for non-RAL regimens and 0.54 for RAL regimens
• Sequence upload function added allowing for sequences to be copies and pasted or uploaded with automatic extraction of mutations.

v2.1.1.0 - Release Date: 18/07/2011

New ‘No Geno’ models added to TRePS

• A committee of 10 RF models trained with 14,964 TCEs and tested with 800 TCEs
• 26 input variables included 5 simple treatment history variables
• Output variable was probability of follow-up viral load <400 copies
• AUC during cross validation = 0.77, sensitivity = 72%, specificity = 71%
• Committee average AUC during testing was 0.77, sensitivity was 71%, specificity 70%
• OOP was 0.55 for non-RAL regimens and 0.85 for RAL regimens
• A single RF models was trained with 14,891 TCEs and tested with 800 TCEs
• 39 input variables including 18 individual drug treatment history variables
• Output variable was probability of follow-up viral load <400 copies
• Committee average AUC during testing was 0.78, sensitivity was 70%, specificity 72%
• Output to TRePS is simple average of the committee average prediction from the Committee and the prediction of the individual treatment history model for each regimen

v1.1.0.1 - Release Date: 05/07/2011

Change to use all 10 of the committee of RF models described below rather than just four

• AUC of these four during cross validation = 0.85, sensitivity = 70%, specificity = 82%
• OOP was 0.41
• Output to TRePS is simple committee average prediction for each regimen

v1.1.0.0 - Release Date: 06/10/2010

Launch of HIV-TRePS containing ‘With Genotype’ Random Forest (RF) models

• A committee of 10 RF models trained with 5,752 TCEs
• 84 input variables include 5 simple treatment history variables
• Output variable was probability of follow-up viral load <50 copies
• The four best performing models (5,7,8 & 9) were selected to be used in TRePS
• AUC of these four during cross validation = 0.85, sensitivity = 70%, specificity = 82%
• Output to TRePS is simple committee average prediction for each regimen