ELTMLE vs. competitors (RA, IPTW, IPTW-RA, and AIPTW)

2017 LONDON STATA USERS GROUP MEETING

************************************************************************************* *! 2017 STATA USERS GROUP MEETING LONDON
*! ELTMLE: Stata module for Ensemble Learning Targeted Maximum Likelihood Estimation
*! by Miguel Angel Luque-Fernandez, PhD [cre,aut]
*! Bug reports:
*! miguel-angel.luque at lshtm.ac.uk
**************************************************************************************

Content

I) Data generation: one sample

II)Dual misspecification: the worts-case scenario

III) Results

Note: Data reproduce a Cancer Epidemiological example. The interpretation of the results is not applicable to real-world

I) Data generation: one sample

Setting your path and working directory

. clear 

. set more off

. cd "D:\Dropbox\SUML\OneSimulation"
D:\Dropbox\SUML\OneSimulation

. //cd "Set your path"

One simulated dataset:

w1=Socieconomic status, w2=Age, w3=Cancer stage, w4=Comorbidities

Note: the interaction between age and comorbidities

. set obs 1000
number of observations (_N) was 0, now 1,000

. set seed 777

. gen w1    = round(runiform(1, 5)) //Quintiles of Socioeconomic Deprivation

. gen w2    = rbinomial(1, 0.45) //Binary: probability age >65 = 0.45

. gen w3    = round(runiform(0, 1) + 0.75*(w2) + 0.8*(w1)) //Stage 

. recode w3 (5/6=1)       //Stage (TNM): categorical 4 levels
(w3: 128 changes made)

. gen w4    = round(runiform(0, 1) + 0.75*(w2) + 0.2*(w1)) //Comorbidites: categorical four levels

. gen A     = (rbinomial(1,invlogit(-1 -  0.15*(w4) + 1.5*(w2) + 0.75*(w3) + 0.25*(w1) + 0.8*(w2)*(w4)))) //Binary treatment

. gen Y     = (rbinomial(1,invlogit(-3 + A + 0.25*(w4) + 0.75*(w3) + 0.8*(w2)*(w4) + 0.05*(w1)))) //Binary outcome

. // Potential outcomes
. gen Yt1 = (invlogit(-3 + 1 + 0.25*(w4) + 0.75*(w3) + 0.8*(w2)*(w4) + 0.05*(w1)))

. gen Yt0 = (invlogit(-3 + 0 + 0.25*(w4) + 0.75*(w3) + 0.8*(w2)*(w4) + 0.05*(w1)))

. // True ATE
. gen psi = Yt1-Yt0

TRUE SIMULATED ATE = 17.8%

. mean psi

Mean estimation                   Number of obs   =      1,000

--------------------------------------------------------------
             |       Mean   Std. Err.     [95% Conf. Interval]
-------------+------------------------------------------------
         psi |   .1787412   .0020142      .1747886    .1826938
--------------------------------------------------------------

Saving the data in your desktop

. save CancerData.dta, replace
file CancerData.dta saved

II) ATE estimation

Note: We misspecified the models excluding the interaction between age and comorbidities

Regression adjustment ATE= 24.1%

. teffects ra (Y w1 w2 w3 w4) (A)

Iteration 0:   EE criterion =  8.701e-32  
Iteration 1:   EE criterion =  1.434e-32  

Treatment-effects estimation                    Number of obs     =      1,000
Estimator      : regression adjustment
Outcome model  : linear
Treatment model: none
------------------------------------------------------------------------------
             |               Robust
           Y |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
ATE          |
           A |
   (1 vs 0)  |   .2410504   .1148066     2.10   0.036     .0160336    .4660672
-------------+----------------------------------------------------------------
POmean       |
           A |
          0  |   .4739475   .1141366     4.15   0.000     .2502439     .697651
------------------------------------------------------------------------------

. estimates store ra

Inverse probability weighting ATE = 37.9%

. teffects ipw (Y) (A w1 w2 w3 w4) 

Iteration 0:   EE criterion =  3.522e-23  
Iteration 1:   EE criterion =  2.840e-33  

Treatment-effects estimation                    Number of obs     =      1,000
Estimator      : inverse-probability weights
Outcome model  : weighted mean
Treatment model: logit
------------------------------------------------------------------------------
             |               Robust
           Y |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
ATE          |
           A |
   (1 vs 0)  |    .378604   .0807516     4.69   0.000     .2203337    .5368743
-------------+----------------------------------------------------------------
POmean       |
           A |
          0  |   .3362695    .079327     4.24   0.000     .1807913    .4917476
------------------------------------------------------------------------------

. estimates store ipw

IPTW-RA ATE = 31.9%

. teffects ipwra (Y w1 w2 w3 w4) (A w1 w2 w3 w4)

Iteration 0:   EE criterion =  3.522e-23  
Iteration 1:   EE criterion =  5.128e-33  

Treatment-effects estimation                    Number of obs     =      1,000
Estimator      : IPW regression adjustment
Outcome model  : linear
Treatment model: logit
------------------------------------------------------------------------------
             |               Robust
           Y |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
ATE          |
           A |
   (1 vs 0)  |   .3188398   .0706036     4.52   0.000     .1804593    .4572202
-------------+----------------------------------------------------------------
POmean       |
           A |
          0  |   .3958477   .0691749     5.72   0.000     .2602674    .5314281
------------------------------------------------------------------------------

. estimates store ipwra

AIPTW ATE = 28.8%

. teffects aipw (Y w1 w2 w3 w4) (A w1 w2 w3 w4)

Iteration 0:   EE criterion =  3.522e-23  
Iteration 1:   EE criterion =  3.600e-33  

Treatment-effects estimation                    Number of obs     =      1,000
Estimator      : augmented IPW
Outcome model  : linear by ML
Treatment model: logit
------------------------------------------------------------------------------
             |               Robust
           Y |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
ATE          |
           A |
   (1 vs 0)  |    .285314   .1152551     2.48   0.013     .0594183    .5112098
-------------+----------------------------------------------------------------
POmean       |
           A |
          0  |   .4293636   .1144905     3.75   0.000     .2049664    .6537608
------------------------------------------------------------------------------

. estimates store aipw

III) RESULTS

. quie reg psi

. estimates store psi

. estout psi ra ipw ipwra aipw

-----------------------------------------------------------------------------
                      psi           ra          ipw        ipwra         aipw
                        b            b            b            b            b
-----------------------------------------------------------------------------
main                                                                         
r1vs0.A                       .2410504      .378604     .3188398      .285314
_cons            .1787412                                                    
-----------------------------------------------------------------------------
POmean                                                                       
0.A                           .4739475     .3362695     .3958477     .4293636
-----------------------------------------------------------------------------
OME0                                                                         
w1                            .0395478                  .1119125     .0395478
w2                            .1770299                  -.098543     .1770299
w3                            .1715455                  .0389883     .1715455
w4                            .1019039                  .2425232     .1019039
_cons                        -.3331923                 -.3427923    -.3331923
-----------------------------------------------------------------------------
OME1                                                                         
w1                            .0100658                  .0071447     .0100658
w2                            .1529128                  .1510302     .1529128
w3                            .0867786                  .0859272     .0867786
w4                            .1157346                  .1193923     .1157346
_cons                         .2148979                  .2211646     .2148979
-----------------------------------------------------------------------------
TME1                                                                         
w1                                          .200913      .200913      .200913
w2                                         3.383105     3.383105     3.383105
w3                                         .6876172     .6876172     .6876172
w4                                        -.1261434    -.1261434    -.1261434
_cons                                       -.85242      -.85242      -.85242
-----------------------------------------------------------------------------

Ensemble Learning Maximum Likelihood Estimation (ELTMLE) ATE = 22.1%

. eltmle Y A w1 w2 w3 w4, tmle
D:\Dropbox\SUML\OneSimulation

. shell "C:\Program Files\R\R-3.3.2\bin\x64\R.exe" CMD BATCH SLS.R 

. 
end of do-file

    Variable |        Obs        Mean    Std. Dev.       Min        Max
-------------+---------------------------------------------------------
        POM1 |      1,000    .7100199     .187985     .20515   .9727612
        POM0 |      1,000    .4891232     .255392   .0454817   .9379262
          WT |      1,000    .2563642    3.599528        -40   2.046998
          PS |      1,000      .86845    .1383139   .4885203       .975


TMLE: Average Causal Effect

ACE (Risk Differences):    0.2209; SE:   0.05458; p-value: 0.0002; 95%CI:(0.1139, 0.3279)


TMLE: Causal Relative Risk

RR:   1.4516; 95%CI:(1.1314, 1.8624)

RELATIVE BIAS IPTW

. display (0.38 - 0.18)/.38 
.52631579

RELATIVE BIAS AIPTW

. display (0.28 - 0.18)/0.28
.35714286

RELATIVE BIAS ELTMLE

. display (0.22 - 0.18)/0.22
.18181818

THANK YOU FOR YOUR ATTENTION