Predicting Glaucomatous Progression using Trees and Random Forests
PPIG
Classification Tree
Random forests
PPIG revisited
Predicting Glaucomatous Progression using Trees and Random Forests Richard A. Levine San Diego State University Joint work with Lucie Sharpsten, Juanjuan Fan, Xiaogang Su, & Shaban Demirel
Discussion
PPIG
Classification Tree
Random forests
PPIG revisited
Discussion
Perimetry and Psychophysics in Glaucoma (PPIG) Study
• 166 subjects with moderate- to high-risk ocular hypertension
• 332 eyes, subjects average age 58.1 (11.0 std), range 33 to 87
years
• 95 females, 71 males recruited in Portland, OR metro area • Median follow-up 6+ years, maximum 9+ years
• Outcome: progressive glaucomatous optic neuropathy (pGON) • 26 subjects pGON both eyes; 41 pGON in one eye • Goal: predict pGON using BL & FU data
PPIG
Classification Tree
Random forests
PPIG revisited
Humphrey Visual Field Analyzer: SAP
Discussion
PPIG
Classification Tree
Random forests
PPIG revisited
Clinical factors used in data analysis Both baseline and follow-up: • 52 test point thresholds • mean deviation (MD)
• pattern standard deviation (PSD)
Baseline only: • intraocular pressure (IOP in mmHg)
• central corneal thickness (CCT in microns) • gender indicator • age (in years)
Discussion
PPIG
Classification Tree
Random forests
PPIG revisited
Decision tree TP44 ≥ 30.17
TP1 < 30.95
13; 0%
TP1 < 32.12
TP32 ≥ 32.29
37; 22%
19; 42%
IOP < 15.5
7; 0%
TP53 ≥ 30.85
7; 14%
30; 23%
12; 100%
Discussion
PPIG
Classification Tree
Random forests
PPIG revisited
CART (Breiman et al., 1984)
• Regression problem: Y outcome/output, X covariates/input
• Why tree methods?
Fewer assumptions; interpretation; prognostic rules
• Partitioning covariate space • Three steps to the CART algorithm: 1. grow a large tree 2. prune to a nested sequence of subtrees 3. select the best-sized tree
Discussion
PPIG
Classification Tree
Random forests
PPIG revisited
Decisions trees for correlated binary data (Sharpsten et al., 2013)
Growing a large tree T0 • Marginal model, i = 1, . . . , n; k = 1, 2; j = 1, . . . , p
logit(pik ) = β0 + β1 I (Xikj ≤ c) I (Xikj ≤ c) or I (Xikj ∈ A) induces a split
Discussion
PPIG
Classification Tree
Random forests
PPIG revisited
Decisions trees for correlated binary data (Sharpsten et al., 2013)
Growing a large tree T0 • Marginal model, i = 1, . . . , n; k = 1, 2; j = 1, . . . , p
logit(pik ) = β0 + β1 I (Xikj ≤ c) I (Xikj ≤ c) or I (Xikj ∈ A) induces a split
• Splitting criteria: maximum robust Z from GEE
(for testing H0 : β1 = 0.)
Discussion
PPIG
Classification Tree
Random forests
PPIG revisited
Summarizing longitudinal measurements (Sharpsten et al., 2013)
• PLRA (Vesti et al., 2003, IOVS);
VFI (Leung et al., 2010, IOVS)
• Generalized least squares: visits l = 1, . . . , vi
Xl = α + βtl + ζl • Time windows (Gardiner et al., 2013, IOVS)
Discussion
PPIG
Classification Tree
Random forests
PPIG revisited
Discussion
Variable importance via extremely randomized trees Algorithm idea from Geurts et al. (2006, ML) • Generate bootstrap samples Lb , b = 1, . . . , B • Grow a tree Tb using Lb
(m randomly selected inputs at each split; 1-3 random cut-points per input; no pruning)
• Compute G (Tb ) using out-of-bag sample L − Lb • For all predictors Zj , j = 1, . . . , p • Permute the values of Zj in L − Lb • Compute Gj (Tb ) using permuted L − Lb
• Compute variable importance
Vj =
1 B
P
b {G (Tb )
− Gj (Tb )}/G (Tb )
PPIG
Classification Tree
Random forests
PPIG revisited
Prognostic groups
Group I II
Definition Less likely More likely
Number of eyes 166 166
Number (%) of progressed eyes 10 (6.4%) 83 (50.0%)
Robust Z II 58.7 –
Discussion
Classification Tree
Random forests
PPIG revisited
0.6 0.4 0.2
True positive rate
0.8
1.0
ROC comparisons
Proposed Method Best Tree via GEE Robust Z Demirel’s average tree 0.0
PPIG
0.0
0.2
0.4
0.6
False positive rate
0.8
1.0
Discussion
PPIG
Classification Tree
Random forests
PPIG revisited
Predicting pGON: Test-point importance
Discussion
PPIG
Classification Tree
Random forests
Current and future work • Joint models
• Bayesian trees
• Random forest variable importance bias
PPIG revisited
Discussion
PPIG
Classification Tree
Random forests
PPIG revisited
Discussion
Relevant decision tree literature
• Segal (1992, JASA): longitudinal quantitative responses
extending CART least squares split function
• Zhang (1998, JASA): multiple binary data using generalized
entropy criterion
• Lee (2005, DMKD): multiple binary data GEE residual-based
approach
• Fan et al. (2006, JASA): multivariate survival data
PPIG
Classification Tree
Random forests
PPIG revisited
Decision tree TP44 ≥ 30.17
TP1 < 30.95
13; 0%
TP1 < 32.12
TP32 ≥ 32.29
37; 22%
19; 42%
IOP < 15.5
7; 0%
TP53 ≥ 30.85
7; 14%
30; 23%
12; 100%
Discussion
PPIG
Classification Tree
Random forests
PPIG revisited
Decisions trees for correlated binary data Pruning algorithm • T0 fully grown tree
• Solve for α at each internal node h
Gα (Th ) = G (Th ) − α|Thi | = 0, P where G (T ) = h∈T i G (h). • Prune tree at the internal node h with smallest α. • Repeat to obtain sequences
α1 , α2 , . . . , αM T0 T1 · · · TM
Discussion
PPIG
Classification Tree
Random forests
PPIG revisited
Decisions trees for correlated binary data Pruning algorithm • T0 fully grown tree
• Solve for α at each internal node h
Gα (Th ) = G (Th ) − α|Thi | = 0, P where G (T ) = h∈T i G (h). • Prune tree at the internal node h with smallest α. • Repeat to obtain sequences
α1 , α2 , . . . , αM T0 T1 · · · TM Theorem: For any α ∈ [αm , αm+1 ], Tm is best subtree.
Discussion
PPIG
Classification Tree
Random forests
PPIG revisited
Decisions trees for correlated binary data Selecting the best-sized subtree • For fixed α in the range of 2 ≤ α ≤ 4,
ˆα (Tm ) max G
1≤m≤M
Discussion
PPIG
Classification Tree
Random forests
PPIG revisited
Discussion
Decisions trees for correlated binary data Selecting the best-sized subtree • For fixed α in the range of 2 ≤ α ≤ 4,
ˆα (Tm ) max G
1≤m≤M
• Bootstrap bias corrected estimate:
0 = bootstrap sample Lb , b = 1, . . . , B; αm
√
αm αm+1
0 ˆ (Tm ) = G (L; L, T (αm G )) −
(1/B)
B X b=1
0 0 G (Lb ; Lb , Tb (αm )) − G (L; Lb , Tb (αm ))
PPIG
Classification Tree
Random forests
PPIG revisited
Discussion
Best tree for predicting pGON
TP44 ≤ 31.46
! " ! " #$ " ! !"
3 39
TP4 ≤ 22.29
+ + +
I
)) )) ))
, , ,#$
!" ** ** II **
TP8 ≤ 28.52
14 23
TP6 ≤ 30.98
) ) )) ))
TP27 ≤ 27.76
%% % %
& &
TP12 ≤ 31.47
!"
I
TP13 ≤ 26.76
&&
TP32 ≤ 30.58
! " ! " #$ " !
1 14
TP44 ≤ 30.53
## $ $$#$ #$ # !"
!"
6 16
20 37
II
II
TP27 ≤ 16.04
** ** **
'' ( (( #$ #$ ' !" !"
2 43
8 17
I
II
## $ $$#$ #$ # !"
!"
15 33
13 28
II
II
'' ( (( #$ #$ ' !" !"
4 70
7 12
I
II
PPIG
Classification Tree
Random forests
PPIG follow-up data
PPIG revisited
Discussion
Classification Tree
Random forests
PPIG revisited
Predicting Glaucomatous Progression using Trees and Random Forests Richard A. Levine San Diego State University Joint work with Lucie Sharpsten, Juanjuan Fan, Xiaogang Su, & Shaban Demirel
Discussion
PPIG
Classification Tree
Random forests
PPIG revisited
Discussion
Perimetry and Psychophysics in Glaucoma (PPIG) Study
• 166 subjects with moderate- to high-risk ocular hypertension
• 332 eyes, subjects average age 58.1 (11.0 std), range 33 to 87
years
• 95 females, 71 males recruited in Portland, OR metro area • Median follow-up 6+ years, maximum 9+ years
• Outcome: progressive glaucomatous optic neuropathy (pGON) • 26 subjects pGON both eyes; 41 pGON in one eye • Goal: predict pGON using BL & FU data
PPIG
Classification Tree
Random forests
PPIG revisited
Humphrey Visual Field Analyzer: SAP
Discussion
PPIG
Classification Tree
Random forests
PPIG revisited
Clinical factors used in data analysis Both baseline and follow-up: • 52 test point thresholds • mean deviation (MD)
• pattern standard deviation (PSD)
Baseline only: • intraocular pressure (IOP in mmHg)
• central corneal thickness (CCT in microns) • gender indicator • age (in years)
Discussion
PPIG
Classification Tree
Random forests
PPIG revisited
Decision tree TP44 ≥ 30.17
TP1 < 30.95
13; 0%
TP1 < 32.12
TP32 ≥ 32.29
37; 22%
19; 42%
IOP < 15.5
7; 0%
TP53 ≥ 30.85
7; 14%
30; 23%
12; 100%
Discussion
PPIG
Classification Tree
Random forests
PPIG revisited
CART (Breiman et al., 1984)
• Regression problem: Y outcome/output, X covariates/input
• Why tree methods?
Fewer assumptions; interpretation; prognostic rules
• Partitioning covariate space • Three steps to the CART algorithm: 1. grow a large tree 2. prune to a nested sequence of subtrees 3. select the best-sized tree
Discussion
PPIG
Classification Tree
Random forests
PPIG revisited
Decisions trees for correlated binary data (Sharpsten et al., 2013)
Growing a large tree T0 • Marginal model, i = 1, . . . , n; k = 1, 2; j = 1, . . . , p
logit(pik ) = β0 + β1 I (Xikj ≤ c) I (Xikj ≤ c) or I (Xikj ∈ A) induces a split
Discussion
PPIG
Classification Tree
Random forests
PPIG revisited
Decisions trees for correlated binary data (Sharpsten et al., 2013)
Growing a large tree T0 • Marginal model, i = 1, . . . , n; k = 1, 2; j = 1, . . . , p
logit(pik ) = β0 + β1 I (Xikj ≤ c) I (Xikj ≤ c) or I (Xikj ∈ A) induces a split
• Splitting criteria: maximum robust Z from GEE
(for testing H0 : β1 = 0.)
Discussion
PPIG
Classification Tree
Random forests
PPIG revisited
Summarizing longitudinal measurements (Sharpsten et al., 2013)
• PLRA (Vesti et al., 2003, IOVS);
VFI (Leung et al., 2010, IOVS)
• Generalized least squares: visits l = 1, . . . , vi
Xl = α + βtl + ζl • Time windows (Gardiner et al., 2013, IOVS)
Discussion
PPIG
Classification Tree
Random forests
PPIG revisited
Discussion
Variable importance via extremely randomized trees Algorithm idea from Geurts et al. (2006, ML) • Generate bootstrap samples Lb , b = 1, . . . , B • Grow a tree Tb using Lb
(m randomly selected inputs at each split; 1-3 random cut-points per input; no pruning)
• Compute G (Tb ) using out-of-bag sample L − Lb • For all predictors Zj , j = 1, . . . , p • Permute the values of Zj in L − Lb • Compute Gj (Tb ) using permuted L − Lb
• Compute variable importance
Vj =
1 B
P
b {G (Tb )
− Gj (Tb )}/G (Tb )
PPIG
Classification Tree
Random forests
PPIG revisited
Prognostic groups
Group I II
Definition Less likely More likely
Number of eyes 166 166
Number (%) of progressed eyes 10 (6.4%) 83 (50.0%)
Robust Z II 58.7 –
Discussion
Classification Tree
Random forests
PPIG revisited
0.6 0.4 0.2
True positive rate
0.8
1.0
ROC comparisons
Proposed Method Best Tree via GEE Robust Z Demirel’s average tree 0.0
PPIG
0.0
0.2
0.4
0.6
False positive rate
0.8
1.0
Discussion
PPIG
Classification Tree
Random forests
PPIG revisited
Predicting pGON: Test-point importance
Discussion
PPIG
Classification Tree
Random forests
Current and future work • Joint models
• Bayesian trees
• Random forest variable importance bias
PPIG revisited
Discussion
PPIG
Classification Tree
Random forests
PPIG revisited
Discussion
Relevant decision tree literature
• Segal (1992, JASA): longitudinal quantitative responses
extending CART least squares split function
• Zhang (1998, JASA): multiple binary data using generalized
entropy criterion
• Lee (2005, DMKD): multiple binary data GEE residual-based
approach
• Fan et al. (2006, JASA): multivariate survival data
PPIG
Classification Tree
Random forests
PPIG revisited
Decision tree TP44 ≥ 30.17
TP1 < 30.95
13; 0%
TP1 < 32.12
TP32 ≥ 32.29
37; 22%
19; 42%
IOP < 15.5
7; 0%
TP53 ≥ 30.85
7; 14%
30; 23%
12; 100%
Discussion
PPIG
Classification Tree
Random forests
PPIG revisited
Decisions trees for correlated binary data Pruning algorithm • T0 fully grown tree
• Solve for α at each internal node h
Gα (Th ) = G (Th ) − α|Thi | = 0, P where G (T ) = h∈T i G (h). • Prune tree at the internal node h with smallest α. • Repeat to obtain sequences
α1 , α2 , . . . , αM T0 T1 · · · TM
Discussion
PPIG
Classification Tree
Random forests
PPIG revisited
Decisions trees for correlated binary data Pruning algorithm • T0 fully grown tree
• Solve for α at each internal node h
Gα (Th ) = G (Th ) − α|Thi | = 0, P where G (T ) = h∈T i G (h). • Prune tree at the internal node h with smallest α. • Repeat to obtain sequences
α1 , α2 , . . . , αM T0 T1 · · · TM Theorem: For any α ∈ [αm , αm+1 ], Tm is best subtree.
Discussion
PPIG
Classification Tree
Random forests
PPIG revisited
Decisions trees for correlated binary data Selecting the best-sized subtree • For fixed α in the range of 2 ≤ α ≤ 4,
ˆα (Tm ) max G
1≤m≤M
Discussion
PPIG
Classification Tree
Random forests
PPIG revisited
Discussion
Decisions trees for correlated binary data Selecting the best-sized subtree • For fixed α in the range of 2 ≤ α ≤ 4,
ˆα (Tm ) max G
1≤m≤M
• Bootstrap bias corrected estimate:
0 = bootstrap sample Lb , b = 1, . . . , B; αm
√
αm αm+1
0 ˆ (Tm ) = G (L; L, T (αm G )) −
(1/B)
B X b=1
0 0 G (Lb ; Lb , Tb (αm )) − G (L; Lb , Tb (αm ))
PPIG
Classification Tree
Random forests
PPIG revisited
Discussion
Best tree for predicting pGON
TP44 ≤ 31.46
! " ! " #$ " ! !"
3 39
TP4 ≤ 22.29
+ + +
I
)) )) ))
, , ,#$
!" ** ** II **
TP8 ≤ 28.52
14 23
TP6 ≤ 30.98
) ) )) ))
TP27 ≤ 27.76
%% % %
& &
TP12 ≤ 31.47
!"
I
TP13 ≤ 26.76
&&
TP32 ≤ 30.58
! " ! " #$ " !
1 14
TP44 ≤ 30.53
## $ $$#$ #$ # !"
!"
6 16
20 37
II
II
TP27 ≤ 16.04
** ** **
'' ( (( #$ #$ ' !" !"
2 43
8 17
I
II
## $ $$#$ #$ # !"
!"
15 33
13 28
II
II
'' ( (( #$ #$ ' !" !"
4 70
7 12
I
II
PPIG
Classification Tree
Random forests
PPIG follow-up data
PPIG revisited
Discussion
