Gateway Course Plenary
Predictive Analytics, Choice Architecture and Bespoke Education
Dr. Tristan Denley
Vice Chancellor for Academic Affairs
Tennessee Board of Regents
[email protected]
Twitter: @TDenley
WHAT IS YOUR VANTAGE POINT?
YOU ARE A CHOICE ARCHITECT
IS DATA ABOUT THE PAST OR THE FUTURE
50000
37500
25000
12500
0
More than 8000 courses system-wide
50000
37500
ENGL1010 ENGL1020
Top 30 most enrolled classes comprise 50% of TBR enrollment
HIST2010 25000
MATH1530 HIST2020
12500
0
TBR Revitalization Initiative
• Partnership between teams of faculty
• 120 projects proposed - 57 selected and at work
50000
37500
25000
12500
0
30000
22500
15000
7500
0
System-wide Math Course Enrollments
30000
24000
18000
12000
Statistics
Top 4 most enrolled classes comprise 50% of TBR enrollment
College Algebra
PreCalculus Liberal Arts Math
6000
0
Calculus
30000
22500
College Algebra, PreCal or Trig 15000
MATH1910 7500
0
30000
22500
College Algebra, PreCal or Trig
Statistics
15000
Calculus 7500
0
Curricular Fit
What is Success??
Traditional Model at APSU
DSPM 0800
DSPM 0850
Elementary Algebra
! ACT 15-16
53.1%
Intermediate Algebra
! ACT 17-18; or
Completed DSPM 0800
51.2%
!
Completed course with a grade of C or higher
!
completed course with a grade of C or higher
Traditional Model at APSU
DSPM 0800
DSPM 0850
MATH1010
Elementary Algebra
! ACT 15-16
53.1%
Intermediate Algebra
! ACT 17-18; or
Completed DSPM 0800
51.2%
Mathematical Thought and Practice
!
Completed course with a grade of C or higher
!
completed course with a grade of C or higher
!
!
(core)
43.5% earned a D or higher
Math !1530
Fundamentals
of Statistics
! 28.8% earned a D or higher
Co-requisite Instruction
Final Grades 30%
Enhanced Regular
23%
15%
8%
0%
A
B
C
D
F
W
Final Grades 30%
Enhanced Regular
Pass%
23%
15%
8%
0%
A
Traditional Redesigned
Liberal Arts
43%
84%
Elementary Stats
28%
71%
B
C
D
F
W
Final Grades 30%
Enhanced Regular
Pass%
23%
15%
8%
0%
A
Traditional Redesigned
Liberal Arts
11%
84%
Elementary Stats
7.5%
71%
B
C
D
F
W
System-wide Math Completion 80%
60%
40%
20%
0% 13
14
15
16
17
18
19
20
21
22
23
ACT Math Subscores
Prepare for Credit Model
Immediate Credit Model
24
25
System-wide Math Completion 80%
60%
40%
20%
0% 13
14
15
16
17
18
19
20
21
22
23
ACT Math Subscores
Prepare for Credit Model
Immediate Credit Model
24
25
System-wide Math Completion 80%
60%
10 Standard Deviations
40%
20%
0% 13
14
15
16
17
18
19
20
21
22
23
ACT Math Subscores
Prepare for Credit Model
Immediate Credit Model
24
25
System-wide Math Completion 80%
60%
40%
20%
0% 13
14
15
16
17
18
19
20
21
22
23
ACT Math Subscores
Prepare for Credit Model
Immediate Credit Model
24
25
COMMUNITY, BELONGING AND
INCLUSION
Rethinking Math Rethinking Reading
g Rethinitikning Wr
Intentionally build into the class design what we know leads to success
Replaced watching with doing
Thinking about
LEARNING SOFTWARE
Controlled Study at University of Mississippi Software A
2 Fall sections of College Algebra
100 students
Software B
2 Fall sections of College Algebra
113 students
No software
4 fall sections of College Algebra
200 students
Final Exams 40
Software A Software B No Software
34.286 28.571 22.857 17.143 11.429 5.714 0
90s
80s
70s
60s
50s
40s
30s
20s
10s
Final Exams 40
Software A Software B No Software
34.286 28.571 22.857 17.143 11.429 5.714 0
90s
80s
70s
60s
50s
40s
30s
20s
10s
Final Exams 40
Software A Software B No Software
34.286 28.571 22.857 17.143 11.429 5.714 0
90s
80s
70s
60s
50s
40s
30s
20s
10s
Final Exams 40
Software A Software B No Software
34.286 28.571 22.857 17.143 11.429 5.714 0
90s
80s
70s
60s
50s
40s
30s
20s
10s
Final Exams 40
Software A Software B No Software
34.286 28.571
Mean
22.857
Hawkes LS
81
17.143
MyMathlab
68
11.429
No Software
70
5.714 0
90s
80s
70s
60s
50s
40s
30s
20s
10s
Typical Math Department
100 level classes 200 level classes 300 level classes Graduate classes
Service classes Major classes Graduate classes
The Story Fall 2000
•College Algebra •Elementary Statistics
966 students
•College Algebra •Elementary Statistics
1070 students
307 students
Fall 2007
864 students
The hidden story !
Fall 2000
• • • • • •
Discrete Mathematics Introduction to Linear Algebra Elementary Differential Equations Modern Algebra I History of Mathematics Advanced Calculus I
Spring 2001
• • • • • • •
Applied Modern Algebra Foundations of Mathematics Elementary Differential Equations Intermediate Differential Equations Modern Algebra II Advanced Calculus II Applied Probability I
• • • • • • • • • • • • •
• • • • • •
Fall 2007 !
Discrete Mathematics Foundations of Mathematics Introduction to Linear Algebra Elementary Differential Equations Introduction to Statistical Methods Techniques in Teaching Sec. Level Math Introduction to Complex Analysis Introduction to Acturial Science Modern Algebra I Topics in Euclidean Geometry History of Mathematics Advanced Calculus I Mathematical Statistics I
!
Spring 2008 ! !
Discrete Mathematics Applied Modern Algebra Foundations of Mathematics Introduction to Linear Algebra Elementary Differential Equations Introduction to Statistical Methods
• • •
Combinatorics Introduction to Abstract Algebra Introduction to Mathematical Statistics
• • • • • •
Topology Number Theory Modern Algebra II Advanced Calculus II Mathematical Statistics II Applied Multiple Regression
! !
The hidden story !
2000-2001
2007-2008
45 Math Majors
81 Math Majors
40 BA students
50 BA students
5 BS Students
31 BS students
PhD program put on probation
23 PhDs over 5 years
INDIVIDUALIZTION?
What ingredients go into the ratings Global centrality Major centrality Grade prediction
GRADE PREDICTIONS
•Predicts grades to within 0.6 of a letter grade on average
•Correctly distinguishes
whether a student will earn an ABC or DF 92% of the time
•5.3SD increase in ABC% campus-wide. •7.7SD increase in ABC% for Pell Grant recipients.
Earned Hours
University #1 Comm. College #1 University #2 Comm. College #2
Recommended Classes
•Average Earned Hours very highly
correlated with recommended classes
Average Gap with Degree Compass
Average Gap for All Schedules
12
Earned Hours
9
12
10.06
2.0 8.06
6
9
3
0
0
Minority
Students
10.30
6
3
White
11.00
0.7
White
Minority
Students
•Achievement gap almost removed when students take recommended schedules
Average Gap with Degree Compass
Average Gap for All Schedules
12
12
Earned Hours
0.74
1.72
9
9
6
6
3
3
0
0
Non Pell
Pell
Students
Non Pell
Pell
Students
•Achievement gap almost removed when students take recommended schedules
Predictive Modeling to anticipate course demand
Students should take the courses they need not what’s left over
WHICH MAJOR?
Fingerprint Courses capture the distinctive flavour and structure of each major
Predicted Grades
Major Fingerprint GPA
Physics Fingerprint Classes Semester 1
Semester 2
Semester 3
Semester 4
PHYS 2110
PHYS 2120
PHYS 3550
PHYS3005
PHYS 2111
PHYS 2121
MATH 2110
PHYS 3700
MATH 1910
MATH 1920
CSCI 2000
MATH 3110
Graduates’ Transcript Analysis Percentage of graduates
90% 75% 60% 45% 30% 15% 0%
0
1
2
3
4
5
Number of F’s on transcript in FINGERPRINT classes
C A R E E R S U G G E S T I O N T O O L F I N D A M A J O R F I N D A C A R E E R
Agriculture Biology Computer Sci HHP
Successful Courses
Job Cloud
C A R E E R S U G G E S T I O N T O O L F I N D A M A J O R F I N D A C A R E E R
Program Description Degree Map
Successful Courses
Job Cloud
C A R E E R S U G G E S T I O N T O O L F I N D A M A J O R F I N D A C A R E E R
Program Description Degree Map
6yr Graduation Rates 45% 40%
All URM White Pell Non-Pell
35% 30% 25% 20% 15% 2008-9
2009-10
2010-11
2011-12
2012-13
Thank you Dr. Tristan Denley
Vice Chancellor for Academic Affairs
Tennessee Board of Regents
[email protected]
Twitter: @TDenley
Dr. Tristan Denley
Vice Chancellor for Academic Affairs
Tennessee Board of Regents
[email protected]
Twitter: @TDenley
WHAT IS YOUR VANTAGE POINT?
YOU ARE A CHOICE ARCHITECT
IS DATA ABOUT THE PAST OR THE FUTURE
50000
37500
25000
12500
0
More than 8000 courses system-wide
50000
37500
ENGL1010 ENGL1020
Top 30 most enrolled classes comprise 50% of TBR enrollment
HIST2010 25000
MATH1530 HIST2020
12500
0
TBR Revitalization Initiative
• Partnership between teams of faculty
• 120 projects proposed - 57 selected and at work
50000
37500
25000
12500
0
30000
22500
15000
7500
0
System-wide Math Course Enrollments
30000
24000
18000
12000
Statistics
Top 4 most enrolled classes comprise 50% of TBR enrollment
College Algebra
PreCalculus Liberal Arts Math
6000
0
Calculus
30000
22500
College Algebra, PreCal or Trig 15000
MATH1910 7500
0
30000
22500
College Algebra, PreCal or Trig
Statistics
15000
Calculus 7500
0
Curricular Fit
What is Success??
Traditional Model at APSU
DSPM 0800
DSPM 0850
Elementary Algebra
! ACT 15-16
53.1%
Intermediate Algebra
! ACT 17-18; or
Completed DSPM 0800
51.2%
!
Completed course with a grade of C or higher
!
completed course with a grade of C or higher
Traditional Model at APSU
DSPM 0800
DSPM 0850
MATH1010
Elementary Algebra
! ACT 15-16
53.1%
Intermediate Algebra
! ACT 17-18; or
Completed DSPM 0800
51.2%
Mathematical Thought and Practice
!
Completed course with a grade of C or higher
!
completed course with a grade of C or higher
!
!
(core)
43.5% earned a D or higher
Math !1530
Fundamentals
of Statistics
! 28.8% earned a D or higher
Co-requisite Instruction
Final Grades 30%
Enhanced Regular
23%
15%
8%
0%
A
B
C
D
F
W
Final Grades 30%
Enhanced Regular
Pass%
23%
15%
8%
0%
A
Traditional Redesigned
Liberal Arts
43%
84%
Elementary Stats
28%
71%
B
C
D
F
W
Final Grades 30%
Enhanced Regular
Pass%
23%
15%
8%
0%
A
Traditional Redesigned
Liberal Arts
11%
84%
Elementary Stats
7.5%
71%
B
C
D
F
W
System-wide Math Completion 80%
60%
40%
20%
0% 13
14
15
16
17
18
19
20
21
22
23
ACT Math Subscores
Prepare for Credit Model
Immediate Credit Model
24
25
System-wide Math Completion 80%
60%
40%
20%
0% 13
14
15
16
17
18
19
20
21
22
23
ACT Math Subscores
Prepare for Credit Model
Immediate Credit Model
24
25
System-wide Math Completion 80%
60%
10 Standard Deviations
40%
20%
0% 13
14
15
16
17
18
19
20
21
22
23
ACT Math Subscores
Prepare for Credit Model
Immediate Credit Model
24
25
System-wide Math Completion 80%
60%
40%
20%
0% 13
14
15
16
17
18
19
20
21
22
23
ACT Math Subscores
Prepare for Credit Model
Immediate Credit Model
24
25
COMMUNITY, BELONGING AND
INCLUSION
Rethinking Math Rethinking Reading
g Rethinitikning Wr
Intentionally build into the class design what we know leads to success
Replaced watching with doing
Thinking about
LEARNING SOFTWARE
Controlled Study at University of Mississippi Software A
2 Fall sections of College Algebra
100 students
Software B
2 Fall sections of College Algebra
113 students
No software
4 fall sections of College Algebra
200 students
Final Exams 40
Software A Software B No Software
34.286 28.571 22.857 17.143 11.429 5.714 0
90s
80s
70s
60s
50s
40s
30s
20s
10s
Final Exams 40
Software A Software B No Software
34.286 28.571 22.857 17.143 11.429 5.714 0
90s
80s
70s
60s
50s
40s
30s
20s
10s
Final Exams 40
Software A Software B No Software
34.286 28.571 22.857 17.143 11.429 5.714 0
90s
80s
70s
60s
50s
40s
30s
20s
10s
Final Exams 40
Software A Software B No Software
34.286 28.571 22.857 17.143 11.429 5.714 0
90s
80s
70s
60s
50s
40s
30s
20s
10s
Final Exams 40
Software A Software B No Software
34.286 28.571
Mean
22.857
Hawkes LS
81
17.143
MyMathlab
68
11.429
No Software
70
5.714 0
90s
80s
70s
60s
50s
40s
30s
20s
10s
Typical Math Department
100 level classes 200 level classes 300 level classes Graduate classes
Service classes Major classes Graduate classes
The Story Fall 2000
•College Algebra •Elementary Statistics
966 students
•College Algebra •Elementary Statistics
1070 students
307 students
Fall 2007
864 students
The hidden story !
Fall 2000
• • • • • •
Discrete Mathematics Introduction to Linear Algebra Elementary Differential Equations Modern Algebra I History of Mathematics Advanced Calculus I
Spring 2001
• • • • • • •
Applied Modern Algebra Foundations of Mathematics Elementary Differential Equations Intermediate Differential Equations Modern Algebra II Advanced Calculus II Applied Probability I
• • • • • • • • • • • • •
• • • • • •
Fall 2007 !
Discrete Mathematics Foundations of Mathematics Introduction to Linear Algebra Elementary Differential Equations Introduction to Statistical Methods Techniques in Teaching Sec. Level Math Introduction to Complex Analysis Introduction to Acturial Science Modern Algebra I Topics in Euclidean Geometry History of Mathematics Advanced Calculus I Mathematical Statistics I
!
Spring 2008 ! !
Discrete Mathematics Applied Modern Algebra Foundations of Mathematics Introduction to Linear Algebra Elementary Differential Equations Introduction to Statistical Methods
• • •
Combinatorics Introduction to Abstract Algebra Introduction to Mathematical Statistics
• • • • • •
Topology Number Theory Modern Algebra II Advanced Calculus II Mathematical Statistics II Applied Multiple Regression
! !
The hidden story !
2000-2001
2007-2008
45 Math Majors
81 Math Majors
40 BA students
50 BA students
5 BS Students
31 BS students
PhD program put on probation
23 PhDs over 5 years
INDIVIDUALIZTION?
What ingredients go into the ratings Global centrality Major centrality Grade prediction
GRADE PREDICTIONS
•Predicts grades to within 0.6 of a letter grade on average
•Correctly distinguishes
whether a student will earn an ABC or DF 92% of the time
•5.3SD increase in ABC% campus-wide. •7.7SD increase in ABC% for Pell Grant recipients.
Earned Hours
University #1 Comm. College #1 University #2 Comm. College #2
Recommended Classes
•Average Earned Hours very highly
correlated with recommended classes
Average Gap with Degree Compass
Average Gap for All Schedules
12
Earned Hours
9
12
10.06
2.0 8.06
6
9
3
0
0
Minority
Students
10.30
6
3
White
11.00
0.7
White
Minority
Students
•Achievement gap almost removed when students take recommended schedules
Average Gap with Degree Compass
Average Gap for All Schedules
12
12
Earned Hours
0.74
1.72
9
9
6
6
3
3
0
0
Non Pell
Pell
Students
Non Pell
Pell
Students
•Achievement gap almost removed when students take recommended schedules
Predictive Modeling to anticipate course demand
Students should take the courses they need not what’s left over
WHICH MAJOR?
Fingerprint Courses capture the distinctive flavour and structure of each major
Predicted Grades
Major Fingerprint GPA
Physics Fingerprint Classes Semester 1
Semester 2
Semester 3
Semester 4
PHYS 2110
PHYS 2120
PHYS 3550
PHYS3005
PHYS 2111
PHYS 2121
MATH 2110
PHYS 3700
MATH 1910
MATH 1920
CSCI 2000
MATH 3110
Graduates’ Transcript Analysis Percentage of graduates
90% 75% 60% 45% 30% 15% 0%
0
1
2
3
4
5
Number of F’s on transcript in FINGERPRINT classes
C A R E E R S U G G E S T I O N T O O L F I N D A M A J O R F I N D A C A R E E R
Agriculture Biology Computer Sci HHP
Successful Courses
Job Cloud
C A R E E R S U G G E S T I O N T O O L F I N D A M A J O R F I N D A C A R E E R
Program Description Degree Map
Successful Courses
Job Cloud
C A R E E R S U G G E S T I O N T O O L F I N D A M A J O R F I N D A C A R E E R
Program Description Degree Map
6yr Graduation Rates 45% 40%
All URM White Pell Non-Pell
35% 30% 25% 20% 15% 2008-9
2009-10
2010-11
2011-12
2012-13
Thank you Dr. Tristan Denley
Vice Chancellor for Academic Affairs
Tennessee Board of Regents
[email protected]
Twitter: @TDenley
