Guidance for Improvement of Transportation Systems ... AWS
Guidance for Improvement of Transportation Systems Management and Operations 1
Capability Maturity – What it is • A conceptual model – establishing the basis for continuous improvement in agency effectiveness – A framework – A tool – Flexible applications
2
TSM&O Capability Maturity Activities • SHRP2 LO 6 Research and validation • FHWA CMM workshops
• SDOT CMM workshops • AASHTO/FHWA Implementation Plans
• Operations Academy/ROFs • Other applications of Capability Maturity
3
“Capabilities” for “Effective” TSM&O? • The Barriers (and capabilities to overcome) surprisingly) NOT : $$, technical knowledge • Real Challenges: lack of appropriate processes institutional arrangements • More difficult management challenges • NEEDED: A PRACTICAL MANAGEMENT TOOL 4
What “Capabilities”? The “Program”
• Identified characteristics of a effective TSM&O program
Processes that support Program
• Recognized dependence on specific business & technical processes
Supporting Institutional • Identified required organization structure & Framework relationships 5
Management Tool: The Capability Maturity concept 1. Objective of “continuous improvement” (mainstreaming)
2. Critical factors to be managed improvement (dimensions)
3. Defined and Doable targets of improvement (levels) 4. Based self-assessment
6
Guidance Component #1: Six Key Capability Dimensions 1. Business processes – planning/programming/budgeting 2. Systems & Technology – systems engineering/technology) 3. Performance -- measurement/data/utilization
4. Culture – understanding/leadership/formalization 5. Organization and workforce– structure/staff development
6. Collaboration –public and public/private 7
Guidance Component #2: Criteria for Capability Levels Goal for the Future LEVEL 4
Optimized LEVEL 3
Most Agencies Today
Integrated LEVEL 2
Managed LEVEL 1
Performed
• Processes developing • Staff training • Limited accountability
• Process documented • Performance measured • Organization/ partners aligned • Program budgeted
• Performance-based improvement • Formal program • Formal partnerships
• Activities & relationships ad hoc • Champion-driven
8
Criteria Define Levels and Targets DIMENSION: PLANNING DIMENSION: ORGANIZAION LEVEL 1 LEVEL 2 LEVEL 3 LEVEL 4 DIMENSION: PERFORMANCE MEASUREMENT LEVEL PERFORME 1 MANAGED INTEGRATED OPTIMIZING LEVEL 2 LEVEL 3 LEVEL 4 LEVEL 1 D LEVEL 2 LEVEL 3 LEVEL 4 PERFORME MANAGED INTEGRATED OPTIMIZING PERFORMED MANAGED INTEGRATED OPTIMIZED D Some outputs Output data used Outcome Performance Some outputs Output data Outcome Some measured outputs Output directly data for used afterOutcome measures Performance Performance measures measured and used directly measures measured and reported directly action forfor debriefings aftermeasures identified measures measures reported reported identified and reported bybysomeafter-action action and debriefings identified (networks,reported modes, reported internally for some debriefings and (networks, internally for by some jurisdictions and improvements; (networks, impacts); modes, and internally utilization for and jurisdictions impacts); utilization jurisdictions improvements; improvements; data easilymodes, impacts); routinely and utilized utilization in and externally and for data easily and routinely externally for for data available easily and routinely common utilized for in externally accountability available andand utilized inobjective-based dashboarded available common for accountability accountability and program dashboarded for dashboarded common justification objective-based program and program and program justification objective-based improvements justification program program improvements improvements 9
Guidance Component # 3: The “Rules” of Capability Maturity 1. Lowest level dimension is the principal constraint 2. All dimensions included are essential/synergistic 3. Levels cannot be skipped
10
CMM Self-Assessment Structure Dimensions
LEVEL 1 PERFORMED
Planning & Programming Systems & Technology
x
Performance
x
LEVEL 2 MANAGED
Lowest level is constraint – cant be skipped
LEVEL 4 OPTIMIZING
x “Ideal”
Culture
x
Organization/ staffing
x
Collaboration
LEVEL 3 INTEGRATED
x
For each dimension guidance provided to get to next level 11
Self-Assessment Workshops Strengths
Level Criteria
Consensus
Weaknesses
Level 1 — Performed
Level 2 — Managed
Level 3 — Integrated
Level 4 — Optimizing
Each jurisdiction doing its own thing according to individual priorities and capabilities
Consensus regional approach developed regarding TSM&O goals, deficiencies, B/C, networks, strategies and common priorities
Regional program integrated into jurisdictions’ overall multimodal transportation plans with related staged program
TSM&O integrated into jurisdictions’ multi-sectoral plans and programs, based on a formal, continuing planning processes
1.5
Workshop Actions to Advance to the Next Level
12
Advantages of CMM Approach • Vision of capability: target of continuous improvement • Small set of critical dimensions of needed capability • Criteria for improvements via doable incremental levels
• Priority actions needed to achieve next level thru self assessment
• Based on self-assessment 13
Product: CMM Utilization
14
Product: Detailed Web-based guidance
Business Processes Planning Scoping
Programming/Budgeting Project
Development Culture
Technical understanding Leadership Outreach Program Authorities
Systems & Technology Regional
architectures Systems engineering Interoperability
Performance Measurement
Measures
definition Data acquisition Measures utilization
Organization/ Workforce Program
status Organizational structure Recruitment/retention Staff development
Collaboration Public
safety agency collaboration Local government coop Outsourcing/PPP
15
F Product: FHWA/AASHTO CMM Workshops & Implementation Planning
16
FHWA/AASHTO CMM Workshops TO DATE • • • • • • • • • • • • • • • • •
Georgia DOT Colorado DOT/Denver Arizona DOT/Phoenix, Nevada DOT/Reno Tennessee DOT Pennsylvania DOT/Philadelphia Kansas City Metropolitan Region Caltrans/Los Angeles Florida DOT/District 4 Wisconsin DOT Dallas-Fort Worth/TXDOT Portland/ODOT Michigan DOT Caltrans (statewide) Maryland SHA/Baltimore New Jersey DOT New Hampshire DOT
UPCOMING in 2014 • • • • • • • • • • • • • • • • • • • •
New Jersey DOT ( Michigan DOT California DOT Oregon DOT Arizona DOT Colorado DOT Utah DOT Washington DOT Iowa DOT Ohio DOT Missouri DOT Kansas DOT District 5 South Dakota DOT Rhode Island DOT Pennsylvania DOT Maryland DOT (Baltimore/Washington)) Tennessee DOT Georgia DOT NITTC (Buffalo/Ontario) Florida DOT District 5)
17
Example Findings -- Current State of Practice: Planning, Programming, Budgeting Between
• Capability Level 1: Each jurisdiction doing its own thing according to individual priorities and capabilities
and
• Capability Level 2 : Consensus regional approach developed regarding TSM&O goals, deficiencies, B/C, networks, strategies and common priorities
State of Play -- Planning for TSM&O still ad hoc and
informal & lack of budget. But conscious of need for Ops Plans and integration into agency level planning. 18
Example Findings-- Current State of Practice: Performance Measurement Almost
• Capability Level 2: Output data used directly for afteraction debriefings and improvements; data easily available and dashboarded
State of Play – High consciousness (MAP-21) agencywide. Limited use by operators to improve procedures . Increased focus on measures and data challenges (including private supply).
19
Example Findings-- Current State of Practice: Organization/Workforce Just at
• Capability Level 2: TSM&O-specific organizational concept developed within/among jurisdictions with core capacity needs identified, collaboration takes place State-of-play -- Considerable consolidation & reorganization, staff limits. Training problems. Outsourcing increasing.
20
Other applications of CMM PLANNING AND PROGRAMMING FOR TSM&O Dimensions
Level 1: Ad Hoc
Level 2: Developing
Level 3: Specified
Cooperation/ Collaboration
No formal planning or programming
consideration at individual unit/agency level
Goals and Objectives
None related specifically to dealing with improving No analysis of current or anticipated
objectives understood/ incorporated
Coordination/ sharing of multiagency planning Overall agency policy/objectives/ strategies adjusted TSM&O-related forecasting used
improvements committed on opportunistic basis
Budget constrained evaluation of strategies
Needs/ Deficiency Analysis Plan Development
Rules of thumb used
Routine lifecycle comparison
Level 4: Mainstreamed integrated into regional interagency multimodal planning
TSM&O given appropriate agency priority Integration of TSM&O within overall forecasting and deficiency analysis integrated into overall agency priority-setting,
21
Lessons Learned • CMM easily understood by participants • CMM structure being adopted in several program areas • CMM self-assessment findings match informed professional observations • CMM results being used in State DOT TSM&O Implementation Plans 22
Capability Maturity – What it is • A conceptual model – establishing the basis for continuous improvement in agency effectiveness – A framework – A tool – Flexible applications
2
TSM&O Capability Maturity Activities • SHRP2 LO 6 Research and validation • FHWA CMM workshops
• SDOT CMM workshops • AASHTO/FHWA Implementation Plans
• Operations Academy/ROFs • Other applications of Capability Maturity
3
“Capabilities” for “Effective” TSM&O? • The Barriers (and capabilities to overcome) surprisingly) NOT : $$, technical knowledge • Real Challenges: lack of appropriate processes institutional arrangements • More difficult management challenges • NEEDED: A PRACTICAL MANAGEMENT TOOL 4
What “Capabilities”? The “Program”
• Identified characteristics of a effective TSM&O program
Processes that support Program
• Recognized dependence on specific business & technical processes
Supporting Institutional • Identified required organization structure & Framework relationships 5
Management Tool: The Capability Maturity concept 1. Objective of “continuous improvement” (mainstreaming)
2. Critical factors to be managed improvement (dimensions)
3. Defined and Doable targets of improvement (levels) 4. Based self-assessment
6
Guidance Component #1: Six Key Capability Dimensions 1. Business processes – planning/programming/budgeting 2. Systems & Technology – systems engineering/technology) 3. Performance -- measurement/data/utilization
4. Culture – understanding/leadership/formalization 5. Organization and workforce– structure/staff development
6. Collaboration –public and public/private 7
Guidance Component #2: Criteria for Capability Levels Goal for the Future LEVEL 4
Optimized LEVEL 3
Most Agencies Today
Integrated LEVEL 2
Managed LEVEL 1
Performed
• Processes developing • Staff training • Limited accountability
• Process documented • Performance measured • Organization/ partners aligned • Program budgeted
• Performance-based improvement • Formal program • Formal partnerships
• Activities & relationships ad hoc • Champion-driven
8
Criteria Define Levels and Targets DIMENSION: PLANNING DIMENSION: ORGANIZAION LEVEL 1 LEVEL 2 LEVEL 3 LEVEL 4 DIMENSION: PERFORMANCE MEASUREMENT LEVEL PERFORME 1 MANAGED INTEGRATED OPTIMIZING LEVEL 2 LEVEL 3 LEVEL 4 LEVEL 1 D LEVEL 2 LEVEL 3 LEVEL 4 PERFORME MANAGED INTEGRATED OPTIMIZING PERFORMED MANAGED INTEGRATED OPTIMIZED D Some outputs Output data used Outcome Performance Some outputs Output data Outcome Some measured outputs Output directly data for used afterOutcome measures Performance Performance measures measured and used directly measures measured and reported directly action forfor debriefings aftermeasures identified measures measures reported reported identified and reported bybysomeafter-action action and debriefings identified (networks,reported modes, reported internally for some debriefings and (networks, internally for by some jurisdictions and improvements; (networks, impacts); modes, and internally utilization for and jurisdictions impacts); utilization jurisdictions improvements; improvements; data easilymodes, impacts); routinely and utilized utilization in and externally and for data easily and routinely externally for for data available easily and routinely common utilized for in externally accountability available andand utilized inobjective-based dashboarded available common for accountability accountability and program dashboarded for dashboarded common justification objective-based program and program and program justification objective-based improvements justification program program improvements improvements 9
Guidance Component # 3: The “Rules” of Capability Maturity 1. Lowest level dimension is the principal constraint 2. All dimensions included are essential/synergistic 3. Levels cannot be skipped
10
CMM Self-Assessment Structure Dimensions
LEVEL 1 PERFORMED
Planning & Programming Systems & Technology
x
Performance
x
LEVEL 2 MANAGED
Lowest level is constraint – cant be skipped
LEVEL 4 OPTIMIZING
x “Ideal”
Culture
x
Organization/ staffing
x
Collaboration
LEVEL 3 INTEGRATED
x
For each dimension guidance provided to get to next level 11
Self-Assessment Workshops Strengths
Level Criteria
Consensus
Weaknesses
Level 1 — Performed
Level 2 — Managed
Level 3 — Integrated
Level 4 — Optimizing
Each jurisdiction doing its own thing according to individual priorities and capabilities
Consensus regional approach developed regarding TSM&O goals, deficiencies, B/C, networks, strategies and common priorities
Regional program integrated into jurisdictions’ overall multimodal transportation plans with related staged program
TSM&O integrated into jurisdictions’ multi-sectoral plans and programs, based on a formal, continuing planning processes
1.5
Workshop Actions to Advance to the Next Level
12
Advantages of CMM Approach • Vision of capability: target of continuous improvement • Small set of critical dimensions of needed capability • Criteria for improvements via doable incremental levels
• Priority actions needed to achieve next level thru self assessment
• Based on self-assessment 13
Product: CMM Utilization
14
Product: Detailed Web-based guidance
Business Processes Planning Scoping
Programming/Budgeting Project
Development Culture
Technical understanding Leadership Outreach Program Authorities
Systems & Technology Regional
architectures Systems engineering Interoperability
Performance Measurement
Measures
definition Data acquisition Measures utilization
Organization/ Workforce Program
status Organizational structure Recruitment/retention Staff development
Collaboration Public
safety agency collaboration Local government coop Outsourcing/PPP
15
F Product: FHWA/AASHTO CMM Workshops & Implementation Planning
16
FHWA/AASHTO CMM Workshops TO DATE • • • • • • • • • • • • • • • • •
Georgia DOT Colorado DOT/Denver Arizona DOT/Phoenix, Nevada DOT/Reno Tennessee DOT Pennsylvania DOT/Philadelphia Kansas City Metropolitan Region Caltrans/Los Angeles Florida DOT/District 4 Wisconsin DOT Dallas-Fort Worth/TXDOT Portland/ODOT Michigan DOT Caltrans (statewide) Maryland SHA/Baltimore New Jersey DOT New Hampshire DOT
UPCOMING in 2014 • • • • • • • • • • • • • • • • • • • •
New Jersey DOT ( Michigan DOT California DOT Oregon DOT Arizona DOT Colorado DOT Utah DOT Washington DOT Iowa DOT Ohio DOT Missouri DOT Kansas DOT District 5 South Dakota DOT Rhode Island DOT Pennsylvania DOT Maryland DOT (Baltimore/Washington)) Tennessee DOT Georgia DOT NITTC (Buffalo/Ontario) Florida DOT District 5)
17
Example Findings -- Current State of Practice: Planning, Programming, Budgeting Between
• Capability Level 1: Each jurisdiction doing its own thing according to individual priorities and capabilities
and
• Capability Level 2 : Consensus regional approach developed regarding TSM&O goals, deficiencies, B/C, networks, strategies and common priorities
State of Play -- Planning for TSM&O still ad hoc and
informal & lack of budget. But conscious of need for Ops Plans and integration into agency level planning. 18
Example Findings-- Current State of Practice: Performance Measurement Almost
• Capability Level 2: Output data used directly for afteraction debriefings and improvements; data easily available and dashboarded
State of Play – High consciousness (MAP-21) agencywide. Limited use by operators to improve procedures . Increased focus on measures and data challenges (including private supply).
19
Example Findings-- Current State of Practice: Organization/Workforce Just at
• Capability Level 2: TSM&O-specific organizational concept developed within/among jurisdictions with core capacity needs identified, collaboration takes place State-of-play -- Considerable consolidation & reorganization, staff limits. Training problems. Outsourcing increasing.
20
Other applications of CMM PLANNING AND PROGRAMMING FOR TSM&O Dimensions
Level 1: Ad Hoc
Level 2: Developing
Level 3: Specified
Cooperation/ Collaboration
No formal planning or programming
consideration at individual unit/agency level
Goals and Objectives
None related specifically to dealing with improving No analysis of current or anticipated
objectives understood/ incorporated
Coordination/ sharing of multiagency planning Overall agency policy/objectives/ strategies adjusted TSM&O-related forecasting used
improvements committed on opportunistic basis
Budget constrained evaluation of strategies
Needs/ Deficiency Analysis Plan Development
Rules of thumb used
Routine lifecycle comparison
Level 4: Mainstreamed integrated into regional interagency multimodal planning
TSM&O given appropriate agency priority Integration of TSM&O within overall forecasting and deficiency analysis integrated into overall agency priority-setting,
21
Lessons Learned • CMM easily understood by participants • CMM structure being adopted in several program areas • CMM self-assessment findings match informed professional observations • CMM results being used in State DOT TSM&O Implementation Plans 22
