Enabling Technologies for Autonomous Vehicles - Amazon Simple ...
Enabling Technologies for Autonomous Vehicles Sanjiv Nanda, VP Technology Qualcomm Research August 2017 Qualcomm Research Teams in Seoul, Amsterdam, Bedminster NJ, Philadelphia and San Diego
2
Delivering significant economic and societal impact Total potential economic impact of over $1 Trillion USD per year Fewer driving fatalities/injuries
More predictable, productive travel
>1.2M
Save Time
Save Lives
people die each year 2 on the roads worldwide
1
Less greenhouse gas emissions
3.1B 14%
gallons of fuels wasted due 3 traffic congestion in the US
Save Energy
of all global warming emissions from transportation4
Vision Zero: End traffic deaths and serious injuries by 2030 1
Rocky Mountain Institute 2016; 2 Global Status Report on Road Safety, World Health Organization 2015; 3 Texas Transportation Institute Urban Mobility Report, 2015;
4
U.S, Environmental Protection Agency (EPA) 2014
3
Autonomous Vehicles and Mobility as a Service Pace of Innovation, Multiple forces of Disruption Automakers & Tier 1’s Vehicle Fleet Providers
Technology Innovation (Autonomous Vehicle)
Autonomy Technology Providers
Business Model Innovation (Mobility as a Service)
Vehicle Ownership 4
Perception – Diverse Sensing Modalities
5
360◦ Perception On-board Sensors – Camera, Radar and Lidar Cocoon V2N
V2P Radar V2V Computer Vision
V2I
6
Perception Beyond the Sensory Horizon HD Maps and V2X Provide Side Information V2N
V2P Radar V2V Computer Vision
V2I
Improved active safety
Better traffic efficiency
Increased situational awareness
Provides 360◦ non-line-of-sight awareness, e.g. intersections/on-ramps, environmental conditions
Allows vehicles to safely drive closer to each other and enables optimization of overall traffic flow
Provides ability to gather data from further ahead to deliver a more predictable driving experience 7
Perception Beyond the Sensory Horizon
8
HD Maps Access to rich, accurate up-to-date HD Maps will be key to Autonomous Driving HD Map Semantic Information Examples Lane attributes − − − −
Lane direction of travel Lane speed limit Lane transition status Lane type
Semantic Information
Localization information
Lane attributes, direction /access restrictions, merges etc
6 DOF pose and precise coordinates of signs and other landmarks
Lane boundary attributes − − − −
Lane boundary traversal Lane marking color Lane marking style Lane marking material
Conditions − −
Access restriction condition Direction of travel condition
Crowdsourced creation and update of HD maps Triangulation & Bundle adjustment Qualcomm Technologies, Inc.
GPSS/GNSS + VIO + Map Fusion Sub-meter accuracy at 95th percentile 9
Precise Positioning: Lane Level Accuracy
10
Prediction and Planning Road World Model: Static and Dynamic Objects, Drivable Space, Road Semantics
11
Data Management Terabytes of data per day; Peta-scale data management
Lidar
Radars
Cameras
Discard
Validation
Annotation
Yes Drive Route Spec
Data Capture Capture (Online Triggers)
External Dataset
Storage
OfflinePreProcessing Processing Data Reduction
Training/ Validation
Testing/ Visualization
Data Extraction Simulated Dataset 12
The Autonomy Brain
Road World Model and Path Prediction Perception Positioning Planning Fusion
• 100x performance compared to smartphones ◦
high throughput sensor processing; sensor fusion; computer vision; machine learning / deep learning; path planning
Driver Monitoring
Algorithms: CV, ML, Fusion HW Architecture
SW Architecture
• High Performance Data Handling & Compute within Thermal Envelope • HW Accelerators
• Toolchains, libraries, SDK • RTOS
Infrastructure • Functional Safety • Fault Tolerance • Timing Synchronization
Product Platform
Next Generation Compute Platform
HMI and Control
• Within a tight power/thermal
budget
• Meeting security & functional safety requirements
Camera • •
Front-Facing Mono/Stereo /Trifocal Surround
L I D A R
RADAR • • •
Short-range Long-range Phased Array
Ultrasound
GNSS IMU
HD MAP
Camera
V2X (V2V, V2I, V2P, V2N)
Sensors 13
Functional Safety Addressed at a System Level Sensing and Algorithms Motion Control Massive Sensing Hardware Fail Diversity, Fusion …
Operational …
Software & Security Testing
Redundancy, Reliability, Diagnostics, Process and Audits, Self Test Certificates … …
Safety Standards, Test Specs, Emulation and Drive Tests …
14
WARNING Challenges, Innovation and Disruption Ahead
Leading to and Era of Safe, Efficient and Green Transportation
Thank you for your kind attention!
15
2
Delivering significant economic and societal impact Total potential economic impact of over $1 Trillion USD per year Fewer driving fatalities/injuries
More predictable, productive travel
>1.2M
Save Time
Save Lives
people die each year 2 on the roads worldwide
1
Less greenhouse gas emissions
3.1B 14%
gallons of fuels wasted due 3 traffic congestion in the US
Save Energy
of all global warming emissions from transportation4
Vision Zero: End traffic deaths and serious injuries by 2030 1
Rocky Mountain Institute 2016; 2 Global Status Report on Road Safety, World Health Organization 2015; 3 Texas Transportation Institute Urban Mobility Report, 2015;
4
U.S, Environmental Protection Agency (EPA) 2014
3
Autonomous Vehicles and Mobility as a Service Pace of Innovation, Multiple forces of Disruption Automakers & Tier 1’s Vehicle Fleet Providers
Technology Innovation (Autonomous Vehicle)
Autonomy Technology Providers
Business Model Innovation (Mobility as a Service)
Vehicle Ownership 4
Perception – Diverse Sensing Modalities
5
360◦ Perception On-board Sensors – Camera, Radar and Lidar Cocoon V2N
V2P Radar V2V Computer Vision
V2I
6
Perception Beyond the Sensory Horizon HD Maps and V2X Provide Side Information V2N
V2P Radar V2V Computer Vision
V2I
Improved active safety
Better traffic efficiency
Increased situational awareness
Provides 360◦ non-line-of-sight awareness, e.g. intersections/on-ramps, environmental conditions
Allows vehicles to safely drive closer to each other and enables optimization of overall traffic flow
Provides ability to gather data from further ahead to deliver a more predictable driving experience 7
Perception Beyond the Sensory Horizon
8
HD Maps Access to rich, accurate up-to-date HD Maps will be key to Autonomous Driving HD Map Semantic Information Examples Lane attributes − − − −
Lane direction of travel Lane speed limit Lane transition status Lane type
Semantic Information
Localization information
Lane attributes, direction /access restrictions, merges etc
6 DOF pose and precise coordinates of signs and other landmarks
Lane boundary attributes − − − −
Lane boundary traversal Lane marking color Lane marking style Lane marking material
Conditions − −
Access restriction condition Direction of travel condition
Crowdsourced creation and update of HD maps Triangulation & Bundle adjustment Qualcomm Technologies, Inc.
GPSS/GNSS + VIO + Map Fusion Sub-meter accuracy at 95th percentile 9
Precise Positioning: Lane Level Accuracy
10
Prediction and Planning Road World Model: Static and Dynamic Objects, Drivable Space, Road Semantics
11
Data Management Terabytes of data per day; Peta-scale data management
Lidar
Radars
Cameras
Discard
Validation
Annotation
Yes Drive Route Spec
Data Capture Capture (Online Triggers)
External Dataset
Storage
OfflinePreProcessing Processing Data Reduction
Training/ Validation
Testing/ Visualization
Data Extraction Simulated Dataset 12
The Autonomy Brain
Road World Model and Path Prediction Perception Positioning Planning Fusion
• 100x performance compared to smartphones ◦
high throughput sensor processing; sensor fusion; computer vision; machine learning / deep learning; path planning
Driver Monitoring
Algorithms: CV, ML, Fusion HW Architecture
SW Architecture
• High Performance Data Handling & Compute within Thermal Envelope • HW Accelerators
• Toolchains, libraries, SDK • RTOS
Infrastructure • Functional Safety • Fault Tolerance • Timing Synchronization
Product Platform
Next Generation Compute Platform
HMI and Control
• Within a tight power/thermal
budget
• Meeting security & functional safety requirements
Camera • •
Front-Facing Mono/Stereo /Trifocal Surround
L I D A R
RADAR • • •
Short-range Long-range Phased Array
Ultrasound
GNSS IMU
HD MAP
Camera
V2X (V2V, V2I, V2P, V2N)
Sensors 13
Functional Safety Addressed at a System Level Sensing and Algorithms Motion Control Massive Sensing Hardware Fail Diversity, Fusion …
Operational …
Software & Security Testing
Redundancy, Reliability, Diagnostics, Process and Audits, Self Test Certificates … …
Safety Standards, Test Specs, Emulation and Drive Tests …
14
WARNING Challenges, Innovation and Disruption Ahead
Leading to and Era of Safe, Efficient and Green Transportation
Thank you for your kind attention!
15
