automated driving data chain challenges
Maxime Flament, ERTICO – ITS Europe
AUTOMATED DRIVING DATA CHAIN CHALLENGES Flament | Automated Vehicle Symposium | July 2017
1
Tomorrow’s Situation: Sensors, Maps and Online Data The Vehicle Looks beyond 300m and Around the Corner
3. Extension of limited in-vehicle resources
1. Highly accurate map model provided and updated via the Backend
2. Extended preview information
Close preview: 10 minutes
4. Fleet based data collection
Vehicle Sensor range 0-300m
GSV-Forum Automatisiertes Fahren Public
October 2016 FF@ Continental AG
2
Automated driving data chain and ecosystem Reference architecture proposed by OADF - - - - -- - - INFRASTRUCTURE - - - - - - - - - -
- - - - - - - - - - - - - - - - - - VEHICLE - - - - - - - - -
N A S T
NDS
Backend (one or multiple) operated by Map supplier / OEM / 3rd Party
ADAS Horizon Provider
OEM-specific Backends
Live Map Updates (Dynamic Data) Maps & Updates
N T
Live Map Delivery
N T
HD Map Delivery
N
Local Live Map (Dynamic Data)
A
HD Map
Environmental Model
A
Analytics/Sensor Fusion
Localization
Data Collection Strategy
Community Data Store
(V2x) provided) Sensors
N
(Dyn.) Data Production
Systematic Data Collection (Mobile Mapping)
(Local) Sensors
Infotainment Map Delivery
S
Data Store
N
Infotainment Map
A
S S
Flament | Automated Vehicle Symposium | July 2017
Data Collection (raw or pre-processed)
3
INTRODUCTION TO ADASIS
Flament | Automated Vehicle Symposium | July 2017
4
ADASIS horizon along the map data chain TPEG: Traffic Information NDS: Incremental Map Update
Map database
In-vehicle systems HMI
Layers e. g. hazard spots
e. g. traffic information
Determination of location and most probable path (MPP)
e. g. speed limits
30
Digital map (static)
Map data (NDS Format)
Enrichment of MPP with road information (e.g. topography, speed limits)
ADAS, Energy Management
Relevant information for road ahead (ADASIS Format)
Conversion into ADASIS format
Flament | Automated Vehicle Symposium | July 2017
5
ADASIS horizon – evolution to the cloud
Cloud
Hybrid
Local
In-vehicle embedded software
Cloud-based backend software
function
horizon provider
map
Link to NDS
function
horizon provider
map
map
function
horizon provider
map
map
Link to Sensoris, NDS & iii. TPEG
function
i.
Link to NDS
ii. dynamic data
horizon provider
map
Link to NDS & TPEG dynamic data
i.
ADASIS Horizon uses data from local map database and provides information for functions in the vehicle
ii.
ADASIS Horizon uses recent data from updated map database via cloud backend or live data from additional databases and provides information for functions in the vehicle
iii. ADASIS Horizon uses cloud-based data and provides information for in-vehicle functions directly from the cloud
Flament | Automated Vehicle Symposium | July 2017
6
ADASIS V3 specification supports highly automated driving New ADASIS V3 specifications supports different aspects of autonomous driving 1.
Support of HAD maps (NDS)
•
Flament | Automated Vehicle Symposium | July 2017
Copyright: Here.com
7
ADASIS V3 specification supports highly automated driving Lane model & Geometry Example: Highway Exit “switch from a map-centric view to a vehicle-centric view of the world to create a simpler representation”
© Wikimedia Commons, licensed by CC-by-SA 2.5
Flament | Automated Vehicle Symposium | July 2017
8
ADASIS V3 specification supports highly automated driving Lane model: Segmentation of road into stretches with Segmentation of road same lane characteristics into stretches with 1stsame level =lane pathcharacteristics description
3 Lanes
level detail: 1st levelof= lanes path description number for each segment level detail: number of lanes for each segment
1 Lane
3+1 Lanes
3 Lanes + shoulder © Wikimedia Commons, licensed by CC-by-SA 2.5
Flament | Automated Vehicle Symposium | July 2017
9
ADASIS V3 specification supports highly automated driving Lane Model & Connectivity Lane Model & Connectivity: Logical Description similar to v2 Lane #3: Lane #2: normal normal Lane #1: normal
2nd level = logical view level detail: lane details and their connectivity
Lane #4: normal
Lane #3: normal
Lane #3: normal
Lane #2: normal
Lane #1: ramp
Lane #2: normal
Lane #1: normal © Wikimedia Commons, licensed by CC-by-SA 2.5
Flament | Automated Vehicle Symposium | July 2017
10
ADASIS V3 specification supports highly automated driving Lane Model & Connectivity Geometry for Linear Objects: • Lane Boundaries Geometry for Linear Objects: • Guardrails • Lane Boundaries level = geometry description •3rdGuardrails level detail: rd level = geometry description 3geometry of road, lane and lines level detail: geometry of road, lane and lines
Lane #3: normal
lines and boundaries
Lane #2: normal
Lane #1: normal © Wikimedia Commons, licensed by CC-by-SA 2.5
Flament | Automated Vehicle Symposium | July 2017
11
ADASIS Forum Membership & Steering Board
From 46 to 52 ADASIS Forum Members (updated 31 may 2017) Vehicle Manufacturers (16)
ADAS Suppliers
(15)
BMW China FAW-RDC ** CRF (FCA) ** Daimler * Ford * Ford-Otosan Honda * Hyundai Motor Company Jaguar Opel * Nissan Renault Toyota Motor Corp. Volkswagen Volvo Car Corp. Volvo Tech. Dev. Corp.
Autonomos Continental Automotive * CTAG Denso dSPACE Fujitsu Ten (Europe) ** Hitachi Ibeo IPG LG Electronic Magna Electronic Europe Magneti Marelli Novero TRW (ZF) Valeo
Navigation System Suppliers (13) AISIN AW Alpine Autoliv Elektrobit Automotive Garmin Harman Mappers Co. ** Mitsubishi Electric Europe MXNAVI NNG LLC Panasonic Robert Bosch GmbH* Telenav
Map & Data Providers (8) * Steering Board Members ** New Members since June 2016
AND Navinfo Co
AutoNavi Holding TomTom*
GeoDigital Automotive** Here * Wuhan Kotei Informatics** Zenrin
Flament | Automated Vehicle Symposium | July 2017
14
Automated driving data chain and ecosystem Reference architecture proposed by OADF - - - - -- - - INFRASTRUCTURE - - - - - - - - - -
- - - - - - - - - - - - - - - - - - VEHICLE - - - - - - - - -
N A S T
NDS
Backend (one or multiple) operated by Map supplier / OEM / 3rd Party
ADAS Horizon Provider
OEM-specific Backends
Live Map Updates (Dynamic Data) Maps & Updates
N T
Live Map Delivery
N T
HD Map Delivery
N
Local Live Map (Dynamic Data)
A
HD Map
Environmental Model
A
Analytics/Sensor Fusion
Localization
Data Collection Strategy
Community Data Store
(V2x) provided) Sensors
N
(Dyn.) Data Production
Systematic Data Collection (Mobile Mapping)
(Local) Sensors
Infotainment Map Delivery
S
Data Store
N
Infotainment Map
A
S S
Flament | Automated Vehicle Symposium | July 2017
Data Collection (raw or pre-processed)
15
INTRODUCTION TO SENSORIS
Flament | Automated Vehicle Symposium | July 2017
16
SENSORIS: Sensor Ingestion Interface Specification Standardization of sensor data content Vehicle-to-Cloud / Cloud-to-Cloud o Sensor manufacturers o Car manufacturers o Location-based services provider o Mobile network operators Part of Open AutoDrive Forum o Focus on Sensor Data upstream
o Alignment with NDS, ADASIS, TISA
Flament | Automated Vehicle Symposium | July 2017
17
SENSORIS architecture: Multi-role model
Flament | Automated Vehicle Symposium | July 2017
18
Use Case: Real-time services
Traffic flow Traffic incidents
Hazard warnings
Environmental conditions
Flament | Automated Vehicle Symposium | July 2017
Traffic signage
19
Use Case: Map maintenance • Road geometry and attributes • Lane geometry and attributes
• POI entries and exits • Road condition
Flament | Automated Vehicle Symposium | July 2017
20
Use Case: Statistical analysis • Historical and real-time data analysis • Personal preference learning
• POI recommendations
Flament | Automated Vehicle Symposium | July 2017
21
SENSORIS Members ADAS manufacturers AISIN AW Continental Automotive GmbH DENSO Fujitsu Ten (Europe) GmbH LG Electronics Valeo Comfort and Driving Assistance Location content & Service providers AutoNavi Software Co. Ltd. HERE Global B.V. INRIX Inc. NavInfo Co.Ltd. TomTom International B.V. Zenrin
Navigation System Suppliers Elektrobit Automotive GmbH Harman Hyundai Mnsoft NNG PIONEER Co. Robert Bosch Car Multimedia GmbH Sensor & Component Manufacturers Qualcomm Telecom & Cloud Infrasrtucture Providers IBM
Flament | Automated Vehicle Symposium | July 2017
Vehicle manufacturers Audi Daimler AG Jaguar Land Rover Limited Volvo Car Other ICCS In process to join BMW AG CTAG Delphi Ericsson Nissan Tass International TeleNav Toyota Trafikverket VTT Wuhan kotei informatics 22
Conclusions • ADASIS and SENSORIS are both industry-wide specification initiatives facilitating the use of map data in automated vehicles • ADASIS V3 transforms the map information into relevant data for the Automated Driving functions • SENSORIS gathers vehicle sensor data to make it accessible in the cloud for various real-time, historical and statistical purposes • ADASIS and SENSORIS are part of the Open Auto Drive Forum Flament | Automated Vehicle Symposium | July 2017
23
Flament | Automated Vehicle Symposium | July 2017
24
HERE Digital Infrastructure Technologies Autonomous Vehicle Symposium July 11, 2017
A history of transforming maps into location technology 2015 2009 1st map for Predictive Cruise Control
2011
3 new Investors
1st pure location cloud
High-precision data collection and map-building technology Use of sensor data for map building
2004 1st map for ADAS
1985 Navigation Technologies founded
HERE Technologies
1994 1st
map for in-car nav
1st map for phone 1st map for Adaptive Cruise Control
2007 Community mapping Offline maps for mobile
1st map for web
© 2017 HERE
HERE Reality IndexTM Location & POIs Beyond roads Things People Spatial/aerial
HERE Technologies
© 2017 HERE
The intelligent car
Assistance
HERE Technologies
Automation
Learning
© 2017 HERE
HERE’s View of Digital Infrastructure to Power Automated Vehicles
5
•
Built on the HERE Open Location Platform
•
Active pilots (Colorado DOT, Iowa DOT) to develop standards for infrastructure data © 2016 HERE | HERE Internal Use Only
High Definition Live Map
HERE Technologies
© 2017 HERE
Sensor Data Self-Healing Map OEM backend
The edge
The cloud
OEM backend
OEM backend ERTICO/OADF*: SENSORIS data standard
Vehicles, IoT devices HERE Technologies
*Open Auto Drive Forum: openautodrive.org
© 2017 HERE
HERE Cooperative V2X Road Hazard Example Incident detection •
Smartphones, aftermarket devices or embedded solutions become a collective community – detecting changing road or environmental conditions
•
Incidents or event information may also be manually input from the community and authorities
8
HERE Location Cloud
© 2016 HERE | HERE Internal Use Only
HERE Cooperative V2X Road Hazard Example Reporting & Location Cloud Analytics •
Receiving and ingesting incident data
•
Running analytics
•
Determining identification of an impact area
9
HERE Location Cloud
© 2016 HERE | HERE Internal Use Only
HERE Cooperative V2X Road Hazard Example Distribution of Road Hazard Warnings •
10
The impact area and incident message type are send to traffic management centers and back to the drivers across all connected infrastructure in the affected area
HERE Location Cloud
City Traffic Management Center
© 2016 HERE | HERE Internal Use Only
Intel® Architecture for Autonomous Driving July 2017
FIVE Things You Should Know about Autonomous VEHICLES
Workloads are more than artificial intelligence (AI).
Some critical algorithms haven’t even been developed yet.
System designs will need to change down the road.
Intel powers hundreds of level 4 and level 5 test vehicles.
Autonomous Vehicles are an End-to-End Solution
2
Building the Best Autonomous Brain
3
Work with Fewer Limits
Design with Flexible, Scalable Intel Architecture
Greater agility
to adapt to evolving system designs
More flexibility to place compute
Optimized performance per watt
Future programmability with FPGAs
Lower TCO
with FPGAs over ASICs
Faster
software development
Reduced
training time in the data center
5
Intel, the Intel logo, Intel Atom, Intel Nervana, and Xeon are trademarks of Intel Corporation or its subsidiaries in the U.S. and/or other countries. *Other names and brands may be claimed as the property of others. © Intel Corporation
ARTIFICIAL INTELLIGENCE & SELF-DRIVING CARS TIM WONG | JULY 2017
DRIVING IS EXTREMELY COMPLICATED
DETECTION DNN: LANES
NVIDIA SELF-DRIVING AI Inputs
Training/Deployment
NVIDIA DGX-1
Testing
NVIDIA DRIVE PX 2
AI THAT MIMICS HUMAN BEHAVIOR
SELF-DRIVING TO STARBUCKS 6 Left Side Camera
1
Left Front Corner Camera 2 Center Front Camera
5 Rear Center Camera
4 Right Side Camera
3 Right Front Corner Camera
MULTIPLE AI’S WORKING TOGETHER NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE.
NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE.
ENABLING THE NEXT-GENERATION OF DRIVER ASSISTANCE
NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE.
Thank you! [email protected]
Recent progress of dynamic map in Japan
12nd July, 2017 Automated Vehicles Symposium 2017 Hilton San Francisco Union Square, USA
Satoru Nakajo, the University of Tokyo
Contents of the presentation
0. Overview of SIP-adus
1. FOT 2017-2018 2. Standardization activities 3. Challenges we are facing (personal view) Ref. Workshop in Tokyo
1
0. Overview of SIP-adus SIP: Cross-ministerial Strategic Innovation Promotion Program Start from FY2013. The Council for Science, Technology and Innovation selects 11 projects. Cross-ministerial Initiatives. Promote focused, end-to-end research and development, from basic research to practical application and commercialization. Total budget for FY2016 was ¥50 billion (around 500 million dollars).
Cyber-Security for Critical Infrastructures were added from FY2015 adus: Automated Driving for Universal Services
¥2.7 billion (FY2016) (around 27 million dollars)
2
0. Overview of SIP-adus (I) Development and verification of automated driving system
(III) International cooperation
Road Transport system Driver Traffic environment
Recognition
Judgment
(5) System Security Recognition (1) Dynamic Map
Operation (4) Driver Model
Judgment
Operation
Area of Cooperation
(2) Prediction based on ITS information (3) Sensors
(1) Open research facility (2) Social acceptance (3) Technology transfer
Area of Competition Vehicle
(1) Traffic fatality reduction effect estimation method & national shared database (2) Macro and micro data analysis and simulation technology (3) Local traffic CO2 emission visualization technology
(II) Basic technologies to reduce traffic fatalities and congestion
(1) Enhanced local traffic management (2) Next generation transport system
(IV) Development for next generation urban transport 3
0. Overview of SIP-adus Research results by SIP-adus http://en.sip-adus.jp/
(results of FY2014-FY2016 are available)
Specification on map features and attributes for ADS by JAMA http://www.jama.or.jp/safe/automated_driving/pdf/recommended_spec.pdf JAMA: Japan Automobile Manufacturers Association Inc. (Japanese) A company “Dynamic Map Platform Inc. (DMP)” had established in June 2017
4
1. FOT 2017-2018 SIP-adus is planning FOTs for 5 technological fields.
Dynamic map HMI Information Security Pedestrian Accidents Reduction Advanced Urban Transit
Call for Participants (first call) was completed. Call for participants (First call) by each technological fields including Dynamic Map Sample Distribution of the map before the test would be available Entry (in English) was possible from following HP http://www.nedo.go.jp/english/sip_ai2017.html (you can also access from http://en.sip-adus.jp/) 5
1. FOT 2017-2018 Overview of Dynamic Map FOT
6
1. FOT 2017-2018 Overview of Dynamic Map FOT
Outline of FOT
Schedule
7
2. Standardization activities Promote standardization activities at ISO/TC204 Discussion for the importance of industrial specifications
Lane-level LR
GDF5.1
Scopes are ONLY a PERSONAL VIEW
Lane-level LR
8
3. Challenges we are facing (personal view) Connection between static and dynamic (inc. semi.) data Map updates (before actual changes) Map definition for various roads including local streets Relationship between Dynamic Map and Navigation data Balance between Map and other key elements such as sensing devices, AI algorithm
Try to go forward by FOTs (real experiments) and discussions 9
Ref. Workshop in Tokyo Workshop 2017 will be held on 14th to 16th Nov. in Tokyo
Just after the 8th OADF meting (13th Nov.) in Tokyo (almost of) all presentations will be done by or translated in English Dynamic Map will be one of the main topic Last day of the workshop will be only available for invited person (around 20 people for each topic, mainly for discussion) Further details will be released at the following website http://en.sip-adus.jp/
10
END [email protected]
AUTOMATED DRIVING DATA CHAIN CHALLENGES Flament | Automated Vehicle Symposium | July 2017
1
Tomorrow’s Situation: Sensors, Maps and Online Data The Vehicle Looks beyond 300m and Around the Corner
3. Extension of limited in-vehicle resources
1. Highly accurate map model provided and updated via the Backend
2. Extended preview information
Close preview: 10 minutes
4. Fleet based data collection
Vehicle Sensor range 0-300m
GSV-Forum Automatisiertes Fahren Public
October 2016 FF@ Continental AG
2
Automated driving data chain and ecosystem Reference architecture proposed by OADF - - - - -- - - INFRASTRUCTURE - - - - - - - - - -
- - - - - - - - - - - - - - - - - - VEHICLE - - - - - - - - -
N A S T
NDS
Backend (one or multiple) operated by Map supplier / OEM / 3rd Party
ADAS Horizon Provider
OEM-specific Backends
Live Map Updates (Dynamic Data) Maps & Updates
N T
Live Map Delivery
N T
HD Map Delivery
N
Local Live Map (Dynamic Data)
A
HD Map
Environmental Model
A
Analytics/Sensor Fusion
Localization
Data Collection Strategy
Community Data Store
(V2x) provided) Sensors
N
(Dyn.) Data Production
Systematic Data Collection (Mobile Mapping)
(Local) Sensors
Infotainment Map Delivery
S
Data Store
N
Infotainment Map
A
S S
Flament | Automated Vehicle Symposium | July 2017
Data Collection (raw or pre-processed)
3
INTRODUCTION TO ADASIS
Flament | Automated Vehicle Symposium | July 2017
4
ADASIS horizon along the map data chain TPEG: Traffic Information NDS: Incremental Map Update
Map database
In-vehicle systems HMI
Layers e. g. hazard spots
e. g. traffic information
Determination of location and most probable path (MPP)
e. g. speed limits
30
Digital map (static)
Map data (NDS Format)
Enrichment of MPP with road information (e.g. topography, speed limits)
ADAS, Energy Management
Relevant information for road ahead (ADASIS Format)
Conversion into ADASIS format
Flament | Automated Vehicle Symposium | July 2017
5
ADASIS horizon – evolution to the cloud
Cloud
Hybrid
Local
In-vehicle embedded software
Cloud-based backend software
function
horizon provider
map
Link to NDS
function
horizon provider
map
map
function
horizon provider
map
map
Link to Sensoris, NDS & iii. TPEG
function
i.
Link to NDS
ii. dynamic data
horizon provider
map
Link to NDS & TPEG dynamic data
i.
ADASIS Horizon uses data from local map database and provides information for functions in the vehicle
ii.
ADASIS Horizon uses recent data from updated map database via cloud backend or live data from additional databases and provides information for functions in the vehicle
iii. ADASIS Horizon uses cloud-based data and provides information for in-vehicle functions directly from the cloud
Flament | Automated Vehicle Symposium | July 2017
6
ADASIS V3 specification supports highly automated driving New ADASIS V3 specifications supports different aspects of autonomous driving 1.
Support of HAD maps (NDS)
•
Flament | Automated Vehicle Symposium | July 2017
Copyright: Here.com
7
ADASIS V3 specification supports highly automated driving Lane model & Geometry Example: Highway Exit “switch from a map-centric view to a vehicle-centric view of the world to create a simpler representation”
© Wikimedia Commons, licensed by CC-by-SA 2.5
Flament | Automated Vehicle Symposium | July 2017
8
ADASIS V3 specification supports highly automated driving Lane model: Segmentation of road into stretches with Segmentation of road same lane characteristics into stretches with 1stsame level =lane pathcharacteristics description
3 Lanes
level detail: 1st levelof= lanes path description number for each segment level detail: number of lanes for each segment
1 Lane
3+1 Lanes
3 Lanes + shoulder © Wikimedia Commons, licensed by CC-by-SA 2.5
Flament | Automated Vehicle Symposium | July 2017
9
ADASIS V3 specification supports highly automated driving Lane Model & Connectivity Lane Model & Connectivity: Logical Description similar to v2 Lane #3: Lane #2: normal normal Lane #1: normal
2nd level = logical view level detail: lane details and their connectivity
Lane #4: normal
Lane #3: normal
Lane #3: normal
Lane #2: normal
Lane #1: ramp
Lane #2: normal
Lane #1: normal © Wikimedia Commons, licensed by CC-by-SA 2.5
Flament | Automated Vehicle Symposium | July 2017
10
ADASIS V3 specification supports highly automated driving Lane Model & Connectivity Geometry for Linear Objects: • Lane Boundaries Geometry for Linear Objects: • Guardrails • Lane Boundaries level = geometry description •3rdGuardrails level detail: rd level = geometry description 3geometry of road, lane and lines level detail: geometry of road, lane and lines
Lane #3: normal
lines and boundaries
Lane #2: normal
Lane #1: normal © Wikimedia Commons, licensed by CC-by-SA 2.5
Flament | Automated Vehicle Symposium | July 2017
11
ADASIS Forum Membership & Steering Board
From 46 to 52 ADASIS Forum Members (updated 31 may 2017) Vehicle Manufacturers (16)
ADAS Suppliers
(15)
BMW China FAW-RDC ** CRF (FCA) ** Daimler * Ford * Ford-Otosan Honda * Hyundai Motor Company Jaguar Opel * Nissan Renault Toyota Motor Corp. Volkswagen Volvo Car Corp. Volvo Tech. Dev. Corp.
Autonomos Continental Automotive * CTAG Denso dSPACE Fujitsu Ten (Europe) ** Hitachi Ibeo IPG LG Electronic Magna Electronic Europe Magneti Marelli Novero TRW (ZF) Valeo
Navigation System Suppliers (13) AISIN AW Alpine Autoliv Elektrobit Automotive Garmin Harman Mappers Co. ** Mitsubishi Electric Europe MXNAVI NNG LLC Panasonic Robert Bosch GmbH* Telenav
Map & Data Providers (8) * Steering Board Members ** New Members since June 2016
AND Navinfo Co
AutoNavi Holding TomTom*
GeoDigital Automotive** Here * Wuhan Kotei Informatics** Zenrin
Flament | Automated Vehicle Symposium | July 2017
14
Automated driving data chain and ecosystem Reference architecture proposed by OADF - - - - -- - - INFRASTRUCTURE - - - - - - - - - -
- - - - - - - - - - - - - - - - - - VEHICLE - - - - - - - - -
N A S T
NDS
Backend (one or multiple) operated by Map supplier / OEM / 3rd Party
ADAS Horizon Provider
OEM-specific Backends
Live Map Updates (Dynamic Data) Maps & Updates
N T
Live Map Delivery
N T
HD Map Delivery
N
Local Live Map (Dynamic Data)
A
HD Map
Environmental Model
A
Analytics/Sensor Fusion
Localization
Data Collection Strategy
Community Data Store
(V2x) provided) Sensors
N
(Dyn.) Data Production
Systematic Data Collection (Mobile Mapping)
(Local) Sensors
Infotainment Map Delivery
S
Data Store
N
Infotainment Map
A
S S
Flament | Automated Vehicle Symposium | July 2017
Data Collection (raw or pre-processed)
15
INTRODUCTION TO SENSORIS
Flament | Automated Vehicle Symposium | July 2017
16
SENSORIS: Sensor Ingestion Interface Specification Standardization of sensor data content Vehicle-to-Cloud / Cloud-to-Cloud o Sensor manufacturers o Car manufacturers o Location-based services provider o Mobile network operators Part of Open AutoDrive Forum o Focus on Sensor Data upstream
o Alignment with NDS, ADASIS, TISA
Flament | Automated Vehicle Symposium | July 2017
17
SENSORIS architecture: Multi-role model
Flament | Automated Vehicle Symposium | July 2017
18
Use Case: Real-time services
Traffic flow Traffic incidents
Hazard warnings
Environmental conditions
Flament | Automated Vehicle Symposium | July 2017
Traffic signage
19
Use Case: Map maintenance • Road geometry and attributes • Lane geometry and attributes
• POI entries and exits • Road condition
Flament | Automated Vehicle Symposium | July 2017
20
Use Case: Statistical analysis • Historical and real-time data analysis • Personal preference learning
• POI recommendations
Flament | Automated Vehicle Symposium | July 2017
21
SENSORIS Members ADAS manufacturers AISIN AW Continental Automotive GmbH DENSO Fujitsu Ten (Europe) GmbH LG Electronics Valeo Comfort and Driving Assistance Location content & Service providers AutoNavi Software Co. Ltd. HERE Global B.V. INRIX Inc. NavInfo Co.Ltd. TomTom International B.V. Zenrin
Navigation System Suppliers Elektrobit Automotive GmbH Harman Hyundai Mnsoft NNG PIONEER Co. Robert Bosch Car Multimedia GmbH Sensor & Component Manufacturers Qualcomm Telecom & Cloud Infrasrtucture Providers IBM
Flament | Automated Vehicle Symposium | July 2017
Vehicle manufacturers Audi Daimler AG Jaguar Land Rover Limited Volvo Car Other ICCS In process to join BMW AG CTAG Delphi Ericsson Nissan Tass International TeleNav Toyota Trafikverket VTT Wuhan kotei informatics 22
Conclusions • ADASIS and SENSORIS are both industry-wide specification initiatives facilitating the use of map data in automated vehicles • ADASIS V3 transforms the map information into relevant data for the Automated Driving functions • SENSORIS gathers vehicle sensor data to make it accessible in the cloud for various real-time, historical and statistical purposes • ADASIS and SENSORIS are part of the Open Auto Drive Forum Flament | Automated Vehicle Symposium | July 2017
23
Flament | Automated Vehicle Symposium | July 2017
24
HERE Digital Infrastructure Technologies Autonomous Vehicle Symposium July 11, 2017
A history of transforming maps into location technology 2015 2009 1st map for Predictive Cruise Control
2011
3 new Investors
1st pure location cloud
High-precision data collection and map-building technology Use of sensor data for map building
2004 1st map for ADAS
1985 Navigation Technologies founded
HERE Technologies
1994 1st
map for in-car nav
1st map for phone 1st map for Adaptive Cruise Control
2007 Community mapping Offline maps for mobile
1st map for web
© 2017 HERE
HERE Reality IndexTM Location & POIs Beyond roads Things People Spatial/aerial
HERE Technologies
© 2017 HERE
The intelligent car
Assistance
HERE Technologies
Automation
Learning
© 2017 HERE
HERE’s View of Digital Infrastructure to Power Automated Vehicles
5
•
Built on the HERE Open Location Platform
•
Active pilots (Colorado DOT, Iowa DOT) to develop standards for infrastructure data © 2016 HERE | HERE Internal Use Only
High Definition Live Map
HERE Technologies
© 2017 HERE
Sensor Data Self-Healing Map OEM backend
The edge
The cloud
OEM backend
OEM backend ERTICO/OADF*: SENSORIS data standard
Vehicles, IoT devices HERE Technologies
*Open Auto Drive Forum: openautodrive.org
© 2017 HERE
HERE Cooperative V2X Road Hazard Example Incident detection •
Smartphones, aftermarket devices or embedded solutions become a collective community – detecting changing road or environmental conditions
•
Incidents or event information may also be manually input from the community and authorities
8
HERE Location Cloud
© 2016 HERE | HERE Internal Use Only
HERE Cooperative V2X Road Hazard Example Reporting & Location Cloud Analytics •
Receiving and ingesting incident data
•
Running analytics
•
Determining identification of an impact area
9
HERE Location Cloud
© 2016 HERE | HERE Internal Use Only
HERE Cooperative V2X Road Hazard Example Distribution of Road Hazard Warnings •
10
The impact area and incident message type are send to traffic management centers and back to the drivers across all connected infrastructure in the affected area
HERE Location Cloud
City Traffic Management Center
© 2016 HERE | HERE Internal Use Only
Intel® Architecture for Autonomous Driving July 2017
FIVE Things You Should Know about Autonomous VEHICLES
Workloads are more than artificial intelligence (AI).
Some critical algorithms haven’t even been developed yet.
System designs will need to change down the road.
Intel powers hundreds of level 4 and level 5 test vehicles.
Autonomous Vehicles are an End-to-End Solution
2
Building the Best Autonomous Brain
3
Work with Fewer Limits
Design with Flexible, Scalable Intel Architecture
Greater agility
to adapt to evolving system designs
More flexibility to place compute
Optimized performance per watt
Future programmability with FPGAs
Lower TCO
with FPGAs over ASICs
Faster
software development
Reduced
training time in the data center
5
Intel, the Intel logo, Intel Atom, Intel Nervana, and Xeon are trademarks of Intel Corporation or its subsidiaries in the U.S. and/or other countries. *Other names and brands may be claimed as the property of others. © Intel Corporation
ARTIFICIAL INTELLIGENCE & SELF-DRIVING CARS TIM WONG | JULY 2017
DRIVING IS EXTREMELY COMPLICATED
DETECTION DNN: LANES
NVIDIA SELF-DRIVING AI Inputs
Training/Deployment
NVIDIA DGX-1
Testing
NVIDIA DRIVE PX 2
AI THAT MIMICS HUMAN BEHAVIOR
SELF-DRIVING TO STARBUCKS 6 Left Side Camera
1
Left Front Corner Camera 2 Center Front Camera
5 Rear Center Camera
4 Right Side Camera
3 Right Front Corner Camera
MULTIPLE AI’S WORKING TOGETHER NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE.
NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE.
ENABLING THE NEXT-GENERATION OF DRIVER ASSISTANCE
NVIDIA CONFIDENTIAL. DO NOT DISTRIBUTE.
Thank you! [email protected]
Recent progress of dynamic map in Japan
12nd July, 2017 Automated Vehicles Symposium 2017 Hilton San Francisco Union Square, USA
Satoru Nakajo, the University of Tokyo
Contents of the presentation
0. Overview of SIP-adus
1. FOT 2017-2018 2. Standardization activities 3. Challenges we are facing (personal view) Ref. Workshop in Tokyo
1
0. Overview of SIP-adus SIP: Cross-ministerial Strategic Innovation Promotion Program Start from FY2013. The Council for Science, Technology and Innovation selects 11 projects. Cross-ministerial Initiatives. Promote focused, end-to-end research and development, from basic research to practical application and commercialization. Total budget for FY2016 was ¥50 billion (around 500 million dollars).
Cyber-Security for Critical Infrastructures were added from FY2015 adus: Automated Driving for Universal Services
¥2.7 billion (FY2016) (around 27 million dollars)
2
0. Overview of SIP-adus (I) Development and verification of automated driving system
(III) International cooperation
Road Transport system Driver Traffic environment
Recognition
Judgment
(5) System Security Recognition (1) Dynamic Map
Operation (4) Driver Model
Judgment
Operation
Area of Cooperation
(2) Prediction based on ITS information (3) Sensors
(1) Open research facility (2) Social acceptance (3) Technology transfer
Area of Competition Vehicle
(1) Traffic fatality reduction effect estimation method & national shared database (2) Macro and micro data analysis and simulation technology (3) Local traffic CO2 emission visualization technology
(II) Basic technologies to reduce traffic fatalities and congestion
(1) Enhanced local traffic management (2) Next generation transport system
(IV) Development for next generation urban transport 3
0. Overview of SIP-adus Research results by SIP-adus http://en.sip-adus.jp/
(results of FY2014-FY2016 are available)
Specification on map features and attributes for ADS by JAMA http://www.jama.or.jp/safe/automated_driving/pdf/recommended_spec.pdf JAMA: Japan Automobile Manufacturers Association Inc. (Japanese) A company “Dynamic Map Platform Inc. (DMP)” had established in June 2017
4
1. FOT 2017-2018 SIP-adus is planning FOTs for 5 technological fields.
Dynamic map HMI Information Security Pedestrian Accidents Reduction Advanced Urban Transit
Call for Participants (first call) was completed. Call for participants (First call) by each technological fields including Dynamic Map Sample Distribution of the map before the test would be available Entry (in English) was possible from following HP http://www.nedo.go.jp/english/sip_ai2017.html (you can also access from http://en.sip-adus.jp/) 5
1. FOT 2017-2018 Overview of Dynamic Map FOT
6
1. FOT 2017-2018 Overview of Dynamic Map FOT
Outline of FOT
Schedule
7
2. Standardization activities Promote standardization activities at ISO/TC204 Discussion for the importance of industrial specifications
Lane-level LR
GDF5.1
Scopes are ONLY a PERSONAL VIEW
Lane-level LR
8
3. Challenges we are facing (personal view) Connection between static and dynamic (inc. semi.) data Map updates (before actual changes) Map definition for various roads including local streets Relationship between Dynamic Map and Navigation data Balance between Map and other key elements such as sensing devices, AI algorithm
Try to go forward by FOTs (real experiments) and discussions 9
Ref. Workshop in Tokyo Workshop 2017 will be held on 14th to 16th Nov. in Tokyo
Just after the 8th OADF meting (13th Nov.) in Tokyo (almost of) all presentations will be done by or translated in English Dynamic Map will be one of the main topic Last day of the workshop will be only available for invited person (around 20 people for each topic, mainly for discussion) Further details will be released at the following website http://en.sip-adus.jp/
10
END [email protected]
