Automation Enabling Technologies AVS 2015 â Breakout Session
Automation Enabling Technologies AVS 2015 – Breakout Session Mohammed Yousuf U.S. Department of Transportation
Agenda Overview of Recent Technological Advances Related to Vehicle Automation □
1:45 – 1:55 Welcome and Introduction (Mohammed Yousuf, FHWA)
□
1:55 – 2:05 Project Background and Goals (Ram Kandarpa, Booz Allen)
□
2:05 – 3:30 Overview of recent advances in technologies that will enable vehicle automation. Presentations and group discussion
□
PNT (Mike Brown, SwRi) Mapping (Mike Brown, SwRi) Communications (Sara Sarkhili, Booz Allen) Sensors (Sudharson Sundararajan, Booz Allen) Human Factors (Ismail Zohdy, Booz Allen)
3:30 – 3:45 Closing Remarks - Based on the technology overview presented in the previous session any additions and next steps for future research (Mohammed Yousuf and Ram Kandarpa)
U.S. Department of Transportation
2
PROJECT BACKGROUND AND GOALS
U.S. Department of Transportation
3
Enabling Technologies Project Background
Vehicle automation relies heavily on technologies such as wireless communications, positioning, mapping, localization, timing, tracking, and sensing Applications planned within the USDOT’s automation research roadmap depends on the understanding and applicability of these technologies
It is important to be aware of the state of these technologies, and more importantly “stay ahead of the curve” Value of this task is not in accurately predicting the future of these technologies for USDOT’s automation program, but to minimize surprises
U.S. Department of Transportation
4
Goals / Objectives
1
Provide guidance to USDOT with respect to technologies necessary to advance vehicle automation efforts
2
Provide a clear understanding of current and future directions of technologies necessary for automated vehicles
3
Provide a framework and content to understand the unique needs of vehicle automation applications and project the likely direction for technologies
U.S. Department of Transportation
5
OVERVIEW OF KEY TECHNOLOGY OBSERVATIONS THAT RELATE TO VEHICLE AUTOMATION
U.S. Department of Transportation
8
High-Level Process
Needs Literature Review Report
Trends
Monthly Technical Reports – 5 completed
Gaps
Next Steps Approximately 250+ popular and technical press sources used
U.S. Department of Transportation
9
Positioning, Navigation, and Timing Needs – Positioning accuracy and reliability – Capability of navigation over a long distance (e.g. cross-country) and through immediate challenges (e.g. collision avoidance, intersection negotiation) – Precise timing for sensor fusion and cooperative aspects – Performance & reliability that is affordable Trends – The market is continuing to drive positioning and navigation systems technology to provide increasing capability at a much lower cost point – These systems are beginning to provide frameworks to fuse other sensor data to enable even more confidence in positioning accuracy – A significant amount of research is focused on using localization approaches(e.g. LIDAR, RADAR, Visual, RF, Cooperative, and fused approaches) for positioning and navigation when GNSS is unavailable or unreliable – Techniques that build redundancy are emerging to provide high reliability of precise timing in case of GNSS unavailability U.S. Department of Transportation
10
Positioning, Navigation, and Timing Gaps – Positioning and navigation systems are advancing but still face many challenges (e.g. urban canyons, tunnels, dense foliage) – Many safety critical cooperative applications are dependent upon reliable GNSS timing and there is typically only one source – GNSS receiver – High performance technology in vehicles is still relatively expensive Next Steps
– Advance the current research of providing corrections from the infrastructure to consider automated vehicle positioning and navigation – Investigate how automated vehicle path planning and navigation would interact with Connected Vehicle applications such as Forward Collison Warning (FCW) – Investigate timing techniques that provide redundancy to ensure higher reliability and security for safety critical, key infrastructure components
U.S. Department of Transportation
11
Discussion 1. What special positioning capabilities do light and heavy vehicles need? 2. What are the challenges associated with getting accurate positioning? 3. Challenges and gaps related to time synchronization? 4. How do you envision PNT technologies will change in the near future? 5. What are lessons learned from other relevant prior transportation technology roll-outs? 6. Recommendations for the automation program?
U.S. Department of Transportation
12
Mapping Needs – Macro / micro level maps to help vehicle navigation – Mapping the vehicle’s surrounding environment accurately – Mapping algorithms need to operate in real-time – Performance and reliability at a cost that is affordable Trends – Multiple companies are continuing to collect and develop extremely high resolution 3D maps – SLAM techniques are constantly advancing to handle challenging conditions (lighting / weather) and dynamic environments (seasonal changes / cities) – Recent research focused on cooperative mapping capabilities
– Graphical Processing Units are increasingly being used to enable real-time localization algorithms
U.S. Department of Transportation
13
Mapping Gaps – Information shared from between the infrastructure and vehicles is relatively limited (e.g. SPAT/MAP/TIM) – Visual SLAM techniques are improving but still face challenges handling challenging conditions and environments and LIDAR based SLAM remains expensive Next Steps
– Research methods for exchanging situational awareness between fixed infrastructure points and vehicles – Investigate techniques to provide vehicles with more accurate, up-to-date, dynamic map data as part of the USDOT Digital Infrastructure project – Research potential infrastructure, signage, and pavement markings that can enhance vehicle perception & localization – Investigate standardization that could be applied to mapping
U.S. Department of Transportation
14
Discussion 1. What special mapping capabilities do trucks or transit vehicles need? 2. What are important challenges/gaps in developing accurate maps? 3. How will mapping evolve in the near future? 4. What are lessons learned from other relevant prior transportation technology roll-outs? 5. Recommendations for the automation program?
U.S. Department of Transportation
15
Communications Needs –
Wireless technologies that are capable of fast network acquisition, and are interoperable
–
Reliable technologies capable of communicating at high data rates, low latency, and efficiently using the spectrum
–
Infrastructure that enables secure and safe network and communication
–
Building necessary infrastructure to enable V2V and enrollment of a critical mass to achieve the potential benefits of V2V
–
Additional standards are needed to ensure full interoperability of different technologies
Gaps –
An integrated wireless communication solution for in-vehicle, inter-vehicle, and long range communications (high bandwidth and low latency)
–
Complementary technologies that can address channel congestion and limited spectrum availability of DSRC
–
Communication security for both CV and AV U.S. Department of Transportation
16
Communications Trends –
Increasing demand for 4G LTE and Wi-Fi hotspots for vehicular communications
–
Advancement of 5G networks enabling gigabits data transmission
–
Expansion of M2M and IoT for in-vehicle technologies; enabling many in-vehicle applications, telematics, and infotainment solutions. There will be a wider integration between connected cars and the Internet of Things. The number of M2M connections is likely to grow three to four times by 2019.
–
Growth of peer-to-peer technologies (e.g. ZigBee, Bluetooth) inside the vehicles for electronic engine control, driving assistants, safety mechanisms, and infotainment.
–
Use of RF communication alternatives such as VLC and Li-Fi for vehicular communication
–
Developments in preventing hackers from intruding and accessing in-vehicle networks
–
Increase of telematics option inside the vehicles
U.S. Department of Transportation
17
Next Steps –
Invest in addressing the issues with DSRC (limited spectrum, expensive infrastructure, adoption issues*)
–
Research techniques for an integrated V2X/Cellular solution
–
Investigate/monitor progress of 5G cellular, M2M, and D2D technologies development and their use for vehicular communication and automation
–
Further research solutions for communication security, and privacy issues
–
Participate in development of necessary standards, laws, and regulations for enabling CV/AV
*DSRC, and any cooperative network require adaptation by a critical mass to be effective U.S. Department of Transportation
18
Discussion 1. What does the future of DSRC look like and what complimentary technologies are needed? 2. How should we address the infrastructure, limited spectrum, and adoption issues w/r to employing DSRC? 3. What does the group think about communications using the visible light spectrum? 4. What are the major challenges / gaps in terms of security and data privacy and how can they be best addressed? 5. What are lessons learned from other relevant prior transportation technology roll-outs? 6. Recommendations for the automation program? U.S. Department of Transportation
19
Sensing Needs – Accurate and reliable 360 degree sensing of vehicle surrounding – Sensor fusion and data accessibility (user interface) – Work in all weather conditions without degraded accuracy – Quality and size of sensors at costs affordable for economy vehicles
Trends – An increasing number of applications are using vision based systems for detecting obstructions and recognizing objects (e.g. 360-degree truck vision technology)
– With evolving sensor technologies, advances in micro-processors have gained popularity (e.g. Nvidia’s Tegra K1 processor for Advanced Driver Assistance Systems) – The market is also looking to use ‘cloud’ systems to analyze real-time data from a variety of sensors and other dynamic events (e.g. Dynamic eHoizon System – Predictive Navigation) U.S. Department of Transportation
20
Sensing Gaps – Accuracy of sensors are increasing but are not completely reliable just yet – Sensor performance in adverse weather conditions – More robust standards to enable data interface & accessibility of different sensors Next Steps – Explore sensors that could be used from the infrastructure perspective. For example use stereo cameras at complex intersections to get a holistic view for adaptive signal timing strategies – Explore infrastructure sensors that could be used to provide more quality data for automated vehicle applications > weather data, intersection blind spot, etc. – Explore changes/updates to traffic signs for improved recognition by sensors for automated vehicle applications
U.S. Department of Transportation
21
Discussion 1. What type of vehicle sensors are the most important? 2. What are the major gaps in today’s sensors? 3. What are the major challenges in sensor development / accuracy? 4. How will vehicle sensors evolve in the near future? 5. What are lessons learned from other relevant prior transportation technology roll-outs? 6. Recommendations for the automation program?
U.S. Department of Transportation
22
Human Factors Needs – Cooperative and non-distracting devices for supporting human machine interaction at different levels of automation – Safe and Smooth transition from human control to automated control – Accommodate the heterogeneity of the drivers reaction/perception in the design – Study how users and non-users will interact in a mixed environment Overview / Trends – The usage of driver state sensing to mitigate driver distraction is underway by several OEMs. For example, Ghost Car technology by Jaguar, zForce Drive Technology and dashboard mounted camera for facial expression monitoring – Understanding Driver behavior in different levels of automation is investigated by many research labs; especially using driver simulators. For example, Transport Research Laboratory (TRL) simulator “DigiCar” – Many driver information/assistance display technologies are claimed to be ready (or almost ready) for market usage. For example, Panasonic’s eCockpit U.S. Department of Transportation
23
Human Factors Gaps – The impact of many of the current technologies (including after-market devices) on safety is still questionable; especially for the different levels of automation – Very few available studies addressed the human factor aspect of joining/disjoining automated vehicle platoons – The research related to automated vehicles interaction with other vulnerable road users (also pedestrians, cyclists, and legacy vehicles) is still lacking
– The interaction of automated vehicles and legacy vehicles (users Vs. non-users) is still not well addressed in the literature – The relationship between automated vehicles and disable/elderly people is still not addressed Next Steps – Investigate the impact of after-market devices on the human level of comfort, distraction and safety
– Study the needs and implications of different levels of automation on the elderly drivers and/or drivers with disabilities – Study the human factor aspect of joining/disjoining automated vehicle platoons under different operation settings. U.S. Department of Transportation
24
Discussion 1. What are the most important features to understand in Human Factors? 2. How do you deal with non-users? 3. How will the HMI market evolve in the near future? 4. What are lessons learned from other relevant prior transportation technology roll-outs? 5. Recommendations for the automation program? (specially for the enabling technologies track on the roadmap)
U.S. Department of Transportation
25
Closing Remarks 1. Other areas of importance? 2. Research strategy moving forward? 3. Top 5 research areas Short Term (1-2 yrs) Medium Term (3-5 yrs) Long Term (6-8 yrs)
U.S. Department of Transportation
26
Agenda Overview of Recent Technological Advances Related to Vehicle Automation □
1:45 – 1:55 Welcome and Introduction (Mohammed Yousuf, FHWA)
□
1:55 – 2:05 Project Background and Goals (Ram Kandarpa, Booz Allen)
□
2:05 – 3:30 Overview of recent advances in technologies that will enable vehicle automation. Presentations and group discussion
□
PNT (Mike Brown, SwRi) Mapping (Mike Brown, SwRi) Communications (Sara Sarkhili, Booz Allen) Sensors (Sudharson Sundararajan, Booz Allen) Human Factors (Ismail Zohdy, Booz Allen)
3:30 – 3:45 Closing Remarks - Based on the technology overview presented in the previous session any additions and next steps for future research (Mohammed Yousuf and Ram Kandarpa)
U.S. Department of Transportation
2
PROJECT BACKGROUND AND GOALS
U.S. Department of Transportation
3
Enabling Technologies Project Background
Vehicle automation relies heavily on technologies such as wireless communications, positioning, mapping, localization, timing, tracking, and sensing Applications planned within the USDOT’s automation research roadmap depends on the understanding and applicability of these technologies
It is important to be aware of the state of these technologies, and more importantly “stay ahead of the curve” Value of this task is not in accurately predicting the future of these technologies for USDOT’s automation program, but to minimize surprises
U.S. Department of Transportation
4
Goals / Objectives
1
Provide guidance to USDOT with respect to technologies necessary to advance vehicle automation efforts
2
Provide a clear understanding of current and future directions of technologies necessary for automated vehicles
3
Provide a framework and content to understand the unique needs of vehicle automation applications and project the likely direction for technologies
U.S. Department of Transportation
5
OVERVIEW OF KEY TECHNOLOGY OBSERVATIONS THAT RELATE TO VEHICLE AUTOMATION
U.S. Department of Transportation
8
High-Level Process
Needs Literature Review Report
Trends
Monthly Technical Reports – 5 completed
Gaps
Next Steps Approximately 250+ popular and technical press sources used
U.S. Department of Transportation
9
Positioning, Navigation, and Timing Needs – Positioning accuracy and reliability – Capability of navigation over a long distance (e.g. cross-country) and through immediate challenges (e.g. collision avoidance, intersection negotiation) – Precise timing for sensor fusion and cooperative aspects – Performance & reliability that is affordable Trends – The market is continuing to drive positioning and navigation systems technology to provide increasing capability at a much lower cost point – These systems are beginning to provide frameworks to fuse other sensor data to enable even more confidence in positioning accuracy – A significant amount of research is focused on using localization approaches(e.g. LIDAR, RADAR, Visual, RF, Cooperative, and fused approaches) for positioning and navigation when GNSS is unavailable or unreliable – Techniques that build redundancy are emerging to provide high reliability of precise timing in case of GNSS unavailability U.S. Department of Transportation
10
Positioning, Navigation, and Timing Gaps – Positioning and navigation systems are advancing but still face many challenges (e.g. urban canyons, tunnels, dense foliage) – Many safety critical cooperative applications are dependent upon reliable GNSS timing and there is typically only one source – GNSS receiver – High performance technology in vehicles is still relatively expensive Next Steps
– Advance the current research of providing corrections from the infrastructure to consider automated vehicle positioning and navigation – Investigate how automated vehicle path planning and navigation would interact with Connected Vehicle applications such as Forward Collison Warning (FCW) – Investigate timing techniques that provide redundancy to ensure higher reliability and security for safety critical, key infrastructure components
U.S. Department of Transportation
11
Discussion 1. What special positioning capabilities do light and heavy vehicles need? 2. What are the challenges associated with getting accurate positioning? 3. Challenges and gaps related to time synchronization? 4. How do you envision PNT technologies will change in the near future? 5. What are lessons learned from other relevant prior transportation technology roll-outs? 6. Recommendations for the automation program?
U.S. Department of Transportation
12
Mapping Needs – Macro / micro level maps to help vehicle navigation – Mapping the vehicle’s surrounding environment accurately – Mapping algorithms need to operate in real-time – Performance and reliability at a cost that is affordable Trends – Multiple companies are continuing to collect and develop extremely high resolution 3D maps – SLAM techniques are constantly advancing to handle challenging conditions (lighting / weather) and dynamic environments (seasonal changes / cities) – Recent research focused on cooperative mapping capabilities
– Graphical Processing Units are increasingly being used to enable real-time localization algorithms
U.S. Department of Transportation
13
Mapping Gaps – Information shared from between the infrastructure and vehicles is relatively limited (e.g. SPAT/MAP/TIM) – Visual SLAM techniques are improving but still face challenges handling challenging conditions and environments and LIDAR based SLAM remains expensive Next Steps
– Research methods for exchanging situational awareness between fixed infrastructure points and vehicles – Investigate techniques to provide vehicles with more accurate, up-to-date, dynamic map data as part of the USDOT Digital Infrastructure project – Research potential infrastructure, signage, and pavement markings that can enhance vehicle perception & localization – Investigate standardization that could be applied to mapping
U.S. Department of Transportation
14
Discussion 1. What special mapping capabilities do trucks or transit vehicles need? 2. What are important challenges/gaps in developing accurate maps? 3. How will mapping evolve in the near future? 4. What are lessons learned from other relevant prior transportation technology roll-outs? 5. Recommendations for the automation program?
U.S. Department of Transportation
15
Communications Needs –
Wireless technologies that are capable of fast network acquisition, and are interoperable
–
Reliable technologies capable of communicating at high data rates, low latency, and efficiently using the spectrum
–
Infrastructure that enables secure and safe network and communication
–
Building necessary infrastructure to enable V2V and enrollment of a critical mass to achieve the potential benefits of V2V
–
Additional standards are needed to ensure full interoperability of different technologies
Gaps –
An integrated wireless communication solution for in-vehicle, inter-vehicle, and long range communications (high bandwidth and low latency)
–
Complementary technologies that can address channel congestion and limited spectrum availability of DSRC
–
Communication security for both CV and AV U.S. Department of Transportation
16
Communications Trends –
Increasing demand for 4G LTE and Wi-Fi hotspots for vehicular communications
–
Advancement of 5G networks enabling gigabits data transmission
–
Expansion of M2M and IoT for in-vehicle technologies; enabling many in-vehicle applications, telematics, and infotainment solutions. There will be a wider integration between connected cars and the Internet of Things. The number of M2M connections is likely to grow three to four times by 2019.
–
Growth of peer-to-peer technologies (e.g. ZigBee, Bluetooth) inside the vehicles for electronic engine control, driving assistants, safety mechanisms, and infotainment.
–
Use of RF communication alternatives such as VLC and Li-Fi for vehicular communication
–
Developments in preventing hackers from intruding and accessing in-vehicle networks
–
Increase of telematics option inside the vehicles
U.S. Department of Transportation
17
Next Steps –
Invest in addressing the issues with DSRC (limited spectrum, expensive infrastructure, adoption issues*)
–
Research techniques for an integrated V2X/Cellular solution
–
Investigate/monitor progress of 5G cellular, M2M, and D2D technologies development and their use for vehicular communication and automation
–
Further research solutions for communication security, and privacy issues
–
Participate in development of necessary standards, laws, and regulations for enabling CV/AV
*DSRC, and any cooperative network require adaptation by a critical mass to be effective U.S. Department of Transportation
18
Discussion 1. What does the future of DSRC look like and what complimentary technologies are needed? 2. How should we address the infrastructure, limited spectrum, and adoption issues w/r to employing DSRC? 3. What does the group think about communications using the visible light spectrum? 4. What are the major challenges / gaps in terms of security and data privacy and how can they be best addressed? 5. What are lessons learned from other relevant prior transportation technology roll-outs? 6. Recommendations for the automation program? U.S. Department of Transportation
19
Sensing Needs – Accurate and reliable 360 degree sensing of vehicle surrounding – Sensor fusion and data accessibility (user interface) – Work in all weather conditions without degraded accuracy – Quality and size of sensors at costs affordable for economy vehicles
Trends – An increasing number of applications are using vision based systems for detecting obstructions and recognizing objects (e.g. 360-degree truck vision technology)
– With evolving sensor technologies, advances in micro-processors have gained popularity (e.g. Nvidia’s Tegra K1 processor for Advanced Driver Assistance Systems) – The market is also looking to use ‘cloud’ systems to analyze real-time data from a variety of sensors and other dynamic events (e.g. Dynamic eHoizon System – Predictive Navigation) U.S. Department of Transportation
20
Sensing Gaps – Accuracy of sensors are increasing but are not completely reliable just yet – Sensor performance in adverse weather conditions – More robust standards to enable data interface & accessibility of different sensors Next Steps – Explore sensors that could be used from the infrastructure perspective. For example use stereo cameras at complex intersections to get a holistic view for adaptive signal timing strategies – Explore infrastructure sensors that could be used to provide more quality data for automated vehicle applications > weather data, intersection blind spot, etc. – Explore changes/updates to traffic signs for improved recognition by sensors for automated vehicle applications
U.S. Department of Transportation
21
Discussion 1. What type of vehicle sensors are the most important? 2. What are the major gaps in today’s sensors? 3. What are the major challenges in sensor development / accuracy? 4. How will vehicle sensors evolve in the near future? 5. What are lessons learned from other relevant prior transportation technology roll-outs? 6. Recommendations for the automation program?
U.S. Department of Transportation
22
Human Factors Needs – Cooperative and non-distracting devices for supporting human machine interaction at different levels of automation – Safe and Smooth transition from human control to automated control – Accommodate the heterogeneity of the drivers reaction/perception in the design – Study how users and non-users will interact in a mixed environment Overview / Trends – The usage of driver state sensing to mitigate driver distraction is underway by several OEMs. For example, Ghost Car technology by Jaguar, zForce Drive Technology and dashboard mounted camera for facial expression monitoring – Understanding Driver behavior in different levels of automation is investigated by many research labs; especially using driver simulators. For example, Transport Research Laboratory (TRL) simulator “DigiCar” – Many driver information/assistance display technologies are claimed to be ready (or almost ready) for market usage. For example, Panasonic’s eCockpit U.S. Department of Transportation
23
Human Factors Gaps – The impact of many of the current technologies (including after-market devices) on safety is still questionable; especially for the different levels of automation – Very few available studies addressed the human factor aspect of joining/disjoining automated vehicle platoons – The research related to automated vehicles interaction with other vulnerable road users (also pedestrians, cyclists, and legacy vehicles) is still lacking
– The interaction of automated vehicles and legacy vehicles (users Vs. non-users) is still not well addressed in the literature – The relationship between automated vehicles and disable/elderly people is still not addressed Next Steps – Investigate the impact of after-market devices on the human level of comfort, distraction and safety
– Study the needs and implications of different levels of automation on the elderly drivers and/or drivers with disabilities – Study the human factor aspect of joining/disjoining automated vehicle platoons under different operation settings. U.S. Department of Transportation
24
Discussion 1. What are the most important features to understand in Human Factors? 2. How do you deal with non-users? 3. How will the HMI market evolve in the near future? 4. What are lessons learned from other relevant prior transportation technology roll-outs? 5. Recommendations for the automation program? (specially for the enabling technologies track on the roadmap)
U.S. Department of Transportation
25
Closing Remarks 1. Other areas of importance? 2. Research strategy moving forward? 3. Top 5 research areas Short Term (1-2 yrs) Medium Term (3-5 yrs) Long Term (6-8 yrs)
U.S. Department of Transportation
26
