Big Data Market Environment
Weather Forecast Improvements at the UK Met Office: Responding to the Big Data Challenge Vicky Pope Input from Rich Carne, Alex Longden, Derrick Ryall
January 2017
Weather Information Value Chain Monitor & Observe
Model & Forecast
Interpret
Produce
Decision making
Economic Value
March 2016
Better use of observations Biggest improvement in a decade
Model and observations improvements
Monitor & Observe
Model & Forecast
November 2016
Spring 2017
2017
New radar network
Extend runs to 5 days
Global model resolution increase to 10km
Improved forecasts of showers
New UK hourly nowcasts
Bigger UK domain
Precipitation type Doppler winds Increased resolution
Autumn 2017
2018-19
Increased vertical resolution Run UK ensembles to 5 days
Couple atmosphere and ocean Better convection
1.2 Pflops
Improved showers
© Crown copyright Met Office
8 Pflops
16 Pflops
Interpret
Integrated Approach to Air Quality Risk: Joining models, data and policy Feedbacks
Global Weather
Local Weather
Emissions
Air Quality
Impacts
Action
Hazard x Vulnerability x Exposure Chronic and acute Environmental Quality of life/health Weather and climate feedbacks Observations
Long term policy e.g. energy mix, vehicle standards, clean air zones, international Tactical e.g. health advice, industrial and traffic management
O3 PM2.5
Weather forecast Multi year weather analysis Climate change
Nested finer scale local weather models Probabilistic representation
Implimented
Anthropogenic and biogenic Temporal and spatial variation Street, local, national, regional, global Observaitons
Coupled with weather Street to global Gas and aerosol chemistry Hourly to climatic variation Deposition Observations
Certain capability/linkages but much is limited, fragmented and not aligned. Range of scales from street to global is a particular challenge.
Should be set in a probabilistic framework.
Charge for service and content
Produce
Data: Open & available
Changes in the information economy © Crown copyright Met Office
Data: Open & free
Charge for data
Climate change adaptation guide for Energy Phase 2 project Coastal infrastructure
Wind Little risk: Action unlikely to be required
CCGTs
Possible risk: Action where assets already close to design limits
TransFormers Urban & Rural
Some understanding: More work to quantify risks
Cabels North & South
Probable risk: Action likely to be required for most assets
Network design
Line rating Network Resilience Climatology for demand forecasting
2010| 2020| 2030| 2040| 2050| 2060| 2070|
© Crown copyright Met Office
Decision making
Decision making
• • • • • • • •
Economic Value
Pipe material Pipe depth Pipe age Soil type Pressure Traffic vibrations Contractor activity Weather
Leakage is complex
Visible leakage / bursts
Customer Side Leakage
Seasonal leakage
© Crown copyright Met Office
Invisible leakage
Decision making
• • • • • • •
Economic Value
Pipe material Pipe depth Pipe age Soil type Pressure Traffic vibrations Contractor activity
Modelling should be complex • Air temperature • Water temperature • Ground temperature • Rainfall • Evapotranspiration • Soil Moisture Deficit Visible leakage / bursts Customer Side Leakage
© Crown copyright Met Office Seasonal leakage
Invisible leakage
New ways of working
Plus social media, IoT, mobile devices, VR & AR, bots, Artificial Intelligence © Crown copyright Met Office
Monitor & Observe
New ways of working Weather Observations Website • Launched June 2011 • Over 28 million observations received… • …from over 1700 sites worldwide
• Users in 145 countries
http://wow.metoffice.gov.uk/
New ways of working Produce
Informatics Lab Created in 2015 Two purposes: Making environmental data/science useful
Transform ways of working and culture
New ways of working Produce
What do we do? High-performing & engaged teams — Transformation
From (big) data to information — Data Analytics From information to decisions — Interactive Media
www.informaticslab.co.uk
Produce
© Crown copyright Met Office
New ways of working
Decision making
The next generation mobile network will have a greater weather elements such as rain have the potential to dependency “Weather degrade the performance of communications networks at these new higher frequency bands. With our expertise in both numerical weather prediction and the remote sensing of the atmosphere – for example, weather satellites and radar – the Met Office is well-placed to contribute realistic high-resolution weather scenarios and the associated impacts on signal transmission to our project partners,”
kext & kabs @ 25 32 41 84 GHz
T+0 UKPP @ 2 km
New ways of working
T+0 UKPP @ 100 m
Project led by Ordinance Survey Stephan Havemann, Robert Scovell, Jean-Claude Thelin, Dave Jones © Crown copyright Met Office
Decision making
New ways of working
HPC Complex – Collaboration Space “A place in which the Met Office creates and builds opportunities for business growth and scientific & technical collaboration”
HPC Complex - Collaboration Space •
Demonstration venue :Hacks, Demos & Exploration
•
Conferences & workshops :Technical, Scientific & Business
•
External Collaboration:Longer term (months) collaborations e.g.ERI (incubation) & Data Analytics Hub
•
Trial Space:Providing a space in which to trial new ways of working &/or an Innovation hub for UK government and industry
Weather Forecast Improvements at the UK Met Office: Responding to the Big Data Challenge Vicky Pope
January 2017
January 2017
Weather Information Value Chain Monitor & Observe
Model & Forecast
Interpret
Produce
Decision making
Economic Value
March 2016
Better use of observations Biggest improvement in a decade
Model and observations improvements
Monitor & Observe
Model & Forecast
November 2016
Spring 2017
2017
New radar network
Extend runs to 5 days
Global model resolution increase to 10km
Improved forecasts of showers
New UK hourly nowcasts
Bigger UK domain
Precipitation type Doppler winds Increased resolution
Autumn 2017
2018-19
Increased vertical resolution Run UK ensembles to 5 days
Couple atmosphere and ocean Better convection
1.2 Pflops
Improved showers
© Crown copyright Met Office
8 Pflops
16 Pflops
Interpret
Integrated Approach to Air Quality Risk: Joining models, data and policy Feedbacks
Global Weather
Local Weather
Emissions
Air Quality
Impacts
Action
Hazard x Vulnerability x Exposure Chronic and acute Environmental Quality of life/health Weather and climate feedbacks Observations
Long term policy e.g. energy mix, vehicle standards, clean air zones, international Tactical e.g. health advice, industrial and traffic management
O3 PM2.5
Weather forecast Multi year weather analysis Climate change
Nested finer scale local weather models Probabilistic representation
Implimented
Anthropogenic and biogenic Temporal and spatial variation Street, local, national, regional, global Observaitons
Coupled with weather Street to global Gas and aerosol chemistry Hourly to climatic variation Deposition Observations
Certain capability/linkages but much is limited, fragmented and not aligned. Range of scales from street to global is a particular challenge.
Should be set in a probabilistic framework.
Charge for service and content
Produce
Data: Open & available
Changes in the information economy © Crown copyright Met Office
Data: Open & free
Charge for data
Climate change adaptation guide for Energy Phase 2 project Coastal infrastructure
Wind Little risk: Action unlikely to be required
CCGTs
Possible risk: Action where assets already close to design limits
TransFormers Urban & Rural
Some understanding: More work to quantify risks
Cabels North & South
Probable risk: Action likely to be required for most assets
Network design
Line rating Network Resilience Climatology for demand forecasting
2010| 2020| 2030| 2040| 2050| 2060| 2070|
© Crown copyright Met Office
Decision making
Decision making
• • • • • • • •
Economic Value
Pipe material Pipe depth Pipe age Soil type Pressure Traffic vibrations Contractor activity Weather
Leakage is complex
Visible leakage / bursts
Customer Side Leakage
Seasonal leakage
© Crown copyright Met Office
Invisible leakage
Decision making
• • • • • • •
Economic Value
Pipe material Pipe depth Pipe age Soil type Pressure Traffic vibrations Contractor activity
Modelling should be complex • Air temperature • Water temperature • Ground temperature • Rainfall • Evapotranspiration • Soil Moisture Deficit Visible leakage / bursts Customer Side Leakage
© Crown copyright Met Office Seasonal leakage
Invisible leakage
New ways of working
Plus social media, IoT, mobile devices, VR & AR, bots, Artificial Intelligence © Crown copyright Met Office
Monitor & Observe
New ways of working Weather Observations Website • Launched June 2011 • Over 28 million observations received… • …from over 1700 sites worldwide
• Users in 145 countries
http://wow.metoffice.gov.uk/
New ways of working Produce
Informatics Lab Created in 2015 Two purposes: Making environmental data/science useful
Transform ways of working and culture
New ways of working Produce
What do we do? High-performing & engaged teams — Transformation
From (big) data to information — Data Analytics From information to decisions — Interactive Media
www.informaticslab.co.uk
Produce
© Crown copyright Met Office
New ways of working
Decision making
The next generation mobile network will have a greater weather elements such as rain have the potential to dependency “Weather degrade the performance of communications networks at these new higher frequency bands. With our expertise in both numerical weather prediction and the remote sensing of the atmosphere – for example, weather satellites and radar – the Met Office is well-placed to contribute realistic high-resolution weather scenarios and the associated impacts on signal transmission to our project partners,”
kext & kabs @ 25 32 41 84 GHz
T+0 UKPP @ 2 km
New ways of working
T+0 UKPP @ 100 m
Project led by Ordinance Survey Stephan Havemann, Robert Scovell, Jean-Claude Thelin, Dave Jones © Crown copyright Met Office
Decision making
New ways of working
HPC Complex – Collaboration Space “A place in which the Met Office creates and builds opportunities for business growth and scientific & technical collaboration”
HPC Complex - Collaboration Space •
Demonstration venue :Hacks, Demos & Exploration
•
Conferences & workshops :Technical, Scientific & Business
•
External Collaboration:Longer term (months) collaborations e.g.ERI (incubation) & Data Analytics Hub
•
Trial Space:Providing a space in which to trial new ways of working &/or an Innovation hub for UK government and industry
Weather Forecast Improvements at the UK Met Office: Responding to the Big Data Challenge Vicky Pope
January 2017
