simple weighting method IMSC2016 RL ETHtemp
A simple weighting method for combining multimodel projections Reto Knutti, Jan Sedláˇcek, Ruth Lorenz, and Ben Sanderson
IMSC 2016, Canmore
Ruth Lorenz
June 9, 2016
1
Uncertainties and ensembles in global climate models Multimodel ensembles are heterogeneous, some models performing better than others for certain purpose
Ruth Lorenz
June 9, 2016
2
Large uncertainty in projected decline in September Arctic sea-ice
Massonnet et al. 2012, Cryosphere Ruth Lorenz
June 9, 2016
3
Uncertainties and ensembles in global climate models Multimodel ensembles are heterogeneous, some models performing better than others for certain purpose → quality for purpose, ”all models are wrong but some are useful“ not independent, developers shared ideas and code
Ruth Lorenz
June 9, 2016
4
Model history and genealogy
Edwards 2011, WIRE
Knutti et al. 2013, GRL Ruth Lorenz
June 9, 2016
5
Uncertainties and ensembles in global climate models Multimodel ensembles are: heterogeneous, some models performing better than others for certain purpose → quality for purpose, ”all models are wrong but some are useful“ not independent, developers shared ideas and code Attempts to move forward: taking into account model performance and dependence e.g. Abramowitz and Gupta, 2008; Sanderson et al. 2015 Problem: rather complex approaches, researchers not focusing on weighting ensembles unlikely to implement Ruth Lorenz
June 9, 2016
6
Method
wi = e
−
D2 i σ2 D
S2 ij M X − 2 / 1 + e σS
(1)
j 6=i
wi : weight for model i Di : distance of model i to observations σD : parameter, determines how strongly model performance is weighted M: number of models Sij : distance between model i and model j σS : parameter, determines how strongly model similarity is weighted Ruth Lorenz
June 9, 2016
7
Prerequisites Available observations to define constraint Uncertainty of observations smaller than model spread Some skill in models
Choices to be made How to measure model performance? How to measure model similarity? Which variables to take into account? How strongly to weight model performance (σD )? When to consider models to be similar (σS )? Ruth Lorenz
June 9, 2016
8
A working example Model performance: Model similarity: Variables:
Root mean squared error between model and observations Root mean squared error between models September arctic sea ice climatology and standard deviation Seasonal cycle arctic temperature climatology and trend
σD and σS :
Ruth Lorenz
June 9, 2016
9
Choosing σD and σS parameters perfect model approach:
correlation:
percent within 5–95% predicted range:
σD
σD
Ruth Lorenz
June 9, 2016
10
September arctic sea-ice and annual temperature
Ruth Lorenz June 9, 2016
11
Unweighted and weighted model mean bias Percent reduction in bias non-weighted to weighted [%]
Ruth Lorenz
June 9, 2016
12
Unweighted and weighted model mean surface warming and sea ice edge
Ruth Lorenz
June 9, 2016
13
Advantages of method Adding identical model does not change weighted ensemble mean Initial conditions ensemble can be included without problems ”Bad“ model obtains zero weight Easy to use for spatial fields and multiple depending variables
Ruth Lorenz
June 9, 2016
14
Conclusion A simple method once some choices are made Possible to decrease uncertainty in climate projections and move away from model democracy in certain cases As for any other method choices to be made, which are not simple Metrics depend on the problem and require thought
Outlook Other examples Influence of performance measure (RMSE)? Other possible metrics? Best ways to determine σ ’s
Ruth Lorenz
June 9, 2016
15
Additional slides
Ruth Lorenz
June 9, 2016
16
S
Chosen σD and σS parameters in σ space
D Ruth Lorenz
June 9, 2016
17
Observations and Models in space
Abramowitz 2010, AMOJ Ruth Lorenz
June 9, 2016
18
Differences between models and obs are within model and model distribution model-obs count artificially increased to have same total count as model-model Histogram of averaged delta matrix model_model model_obs
1400 1200
Count
1000 800 600 400 200 0 0.4
0.6
0.8
1.0 Delta
1.2
1.4
1.6
Ruth Lorenz
June 9, 2016
19
A temperature example σS =0.6, σD =0.6, 0.4, 0.2 CNA_flip tasmax clim seas [K] timeseries
314 312 310 308 306 304 302 300 298 296 1940
312 310
1960
1980
2000
306 304 302 300 298 1960
1980
2040
314
ERAint non-weighted MMM weighted MMM
308
296 1940
2020 Year
CNA_flip tasmax clim seas [K] timeseries
CNA_flip tasmax clim seas [K] timeseries
314
ERAint non-weighted MMM weighted MMM
2000
2020 Year
2040
2060
2080
2100
312 310
2060
2080
2100
ERAint non-weighted MMM weighted MMM
308 306 304 302 300 298 296 1940
1960
1980
2000
2020 Year
2040
2060
2080
2100
Ruth Lorenz
June 9, 2016
20
IMSC 2016, Canmore
Ruth Lorenz
June 9, 2016
1
Uncertainties and ensembles in global climate models Multimodel ensembles are heterogeneous, some models performing better than others for certain purpose
Ruth Lorenz
June 9, 2016
2
Large uncertainty in projected decline in September Arctic sea-ice
Massonnet et al. 2012, Cryosphere Ruth Lorenz
June 9, 2016
3
Uncertainties and ensembles in global climate models Multimodel ensembles are heterogeneous, some models performing better than others for certain purpose → quality for purpose, ”all models are wrong but some are useful“ not independent, developers shared ideas and code
Ruth Lorenz
June 9, 2016
4
Model history and genealogy
Edwards 2011, WIRE
Knutti et al. 2013, GRL Ruth Lorenz
June 9, 2016
5
Uncertainties and ensembles in global climate models Multimodel ensembles are: heterogeneous, some models performing better than others for certain purpose → quality for purpose, ”all models are wrong but some are useful“ not independent, developers shared ideas and code Attempts to move forward: taking into account model performance and dependence e.g. Abramowitz and Gupta, 2008; Sanderson et al. 2015 Problem: rather complex approaches, researchers not focusing on weighting ensembles unlikely to implement Ruth Lorenz
June 9, 2016
6
Method
wi = e
−
D2 i σ2 D
S2 ij M X − 2 / 1 + e σS
(1)
j 6=i
wi : weight for model i Di : distance of model i to observations σD : parameter, determines how strongly model performance is weighted M: number of models Sij : distance between model i and model j σS : parameter, determines how strongly model similarity is weighted Ruth Lorenz
June 9, 2016
7
Prerequisites Available observations to define constraint Uncertainty of observations smaller than model spread Some skill in models
Choices to be made How to measure model performance? How to measure model similarity? Which variables to take into account? How strongly to weight model performance (σD )? When to consider models to be similar (σS )? Ruth Lorenz
June 9, 2016
8
A working example Model performance: Model similarity: Variables:
Root mean squared error between model and observations Root mean squared error between models September arctic sea ice climatology and standard deviation Seasonal cycle arctic temperature climatology and trend
σD and σS :
Ruth Lorenz
June 9, 2016
9
Choosing σD and σS parameters perfect model approach:
correlation:
percent within 5–95% predicted range:
σD
σD
Ruth Lorenz
June 9, 2016
10
September arctic sea-ice and annual temperature
Ruth Lorenz June 9, 2016
11
Unweighted and weighted model mean bias Percent reduction in bias non-weighted to weighted [%]
Ruth Lorenz
June 9, 2016
12
Unweighted and weighted model mean surface warming and sea ice edge
Ruth Lorenz
June 9, 2016
13
Advantages of method Adding identical model does not change weighted ensemble mean Initial conditions ensemble can be included without problems ”Bad“ model obtains zero weight Easy to use for spatial fields and multiple depending variables
Ruth Lorenz
June 9, 2016
14
Conclusion A simple method once some choices are made Possible to decrease uncertainty in climate projections and move away from model democracy in certain cases As for any other method choices to be made, which are not simple Metrics depend on the problem and require thought
Outlook Other examples Influence of performance measure (RMSE)? Other possible metrics? Best ways to determine σ ’s
Ruth Lorenz
June 9, 2016
15
Additional slides
Ruth Lorenz
June 9, 2016
16
S
Chosen σD and σS parameters in σ space
D Ruth Lorenz
June 9, 2016
17
Observations and Models in space
Abramowitz 2010, AMOJ Ruth Lorenz
June 9, 2016
18
Differences between models and obs are within model and model distribution model-obs count artificially increased to have same total count as model-model Histogram of averaged delta matrix model_model model_obs
1400 1200
Count
1000 800 600 400 200 0 0.4
0.6
0.8
1.0 Delta
1.2
1.4
1.6
Ruth Lorenz
June 9, 2016
19
A temperature example σS =0.6, σD =0.6, 0.4, 0.2 CNA_flip tasmax clim seas [K] timeseries
314 312 310 308 306 304 302 300 298 296 1940
312 310
1960
1980
2000
306 304 302 300 298 1960
1980
2040
314
ERAint non-weighted MMM weighted MMM
308
296 1940
2020 Year
CNA_flip tasmax clim seas [K] timeseries
CNA_flip tasmax clim seas [K] timeseries
314
ERAint non-weighted MMM weighted MMM
2000
2020 Year
2040
2060
2080
2100
312 310
2060
2080
2100
ERAint non-weighted MMM weighted MMM
308 306 304 302 300 298 296 1940
1960
1980
2000
2020 Year
2040
2060
2080
2100
Ruth Lorenz
June 9, 2016
20
