Fast Simulation & Optimization
Simulation Optimization for Discrete-event Systems: Ordinal Optimization and Beyond Chun-Hung Chen Dept. of Systems Engineering & Operations Research George Mason University Fairfax, VA, USA
GREAT APPRECIATIONS TO PROF. HO
• Studied at Harvard from 1991-94 • #39 • One year overlapping with Leyuan Shi
FLIGHT DELAY IN CHINA
> • Arrive before the queue starts to build up • Avoid short flights ‒ ‒
Longer flights have higher priority GDP: Ground Delay Program
• Avoid congested areas ‒ ‒
Enough capacity is needed for the entire flight Shanghai, for example
CAN BE DRAMATICALLY INCREASED, BUT… • Separation: 45 sec vs. 2 min • Safety standard: 10-9 vs. 0.0
Continuous systems
Discrete event dynamic systems
• • •
From 1990, students worked on a new subject – OO Fast solution for hard optimization problem Extremely important in the new era of big data
NETWORKED SYSTEMS WITH BIG DATA • IOT / Internet Plus • Industry 4.0 • Cyber Physical Systems
IN THE NEW ERA OF BIG DATA • Challenges and Opportunities ‒ Connection and integration system becomes larger and more complicated ‒ Global sensors more factors (input variables) to include for decision making ‒ Real-time dynamic data data keep arriving and changing less time to make a decision ‒ Smarter decision smart city, smart grids, smartcare, smart buildings
• Useful Methodology ‒ ‒
Large-scale Optimization Efficient Simulation
Ordinal Opt.
SIMULATION OPTIMIZATION
Control policy, Decision, Design alternative
X
System performance, Output
Stochastic Simulation
min J ( X ) X
J(X)
COMPUTATIONAL ISSUES
Simulation
Multiple Simulation Runs (replications)
1 Min J ( X ) E[ f ( X , W )] X N Many Alternatives in Design Space
N
f (X ,Wi ) i 1
Optimization
EFFICIENCY CONCERN FOR SIMULATING K DESIGNS Alternative Design
K K-1
Challenges: K*N can be very large
6 5 4 3 2 1 # of simulation runs
N
METHODOLOGIES Alternative Design
K K-1
6 5 4 3 2 1 # of simulation runs
•
N
Ordinal Optimization (Ho et al. 1992) 1. Focus on good enough solutions 2. Concentrate on relative order comparison only need to conduct a very small fraction of simulations
•
Optimal Computing Budget Allocation (OCBA) Further enhance efficiency via optimal control of simulation
•
Ordinal Transformation
OO1: GOOD ENOUGH SOLUTION •
Basic Idea ‒ Instead of asking the best design, OO focuses on a good enough solution ‒ Only need to simulate a very small fraction of designs
•
Conservative Case – Blind Picking ‒ Assume simulation estimation noise is extremely large (e.g., before simulation) ‒ If we are willing to accept a good design, say within top-0.1% (99.9 percentile), with a confidence probability Psat Psat
90%
99%
# of designs for simulation
2301
4603
99.99% 99.999% 9206
11,508
• Utilize Simple Analytic Approximation Model – Can do much better than blind picking in the above worst cast analysis
• See Lau & Ho (1997), Luo, Chen, Guignard-Spielberg (2001), Lin & Ho (2002), Lin et al. (2004)
OO2: ORDINAL COMPARISON •
Basic Idea ‒ Instead of accurately estimating the performance measures for all designs, OO concentrates on relative order comparison 1 ‒ Can obtain exponential convergence rate (vs. O( ) for confidence N
interval)
‒ Only need to conduct a smaller number of simulation runs
•
Correct Selection Probability P{CS} P{ The selected design is indeed better than others } = P{ Correct Selection of the Best Alternative }
= 1 - e- N
(, > 0) (Dai 1996, Dai & Chen 1997, Ho et al. 2000)
OO: MUCH SMALLER SUBSET OF SIMULATION Alternative Design
K K-1
6 5 4 3 2 1 # of simulation runs
N
Alternative Design
Ordinal Optimization
Ranking & Selection
k k-1
2 1 # of simulation runs
RANKING & SELECTION: TRADITIONAL PROCEDURES •
Many developments in simulation society : ‒ Rinott (1978), Dudewicz and Dalal (1975), Goldsman and Nelson (1994), Matejcik and Nelson (1993, 1995), Bechhofer, et al. (1995), …
•
Ideas ‒ Based on least favorable configuration
•
Main results ‒ Find the required Ni to asymptotically guarantee a desired P{CS} ‒ Conservative ‒ Simulation allocation is proportional to variance: Ni = ci i2
SMART SIMULATION ALLOCATION
99% Confidence Intervals for J(X) after some simulations
x1
•
x2
x3
x4
x5
Which designs should we simulate more? ‒ 2 & 3 are clearly superior ‒ 1, 4 & 5 have larger variances
•
Chen (1995) & Chen (1996) propose smarter allocations for efficiency
OPTIMAL COMPUTING BUDGET ALLOCATION (OCBA) • Maximize the Probability of Correctly Selecting the Best Design 𝐦𝐚𝐱
𝑁1 ,…,𝑁𝑠
s.t.
𝑷{𝐂𝐒} 𝑁1 + 𝑁2 + ⋯ + 𝑁𝑠 = 𝑇 (total number of runs)
• Asymptotically Optimal Solution Ni
(b, j / j)2
Nj
(b,i / i)2
Nb = b
ib (Ni2 / i2)
for i j b
S OME I NSIGHTS OF OCBA R ULE
Signal to Noise Ratio
c3
c2
1
2
1,2
3
1,3
1, 3 N2 3 2 N 3 1, 2 2 2
Designs
inversely proportional to the square of the signal to noise ratio
SELECTED GENERALIZATIONS & EXTENSIONS (1) • Non-normal Distributions - P. Glynn (Stanford Univ.) - S. Juneja (Columbia Univ.)
• Heavy-tailed Distributions - M. Broadie, M. Han, and A. Zeevi (Columbia University)
• Minimizing Variance Instead of Minimizing Mean - Lucy Pao & Lidija Trailovic (U. of Colorado)
• Correlated Sampling - Michael Fu (U. of Maryland at College Park) - J.Q. Hu & Y.J. Peng (Fudan Univ.)
• Finding both Simple and Good Designs - E. Zhou (Georgia Tech) - Q.S. Jia (Tsinghua University)
SELECTED GENERALIZATIONS & EXTENSIONS (2) • Multiple Objectives - L. Lee & E. Chew (National University of Singapore)
• Small Computing Budget - J. LaPorte (US Military Academy at West Point) - J. Branke (Warwick Univ.)
• Transient Simulation - D. J. Morrice (University of Texas at Austin)
• Expected opportunity cost instead of the probability of correct selection - S. Gao (City U of HK) - W. Chen (Rutgers Univ.)
- L. Shi (Peking Univ.)
• Optimal Subset Selection - S. Zhang (Shanghai University)
EFFICIENCY USING MULTI-FIDELITY MODELS Full Simulation Model
Simplified Model
Some examples: High-fidelity model Low fidelity model Discrete-event simulation Queueing theory Fine model Coarse model Capturing uncertainty Ignoring uncertainty
Complex
Much simpler
Good accuracy, but very time consuming
Biased, but fast
MULTI-FIDELITY TIME-SENSITIVE DATA Now Freshness of Data
Decision point
Fast Time Data from last hour Data from yesterday
Data from last week Data from last month
Time Availability to Decision Point
EXAMPLE: RESOURCE ALLOCATION PROBLEM • Flexible Manufacturing System • • • • •
2 product types 5 workstations Non-exponential service times Re-entrant manufacturing process Product 1 has higher priority than product 2
• Decision variable: number of machines allocated to each workstation 𝐌𝐢𝐧𝐢𝐦𝐢𝐳𝐞 Expected Total Processing Time 𝐒𝐮𝐛𝐣𝐞𝐜𝐭 𝐭𝐨 𝟓 ≤ # of machines at each workstation ≤ 𝟏𝟎 Total # of machines at all workstatiosn = 𝟑𝟖
• # of alternatives: 780
P1
P2
Workstation 1 Workstation 2 Workstation 3 Workstation 4 Workstation 5
FULL SIMULATION VS. QUEUEING APPROXIMATION
• Bias is non-homogeneous and can be quite large
ORDINAL TRANSFORMATION
High Fidelity
• •
Low Fidelity
Quickly evaluate each alternative using low-fidelity model Transform the decision space into an ordinal space
ORDINAL TRANSFORMATION
OT
BENEFITS
OT
– – – –
Non-smooth response can become much smoother Neighborhood connection is strengthened Designs with similar performance are grouped together Search/optimization efficiency is enhanced
SOME PROPERTIES • Theorem 1. Ordinal transformation can reduce the variability of each group by at least
100 𝟏 − ‒
𝟑 𝒎+𝟐
+
𝟔 𝒌𝟐
𝝆𝟐 %
is the correlation between original and ordinal models
• Theorem 2. The differences between the means of two neighboring groups can be increased by 100
𝟏𝟐𝒎 𝒌(𝒎+𝟏)
𝝆%
MACHINE ALLOCATION PROBLEM • 10 Groups (k=10)
MULTI-FIDELITY DATA AND MODELS • Full simulation (high-fidelity) New Demand
Product 1
Product 2
250
150
• Low-fidelity model: queueing approximation ‒
Very fast but poor approximation
New Demand
Product 1
Product 2
250
150
• Multi-fidelity data (with old simulations) ‒
Minimum additional cost but poor approximation Product 1
Product 2
Old Demand 1
210
170
Old Demand 2
280
140
OPTIMAL LINEAR COMBINATION OF TWO APPROXIMATION MODELS • Suppose we have 2 approximation models: g1 and g2 𝑔 𝑋 = 𝑎1 𝑔1 𝑋 + 𝑎2 𝑔2 𝑋
• To maximize the correlation with the true model 𝐶𝑜𝑣 𝑔 𝑋 , 𝑓 𝑋
𝑚𝑎𝑥𝑎1,𝑎2 𝜌 ≡
𝑉𝑎𝑟 𝑔 𝑋
𝑉𝑎𝑟 𝑓 𝑋
• Optimal weighting factors 𝑎1∗ 𝜌1 − 𝜌12 𝜌2 𝜎2 = 𝑎2∗ 𝜌2 − 𝜌12 𝜌1 𝜎1
** Joint work with Si Zhang
LEAD TO HIGH CORRELATION & ALIGNMENT • Ordinal Transformation using single model/data Rank Correlation
Alignment (top-5)
Old Demand 1 only
0.623
2
Old Demand 2 only
0.596
2
• Ordinal Transformation with intelligent combining use of two models
Old 1 + Old 2 Demand Models
Rank Correlation
Alignment (top-5)
0.875
5
DOES A SECOND MODEL ALWAYS HELP? • Consider a case where 1 = 0.6
12
2
helpful area, i.e., 12 < 0.2
If 2 = 0.6
SUMMARY •
Ordinal Optimization (Ho et al. 1992) 1. Focus on good enough solutions 2. Concentrate on relative order comparison only need to conduct a very small fraction of simulations
•
Optimal Computing Budget Allocation (OCBA) ‒ Further enhance OO efficiency via optimal control of simulation
•
Ordinal Transformation ‒ Utilize low-fidelity models/data to transform the decision space into a better space ‒ Enhance search efficiency
GREAT APPRECIATIONS TO PROF. HO
• Studied at Harvard from 1991-94 • #39 • One year overlapping with Leyuan Shi
FLIGHT DELAY IN CHINA
> • Arrive before the queue starts to build up • Avoid short flights ‒ ‒
Longer flights have higher priority GDP: Ground Delay Program
• Avoid congested areas ‒ ‒
Enough capacity is needed for the entire flight Shanghai, for example
CAN BE DRAMATICALLY INCREASED, BUT… • Separation: 45 sec vs. 2 min • Safety standard: 10-9 vs. 0.0
Continuous systems
Discrete event dynamic systems
• • •
From 1990, students worked on a new subject – OO Fast solution for hard optimization problem Extremely important in the new era of big data
NETWORKED SYSTEMS WITH BIG DATA • IOT / Internet Plus • Industry 4.0 • Cyber Physical Systems
IN THE NEW ERA OF BIG DATA • Challenges and Opportunities ‒ Connection and integration system becomes larger and more complicated ‒ Global sensors more factors (input variables) to include for decision making ‒ Real-time dynamic data data keep arriving and changing less time to make a decision ‒ Smarter decision smart city, smart grids, smartcare, smart buildings
• Useful Methodology ‒ ‒
Large-scale Optimization Efficient Simulation
Ordinal Opt.
SIMULATION OPTIMIZATION
Control policy, Decision, Design alternative
X
System performance, Output
Stochastic Simulation
min J ( X ) X
J(X)
COMPUTATIONAL ISSUES
Simulation
Multiple Simulation Runs (replications)
1 Min J ( X ) E[ f ( X , W )] X N Many Alternatives in Design Space
N
f (X ,Wi ) i 1
Optimization
EFFICIENCY CONCERN FOR SIMULATING K DESIGNS Alternative Design
K K-1
Challenges: K*N can be very large
6 5 4 3 2 1 # of simulation runs
N
METHODOLOGIES Alternative Design
K K-1
6 5 4 3 2 1 # of simulation runs
•
N
Ordinal Optimization (Ho et al. 1992) 1. Focus on good enough solutions 2. Concentrate on relative order comparison only need to conduct a very small fraction of simulations
•
Optimal Computing Budget Allocation (OCBA) Further enhance efficiency via optimal control of simulation
•
Ordinal Transformation
OO1: GOOD ENOUGH SOLUTION •
Basic Idea ‒ Instead of asking the best design, OO focuses on a good enough solution ‒ Only need to simulate a very small fraction of designs
•
Conservative Case – Blind Picking ‒ Assume simulation estimation noise is extremely large (e.g., before simulation) ‒ If we are willing to accept a good design, say within top-0.1% (99.9 percentile), with a confidence probability Psat Psat
90%
99%
# of designs for simulation
2301
4603
99.99% 99.999% 9206
11,508
• Utilize Simple Analytic Approximation Model – Can do much better than blind picking in the above worst cast analysis
• See Lau & Ho (1997), Luo, Chen, Guignard-Spielberg (2001), Lin & Ho (2002), Lin et al. (2004)
OO2: ORDINAL COMPARISON •
Basic Idea ‒ Instead of accurately estimating the performance measures for all designs, OO concentrates on relative order comparison 1 ‒ Can obtain exponential convergence rate (vs. O( ) for confidence N
interval)
‒ Only need to conduct a smaller number of simulation runs
•
Correct Selection Probability P{CS} P{ The selected design is indeed better than others } = P{ Correct Selection of the Best Alternative }
= 1 - e- N
(, > 0) (Dai 1996, Dai & Chen 1997, Ho et al. 2000)
OO: MUCH SMALLER SUBSET OF SIMULATION Alternative Design
K K-1
6 5 4 3 2 1 # of simulation runs
N
Alternative Design
Ordinal Optimization
Ranking & Selection
k k-1
2 1 # of simulation runs
RANKING & SELECTION: TRADITIONAL PROCEDURES •
Many developments in simulation society : ‒ Rinott (1978), Dudewicz and Dalal (1975), Goldsman and Nelson (1994), Matejcik and Nelson (1993, 1995), Bechhofer, et al. (1995), …
•
Ideas ‒ Based on least favorable configuration
•
Main results ‒ Find the required Ni to asymptotically guarantee a desired P{CS} ‒ Conservative ‒ Simulation allocation is proportional to variance: Ni = ci i2
SMART SIMULATION ALLOCATION
99% Confidence Intervals for J(X) after some simulations
x1
•
x2
x3
x4
x5
Which designs should we simulate more? ‒ 2 & 3 are clearly superior ‒ 1, 4 & 5 have larger variances
•
Chen (1995) & Chen (1996) propose smarter allocations for efficiency
OPTIMAL COMPUTING BUDGET ALLOCATION (OCBA) • Maximize the Probability of Correctly Selecting the Best Design 𝐦𝐚𝐱
𝑁1 ,…,𝑁𝑠
s.t.
𝑷{𝐂𝐒} 𝑁1 + 𝑁2 + ⋯ + 𝑁𝑠 = 𝑇 (total number of runs)
• Asymptotically Optimal Solution Ni
(b, j / j)2
Nj
(b,i / i)2
Nb = b
ib (Ni2 / i2)
for i j b
S OME I NSIGHTS OF OCBA R ULE
Signal to Noise Ratio
c3
c2
1
2
1,2
3
1,3
1, 3 N2 3 2 N 3 1, 2 2 2
Designs
inversely proportional to the square of the signal to noise ratio
SELECTED GENERALIZATIONS & EXTENSIONS (1) • Non-normal Distributions - P. Glynn (Stanford Univ.) - S. Juneja (Columbia Univ.)
• Heavy-tailed Distributions - M. Broadie, M. Han, and A. Zeevi (Columbia University)
• Minimizing Variance Instead of Minimizing Mean - Lucy Pao & Lidija Trailovic (U. of Colorado)
• Correlated Sampling - Michael Fu (U. of Maryland at College Park) - J.Q. Hu & Y.J. Peng (Fudan Univ.)
• Finding both Simple and Good Designs - E. Zhou (Georgia Tech) - Q.S. Jia (Tsinghua University)
SELECTED GENERALIZATIONS & EXTENSIONS (2) • Multiple Objectives - L. Lee & E. Chew (National University of Singapore)
• Small Computing Budget - J. LaPorte (US Military Academy at West Point) - J. Branke (Warwick Univ.)
• Transient Simulation - D. J. Morrice (University of Texas at Austin)
• Expected opportunity cost instead of the probability of correct selection - S. Gao (City U of HK) - W. Chen (Rutgers Univ.)
- L. Shi (Peking Univ.)
• Optimal Subset Selection - S. Zhang (Shanghai University)
EFFICIENCY USING MULTI-FIDELITY MODELS Full Simulation Model
Simplified Model
Some examples: High-fidelity model Low fidelity model Discrete-event simulation Queueing theory Fine model Coarse model Capturing uncertainty Ignoring uncertainty
Complex
Much simpler
Good accuracy, but very time consuming
Biased, but fast
MULTI-FIDELITY TIME-SENSITIVE DATA Now Freshness of Data
Decision point
Fast Time Data from last hour Data from yesterday
Data from last week Data from last month
Time Availability to Decision Point
EXAMPLE: RESOURCE ALLOCATION PROBLEM • Flexible Manufacturing System • • • • •
2 product types 5 workstations Non-exponential service times Re-entrant manufacturing process Product 1 has higher priority than product 2
• Decision variable: number of machines allocated to each workstation 𝐌𝐢𝐧𝐢𝐦𝐢𝐳𝐞 Expected Total Processing Time 𝐒𝐮𝐛𝐣𝐞𝐜𝐭 𝐭𝐨 𝟓 ≤ # of machines at each workstation ≤ 𝟏𝟎 Total # of machines at all workstatiosn = 𝟑𝟖
• # of alternatives: 780
P1
P2
Workstation 1 Workstation 2 Workstation 3 Workstation 4 Workstation 5
FULL SIMULATION VS. QUEUEING APPROXIMATION
• Bias is non-homogeneous and can be quite large
ORDINAL TRANSFORMATION
High Fidelity
• •
Low Fidelity
Quickly evaluate each alternative using low-fidelity model Transform the decision space into an ordinal space
ORDINAL TRANSFORMATION
OT
BENEFITS
OT
– – – –
Non-smooth response can become much smoother Neighborhood connection is strengthened Designs with similar performance are grouped together Search/optimization efficiency is enhanced
SOME PROPERTIES • Theorem 1. Ordinal transformation can reduce the variability of each group by at least
100 𝟏 − ‒
𝟑 𝒎+𝟐
+
𝟔 𝒌𝟐
𝝆𝟐 %
is the correlation between original and ordinal models
• Theorem 2. The differences between the means of two neighboring groups can be increased by 100
𝟏𝟐𝒎 𝒌(𝒎+𝟏)
𝝆%
MACHINE ALLOCATION PROBLEM • 10 Groups (k=10)
MULTI-FIDELITY DATA AND MODELS • Full simulation (high-fidelity) New Demand
Product 1
Product 2
250
150
• Low-fidelity model: queueing approximation ‒
Very fast but poor approximation
New Demand
Product 1
Product 2
250
150
• Multi-fidelity data (with old simulations) ‒
Minimum additional cost but poor approximation Product 1
Product 2
Old Demand 1
210
170
Old Demand 2
280
140
OPTIMAL LINEAR COMBINATION OF TWO APPROXIMATION MODELS • Suppose we have 2 approximation models: g1 and g2 𝑔 𝑋 = 𝑎1 𝑔1 𝑋 + 𝑎2 𝑔2 𝑋
• To maximize the correlation with the true model 𝐶𝑜𝑣 𝑔 𝑋 , 𝑓 𝑋
𝑚𝑎𝑥𝑎1,𝑎2 𝜌 ≡
𝑉𝑎𝑟 𝑔 𝑋
𝑉𝑎𝑟 𝑓 𝑋
• Optimal weighting factors 𝑎1∗ 𝜌1 − 𝜌12 𝜌2 𝜎2 = 𝑎2∗ 𝜌2 − 𝜌12 𝜌1 𝜎1
** Joint work with Si Zhang
LEAD TO HIGH CORRELATION & ALIGNMENT • Ordinal Transformation using single model/data Rank Correlation
Alignment (top-5)
Old Demand 1 only
0.623
2
Old Demand 2 only
0.596
2
• Ordinal Transformation with intelligent combining use of two models
Old 1 + Old 2 Demand Models
Rank Correlation
Alignment (top-5)
0.875
5
DOES A SECOND MODEL ALWAYS HELP? • Consider a case where 1 = 0.6
12
2
helpful area, i.e., 12 < 0.2
If 2 = 0.6
SUMMARY •
Ordinal Optimization (Ho et al. 1992) 1. Focus on good enough solutions 2. Concentrate on relative order comparison only need to conduct a very small fraction of simulations
•
Optimal Computing Budget Allocation (OCBA) ‒ Further enhance OO efficiency via optimal control of simulation
•
Ordinal Transformation ‒ Utilize low-fidelity models/data to transform the decision space into a better space ‒ Enhance search efficiency
