Optimal Computing Budget Allocation for Simulation ... AWS

A Simulation Budget Allocation Procedure for Enhancing the Efficiency of Optimal Subset Selection Si Zhang Dept. of Management Science & Engineering School of Management Shanghai University

Outline 1. Introduction 2. Literature Review 3. Optimal Computing Budget Allocation for optimal subset selection 4. Convergence Rate Analysis

5. Conclusions and future research

2

Simulation & Optimization ➢ Optimization problems for complex system Manufacturing industry

Financial investment

Service industry

electronic circuit design

Portfolio selection

Spare parts inventory planning for airlines

Difficulties:

min f  X  X 

Feasible region

• How to evaluate?

 simulation

• How to find the best?  optimization Simulation Optimization 3

Challenges Multiple Simulation Runs (replications)

1 N Min f ( X )  E[ f ( X ,W )]   f (X ,Wi ) X  N i1 Many Alternatives in Design Space  Goal: maximize the overall efficiency  OCBA: Optimal Computing Budget Allocation

1. Introduction 1.2 Computing cost for simulation optimization New solutions to evaluate

Optimization Engine

Simulator

The performance of solutions

Noise

min f  X  X 

Best X 

1 n ˆ f ( X )   f (X , i ) n i 1

• n∞, 100% correct • n is finite, for each solution, make sure the correctness of selection maximal or at a high level (e.g. 95%)

100% Correct Time consuming

Good enough 5

1. Introduction 1.3 How to run simulation efficiently? 90% Confidence Interval

x1

x2 x3 x4

x5

Allocate 100 replications to this 5 solutions. How?

x1

x2 x3 x4

x5

x1

x2 x3 x4

x5

Equal simulation

Intelligent way

Each 20 replications

More important, more

How to efficiently allocate simulation replications budget?

Optimal Computing Budget Allocation (OCBA)

6

2. Literature Review 2.1 Literature review for Ranking and Selection (R&S) Ranking and Selection

determine the number of simulation replications in selecting the best solution(s) from a finite number of alternative solutions. Bechhofer et al. (1995), Swisher et al. (2003), Kim and Nelson (2006, 2007) Get

Computing replications

Max

P{CS}

Subject to P{CS}>=P*

Subject to Computing Budget

➢ Two-stage procedure Dudewicz and Dalal (1975), Rinott (1978)

➢ Optimal Computing Budget Allocation (OCBA) Chen et al. (1996), Chen et al. (1997), Chen et al. (2000), …

➢ Indifference-zone procedure (IZ) Kim and Nelson (2001), Nelson et al. (2001)

7

2. Literature Review 2.2 Literature review for OCBA ➢ Basic framework for OCBA Chen et al. (1996), Chen et al. (1997), Chen et al. (2000) ➢ • • • • •

Extension work Constraints: Pujowidianto et al. (2009) Multiple objectives: Chen and Lee (2009); Lee et al. (2010) Optimal subset selection: Chen et al. (2008) Correlation among solutions: Fu et al. (2004, 2007) Performance not normally distributed: Glynn and Juneja (2004)

➢ Application of OCBA • Problems given a fixed set of alternatives semiconductor wafer fab scheduling (Hsieh et al., 2001; Hsieh et al. 2007) • Problems with enormous size or continuous solution space Nested Partition (Shi et al. 1999); Cross Entropy (He et al. 2010). 8

In This Talk… 1. Introduction of the Optimal Computing Budget Allocation C. H. Chen and L. H. Lee, (2010). Stochastic Simulation Optimization: An Optimal Computing Budget Allocation. World Scientific Publishing.

2. Optimal Computing Budget Allocation for Optimal Subset Selection S. Zhang, J. Xu, L.H. Lee, E.P. Chew and C.H. Chen (2016). A Simulation Budget Allocation Procedure for Enhancing the Efficiency of Optimal Subset Selection, IEEE Transactions on Automatic Control，61（1）：62 ~ 75

9

Precision of Stochastic Simulation Estimator Confidence Interval (C.I.) 1 N f(X, w )  z j  j 1 N

 N

– z is the critical value for the standard normal distribution 99% Confidence Interval

Increase the number of simulation runs (N) N

Probability of Correct Selection: P{CS} 99% Confidence Intervals for f(X)

Increase N

x1

x2

x3

x4

x5

x1

x2

x3

As N increases • Simulation precision enhances • Confidence intervals become narrower • P{CS} increases

x4

x5

Smarter Simulation Allocation 99% Confidence Intervals for J(X) after some simulations

x1

x2

x3

x4

x5

• Which designs should we simulate more? – 2 & 3 are clearly superior – 1, 4 & 5 have larger variances

• When is the optimal screening point?

Optimal Computing Budget Allocation (OCBA) Selecting the best (P1) Minimize the total number of simulation runs in order to achieve a desired simulation quality:

min N1 ,.., N k

[ N1 + N2 + .. + Nk ] s.t. P{CS} > Psat (a satisfactory level)

(P2) Maximize the simulation quality with a given simulation budget:

max P{CS}

N1,.., N k

s.t. N1 + N2 + .. + Nk = T (total comp. budget)

In This Talk… 1. Introduction of the Optimal Computing Budget Allocation C. H. Chen and L. H. Lee, (2010). Stochastic Simulation Optimization: An Optimal Computing Budget Allocation. World Scientific Publishing.

2. Optimal Computing Budget Allocation for optimal subset selection S. Zhang, J. Xu, L.H. Lee, E.P. Chew and C.H. Chen (2016). A Simulation Budget Allocation Procedure for Enhancing the Efficiency of Optimal Subset Selection, IEEE Transactions on Automatic Control，61（1）：62 ~ 75

14

Problem Motivation Alternatives

Best

A, B, C, D, E, …

C Simulator

 a more flexible and people oriented way

 screen procedure

D1, D2, D3,…, D100

not enough!

 population based searching algorithms Get certain solution(s)

Rough Experiment Good designs: D4, D20, D46, D64, D82 How about qualitative criteria and political feasibility ?

Accurate Experiment

Best design: D64

Evaluate the performance of these solution(s) Select an elite subset and determine the search direction Move to the new generated solution(s) 15

Problem Formulation for optimal subset selection k alternatives 1, 2, …, k



N i ,  i2

Top-m Simulation



Total computing budget: T Replications allocated

N i   iT

max P CS 

N1 , N 2 ,, N k

k

s.t.

N i 1

i

How to formulate it?

T

How to get the optimal allocation rule?

16

Expression of P{CS} The optimal subset:

The ordering of means:

1, 2,

1  2  ...  k Xm

X1 X 2

  P CS   P   

X i

i



 X j , for i  1, 2,

X m 1 X m  2

, m Xk

, m and j  m  1, m  2,

j

  ,k  

▪P{CS} does not have close form expression ▪Use lower bounds to approximate its true value, which are called the Approximated Probability of Correct Selection (APCS). 7/28

Expression of P{CS} X i1 X i2



X im2 X im1 X im

X ik 1 X ik

X im1 X im2 X im3

Boundaries Use X im as a threshold X i1 X i2

X im2 X im1



X im



X ik 1 X ik

X im1 X im2 X im3

One lower bound of P CS  m 1



  PX

APCSm1  1   P X ip  X im  p 1

k

q  m 1

im

 X iq

 9/28

Expression of P{CS} Use X im1as a threshold X i1 X i2

X im2 X im1 X im

 Xi  m1

X ik 1 X ik

X im2 X im3

Another lower bound of P CS  m



  PX

APCSm2  1   P X ip  X im1  p 1

k

q m 2

im1

 X iq



Lemma 1.

P{CS}  APCSm  max  APCSm1 , APCSm2 

10/28

Problem Formulation for optimal subset selection Sub-problem 1

m 1

max APCSm1  1   P  X i  X m  

➢ OCBA model

1 , 2 ,, k

i 1

k

max APCSm 

s.t.

i 1

N1 , N 2 ,, N k

k

s.t.

N i 1

i

T

Ni  0



i

 PX k

j  m 1

m

 X j

1

 i  0, for i  1, 2,

, k.

Sub-problem 2 max APCSm2  1   P  X i  X m 1  m

1 , 2 ,, k

i 1

k

s.t.

 i 1

i

 PX k

j m 2

m 1

 X j

1

 i  0, for i  1, 2,

, k.

Lemma 2. There exists a large enough T * such that both subproblems 1 and 2 are convex with respect to the vector  >0 when T  T * . 20

OCBA for optimal subset selection 

3 Noise to signal ratio

Square root rule

21

OCBA for optimal subset selection 1 The asymptotically optimal allocation rule, named as OCBAm+, Theorem 3.1. to maximize APCSm+ is    ,  , *

* 1

,

* 2

* k



 *1   *2   

    APCSm    APCSm  

if APCSm1  *1  APCSm2  *2 if

*1

1

*2

2

Corollary 1. If m equals to one, the allocation rule OCBAm+, will be the OCBA1 rule expressed as follows.  i  i  1 N1   1

k

N i2

 i 2

2 i

2

,

Ni  i   i  i     N j  j   j  j  for i, j  1

22

Asymptotic Convergence Rate Analysis • How does an allocation rule perform? • How do we compare the performance of different allocation rules?

Numerical experiments

?

Theoretical framework

❖ When T goes to infinity, P{CS} convergences to 1 and P{IS} convergences to 0. ❖Given an allocation rule, we can calculate the convergence rate of P{CS} (or P{IS}) as a measure to characterize the performance of this allocation rule. ❖The allocation rule with higher convergence rate is better than rules with lower convergence rates.

16/28

Asymptotic Convergence Rate Analysis •

Suppose for k designs, we have 1  2  S1  1, 2,

S2  m  1, m  2,

, m



 m  m1 

 k 1  k

, k



P IS   P max X i ( iT )  min X j ( jT )

•

iS1

jS2

A lower bound and an upper bound of P IS max P  X i ( iT )  X j ( jT )  P IS   S1  S 2 max P  X i ( iT )  X j ( jT )

iS1 , jS2

•

If we have

iS1 , jS2

1 log P  X i ( iT )  X j ( jT )  Gij  i ,  j  , T  T lim

•

Then

1 log P IS    min Gij  i ,  j  T  T iS1 , jS 2 lim

i  S1 , j  S 2

Convergence rate

17/28

Asymptotic Convergence Rate Analysis • Large deviation theory Dembo, A. and Zeitouni, O. (1992); Peter Glynn and Sandeep Juneja (2004)

Gij  i ,  j  



    i

2

j

2  i2  i   2j  j



• The convergence rate for equal allocation rule (EA) under equal variance case  12   22 

  k2   2

EA rule:  i  1 k Gij  i ,  j  

    i

2

j

4k 2

min Gij  i ,  j   Gm m1  m ,  m1 

iS1 , jS2

   m1   m

2

4k 2

19/28

Asymptotic Convergence Rate Analysis Lemma 5(a) The asymptotic convergence rate obtained by OCBAm in equal variance case





min Gmk  m ,  k  , G1 m1 1 ,   m1



2   m  k 2  1  m1    min  2 , 2  2  1   1  2 1 1  1  m1     m k  

Lemma 5(b) The asymptotic convergence rate obtained by OCBAm+ in equal variance case





Gm m 1  mL ,  mL 1 

 m  m1 



2

2 2 1  mL  1  mL 1



20/28

Asymptotic Convergence Rate Analysis If the means of all designs form an arithmetical progression, and the variances of all populations are equal, that is     d , for i  1, 2, , k 1 and         i 1

i

2 1

2 2

2 k

2

Theorem 2. The asymptotic convergence rates gained by OCBAm+ and OCBAm are greater than the rate gained by equal allocation rule. Theorem 3. The asymptotic convergence rate gained by OCBAm+ is always no less than OCBAm when m=1 or m  2 and k  m  5 Conclusions: 1.OCBAm+ and OCBAm are better than the equal allocation rule. 2.OCBAm+ is better than OCBAm in most situations. 21/28

Numerical Experiment for OCBAm+ ➢ Base Experiment: k=50; m=5; Solution i ~ N(i , 102) 1 0.95 0.9 0.8

8800

3750

0.75

EA

0.7

OCBAm+

0.65 0.6 0.55

0.5

3000 3200 3400 3600 3800 4000 4200 4400 4600 4800 5000 5200 5400 5600 5800 6000 6200 6400 6600 6800 7000 7200 7400 7600 7800 8000 8200 8400 8600 8800 9000 9200 9400 9600 9800 10000

P{CS}

0.85

T

Rule

EA

Convergence Rate

0.50×10-4

OCBAm+