An Improved Dynamic Programming ... - Semantic Scholar
An Improved Dynamic Programming Decomposition Approach for Network Revenue Management
Dan Zhang Leeds School of Business University of Colorado at Boulder
May 21, 2012
www.danzhang.com
Improved DP Decomposition
Outline
Background Network revenue management formulation Classical dynamic programming decomposition An improved dynamic programming decomposition Numerical results Summary
www.danzhang.com
Improved DP Decomposition
Network RM Formulation (Gallego and van Ryzin, 1997; Gallego et. al., 2004; Liu and van Ryzin, 2008) m resources with capacity c (an m-vector) Capacity for resource i is ci .
n products N = {1, . . . , n} Fare for product j is fj Product consumption matrix A = [aij ]
Finite time horizon with length τ In each period, there is one customer arrival with probability λ, and no customer arrival with probability 1 − λ. Given a set of products S ⊆ N, a customer chooses product j with probability Pj (S). No-purchase probability P0 (S) = 1 −
P
j∈S
Pj (S).
Objective: Maximize expected total revenue www.danzhang.com
Improved DP Decomposition
Applications
Industry Airlines Hotels Car rentals Air Cargo
Resources Scheduled flights Room-days Car-days Scheduled flights (weight) Scheduled flights (volume) www.danzhang.com
Products O-D itineraries at certain fare levels Single(multi)-day stays at certain rates Single(multi)-day rentals at certain rates O-D shipments at certain rates
Improved DP Decomposition
Dynamic Programming Formulation DP optimality equations: n X vt (x) = max λ Pj (S)(fj + vt+1 (x − Aj )) S⊆N(x)
j∈S
o + (λP0 (S) + 1 − λ)vt+1 (x) , vτ +1 (x) = 0,
∀x.
Notations vt (x): DP value function Aj : resource incidence vector of product j N(x): {j ∈ N : x ≥ Aj }
www.danzhang.com
Improved DP Decomposition
∀t, x,
Dynamic Programming Formulation DP optimality equations: n X vt (x) = max λ Pj (S)(fj + vt+1 (x − Aj )) S⊆N(x)
j∈S
o + (λP0 (S) + 1 − λ)vt+1 (x) , vτ +1 (x) = 0,
∀t, x,
∀x.
Notations vt (x): DP value function Aj : resource incidence vector of product j N(x): {j ∈ N : x ≥ Aj }
Curse of dimensionality: state space grows exponentially with the number of resources
www.danzhang.com
Improved DP Decomposition
Choice-based Deterministic Linear Program (CDLP) zCDLP = max h
X
X
λR(S)h(S)
S⊆N
λQ(S)h(S) ≤ c,
(Resource constraint)
h(S) ≤ τ,
(Time constraint)
S⊆N
X S⊆N
h(S) ≥ 0,
∀S ⊆ N.
(Non-negativity)
Replace stochastic demand with deterministic fluid with rate λ Given offer set S ⊆ N Total time S is offered: h(S) P Revenue from unit demand: R(S) = j∈S fj Pj (S) Consumption P of resource i from unit demand: Qi (S) = j∈S aij Pj (S) www.danzhang.com
Improved DP Decomposition
CDLP (Gallego et. al, 2004; Liu and van Ryzin, 2008)
CDLP can by efficiently solved for certain class of choice models. The vector of dual values π ∗ associated with resource constraints can be used as “bid-prices” for resources zCDLP provides an upper bound on revenue Some recent references: Talluri (2010): Concave programming formulation Gallego, Ratliff, Shebalov (2011): Efficient reformulation
www.danzhang.com
Improved DP Decomposition
Classical Dynamic Programming Decomposition For each i, approximate the DP value function with X ∗ + πk xk , vt (x) ≈ vt,i (xi ) | {z } k6=i | {z } Value of the Value of all i-th resource other resources
www.danzhang.com
Improved DP Decomposition
∀t, x.
Classical Dynamic Programming Decomposition For each i, approximate the DP value function with X ∗ + πk xk , vt (x) ≈ vt,i (xi ) | {z } k6=i | {z } Value of the Value of all i-th resource other resources
∀t, x.
Using the approximation in DP recursion leads to X X vt,i (xi ) = max λPj (S) fj − akj πk∗ +vt+1,i (xi − aij ) S⊆N(xi ,c−i )
j∈S
k6=i
| {z } Fare proration + (λP0 (S) + 1 − λ)vt+1 (xi ),
www.danzhang.com
∀t, xi .
Improved DP Decomposition
Classical Dynamic Programming Decomposition For each i, approximate the DP value function with X ∗ + πk xk , vt (x) ≈ vt,i (xi ) | {z } k6=i | {z } Value of the Value of all i-th resource other resources
∀t, x.
Using the approximation in DP recursion leads to X X vt,i (xi ) = max λPj (S) fj − akj πk∗ +vt+1,i (xi − aij ) S⊆N(xi ,c−i )
j∈S
k6=i
| {z } Fare proration + (λP0 (S) + 1 − λ)vt+1 (xi ),
∀t, xi .
Compute offer sets dynamically using the approximate value function X vt (x) ≈ vt,i (xi ), ∀t, i. i
www.danzhang.com
Improved DP Decomposition
Classical Dynamic Programming Decomposition A DP with m-dimensional state space is reduced to m one-dimensional DPs, one for each resource.
1014 states (Assume 100 seats per flight)
www.danzhang.com
⇒
4 × 101 states
Improved DP Decomposition
Classical Dynamic Programming Decomposition A DP with m-dimensional state space is reduced to m one-dimensional DPs, one for each resource.
1014 states (Assume 100 seats per flight)
⇒
4 × 101 states
Variants of the approach are widely used in practice. Review: Talluri and van Ryzin (2004a) www.danzhang.com
Improved DP Decomposition
DP Decomposition Bounds
Proposition (Zhang and Adelman, 2009) The following relationships hold: n o P (i) vt (x) ≤ minl=1,...,m vt,l (xl ) + k6=l πk∗ xk ≤ P vt,i (xi ) + k6=i πk∗ xk , ∀i, t, x; P (ii) v1 (c) ≤ v1,i (ci ) + k6=i πk∗ ck ≤ zCDLP , ∀i.
Decomposition value for each leg provides an upper bound on revenue Decomposition bounds are tighter than the bound from CDLP
www.danzhang.com
Improved DP Decomposition
Linear Programming Formulation of DP (Adelman, 2007)
min v1 (c) X vt (x) ≥ λPj (S)(fj + vt+1 (x − Aj )) {vt (·)}∀t
j∈S
+ (λP0 (S) + 1 − λ)vt+1 (x),
www.danzhang.com
∀t, x, S ⊆ N(x).
Improved DP Decomposition
Linear Programming Formulation of DP (Adelman, 2007)
min v1 (c) X vt (x) ≥ λPj (S)(fj + vt+1 (x − Aj )) {vt (·)}∀t
j∈S
+ (λP0 (S) + 1 − λ)vt+1 (x),
∀t, x, S ⊆ N(x).
Huge number of decision variables and constraints
www.danzhang.com
Improved DP Decomposition
Linear Programming Formulation of DP (Adelman, 2007)
min v1 (c) X vt (x) ≥ λPj (S)(fj + vt+1 (x − Aj )) {vt (·)}∀t
j∈S
+ (λP0 (S) + 1 − λ)vt+1 (x),
∀t, x, S ⊆ N(x).
Huge number of decision variables and constraints Functional approximation idea: use a parameterized representation of the value function to reduce the number of decision variables
www.danzhang.com
Improved DP Decomposition
The Affine Functional Approximation (Zhang and Adelman, 2009)
Affine approximation is given by vt (x) ≈ θt +
X
Vt,i xi ,
∀t, x.
i
www.danzhang.com
Improved DP Decomposition
(1)
The Affine Functional Approximation (Zhang and Adelman, 2009)
Affine approximation is given by vt (x) ≈ θt +
X
∀t, x.
Vt,i xi ,
(1)
i
Using (1) in the linear programming formulation leads to min θ1 + θ,V
X
V1,i ci
i
! θt +
X i
Vt,i xi ≥
X
λPj (S) fj + θt+1 +
X
Vt+1,i (xi − aij )
i
j∈S
! + (λP0 (S) + 1 − λ) θt+1 +
X
Vt+1,i xi
,
∀t, x, S ⊆ N(x).
i
www.danzhang.com
Improved DP Decomposition
The Affine Functional Approximation
The dual program is given by zP1 = max Y
X
X
λPj (S)fj Yt,x,S
j∈S
t,x,S⊆N(x)
( X
xi Yt,x,S =
P
x,S⊆N(x)
x,S⊆N(x)
X x,S⊆N(x)
Yt,x,S =
ci , P xi − j∈S λPj (S)aij Yt−1,x,S ,
1, P
x,S⊆N(x)
Yt−1,x,S ,
if t = 1, ∀t = 2, . . . , τ
if t = 1, ∀t = 2, . . . , τ.
Y ≥ 0.
www.danzhang.com
Improved DP Decomposition
∀i, t,
The Affine Functional Approximation
The dual program is given by zP1 = max Y
X
X
λPj (S)fj Yt,x,S
j∈S
t,x,S⊆N(x)
( X
xi Yt,x,S =
P
x,S⊆N(x)
x,S⊆N(x)
X x,S⊆N(x)
Yt,x,S =
ci , P xi − j∈S λPj (S)aij Yt−1,x,S ,
1, P
x,S⊆N(x)
Yt−1,x,S ,
if t = 1, ∀t = 2, . . . , τ
∀i, t,
if t = 1, ∀t = 2, . . . , τ.
Y ≥ 0.
Due to the large number of columns, solving the linear program above still requires considerable computational effort.
www.danzhang.com
Improved DP Decomposition
Functional Approximation Approaches for Network RM
Citation
Choice Model
Functional approximation
Solution strategy
Adelman (2007) Zhang and Adelman (2009) Zhang (2011) Liu and van Ryzin (2008) Miranda Bront et. al. (2009) Farias and Van Roy (2008) Meissner and Strauss (2012) Kunnumkal and Topaloglu (2011) Tong and Topaloglu (2011)
Independent demand MNLD MNLD MNLD MNLO Independent demand MNLD MNLD
Affine Affine Nonlinear Separable Separable Separable Separable Separable
Independent demand
Affine
Vossen and Zhang
Independent demand + MNLD
Affine
Column generation Column generation CDLP+Simultaneous DP CDLP+DP Decomposition CDLP+DP Decomposition Constraint sampling Column generation Convex programming +DP Decomposition Reduction + Constraint generation Reduction + Dynamic disaggregation
non-separable (fare proration) (fare proration) concave concave (fare proration)
MNLD: Multinomial logit model with disjoint consideration sets MNLO: Multinomial logit model with overlapping consideration sets
www.danzhang.com
Improved DP Decomposition
Research Questions
Computational cost: ADP (affine or separable concave approximation) classical DP decomposition
www.danzhang.com
Improved DP Decomposition
Research Questions
Computational cost: ADP (affine or separable concave approximation) classical DP decomposition How can we balance solution quality with solution time? Can we improve the classical DP decomposition?
www.danzhang.com
Improved DP Decomposition
A Strong Functional Approximation (Zhang, 2011)
X πk∗ xk , vt (x) ≈ min vˆt,i (xi ) + i
∀t, x.
k6=i
Nonlinear and non-separable functional approximation Each value vt (x) is approximated by a single value across legs Motivated by the decomposition bounds (Zhang and Adelman, 2009)
www.danzhang.com
Improved DP Decomposition
A Nonlinear Optimization Problem Using the new functional approximation leads to zNLP =
min min vˆt,i (·)∀t,i
i
vˆ1,i (ci ) +
X k6=i
πk∗ ck
X ∗ min vˆt,i (xi ) + πk xk ≥ i k6=i X X ∗ λPj (S) fj + min vˆt+1,l (xl − alj ) + πk (xk − akj ) l j∈S k6=l X ∗ + (λP0 (S) + 1 − λ)min vˆt+1,l (xl ) + πk xk , ∀t, x, S ⊆ N(x). l k6=l
The problem is a nonlinear optimization problem with a huge number of nonlinear constraints. www.danzhang.com
Improved DP Decomposition
A Restricted Optimization Problem
Step 1: Writing each constraint as m equivalent constraints Step 2: Restricting the constraints so that each constraint only involves one resource The restricted problem provides a relaxed bound: Proposition The objective value of the restricted program, zNLP d , is bigger than zNLP .
www.danzhang.com
Improved DP Decomposition
An Equivalent Simultaneous Dynamic Program ( ∗ vˆt,i (xi )
=
X
max S⊆N(xi ,c−i )
∗ λPj (S) fj + min vˆt+1,i (xi − aij ) −
j∈S
l6=i
max
πk∗ akj ,
k6=i
min
X
0≤yl ≤cl −alj
∗ [ˆ vt+1,l (yl )
−
yl πl∗ ]
−
X
akj πk∗
+
πi∗ xi
)!
k
∗ ∗ vt+1,l (yl ) − πl∗ yl ] + πi∗ xi + (λP0 (S) + 1 − λ) min vˆt+1,i (xi ), min max [ˆ l6=i
0≤yl ≤cl
∀i, t, xi .
DP recursion for resource i involves values from all other resources
www.danzhang.com
Improved DP Decomposition
An Equivalent Simultaneous Dynamic Program ( ∗ vˆt,i (xi )
=
X
max S⊆N(xi ,c−i )
∗ λPj (S) fj + min vˆt+1,i (xi − aij ) −
j∈S
l6=i
max
πk∗ akj ,
k6=i
min
X
0≤yl ≤cl −alj
∗ [ˆ vt+1,l (yl )
−
yl πl∗ ]
−
X
akj πk∗
+
πi∗ xi
)!
k
∗ ∗ vt+1,l (yl ) − πl∗ yl ] + πi∗ xi + (λP0 (S) + 1 − λ) min vˆt+1,i (xi ), min max [ˆ l6=i
0≤yl ≤cl
∀i, t, xi .
DP recursion for resource i involves values from all other resources The dynamic program is equivalent to the restricted nonlinear program can be solved efficiently via a simultaneous dynamic programming algorithm leads to tighter revenue bounds www.danzhang.com
Improved DP Decomposition
New Bounds
Proposition (Zhang, 2011) ∗ (·)} Let {ˆ vt,i ∀t,i,xi be the optimal solution from the simultaneous dynamic program. The following results hold: ∗ (x ) ≤ v (x ), ∀i, x ; (i) vˆt,i i t,i i i
n o P ∗ (c ) + ∗c (ii) v1 (c) ≤ zNLP ≤ zNLP = min v ˆ π i i d k6=i k k ≤ n o 1,i P mini v1,i (ci ) + k6=i πk∗ ck ≤ zCDLP . The simultaneous dynamic program provides tighter bounds on revenue than the classical decomposition.
www.danzhang.com
Improved DP Decomposition
Recap
High dimensional dynamic program
www.danzhang.com
Improved DP Decomposition
Recap
High dimensional dynamic program m Large scale linear program
www.danzhang.com
Improved DP Decomposition
Recap
High dimensional dynamic program m Large scale linear program ⇓ Large scale nonlinear program with nonlinear constraints
www.danzhang.com
Improved DP Decomposition
Recap
High dimensional dynamic program m Large scale linear program ⇓ Large scale nonlinear program with nonlinear constraints ⇓ Restricted nonlinear program with nonlinear constraints
www.danzhang.com
Improved DP Decomposition
Recap
High dimensional dynamic program m Large scale linear program ⇓ Large scale nonlinear program with nonlinear constraints ⇓ Restricted nonlinear program with nonlinear constraints m Simultaneous dynamic program
www.danzhang.com
Improved DP Decomposition
Comparison: Classical vs. Improved Approaches Classical dynamic programming decomposition: Solve m single-leg DPs Prorated fares
www.danzhang.com
Fare proration
Static bid-prices
Solve CDLP
Improved DP Decomposition
Comparison: Classical vs. Improved Approaches Classical dynamic programming decomposition: Solve m single-leg DPs Prorated fares
Fare proration
Static bid-prices
Solve CDLP
Network effects only captured through fare proration
www.danzhang.com
Improved DP Decomposition
Comparison: Classical vs. Improved Approaches Classical dynamic programming decomposition: Solve m single-leg DPs Prorated fares
Fare proration
Static bid-prices
Solve CDLP
Network effects only captured through fare proration Improved dynamic programming decomposition: Solve one simultaneous DP Static bid-prices
www.danzhang.com
Solve CDLP
Improved DP Decomposition
Comparison: Classical vs. Improved Approaches Classical dynamic programming decomposition: Solve m single-leg DPs Prorated fares
Fare proration
Static bid-prices
Solve CDLP
Network effects only captured through fare proration Improved dynamic programming decomposition: Solve one simultaneous DP Static bid-prices
Solve CDLP
Network effects captured during DP recursion! www.danzhang.com
Improved DP Decomposition
Computational Study: Problem Instances
Randomly generated hub-and-spoke instances Number of non-hub locations (flights) in the set {4, 8, 16, 24} Number of periods in the set {100, 200, 400, 800} Two products for each possible itinerary Multinomial Logit Choice Model with Disjoint Consideration Sets (MNLD) Largest problem instance: 24 non-hub locations (flights), 336 products, 800 periods
www.danzhang.com
Improved DP Decomposition
Numerical Study: Policies
DCOMP1: the new decomposition approach where the approximation m X ∗ vt (x) ≈ vˆt,i (xi ), ∀t, x i=1
is used to compute control policies. DCOMP: the classical dynamic programming decomposition CDLP: static bid-price policy based on the dual values of resource constraints in CDLP CDLP10: A version of CDLP that resolves 10 times with equally spaced resolving intervals Each policy is simulated 20000 times
www.danzhang.com
Improved DP Decomposition
Computational Time
Case # A1 A2 A3 A4 A5 A6 A7 A8 A9 A10 A11 A12 A13 A14 A15 A16
Parameters (100,4,4,16) (200,4,4,16) (400,4,4,16) (800,4,4,16) (100,8,8,48) (200,8,8,48) (400,8,8,48) (800,8,8,48) (100,16,16,160) (200,16,16,160) (400,16,16,160) (800,16,16,160) (100,24,24,336) (200,24,24,336) (400,24,24,336) (800,24,24,336)
Capacity
Load
per leg
factor
CDLP
DCOMP
DCOMP1
1.17 1.27 1.19 1.28 1.43 1.36 1.35 1.21 1.65 1.45 1.29 1.40 1.45 1.35 1.29 1.38
0.16 0.23 0.16 0.20 1.52 0.72 1.61 0.72 4.64 4.69 2.92 3.64 3.69 4.59 4.39 4.23
2.03 7.89 31.92 127.66 5.75 22.83 91.67 362.84 15.09 75.84 303.66 1218.67 24.72 98.52 492.73 1978.14
2.75 10.88 43.73 174.48 7.47 29.58 118.92 471.73 19.42 96.97 388.97 1560.19 31.81 127.36 630.84 2532.20
10 20 40 80 5 10 20 40 2 5 10 20 1 2 5 10
www.danzhang.com
CPU seconds
DCOMP1−DCOMP DCOMP
Improved DP Decomposition
35.38% 37.82% 37.00% 36.68% 29.89% 29.57% 29.73% 30.01% 28.67% 27.85% 28.10% 28.02% 28.70% 29.28% 28.03% 28.01%
Bound Performance
Case # A1 A2 A3 A4 A5 A6 A7 A8 A9 A10 A11 A12 A13 A14 A15 A16
Bound improvement
%-difference across legs
CDLP bound
DCOMP bound
DCOMP1 bound
%-CDLP
%-DCOMP
DCOMP
DCOMP1
24078.90 48367.58 89312.44 213102.50 32521.30 70541.63 107831.01 216080.83 26347.76 60629.35 101616.47 224780.69 13074.04 26296.19 74112.13 131457.79
23985.56 48328.43 87576.49 211854.85 31029.90 68760.67 106339.36 212915.61 24953.24 58489.12 100069.27 222558.53 11845.73 24926.41 72089.14 129589.28
22900.49 47588.56 86729.90 211087.37 30726.17 68617.41 106153.32 212848.05 24764.75 58118.33 99771.63 222231.72 10386.38 24373.33 71617.55 129273.91
5.15% 1.64% 2.98% 0.95% 5.84% 2.80% 1.58% 1.52% 6.39% 4.32% 1.85% 1.15% 25.88% 7.89% 3.48% 1.69%
4.74% 1.55% 0.98% 0.36% 0.99% 0.21% 0.18% 0.03% 0.76% 0.64% 0.30% 0.15% 14.05% 2.27% 0.66% 0.24%
4.46% 1.87% 2.36% 0.58% 3.18% 2.18% 1.09% 1.49% 4.69% 2.95% 1.52% 0.94% 10.37% 5.50% 2.80% 1.44%
0.00% 0.36% 0.00% 0.00% 0.05% 0.22% 0.00% 0.00% 0.00% 0.03% 0.01% 0.00% 0.00% 0.00% 0.03% 0.00%
www.danzhang.com
Improved DP Decomposition
Bounds from Individual Legs
Bounds from individual Legs s from individual Legs dividual Legs al Legs
132000 131500 131000 130500 130000
DCOMP
129500
DCOMP1
129000 128500 128000 1
2
3
4
5
6
7
8
9
10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Leg
www.danzhang.com
Improved DP Decomposition
Bounds from Individual Legs
Bounds from individual Legs s from individual Legs dividual Legs al Legs
132000 131500 131000 130500 130000
DCOMP
129500
DCOMP1
129000 128500 128000 1
2
3
4
5
6
7
8
9
10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Leg
DCOMP1 bounds are more homogeneous across legs
www.danzhang.com
Improved DP Decomposition
A Hub-and-spoke Network with 2 Non-Hub Locations
Case # B1 B2 B3 B4 B5 B6 B7 B8 B9 B10 B11 B12 B13 B14 B15 B16 B17 B18 B19 B20
τ 100 200 400 800 100 200 400 800 100 200 400 800 100 200 400 800 100 200 400 800
DCOMP1 Revenue Gains
Load factor
Capacity per leg
DCOMP1 REV
OPT-GAP
2.40 2.13 2.13 2.13 1.60 1.60 1.60 1.60 1.37 1.28 1.28 1.28 1.07 1.07 1.07 1.07 0.96 0.91 0.91 0.91
4 9 18 36 6 12 24 48 7 15 30 60 9 18 36 72 10 21 42 84
5775.47 13262.92 25456.41 53946.59 8034.27 17318.40 35472.06 65618.95 9269.56 20521.01 42471.91 86841.58 11107.51 23268.75 47824.97 96993.79 11854.02 25259.70 51593.10 104376.37
-3.17% -2.04% -6.53% -1.20% -7.25% -2.31% -1.25% -9.20% -5.85% -3.70% -2.10% -1.15% -1.68% -0.80% -0.27% -0.08% -0.85% -0.07% 0.03% 0.01%
www.danzhang.com
%-CDLP
%-CDLP10
%-DCOMP
182.90% 209.79% -1.00% 212.87% 48.88% 13.42% 311.96% 42.16% 13.27% 15.05% 15.46% 15.47% 2.89% 4.84% 22.11% 7.92% 23.82% 2.61% 30.16% 31.46%
8.56% 5.74% -0.33% 4.41% 3.00% 3.62% 4.78% -4.49% 1.33% 0.98% 1.70% 2.45% 0.47% 1.14% 2.22% 2.14% 1.46% 1.24% 2.48% 2.60%
2.74% 4.11% 0.03% 7.17% 0.09% 5.77% 7.97% 0.00% 1.94% 4.65% 8.14% 5.52% 0.34% 0.11% 0.04% 0.01% 0.18% 0.04% 0.01% 0.00%
Improved DP Decomposition
DCOMP1 Percentage Revenue Gain vs. Load Factor
DCOMP1 percentage revenue gain
10 %−CDLP10 %−DCOMP
5
0
−5 0.5
1
1.5 Load factor
www.danzhang.com
2
Improved DP Decomposition
2.5
DCOMP1 Percentage Revenue Gain vs. Load Factor
DCOMP1 percentage revenue gain
10 %−CDLP10 %−DCOMP
5
0
−5 0.5
1
1.5 Load factor
2
2.5
Higher load factor ⇒ Higher revenue gains www.danzhang.com
Improved DP Decomposition
DCOMP1 Percentage Revenue Gain vs. Number of Periods
DCOMP1 percentage revenue gain
10 %−CDLP10 %−DCOMP
5
0
−5 0
100
200
300 400 500 600 Number of periods
www.danzhang.com
700
800
Improved DP Decomposition
900
DCOMP1 Percentage Revenue Gain vs. Number of Periods
DCOMP1 percentage revenue gain
10 %−CDLP10 %−DCOMP
5
0
−5 0
100
200
300 400 500 600 Number of periods
700
800
900
Significant revenue gains for problems with long selling horizons! www.danzhang.com
Improved DP Decomposition
A Hub-and-spoke Network with 4 Non-Hub Locations
Case # C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11 C12 C13 C14 C15 C16 C17 C18 C19 C20
τ 100 200 400 800 100 200 400 800 100 200 400 800 100 200 400 800 100 200 400 800
DCOMP1 Revenue Gains
Load factor
Capacity per leg
DCOMP1 REV
OPT-GAP
1.99 1.99 1.99 1.99 1.49 1.49 1.49 1.49 1.19 1.19 1.19 1.19 1.00 1.00 1.00 1.00 0.85 0.85 0.85 0.85
6 12 24 48 8 16 32 64 10 20 40 80 12 24 48 96 14 28 56 112
16795.66 35028.96 70163.84 143921.85 21860.34 45171.83 88532.95 184410.85 26270.03 54509.68 111520.14 226059.91 29208.18 61175.75 125854.79 256236.11 32057.27 66527.87 135897.71 274817.89
-5.08% -2.63% -3.35% -1.34% -4.23% -2.54% -5.29% -1.79% -5.00% -3.02% -1.79% -1.05% -5.00% -3.02% -1.79% -1.00% -2.55% -1.28% -0.51% -0.17%
www.danzhang.com
%-CDLP
%-CDLP10
%-DCOMP
14.60% 52.79% 12.35% 52.31% 44.57% 14.18% 29.01% 1.02% 4.98% 4.18% 3.44% 57.07% 1.30% 2.18% 6.38% 2.66% 2.50% 3.05% 3.24% 3.21%
0.46% 1.95% -0.11% 1.36% 1.71% 1.72% -2.27% 0.49% 1.46% 1.61% 1.51% 1.92% -0.90% 0.21% 0.74% 0.89% -0.81% -0.03% 0.26% 0.41%
0.10% 1.32% 0.03% 0.66% 1.96% 2.67% -0.81% 0.18% 2.67% 4.11% 4.43% 3.08% 0.21% 0.08% 0.00% -0.06% 0.44% 0.20% 0.05% 0.02%
Improved DP Decomposition
DCOMP1 Percentage Revenue Gain vs. Load Factor
DCOMP1 percentage revenue gain
5 %−CDLP10 %−DCOMP
4 3 2 1 0 −1 −2 −3 0.5
1
1.5 Load factor
www.danzhang.com
2
Improved DP Decomposition
2.5
DCOMP1 Percentage Revenue Gain vs. Number of Periods
DCOMP1 percentage revenue gain
5 %−CDLP10 %−DCOMP
4 3 2 1 0 −1 −2 −3 0
100
200
300 400 500 600 Number of periods
www.danzhang.com
700
800
Improved DP Decomposition
900
Summary and Future Directions Functional approximation approach is promising for solving large scale stochastic dynamic programs. However, implementations of the approach often require very high computational cost.
The first nonlinear non-separable functional approximation for network RM problem Novel approximation architecture Better revenue bounds Improved heuristic policies Moderate computational cost
Current work Exploiting special structures of the LP formulations of dynamic programs in value function approximation (Vossen and Zhang, 2012) Applications with real data (Zhang and Weatherford, 2012)
www.danzhang.com
Improved DP Decomposition
Thank you!
Questions? Comments?
www.danzhang.com
Improved DP Decomposition
Dan Zhang Leeds School of Business University of Colorado at Boulder
May 21, 2012
www.danzhang.com
Improved DP Decomposition
Outline
Background Network revenue management formulation Classical dynamic programming decomposition An improved dynamic programming decomposition Numerical results Summary
www.danzhang.com
Improved DP Decomposition
Network RM Formulation (Gallego and van Ryzin, 1997; Gallego et. al., 2004; Liu and van Ryzin, 2008) m resources with capacity c (an m-vector) Capacity for resource i is ci .
n products N = {1, . . . , n} Fare for product j is fj Product consumption matrix A = [aij ]
Finite time horizon with length τ In each period, there is one customer arrival with probability λ, and no customer arrival with probability 1 − λ. Given a set of products S ⊆ N, a customer chooses product j with probability Pj (S). No-purchase probability P0 (S) = 1 −
P
j∈S
Pj (S).
Objective: Maximize expected total revenue www.danzhang.com
Improved DP Decomposition
Applications
Industry Airlines Hotels Car rentals Air Cargo
Resources Scheduled flights Room-days Car-days Scheduled flights (weight) Scheduled flights (volume) www.danzhang.com
Products O-D itineraries at certain fare levels Single(multi)-day stays at certain rates Single(multi)-day rentals at certain rates O-D shipments at certain rates
Improved DP Decomposition
Dynamic Programming Formulation DP optimality equations: n X vt (x) = max λ Pj (S)(fj + vt+1 (x − Aj )) S⊆N(x)
j∈S
o + (λP0 (S) + 1 − λ)vt+1 (x) , vτ +1 (x) = 0,
∀x.
Notations vt (x): DP value function Aj : resource incidence vector of product j N(x): {j ∈ N : x ≥ Aj }
www.danzhang.com
Improved DP Decomposition
∀t, x,
Dynamic Programming Formulation DP optimality equations: n X vt (x) = max λ Pj (S)(fj + vt+1 (x − Aj )) S⊆N(x)
j∈S
o + (λP0 (S) + 1 − λ)vt+1 (x) , vτ +1 (x) = 0,
∀t, x,
∀x.
Notations vt (x): DP value function Aj : resource incidence vector of product j N(x): {j ∈ N : x ≥ Aj }
Curse of dimensionality: state space grows exponentially with the number of resources
www.danzhang.com
Improved DP Decomposition
Choice-based Deterministic Linear Program (CDLP) zCDLP = max h
X
X
λR(S)h(S)
S⊆N
λQ(S)h(S) ≤ c,
(Resource constraint)
h(S) ≤ τ,
(Time constraint)
S⊆N
X S⊆N
h(S) ≥ 0,
∀S ⊆ N.
(Non-negativity)
Replace stochastic demand with deterministic fluid with rate λ Given offer set S ⊆ N Total time S is offered: h(S) P Revenue from unit demand: R(S) = j∈S fj Pj (S) Consumption P of resource i from unit demand: Qi (S) = j∈S aij Pj (S) www.danzhang.com
Improved DP Decomposition
CDLP (Gallego et. al, 2004; Liu and van Ryzin, 2008)
CDLP can by efficiently solved for certain class of choice models. The vector of dual values π ∗ associated with resource constraints can be used as “bid-prices” for resources zCDLP provides an upper bound on revenue Some recent references: Talluri (2010): Concave programming formulation Gallego, Ratliff, Shebalov (2011): Efficient reformulation
www.danzhang.com
Improved DP Decomposition
Classical Dynamic Programming Decomposition For each i, approximate the DP value function with X ∗ + πk xk , vt (x) ≈ vt,i (xi ) | {z } k6=i | {z } Value of the Value of all i-th resource other resources
www.danzhang.com
Improved DP Decomposition
∀t, x.
Classical Dynamic Programming Decomposition For each i, approximate the DP value function with X ∗ + πk xk , vt (x) ≈ vt,i (xi ) | {z } k6=i | {z } Value of the Value of all i-th resource other resources
∀t, x.
Using the approximation in DP recursion leads to X X vt,i (xi ) = max λPj (S) fj − akj πk∗ +vt+1,i (xi − aij ) S⊆N(xi ,c−i )
j∈S
k6=i
| {z } Fare proration + (λP0 (S) + 1 − λ)vt+1 (xi ),
www.danzhang.com
∀t, xi .
Improved DP Decomposition
Classical Dynamic Programming Decomposition For each i, approximate the DP value function with X ∗ + πk xk , vt (x) ≈ vt,i (xi ) | {z } k6=i | {z } Value of the Value of all i-th resource other resources
∀t, x.
Using the approximation in DP recursion leads to X X vt,i (xi ) = max λPj (S) fj − akj πk∗ +vt+1,i (xi − aij ) S⊆N(xi ,c−i )
j∈S
k6=i
| {z } Fare proration + (λP0 (S) + 1 − λ)vt+1 (xi ),
∀t, xi .
Compute offer sets dynamically using the approximate value function X vt (x) ≈ vt,i (xi ), ∀t, i. i
www.danzhang.com
Improved DP Decomposition
Classical Dynamic Programming Decomposition A DP with m-dimensional state space is reduced to m one-dimensional DPs, one for each resource.
1014 states (Assume 100 seats per flight)
www.danzhang.com
⇒
4 × 101 states
Improved DP Decomposition
Classical Dynamic Programming Decomposition A DP with m-dimensional state space is reduced to m one-dimensional DPs, one for each resource.
1014 states (Assume 100 seats per flight)
⇒
4 × 101 states
Variants of the approach are widely used in practice. Review: Talluri and van Ryzin (2004a) www.danzhang.com
Improved DP Decomposition
DP Decomposition Bounds
Proposition (Zhang and Adelman, 2009) The following relationships hold: n o P (i) vt (x) ≤ minl=1,...,m vt,l (xl ) + k6=l πk∗ xk ≤ P vt,i (xi ) + k6=i πk∗ xk , ∀i, t, x; P (ii) v1 (c) ≤ v1,i (ci ) + k6=i πk∗ ck ≤ zCDLP , ∀i.
Decomposition value for each leg provides an upper bound on revenue Decomposition bounds are tighter than the bound from CDLP
www.danzhang.com
Improved DP Decomposition
Linear Programming Formulation of DP (Adelman, 2007)
min v1 (c) X vt (x) ≥ λPj (S)(fj + vt+1 (x − Aj )) {vt (·)}∀t
j∈S
+ (λP0 (S) + 1 − λ)vt+1 (x),
www.danzhang.com
∀t, x, S ⊆ N(x).
Improved DP Decomposition
Linear Programming Formulation of DP (Adelman, 2007)
min v1 (c) X vt (x) ≥ λPj (S)(fj + vt+1 (x − Aj )) {vt (·)}∀t
j∈S
+ (λP0 (S) + 1 − λ)vt+1 (x),
∀t, x, S ⊆ N(x).
Huge number of decision variables and constraints
www.danzhang.com
Improved DP Decomposition
Linear Programming Formulation of DP (Adelman, 2007)
min v1 (c) X vt (x) ≥ λPj (S)(fj + vt+1 (x − Aj )) {vt (·)}∀t
j∈S
+ (λP0 (S) + 1 − λ)vt+1 (x),
∀t, x, S ⊆ N(x).
Huge number of decision variables and constraints Functional approximation idea: use a parameterized representation of the value function to reduce the number of decision variables
www.danzhang.com
Improved DP Decomposition
The Affine Functional Approximation (Zhang and Adelman, 2009)
Affine approximation is given by vt (x) ≈ θt +
X
Vt,i xi ,
∀t, x.
i
www.danzhang.com
Improved DP Decomposition
(1)
The Affine Functional Approximation (Zhang and Adelman, 2009)
Affine approximation is given by vt (x) ≈ θt +
X
∀t, x.
Vt,i xi ,
(1)
i
Using (1) in the linear programming formulation leads to min θ1 + θ,V
X
V1,i ci
i
! θt +
X i
Vt,i xi ≥
X
λPj (S) fj + θt+1 +
X
Vt+1,i (xi − aij )
i
j∈S
! + (λP0 (S) + 1 − λ) θt+1 +
X
Vt+1,i xi
,
∀t, x, S ⊆ N(x).
i
www.danzhang.com
Improved DP Decomposition
The Affine Functional Approximation
The dual program is given by zP1 = max Y
X
X
λPj (S)fj Yt,x,S
j∈S
t,x,S⊆N(x)
( X
xi Yt,x,S =
P
x,S⊆N(x)
x,S⊆N(x)
X x,S⊆N(x)
Yt,x,S =
ci , P xi − j∈S λPj (S)aij Yt−1,x,S ,
1, P
x,S⊆N(x)
Yt−1,x,S ,
if t = 1, ∀t = 2, . . . , τ
if t = 1, ∀t = 2, . . . , τ.
Y ≥ 0.
www.danzhang.com
Improved DP Decomposition
∀i, t,
The Affine Functional Approximation
The dual program is given by zP1 = max Y
X
X
λPj (S)fj Yt,x,S
j∈S
t,x,S⊆N(x)
( X
xi Yt,x,S =
P
x,S⊆N(x)
x,S⊆N(x)
X x,S⊆N(x)
Yt,x,S =
ci , P xi − j∈S λPj (S)aij Yt−1,x,S ,
1, P
x,S⊆N(x)
Yt−1,x,S ,
if t = 1, ∀t = 2, . . . , τ
∀i, t,
if t = 1, ∀t = 2, . . . , τ.
Y ≥ 0.
Due to the large number of columns, solving the linear program above still requires considerable computational effort.
www.danzhang.com
Improved DP Decomposition
Functional Approximation Approaches for Network RM
Citation
Choice Model
Functional approximation
Solution strategy
Adelman (2007) Zhang and Adelman (2009) Zhang (2011) Liu and van Ryzin (2008) Miranda Bront et. al. (2009) Farias and Van Roy (2008) Meissner and Strauss (2012) Kunnumkal and Topaloglu (2011) Tong and Topaloglu (2011)
Independent demand MNLD MNLD MNLD MNLO Independent demand MNLD MNLD
Affine Affine Nonlinear Separable Separable Separable Separable Separable
Independent demand
Affine
Vossen and Zhang
Independent demand + MNLD
Affine
Column generation Column generation CDLP+Simultaneous DP CDLP+DP Decomposition CDLP+DP Decomposition Constraint sampling Column generation Convex programming +DP Decomposition Reduction + Constraint generation Reduction + Dynamic disaggregation
non-separable (fare proration) (fare proration) concave concave (fare proration)
MNLD: Multinomial logit model with disjoint consideration sets MNLO: Multinomial logit model with overlapping consideration sets
www.danzhang.com
Improved DP Decomposition
Research Questions
Computational cost: ADP (affine or separable concave approximation) classical DP decomposition
www.danzhang.com
Improved DP Decomposition
Research Questions
Computational cost: ADP (affine or separable concave approximation) classical DP decomposition How can we balance solution quality with solution time? Can we improve the classical DP decomposition?
www.danzhang.com
Improved DP Decomposition
A Strong Functional Approximation (Zhang, 2011)
X πk∗ xk , vt (x) ≈ min vˆt,i (xi ) + i
∀t, x.
k6=i
Nonlinear and non-separable functional approximation Each value vt (x) is approximated by a single value across legs Motivated by the decomposition bounds (Zhang and Adelman, 2009)
www.danzhang.com
Improved DP Decomposition
A Nonlinear Optimization Problem Using the new functional approximation leads to zNLP =
min min vˆt,i (·)∀t,i
i
vˆ1,i (ci ) +
X k6=i
πk∗ ck
X ∗ min vˆt,i (xi ) + πk xk ≥ i k6=i X X ∗ λPj (S) fj + min vˆt+1,l (xl − alj ) + πk (xk − akj ) l j∈S k6=l X ∗ + (λP0 (S) + 1 − λ)min vˆt+1,l (xl ) + πk xk , ∀t, x, S ⊆ N(x). l k6=l
The problem is a nonlinear optimization problem with a huge number of nonlinear constraints. www.danzhang.com
Improved DP Decomposition
A Restricted Optimization Problem
Step 1: Writing each constraint as m equivalent constraints Step 2: Restricting the constraints so that each constraint only involves one resource The restricted problem provides a relaxed bound: Proposition The objective value of the restricted program, zNLP d , is bigger than zNLP .
www.danzhang.com
Improved DP Decomposition
An Equivalent Simultaneous Dynamic Program ( ∗ vˆt,i (xi )
=
X
max S⊆N(xi ,c−i )
∗ λPj (S) fj + min vˆt+1,i (xi − aij ) −
j∈S
l6=i
max
πk∗ akj ,
k6=i
min
X
0≤yl ≤cl −alj
∗ [ˆ vt+1,l (yl )
−
yl πl∗ ]
−
X
akj πk∗
+
πi∗ xi
)!
k
∗ ∗ vt+1,l (yl ) − πl∗ yl ] + πi∗ xi + (λP0 (S) + 1 − λ) min vˆt+1,i (xi ), min max [ˆ l6=i
0≤yl ≤cl
∀i, t, xi .
DP recursion for resource i involves values from all other resources
www.danzhang.com
Improved DP Decomposition
An Equivalent Simultaneous Dynamic Program ( ∗ vˆt,i (xi )
=
X
max S⊆N(xi ,c−i )
∗ λPj (S) fj + min vˆt+1,i (xi − aij ) −
j∈S
l6=i
max
πk∗ akj ,
k6=i
min
X
0≤yl ≤cl −alj
∗ [ˆ vt+1,l (yl )
−
yl πl∗ ]
−
X
akj πk∗
+
πi∗ xi
)!
k
∗ ∗ vt+1,l (yl ) − πl∗ yl ] + πi∗ xi + (λP0 (S) + 1 − λ) min vˆt+1,i (xi ), min max [ˆ l6=i
0≤yl ≤cl
∀i, t, xi .
DP recursion for resource i involves values from all other resources The dynamic program is equivalent to the restricted nonlinear program can be solved efficiently via a simultaneous dynamic programming algorithm leads to tighter revenue bounds www.danzhang.com
Improved DP Decomposition
New Bounds
Proposition (Zhang, 2011) ∗ (·)} Let {ˆ vt,i ∀t,i,xi be the optimal solution from the simultaneous dynamic program. The following results hold: ∗ (x ) ≤ v (x ), ∀i, x ; (i) vˆt,i i t,i i i
n o P ∗ (c ) + ∗c (ii) v1 (c) ≤ zNLP ≤ zNLP = min v ˆ π i i d k6=i k k ≤ n o 1,i P mini v1,i (ci ) + k6=i πk∗ ck ≤ zCDLP . The simultaneous dynamic program provides tighter bounds on revenue than the classical decomposition.
www.danzhang.com
Improved DP Decomposition
Recap
High dimensional dynamic program
www.danzhang.com
Improved DP Decomposition
Recap
High dimensional dynamic program m Large scale linear program
www.danzhang.com
Improved DP Decomposition
Recap
High dimensional dynamic program m Large scale linear program ⇓ Large scale nonlinear program with nonlinear constraints
www.danzhang.com
Improved DP Decomposition
Recap
High dimensional dynamic program m Large scale linear program ⇓ Large scale nonlinear program with nonlinear constraints ⇓ Restricted nonlinear program with nonlinear constraints
www.danzhang.com
Improved DP Decomposition
Recap
High dimensional dynamic program m Large scale linear program ⇓ Large scale nonlinear program with nonlinear constraints ⇓ Restricted nonlinear program with nonlinear constraints m Simultaneous dynamic program
www.danzhang.com
Improved DP Decomposition
Comparison: Classical vs. Improved Approaches Classical dynamic programming decomposition: Solve m single-leg DPs Prorated fares
www.danzhang.com
Fare proration
Static bid-prices
Solve CDLP
Improved DP Decomposition
Comparison: Classical vs. Improved Approaches Classical dynamic programming decomposition: Solve m single-leg DPs Prorated fares
Fare proration
Static bid-prices
Solve CDLP
Network effects only captured through fare proration
www.danzhang.com
Improved DP Decomposition
Comparison: Classical vs. Improved Approaches Classical dynamic programming decomposition: Solve m single-leg DPs Prorated fares
Fare proration
Static bid-prices
Solve CDLP
Network effects only captured through fare proration Improved dynamic programming decomposition: Solve one simultaneous DP Static bid-prices
www.danzhang.com
Solve CDLP
Improved DP Decomposition
Comparison: Classical vs. Improved Approaches Classical dynamic programming decomposition: Solve m single-leg DPs Prorated fares
Fare proration
Static bid-prices
Solve CDLP
Network effects only captured through fare proration Improved dynamic programming decomposition: Solve one simultaneous DP Static bid-prices
Solve CDLP
Network effects captured during DP recursion! www.danzhang.com
Improved DP Decomposition
Computational Study: Problem Instances
Randomly generated hub-and-spoke instances Number of non-hub locations (flights) in the set {4, 8, 16, 24} Number of periods in the set {100, 200, 400, 800} Two products for each possible itinerary Multinomial Logit Choice Model with Disjoint Consideration Sets (MNLD) Largest problem instance: 24 non-hub locations (flights), 336 products, 800 periods
www.danzhang.com
Improved DP Decomposition
Numerical Study: Policies
DCOMP1: the new decomposition approach where the approximation m X ∗ vt (x) ≈ vˆt,i (xi ), ∀t, x i=1
is used to compute control policies. DCOMP: the classical dynamic programming decomposition CDLP: static bid-price policy based on the dual values of resource constraints in CDLP CDLP10: A version of CDLP that resolves 10 times with equally spaced resolving intervals Each policy is simulated 20000 times
www.danzhang.com
Improved DP Decomposition
Computational Time
Case # A1 A2 A3 A4 A5 A6 A7 A8 A9 A10 A11 A12 A13 A14 A15 A16
Parameters (100,4,4,16) (200,4,4,16) (400,4,4,16) (800,4,4,16) (100,8,8,48) (200,8,8,48) (400,8,8,48) (800,8,8,48) (100,16,16,160) (200,16,16,160) (400,16,16,160) (800,16,16,160) (100,24,24,336) (200,24,24,336) (400,24,24,336) (800,24,24,336)
Capacity
Load
per leg
factor
CDLP
DCOMP
DCOMP1
1.17 1.27 1.19 1.28 1.43 1.36 1.35 1.21 1.65 1.45 1.29 1.40 1.45 1.35 1.29 1.38
0.16 0.23 0.16 0.20 1.52 0.72 1.61 0.72 4.64 4.69 2.92 3.64 3.69 4.59 4.39 4.23
2.03 7.89 31.92 127.66 5.75 22.83 91.67 362.84 15.09 75.84 303.66 1218.67 24.72 98.52 492.73 1978.14
2.75 10.88 43.73 174.48 7.47 29.58 118.92 471.73 19.42 96.97 388.97 1560.19 31.81 127.36 630.84 2532.20
10 20 40 80 5 10 20 40 2 5 10 20 1 2 5 10
www.danzhang.com
CPU seconds
DCOMP1−DCOMP DCOMP
Improved DP Decomposition
35.38% 37.82% 37.00% 36.68% 29.89% 29.57% 29.73% 30.01% 28.67% 27.85% 28.10% 28.02% 28.70% 29.28% 28.03% 28.01%
Bound Performance
Case # A1 A2 A3 A4 A5 A6 A7 A8 A9 A10 A11 A12 A13 A14 A15 A16
Bound improvement
%-difference across legs
CDLP bound
DCOMP bound
DCOMP1 bound
%-CDLP
%-DCOMP
DCOMP
DCOMP1
24078.90 48367.58 89312.44 213102.50 32521.30 70541.63 107831.01 216080.83 26347.76 60629.35 101616.47 224780.69 13074.04 26296.19 74112.13 131457.79
23985.56 48328.43 87576.49 211854.85 31029.90 68760.67 106339.36 212915.61 24953.24 58489.12 100069.27 222558.53 11845.73 24926.41 72089.14 129589.28
22900.49 47588.56 86729.90 211087.37 30726.17 68617.41 106153.32 212848.05 24764.75 58118.33 99771.63 222231.72 10386.38 24373.33 71617.55 129273.91
5.15% 1.64% 2.98% 0.95% 5.84% 2.80% 1.58% 1.52% 6.39% 4.32% 1.85% 1.15% 25.88% 7.89% 3.48% 1.69%
4.74% 1.55% 0.98% 0.36% 0.99% 0.21% 0.18% 0.03% 0.76% 0.64% 0.30% 0.15% 14.05% 2.27% 0.66% 0.24%
4.46% 1.87% 2.36% 0.58% 3.18% 2.18% 1.09% 1.49% 4.69% 2.95% 1.52% 0.94% 10.37% 5.50% 2.80% 1.44%
0.00% 0.36% 0.00% 0.00% 0.05% 0.22% 0.00% 0.00% 0.00% 0.03% 0.01% 0.00% 0.00% 0.00% 0.03% 0.00%
www.danzhang.com
Improved DP Decomposition
Bounds from Individual Legs
Bounds from individual Legs s from individual Legs dividual Legs al Legs
132000 131500 131000 130500 130000
DCOMP
129500
DCOMP1
129000 128500 128000 1
2
3
4
5
6
7
8
9
10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Leg
www.danzhang.com
Improved DP Decomposition
Bounds from Individual Legs
Bounds from individual Legs s from individual Legs dividual Legs al Legs
132000 131500 131000 130500 130000
DCOMP
129500
DCOMP1
129000 128500 128000 1
2
3
4
5
6
7
8
9
10 11 12 13 14 15 16 17 18 19 20 21 22 23 24
Leg
DCOMP1 bounds are more homogeneous across legs
www.danzhang.com
Improved DP Decomposition
A Hub-and-spoke Network with 2 Non-Hub Locations
Case # B1 B2 B3 B4 B5 B6 B7 B8 B9 B10 B11 B12 B13 B14 B15 B16 B17 B18 B19 B20
τ 100 200 400 800 100 200 400 800 100 200 400 800 100 200 400 800 100 200 400 800
DCOMP1 Revenue Gains
Load factor
Capacity per leg
DCOMP1 REV
OPT-GAP
2.40 2.13 2.13 2.13 1.60 1.60 1.60 1.60 1.37 1.28 1.28 1.28 1.07 1.07 1.07 1.07 0.96 0.91 0.91 0.91
4 9 18 36 6 12 24 48 7 15 30 60 9 18 36 72 10 21 42 84
5775.47 13262.92 25456.41 53946.59 8034.27 17318.40 35472.06 65618.95 9269.56 20521.01 42471.91 86841.58 11107.51 23268.75 47824.97 96993.79 11854.02 25259.70 51593.10 104376.37
-3.17% -2.04% -6.53% -1.20% -7.25% -2.31% -1.25% -9.20% -5.85% -3.70% -2.10% -1.15% -1.68% -0.80% -0.27% -0.08% -0.85% -0.07% 0.03% 0.01%
www.danzhang.com
%-CDLP
%-CDLP10
%-DCOMP
182.90% 209.79% -1.00% 212.87% 48.88% 13.42% 311.96% 42.16% 13.27% 15.05% 15.46% 15.47% 2.89% 4.84% 22.11% 7.92% 23.82% 2.61% 30.16% 31.46%
8.56% 5.74% -0.33% 4.41% 3.00% 3.62% 4.78% -4.49% 1.33% 0.98% 1.70% 2.45% 0.47% 1.14% 2.22% 2.14% 1.46% 1.24% 2.48% 2.60%
2.74% 4.11% 0.03% 7.17% 0.09% 5.77% 7.97% 0.00% 1.94% 4.65% 8.14% 5.52% 0.34% 0.11% 0.04% 0.01% 0.18% 0.04% 0.01% 0.00%
Improved DP Decomposition
DCOMP1 Percentage Revenue Gain vs. Load Factor
DCOMP1 percentage revenue gain
10 %−CDLP10 %−DCOMP
5
0
−5 0.5
1
1.5 Load factor
www.danzhang.com
2
Improved DP Decomposition
2.5
DCOMP1 Percentage Revenue Gain vs. Load Factor
DCOMP1 percentage revenue gain
10 %−CDLP10 %−DCOMP
5
0
−5 0.5
1
1.5 Load factor
2
2.5
Higher load factor ⇒ Higher revenue gains www.danzhang.com
Improved DP Decomposition
DCOMP1 Percentage Revenue Gain vs. Number of Periods
DCOMP1 percentage revenue gain
10 %−CDLP10 %−DCOMP
5
0
−5 0
100
200
300 400 500 600 Number of periods
www.danzhang.com
700
800
Improved DP Decomposition
900
DCOMP1 Percentage Revenue Gain vs. Number of Periods
DCOMP1 percentage revenue gain
10 %−CDLP10 %−DCOMP
5
0
−5 0
100
200
300 400 500 600 Number of periods
700
800
900
Significant revenue gains for problems with long selling horizons! www.danzhang.com
Improved DP Decomposition
A Hub-and-spoke Network with 4 Non-Hub Locations
Case # C1 C2 C3 C4 C5 C6 C7 C8 C9 C10 C11 C12 C13 C14 C15 C16 C17 C18 C19 C20
τ 100 200 400 800 100 200 400 800 100 200 400 800 100 200 400 800 100 200 400 800
DCOMP1 Revenue Gains
Load factor
Capacity per leg
DCOMP1 REV
OPT-GAP
1.99 1.99 1.99 1.99 1.49 1.49 1.49 1.49 1.19 1.19 1.19 1.19 1.00 1.00 1.00 1.00 0.85 0.85 0.85 0.85
6 12 24 48 8 16 32 64 10 20 40 80 12 24 48 96 14 28 56 112
16795.66 35028.96 70163.84 143921.85 21860.34 45171.83 88532.95 184410.85 26270.03 54509.68 111520.14 226059.91 29208.18 61175.75 125854.79 256236.11 32057.27 66527.87 135897.71 274817.89
-5.08% -2.63% -3.35% -1.34% -4.23% -2.54% -5.29% -1.79% -5.00% -3.02% -1.79% -1.05% -5.00% -3.02% -1.79% -1.00% -2.55% -1.28% -0.51% -0.17%
www.danzhang.com
%-CDLP
%-CDLP10
%-DCOMP
14.60% 52.79% 12.35% 52.31% 44.57% 14.18% 29.01% 1.02% 4.98% 4.18% 3.44% 57.07% 1.30% 2.18% 6.38% 2.66% 2.50% 3.05% 3.24% 3.21%
0.46% 1.95% -0.11% 1.36% 1.71% 1.72% -2.27% 0.49% 1.46% 1.61% 1.51% 1.92% -0.90% 0.21% 0.74% 0.89% -0.81% -0.03% 0.26% 0.41%
0.10% 1.32% 0.03% 0.66% 1.96% 2.67% -0.81% 0.18% 2.67% 4.11% 4.43% 3.08% 0.21% 0.08% 0.00% -0.06% 0.44% 0.20% 0.05% 0.02%
Improved DP Decomposition
DCOMP1 Percentage Revenue Gain vs. Load Factor
DCOMP1 percentage revenue gain
5 %−CDLP10 %−DCOMP
4 3 2 1 0 −1 −2 −3 0.5
1
1.5 Load factor
www.danzhang.com
2
Improved DP Decomposition
2.5
DCOMP1 Percentage Revenue Gain vs. Number of Periods
DCOMP1 percentage revenue gain
5 %−CDLP10 %−DCOMP
4 3 2 1 0 −1 −2 −3 0
100
200
300 400 500 600 Number of periods
www.danzhang.com
700
800
Improved DP Decomposition
900
Summary and Future Directions Functional approximation approach is promising for solving large scale stochastic dynamic programs. However, implementations of the approach often require very high computational cost.
The first nonlinear non-separable functional approximation for network RM problem Novel approximation architecture Better revenue bounds Improved heuristic policies Moderate computational cost
Current work Exploiting special structures of the LP formulations of dynamic programs in value function approximation (Vossen and Zhang, 2012) Applications with real data (Zhang and Weatherford, 2012)
www.danzhang.com
Improved DP Decomposition
Thank you!
Questions? Comments?
www.danzhang.com
Improved DP Decomposition
