Column Generation Based Primal Heuristics - Semantic Scholar
Column Generation Based Primal Heuristics C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
University Bordeaux 1 & INRIA team RealOpt
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Outline
1
Context Generic Primal Heuristics The Branch-and-Price Approach
2
Column Generation based Primal Heuristics Specificity of the B-a-P context Literature Review
3
Diving Heuristic An implementation with Limited Discrepancy Search Numerical tests
4
Discussion
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Context
Generic Primal Heuristics for MIPs “good” feasible solutions using the tools of exact optimization Truncating an exact method Building from the relaxation used for the exact approach Defining a target based on the relaxation Using dual information to price choices in greedy heuristics Exact approach used to explore a neighborhood Examples: [Berthold’06] 1
Relaxation Induced Neighborhood Search [Dana al’05]
2
Local Branching [Fischetti al’03]
3
Feasibility Pump [Fischetti al’05]
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Context
Generic Primal Heuristics for MIPs “good” feasible solutions using the tools of exact optimization Truncating an exact method Building from the relaxation used for the exact approach Defining a target based on the relaxation Using dual information to price choices in greedy heuristics Exact approach used to explore a neighborhood Examples: [Berthold’06] 1
Relaxation Induced Neighborhood Search [Dana al’05]
2
Local Branching [Fischetti al’03]
3
Feasibility Pump [Fischetti al’05]
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Context
Generic Primal Heuristics for MIPs “good” feasible solutions using the tools of exact optimization Truncating an exact method Building from the relaxation used for the exact approach Defining a target based on the relaxation Using dual information to price choices in greedy heuristics Exact approach used to explore a neighborhood Examples: [Berthold’06] 1
Relaxation Induced Neighborhood Search [Dana al’05]
2
Local Branching [Fischetti al’03]
3
Feasibility Pump [Fischetti al’05]
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Context
Generic Primal Heuristics for MIPs “good” feasible solutions using the tools of exact optimization Truncating an exact method Building from the relaxation used for the exact approach Defining a target based on the relaxation Using dual information to price choices in greedy heuristics Exact approach used to explore a neighborhood Examples: [Berthold’06] 1
Relaxation Induced Neighborhood Search [Dana al’05]
2
Local Branching [Fischetti al’03]
3
Feasibility Pump [Fischetti al’05]
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Context
Generic Primal Heuristics for MIPs “good” feasible solutions using the tools of exact optimization Truncating an exact method Building from the relaxation used for the exact approach Defining a target based on the relaxation Using dual information to price choices in greedy heuristics Exact approach used to explore a neighborhood Examples: [Berthold’06] 1
Relaxation Induced Neighborhood Search [Dana al’05]
2
Local Branching [Fischetti al’03]
3
Feasibility Pump [Fischetti al’05]
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Context
Generic LP based heuristics min{
P
j
cj xj :
P
j
aij xj ≥ bi ∀i, lj ≤ xj ≤ uj ∀j}
Rounding: Iteratively select a var xj and bound/fix it least fractional: argminj {min{xj − bxj c, dxj e − xj }} least infeasibility: argminj {up-lockj + down-lockj } where up-lockj = # constraint i with aij < 0 and down-lockj = # constraint i with aij > 0 guided search: argminj {|xj − xjinc |} least pseudo-cost: argminj {LP val increase} c greedy: argminj { P jaij } i
Diving: rounding + LP resolve + reiterate heuristic depth search in branch-and-bound tree branching rule 6= that of exact branch-and-bound sub-MIPing: rounding + MIP sol of the residual prob. C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Context
Generic LP based heuristics min{
P
j
cj xj :
P
j
aij xj ≥ bi ∀i, lj ≤ xj ≤ uj ∀j}
Rounding: Iteratively select a var xj and bound/fix it least fractional: argminj {min{xj − bxj c, dxj e − xj }} least infeasibility: argminj {up-lockj + down-lockj } where up-lockj = # constraint i with aij < 0 and down-lockj = # constraint i with aij > 0 guided search: argminj {|xj − xjinc |} least pseudo-cost: argminj {LP val increase} c greedy: argminj { P jaij } i
Diving: rounding + LP resolve + reiterate heuristic depth search in branch-and-bound tree branching rule 6= that of exact branch-and-bound sub-MIPing: rounding + MIP sol of the residual prob. C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Context
Generic LP based heuristics min{
P
j
cj xj :
P
j
aij xj ≥ bi ∀i, lj ≤ xj ≤ uj ∀j}
Rounding: Iteratively select a var xj and bound/fix it least fractional: argminj {min{xj − bxj c, dxj e − xj }} least infeasibility: argminj {up-lockj + down-lockj } where up-lockj = # constraint i with aij < 0 and down-lockj = # constraint i with aij > 0 guided search: argminj {|xj − xjinc |} least pseudo-cost: argminj {LP val increase} c greedy: argminj { P jaij } i
Diving: rounding + LP resolve + reiterate heuristic depth search in branch-and-bound tree branching rule 6= that of exact branch-and-bound sub-MIPing: rounding + MIP sol of the residual prob. C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Context
Generic LP based heuristics min{
P
j
cj xj :
P
j
aij xj ≥ bi ∀i, lj ≤ xj ≤ uj ∀j}
Rounding: Iteratively select a var xj and bound/fix it least fractional: argminj {min{xj − bxj c, dxj e − xj }} least infeasibility: argminj {up-lockj + down-lockj } where up-lockj = # constraint i with aij < 0 and down-lockj = # constraint i with aij > 0 guided search: argminj {|xj − xjinc |} least pseudo-cost: argminj {LP val increase} c greedy: argminj { P jaij } i
Diving: rounding + LP resolve + reiterate heuristic depth search in branch-and-bound tree branching rule 6= that of exact branch-and-bound sub-MIPing: rounding + MIP sol of the residual prob. C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Context
Heuristic search in branch-and-bound tree
Diving
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
sub-MIPing
Column Generation Based Primal Heuristics
Context
The Branch-and-Price Approach min
c1x 1 D x1 B x1
c2 x 2 D x2
+ +
+ +
... ...
+ +
cK x K D xK
K
B xK ∈ Nn .
B x2 .. x
1
n
∈N ,
x
2
n
∈N ,
.
...
x
≥d ≥b ≥b . ≥ .. ≥b
Relax Dx ≥ d = decomposition: subproblem {B x ≥ b, x ∈ Nn } and a reformulation solved by Branch-and-Price: P min P S∈S cx s λs s s∈S PDx λs ≥ = S∈S λs λ ∈ X k X s y := x = cx k
Solve Master LP
d K N|S| λs
S∈S
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Pricing Problem
Solve Master LP
Solve Master LP
Pricing Problem
Pricing Problem
Column Generation Based Primal Heuristics
Context
The Branch-and-Price Approach min
c1x 1 D x1 B x1
c2 x 2 D x2
+ +
+ +
... ...
+ +
cK x K D xK
K
B xK ∈ Nn .
B x2 .. x
1
n
∈N ,
x
2
n
∈N ,
.
...
x
≥d ≥b ≥b . ≥ .. ≥b
Relax Dx ≥ d = decomposition: subproblem {B x ≥ b, x ∈ Nn } and a reformulation solved by Branch-and-Price: P min P S∈S cx s λs s s∈S PDx λs ≥ = S∈S λs λ ∈ X k X s y := x = cx k
Solve Master LP
d K N|S| λs
S∈S
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Pricing Problem
Solve Master LP
Solve Master LP
Pricing Problem
Pricing Problem
Column Generation Based Primal Heuristics
Context
The Branch-and-Price Approach min
c1x 1 D x1 B x1
c2 x 2 D x2
+ +
+ +
... ...
+ +
cK x K D xK
K
B xK ∈ Nn .
B x2 .. x
1
n
∈N ,
x
2
n
∈N ,
.
...
x
≥d ≥b ≥b . ≥ .. ≥b
Relax Dx ≥ d = decomposition: subproblem {B x ≥ b, x ∈ Nn } and a reformulation solved by Branch-and-Price: P min P S∈S cx s λs s s∈S PDx λs ≥ = S∈S λs λ ∈ X k X s y := x = cx k
Solve Master LP
d K N|S| λs
S∈S
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Pricing Problem
Solve Master LP
Solve Master LP
Pricing Problem
Pricing Problem
Column Generation Based Primal Heuristics
Column Generation based Primal Heuristics
Difficulties in a B-a-P context Heuristic paradigm in original space or the reformulation? On master variables: λ Cannot fix bounds (as in rounding) Cannot define constraints (as in local branching) On original variables: x Cannot grasp individual SP var. after aggregation in the common case of identical SPs Cannot modify the SP structure required by the oracle What can be done: define a partial solution iteratively updated (lb on λ) apply col. gen. to the residual prob. (prepr. = bd x, λ)
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Column Generation based Primal Heuristics
Difficulties in a B-a-P context Heuristic paradigm in original space or the reformulation? On master variables: λ Cannot fix bounds (as in rounding) Cannot define constraints (as in local branching) On original variables: x Cannot grasp individual SP var. after aggregation in the common case of identical SPs Cannot modify the SP structure required by the oracle What can be done: define a partial solution iteratively updated (lb on λ) apply col. gen. to the residual prob. (prepr. = bd x, λ)
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Column Generation based Primal Heuristics
Difficulties in a B-a-P context Heuristic paradigm in original space or the reformulation? On master variables: λ Cannot fix bounds (as in rounding) Cannot define constraints (as in local branching) On original variables: x Cannot grasp individual SP var. after aggregation in the common case of identical SPs Cannot modify the SP structure required by the oracle What can be done: define a partial solution iteratively updated (lb on λ) apply col. gen. to the residual prob. (prepr. = bd x, λ)
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Column Generation based Primal Heuristics
Difficulties in a B-a-P context Heuristic paradigm in original space or the reformulation? On master variables: λ Cannot fix bounds (as in rounding) Cannot define constraints (as in local branching) On original variables: x Cannot grasp individual SP var. after aggregation in the common case of identical SPs Cannot modify the SP structure required by the oracle What can be done: define a partial solution iteratively updated (lb on λ) apply col. gen. to the residual prob. (prepr. = bd x, λ)
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Column Generation based Primal Heuristics
Review of column generation based heuristics 1
Restricted Master P P P min{ S cs λs : s Dx s λs ≥ d, s λs = K , λs ∈ N s ∈ S} restricted set build either heuristically / from master LP
2
Greedy iteratively select column with best “pseudo-cost / red-cost”
3
Rounding round down master LP solution + iterative round-up complete heuristically
4
Local search delete columns from incumbent rebuild heuristically
genericity & feasibility issue C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Column Generation based Primal Heuristics
Review of column generation based heuristics 1
Restricted Master P P P min{ S cs λs : s Dx s λs ≥ d, s λs = K , λs ∈ N s ∈ S} restricted set build either heuristically / from master LP
2
Greedy iteratively select column with best “pseudo-cost / red-cost”
3
Rounding round down master LP solution + iterative round-up complete heuristically
4
Local search delete columns from incumbent rebuild heuristically
genericity & feasibility issue C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Column Generation based Primal Heuristics
Review of column generation based heuristics 1
Restricted Master P P P min{ S cs λs : s Dx s λs ≥ d, s λs = K , λs ∈ N s ∈ S} restricted set build either heuristically / from master LP
2
Greedy iteratively select column with best “pseudo-cost / red-cost”
3
Rounding round down master LP solution + iterative round-up complete heuristically
4
Local search delete columns from incumbent rebuild heuristically
genericity & feasibility issue C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Column Generation based Primal Heuristics
Review of column generation based heuristics 1
Restricted Master P P P min{ S cs λs : s Dx s λs ≥ d, s λs = K , λs ∈ N s ∈ S} restricted set build either heuristically / from master LP
2
Greedy iteratively select column with best “pseudo-cost / red-cost”
3
Rounding round down master LP solution + iterative round-up complete heuristically
4
Local search delete columns from incumbent rebuild heuristically
genericity & feasibility issue C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Column Generation based Primal Heuristics
Review of column generation based heuristics 1
Restricted Master P P P min{ S cs λs : s Dx s λs ≥ d, s λs = K , λs ∈ N s ∈ S} restricted set build either heuristically / from master LP
2
Greedy iteratively select column with best “pseudo-cost / red-cost”
3
Rounding round down master LP solution + iterative round-up complete heuristically
4
Local search delete columns from incumbent rebuild heuristically
genericity & feasibility issue C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Column Generation based Primal Heuristics
Review of column generation based heuristics 1
Restricted Master (generic, often leads to infeasibility) P P P min{ S cs λs : s Dx s λs ≥ d, s λs = K , λs ∈ N s ∈ S} restricted set build either heuristically / from master LP
2
Greedy (application specific) iteratively select column with best “pseudo-cost / red-cost”
3
Rounding (application specific) round down master LP solution + iterative round-up complete heuristically
4
Local search (application specific) delete columns from incumbent rebuild heuristically
genericity & feasibility issue C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Diving Heuristic
“Diving” Heuristic: generic algorithm Generic way to restore feasibility: further column generation Step 1: Solve the current master LP. Step 2: Select columns into the current solution at ¯sc heuristically set values: λs ← dλ Step 3: If the partial solution defines a complete primal solution, record this solution and goto Step 6. Step 4: Update the master (RHS) and the pricing problems (SP var. bounds). Step 5: If residual master problem is shown infeasible through preprocessing, goto Step 6. Else, return to Step 1. Step 6: + diversification procedure: limited backtracking
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Diving Heuristic
“Diving” Heuristic: generic algorithm Generic way to restore feasibility: further column generation Step 1: Solve the current master LP. Step 2: Select columns into the current solution at ¯sc heuristically set values: λs ← dλ Step 3: If the partial solution defines a complete primal solution, record this solution and goto Step 6. Step 4: Update the master (RHS) and the pricing problems (SP var. bounds). Step 5: If residual master problem is shown infeasible through preprocessing, goto Step 6. Else, return to Step 1. Step 6: Stop. + diversification procedure: limited backtracking
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Diving Heuristic
“Diving” Heuristic: generic algorithm Generic way to restore feasibility: further column generation Step 1: Solve the current master LP. Step 2: Select columns into the current solution at ¯sc heuristically set values: λs ← dλ Step 3: If the partial solution defines a complete primal solution, record this solution and goto Step 6. Step 4: Update the master (RHS) and the pricing problems (SP var. bounds). Step 5: If residual master problem is shown infeasible through preprocessing, goto Step 6. Else, return to Step 1. Step 6: Backtrack. + diversification procedure: limited backtracking
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Diving Heuristic
“Diving” plus “diversification” Depth-first search: select column: ¯sc λs ← d λ update master and SP apply preprocessing
Backtracking: Limited Discrepancy Search
MaxDiscrepancy = 2, MaxDepth = 3 C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Diving Heuristic
Numerical tests: Cutting Stock & Vertex Coloring
2 1
3 5
problem CSP CSP VCP
size 50 80 11–211
#inst. 10 10 18
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
pure B-a-P #nodes time 245 26 417 192 17 116
4 DH w LDS gap time 0% 2 0% 14 0% 26
Column Generation Based Primal Heuristics
Diving Heuristic
Numerical tests: Bin Packing With Conflicts con
flict
conflict
conflict
20 inst. class t60 t120 t249 t501 u120 u250 u500 u1000
b-a-p not opt. av. time 0 1.7 1 156.0 2 334.2 0 245.9 0 3.4 0 23.9 1 318.0 0 1401.2
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
b-a-p + diving w LDS not opt. av. time max. time 0 1.6 7.2 0 14.9 846.3 0 32.1 172.1 0 174.9 793.5 0 3.6 36.2 0 14.6 30.9 0 74.8 136.3 0 552.8 5359.6 Column Generation Based Primal Heuristics
Diving Heuristic
Numerical tests: Bin Packing With Conflicts con
flict
conflict
conflict
class t60 t120 t249 t501 u120 u250 u500 u1000
Tabu Search gap time 0.45% 37.5 0.62% 40.0 0.39% 51.9 0.21% 58.9 0.10% 22.4 0.21% 52.1 0.20% 69.7 0.22% 107.8
Diving H gap time 0.11% 1.3 0.66% 2.5 0.35% 5.1 0.16% 10.3 0.18% 2.6 0.07% 5.4 0.03% 10.7 0.01% 21.5
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
DH LDS gap time 0% 1.5 0% 5.9 0% 32.1 0% 174.9 0% 3.3 0% 14.6 0% 74.8 0.002% 514.1
Column Generation Based Primal Heuristics
Diving Heuristic
Numerical tests: Vehicle Routing
problem VRP
size An32-80
#inst. 27
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Diving H # unsolved gap 17 5.25%
time 70
Column Generation Based Primal Heuristics
Discussion
Feedback on experimentation
Rounding/Diving based on fixing master var. λ works when 1
Sufficiently many P columns in the solution: s λs = K with K 1
2
Objective function is “flat”: many solutions of the same costs
3
Most of the combinatorial difficulty is in the subproblem
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Discussion
Further research
Exact Branch-and-Price with heuristic oracle First experiment on CSP with greedy knapsack solver Seems no good when most of the difficulty is in the SP
Sub-MIPing Fixing original variables in aggregate space
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
University Bordeaux 1 & INRIA team RealOpt
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Outline
1
Context Generic Primal Heuristics The Branch-and-Price Approach
2
Column Generation based Primal Heuristics Specificity of the B-a-P context Literature Review
3
Diving Heuristic An implementation with Limited Discrepancy Search Numerical tests
4
Discussion
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Context
Generic Primal Heuristics for MIPs “good” feasible solutions using the tools of exact optimization Truncating an exact method Building from the relaxation used for the exact approach Defining a target based on the relaxation Using dual information to price choices in greedy heuristics Exact approach used to explore a neighborhood Examples: [Berthold’06] 1
Relaxation Induced Neighborhood Search [Dana al’05]
2
Local Branching [Fischetti al’03]
3
Feasibility Pump [Fischetti al’05]
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Context
Generic Primal Heuristics for MIPs “good” feasible solutions using the tools of exact optimization Truncating an exact method Building from the relaxation used for the exact approach Defining a target based on the relaxation Using dual information to price choices in greedy heuristics Exact approach used to explore a neighborhood Examples: [Berthold’06] 1
Relaxation Induced Neighborhood Search [Dana al’05]
2
Local Branching [Fischetti al’03]
3
Feasibility Pump [Fischetti al’05]
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Context
Generic Primal Heuristics for MIPs “good” feasible solutions using the tools of exact optimization Truncating an exact method Building from the relaxation used for the exact approach Defining a target based on the relaxation Using dual information to price choices in greedy heuristics Exact approach used to explore a neighborhood Examples: [Berthold’06] 1
Relaxation Induced Neighborhood Search [Dana al’05]
2
Local Branching [Fischetti al’03]
3
Feasibility Pump [Fischetti al’05]
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Context
Generic Primal Heuristics for MIPs “good” feasible solutions using the tools of exact optimization Truncating an exact method Building from the relaxation used for the exact approach Defining a target based on the relaxation Using dual information to price choices in greedy heuristics Exact approach used to explore a neighborhood Examples: [Berthold’06] 1
Relaxation Induced Neighborhood Search [Dana al’05]
2
Local Branching [Fischetti al’03]
3
Feasibility Pump [Fischetti al’05]
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Context
Generic Primal Heuristics for MIPs “good” feasible solutions using the tools of exact optimization Truncating an exact method Building from the relaxation used for the exact approach Defining a target based on the relaxation Using dual information to price choices in greedy heuristics Exact approach used to explore a neighborhood Examples: [Berthold’06] 1
Relaxation Induced Neighborhood Search [Dana al’05]
2
Local Branching [Fischetti al’03]
3
Feasibility Pump [Fischetti al’05]
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Context
Generic LP based heuristics min{
P
j
cj xj :
P
j
aij xj ≥ bi ∀i, lj ≤ xj ≤ uj ∀j}
Rounding: Iteratively select a var xj and bound/fix it least fractional: argminj {min{xj − bxj c, dxj e − xj }} least infeasibility: argminj {up-lockj + down-lockj } where up-lockj = # constraint i with aij < 0 and down-lockj = # constraint i with aij > 0 guided search: argminj {|xj − xjinc |} least pseudo-cost: argminj {LP val increase} c greedy: argminj { P jaij } i
Diving: rounding + LP resolve + reiterate heuristic depth search in branch-and-bound tree branching rule 6= that of exact branch-and-bound sub-MIPing: rounding + MIP sol of the residual prob. C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Context
Generic LP based heuristics min{
P
j
cj xj :
P
j
aij xj ≥ bi ∀i, lj ≤ xj ≤ uj ∀j}
Rounding: Iteratively select a var xj and bound/fix it least fractional: argminj {min{xj − bxj c, dxj e − xj }} least infeasibility: argminj {up-lockj + down-lockj } where up-lockj = # constraint i with aij < 0 and down-lockj = # constraint i with aij > 0 guided search: argminj {|xj − xjinc |} least pseudo-cost: argminj {LP val increase} c greedy: argminj { P jaij } i
Diving: rounding + LP resolve + reiterate heuristic depth search in branch-and-bound tree branching rule 6= that of exact branch-and-bound sub-MIPing: rounding + MIP sol of the residual prob. C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Context
Generic LP based heuristics min{
P
j
cj xj :
P
j
aij xj ≥ bi ∀i, lj ≤ xj ≤ uj ∀j}
Rounding: Iteratively select a var xj and bound/fix it least fractional: argminj {min{xj − bxj c, dxj e − xj }} least infeasibility: argminj {up-lockj + down-lockj } where up-lockj = # constraint i with aij < 0 and down-lockj = # constraint i with aij > 0 guided search: argminj {|xj − xjinc |} least pseudo-cost: argminj {LP val increase} c greedy: argminj { P jaij } i
Diving: rounding + LP resolve + reiterate heuristic depth search in branch-and-bound tree branching rule 6= that of exact branch-and-bound sub-MIPing: rounding + MIP sol of the residual prob. C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Context
Generic LP based heuristics min{
P
j
cj xj :
P
j
aij xj ≥ bi ∀i, lj ≤ xj ≤ uj ∀j}
Rounding: Iteratively select a var xj and bound/fix it least fractional: argminj {min{xj − bxj c, dxj e − xj }} least infeasibility: argminj {up-lockj + down-lockj } where up-lockj = # constraint i with aij < 0 and down-lockj = # constraint i with aij > 0 guided search: argminj {|xj − xjinc |} least pseudo-cost: argminj {LP val increase} c greedy: argminj { P jaij } i
Diving: rounding + LP resolve + reiterate heuristic depth search in branch-and-bound tree branching rule 6= that of exact branch-and-bound sub-MIPing: rounding + MIP sol of the residual prob. C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Context
Heuristic search in branch-and-bound tree
Diving
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
sub-MIPing
Column Generation Based Primal Heuristics
Context
The Branch-and-Price Approach min
c1x 1 D x1 B x1
c2 x 2 D x2
+ +
+ +
... ...
+ +
cK x K D xK
K
B xK ∈ Nn .
B x2 .. x
1
n
∈N ,
x
2
n
∈N ,
.
...
x
≥d ≥b ≥b . ≥ .. ≥b
Relax Dx ≥ d = decomposition: subproblem {B x ≥ b, x ∈ Nn } and a reformulation solved by Branch-and-Price: P min P S∈S cx s λs s s∈S PDx λs ≥ = S∈S λs λ ∈ X k X s y := x = cx k
Solve Master LP
d K N|S| λs
S∈S
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Pricing Problem
Solve Master LP
Solve Master LP
Pricing Problem
Pricing Problem
Column Generation Based Primal Heuristics
Context
The Branch-and-Price Approach min
c1x 1 D x1 B x1
c2 x 2 D x2
+ +
+ +
... ...
+ +
cK x K D xK
K
B xK ∈ Nn .
B x2 .. x
1
n
∈N ,
x
2
n
∈N ,
.
...
x
≥d ≥b ≥b . ≥ .. ≥b
Relax Dx ≥ d = decomposition: subproblem {B x ≥ b, x ∈ Nn } and a reformulation solved by Branch-and-Price: P min P S∈S cx s λs s s∈S PDx λs ≥ = S∈S λs λ ∈ X k X s y := x = cx k
Solve Master LP
d K N|S| λs
S∈S
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Pricing Problem
Solve Master LP
Solve Master LP
Pricing Problem
Pricing Problem
Column Generation Based Primal Heuristics
Context
The Branch-and-Price Approach min
c1x 1 D x1 B x1
c2 x 2 D x2
+ +
+ +
... ...
+ +
cK x K D xK
K
B xK ∈ Nn .
B x2 .. x
1
n
∈N ,
x
2
n
∈N ,
.
...
x
≥d ≥b ≥b . ≥ .. ≥b
Relax Dx ≥ d = decomposition: subproblem {B x ≥ b, x ∈ Nn } and a reformulation solved by Branch-and-Price: P min P S∈S cx s λs s s∈S PDx λs ≥ = S∈S λs λ ∈ X k X s y := x = cx k
Solve Master LP
d K N|S| λs
S∈S
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Pricing Problem
Solve Master LP
Solve Master LP
Pricing Problem
Pricing Problem
Column Generation Based Primal Heuristics
Column Generation based Primal Heuristics
Difficulties in a B-a-P context Heuristic paradigm in original space or the reformulation? On master variables: λ Cannot fix bounds (as in rounding) Cannot define constraints (as in local branching) On original variables: x Cannot grasp individual SP var. after aggregation in the common case of identical SPs Cannot modify the SP structure required by the oracle What can be done: define a partial solution iteratively updated (lb on λ) apply col. gen. to the residual prob. (prepr. = bd x, λ)
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Column Generation based Primal Heuristics
Difficulties in a B-a-P context Heuristic paradigm in original space or the reformulation? On master variables: λ Cannot fix bounds (as in rounding) Cannot define constraints (as in local branching) On original variables: x Cannot grasp individual SP var. after aggregation in the common case of identical SPs Cannot modify the SP structure required by the oracle What can be done: define a partial solution iteratively updated (lb on λ) apply col. gen. to the residual prob. (prepr. = bd x, λ)
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Column Generation based Primal Heuristics
Difficulties in a B-a-P context Heuristic paradigm in original space or the reformulation? On master variables: λ Cannot fix bounds (as in rounding) Cannot define constraints (as in local branching) On original variables: x Cannot grasp individual SP var. after aggregation in the common case of identical SPs Cannot modify the SP structure required by the oracle What can be done: define a partial solution iteratively updated (lb on λ) apply col. gen. to the residual prob. (prepr. = bd x, λ)
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Column Generation based Primal Heuristics
Difficulties in a B-a-P context Heuristic paradigm in original space or the reformulation? On master variables: λ Cannot fix bounds (as in rounding) Cannot define constraints (as in local branching) On original variables: x Cannot grasp individual SP var. after aggregation in the common case of identical SPs Cannot modify the SP structure required by the oracle What can be done: define a partial solution iteratively updated (lb on λ) apply col. gen. to the residual prob. (prepr. = bd x, λ)
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Column Generation based Primal Heuristics
Review of column generation based heuristics 1
Restricted Master P P P min{ S cs λs : s Dx s λs ≥ d, s λs = K , λs ∈ N s ∈ S} restricted set build either heuristically / from master LP
2
Greedy iteratively select column with best “pseudo-cost / red-cost”
3
Rounding round down master LP solution + iterative round-up complete heuristically
4
Local search delete columns from incumbent rebuild heuristically
genericity & feasibility issue C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Column Generation based Primal Heuristics
Review of column generation based heuristics 1
Restricted Master P P P min{ S cs λs : s Dx s λs ≥ d, s λs = K , λs ∈ N s ∈ S} restricted set build either heuristically / from master LP
2
Greedy iteratively select column with best “pseudo-cost / red-cost”
3
Rounding round down master LP solution + iterative round-up complete heuristically
4
Local search delete columns from incumbent rebuild heuristically
genericity & feasibility issue C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Column Generation based Primal Heuristics
Review of column generation based heuristics 1
Restricted Master P P P min{ S cs λs : s Dx s λs ≥ d, s λs = K , λs ∈ N s ∈ S} restricted set build either heuristically / from master LP
2
Greedy iteratively select column with best “pseudo-cost / red-cost”
3
Rounding round down master LP solution + iterative round-up complete heuristically
4
Local search delete columns from incumbent rebuild heuristically
genericity & feasibility issue C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Column Generation based Primal Heuristics
Review of column generation based heuristics 1
Restricted Master P P P min{ S cs λs : s Dx s λs ≥ d, s λs = K , λs ∈ N s ∈ S} restricted set build either heuristically / from master LP
2
Greedy iteratively select column with best “pseudo-cost / red-cost”
3
Rounding round down master LP solution + iterative round-up complete heuristically
4
Local search delete columns from incumbent rebuild heuristically
genericity & feasibility issue C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Column Generation based Primal Heuristics
Review of column generation based heuristics 1
Restricted Master P P P min{ S cs λs : s Dx s λs ≥ d, s λs = K , λs ∈ N s ∈ S} restricted set build either heuristically / from master LP
2
Greedy iteratively select column with best “pseudo-cost / red-cost”
3
Rounding round down master LP solution + iterative round-up complete heuristically
4
Local search delete columns from incumbent rebuild heuristically
genericity & feasibility issue C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Column Generation based Primal Heuristics
Review of column generation based heuristics 1
Restricted Master (generic, often leads to infeasibility) P P P min{ S cs λs : s Dx s λs ≥ d, s λs = K , λs ∈ N s ∈ S} restricted set build either heuristically / from master LP
2
Greedy (application specific) iteratively select column with best “pseudo-cost / red-cost”
3
Rounding (application specific) round down master LP solution + iterative round-up complete heuristically
4
Local search (application specific) delete columns from incumbent rebuild heuristically
genericity & feasibility issue C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Diving Heuristic
“Diving” Heuristic: generic algorithm Generic way to restore feasibility: further column generation Step 1: Solve the current master LP. Step 2: Select columns into the current solution at ¯sc heuristically set values: λs ← dλ Step 3: If the partial solution defines a complete primal solution, record this solution and goto Step 6. Step 4: Update the master (RHS) and the pricing problems (SP var. bounds). Step 5: If residual master problem is shown infeasible through preprocessing, goto Step 6. Else, return to Step 1. Step 6: + diversification procedure: limited backtracking
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Diving Heuristic
“Diving” Heuristic: generic algorithm Generic way to restore feasibility: further column generation Step 1: Solve the current master LP. Step 2: Select columns into the current solution at ¯sc heuristically set values: λs ← dλ Step 3: If the partial solution defines a complete primal solution, record this solution and goto Step 6. Step 4: Update the master (RHS) and the pricing problems (SP var. bounds). Step 5: If residual master problem is shown infeasible through preprocessing, goto Step 6. Else, return to Step 1. Step 6: Stop. + diversification procedure: limited backtracking
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Diving Heuristic
“Diving” Heuristic: generic algorithm Generic way to restore feasibility: further column generation Step 1: Solve the current master LP. Step 2: Select columns into the current solution at ¯sc heuristically set values: λs ← dλ Step 3: If the partial solution defines a complete primal solution, record this solution and goto Step 6. Step 4: Update the master (RHS) and the pricing problems (SP var. bounds). Step 5: If residual master problem is shown infeasible through preprocessing, goto Step 6. Else, return to Step 1. Step 6: Backtrack. + diversification procedure: limited backtracking
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Diving Heuristic
“Diving” plus “diversification” Depth-first search: select column: ¯sc λs ← d λ update master and SP apply preprocessing
Backtracking: Limited Discrepancy Search
MaxDiscrepancy = 2, MaxDepth = 3 C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Diving Heuristic
Numerical tests: Cutting Stock & Vertex Coloring
2 1
3 5
problem CSP CSP VCP
size 50 80 11–211
#inst. 10 10 18
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
pure B-a-P #nodes time 245 26 417 192 17 116
4 DH w LDS gap time 0% 2 0% 14 0% 26
Column Generation Based Primal Heuristics
Diving Heuristic
Numerical tests: Bin Packing With Conflicts con
flict
conflict
conflict
20 inst. class t60 t120 t249 t501 u120 u250 u500 u1000
b-a-p not opt. av. time 0 1.7 1 156.0 2 334.2 0 245.9 0 3.4 0 23.9 1 318.0 0 1401.2
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
b-a-p + diving w LDS not opt. av. time max. time 0 1.6 7.2 0 14.9 846.3 0 32.1 172.1 0 174.9 793.5 0 3.6 36.2 0 14.6 30.9 0 74.8 136.3 0 552.8 5359.6 Column Generation Based Primal Heuristics
Diving Heuristic
Numerical tests: Bin Packing With Conflicts con
flict
conflict
conflict
class t60 t120 t249 t501 u120 u250 u500 u1000
Tabu Search gap time 0.45% 37.5 0.62% 40.0 0.39% 51.9 0.21% 58.9 0.10% 22.4 0.21% 52.1 0.20% 69.7 0.22% 107.8
Diving H gap time 0.11% 1.3 0.66% 2.5 0.35% 5.1 0.16% 10.3 0.18% 2.6 0.07% 5.4 0.03% 10.7 0.01% 21.5
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
DH LDS gap time 0% 1.5 0% 5.9 0% 32.1 0% 174.9 0% 3.3 0% 14.6 0% 74.8 0.002% 514.1
Column Generation Based Primal Heuristics
Diving Heuristic
Numerical tests: Vehicle Routing
problem VRP
size An32-80
#inst. 27
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Diving H # unsolved gap 17 5.25%
time 70
Column Generation Based Primal Heuristics
Discussion
Feedback on experimentation
Rounding/Diving based on fixing master var. λ works when 1
Sufficiently many P columns in the solution: s λs = K with K 1
2
Objective function is “flat”: many solutions of the same costs
3
Most of the combinatorial difficulty is in the subproblem
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
Discussion
Further research
Exact Branch-and-Price with heuristic oracle First experiment on CSP with greedy knapsack solver Seems no good when most of the difficulty is in the SP
Sub-MIPing Fixing original variables in aggregate space
C. Joncour, S. Michel, R. Sadykov, D. Sverdlov, F. Vanderbeck
Column Generation Based Primal Heuristics
