Approximate Potential Game Approach for Cooperative Sensor ...
InfoSymbiotics/DDDAS Conference 2016
Approximate Potential Game Approach for Cooperative Sensor Network Planning Su-Jin Lee & Han-Lim Choi Dept. of Aerospace Eng., KAIST
Laboratory for Information and Control Systems Aug. 10, 2016 Department of Aerospace Engineering
Cooperative Sensor Network Planning Allocate sensing resources to extract from environment
■ Robotic sensor targeting for weather forecast
[Choi08]
■ Wireless sensor networks for object tracking
Goal
■ Reduce uncertainty in some quantities of interest termed verification variables
Formulation
■ Maximize mutual information between verification variables and measurement variables
Sensor network Measurement variables
Target states
Environment Verification variables 2/30
Approximate Potential Game Approach for Cooperative Sensor Network Planning
1/16
Sensor Network Planning Problems Sensor 1 Sensor 𝑖𝑖‘s search space 𝒮𝒮𝑖𝑖 Sensor 𝒊𝒊’s sensing choice 𝒔𝒔𝒊𝒊
Sensor j
Verification Variables 𝐱𝐱𝒕𝒕 Sensor N
Objective
■ Find optimal sensing points in a finite space
■ Maximize uncertainty reduction in verification variables
.
sensing point for a sensor network
mutual information = difference between prior and conditioned entropy Prior entropy of verification variables
Posterior entropy conditioned on measurement
Approximate Potential Game Approach for Cooperative Sensor Network Planning
2/16
Computational Issues Computational complexity = NP-Hard
■ Combinatorial search of all possible solution candidates
Greedy approximations ■ Local greedy
Maximize information gain by its own decision Simple, decentralized Suboptimal results: ignore the coupling between agents’ decisions
■ Sequential greedy
Maximize information gain conditioned on preceding decisions Performance guarantee when global objective is submodular Not fully take advantage of possible information flows
Approximate Potential Game Approach for Cooperative Sensor Network Planning
3/16
Potential Game Approach Approach – Sensor planning as a non-cooperative game
■ New framework for analysis/synthesis of coordination of multi-agent systems[Marden09] Players(sensing agents) Actions(search space) Global objective
Potential Game
■ Local utility functions written in terms of potential function An increment of utility function
■ Pure Nash equilibrium (NE)
An increment of global objective
Always exist pure NE
Simple learning algorithms to converge to NE Approximate Potential Game Approach for Cooperative Sensor Network Planning
Desirable properties of PG 4/16
In our previous work Design of local utility function[Choi15]
■ Maximize own information gain conditioned on others’ decisions Potential game with global objective,
Learning algorithm – repeated game Repeat for 𝑖𝑖 = 1 to 𝑁𝑁 Perform local optimization at sensing agent 𝑖𝑖 Update sensing points s𝑖𝑖 end for until Convergence criteria satisfied
Update rule
■ Joint strategy fictitious play (JSFP) [Marden09]
Expected utility when other agents are assumed to play according to joint empirical frequencies.
JSFP often results in close to optimal solutions [Choi15] H.-L. Choi and S.-J. Lee, “A potential-game approach for information-maximizing cooperative planning of sensor networks,” Control systems technology, ieee transactions on, vol. 23, iss. 6, pp. 2326-2335, 2015.
Approximate Potential Game Approach for Cooperative Sensor Network Planning
5/16
Potential Game Issues Computational complexity ■ Entropy
Covariance for Gaussian: (N-1) square matrix inversion Particles for non-Gaussian: N-th integral
Communication load
■ Player needs all other agents’ decisions
Approach Approximate local utility ■ By conditioning on smaller number of variables (Gaussian) ■ By sampling to avoid multiple integrals (Particles)
Approximate Potential Game Approach for Cooperative Sensor Network Planning
6/16
Approximation by Neighbor Reduction Conditioned only on neighbors’ decisions
Neighbor set, Difference between the approximate utility and the true one
where
■ Common term
:non-neighboring agents’ sensing locations
does not affect the preference structure of the game
Sufficient Condition
■ Conditional independence between non-neighboring and verification variables conditioned on agent’s + neighbors’ decision No information in -Ni
Approximate Potential Game Approach for Cooperative Sensor Network Planning
7/16
Selection of Neighboring Agents Greedy neighbor selection algorithm for weather forecast ■ Select 𝑦𝑦 ∗ 𝑘𝑘 ∈ s−𝑁𝑁𝑖𝑖 (𝑘𝑘−1) that maximizes until the number of neighbors is 𝑛𝑛. Add to the neighbor set one by one.
Neighbor selection algorithm for agent 𝑖𝑖 s𝑁𝑁𝑖𝑖 ≔∅ s−𝑁𝑁𝑖𝑖 ≔𝒮𝒮−𝑖𝑖 for 𝑘𝑘 = 1 to 𝑛𝑛 for 𝑦𝑦 ∈ 𝑠𝑠−𝑁𝑁𝑖𝑖 do Compute end for 𝑦𝑦 ∗ = arg max 𝑒𝑒𝑦𝑦 s𝑁𝑁𝑖𝑖 ≔s𝑁𝑁𝑖𝑖 ∪ 𝑦𝑦 ∗ s−𝑁𝑁𝑖𝑖 :=s−𝑁𝑁𝑖𝑖 ∖ 𝑦𝑦 ∗ end for Approximate Potential Game Approach for Cooperative Sensor Network Planning
8/16
Numerical Example 1 Lorenz-95 weather targeting
■ Improve 2.5-day forecast by picking sensing points at 6 hrs ■ JSFP w/ approximate local utility > Sequential greedy Example cases Case
1
2
𝑁𝑁
9
9
9X6
9X6
3X2
3X2
𝒮𝒮1:𝑁𝑁 𝒮𝒮𝑖𝑖
Strategies
■ Optimal
■ Local/Sequential greedy ■ JSFP-inertia
■ JSFP-inertia approximate utility
Approximate Potential Game Approach for Cooperative Sensor Network Planning
9/16
Numerical Example 1 Lorenz-95 weather targeting
■ Larger sensor network ■ JSFP w/ approximate local utility > Sequential greedy ■ JSFP w/ approximate local utility ≈ JSFP Example cases Case
3
𝑁𝑁
15
𝒮𝒮1:𝑁𝑁 𝒮𝒮𝑖𝑖
10X9 2X3 Strategy
Global Obj.
# to converge
# of neighbors
Local greedy
2.8238
1
14
Seq. greedy
3.0218
1
14
JSFP
3.2319
12
14
JSFP-inertia
3.2236
25
14
JSFP-inertia appr
3.1731
37
7
Approximate Potential Game Approach for Cooperative Sensor Network Planning
10/16
Approximation by Sampling-Based Integration Computation of mutual information with Particle Filter Target tracking
Conditional independence
Target tracking with particle filter, ■ Local utility function
■ Approximation of Entropy with Particle Filter
Integration over
𝐱𝐱 −𝒔𝒔
𝐱𝐱𝑡𝑡
𝐱𝐱𝐬𝐬 𝐳𝐳𝐬𝐬
summation with samples
Approximate Potential Game Approach for Cooperative Sensor Network Planning
12/16
Numerical Example 2 Target Tracking with UAVs
■ One-step look ahead planning and re-planning
■ UAVs w/ range sensors Optimal behavior separating w/ equal angles
Optimality gaps for different strategies
Approximate Potential Game Approach for Cooperative Sensor Network Planning
13/16
Unified Performance Analysis Potential game, 𝒢𝒢
Approximation, 𝒢𝒢̃ Pure NE
exist ?
Approximate game 𝒢𝒢̃
■ Not aligned with global objective not potential game ■ No guarantee for the existence of NE
Existence of Near Nash Equilibria
■ If then, every Nash equilibrium of 𝒢𝒢 is an 𝜖𝜖-equilibrium of 𝒢𝒢̃ for some 𝜖𝜖 ≤ 2Δ𝑢𝑢 . �𝜖𝜖 𝒳𝒳0 ⊂ 𝒳𝒳
Near Nash equilibrium A strategy profile 𝑎𝑎� is an 𝜖𝜖-equilibrium if 𝑢𝑢𝑖𝑖 𝑎𝑎�𝑖𝑖 , 𝑎𝑎� for all 𝑖𝑖 ∈ 𝒩𝒩 and 𝑎𝑎𝑖𝑖 ∈ 𝒜𝒜𝑖𝑖 −𝑖𝑖 + 𝜖𝜖 ≥ 𝑢𝑢𝑖𝑖 (𝑎𝑎𝑖𝑖 , 𝑎𝑎� −𝑖𝑖 )
Approximate Potential Game Approach for Cooperative Sensor Network Planning
14/16
Concluding Remarks In our previous work,
■ Formulation as a potential game
■ Max information gain conditioned on others
In this work,
■ Approximation of local utility ■ Existence of Near Nash equilibrium
Future work
■ More analysis on quality of near NE
■ “How close the near NE to the optimal solution?”
Approximate Potential Game Approach for Cooperative Sensor Network Planning
15/16
Thank you.
Approximate Potential Game Approach for Cooperative Sensor Network Planning Su-Jin Lee & Han-Lim Choi Dept. of Aerospace Eng., KAIST
Laboratory for Information and Control Systems Aug. 10, 2016 Department of Aerospace Engineering
Cooperative Sensor Network Planning Allocate sensing resources to extract from environment
■ Robotic sensor targeting for weather forecast
[Choi08]
■ Wireless sensor networks for object tracking
Goal
■ Reduce uncertainty in some quantities of interest termed verification variables
Formulation
■ Maximize mutual information between verification variables and measurement variables
Sensor network Measurement variables
Target states
Environment Verification variables 2/30
Approximate Potential Game Approach for Cooperative Sensor Network Planning
1/16
Sensor Network Planning Problems Sensor 1 Sensor 𝑖𝑖‘s search space 𝒮𝒮𝑖𝑖 Sensor 𝒊𝒊’s sensing choice 𝒔𝒔𝒊𝒊
Sensor j
Verification Variables 𝐱𝐱𝒕𝒕 Sensor N
Objective
■ Find optimal sensing points in a finite space
■ Maximize uncertainty reduction in verification variables
.
sensing point for a sensor network
mutual information = difference between prior and conditioned entropy Prior entropy of verification variables
Posterior entropy conditioned on measurement
Approximate Potential Game Approach for Cooperative Sensor Network Planning
2/16
Computational Issues Computational complexity = NP-Hard
■ Combinatorial search of all possible solution candidates
Greedy approximations ■ Local greedy
Maximize information gain by its own decision Simple, decentralized Suboptimal results: ignore the coupling between agents’ decisions
■ Sequential greedy
Maximize information gain conditioned on preceding decisions Performance guarantee when global objective is submodular Not fully take advantage of possible information flows
Approximate Potential Game Approach for Cooperative Sensor Network Planning
3/16
Potential Game Approach Approach – Sensor planning as a non-cooperative game
■ New framework for analysis/synthesis of coordination of multi-agent systems[Marden09] Players(sensing agents) Actions(search space) Global objective
Potential Game
■ Local utility functions written in terms of potential function An increment of utility function
■ Pure Nash equilibrium (NE)
An increment of global objective
Always exist pure NE
Simple learning algorithms to converge to NE Approximate Potential Game Approach for Cooperative Sensor Network Planning
Desirable properties of PG 4/16
In our previous work Design of local utility function[Choi15]
■ Maximize own information gain conditioned on others’ decisions Potential game with global objective,
Learning algorithm – repeated game Repeat for 𝑖𝑖 = 1 to 𝑁𝑁 Perform local optimization at sensing agent 𝑖𝑖 Update sensing points s𝑖𝑖 end for until Convergence criteria satisfied
Update rule
■ Joint strategy fictitious play (JSFP) [Marden09]
Expected utility when other agents are assumed to play according to joint empirical frequencies.
JSFP often results in close to optimal solutions [Choi15] H.-L. Choi and S.-J. Lee, “A potential-game approach for information-maximizing cooperative planning of sensor networks,” Control systems technology, ieee transactions on, vol. 23, iss. 6, pp. 2326-2335, 2015.
Approximate Potential Game Approach for Cooperative Sensor Network Planning
5/16
Potential Game Issues Computational complexity ■ Entropy
Covariance for Gaussian: (N-1) square matrix inversion Particles for non-Gaussian: N-th integral
Communication load
■ Player needs all other agents’ decisions
Approach Approximate local utility ■ By conditioning on smaller number of variables (Gaussian) ■ By sampling to avoid multiple integrals (Particles)
Approximate Potential Game Approach for Cooperative Sensor Network Planning
6/16
Approximation by Neighbor Reduction Conditioned only on neighbors’ decisions
Neighbor set, Difference between the approximate utility and the true one
where
■ Common term
:non-neighboring agents’ sensing locations
does not affect the preference structure of the game
Sufficient Condition
■ Conditional independence between non-neighboring and verification variables conditioned on agent’s + neighbors’ decision No information in -Ni
Approximate Potential Game Approach for Cooperative Sensor Network Planning
7/16
Selection of Neighboring Agents Greedy neighbor selection algorithm for weather forecast ■ Select 𝑦𝑦 ∗ 𝑘𝑘 ∈ s−𝑁𝑁𝑖𝑖 (𝑘𝑘−1) that maximizes until the number of neighbors is 𝑛𝑛. Add to the neighbor set one by one.
Neighbor selection algorithm for agent 𝑖𝑖 s𝑁𝑁𝑖𝑖 ≔∅ s−𝑁𝑁𝑖𝑖 ≔𝒮𝒮−𝑖𝑖 for 𝑘𝑘 = 1 to 𝑛𝑛 for 𝑦𝑦 ∈ 𝑠𝑠−𝑁𝑁𝑖𝑖 do Compute end for 𝑦𝑦 ∗ = arg max 𝑒𝑒𝑦𝑦 s𝑁𝑁𝑖𝑖 ≔s𝑁𝑁𝑖𝑖 ∪ 𝑦𝑦 ∗ s−𝑁𝑁𝑖𝑖 :=s−𝑁𝑁𝑖𝑖 ∖ 𝑦𝑦 ∗ end for Approximate Potential Game Approach for Cooperative Sensor Network Planning
8/16
Numerical Example 1 Lorenz-95 weather targeting
■ Improve 2.5-day forecast by picking sensing points at 6 hrs ■ JSFP w/ approximate local utility > Sequential greedy Example cases Case
1
2
𝑁𝑁
9
9
9X6
9X6
3X2
3X2
𝒮𝒮1:𝑁𝑁 𝒮𝒮𝑖𝑖
Strategies
■ Optimal
■ Local/Sequential greedy ■ JSFP-inertia
■ JSFP-inertia approximate utility
Approximate Potential Game Approach for Cooperative Sensor Network Planning
9/16
Numerical Example 1 Lorenz-95 weather targeting
■ Larger sensor network ■ JSFP w/ approximate local utility > Sequential greedy ■ JSFP w/ approximate local utility ≈ JSFP Example cases Case
3
𝑁𝑁
15
𝒮𝒮1:𝑁𝑁 𝒮𝒮𝑖𝑖
10X9 2X3 Strategy
Global Obj.
# to converge
# of neighbors
Local greedy
2.8238
1
14
Seq. greedy
3.0218
1
14
JSFP
3.2319
12
14
JSFP-inertia
3.2236
25
14
JSFP-inertia appr
3.1731
37
7
Approximate Potential Game Approach for Cooperative Sensor Network Planning
10/16
Approximation by Sampling-Based Integration Computation of mutual information with Particle Filter Target tracking
Conditional independence
Target tracking with particle filter, ■ Local utility function
■ Approximation of Entropy with Particle Filter
Integration over
𝐱𝐱 −𝒔𝒔
𝐱𝐱𝑡𝑡
𝐱𝐱𝐬𝐬 𝐳𝐳𝐬𝐬
summation with samples
Approximate Potential Game Approach for Cooperative Sensor Network Planning
12/16
Numerical Example 2 Target Tracking with UAVs
■ One-step look ahead planning and re-planning
■ UAVs w/ range sensors Optimal behavior separating w/ equal angles
Optimality gaps for different strategies
Approximate Potential Game Approach for Cooperative Sensor Network Planning
13/16
Unified Performance Analysis Potential game, 𝒢𝒢
Approximation, 𝒢𝒢̃ Pure NE
exist ?
Approximate game 𝒢𝒢̃
■ Not aligned with global objective not potential game ■ No guarantee for the existence of NE
Existence of Near Nash Equilibria
■ If then, every Nash equilibrium of 𝒢𝒢 is an 𝜖𝜖-equilibrium of 𝒢𝒢̃ for some 𝜖𝜖 ≤ 2Δ𝑢𝑢 . �𝜖𝜖 𝒳𝒳0 ⊂ 𝒳𝒳
Near Nash equilibrium A strategy profile 𝑎𝑎� is an 𝜖𝜖-equilibrium if 𝑢𝑢𝑖𝑖 𝑎𝑎�𝑖𝑖 , 𝑎𝑎� for all 𝑖𝑖 ∈ 𝒩𝒩 and 𝑎𝑎𝑖𝑖 ∈ 𝒜𝒜𝑖𝑖 −𝑖𝑖 + 𝜖𝜖 ≥ 𝑢𝑢𝑖𝑖 (𝑎𝑎𝑖𝑖 , 𝑎𝑎� −𝑖𝑖 )
Approximate Potential Game Approach for Cooperative Sensor Network Planning
14/16
Concluding Remarks In our previous work,
■ Formulation as a potential game
■ Max information gain conditioned on others
In this work,
■ Approximation of local utility ■ Existence of Near Nash equilibrium
Future work
■ More analysis on quality of near NE
■ “How close the near NE to the optimal solution?”
Approximate Potential Game Approach for Cooperative Sensor Network Planning
15/16
Thank you.
