Low-Complexity Approaches to Spectrum ... - Semantic Scholar
c °Zhao & Krishnamachari & Liu. CrownCom, August, 2007.
Low-Complexity Approaches to Spectrum Opportunity Tracking Qing Zhao∗, Bhaskar Krishnamachari¦, and Keqin Liu∗ ∗ ¦
University of California at Davis
University of Southern California
Supported by NSF and ARL-CTA.
1
c °Zhao & Krishnamachari & Liu. CrownCom, August, 2007.
2
Opportunistic Spectrum Access Pervasive Opportunities
Crowded Spectrum −2
10
signal [V]
−3
10
−4
10
0
0.01
0.02
0.03
0.04
time [s]
Three Basic Components: Spectrum Sensor: opportunity identification (PHY) I Sensing policy: where to sense for opportunity tracking (MAC) I Access policy: whether to tx given that sensing errors may occur (MAC) I
A joint PHY-MAC design is necessary for optimality.
0.05
c °Zhao & Krishnamachari & Liu. CrownCom, August, 2007.
3
Outline I
Network model
I
Joint design of spectrum sensor, sensing policy, and access policy 2A
constrained POMDP formulation
2 The
separation principle:
1. Spectrum sensor and access policy: myopic design in closed form 2. Sensing policy: an unconstrained POMDP I
Low-complexity sensing policies for opportunity tracking 2 Structure
and optimality of myopic sensing policy
2 Low-complexity I
index policies
Conclusion
Y. Chen, Q. Zhao, and A. Swami, “Joint Design and Separation Principle for Opportunistic Spectrum Access in the Presence of Sensing Errors,” submitted to IEEE Transactions on Information Theory, Feb. 2007; available at http://arxiv.org/PS cache/cs/pdf/0702/0702158.pdf.
A primary slotted network. I
Markovian spectrum usage (2N states). I
Usage statistics unchanged for T slots. (1, 1)
channels, each with bandwidth Bi. IN
SN (T ) = 0 SN (1)replacements = 1 SN (2) = 0 SN (3) = 0 PSfrag
t T 3 2 1 0
I
(1, 0)
Channel N
t S1 (T ) = 0 S1 (3) = 0 S1 (2) = 1 S1 (1) = 0
(0, 1) (0, 0)
Channel 1
4 c °Zhao & Krishnamachari & Liu. CrownCom, August, 2007.
Network Model
Opportunities
SN (T ) = 0 SN (1) = 1 SN (2) = 0 SN (3) = 0
t T 3
t S1 (T ) = 0 S1 (3) = 0 S1 (2) = 1 S1 (1) = 0
2 1 0
Channel N
Channel 1
5 c °Zhao & Krishnamachari & Liu. CrownCom, August, 2007.
Network Model
Opportunities
I
Secondary users search for opportunities independently. I
Can sense and access only one channel in each slot. I
A successful transmission is acknowledged at the end of the slot.
c °Zhao & Krishnamachari & Liu. CrownCom, August, 2007.
6
Spectrum Sensor at PHY Binary Hypotheses Test: H0
(idle)
vs. H1 (busy)
Two Types of Sensing Errors: ∆
prob. of overlook
∆
prob. of misidentification
I
opportunity overlook: H0 → H1
²=
I
opportunity misidentification: H1 → H0
δ=
Receiver Operating Characteristics (ROC): 1 − δ vs. ² Probability of Detection 1 − δ
1 0.9
δ
ε
0.8 0.7
Which point δ to operate at?
0.6 0.5 0.4
overlook vs. misidentification
0.3 0.2 0.1 0
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
Probability of False Alarm ε
0.9
1
c °Zhao & Krishnamachari & Liu. CrownCom, August, 2007.
7
Spectrum Sensor at PHY Binary Hypotheses Test: H0
(idle)
vs. H1 (busy)
Two Types of Sensing Errors: ∆
prob. of overlook
∆
prob. of misidentification
I
opportunity overlook: H0 → H1
²=
I
opportunity misidentification: H1 → H0
δ=
Receiver Operating Characteristics (ROC): 1 − δ vs. ² Probability of Detection 1 − δ
1 0.9
δ
ε
0.8 0.7
Which point δ to operate at?
0.6 0.5 0.4
overlook vs. misidentification
0.3 0.2 0.1 0
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
Probability of False Alarm ε
0.9
1
c °Zhao & Krishnamachari & Liu. CrownCom, August, 2007.
8
Sensing and Access Policies Opportunities !
!
!
!
!
! !
!
!
!
!
! !
!
!
!
!
! !
!
!
!
!
!
!
!
%
$
$
%
!
!
%
$
S1 (T ) = 0
"
# "
#
"
"
#
#
"
#
"
#
"
"
#
#
"
3
#
"
#
"
"
#
#
"
#
"
#
"
"
#
#
"
#
#
"
#
"
"
"
#
#
#
"
"
#
"
#
#
"
"
#
"
#
"
#
"
#
"
#
"
#
SN (1) = 1 SN (2) = 0 SN (3) = 0
"
&
'
2
#
&
'
&
&
'
'
&
'
&
'
&
&
'
'
&
'
&
'
&
&
'
'
&
'
&
'
&
&
'
'
&
'
&
'
&
&
'
'
&
'
&
'
&
&
'
'
&
'
&
'
&
&
'
'
&
'
&
'
&
'
&
'
1
0
Channel N
!
!
%
$
S1 (1) = 0 S1 (2) = 1 S1 (3) = 0
!
!
%
$
$
%
$
$
%
%
$
%
$
%
$
$
%
%
$
%
$
%
$
$
%
%
$
%
$
%
$
$
%
%
$
%
$
%
$
$
%
%
$
%
$
%
$
%
$
%
$
%
$
%
$
%
$
(
)
%
(
)
(
(
)
)
(
)
(
)
(
(
)
)
(
)
(
)
(
(
)
)
(
)
(
)
(
(
)
)
(
)
(
)
(
(
)
)
(
)
(
)
(
(
)
)
(
)
(
)
(
)
(
)
(
)
(
)
(
)
(
)
Channel 1
T
t
t
SN (T ) = 0
Sensing Policy πs 2 Deterministic: choose the sensing action a in each slot 2 Randomized:
choose PMF of the sensing action in each slot
Access Policy πc 2 Deterministic: whether to transmit based on sensing outcome 2 Randomized:
transmission probability based on sensing outcome
c °Zhao & Krishnamachari & Liu. CrownCom, August, 2007.
9
Sensing and Access Policies Opportunities .
/ .
/ .
/
.
/
.
.
/
/
.
/
.
/
.
.
/
/
.
/
.
/
.
.
/
/
.
/
.
/
.
.
/
/
.
/
.
/
/
.
.
/
.
/
/
.
.
/
.
/
.
/
3
.
/
S1 (T ) = 0
0
1
0
0
1
1
0
1
SN (1) = 1 SN (2) = 0 SN (3) = 0
0
3
2
1
0
1
0
0
1
1
0
1
0
1
0
0
1
1
0
1
0
1
0
0
1
1
0
1
1
0
1
0
0
0
1
1
1
0
0
1
0
1
1
0
0
1
0
1
0
1
0
1
0
1
0
1
0
4
5
1
4
5
4
4
5
5
4
5
4
5
4
4
5
5
4
5
4
5
4
4
5
5
4
5
4
5
4
4
5
5
4
5
4
5
4
4
5
5
4
5
4
5
4
4
5
5
4
5
4
5
4
4
5
5
4
5
4
5
*
4
5
+
4
5
*
+
*
*
+
+
*
+
*
+
*
*
+
+
*
+
*
+
*
*
+
+
*
+
*
+
*
*
+
+
*
1
+
*
+
*
*
+
+
*
+
*
+
*
*
+
+
*
+
*
+
*
+
*
+
*
+
*
+
*
+
*
0
+
Channel N
.
S1 (1) = 0 S1 (2) = 1 S1 (3) = 0
/
.
/
2
.
2
3
/
3
2
.
3
2
/
3
2
2
3
2
2
3
3
2
3
2
3
2
2
3
3
2
3
2
3
2
2
3
3
2
3
2
3
2
2
3
3
2
3
2
3
2
2
3
3
2
3
2
3
2
3
2
3
2
3
2
3
2
-
,
-
,
-
,
-
,
-
,
-
,
-
,
-
,
3
-
,
-
,
-
,
-
,
-
,
-
,
-
,
-
,
2
-
,
-
,
-
,
-
,
-
,
-
,
-
,
-
,
6
7
3
-
,
-
,
-
,
-
,
-
,
-
,
-
,
-
,
6
7
6
6
7
7
6
7
6
7
6
6
7
7
6
7
6
7
6
6
7
7
6
7
6
7
6
6
7
7
6
7
6
7
6
6
7
7
6
7
6
7
6
6
7
7
6
7
6
7
6
7
6
7
6
7
6
7
6
7
6
7
Channel 1
T
t
t
SN (T ) = 0
Reward and Collision 2A
reward R(t) = Ba is accrued when access an idle channel
2A
collision with primary users occurs when access a busy channel
Objective: choose sensor operating policy πδ , sensing policy πs, access policy πc max E[
T X t=1
R(t)] s.t.
collision probability Pc ≤ ζ
c °Zhao & Krishnamachari & Liu. CrownCom, August, 2007.
10
A Constrained POMDP Formulation
{πδ∗, πs∗, πc∗}
= arg max E[
T X
R(t)] s.t.
collision probability Pc ≤ ζ
t=1
I
A constrained Partially Observable Markov Decision Process (POMDP)
I
Often requires randomized policy for optimality
I
Analytically intractable
I
Approximate numerical solutions provide little insight
c °Zhao & Krishnamachari & Liu. CrownCom, August, 2007.
11
Separation Principle
{πδ∗, πs∗, πc∗}
= arg max E[
T X
R(t)],
subject to Pc ≤ ζ
t=1
Separation principle: π δ and π c can be decoupled from π s I
Choose sensor operating policy π δ and access policy π c
I
to maximize immediate reward R(t) under constraint Pc = ζ .
I =⇒
Static optimization problem
I =⇒
Deterministic policy {δ ∗, π ∗c } in closed form.
I
Choose sensing policy π s to maximize total reward E
I =⇒
One unconstrained POMDP
I =⇒
Optimality achieved with deterministic policies.
hP
T t=1 R(t)
i
.
c °Zhao & Krishnamachari & Liu. CrownCom, August, 2007.
12
Separation Principle
{πδ∗, πs∗, πc∗}
= arg max E[
T X
R(t)],
subject to Pc ≤ ζ
t=1
Separation principle: π δ and π c can be decoupled from π s I
Choose sensor operating policy π δ and access policy π c
I
to maximize immediate reward R(t) under constraint Pc = ζ .
I =⇒
Myopic is optimal: static optimization
I =⇒
Deterministic policies {π ∗δ , π ∗c } in closed form.
I =⇒ π ∗δ : δ ∗ = ζ ; I
π ∗c :
trust the sensor.
Choose sensing policy π s to maximize total reward E
I =⇒
An unconstrained POMDP
I =⇒
Optimality achieved with deterministic policies.
hP
T t=1 R(t)
i
.
c °Zhao & Krishnamachari & Liu. CrownCom, August, 2007.
13
The Optimal Sensing Policy for Opportunity Tracking πs∗ = arg max E[
T X
R(t)]
t=1
a(t) Max total remaining reward R(t) 0
K(1)
K(2)
···
K(t − 1) t
R(t + 1)
···
K(t) t + 1
Use entire observation history I
Use common observations (ACK) to ensure Tx-Rx synchronous hopping.
I
Computational complexity: O(N T )
I
Due to continuously growing observation history and foresighted planning
T
PSfrag replacements14
c °Zhao & Krishnamachari & Liu. CrownCom, August, 2007.
The Myopic Sensing Policy πs∗ = arg max E[
T X
R(t)]
t=1
a(t) Max R(t) 0
K(1)
K(2)
···
K(t − 1) t
K(t) t + 1
Use entire observation history I
Myopic policy: maximize immediate reward R(t) a(t) = arg max Ra(t) a=1,··· ,N
= arg max Pr[a a=1,··· ,N
is idle | K(1), · · · , K(t − 1) ]Ba {z } | common observ.
T
0 1
c °Zhao & Krishnamachari & Liu. CrownCom, August, 2007.
(busy) (idle) Structure of Myopic Sensing for i.i.d. Markov Processes PSfrag replacements p0,1 1−δ
optimal (δ ∗ = ζp)1,1 p0,0
1−ζ
p1,0
p0,1
p0,0
0
1
(busy)
(idle)
p1,1
²∗
p1,0
p0,1 The Structure of Myopic Sensing Policy: p1,1 > p0,1 and ²∗ < pp1,0 1,1 p0,0
Stay in the same channel after ACK and switch after NAK. I Switch to the channel visited the longest time ago. I A sufficient statistic: current acknowledgement. PSfrag replacements I
NAK
Ch 1
NAK
Ch 2 NAK
Ch 3
²
15
0 1
c °Zhao & Krishnamachari & Liu. CrownCom, August, 2007.
16
(busy) (idle) Structure of Myopic Sensing for i.i.d. Markov Processes PSfrag replacements p0,1 1−δ
optimal (δ ∗ = ζp)1,1 p0,0
1−ζ
p1,0
p0,1
p0,0
0
1
(busy)
(idle) p1,0
p1,1
²∗
²
p1,1 The Structure of Myopic Sensing Policy: p0,1 > p1,1 and ²∗ < pp0,0 0,1 p1,0
I
Stay in the same channel after a NAK and switch after an ACK.
I
Among channels visited an even number of slots ago, choose the most recent.
I
If no such channels, choose the one visited the longest time ago.
I
A sufficient statistic: current acknowledgement and last visit to each channel.
c °Zhao & Krishnamachari & Liu. CrownCom, August, 2007.
17
PSfrag Optimality of Myopic Sensing for i.i.d.replacements Markov Processes p0,1
p0,0
0
1
(busy)
(idle) p1,0
The Optimality of Myopic Sensing Policy I
Proven to be optimal for N = 2
I
Numerical results indicate its optimality for N > 2.
p1,1
c °Zhao & Krishnamachari & Liu. CrownCom, August, 2007.
18
Throughput(bits per slot)
A Numerical Example
0.65 0.6 Optimal Myopic Random
0.55 0.5
Throughput(bits per slot)
0
5
Time Slot
10
15
0.54 0.53 0.52 0.51
Optimal Myopic Random
0.5 0.49 0.48
0
5
Time Slot
10
15
Cognitive: improved performance due to increasingly accurate state I information drawn from accumulating observations. I
c °Zhao & Krishnamachari & Liu. CrownCom, August, 2007.
19
Conclusion Three Basic Components:
Spectrum Sensor: opportunity identification (PHY) I Sensing policy: where to sense for opportunity tracking (MAC) I Access policy: whether to tx given that sensing errors may occur (MAC)
I
The Separation Principle:
Spectrum sensor and access policy: myopic design in closed form I Sensing policy: an unconstrained POMDP I
Low-complexity Sensing:
Structure and optimality of myopic sensing for i.i.d. Markov channels I Low-complexity index policies for non-i.i.d. channels I
Y. Chen, Q. Zhao, and A. Swami, “Joint Design and Separation Principle for Opportunistic Spectrum Access in the Presence of Sensing Errors,” submitted to IEEE Transactions on Information Theory, Feb. 2007; available at http://arxiv.org/PS cache/cs/pdf/0702/0702158.pdf.
Low-Complexity Approaches to Spectrum Opportunity Tracking Qing Zhao∗, Bhaskar Krishnamachari¦, and Keqin Liu∗ ∗ ¦
University of California at Davis
University of Southern California
Supported by NSF and ARL-CTA.
1
c °Zhao & Krishnamachari & Liu. CrownCom, August, 2007.
2
Opportunistic Spectrum Access Pervasive Opportunities
Crowded Spectrum −2
10
signal [V]
−3
10
−4
10
0
0.01
0.02
0.03
0.04
time [s]
Three Basic Components: Spectrum Sensor: opportunity identification (PHY) I Sensing policy: where to sense for opportunity tracking (MAC) I Access policy: whether to tx given that sensing errors may occur (MAC) I
A joint PHY-MAC design is necessary for optimality.
0.05
c °Zhao & Krishnamachari & Liu. CrownCom, August, 2007.
3
Outline I
Network model
I
Joint design of spectrum sensor, sensing policy, and access policy 2A
constrained POMDP formulation
2 The
separation principle:
1. Spectrum sensor and access policy: myopic design in closed form 2. Sensing policy: an unconstrained POMDP I
Low-complexity sensing policies for opportunity tracking 2 Structure
and optimality of myopic sensing policy
2 Low-complexity I
index policies
Conclusion
Y. Chen, Q. Zhao, and A. Swami, “Joint Design and Separation Principle for Opportunistic Spectrum Access in the Presence of Sensing Errors,” submitted to IEEE Transactions on Information Theory, Feb. 2007; available at http://arxiv.org/PS cache/cs/pdf/0702/0702158.pdf.
A primary slotted network. I
Markovian spectrum usage (2N states). I
Usage statistics unchanged for T slots. (1, 1)
channels, each with bandwidth Bi. IN
SN (T ) = 0 SN (1)replacements = 1 SN (2) = 0 SN (3) = 0 PSfrag
t T 3 2 1 0
I
(1, 0)
Channel N
t S1 (T ) = 0 S1 (3) = 0 S1 (2) = 1 S1 (1) = 0
(0, 1) (0, 0)
Channel 1
4 c °Zhao & Krishnamachari & Liu. CrownCom, August, 2007.
Network Model
Opportunities
SN (T ) = 0 SN (1) = 1 SN (2) = 0 SN (3) = 0
t T 3
t S1 (T ) = 0 S1 (3) = 0 S1 (2) = 1 S1 (1) = 0
2 1 0
Channel N
Channel 1
5 c °Zhao & Krishnamachari & Liu. CrownCom, August, 2007.
Network Model
Opportunities
I
Secondary users search for opportunities independently. I
Can sense and access only one channel in each slot. I
A successful transmission is acknowledged at the end of the slot.
c °Zhao & Krishnamachari & Liu. CrownCom, August, 2007.
6
Spectrum Sensor at PHY Binary Hypotheses Test: H0
(idle)
vs. H1 (busy)
Two Types of Sensing Errors: ∆
prob. of overlook
∆
prob. of misidentification
I
opportunity overlook: H0 → H1
²=
I
opportunity misidentification: H1 → H0
δ=
Receiver Operating Characteristics (ROC): 1 − δ vs. ² Probability of Detection 1 − δ
1 0.9
δ
ε
0.8 0.7
Which point δ to operate at?
0.6 0.5 0.4
overlook vs. misidentification
0.3 0.2 0.1 0
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
Probability of False Alarm ε
0.9
1
c °Zhao & Krishnamachari & Liu. CrownCom, August, 2007.
7
Spectrum Sensor at PHY Binary Hypotheses Test: H0
(idle)
vs. H1 (busy)
Two Types of Sensing Errors: ∆
prob. of overlook
∆
prob. of misidentification
I
opportunity overlook: H0 → H1
²=
I
opportunity misidentification: H1 → H0
δ=
Receiver Operating Characteristics (ROC): 1 − δ vs. ² Probability of Detection 1 − δ
1 0.9
δ
ε
0.8 0.7
Which point δ to operate at?
0.6 0.5 0.4
overlook vs. misidentification
0.3 0.2 0.1 0
0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
Probability of False Alarm ε
0.9
1
c °Zhao & Krishnamachari & Liu. CrownCom, August, 2007.
8
Sensing and Access Policies Opportunities !
!
!
!
!
! !
!
!
!
!
! !
!
!
!
!
! !
!
!
!
!
!
!
!
%
$
$
%
!
!
%
$
S1 (T ) = 0
"
# "
#
"
"
#
#
"
#
"
#
"
"
#
#
"
3
#
"
#
"
"
#
#
"
#
"
#
"
"
#
#
"
#
#
"
#
"
"
"
#
#
#
"
"
#
"
#
#
"
"
#
"
#
"
#
"
#
"
#
"
#
SN (1) = 1 SN (2) = 0 SN (3) = 0
"
&
'
2
#
&
'
&
&
'
'
&
'
&
'
&
&
'
'
&
'
&
'
&
&
'
'
&
'
&
'
&
&
'
'
&
'
&
'
&
&
'
'
&
'
&
'
&
&
'
'
&
'
&
'
&
&
'
'
&
'
&
'
&
'
&
'
1
0
Channel N
!
!
%
$
S1 (1) = 0 S1 (2) = 1 S1 (3) = 0
!
!
%
$
$
%
$
$
%
%
$
%
$
%
$
$
%
%
$
%
$
%
$
$
%
%
$
%
$
%
$
$
%
%
$
%
$
%
$
$
%
%
$
%
$
%
$
%
$
%
$
%
$
%
$
%
$
(
)
%
(
)
(
(
)
)
(
)
(
)
(
(
)
)
(
)
(
)
(
(
)
)
(
)
(
)
(
(
)
)
(
)
(
)
(
(
)
)
(
)
(
)
(
(
)
)
(
)
(
)
(
)
(
)
(
)
(
)
(
)
(
)
Channel 1
T
t
t
SN (T ) = 0
Sensing Policy πs 2 Deterministic: choose the sensing action a in each slot 2 Randomized:
choose PMF of the sensing action in each slot
Access Policy πc 2 Deterministic: whether to transmit based on sensing outcome 2 Randomized:
transmission probability based on sensing outcome
c °Zhao & Krishnamachari & Liu. CrownCom, August, 2007.
9
Sensing and Access Policies Opportunities .
/ .
/ .
/
.
/
.
.
/
/
.
/
.
/
.
.
/
/
.
/
.
/
.
.
/
/
.
/
.
/
.
.
/
/
.
/
.
/
/
.
.
/
.
/
/
.
.
/
.
/
.
/
3
.
/
S1 (T ) = 0
0
1
0
0
1
1
0
1
SN (1) = 1 SN (2) = 0 SN (3) = 0
0
3
2
1
0
1
0
0
1
1
0
1
0
1
0
0
1
1
0
1
0
1
0
0
1
1
0
1
1
0
1
0
0
0
1
1
1
0
0
1
0
1
1
0
0
1
0
1
0
1
0
1
0
1
0
1
0
4
5
1
4
5
4
4
5
5
4
5
4
5
4
4
5
5
4
5
4
5
4
4
5
5
4
5
4
5
4
4
5
5
4
5
4
5
4
4
5
5
4
5
4
5
4
4
5
5
4
5
4
5
4
4
5
5
4
5
4
5
*
4
5
+
4
5
*
+
*
*
+
+
*
+
*
+
*
*
+
+
*
+
*
+
*
*
+
+
*
+
*
+
*
*
+
+
*
1
+
*
+
*
*
+
+
*
+
*
+
*
*
+
+
*
+
*
+
*
+
*
+
*
+
*
+
*
+
*
0
+
Channel N
.
S1 (1) = 0 S1 (2) = 1 S1 (3) = 0
/
.
/
2
.
2
3
/
3
2
.
3
2
/
3
2
2
3
2
2
3
3
2
3
2
3
2
2
3
3
2
3
2
3
2
2
3
3
2
3
2
3
2
2
3
3
2
3
2
3
2
2
3
3
2
3
2
3
2
3
2
3
2
3
2
3
2
-
,
-
,
-
,
-
,
-
,
-
,
-
,
-
,
3
-
,
-
,
-
,
-
,
-
,
-
,
-
,
-
,
2
-
,
-
,
-
,
-
,
-
,
-
,
-
,
-
,
6
7
3
-
,
-
,
-
,
-
,
-
,
-
,
-
,
-
,
6
7
6
6
7
7
6
7
6
7
6
6
7
7
6
7
6
7
6
6
7
7
6
7
6
7
6
6
7
7
6
7
6
7
6
6
7
7
6
7
6
7
6
6
7
7
6
7
6
7
6
7
6
7
6
7
6
7
6
7
6
7
Channel 1
T
t
t
SN (T ) = 0
Reward and Collision 2A
reward R(t) = Ba is accrued when access an idle channel
2A
collision with primary users occurs when access a busy channel
Objective: choose sensor operating policy πδ , sensing policy πs, access policy πc max E[
T X t=1
R(t)] s.t.
collision probability Pc ≤ ζ
c °Zhao & Krishnamachari & Liu. CrownCom, August, 2007.
10
A Constrained POMDP Formulation
{πδ∗, πs∗, πc∗}
= arg max E[
T X
R(t)] s.t.
collision probability Pc ≤ ζ
t=1
I
A constrained Partially Observable Markov Decision Process (POMDP)
I
Often requires randomized policy for optimality
I
Analytically intractable
I
Approximate numerical solutions provide little insight
c °Zhao & Krishnamachari & Liu. CrownCom, August, 2007.
11
Separation Principle
{πδ∗, πs∗, πc∗}
= arg max E[
T X
R(t)],
subject to Pc ≤ ζ
t=1
Separation principle: π δ and π c can be decoupled from π s I
Choose sensor operating policy π δ and access policy π c
I
to maximize immediate reward R(t) under constraint Pc = ζ .
I =⇒
Static optimization problem
I =⇒
Deterministic policy {δ ∗, π ∗c } in closed form.
I
Choose sensing policy π s to maximize total reward E
I =⇒
One unconstrained POMDP
I =⇒
Optimality achieved with deterministic policies.
hP
T t=1 R(t)
i
.
c °Zhao & Krishnamachari & Liu. CrownCom, August, 2007.
12
Separation Principle
{πδ∗, πs∗, πc∗}
= arg max E[
T X
R(t)],
subject to Pc ≤ ζ
t=1
Separation principle: π δ and π c can be decoupled from π s I
Choose sensor operating policy π δ and access policy π c
I
to maximize immediate reward R(t) under constraint Pc = ζ .
I =⇒
Myopic is optimal: static optimization
I =⇒
Deterministic policies {π ∗δ , π ∗c } in closed form.
I =⇒ π ∗δ : δ ∗ = ζ ; I
π ∗c :
trust the sensor.
Choose sensing policy π s to maximize total reward E
I =⇒
An unconstrained POMDP
I =⇒
Optimality achieved with deterministic policies.
hP
T t=1 R(t)
i
.
c °Zhao & Krishnamachari & Liu. CrownCom, August, 2007.
13
The Optimal Sensing Policy for Opportunity Tracking πs∗ = arg max E[
T X
R(t)]
t=1
a(t) Max total remaining reward R(t) 0
K(1)
K(2)
···
K(t − 1) t
R(t + 1)
···
K(t) t + 1
Use entire observation history I
Use common observations (ACK) to ensure Tx-Rx synchronous hopping.
I
Computational complexity: O(N T )
I
Due to continuously growing observation history and foresighted planning
T
PSfrag replacements14
c °Zhao & Krishnamachari & Liu. CrownCom, August, 2007.
The Myopic Sensing Policy πs∗ = arg max E[
T X
R(t)]
t=1
a(t) Max R(t) 0
K(1)
K(2)
···
K(t − 1) t
K(t) t + 1
Use entire observation history I
Myopic policy: maximize immediate reward R(t) a(t) = arg max Ra(t) a=1,··· ,N
= arg max Pr[a a=1,··· ,N
is idle | K(1), · · · , K(t − 1) ]Ba {z } | common observ.
T
0 1
c °Zhao & Krishnamachari & Liu. CrownCom, August, 2007.
(busy) (idle) Structure of Myopic Sensing for i.i.d. Markov Processes PSfrag replacements p0,1 1−δ
optimal (δ ∗ = ζp)1,1 p0,0
1−ζ
p1,0
p0,1
p0,0
0
1
(busy)
(idle)
p1,1
²∗
p1,0
p0,1 The Structure of Myopic Sensing Policy: p1,1 > p0,1 and ²∗ < pp1,0 1,1 p0,0
Stay in the same channel after ACK and switch after NAK. I Switch to the channel visited the longest time ago. I A sufficient statistic: current acknowledgement. PSfrag replacements I
NAK
Ch 1
NAK
Ch 2 NAK
Ch 3
²
15
0 1
c °Zhao & Krishnamachari & Liu. CrownCom, August, 2007.
16
(busy) (idle) Structure of Myopic Sensing for i.i.d. Markov Processes PSfrag replacements p0,1 1−δ
optimal (δ ∗ = ζp)1,1 p0,0
1−ζ
p1,0
p0,1
p0,0
0
1
(busy)
(idle) p1,0
p1,1
²∗
²
p1,1 The Structure of Myopic Sensing Policy: p0,1 > p1,1 and ²∗ < pp0,0 0,1 p1,0
I
Stay in the same channel after a NAK and switch after an ACK.
I
Among channels visited an even number of slots ago, choose the most recent.
I
If no such channels, choose the one visited the longest time ago.
I
A sufficient statistic: current acknowledgement and last visit to each channel.
c °Zhao & Krishnamachari & Liu. CrownCom, August, 2007.
17
PSfrag Optimality of Myopic Sensing for i.i.d.replacements Markov Processes p0,1
p0,0
0
1
(busy)
(idle) p1,0
The Optimality of Myopic Sensing Policy I
Proven to be optimal for N = 2
I
Numerical results indicate its optimality for N > 2.
p1,1
c °Zhao & Krishnamachari & Liu. CrownCom, August, 2007.
18
Throughput(bits per slot)
A Numerical Example
0.65 0.6 Optimal Myopic Random
0.55 0.5
Throughput(bits per slot)
0
5
Time Slot
10
15
0.54 0.53 0.52 0.51
Optimal Myopic Random
0.5 0.49 0.48
0
5
Time Slot
10
15
Cognitive: improved performance due to increasingly accurate state I information drawn from accumulating observations. I
c °Zhao & Krishnamachari & Liu. CrownCom, August, 2007.
19
Conclusion Three Basic Components:
Spectrum Sensor: opportunity identification (PHY) I Sensing policy: where to sense for opportunity tracking (MAC) I Access policy: whether to tx given that sensing errors may occur (MAC)
I
The Separation Principle:
Spectrum sensor and access policy: myopic design in closed form I Sensing policy: an unconstrained POMDP I
Low-complexity Sensing:
Structure and optimality of myopic sensing for i.i.d. Markov channels I Low-complexity index policies for non-i.i.d. channels I
Y. Chen, Q. Zhao, and A. Swami, “Joint Design and Separation Principle for Opportunistic Spectrum Access in the Presence of Sensing Errors,” submitted to IEEE Transactions on Information Theory, Feb. 2007; available at http://arxiv.org/PS cache/cs/pdf/0702/0702158.pdf.
