CS 188: ArVficial Intelligence Bayes' Nets: Sampling Bayes' Net ...
Bayes Net Representa)on
CS 188: Ar)ficial Intelligence
Bayes’ Nets: Sampling
§ A directed, acyclic graph, one node per random variable § A condi)onal probability table (CPT) for each node § A collec)on of distribu)ons over X, one for each combina)on of parents values
§ Bayes nets implicitly encode joint distribu)ons § As a product of local condi)onal distribu)ons § To see what probability a BN gives to a full assignment, mul)ply all the relevant condi)onals together: Pieter Abbeel and Dan Klein University of California, Berkeley
Example: Alarm Network B
P(B)
+b 0.001 -‐b
B
E
0.999
A
E
P(E)
B
+b 0.001
-‐e
-‐b
0.998
A
J
P(J|A)
A
M
P(M|A)
+j
0.9
+a
+m
0.7
+a
-‐j
0.1
+a
-‐m
0.3
-‐a
+j
0.05
-‐a
+m
0.01
-‐a
-‐j
0.95
-‐a
-‐m
0.99
M
B
E
A
P(A|B,E)
+b
+e
+a
0.95
+b
+e
-‐a
0.05
+b
-‐e
+a
0.94
+b
-‐e
-‐a
0.06
-‐b
+e
+a
-‐b
+e
-‐b -‐b
E
0.999
E
-‐e
A
P(E)
+e 0.002 0.998
A
J
P(J|A)
A
M
P(M|A)
+a
+j
0.9
+a
+m
0.7
+a
-‐j
0.1
+a
-‐m
0.3
-‐a
+j
0.05
-‐a
+m
0.01
-‐a
-‐j
0.95
-‐a
-‐m
0.99
B
E
A
P(A|B,E)
+e
+a
0.95
+b
+e
-‐a
0.05
+b
-‐e
+a
0.94
+b
-‐e
-‐a
0.06
0.29
-‐b
+e
+a
0.29
-‐a
0.71
-‐b
+e
-‐a
0.71
-‐e
+a
0.001
-‐b
-‐e
+a
0.001
-‐e
-‐a
0.999
-‐b
-‐e
-‐a
0.999
J
M
Approximate Inference: Sampling
§ Interleave joining and marginalizing § dk entries computed for a factor over k variables with domain sizes d
§ Worst case: running )me exponen)al in the size of the Bayes’ net
B
+b
Variable Elimina)on
§ Ordering of elimina)on of hidden variables can affect size of factors generated
P(B)
+e 0.002
+a
J
Example: Alarm Network
… …
Sampling § Sampling is a lot like repeated simula)on
Sampling
§ Why sample? § Learning: get samples from a distribu)on you don’t know
§ Predic)ng the weather, basketball games, …
§ Basic idea
§ Inference: geeng a sample is faster than compu)ng the right answer (e.g. with variable elimina)on)
§ Draw N samples from a sampling distribu)on S § Compute an approximate posterior probability § Show this converges to the true probability P
Sampling in Bayes’ Nets
§ Sampling from given distribu)on
§ Example
§ Step 1: Get sample u from uniform distribu)on over [0, 1) § E.g. random() in python
§ Step 2: Convert this sample u into an outcome for the given distribu)on by having each outcome associated with a sub-‐interval of [0,1) with sub-‐interval size equal to probability of the outcome
C red
P(C) 0.6
green
0.1
blue
0.3
§ If random() returns u = 0.83, then our sample is C = blue § E.g, ajer sampling 8 )mes:
Prior Sampling
§ Prior Sampling § Rejec)on Sampling § Likelihood Weigh)ng § Gibbs Sampling
Prior Sampling +c -‐c
Prior Sampling § For i=1, 2, …, n
0.5 0.5
§ Sample xi from P(Xi | Parents(Xi))
Cloudy +c +s 0.1 -‐s 0.9 -‐c +s 0.5 -‐s 0.5
+s -‐s
+r -‐r +r -‐r
Sprinkler
+w -‐w +w -‐w +w -‐w +w -‐w
0.99 0.01 0.90 0.10 0.90 0.10 0.01 0.99
Rain
WetGrass
+c -‐c
+r -‐r +r -‐r
0.8 0.2 0.2 0.8
Samples: +c, -‐s, +r, +w -‐c, +s, -‐r, +w …
§ Return (x1, x2, …, xn)
Prior Sampling § This process generates samples with probability:
Example § We ll get a bunch of samples from the BN: +c, -‐s, +r, +w +c, +s, +r, +w -‐c, +s, +r, -‐w +c, -‐s, +r, +w -‐c, -‐s, -‐r, +w
…i.e. the BN s joint probability § Let the number of samples of an event be § Then
§ I.e., the sampling procedure is consistent
C S
R W
§ If we want to know P(W) § § § § § §
We have counts Normalize to get P(W) = This will get closer to the true distribu)on with more samples Can es)mate anything else, too What about P(C| +w)? P(C| +r, +w)? P(C| -‐r, -‐w)? Fast: can use fewer samples if less )me (what’s the drawback?)
Rejec)on Sampling
Rejec)on Sampling § Let s say we want P(C) § No point keeping all samples around § Just tally counts of C as we go C
§ Let s say we want P(C| +s) § Same thing: tally C outcomes, but ignore (reject) samples which don t have S=+s § This is called rejec)on sampling § It is also consistent for condi)onal probabili)es (i.e., correct in the limit)
Rejec)on Sampling § IN: evidence instan)a)on § For i=1, 2, …, n § Sample xi from P(Xi | Parents(Xi)) § If xi not consistent with evidence
S
R W
+c, -‐s, +r, +w +c, +s, +r, +w -‐c, +s, +r, -‐w +c, -‐s, +r, +w -‐c, -‐s, -‐r, +w
Sampling Example § There are 2 cups. § The first contains 1 penny and 1 quarter § The second contains 2 quarters
§ Reject: Return, and no sample is generated in this cycle
§ Return (x1, x2, …, xn)
§ Say I pick a cup uniformly at random, then pick a coin randomly from that cup. It's a quarter (yes!). What is the probability that the other coin in that cup is also a quarter?
Likelihood Weigh)ng
Likelihood Weigh)ng § Problem with rejec)on sampling: § If evidence is unlikely, rejects lots of samples § Evidence not exploited as you sample § Consider P(Shape|blue)
Shape
Likelihood Weigh)ng +c -‐c
Color
pyramid, green pyramid, red sphere, blue cube, red sphere, green
§ IN: evidence instan)a)on § w = 1.0 § for i=1, 2, …, n
0.5 0.5
Sprinkler
§ Problem: sample distribu)on not consistent! § Solu)on: weight by probability of evidence given parents pyramid, blue pyramid, blue sphere, blue Shape Color cube, blue sphere, blue
Likelihood Weigh)ng
§ if Xi is an evidence variable
Cloudy +c +s 0.1 -‐s 0.9 -‐c +s 0.5 -‐s 0.5
§ Idea: fix evidence variables and sample the rest
+c -‐c
Rain
+r -‐r +r -‐r
§ Xi = observa)on xi for Xi § Set w = w * P(xi | Parents(Xi))
0.8 0.2 0.2 0.8
§ else § Sample xi from P(Xi | Parents(Xi))
§ return (x1, x2, …, xn), w +s -‐s
+r -‐r +r -‐r
+w -‐w +w -‐w +w -‐w +w -‐w
0.99 0.01 0.90 0.10 0.90 0.10 0.01 0.99
WetGrass
Samples: +c, +s, +r, +w …
Likelihood Weigh)ng
Likelihood Weigh)ng § Likelihood weigh)ng is good
§ Sampling distribu)on if z sampled and e fixed evidence
§ We have taken evidence into account as we generate the sample § E.g. here, W s value will get picked based on the evidence values of S, R § More of our samples will reflect the state of the world suggested by the evidence
Cloudy C
§ Now, samples have weights
S
R W
§ Together, weighted sampling distribu)on is consistent
§ Likelihood weigh)ng doesn t solve all our problems § Evidence influences the choice of downstream variables, but not upstream ones (C isn t more likely to get a value matching the evidence)
§ We would like to consider evidence when we sample every variable à Gibbs sampling
Gibbs Sampling
Gibbs Sampling § Procedure: keep track of a full instan)a)on x1, x2, …, xn. Start with an arbitrary instan)a)on consistent with the evidence. Sample one variable at a )me, condi)oned on all the rest, but keep evidence fixed. Keep repea)ng this for a long )me. § Property: in the limit of repea)ng this infinitely many )mes the resul)ng sample is coming from the correct distribu)on § Ra7onale: both upstream and downstream variables condi)on on evidence.
§ In contrast: likelihood weigh)ng only condi)ons on upstream evidence, and hence weights obtained in likelihood weigh)ng can some)mes be very small. Sum of weights over all samples is indica)ve of how many “effec)ve” samples were obtained, so want high weight.
Gibbs Sampling Example: P( S | +r) § Step 1: Fix evidence § R = +r
§ Step 2: Ini)alize other variables
C S
Efficient Resampling of One Variable § Sample from P(S | +c, +r, -‐w)
C
§ Randomly
+r
S
C S
+r
W
+r W
W
§ Steps 3: Repeat § Choose a non-‐evidence variable X § Resample X from P( X | all other variables) C S
C +r
W
S
C +r
W
S
C +r
W
S
C +r
S
W
C +r
W
S
+r W
§ Many things cancel out – only CPTs with S remain! § More generally: only CPTs that have resampled variable need to be considered, and joined together
Further Reading* § Gibbs sampling produces sample from the query distribu)on P( Q | e ) in limit of re-‐sampling infinitely ojen
Bayes’ Net Sampling Summary § Prior Sampling P
§ Rejec)on Sampling P( Q | e )
§ Likelihood Weigh)ng P( Q | e)
§ Gibbs Sampling P( Q | e )
§ Gibbs sampling is a special case of more general methods called Markov chain Monte Carlo (MCMC) methods § Metropolis-‐Has)ngs is one of the more famous MCMC methods (in fact, Gibbs sampling is a special case of Metropolis-‐Has)ngs)
§ You may read about Monte Carlo methods – they’re just sampling
CS 188: Ar)ficial Intelligence
Bayes’ Nets: Sampling
§ A directed, acyclic graph, one node per random variable § A condi)onal probability table (CPT) for each node § A collec)on of distribu)ons over X, one for each combina)on of parents values
§ Bayes nets implicitly encode joint distribu)ons § As a product of local condi)onal distribu)ons § To see what probability a BN gives to a full assignment, mul)ply all the relevant condi)onals together: Pieter Abbeel and Dan Klein University of California, Berkeley
Example: Alarm Network B
P(B)
+b 0.001 -‐b
B
E
0.999
A
E
P(E)
B
+b 0.001
-‐e
-‐b
0.998
A
J
P(J|A)
A
M
P(M|A)
+j
0.9
+a
+m
0.7
+a
-‐j
0.1
+a
-‐m
0.3
-‐a
+j
0.05
-‐a
+m
0.01
-‐a
-‐j
0.95
-‐a
-‐m
0.99
M
B
E
A
P(A|B,E)
+b
+e
+a
0.95
+b
+e
-‐a
0.05
+b
-‐e
+a
0.94
+b
-‐e
-‐a
0.06
-‐b
+e
+a
-‐b
+e
-‐b -‐b
E
0.999
E
-‐e
A
P(E)
+e 0.002 0.998
A
J
P(J|A)
A
M
P(M|A)
+a
+j
0.9
+a
+m
0.7
+a
-‐j
0.1
+a
-‐m
0.3
-‐a
+j
0.05
-‐a
+m
0.01
-‐a
-‐j
0.95
-‐a
-‐m
0.99
B
E
A
P(A|B,E)
+e
+a
0.95
+b
+e
-‐a
0.05
+b
-‐e
+a
0.94
+b
-‐e
-‐a
0.06
0.29
-‐b
+e
+a
0.29
-‐a
0.71
-‐b
+e
-‐a
0.71
-‐e
+a
0.001
-‐b
-‐e
+a
0.001
-‐e
-‐a
0.999
-‐b
-‐e
-‐a
0.999
J
M
Approximate Inference: Sampling
§ Interleave joining and marginalizing § dk entries computed for a factor over k variables with domain sizes d
§ Worst case: running )me exponen)al in the size of the Bayes’ net
B
+b
Variable Elimina)on
§ Ordering of elimina)on of hidden variables can affect size of factors generated
P(B)
+e 0.002
+a
J
Example: Alarm Network
… …
Sampling § Sampling is a lot like repeated simula)on
Sampling
§ Why sample? § Learning: get samples from a distribu)on you don’t know
§ Predic)ng the weather, basketball games, …
§ Basic idea
§ Inference: geeng a sample is faster than compu)ng the right answer (e.g. with variable elimina)on)
§ Draw N samples from a sampling distribu)on S § Compute an approximate posterior probability § Show this converges to the true probability P
Sampling in Bayes’ Nets
§ Sampling from given distribu)on
§ Example
§ Step 1: Get sample u from uniform distribu)on over [0, 1) § E.g. random() in python
§ Step 2: Convert this sample u into an outcome for the given distribu)on by having each outcome associated with a sub-‐interval of [0,1) with sub-‐interval size equal to probability of the outcome
C red
P(C) 0.6
green
0.1
blue
0.3
§ If random() returns u = 0.83, then our sample is C = blue § E.g, ajer sampling 8 )mes:
Prior Sampling
§ Prior Sampling § Rejec)on Sampling § Likelihood Weigh)ng § Gibbs Sampling
Prior Sampling +c -‐c
Prior Sampling § For i=1, 2, …, n
0.5 0.5
§ Sample xi from P(Xi | Parents(Xi))
Cloudy +c +s 0.1 -‐s 0.9 -‐c +s 0.5 -‐s 0.5
+s -‐s
+r -‐r +r -‐r
Sprinkler
+w -‐w +w -‐w +w -‐w +w -‐w
0.99 0.01 0.90 0.10 0.90 0.10 0.01 0.99
Rain
WetGrass
+c -‐c
+r -‐r +r -‐r
0.8 0.2 0.2 0.8
Samples: +c, -‐s, +r, +w -‐c, +s, -‐r, +w …
§ Return (x1, x2, …, xn)
Prior Sampling § This process generates samples with probability:
Example § We ll get a bunch of samples from the BN: +c, -‐s, +r, +w +c, +s, +r, +w -‐c, +s, +r, -‐w +c, -‐s, +r, +w -‐c, -‐s, -‐r, +w
…i.e. the BN s joint probability § Let the number of samples of an event be § Then
§ I.e., the sampling procedure is consistent
C S
R W
§ If we want to know P(W) § § § § § §
We have counts Normalize to get P(W) = This will get closer to the true distribu)on with more samples Can es)mate anything else, too What about P(C| +w)? P(C| +r, +w)? P(C| -‐r, -‐w)? Fast: can use fewer samples if less )me (what’s the drawback?)
Rejec)on Sampling
Rejec)on Sampling § Let s say we want P(C) § No point keeping all samples around § Just tally counts of C as we go C
§ Let s say we want P(C| +s) § Same thing: tally C outcomes, but ignore (reject) samples which don t have S=+s § This is called rejec)on sampling § It is also consistent for condi)onal probabili)es (i.e., correct in the limit)
Rejec)on Sampling § IN: evidence instan)a)on § For i=1, 2, …, n § Sample xi from P(Xi | Parents(Xi)) § If xi not consistent with evidence
S
R W
+c, -‐s, +r, +w +c, +s, +r, +w -‐c, +s, +r, -‐w +c, -‐s, +r, +w -‐c, -‐s, -‐r, +w
Sampling Example § There are 2 cups. § The first contains 1 penny and 1 quarter § The second contains 2 quarters
§ Reject: Return, and no sample is generated in this cycle
§ Return (x1, x2, …, xn)
§ Say I pick a cup uniformly at random, then pick a coin randomly from that cup. It's a quarter (yes!). What is the probability that the other coin in that cup is also a quarter?
Likelihood Weigh)ng
Likelihood Weigh)ng § Problem with rejec)on sampling: § If evidence is unlikely, rejects lots of samples § Evidence not exploited as you sample § Consider P(Shape|blue)
Shape
Likelihood Weigh)ng +c -‐c
Color
pyramid, green pyramid, red sphere, blue cube, red sphere, green
§ IN: evidence instan)a)on § w = 1.0 § for i=1, 2, …, n
0.5 0.5
Sprinkler
§ Problem: sample distribu)on not consistent! § Solu)on: weight by probability of evidence given parents pyramid, blue pyramid, blue sphere, blue Shape Color cube, blue sphere, blue
Likelihood Weigh)ng
§ if Xi is an evidence variable
Cloudy +c +s 0.1 -‐s 0.9 -‐c +s 0.5 -‐s 0.5
§ Idea: fix evidence variables and sample the rest
+c -‐c
Rain
+r -‐r +r -‐r
§ Xi = observa)on xi for Xi § Set w = w * P(xi | Parents(Xi))
0.8 0.2 0.2 0.8
§ else § Sample xi from P(Xi | Parents(Xi))
§ return (x1, x2, …, xn), w +s -‐s
+r -‐r +r -‐r
+w -‐w +w -‐w +w -‐w +w -‐w
0.99 0.01 0.90 0.10 0.90 0.10 0.01 0.99
WetGrass
Samples: +c, +s, +r, +w …
Likelihood Weigh)ng
Likelihood Weigh)ng § Likelihood weigh)ng is good
§ Sampling distribu)on if z sampled and e fixed evidence
§ We have taken evidence into account as we generate the sample § E.g. here, W s value will get picked based on the evidence values of S, R § More of our samples will reflect the state of the world suggested by the evidence
Cloudy C
§ Now, samples have weights
S
R W
§ Together, weighted sampling distribu)on is consistent
§ Likelihood weigh)ng doesn t solve all our problems § Evidence influences the choice of downstream variables, but not upstream ones (C isn t more likely to get a value matching the evidence)
§ We would like to consider evidence when we sample every variable à Gibbs sampling
Gibbs Sampling
Gibbs Sampling § Procedure: keep track of a full instan)a)on x1, x2, …, xn. Start with an arbitrary instan)a)on consistent with the evidence. Sample one variable at a )me, condi)oned on all the rest, but keep evidence fixed. Keep repea)ng this for a long )me. § Property: in the limit of repea)ng this infinitely many )mes the resul)ng sample is coming from the correct distribu)on § Ra7onale: both upstream and downstream variables condi)on on evidence.
§ In contrast: likelihood weigh)ng only condi)ons on upstream evidence, and hence weights obtained in likelihood weigh)ng can some)mes be very small. Sum of weights over all samples is indica)ve of how many “effec)ve” samples were obtained, so want high weight.
Gibbs Sampling Example: P( S | +r) § Step 1: Fix evidence § R = +r
§ Step 2: Ini)alize other variables
C S
Efficient Resampling of One Variable § Sample from P(S | +c, +r, -‐w)
C
§ Randomly
+r
S
C S
+r
W
+r W
W
§ Steps 3: Repeat § Choose a non-‐evidence variable X § Resample X from P( X | all other variables) C S
C +r
W
S
C +r
W
S
C +r
W
S
C +r
S
W
C +r
W
S
+r W
§ Many things cancel out – only CPTs with S remain! § More generally: only CPTs that have resampled variable need to be considered, and joined together
Further Reading* § Gibbs sampling produces sample from the query distribu)on P( Q | e ) in limit of re-‐sampling infinitely ojen
Bayes’ Net Sampling Summary § Prior Sampling P
§ Rejec)on Sampling P( Q | e )
§ Likelihood Weigh)ng P( Q | e)
§ Gibbs Sampling P( Q | e )
§ Gibbs sampling is a special case of more general methods called Markov chain Monte Carlo (MCMC) methods § Metropolis-‐Has)ngs is one of the more famous MCMC methods (in fact, Gibbs sampling is a special case of Metropolis-‐Has)ngs)
§ You may read about Monte Carlo methods – they’re just sampling
