Cliques in Graphs Cliques and 3-SAT
Cliques in Graphs Definition. A clique in a graph is a set S of vertices such that every pair of vertices in S are adjacent. If the clique has n vertices, it is denoted by Kn .
K5
Finding maximal sized cliques in large graphs is a challenging problem in data mining, i.e., analyzing data clustering. Maximal clique detection arises in bioinformatics, analysis of social networking patterns, web page clustering etc.
Cliques and 3-SAT Recall 3-SAT: Given a set of clauses C = {C1 , C2 , . . . , Ck } each of size 3, over a set of variables X = {x1 , x2 , . . . , xn }, does there exist a truth assignment that satisfies C?
1
3-SAT and Cliques Suppose we have a 3-SAT formula: F : (a OR b OR d) AND (b OR !c OR d) AND (!a OR c OR !d) AND (a OR !b OR !c)
Consider the following algorithm to construct a graph from F : 1. Create a vertex for each literal in the formula. Use the literal to label the vertex. 2. Group the vertices corresponding to a clause together. 3. Connect two vertices in the graph if they are: • In different clauses • and are not a negation of each other. Example
a
b
d
b
a
!c
!b
d
!c !a
c
!d 2
3-SAT and Cliques Q. If there is a clique in our graph equal to the number of clauses in F (so 4), what does this tell us about F ? A.
a
b
d
b
a
!c
!b
d
!c !a
c
!d
F : (a OR b OR d) AND (b OR !c OR d) AND (!a OR c OR !d) AND (a OR !b OR !c)
Q. Notice that there are several possible K4 s in G. What does this mean? A. Q. If there does not exist a clique of size equal to the number of clauses then what does this tell about F ? A. 3
Clique Q. Why? A.
Definition. The problem known as Clique is the following: Given a graph G and a natural number k, does there exist a clique of size k in G? Q. What have we shown about Clique and 3-SAT? A. Q. Why? A. This technique of showing that one problem is as hard as another is called a reduction.
4
Proof By Induction Proof by Induction is a very powerful tool when used correctly. Visual: The Domino Argument Think of a row of dominos. If set up properly, when the ith domino falls, so too will the i + 1st domino for all values of i. Therefore, when the first domino is knocked over so too are all the dominos.
We use it to prove a statement S(n) is true for all natural numbers n larger than b where b is a natural number.
Note: The natural numbers, denoted N, are the “counting numbers” *: 0, 1, 2, 3, . . . , i, i + 1, i + 2, . . .
*In computer science we usually include 0 in N.
5
Inductive Proof Structure There are three steps to an inductive proof that: for all n ≥ b where n ∈ N, S (n ) is true. 1. Base Case: Prove that S(·) holds for a set B of “smallest” consecutive values. 2. Inductive Hypothesis: Assume that S(k) holds for all values of k ∈ N such that d < k < n where d is the largest value in B. 3. Inductive Step: Derive that S(n) is true from the fact that all values smaller than n make S() true. Conclusion. For all natural numbers n > b, S(n) holds. Q. How does each step relate to our domino analogy? A. •
•
6
Mmmmm....CHOCOLATE! Suppose we have a chocolate bar consisting of a number of squares arranged in a rectangular pattern. The task is to split the bar into small squares with a minimum number of breaks. Q: How many breaks will it take (assume we only break along the lines)?
Make an educated guess, and prove it by induction. Dimensions 1x1 1x2 2x2 3x2 4x3
Breaks 0 1 3 5 11
Formula: l × w chocolate bar needs
7
Proof By Induction Prove that an l × w chocolate bar needs Define S(n): If n ≥ 1 then a chocolate bar with n squares requires n − 1 breaks.
Prove that ∀n ∈ N where n ≥ 1, S(n) is true. The symbol ∀ means “for all”.
Stay tuned....we will come back to this.
8
Another Example Prove that ∀n ∈ N, n(n2 + 5) is divisible by 6. Define P (n): Let P (n) be ∃m ∈ N, n(n2 + 5) = 6m Prove ∀n ∈ N, P (n) by simple induction. Base Case. X n = 0. Let m = 0, then 0 = 0 · 6. Inductive Hypothesis. Let k ∈ N. Suppose P (k). Inductive Step. Prove that if P (k) is true then P (k + 1) is true.
= = = = =
(k + 1)((k + 1)2 + 5) (k + 1)(k2 + 2k + 1 + 5) (k2 + 2k + 6) + k(k2 + 2k + 1 + 5) (k2 + 2k + 6 + 2k2 + k) + k(k2 + 5) (3k2 + 3k + 6) + k(k2 + 5) 3k(k + 1) + 6 + k(k2 + 5)
Notice that the first term is divisble by 6 since it is a multiple of 3 and further either k or (k + 1) is even, or divisible by 2. The second term is divisible by 6 and by the induction hypothesis, so is the third term. 9
Stamp Example – Simple Induction Given an unlimited supply of 4-cent and 7-cent stamps, prove that there exists a combination of stamps to make any amount of postage that is 18-cents or more. Define P (n), what we are proving: In English: If n ≥ 18, then postage of exactly n cents can be made using only 4-cent and 7-cent stamp. Formally: n ≥ 18 → ∃a ∈ N, ∃b ∈ N, n = 4a + 7b RTP: ∀n ∈ N, P (n). Steps 1. Base Case: P (0): This is vacuously true as 0 ≥ 18 is false. Use: 1x 4-cent stamp, 2x 7-cent stamp Therefore, P (0) is true. 2. Inductive Hypothesis (IH) Let k ∈ N, suppose P (k). i.e., for each k, if k ≥ 18 then there exists i, j ∈ N such that i represents the number of 4 cent stamps and j represents the number of 7 cent stamps and k = 4i + 7j 10
4. Inductive Step (IS): RTP if P (k) holds then P (k + 1) holds. I.e., that there exist a ∈ N and b ∈ N such that: 4a + 7b = k + 1 Case I k + 1 = 18 We need to prove this directly (why?). 18 = 1 · 4 + 2 · 7 so pick a = 1 and b = 2. Case II k + 1 > 18 Case a j > 0 Since k + 1 > 18, k ≥ 18 and P (k) holds by the IH, ∃i, j ∈ N such that k = 4i + 7j k + 1 = 4i + 7j + 8 − 7 = 4i + 7j + 2 · 4 − 1 · 7 = 4(i + 2) + (j − 1) · 7 Therefore, take a = i + 2 and b = j − 1. This is okay because j > 0. Case b j = 0, Then take away some 4-cent stamps and add some 7 cent stamps. Since k ≥ 18, i ≥ 5, we can take away 5 4-cent stamps without having ”negative” stamps. Also, notice that 3 · 7 − 4 · 5 = 1. By IH, k = 4i for i ≥ 5. k + 1 = 4i + 1 = 4i + 21 − 20 = 4i + 3 · 7 − 4 · 5 = 4(i − 5) + 3 · 7 Therefore, take a = (i − 5) and b = 3. 11
Rigorous Proof of Why Induction Works Contradiction Argument Needs: The Well Ordering Principle
Any non-empty subset C of N contains a smallest element. We say that C is well ordered. 1 2 3 4 5 6 7 8 9 10 . . . i − 1 i i+1 i+2 . . . n − 1 n n+1 . . . Suppose C = {3, 7, 9, 10, i, n, . . .} then 3 is the smallest element. Assume that we have done the 3 steps of an inductive proof properly. For simplicity assume that we are proving that S(n) is true for all natural numbers n.
1. Proved that S(x) is true for all values x ∈ B. 2. Suppose that S(k) is true, d < k < n, k ∈ N where d = max(B). 3. Proved that S(n) is true. 12
Proof that Induction Proofs are valid Construct a contradiction. • Assume that ∃z ∈ N, S(z) is not true. • Let C = {x | S(x) is false and x ∈ N} be the set of all such values. This is where the Well Ordering Principle comes into play. • By the Well Ordering Principle there exists a smallest element of C. • Let a be the smallest element in C. • By the Base Case and our initial assumption, it must be that a > d. • Since a is the smallest value such that S(a) is false, it must be that S(a − 1) is true. • But by our Inductive Step, if S(a − 1) is true, then S(a) is true. • Therefore a ∈ / C, contradiction. Therefore, our assumption must be false, and S(n) holds for all n ∈ N. 13
Back to the Chocolate Bar Problem... Prove that an l × w chocolate bar needs l × w − 1 breaks. Define S(n): If n ≥ 1 then a chocolate bar with n squares requires n − 1 breaks. Prove that ∀n ∈ N, S(n). Steps: Base Case: S(1). 1 × 1 requires 0 = 1-1 breaks as we had shown on the previous slide. Therefore the Base Case holds. Inductive Hypothesis: Suppose that we know that up to a specified size of chocolate bar, our claim holds. Let n ∈ N. Suppose that all chocolate bars with less than n squares satisfy our claim. Q: How can we express this mathematically? Let n ∈ N. Suppose ∀k ∈ N such that 0 < k < n, that S(k) holds. 14
Proof Continued Inductive Step: Prove S(n) Break the bar into two smaller pieces of a and b squares where a, b < n and a + b =n. Since a < n, b < n the IH applies and S(a) and S(b) hold. What does it mean for S(a) to hold? takes a − 1 breaks to split this part of the bar up. We used one break to split the bar into the smaller pieces so the total number of breaks is (a − 1) + (b − 1) + 1 = (a + b) − 1 = n − 1 Done.
15
Strong or Complete Induction Idea. Recall the domino argument. To show that P (a) is true, we can show that:
P (0)∧(P (0) → P (1))∧(P (1) → P (2))∧. . .∧(P (a−1) → P (a))
What we’ve really done, is show that
(P (0) ∧ P (1) ∧ P (2) ∧ . . . P (a − 1)) → P (a)
So to show ∀n ∈ N, P (n), we can show:
((P (0) ∧ P (1) ∧ P (2) ∧ . . . P (a − 1)) → P (a)) → ∀nP (n)
16
Stamp Example – Strong Induction Given an unlimited supply of 4-cent and 7-cent stamps, prove that there exists a combination of stamps to make any amount of postage that is 18-cents or more. Proof by strong induction: Define P(n): Let P (n) be, if n ≥ 18 → ∃a ∈ N, ∃b ∈ N, n = 4a + 7b. RTP: ∀n ∈ N, P (n). Q: What is our proof strategy? Want to be able to prove P (n) simply be looking back at the value (n − 4). Base Case: With strong induction we typically include the base case in the induction step. Inductive Hypthesis: Let m ∈ N, suppose P (k) is true for all k < m, k ∈ N.
17
Inductive Step: Prove that P (n) is true. Case 1 n < 18. Why is this true?
Case 2 n ≥ 18. What is our strategy? Want P (n − 4) to be true, because then we can just add on a 4-cent stamp. This only works if m ≥ 22 and we know P (18), P (19), P (20) and P (21) hold. Case 2a 18 ≤ n ≤ 21. 18 19 20 21
= = = =
1·4+2·7 3·4+1·7 5·4 3·7
Case 2b n ≥ 22. Check, we need (n − 4) ≥ 18. n ≥ 22 and n − 4 ≥ 18 so P (n − 4) holds and by the inductive hypothesis there exists a,b such that 4a + 7b = n − 4, so m = 4(a + 1) + 7b.
18
K5
Finding maximal sized cliques in large graphs is a challenging problem in data mining, i.e., analyzing data clustering. Maximal clique detection arises in bioinformatics, analysis of social networking patterns, web page clustering etc.
Cliques and 3-SAT Recall 3-SAT: Given a set of clauses C = {C1 , C2 , . . . , Ck } each of size 3, over a set of variables X = {x1 , x2 , . . . , xn }, does there exist a truth assignment that satisfies C?
1
3-SAT and Cliques Suppose we have a 3-SAT formula: F : (a OR b OR d) AND (b OR !c OR d) AND (!a OR c OR !d) AND (a OR !b OR !c)
Consider the following algorithm to construct a graph from F : 1. Create a vertex for each literal in the formula. Use the literal to label the vertex. 2. Group the vertices corresponding to a clause together. 3. Connect two vertices in the graph if they are: • In different clauses • and are not a negation of each other. Example
a
b
d
b
a
!c
!b
d
!c !a
c
!d 2
3-SAT and Cliques Q. If there is a clique in our graph equal to the number of clauses in F (so 4), what does this tell us about F ? A.
a
b
d
b
a
!c
!b
d
!c !a
c
!d
F : (a OR b OR d) AND (b OR !c OR d) AND (!a OR c OR !d) AND (a OR !b OR !c)
Q. Notice that there are several possible K4 s in G. What does this mean? A. Q. If there does not exist a clique of size equal to the number of clauses then what does this tell about F ? A. 3
Clique Q. Why? A.
Definition. The problem known as Clique is the following: Given a graph G and a natural number k, does there exist a clique of size k in G? Q. What have we shown about Clique and 3-SAT? A. Q. Why? A. This technique of showing that one problem is as hard as another is called a reduction.
4
Proof By Induction Proof by Induction is a very powerful tool when used correctly. Visual: The Domino Argument Think of a row of dominos. If set up properly, when the ith domino falls, so too will the i + 1st domino for all values of i. Therefore, when the first domino is knocked over so too are all the dominos.
We use it to prove a statement S(n) is true for all natural numbers n larger than b where b is a natural number.
Note: The natural numbers, denoted N, are the “counting numbers” *: 0, 1, 2, 3, . . . , i, i + 1, i + 2, . . .
*In computer science we usually include 0 in N.
5
Inductive Proof Structure There are three steps to an inductive proof that: for all n ≥ b where n ∈ N, S (n ) is true. 1. Base Case: Prove that S(·) holds for a set B of “smallest” consecutive values. 2. Inductive Hypothesis: Assume that S(k) holds for all values of k ∈ N such that d < k < n where d is the largest value in B. 3. Inductive Step: Derive that S(n) is true from the fact that all values smaller than n make S() true. Conclusion. For all natural numbers n > b, S(n) holds. Q. How does each step relate to our domino analogy? A. •
•
6
Mmmmm....CHOCOLATE! Suppose we have a chocolate bar consisting of a number of squares arranged in a rectangular pattern. The task is to split the bar into small squares with a minimum number of breaks. Q: How many breaks will it take (assume we only break along the lines)?
Make an educated guess, and prove it by induction. Dimensions 1x1 1x2 2x2 3x2 4x3
Breaks 0 1 3 5 11
Formula: l × w chocolate bar needs
7
Proof By Induction Prove that an l × w chocolate bar needs Define S(n): If n ≥ 1 then a chocolate bar with n squares requires n − 1 breaks.
Prove that ∀n ∈ N where n ≥ 1, S(n) is true. The symbol ∀ means “for all”.
Stay tuned....we will come back to this.
8
Another Example Prove that ∀n ∈ N, n(n2 + 5) is divisible by 6. Define P (n): Let P (n) be ∃m ∈ N, n(n2 + 5) = 6m Prove ∀n ∈ N, P (n) by simple induction. Base Case. X n = 0. Let m = 0, then 0 = 0 · 6. Inductive Hypothesis. Let k ∈ N. Suppose P (k). Inductive Step. Prove that if P (k) is true then P (k + 1) is true.
= = = = =
(k + 1)((k + 1)2 + 5) (k + 1)(k2 + 2k + 1 + 5) (k2 + 2k + 6) + k(k2 + 2k + 1 + 5) (k2 + 2k + 6 + 2k2 + k) + k(k2 + 5) (3k2 + 3k + 6) + k(k2 + 5) 3k(k + 1) + 6 + k(k2 + 5)
Notice that the first term is divisble by 6 since it is a multiple of 3 and further either k or (k + 1) is even, or divisible by 2. The second term is divisible by 6 and by the induction hypothesis, so is the third term. 9
Stamp Example – Simple Induction Given an unlimited supply of 4-cent and 7-cent stamps, prove that there exists a combination of stamps to make any amount of postage that is 18-cents or more. Define P (n), what we are proving: In English: If n ≥ 18, then postage of exactly n cents can be made using only 4-cent and 7-cent stamp. Formally: n ≥ 18 → ∃a ∈ N, ∃b ∈ N, n = 4a + 7b RTP: ∀n ∈ N, P (n). Steps 1. Base Case: P (0): This is vacuously true as 0 ≥ 18 is false. Use: 1x 4-cent stamp, 2x 7-cent stamp Therefore, P (0) is true. 2. Inductive Hypothesis (IH) Let k ∈ N, suppose P (k). i.e., for each k, if k ≥ 18 then there exists i, j ∈ N such that i represents the number of 4 cent stamps and j represents the number of 7 cent stamps and k = 4i + 7j 10
4. Inductive Step (IS): RTP if P (k) holds then P (k + 1) holds. I.e., that there exist a ∈ N and b ∈ N such that: 4a + 7b = k + 1 Case I k + 1 = 18 We need to prove this directly (why?). 18 = 1 · 4 + 2 · 7 so pick a = 1 and b = 2. Case II k + 1 > 18 Case a j > 0 Since k + 1 > 18, k ≥ 18 and P (k) holds by the IH, ∃i, j ∈ N such that k = 4i + 7j k + 1 = 4i + 7j + 8 − 7 = 4i + 7j + 2 · 4 − 1 · 7 = 4(i + 2) + (j − 1) · 7 Therefore, take a = i + 2 and b = j − 1. This is okay because j > 0. Case b j = 0, Then take away some 4-cent stamps and add some 7 cent stamps. Since k ≥ 18, i ≥ 5, we can take away 5 4-cent stamps without having ”negative” stamps. Also, notice that 3 · 7 − 4 · 5 = 1. By IH, k = 4i for i ≥ 5. k + 1 = 4i + 1 = 4i + 21 − 20 = 4i + 3 · 7 − 4 · 5 = 4(i − 5) + 3 · 7 Therefore, take a = (i − 5) and b = 3. 11
Rigorous Proof of Why Induction Works Contradiction Argument Needs: The Well Ordering Principle
Any non-empty subset C of N contains a smallest element. We say that C is well ordered. 1 2 3 4 5 6 7 8 9 10 . . . i − 1 i i+1 i+2 . . . n − 1 n n+1 . . . Suppose C = {3, 7, 9, 10, i, n, . . .} then 3 is the smallest element. Assume that we have done the 3 steps of an inductive proof properly. For simplicity assume that we are proving that S(n) is true for all natural numbers n.
1. Proved that S(x) is true for all values x ∈ B. 2. Suppose that S(k) is true, d < k < n, k ∈ N where d = max(B). 3. Proved that S(n) is true. 12
Proof that Induction Proofs are valid Construct a contradiction. • Assume that ∃z ∈ N, S(z) is not true. • Let C = {x | S(x) is false and x ∈ N} be the set of all such values. This is where the Well Ordering Principle comes into play. • By the Well Ordering Principle there exists a smallest element of C. • Let a be the smallest element in C. • By the Base Case and our initial assumption, it must be that a > d. • Since a is the smallest value such that S(a) is false, it must be that S(a − 1) is true. • But by our Inductive Step, if S(a − 1) is true, then S(a) is true. • Therefore a ∈ / C, contradiction. Therefore, our assumption must be false, and S(n) holds for all n ∈ N. 13
Back to the Chocolate Bar Problem... Prove that an l × w chocolate bar needs l × w − 1 breaks. Define S(n): If n ≥ 1 then a chocolate bar with n squares requires n − 1 breaks. Prove that ∀n ∈ N, S(n). Steps: Base Case: S(1). 1 × 1 requires 0 = 1-1 breaks as we had shown on the previous slide. Therefore the Base Case holds. Inductive Hypothesis: Suppose that we know that up to a specified size of chocolate bar, our claim holds. Let n ∈ N. Suppose that all chocolate bars with less than n squares satisfy our claim. Q: How can we express this mathematically? Let n ∈ N. Suppose ∀k ∈ N such that 0 < k < n, that S(k) holds. 14
Proof Continued Inductive Step: Prove S(n) Break the bar into two smaller pieces of a and b squares where a, b < n and a + b =n. Since a < n, b < n the IH applies and S(a) and S(b) hold. What does it mean for S(a) to hold? takes a − 1 breaks to split this part of the bar up. We used one break to split the bar into the smaller pieces so the total number of breaks is (a − 1) + (b − 1) + 1 = (a + b) − 1 = n − 1 Done.
15
Strong or Complete Induction Idea. Recall the domino argument. To show that P (a) is true, we can show that:
P (0)∧(P (0) → P (1))∧(P (1) → P (2))∧. . .∧(P (a−1) → P (a))
What we’ve really done, is show that
(P (0) ∧ P (1) ∧ P (2) ∧ . . . P (a − 1)) → P (a)
So to show ∀n ∈ N, P (n), we can show:
((P (0) ∧ P (1) ∧ P (2) ∧ . . . P (a − 1)) → P (a)) → ∀nP (n)
16
Stamp Example – Strong Induction Given an unlimited supply of 4-cent and 7-cent stamps, prove that there exists a combination of stamps to make any amount of postage that is 18-cents or more. Proof by strong induction: Define P(n): Let P (n) be, if n ≥ 18 → ∃a ∈ N, ∃b ∈ N, n = 4a + 7b. RTP: ∀n ∈ N, P (n). Q: What is our proof strategy? Want to be able to prove P (n) simply be looking back at the value (n − 4). Base Case: With strong induction we typically include the base case in the induction step. Inductive Hypthesis: Let m ∈ N, suppose P (k) is true for all k < m, k ∈ N.
17
Inductive Step: Prove that P (n) is true. Case 1 n < 18. Why is this true?
Case 2 n ≥ 18. What is our strategy? Want P (n − 4) to be true, because then we can just add on a 4-cent stamp. This only works if m ≥ 22 and we know P (18), P (19), P (20) and P (21) hold. Case 2a 18 ≤ n ≤ 21. 18 19 20 21
= = = =
1·4+2·7 3·4+1·7 5·4 3·7
Case 2b n ≥ 22. Check, we need (n − 4) ≥ 18. n ≥ 22 and n − 4 ≥ 18 so P (n − 4) holds and by the inductive hypothesis there exists a,b such that 4a + 7b = n − 4, so m = 4(a + 1) + 7b.
18
