Methods for solving recurrences Iteration method
MATH 641 Module 05b Course Slides (Last Updated: November 28, 2012)
Methods for solving recurrences
Iteration method Substitution method Master theorem method Aside Other methods exist, such as using generating functions and characteristic equations.
Iteration method
Step 1 Iteratively expand the recurrence. Step 2 “Unwind” until the base case is reached. Step 3 Determine the total. Example: T (1) = 3; T (n) = T (dn/2e) + 1 T (n) = T (dn/2e) + 1 = T (dn/4e) + 1 + 1 = T (dn/8e) + 1 + 1 + 1 = T (dn/2k e) + k
Page 1 of 7 c University of Waterloo and others
MATH 641 Module 05b Course Slides (Last Updated: November 28, 2012)
Finding a closed form
T (n) = T (dn/2k e) + k Idea: keep “unwinding” until we reach the base case. T (1) = 3 T (dn/2k e) = T (1) when k = dlog2 ne T (n) = T (1) + dlog2 ne T (n) is in Θ(log n)
Substitution method
Step 1 Guess an upper bound for T (n). Step 2 Plug in the answer for smaller values on the right hand side. Step 3 Simplify the right hand side to prove the bound. Tips: Do not use order notation in the guess. Use constants as placeholders if needed.
Page 2 of 7 c University of Waterloo and others
MATH 641 Module 05b Course Slides (Last Updated: November 28, 2012)
How to guess an answer
Modify the result of a similar recurrence. Prove rough lower and upper bounds and choose something in the middle (repeat as needed). Use iteration method for a few iterations, then guess.
Substitution example: guess using iteration Example: T (2k ) = 2T (2k−1 ) + c2k , T (1) = c k
0
1
2
3
4
5
2k
1
2
4
8
16
32
T (2k )
c
4c
12c
32c
80c
192c
T (2k )/c2k
1
2
3
4
5
6
Guess T (2k )/c2k = k + 1, or T (2k ) = c2k (k + 1). OK for k = 0. Result: O(n log n) T (2k ) = 2T (2k−1 ) + c2k = 2c2k−1 k + c2k = c2k k + c2k = c2k (k + 1)
Page 3 of 7 c University of Waterloo and others
MATH 641 Module 05b Course Slides (Last Updated: November 28, 2012)
Master method For T (n) = aT (n/b) + f (n), compare f (n) and x = nlogb a . (Ignore floors and ceilings) If f (n) is “smaller than” x, then T (n) is in Θ(x). If f (n) is “bigger than” x, then T (n) is in Θ(f (n)). If f (n) and x are “the same size” then T (n) is in Θ(x log n). Aside The Master Theorem does not cover all cases: Suppose T (n) = aT (n/b) + f (n) for constants a ≥ 1 and b > 1. Then I
If f (n) is in O(nlogb a− ) for some constant > 0, then T (n) is in Θ(nlogb a ).
I
If f (n) is in Ω(nlogb a+ ) for some constant > 0, and if af (n/b) ≤ cf (n) for some constant c < 1 and all sufficiently large n, then T (n) is in Θ(f (n)).
I
If f (n) is in Θ(nlogb a ), then T (n) is in Θ(nlogb a log n).
Master method examples For T (n) = aT (n/b) + f (n), compare f (n) and nlogb a . If f (n) is “smaller than” x, then T (n) is in Θ(x). If f (n) is “bigger than” x, then T (n) is in Θ(f (n)). If f (n) and x are “the same size” then T (n) is in Θ(x log n). Example: T (n) = 4T (n/2) + n a = 4, b = 2, f (n) = n, nlogb a = n2 T (n) is in Θ(n2 ) Example: T (n) = T (n/2) + n a = 1, b = 2, f (n) = n, nlogb a = n0 T (n) is in Θ(n) Example: T (n) = 2T (n/2) + n a = 2, b = 2, f (n) = n, nlogb a = n T (n) is in Θ(n log n)
Page 4 of 7 c University of Waterloo and others
MATH 641 Module 05b Course Slides (Last Updated: November 28, 2012)
Properties of divide-and-conquer
Original problem is easily decomposable into subproblems Subproblems should be roughly the same size Combining solutions shouldn’t be too costly
Maximum subtotal Maximum subtotal Input: A sequence of n numbers a1 through an P Output: Values i and j such that jk=i ak is the largest possible 1
2
3
4
5
6
7
2 -10 4
-2 16 -5
i
j
4
6
Page 5 of 7 c University of Waterloo and others
7
MATH 641 Module 05b Course Slides (Last Updated: November 28, 2012)
Solving maximum subtotal Divide: divide list into two pieces Conquer: find maximum of left half and maximum of right half Combine:
1
2
3
4
5
6
7
7
2 -10 4
-2 16 -5
maximum
maximum
9
16
Solving maximum subtotal Divide: divide list into two pieces Conquer: find maximum of left half and maximum of right half Combine: maximum of maximum of left half maximum of right half sum of the maximum suffix of the left half and the maximum prefix of the right half 1
2
3
4
5
6
7
2 -10 4
-2 16 -5
maximum suffix
maximum prefix
4
14
Page 6 of 7 c University of Waterloo and others
7
MATH 641 Module 05b Course Slides (Last Updated: November 28, 2012)
Pseudocode for maximum subtotal Assume linear-time procedures MAXSUFFIX and MAXPREFIX MAXOF(A,p,r) 1 if p = r 2 then return ITEM(A,p) 3 else 4 q ← b(p + r )/2c 5 Leftmax ← MAXOF(A,p,q) 6 Rightmax ← MAXOF(A,q+1,r) 7 Suffix ← MAXSUFFIX(A,p,q) 8 Prefix ← MAXPREFIX(A,q+1,r) 9 return max(Leftmax,Rightmax,Suffix+Prefix)
T (n) = T (bn/2c) + T (dn/2e) + O(n) T (1) is constant time
Page 7 of 7 c University of Waterloo and others
Methods for solving recurrences
Iteration method Substitution method Master theorem method Aside Other methods exist, such as using generating functions and characteristic equations.
Iteration method
Step 1 Iteratively expand the recurrence. Step 2 “Unwind” until the base case is reached. Step 3 Determine the total. Example: T (1) = 3; T (n) = T (dn/2e) + 1 T (n) = T (dn/2e) + 1 = T (dn/4e) + 1 + 1 = T (dn/8e) + 1 + 1 + 1 = T (dn/2k e) + k
Page 1 of 7 c University of Waterloo and others
MATH 641 Module 05b Course Slides (Last Updated: November 28, 2012)
Finding a closed form
T (n) = T (dn/2k e) + k Idea: keep “unwinding” until we reach the base case. T (1) = 3 T (dn/2k e) = T (1) when k = dlog2 ne T (n) = T (1) + dlog2 ne T (n) is in Θ(log n)
Substitution method
Step 1 Guess an upper bound for T (n). Step 2 Plug in the answer for smaller values on the right hand side. Step 3 Simplify the right hand side to prove the bound. Tips: Do not use order notation in the guess. Use constants as placeholders if needed.
Page 2 of 7 c University of Waterloo and others
MATH 641 Module 05b Course Slides (Last Updated: November 28, 2012)
How to guess an answer
Modify the result of a similar recurrence. Prove rough lower and upper bounds and choose something in the middle (repeat as needed). Use iteration method for a few iterations, then guess.
Substitution example: guess using iteration Example: T (2k ) = 2T (2k−1 ) + c2k , T (1) = c k
0
1
2
3
4
5
2k
1
2
4
8
16
32
T (2k )
c
4c
12c
32c
80c
192c
T (2k )/c2k
1
2
3
4
5
6
Guess T (2k )/c2k = k + 1, or T (2k ) = c2k (k + 1). OK for k = 0. Result: O(n log n) T (2k ) = 2T (2k−1 ) + c2k = 2c2k−1 k + c2k = c2k k + c2k = c2k (k + 1)
Page 3 of 7 c University of Waterloo and others
MATH 641 Module 05b Course Slides (Last Updated: November 28, 2012)
Master method For T (n) = aT (n/b) + f (n), compare f (n) and x = nlogb a . (Ignore floors and ceilings) If f (n) is “smaller than” x, then T (n) is in Θ(x). If f (n) is “bigger than” x, then T (n) is in Θ(f (n)). If f (n) and x are “the same size” then T (n) is in Θ(x log n). Aside The Master Theorem does not cover all cases: Suppose T (n) = aT (n/b) + f (n) for constants a ≥ 1 and b > 1. Then I
If f (n) is in O(nlogb a− ) for some constant > 0, then T (n) is in Θ(nlogb a ).
I
If f (n) is in Ω(nlogb a+ ) for some constant > 0, and if af (n/b) ≤ cf (n) for some constant c < 1 and all sufficiently large n, then T (n) is in Θ(f (n)).
I
If f (n) is in Θ(nlogb a ), then T (n) is in Θ(nlogb a log n).
Master method examples For T (n) = aT (n/b) + f (n), compare f (n) and nlogb a . If f (n) is “smaller than” x, then T (n) is in Θ(x). If f (n) is “bigger than” x, then T (n) is in Θ(f (n)). If f (n) and x are “the same size” then T (n) is in Θ(x log n). Example: T (n) = 4T (n/2) + n a = 4, b = 2, f (n) = n, nlogb a = n2 T (n) is in Θ(n2 ) Example: T (n) = T (n/2) + n a = 1, b = 2, f (n) = n, nlogb a = n0 T (n) is in Θ(n) Example: T (n) = 2T (n/2) + n a = 2, b = 2, f (n) = n, nlogb a = n T (n) is in Θ(n log n)
Page 4 of 7 c University of Waterloo and others
MATH 641 Module 05b Course Slides (Last Updated: November 28, 2012)
Properties of divide-and-conquer
Original problem is easily decomposable into subproblems Subproblems should be roughly the same size Combining solutions shouldn’t be too costly
Maximum subtotal Maximum subtotal Input: A sequence of n numbers a1 through an P Output: Values i and j such that jk=i ak is the largest possible 1
2
3
4
5
6
7
2 -10 4
-2 16 -5
i
j
4
6
Page 5 of 7 c University of Waterloo and others
7
MATH 641 Module 05b Course Slides (Last Updated: November 28, 2012)
Solving maximum subtotal Divide: divide list into two pieces Conquer: find maximum of left half and maximum of right half Combine:
1
2
3
4
5
6
7
7
2 -10 4
-2 16 -5
maximum
maximum
9
16
Solving maximum subtotal Divide: divide list into two pieces Conquer: find maximum of left half and maximum of right half Combine: maximum of maximum of left half maximum of right half sum of the maximum suffix of the left half and the maximum prefix of the right half 1
2
3
4
5
6
7
2 -10 4
-2 16 -5
maximum suffix
maximum prefix
4
14
Page 6 of 7 c University of Waterloo and others
7
MATH 641 Module 05b Course Slides (Last Updated: November 28, 2012)
Pseudocode for maximum subtotal Assume linear-time procedures MAXSUFFIX and MAXPREFIX MAXOF(A,p,r) 1 if p = r 2 then return ITEM(A,p) 3 else 4 q ← b(p + r )/2c 5 Leftmax ← MAXOF(A,p,q) 6 Rightmax ← MAXOF(A,q+1,r) 7 Suffix ← MAXSUFFIX(A,p,q) 8 Prefix ← MAXPREFIX(A,q+1,r) 9 return max(Leftmax,Rightmax,Suffix+Prefix)
T (n) = T (bn/2c) + T (dn/2e) + O(n) T (1) is constant time
Page 7 of 7 c University of Waterloo and others
