Transcript 投影片 1

Discrete
Mathematics
Chapter 3
The Fundamentals : Algorithms,
the Integers, and Matrices
大葉大學 資訊工程系 黃鈴玲
3.1 Algorithms
Def 1. An algorithm is a finite sequence of
precise instructions for performing a
computation or for solving a problem.
 Example 1. Describe an algorithm for finding
the maximum value in a finite sequence of
integers.(假設給定的sequence是a1,a2,…,an)
Ch3-2
Solution : ( English language)
1. Set the temporary maximum equal to the first
integer in the sequence.
2. Compare the next integer in the sequence to the
temporary maximum, and if it is larger than the
temporary maximum, set the temporary maximum
equal to this integer.
3. Repeat the previous step if there are more integers
in the sequence.
4. Stop when there are no integers left in the
sequence. The temporary maximum at this point is
the largest integer in the sequence.
Ch3-3
Solution : (pseudo-code)
Algorithm 1. Finding the Maximum Element
procedure max(a1, a2, …, an : integers)
max := a1
for i := 2 to n
if max < ai then max := ai
{ max is the largest element}
Ch3-4
※ There are several properties that algorithms generally
share :
 Input
 Output
 Definiteness : The steps of an algorithm must be defined
precisely.
 Correctness : produce correct output values
 Finiteness : produce the desired output after a finite
number of step.
 Effectiveness
 Generality : The procedure should be applicable for all
problems of the desired form, not just for a
particular set of input values.
Ch3-5
※ Searching Algorithms
Problem : Locate an element x in a list of distinct elements
a1,a2,…,an, or determine that it is not in the list.
做法 : linear search, binary search.
Algorithm 2. The linear search algorithm
procedure linear_search( x : integer, a1,a2,…,an: distinct integers)
i := 1
While ( i ≤ n and x≠ai )
i := i + 1
if i ≤ n then location := i
else location := 0
{ location = j if x = aj; location = 0 if x≠ai, ∀i }
Ch3-6

兩種search方式的概念 :
Linear Search : 從 a1 開始，逐一比對 x 是否等於 ai，若找到
則 location = i , 若到 an 比完後還找不到，則 location = 0。
Binary Search : (必須具備 a1 < a2 < … < an 的性質才能用)
(1) 每次將 list 切成兩半，( ai , … , am )，(am+1 , … , aj )
若 x > am 表示 x 應在右半，否則在左半。
(2) 重覆上一步驟至 list 只剩一個元素 ai，
若 x = ai 則 location = i，否則 location = 0。
Ch3-7
Example 3. Search 19 from
a1 a2 a3 a4 a5 a6 a7 a8
1 2 3 5 6 7 8 10
1. ( 切兩半 )
( 因 19 > 10，取右半 )
2. ( 再切二半 )
( 因 19 > 16，取右半 )
3. ( 再切二半 )
( 因 19 ≦19，取左半 )
4. ( 再切二半 )
( 因 19 > 18，取右半 )
5 此時只剩一個元素 a14 = 19
因 19 = 19，故 location =14
a9
12
12
a10
13
13
a11
15
15
a12
16
16
a13
18
18
18
18
a14
19
19
19
19
19
a15
20
20
20
a16
22
22
22
Note : ai, ai+1, …, aj 數列的切法 :
i  j 
令m= 
 2 
則 am 即切開紅線左邊那點。
Ch3-8
Algorithm 3. The Binary Search Algorithm
procedure binary_search( x : integer, a1,a2,…,an : increasing integers)
i :=1 { i is left endpoint of search interval }
j := n { j is right endpoint of search interval }
while i < j
begin
m := (i  j ) / 2
if x > am then i := m+1
else j := m
end
if x = ai then location := i
else location := 0
{ location = i if x = ai , location = 0 if x≠ai , ∀i }
Ch3-9
※ Sorting Algorithms




Problem : Suppose that we have a list of elements,
a sorting is putting these elements into a list in
which the elements are in increasing order.
eg. 7, 2, 1, 4, 5, 9 => 1, 2, 4, 5, 7, 9
d, t, c, a, f => a, c, d, f, t
解法有很多，此處僅介紹 : bubble sort (氣泡排序
法)，及 insertion sort (插入排序法)。
Bubble Sort 概念 : 設原 list 為 a1,…,an。
 從a1,a2開始，向後兩兩比較，若ai
> ai+1 則交換，當檢查
完 an 時，an 必定是最大數。
 再從 a1,a2 開始向後比較，若ai > ai+1 則交換，此時只需檢
查到 an-1 即可。
 依此類推。
Ch3-10


Example 4. Use the bubble sort to put 3, 2, 4, 1, 5
into increasing order.
Sol :
First pass (i=1) :
3
2
4
1
5
2
3
4
1
5
2
3
4
1
5
Third pass (i=3) :
2
1
3
4
5
1
2
3
4
5
1
2
3
4
5
Second pass (i=2) :
2
3
1
4
5
2
3
1
4
5
2
3
1
4
5
2
3
1
4
5
2
1
3
4
5
2
1
3
4
5
Fourth pass (i=4) :
1
2
3
4
5
1
2
3
4
5
Ch3-11
Algorithm 4 The Bubble Sort
procedure bubble_sort (a1,…,an )
for i := 1 to n-1
for j := 1 to n-i
if aj > aj+1 then interchange aj and aj+1
{ a1,a2,…,an is in increasing order }
Ch3-12

Insertion Sort 的概念 :
j = 2 開始，將 aj 插入已排序好的 a1,…,aj-1間的
位置，使得 a1,…,aj 都由小 → 大排好。
 j 逐次遞增，重複上一步驟至做完。
從
Ch3-13


a1 a2 a3 a4 a5
Example 5. Use insertion sort to sort 3, 2, 4, 1, 5
Sol :
 (j=2時，a1=3可看成已經排序好的數列，此時要插
入a2) :
3 < 2  2, 3 交換  2, 3, 4, 1, 5
 (j=3時，a1,a2已經排序好，此時要插入a3) :
4 > 2, 4 > 3  4的位置不變  2, 3, 4, 1, 5
 (j=4時，a1,a2 ,a3已經排序好，此時要插入a4) :
1 < 2  將 1 插在最前面  1, 2, 3, 4, 5
 (j=5時，a1,a2 ,a3 ,a4已經排序好，此時要插入a5) :
5 > 1, 5 > 2, 5 > 3, 5 > 4  5不變  1, 2, 3, 4, 5
Ch3-14
Algorithm 5 The Insertion Sort
procedure insertion_sort ( a1,…,an : real numbers with n ≥ 2 )
for j := 2 to n
begin
i := 1
找出 aj 應插入的位置
while aj > ai
最後ai-1 < aj <= ai
i := i + 1
m := aj
for k := 0 to j – i – 1
將 ai, ai+1, …, aj-1
aj-k := aj-k-1
全部往右移一格
ai := m
end
{ a1,a2,…,an are sorted }
( Exercise : 3, 9, 13, 23, 35, 39 )
Ch3-15
3.2 The Growth of Functions


To analyze the practicality of the program, we need
to understand how quickly the function (number of
operations used by this algorithm) grows as n
(number of input elements) grows.
eg. sort n objects
 Alg. 1 : n2次計算
 Alg. 2 : 8n次計算
n
# Alg.1
of Alg.2
op.
1
1
8
2
4
16
3
9
24
…
…
…
8
64
64
9
81
72
10
100
80 better!
Ch3-16
Def 1. ( Big-O notation )
Let f and g be functions from the set of integers to
the set of real numbers. We say that f (x) is O(g(x))
if there are constants C and k such that
| f (x) | ≤ C | g(x) |
whenever x > k . ( read as “f (x) is big-oh of g(x)” )
Ch3-17
Example 1. Show that f (x) = x2+2x+1 is O(x2)
Sol : Since
x2+2x+1 ≤ x2+2x2+x2 = 4x2
whenever x > 1 , it follows that f (x) is O(x2)
(take C = 4 and k =1 )
另法：
If x > 2, we see that
x2+2x+1 ≤ x2+x2+x2 = 3x2
( take C = 3 and k = 2 )
Ch3-18
Figure 2. The function f (x) is O(g(x))
Cg(x)
f (x)
g(x)
f (x) < C g(x) for x > k
k
Example 1(補充). Show that f (n)= n 2+2n +2 is O(n3)
Sol : Since
n2+2n+2 ≤ n3+n3+n3 = 3n3
whenever n > 1, we see that f (n) is O(n3)
( take C = 3 and k = 1 )
Note. The function g is chosen to be as small as possible.
Ch3-19
Theorem 1.
Let f (x) = anxn+an-1xn-1+…+a1x+a0
where a0, a1, …, an are real numbers.
Then f (x) is O(xn).
Example 5. How can big-O notation be used to
estimate the sum of the first n positive integers?
n
( i.e.,  i )
i 1
Sol :
1 + 2 + 3 + … + n ≤ n + n + … + n = n2
n
∴  iis O(n2), taking C =1 and k =1.
i 1
Ch3-20
Example 6. Give big-O estimates for f (n) = n!
Sol :
n! = 12  3  …  n ≤ n  n  …  n = nn
∴ n! is O(nn) , taking C =1 and k =1.
Example 7. (see Figure 3)
常見function的成長速度由小至大排列：
1 < log n < n < n log n < n2 < 2n < n!
Theorem 2,3 Suppose that f1(x) is O(g1(x)) and f2(x)
is O(g2(x)), then
(f1+f2)(x) is O(max(|g1(x)|, |g2(x)|)),
(f1 f2)(x) is O(g1(x) g2(x)).
Ch3-21
Exercise 7,11,19
Exercise 19(c) : f (n) = (n!+2n)(n3+log(n2+1))
 (n!+n!)(n3+n3)
= 4n3n!
∴ f (n) is O(n3n!) 取 C = 4, k = 3
Ch3-22
3.3 Complexity of Algorithms
Q : How can the efficiency of an algorithm be
analyzed ?
Ans : (1) time (2) memory
Def :
 Time
complexity : an analysis of the time required to solve
a problem of a particular size.
(評量方式 : 計算 # of operations，如 “comparison”次數，
“加法” 或 “乘法” 次數等)
 Space complexity : an analysis of the computer memory
required to solve a problem of a particular size. (通常是資
料結構探討的範圍)
Ch3-23
Example 1. Describe the time complexity of
Algorithm 1.
Sol : (計算 # of comparisons)
Algorithm 1. ( Find Max )
procedure max(a1,…,an : integers)
max := a1
for i := 2 to n
if max < ai then max := ai
i 值一開始 = 2
逐次加一，並比較是否>n.
當 i 變成 n+1 時
因比 n 大，故結束 for 迴圈。
∴ 共有 n 次 comparison
{ max is the largest element }
共有 n-1 次 comparison
故整個演算法共做 2n-1 次
comparison
其 time complexity 為 O(n).
Ch3-24
Example 2. Describe the time complexity of the
linear search algorithm.
Algorithm 2 ( Linear Search )
procedure ls ( x : integer , a1,…,an : distinct integers )
i := 1
Sol : ( 計算 # of comparisons )
While ( i  n and x ≠ai )
(Case 1) 當 x = ai for some i  n 時
i := i +1
此行只執行 i 次，故此行共2i次比較
if i  n then location := i
加上if，共計 2i +1次 comparisons.
else location := 0
(Case 2) 當 x ≠ ai for all i 時
location = i  x = ai
= 0  x  ai i
此行執行 n 次後
第 n + 1 次時 i = n + 1 > n 即跳出
∴共計 2n+2 次 comparisons
由(1)、(2)取 worst-case
演算法的 time complexity為 O(n)
Ch3-25
Example 4. Describe the average-case performance of the
linear search algorithm, assuming that x is in the list.
Alg. 2 ( Linear Search )
procedure ls ( x,a1,…,an)
i := 1
While ( i  n and x ≠ai )
i := i +1
if i  n then location := i
else location := 0
Sol : ( 計算 “平均比較次數” )
已知當 x = ai 時，共需 2i + 1 次比較.
( by Example 2 )
x = a1,a2, …, 或 an 的機率都是 1/n.
∴平均比較次數 (即期望值)
= ( x = a1 的比較次數 ) × ( x = a1 的機率 )
+ ( x = a2 的比較次數 ) × ( x = a2 的機率 )
+…
+ ( x = an 的比較次數 ) × ( x = an 的機率 )
= 3 × 1/n + 5 × 1/n + … + ( 2n+1) × 1/n
= ( 3+5+…+(2n+1)) / n
=
(2n  4)  n
2
/n=n+2
∴average-case的time complexity為O(n)
Ch3-26
Example 3. Describe the time complexity of the binary
search algorithm.
Alg. 3 ( Binary Search )
procedure bs ( x : integer, a1,…,an : increasing integers )
i := 1 { left endpoint }
j := n { right endpoint }
Sol : 設 n = 2k 以簡化計算
while i < j
/* ( k+1 次)
(若 n < 2k，其比較次數必小於等
begin
於 n = 2k 的情況)
m :=  ( i + j ) / 2 
因while迴圈每次執行後
if x > am then i := m+1 /* ( k次 ) 整個 list 會切成兩半
else j := m
故最多只能切 k 次
end
就會因 i = j 而跳出迴圈
if x = ai then location := i /* ( 1次 ) ∴共比較 2k+2 次
else location := 0
time complexity 為
O(k) = O(log n)
Ch3-27
Example 5. What is the worst-case complexity of the
bubble sort in terms of the number of comparisons
made ?
Sol : 共 n-1 個 pass
第 i 個 pass 需 n – i 次比較
∴共計
(n-1)+(n-2)+…+1
procedure bubble_sort ( a1,…,an )
for i := 1 to n -1
for j := 1 to n – i
if aj > aj+1 then
interchange aj and ai+1
{ a1,…,an is in increasing order }
n( n  1)
=
2
次比較
∴ O(n2)
Note 1. 不管何種 case 都需做 n( n  1)
次比較。
2
Note 2. For 迴圈所需比較次數通常會省略，因此Example 5,6 不再考慮。
Ch3-28

Example 6. What is the worst-case complexity of
the insertion sort in terms of the number of
comparisons made ?
procedure insertion_sort ( a1,…,an ) Sol :
for j := 2 to n
做最多次比較的情況如下：
begin
在考慮 aj 時
i := 1
a1 < a2 < … < aj-1 < aj
while aj > ai
此時共做 j 次比較
i := i +1
m := aj
故共計
n( n  1)
2+3+…+n =
-1 次比較
for k := 0 to j - i -1
2
aj-k := aj-k-1
ai := m
 O(n2)
end
(即 worst case 是 a1 < a2 < … < an)
{ a1,…,an are sorted }
Ch3-29
Table 1. Commonly Used Terminology
Complexity
O(1)
O(log n)
O(n)
O(n log n)
O(nb)
O(bn) , b >1
O(n!)
Terminology
constant complexity
Logarithmic complexity
Linear complexity
n log n complexity
Polynomial complexity
Exponential complexity
Factorial complexity
Exercise : 7,8,13
Ch3-30
3.4 The integers and division
※探討一些 Number Theory 的基本觀念
Def 1. a,b : integers, a≠0.
a divides b (denote a | b) if cZ , b=ac .
(a : a factor of b, b : a multiple of a)
(a b if a does not divide b)
Corollary 1. If a,b,c Z and a | b , a | c.
then a | mb+nc whenever m,nZ
Def 2. In the quality a = dq + r with 0  r < d, d is called
the divisor (除數), a is called the dividend (被除數),
q is called the quotient (商數), and r is called the
remainder (餘數).
 q = a div d, r = a mod d
Ch3-31
If a,bZ, mZ+, then
a is congruent (同餘) to b modulo m if m | (a-b).
(denote a≡b (mod m)).
Thm 4. Let mZ+, a,bZ.
a≡b (mod m) iff kZ, s.t. a=b+km.
Thm 5. Let mZ+, a,bZ.
If a≡b (mod m) and c≡d (mod m),
then a+c≡b+d (mod m) and ac≡bd (mod m).
Def 3.
Ch3-32
3.5 Primes and Greatest Common
Divisors
Def 1. pZ+ -{1} is called prime (質數)
if a p, 1<a< p, aZ+.
p is called composite (合成數) otherwise.
Thm 1. (The fundamental theorem of arithmetic)
Every positive integer greater than 1 can be
written uniquely as a prime or as the product of
two or more primes where the prime factors are
written in order of nondecreasing size.
Example 2.
The prime factorization (因數分解) of 100 is 2252.
Ch3-33
Thm 2. If n is a composite integer, then n has a
prime divisor less than or equal to n .
Thm 3. There are infinitely many primes.
Pf. 假設質數只有n個：p1, p2, …, 及 pn，
令Q = p1p2…pn+1, 因 p1, …, pn 都不整除Q，得證。
※目前為止所知最大的質數是2p -1的形式, where p is
prime. 稱為 Mersenne primes (梅森質數).
Example 5. 22-1=3, 23-1=7, 25-1=31 are primes,
but 211-1=2047=2389 is not a prime.
Def 2. gcd ( greatest common divisor )
Def 3. relatively prime (互質)
Def 5. lcm ( least common multiple )
Ch3-34
Exercise 14. How many zeros are there at the end of
100! ?
Sol : 計算12  3  …  100=10k m, where 10 m
∵ 10=2  5，又2的次數必定比5多
∴ 計算1  2  3  …  100=5k  n, where 5 n
∵ 5,10,15,20,…,100才有因數5,
而25,50,75,100有因數25
∴ k=24  共有24個0
Homework : 試寫一alg.
求出  n 的所有質數
Ch3-35
3.6 Integers and Algorithms
※The Euclidean Algorithm (輾轉相除法求 gcd )
Example : Find gcd(91,287)
Sol:
if x|91 & x|287  x|14
287 = 91 3 + 14
∴gcd (91,287) = gcd(91,14)
gcd (91,14) = gcd (14,7)
91 = 14  6 + 7
gcd (14,7) = 7
14 = 7  2
∴ gcd(91,287) = 7
Lemma 1 Let a = bq + r, where a, b, q, and r Z.
Then gcd(a, b) = gcd (b, r).
Ch3-36
Algorithm 6. ( The Euclidean Algorithm)
procedure gcd ( a, b : positive integers)
x := a
y := b
while y≠0
begin
r := x mod y ( if y > x then r = x)
x := y
y := r
end
{ gcd (a, b) = x }
Exercise : 23,25
eg. 求 gcd (6,12)
x=6
y = 12
while y≠0
r = 6 mod 12 =6
x = 12
y=6
while y≠0
r = 12 mod 6 = 0
x=6
y=0
while y = 0 , end.
∴ gcd (6,12) = 6
Ch3-37
3.7 Applications of Number Theory
※介紹中國餘數定理
Example 5. 孫子算經 :「某物不知其數，三三數之餘二，五五數之餘
三，七七數之餘二，問物幾何 ?」
i.e.
x ≡ 2 (mod 3)
x ≡ 3 (mod 5)
x=?
x ≡ 2 (mod 7)
Theorem 4. (The Chinese Remainder Theorem)
Let m1,m2,…,mn be pairwise relatively prime positive integers.
The system
x ≡ a1 (mod m1)
x ≡ a2 (mod m2)
:
x ≡ an (mod mn)
has a unique solution modulo m = m1m2…mn.
(即有一解 x, where 0 x < m , 且所有其他解 mod m都等於 x)
Ch3-38
Proof of Thm 4:
Let Mk = m / mk  1 k  n
∵ m1, m2,…, mn are pairwise relatively prime
∴ gcd (Mk , mk) = 1
 integer yk s.t. Mk yk≡1 (mod mk)
( by Thm. 3, 此處不証)
 ak Mk yk≡ak (mod mk) ,  1  k  n
Let x = a1M1y1+a2M2y2+…+anMnyn
∵ mi | Mj , i≠j
∴ x≡ak Mk yk≡ak (mod mk)  1 k  n
x 即為一解
Ch3-39
Example 6. (承 Example 5 題目敘述，求解)
Let m = 357 = 105
M1 = m / 3 = 35
M2 = m / 5 = 21
M3 = m / 7 = 15
M1 y1
∵ 35≡2 (mod 3)  35  2 ≡ 1 (mod 3)
21≡1 (mod 5)
15≡1 (mod 7)
M2
y2
M3
y3
 21  1 ≡ 1 (mod 5)
 15  1 ≡ 1 (mod 7)
∴ x = a1M1y1 + a2M2y2 + a3M3y3
= 2  35  2 + 3  21  1 + 2  15  1 = 233 ≡ 23 (mod 105)
∴ 最小的解為23，其餘解都等於 23+105t for some tZ+
Exercise : 19
Ch3-40
Exercise 18. Find all solutions to the system of
congruences x≡2 (mod 3)
x≡1 (mod 4)
x≡3 (mod 5)
Sol :
a1=2 , a2=1 , a3=3,
m1=3 , m2=4 , m3=5
m=345=60
M1=20 , M2=15 , M3=12
20≡2 (mod 3)  202≡1 (mod 3)
15≡3 (mod 4)  153≡1 (mod 4)
12≡2 (mod 5)  123≡1 (mod 5)
∴ x = 2202+1153+3123
= 80+45+108=233≡53 (mod 60)
Ch3-41
3.8 Matrices
Algorithm 1. Matrix multiplication
procedure matrix_multiplication(A : mk matrix, B : kn matrix )
for i := 1 to m
for j :=1 to n
begin
cij := 0
 b1 j 
for q := 1 to k
b 
2j
cij := cij + aiqbqj
cij  ai1 ai 2  aik   
  
end
 
 bkj 
{ c =[cij] = AB }
Exercise : 試寫一algorithm求At.
Ch3-42