Transcript Hash Tables - 中山大学信息科学与技术学院
Algorithms Design and its Applications Lecture 1 Introduction
Something about me 刘晓铭 • Home Page: http://sist.sysu.edu.cn/~isslxm • Email: [email protected]
• Office: 信息学院楼 526B
Something about this course • Course homepage: http://sist.sysu.edu.cn/~isslxm/ada/cs2012/inde x.html
• BBS: http://bbs.sysu.edu.cn/bbsdoc?board=ACMICPC • Office Hours: 每周四下午 2:00 – 4:00 ，信息学院楼 526B 其它时间可邮件预约。
Something about this course • 1. 平时作业：书面作业 + 编程练习 http://soj.me/ • 2. 研究型的小 Project ：选一个问题，编程 实现解决这一问题的若干种算法，在不同 规模的测试数据上，比较各种算法的性能 • 3. 期末考试： ACM 方式 , 自动测评
FAQ • Q: 这门课是为了培养参加 ACM / ICPC 比赛 的队员而专门开设的吗？ • A: No. Some important topics not included in ACMICPC • Advanced Algorithm Analysis Techniques • NP-Completeness Proofs • Randomized and Approximation Algorithms • Quantum Algorithms
推荐 书籍
武林秘籍
Why take this course?
You want to be a computer scientist?
Is your goal to be a mundane programmer?
Or a great leader and thinker?
Original Thinking
Boss assigns task: – Given today’s prices of pork, grain, sawdust, … – Given constraints on what constitutes a hotdog.
– Make the cheapest hotdog.
Everyday industry asks these questions.
Your answer: • Um? Tell me what to code.
With more sophisticated software engineering systems, the demand for mundane programmers will diminish.
Your answer: • I learned this great algorithm that will work.
Soon all known algorithms will be available in libraries.
Your answer: • I can develop a new algorithm for you.
Great thinkers will always be needed.
Course Content • A list of algorithms. – Learn their code.
– Trace them until you are convinced that they work.
– Implement them.
– Worry about details.
class InsertionSortAlgorithm extends SortAlgorithm { void sort(int a[]) throws Exception { for (int i = 1; i < a.length; i++) { int j = i; int B = a[i]; while ((j > 0) && (a[j-1] > B)) { a[j] = a[j-1]; j--; } a[j] = B; }}
Course Content • A survey of algorithmic design techniques.
• Abstract thinking.
• How to develop new algorithms for any problem that may arise.
Study: • Many experienced programmers were asked to code up binary search.
Study: • Many experienced programmers were asked to code up binary search.
80% got it wrong
What did they lack?
What did they lack?
• Formal proof methods?
What did they lack?
• Formal proof methods?
Yes, likely Industry is starting to realize that formal methods are important.
But even without formal methods …. ?
What did they lack?
• Fundamental understanding of the algorithmic design techniques.
• Abstract thinking.
Course Content Notations, analogies, and abstractions for developing, thinking about, and describing algorithms so correctness is transparent
A survey of fundamental ideas and algorithmic design techniques
For example . . .
Some Math Logic Quantifiers g " b Loves(b,g) " b g Loves(b,g ) Classifying Functions f(i) = n Q(n) Time Complexity t(n) = Q(n 2 )
Input Size
Adding Made Easy ∑ i=1 f(i).
Logs and Exps 2 a × 2 b 2 log n = 2 = n a+b Recurrence Relations T(n) = a T(n/b) + f(n)
Recursive Algorithms
Recursive Back Tracking
Greedy Algorithms
Dynamic Programing 5 2 4 3 2 4 5 3 7 2 8 4 5 5 9 6 6 4 5 2 1 8 8 3
Graph Search Algorithms
Network Flows
Reduction
=
Rudich www.discretemath.com
Useful Learning Techniques
Read Ahead You are expected to read the lecture notes
before
the lecture.
This will facilitate more productive discussion during class.
Also please proof read assignments & tests.
Be Creative •Ask questions. • Why is it done this way and not that way?
Guesses and Counter Examples • Guess at potential algorithms for solving a problem. • Look for input instances for which your algorithm gives the wrong answer.
• Treat it as a game between these two players.
Refinement: The best solution comes from a process of repeatedly refining and inventing alternative solutions Rudich www.discretemath.com
Please feel free to ask questions!
Help me know what people are not understanding Slow down the slides We do have a lot of material Please don’t shout out answers for at least 10sec.
什么是算法？
Informally, an algorithm is any well – defined computational procedure that takes some value, or set of values, as input and produces some value, or set of values, as output .
-- Introduction to Algorithms
Knuth said
Knuth (1968, 1973) has given a list of five properties that are widely accepted as requirements for an algorithm: Finiteness: "An algorithm must always terminate after a finite number of steps ... a very finite number, a reasonable number" Definiteness: "Each step of an algorithm must be precisely defined; the actions to be carried out must be rigorously and unambiguously specified for each case" Input: "...quantities which are given to it initially before the algorithm begins. These inputs are taken from specified sets of objects" Output: "...quantities which have a specified relation to the inputs" Effectiveness: "... all of the operations to be performed in the algorithm must be sufficiently basic that they can in principle be done exactly and in a finite length of time by a man using paper and pencil"
Finally … The word
algorithm
does not have a generally accepted definition.
From wikipedia
Analysis of algorithms • The theoretical study of computer-program performance and resource usage.
• What’s more important than performance?
• modularity • correctness • maintainability • robustness • functionality • user-friendliness • programmer time • simplicity • extensibility • reliability
Why study algorithms and performance?
• Algorithms help us to understand scalability.
• Performance often draws the line between what is feasible and what is impossible.
• Algorithmic mathematics provides a language for talking about program behavior.
• Performance is the currency of computing.
• The lessons of program performance generalize to other computing resources. • Speed is fun! Need for speed!
Three questions on a algorithm 1. Is it correct?
2. How much time does it take, as a function of n?
3. And can we do better?
Fibonacci number 0, 1, 1, 2, 3, 5, 8, 13, 21, 34, …, Leonardo of Pisa (Fibonacci) 1170-1250
Fibonacci number •
An exponential algorithm int fib1(n) { if ( n==0 ) return 0; if ( n==1 ) return 1; return fib1(n-1) + fib1(n-2); }
How much time does it take?
Let T(n) be the number of computer steps needed to compute fib1(n).
T(n)<=2 for n<=1 T(n)=T(n-1) + T(n-2) + 3 for n>1 (two recursive invocations of fib1, plus three computer steps to check on the value of n and a final addition.) Therefore, T(n) >= F n .
The running time of the algorithm grows as fast as the Fibonacci numbers! T(n) is exponential in n.
How much time does it take?
How bad exponential time is?
To compute F 200 , the fib1 algorithm executes T(200)>=F 200 >=2 138 elementary computer steps.
On a computer which clocks 40 trillion steps per second, fib1(200) would take at least 2 92 seconds.
How much time does it take?
Moore’s law: computer speeds doubles roughly every 18 months. Computers get roughly 1.6 times faster each year. It helps?
The running time of fib1(n) is proportional to
2 0.694n
= (1.6) n
, so it takes 1.6 times longer to compute F n+1 than F n .
So if we reasonably compute F 100 ith this year’s technology, then next year we will manage F 101 . And the year after, F 102 . And so on.
Can we do better?
• Why fib1 is so slow?
Can we do better?
•
An polynomial algorithm int fib2(n) { if ( n==0 ) return 0; int f[n]; // create an array f[0…n] f[0] = 0, f[1] = 1; for (int i=2; i<=n; i++) f[i] = f(i-1) + f(i-2); return f[n]; }
How long does fib2 take?
• The inner loop consists of a single computer step and it executed n-1 times. Therefore the number of computers steps used by fib2 is linear in n.
• It is now perfectly reasonable to compute F 200 or even F 200000 .
Can we do even better?
Write the equations F 1 = F 1 and F 2 = F 0 +F 1 in matrix notation: So, in order to compute Fn, it suffices to raise this 2*2 matrix, call it X, to the nth power.
Two 2*2 matrices can be multiplied using 4 additions and 8 multiplications. How many matrix multiplications suffice for computing X n ?
Efficient Exponentiation
long pow( long x, int n) { if( n == 0 ) return 1; if( n == 1 ) return x; if( n%2==0 ) return pow(x*x,n/2); else return pow(x*x,n/2) * x; } O(log n)
matrix multiplications suffice for computing X n ?
Can we do even better?
Thus the number of arithmetic operations needed by our matrix-based algorithm, callit fib3, is just O(logn), as compared to O(n) for fib2.
Have we broken another exponential barrier?
为什么学习算法
原因之一：广泛的应用 • Human Genome Project • Internet • Electronic commerce security • Other manufacturing and commercial setting • Etc.
为什么学习算法
原因之二：强大的力量
为什么学习算法
原因之三： Just for fun!
The RAM Model of Computation • 用基本指令条数来精确度量一个算法的效 率很困难： – 从理论上分析及其繁琐，从实际中估计也颇为 困难。 – 不同的计算机采用的指令集、缓存策略等技术 细节不同。 • 寻找一种简便易行的，与具体机器无关的 度量方法，它是以输入规模为自变量的函 数。
The RAM Model of Computation • Random Access Machine (RAM) – Each simple operation (+, *, -, =, if, call) takes exactly one time step.
– Loops and subroutines are not considered simple operations. Instead, they are the composition of many single-step operations.
– Each memory access takes exactly one time step. Further, we have as much memory as we need. The RAM model takes no notice of whether an item is in cache or on the disk.
Definition
： The Big-O Notation
f ( n )
，
g ( n ) are positive function about natural number. If there is a constant c >0
，
and natural number n 0
，
When n > n 0
，
there is f ( n )= O ( g ( n )) f ( n ) < c g ( n ), so If there is a constant c >0
，
and natural number n 0
，
When n > n 0
，
there is f ( n )= Ω( g ( n )) f ( n ) > c g ( n ), so if f ( n )= O ( g ( n )), f ( n )= Ω( g ( n )) , then f ( n )= Θ( g ( n ))
The Big Oh Notation
Working with the Big Oh • Adding Functions
Working with the Big Oh • Multiplying Functions O(cf(n)) = O(f(n)) Ω(cf(n)) = Ω(f(n)) Θ(cf(n)) = Θ(f(n)) O(f(n)) * O(g(n)) = O(f(n) * g(n)) Ω(f(n)) * Ω(g(n)) = Ω(f(n) * g(n)) Θ(f(n)) * Θ(g(n)) = Θ(f(n) * g(n))
Working with the Big Oh • Transitive Experience If f(n) = O(g(n)) and g(n) = O(h(n)), then f(n)=O(h(n)).
Grow Rates
Popular ：
O
(1) <
O
(log
n
) <
O
(
n
) <
O
(
n
log
n
) <
O
(
n
2 )<
O
(
n
3 )
O
(2
n
) <
O
(
n
!) <
O
(
n n
)
Some rules
： O( f )+O( g )=O(max( f , g )) O( f )+O( g )=O( f+g ) O( f )O( g )=O( f *g ) O( c f (n)=O( f (n))
•
Popular: O(1), O(log 2 n), O(n), O(nlog 2 n) , O(n 2 ), O(2 n ), O(n!)
2 n T(n) 2000 20 10 5n 2 100n 20 200log 2 n n
Best, Worst and Average-Case Complexity
空间复杂度 • • • • 计算算法运行过程所占用的存储量作为问题规模 的函数 S(n) = O(f(n)).
通常以 简单变量的存储空间为单位 算法的存 储空间包括： 1 ）程序本身、常量和 变量等所需空间 2 ）程序运行中 动态申请的空间和递归调用所 需空 间 通常考 虑除输入和程序以外的空间 . 如果相 对于 输入是常数 , 则称为原地工作 . 如 sum 和 selectSort.
Running Time Calculations • A Simple Example • General Rules • Solutions for the maximum Subsequence Sum Problem
A Simple Example
3 4 } A simple program to calculate 1 3 +2 3 +..+n 3 . int sum(int n) { 1 2 int partialSum; partialSum = 0; for (int i=0; i<=n; i++) partialSum += i*i*i; return partialSum;
General Rules
Rule 1-For loops.
The running time of a for loop is at most the running time of the statements inside the for loop times the number of iterations.
Rule 2-Nested loops.
Analyze these inside out. The total running time of a statement inside a group of nested loops is the running time of the statement multiplied by the product of the size of all the loops.
General Rules
Rule 3-Consecutive Statements.
These just add.
for (i=0; i
General Rules
Rule 4-If/Else.
For the fragment if (condition) S1 else S2 the running time of an if/else statement is never more than the running time of the test plus the larger of the running times of S1 and S2.
• • The Maximum Subsequence Sum Problem 给一串整数
a
[1…
n
] ，求出它和最大的子序 列，即找出 1<=
i
<=
j
<=
n
，使
a
[
i
]+
a
[
i
+1]+…+
a
[
j
] 最大 介绍四个算法并分析时间复杂度 – – – – 枚举：
O
(
n
3 ) 优化的枚举： O(
n
2 ) 分治：
O
(
n
log
n
) 贪心：
O
(
n
)
算法一 • •
max := a[1]; for i:=1 to n do for j:=i to n do begin sum := 0; for k:=i to j do sum := sum + a[k]; if sum > max then max := sum; end;
思想：枚举所有的
i
和
j
，计算
a
[
i
]+…+
a
[
j
] ，选择最大的 时间复杂度如何分析？ – – – 三层循环 内层操作次数取决于
i
,
j
中层操作次数取决于
i
算法一分析 • • • • 当 i 和 j 一定的时候，内层循环执行 j-i+1 次
n n
总次数为
i
1 ( 应该如何计算？ 1) – – – – 方法一：直接计算 先计算里层求和号，得
i n
1 1 2 ( 1)( 2) 再加起来？好复杂 … 直接算法需要利用公式 1 2 +…+
n
2 =
O
(
n
3 ) 一般地，有 1
k
+…+
n k
=O(
n k
+1 ). 证明 : 数学归纳法
算法一分析（续） • • • • 总次数为：
n n
– (
j i
1 完全的计算太麻烦 – 1) 能不能不动笔就知道渐进时间复杂度呢？ 何必非要计算出详细的公式再化简呢？ – 里层求和计算出的结果是 – 1 2 +2 2 +…+
n
2 =
O
(
n
3 )
O
((
n
-
i
) 2 ) – 每步都化简！但是要保留外层需要的变量 结论：算法一时间复杂度为 O(
n
3 ) 经验：一般只能支持
n
<=200
算法二 • 思路 – – 枚举 i 和 j 后，能否快速算出 预处理！
a
[
i
]+…+
a
[
j
] 呢？ – – 记
s
[
i
] =
a
[1] +
a
[2] + … +
a
[
i
], 规定
s
[0] = 0 则可以在
O
(
n
) 的时间内计算出
s
[1], …,
s
[
n
]
s[0] := 0; for i:=1 to n do s[i] := s[i–1] + a[i];
算法二（续） • 有了
s
[
i
] ，怎么快速求
a
[
i
]+…+
a
[
j
] 呢？
a
[
i
]+…+
a
[
j
] = (
a
[1] + … +
a
[
j
]) – (
a
[1] + … +
a
[
i
-1]) =
s
[
j
] –
s
[
i
-1] 而
s
[
i
] 经过预处理以后可以直接读出！
max := a[1]; for i:=1 to n do for j:=i to n do if s[j] – s[i-1] > max then max := s[j] – s[i-1];
时间复杂度：预处理 + 主程序 =
O
(
n
+
n
2 )=
O
(
n
2 ).
n
<=5000
算法三 • • • 用一种完全不同的思路 最大子序列的位置有三种可能 – – – 完全处于序列的左半，即
j
<=
n
/2 完全处于序列的右半，即
i
>=
n
/2 跨越序列中间，即
i
<
n
/2<
j
用递归的思路解决！ – 设
max
(
i
,
j
) 为序列
a
[
i
…
j
] 的最大子序列 , 那么 …
算法三（续） • • 用递归的思路解决！ – – 设
max
(
i
,
j
) 为序列
a
[
i
…
j
] 的最大子序列 设
mid
• = (
i
+
j
)/2 ，即区间
a
[
i
…
j
] 的中点 最大的第一种序列为
max
(
i
,
mid
) • 最大的第二种序列为
max
(
mid
+1,
j
) 问题：最大的第三种序列为？？？ – 既然跨越中点，把序列
a
[
i
…
j
] 划分为两部分 •
a
[
i
…
mid
] ：最大序列用扫描法在
n
/2 时间内找到 – 一共只有
mid
-1=
n
/2 种可能的序列，一一比较即可 • •
a
[
mid
+1…
j
] ：同理 一共花费时间为
j
-
i
+1
算法三（续） // 返回数组在区间 [x,y) 中的最大连续和 int maxsum(int *A, int x, int y) { int i, m, v, L, R, max; if (y-x == 1) return A[x]; // 只有一个元素，直接返回 m = x + (y – x) / 2; // 分治第一步：划分成 [x,m) 和 [m,y) // 分治第二步，递归求解 max = maxsum(A, x, m) >? maxsum(A, m, y); // 分治第三步：合并 (1), 求分界点开始向左的最大连续和； v = 0; L = A[m-1]; for(i=m-1;i>=x;i--) L >?= v += A[i]; // 分治第三步：合并 [2], 求分界点开始向右的最大连续和； v = 0; R = A[m]; for(i=m;i
算法三分析 • 如何分析时间复杂度呢？ – – – 我们没有直接得到具体的
T
(
n
) 的式子 但是容易得到递推关系
T
(
n
) = 2
T
(
n
/2) +
n
• • 设时间复杂度的精确式子是
T
(
n
) 第一、二种序列的计算时间是
T
(
n
/2) ，因为序列长度 缩小了一半 • 第三种序列的计算时间是
n
由递归式四 , 得
T
(
n
)=O(
n
log
n
)
算法四 • 算法二的实质是求出
i
<=
j
, 让
s
[
j
]-
s
[
i
-1] 最大 – 对于给定的
j
，能否直接找到在
j
之前的最小
s
值呢？ – 从小到大扫描
j
•
j
=1 时，只有一个合法的
i
，即
i
=1, s[1-1]=0 • • 如果
s
[
j
] 变大，则最小的
s
值和上次一样 如果
s
[
j
] 再创新低，应该让
s
[
j
] 作为今后的最优
s
值
min_s := 0; For j :=1 to n do begin if s[j] < min_s then min_s := s[j]; if s[j] – min_s > max then max := s[j] – min_s; end;
时间复杂度很明显：
O
(
n
).
n
<= 1,000,000
总结 算法 枚举 扫描 时间复杂度 规模 O(n 优化枚举 O(n 2 ) 分治 O(nlogn) O(n) 3 ) 分析方法 约 200 分层求和 约 3000 明显 约 10 5 递归树 约 10 6 明显