Transcript Design & Analysis Algoritms

Design & Analysis
of Algoritms
Ernastuti
Algorithm
• The word algorithm comes from the name
of a Persian author,
Abu Ja’far Mohammed ibn Musa al
Khowarizmi (825 A.D)
who wrote a textbook on mathematics.
Algorithm
• Algorithm is any special method of solving
a certain kind of problem.
• But this word has taken on a special
significance in computer science, where
algorithm has come to refer to a precise
method useable by a computer for the
solution of a problem.
• This is what makes the notion of an
algorithm different from words such as
process, technique or method.
• An Algorithm is composed of finite set of
steps, each of which may require one or
more operations.
• Each operation must definite, meaning
that it must be perfectly clear what should
be done.
• Directions such as “compute 5/0” or “add 6
or 7 to x” are not permitted because it is
not clear what the result is or which of the
two possibilities should be done.
• Another important property each operation
should have is that it be effective; each
step must be such that it can, at least in
principle, be done by a person using pencil
and paper in a finite amount of time.
• Performing arithmetic on integers is an
example of an effective operation, but
arithmetic with real numbers is not, since
some values may be expressible only by
an infinitely long decimal expansion.
Adding two such numbers would violate
the effectiveness property.
• An algorithm produces
one or more outputs
and may have
zero or more inputs
which are externally supplied.
• Another important criterion which will be
assumed about algorithms is that they
terminate after a finite number of
operations.
• There is another word for an algorithm which
obeys all of the above properties (finite set of
steps, definite, effective) except termination, and
that is computational procedure.
• One important example of a computational
procedure is the operating system of a digital
computer.
• This procedure is designed to control the
execution of jobs, such that when no jobs are
available, it does not terminate, but continues in
a waiting state until a new job is entered.
• Though computational procedures include
important examples such as this one, we will
restrict our study to these computional
procedures which always terminate.
Algorithm
• Informally, an Algorithm is any welldefined computational procedure that
takes some value, or set of values, as
input and produces some value, or set of
values, as output.
• An Algorithm is thus a squence of
computational steps that transform the
input into the output.
Program
• A Program is the expression of an
algorithm in a programming language.
• In order to help us achieve the criterion of
definiteness, algorithm will be written in a
programming language.
• Most of you have already programmed
and run some algorithms on a computer.
• This is desirable because before one
studies a concept in general it helps if one
has had some practical experience with it.
• Perhaps you had some difficulty getting
started in formulating an initial solution to a
problem, or perhaps you were unable to
decide which of two algorithms was better.
• The goal of this course is to teach you how
to make these decisions.
• The study of algorithms includes many important
and active areas of research.
• There are perhaps five distinct areas of study
one can identify :
1. How to devise algorithms
2. How to analyze algorithms
3. How to express algorithms
4. How to validate algorithms
5. How to test algorithms
1. How to devise algorithms
• The act of creating an algorithm is an art
which may never be fully automated.
• A major goal of this course is to study
various design techniques which have
proven to be useful in that they have often
yielded good algorithms.
Design of algorithm
• A knowledge of design will certainly help one to create a
good algorithms, yet without the tools of analysis there is
no way to determine the quality of the result.
• There are several design strategies/ techniques , such
as :
devide-and-conquer,
the greedy method,
dynamic programming,
search- and-traversal,
backtracking
& branch-and-bound.
2. How to analyze algorithms
• This field of study is called analysis algorithms.
• As an algorithm is executed, it makes use of the
computer’s central processing unit (CPU) to
perform operations and it uses the memory to
hold the program and its data.
• Analysis of algorithms refers to the process of
determining how much computing time and
storage an algorithm will require.
• This is a challenging one which sometimes
requires great mathematical skill.
Analysis of algorithm
• Analyzing even a simple algorithm can be
a challenge.
• The mathematical tools required may
include discrete combinatorics, elementary
probability theory, algebraic dexterity, and
the ability to identify the most significant
terms in a formula.
3. How to express algorithms
• A Program is the expression of an
algorithm in a programming language.
4. How to validate algorithms
• Once an algorithm is devised it is necessary to
show that it computes the correct answer for all
possible legal inputs.
• We refer to this process as algorithm validation.
• The purpose of the validation is to assure us that
this algorithm will work correctly independent of
the issues concerning the programming
language it will eventually be written in.
• Once the validity of the method has been shown, a
program can be written and a second phase begins. This
phase is referred to as program verification.
• A proof of correctness requires that the solution be
stated in two forms.
• One form is usually as a program which is annotated by
a set of assertions about the input and output variables
of the program. These assertions are often expressed in
the predicate calculus.
• The second form is called a specification and this may
also be expressed in the predicate calculus.
• A complete proof of program correctness requires that
each statement of the programming language be
pricisely defined and that all basic operations be proved
correct.
• All these details may cause a proof to be very much
longer than the program.
5. How to test a program
• Testing a program really consists of two phases
Debugging and Profilling.
• Debugging is the process of executing
programs on sample data sets to determine if
faulty results occur and, if so, to correct them !
• A proof of correctness is much more valuable
than a thousand tests, ( if that proof is correct ),
since it guarantees that the program will work
correctly for all possible inputs.
• Profilling is the process of executing a
correct program on data sets and
measuring the time and space it takes to
compute the results.
• This timing figures are useful in that they
may confirm a previously done analysis
and point out logical places to perform
useful optimization.
• As we can’t hope to cover all of these five
categories completely, we will content
ourselves with concentrating on design
and analysis, spending less time on
program construction and correctness.
• One can see that the subject of algorithms
is a very diverse and challenging one.
What are the prerequisites for
understanding the D&AoA course
• You should have some programming
experience. In particular, you should
understand recursive procedures and
simple data structures as arrays and
linked lists.
• You should have some facility with proofs
by mathematical induction.
• Algorithm devised to solve the same problem
often differ dramatically in their efficiency.
• These differences can be much more significant
than the difference between a personal
computer and supercomputer.
• As an example, let us pit a supercomputer
running insertion sort againts a small personal
computer running mergesort.
• They each must sort an array of one million
numbers.
• Suppose the supercomputer executes 100 million
instructions per second, while the personal computer
executes only one million instructions per second.
• To make the difference even more dramatic, suppose
that the world’s craftiest programmer codes insertion sort
in machine language for the supercomputer, and the
resulting code requires 2n2 supercomputer instructions
to sort n numbers.
• Mergesort, on the other hand, is programmed for
personal computer by an average programming using a
high level language with an inefficient compiler, with the
resulting code taking 50n log n .
• To sort a million numbers :
The supercomputer takes
2.(106)2 instructions / 108 instructions/seconds
= 20,000 seconds  5.56 hours
While the personal computer takes
50.(106) log 106 instr / 106 instructions/seconds
 1,000 seconds  16.67 minutes
• By using an algorithm whose running time has a lower
order of growth, even with a poor compiler, the personal
computer runs 20 times faster than the supercomputer.
• This example shows that algorithms, like computer
hardware, are a technology.
• Total system performance depends on choosing efficient
algorithms as much as on choosing fast hardware.
• Just as rapid advances are being made in other
computer technologies, they are being made in
algorithms as well.