Week 12 - Wednesday

    What did we talk about last time?

Hunters and prey Class variables Big Oh notation

  We want to compare the running time of one program to another We want a mathematical description with the following characteristics: 

Worst case

We care mostly about how bad things could be 

Asymptotic

We focus on the behavior as the input size gets larger and larger

      Enter Big Oh notation Big Oh simplifies a complicated running time function into a simple statement about its worst case growth rate All constant coefficients are ignored All low order terms are ignored 3n + 3 is O(n) Big Oh is a statement that a particular running time is no worse than some function, with appropriate constants

 147n 3 O(n 3 ) + 2n 2 + 5n + 12083 is 

n

1000 O(2

n

) + 2

n

is  15n 2 O(n 2 ) + 6n + 7log n + 145 is  659n + nlog n + 87829 is O(n log n)  Note: In CS, we use log 2 unless stated otherwise

 How long does it take to do multiplication by hand?

123 x 456 738 615 492__ 56088

   Let’s assume that the length of the numbers is n digits (n multiplications + n carries) x n digits + (n + 1 digits) x n additions Running time: O(n 2 )

 How do we find the largest element in an array?

int largest = array[0]; for ( int i = 1; i < array.length; i++ ) if ( array[i] > largest ) largest = array[i]; System.out.println

("Largest: " + largest);

  Running time: O(n) if n is the length of the array What if the array is sorted in ascending order?

System.out.println( "Largest: " + array[array.length-1]);

 Running time: O(1)

  Here is some code that sorts an array in ascending order What is its running time?

for ( int i = 0; i < array.length; i++ ) for ( int j = 0; j < array.length - 1; j++ ) if ( array[j] > array[j + 1] ) { int temp = array[j]; array[j] = array[j + 1]; array[j + 1] = temp; }

 Running time: O(n 2 )

 Here is a table of several different complexity measures, in ascending order, with their functions evaluated at n = 100

Description

Constant Logarithmic Linear Linearithmic Quadratic Cubic Exponential Factorial

Big Oh

O(1) O(log n) O(n) O(n log n) O(n 2 ) O(n 3 ) O(2

n

) O(n!)

f(100)

1 6.64

100 664.39

10000 1000000 1.27 x 10 30 9.33 x 10 157

  Computers get faster, but not in unlimited ways If computers get 10 times faster, here is how much a problem from each class could grow and still be solvable

Description

Constant Logarithmic Linear Linearithmic Quadratic Cubic Exponential Factorial

Big Oh

O(1) O(log n) O(n) O(n log n) O(n 2 ) O(n 3 ) O(2

n

) O(n!)

Increase in Size

Unlimited 1000 10 10 3-4 2-3 Hardly changes Hardly changes

       There is nothing better than constant time Logarithmic time means that the problem can become much larger and only take a little longer Linear time means that time grows with the problem Linearithmic time is just a little worse than linear Quadratic time means that expanding the problem size significantly could make it impractical Cubic time is about the reasonable maximum if we expect the problem to grow Exponential and factorial time mean that we cannot solve anything but the most trivial problem instances

   Memory usage can be a problem If you run out of memory, your program can crash Memory usage can have serious performance consequences too

  Remember, there are multiple levels of memory on a computer Each next level is on the order of 500 times larger and 500 times slower

Size Cache

• Actually on the CPU • Fast and expensive

Speed 1000X 1000X RAM

• Primary memory for a desktop computer • Pretty fast and relatively expensive

1000X 1000X Hard Drive

• Secondary memory for a desktop computer • Slow and cheap

   If you can do a lot of number crunching without leaving cache, that will be very fast If you have to fetch data from RAM, that will slow things down If you have to read and write data to the hard drive (unavoidable with large pieces of data like digital movies), you will slow things down a lot

    Memory can be easier to estimate than running time Depending on your input, you will allocate a certain number of objects, arrays, and primitive data types It is possible to count the storage for each item allocated Remember that a reference to an object or an array costs an additional 4 bytes on top of the size of the object

 Here are the sizes of various types in Java

Type boolean char int double Bytes 1 2 4 8 Type reference String object array of objects Type boolean[] char[] int[] double[] Bytes 16 + N 16 + 2N 16 + 4N 16 + 8N Bytes 4 40 + 2N 8 + size of members 16 + (4 + size of members)N

 Note that the N refers to the number of elements in the array or

String

   Lets say that I give you a list of numbers, and I ask you, “Is 37 on this list?” As a human, you have no problem answering this question, as long as the list is reasonably short What if the list is an array, and I want you to write a Java program to find some number?

  Easy!

We just look through every element in the array until we find it or run out

public static int for ( int find( int [] array, int i = 0; i < array.length; i++ ) if ( array[i] == number ) return i; number ) { } return -1;

 If we find it, we return the index, otherwise we return -1

    Unfortunately for you, we know about Big Oh notation Now we have some way to measure how long this algorithm takes How long, if n is the length of the array?

O(n) time because we have to look through every element in the array, in the worst case

  Is there any way to go smaller than O(n)?

What complexity classes even exist that are smaller than O(n)?

 O(1)    O(log n) Well, on average, we only need to check half the numbers, that’s ½ n which is still O(n) Darn…

    We can do better with more information For example, if the list is sorted, then we can use that information somehow How?

We can play a High-Low game

  Repeatedly divide the search space in half We’re looking for 37, let’s say

23 31 37 54 (Too low)

    How long can it take?

What if you never find what you’re looking for?

Well, then, you’ve narrowed it down to a single spot in the array that doesn’t have what you want And what’s the maximum amount of time that could have taken?

   We cut the search space in half every time At worst, we keep cutting n in half until we get 1 Let’s say x is the number of times we look: 1

x n

= 1 2

n

log n = = 2x

x

 The running time is O(log n)

     We can apply this idea to a guessing game First we tell the computer that we are going to pick a number between 1 and n We pick, and it tries to narrow down the number It should only take log n tries Remember log 2 (1,000,000) is only about 20

     This is a classic interview question asked by Microsoft, Amazon, and similar companies Imagine that you have 9 red balls One of them is just slightly heavier than the others, but so slightly that you can’t feel it You have a very accurate two pan balance you can use to compare balls Find the heaviest ball in the smallest number of weighings

  It’s got to be 8 or fewer We could easily test one ball against every other ball   There must be some cleverer way to divide them up Something that is related somehow to binary search

 We can divide the balls in half each time  If those all balance, it must be the one we left out to begin with

  How?

They key is that you can actually cut the number of balls into three parts each time   We weigh 3 against 3, if they balance, then we know the 3 left out have the heavy ball When it’s down to 3, weigh 1 against 1, again knowing that it’s the one left out that’s heavy if they balance

    The cool thing is…  Yes, this is “cool” in the CS sense, not in the real sense Anyway, the cool thing is that we are trisecting the search space each time This means that it takes log 3 find the heaviest ball n weighings to We could do 27 balls in 3 weighings, 81 balls in 4 weighings, etc.

  Sorting Lab 12

 Finish Project 4 

Due Friday before midnight