Intro to Analysis

San Diego State University

Intro to Analysis Slide # 1

Intro to Analysis Slide # 2

Intro to Analysis Slide # 3

If analysis of algorithms is the answer, what is the question?

Given two or more algorithms for the same task, which is better?

- Under which condition is bubble sort better than insertion sort?

What computing resources does an algorithm require?

- How long will it take bubble sort to sort a list of N items?

Goal of mathematical analysis is a function of the resources required of an algorithm

On what computer?

Intro to Analysis Slide # 4

- Single processor
- Instructions executed sequentially
- Each operation requires the same amount of time
- "Standard" set of operations

Single cost vs. Lg(N) cost

- Time required for basic operation?
- 3 + 6
- 1234!

Intro to Analysis Slide # 5

A finite set of rules which gives a sequence of operations for solving a specific type of problem.

The set of rules must have the following features.

1) Finiteness

- Must terminate after a finite number of steps

2) Definiteness

- Each step of an algorithm must be precisely defined
- No ambiguous statements allowed

3) Effectiveness

- A human must be able to perform each step using paper and pencil in a finite amount of time.

4) Input

- Each algorithm has some input.

5) Output

- Each algorithm has some output.

Intro to Analysis Slide # 6

Cost | times | ||

A[0] = - infinity | 1 | ||

for K = 2 to N do | N | ||

begin | 0 | N-1 | |

J = K; | N-1 | ||

Key = A[J] | N-1 | ||

| |||

| 0 | ||

A[J] = A[J-1]; | |||

J = J - 1; | |||

| |||

A[J] = Key; | N-1 | ||

end for; | N-1 |

Intro to Analysis Slide # 7

where

What is ?

Intro to Analysis Slide # 8

Intro to Analysis Slide # 9

Intro to Analysis Slide # 10

Intro to Analysis Slide # 11

- Everyone incorrectly writes:

- instead of:

- or does not mean

- or even that there is an n such that

- Let f(n) = 2n + 10, and g(n) = n then
- f(n) = O(g(n)) but f(n) > g(n)

- Using O( ) when mean

Intro to Analysis Slide # 12

Complexity

- Resources required by the algorithm as a function of the input size

Worst-case Analysis

- Complexity of an algorithm based on worst input of each size

Average-case Analysis

- Complexity of an algorithm averaged over all inputs of each size

Comparisons | Element moves | |

worst case | (N+1)N/2 - 1 | (N-1)N/2 |

average case | (N+1)N/4 - 1/2 | (N-1)N/4 |

Intro to Analysis Slide # 13

Input A - array of keys to be sorted

L - index of left most item in array to be sorted

R - index of right most item in array to be sorted

Output A with A[L], A[L+1], ...,A[R] in order

Quicksort(A: array[1..N]; L,R:integer) if L < R then begin Mid := partition(L, R); Quicksort(A,L,Mid-1); Quicksort(A,Mid+1,R); end;

Intro to Analysis Slide # 14

Input A - array to be split

Left - index of left most element to be split

Right - index of right most element to be split

partition(A,Left,Right) begin L := Left + 1; R := Right; Key := A[Left]; repeat repeat L = L + 1 until A[L] >= Key; repeat R = R - 1 until A[R] <= Key; swap(A[L],A[R]); until R <= L A[Left] := A[L]; A[L] := A[R]; A[R] := Key; return R; end

Intro to Analysis Slide # 15

Let Q(N) = time complexity of quicksort on input of size N

How to solve for Q(N)?

Iteration

Substitution

Master theorem

Intro to Analysis Slide # 16

Theorem 4.1

Let a>= 1 and B > 1 be constants. Let f(n) be a function, and let T(n) be defined by

- T(n) = aT(n/b) + f(n)

where n/b may be either or . Then:

- if for some constant > 0 then

- if then

- if for some constant and if a*f(n/b)<= c*f(n) for some constant c < 1 and all large n, then

Intro to Analysis Slide # 17Bubble vs. Insertion Sort

Worst case | Average case | |

Bubble sort | ||

Insertion Sort |

Comparisons | Element moves | |

worst case | (N-1)N/2 | 3(N-1)N/2 |

average case | (N-1)N/2 | 3(N-1)N/4 |

best case | (N-1)N/2 | 0 |

Comparisons | Element moves | |

worst case | (N+1)N/2 - 1 | (N-1)N/2 |

average case | (N+1)N/4 - 1/2 | (N-1)N/4 |

best case | N - 1 | 0 |

Intro to Analysis Slide # 18

N | Bubble | Insertion |

100 | 1 | 1 |

200 | 5 | 3 |

400 | 19 | 11 |

800 | 79 | 42 |

1600 | 317 | 166 |

Average Case

N | Bubble | Insertion |

100 | 1 | 0 |

200 | 3 | 1 |

400 | 14 | 5 |

800 | 56 | 21 |

1600 | 228 | 84 |