## CS660 Combinatorial Algorithms Fall Semester, 1996 Intro to Analysis

[To Lecture Notes Index]
San Diego State University -- This page last updated Monday, 02 September, 1996

### Contents of Intro to Analysis

```                                          Intro to Analysis Slide # 1
```

## References

Introduction To Algorithms, Corman, Leiserson,Rivest, Chapters 1-4

Introduction To Algorithms, Corman, Leiserson,Rivest, Chapters 1-4 by Tuesday Sept. 10

```                                          Intro to Analysis Slide # 2
```

## Mathematical Analysis of Algorithms

```                                          Intro to Analysis Slide # 3
```

### Model of Computing

If analysis of algorithms is the answer, what is the question?

Given two or more algorithms for the same task, which is better?
Under which condition is bubble sort better than insertion sort?

What computing resources does an algorithm require?
How long will it take bubble sort to sort a list of N items?

Goal of mathematical analysis is a function of the resources required of an algorithm

On what computer?
```                                          Intro to Analysis Slide # 4
```
What is a Computer?
Random-access machine (RAM)
Single processor
Instructions executed sequentially
Each operation requires the same amount of time
"Standard" set of operations

Single cost vs. Lg(N) cost
Time required for basic operation?
3 + 6
1234!

```                                          Intro to Analysis Slide # 5
```
What is an Algorithm?

A finite set of rules which gives a sequence of operations for solving a specific type of problem.

The set of rules must have the following features.

1) Finiteness
Must terminate after a finite number of steps

2) Definiteness
Each step of an algorithm must be precisely defined
No ambiguous statements allowed

3) Effectiveness
A human must be able to perform each step using paper and pencil in a finite amount of time.

4) Input
Each algorithm has some input.

5) Output
Each algorithm has some output.

```                                          Intro to Analysis Slide # 6
```
Insertion Sort
```
```
 Cost times A[0] = - infinity 1 for K = 2 to N do N begin 0 N-1 ` J = K; ` N-1 ` Key = A[J]` N-1 ` while Key < A[J-1] do` ` begin` 0 ` A[J] = A[J-1];` ` J = J - 1;` ` end while;` ` A[J] = Key;` N-1 end for; N-1

```                                          Intro to Analysis Slide # 7
```
T(N) = cost of running Insertion Sort

where

What is ?
```                                          Intro to Analysis Slide # 8
```
Summations

```                                          Intro to Analysis Slide # 9
```

### Asymptotic Notation

Asymptotically tight bound - Theta

Asymptotic upper boundsBig-O
Little o

```                                          Intro to Analysis Slide # 10
```
Asymptotic lower bounds
Omega
Little Omega

```                                          Intro to Analysis Slide # 11
```
Common Myths and Errors
• Everyone incorrectly writes:
• or does not mean
or even that there is an n such that

Let f(n) = 2n + 10, and g(n) = n then
f(n) = O(g(n)) but f(n) > g(n)

• Using O( ) when mean

```                                          Intro to Analysis Slide # 12
```
Terms

Complexity
Resources required by the algorithm as a function of the input size

Worst-case Analysis
Complexity of an algorithm based on worst input of each size

Average-case Analysis
Complexity of an algorithm averaged over all inputs of each size
Insertion Sort
 Comparisons Element moves worst case (N+1)N/2 - 1 (N-1)N/2 average case (N+1)N/4 - 1/2 (N-1)N/4

```                                          Intro to Analysis Slide # 13
```

### Recurrences

Quicksort

Input A - array of keys to be sorted
L - index of left most item in array to be sorted
R - index of right most item in array to be sorted

Output A with A[L], A[L+1], ...,A[R] in order

```Quicksort(A: array[1..N]; L,R:integer)

if L < R then
begin
Mid := partition(L, R);
Quicksort(A,L,Mid-1);
Quicksort(A,Mid+1,R);
end;
```

```                                          Intro to Analysis Slide # 14
```
Partition procedure

Input A - array to be split
Left - index of left most element to be split
Right - index of right most element to be split
```
partition(A,Left,Right)

begin
L := Left + 1;
R := Right;
Key := A[Left];

repeat
repeat L = L + 1 until A[L] >= Key;
repeat R = R - 1 until A[R] <= Key;
swap(A[L],A[R]);
until  R <= L

A[Left] := A[L];
A[L] := A[R];
A[R] := Key;
return R;
end
```

```                                          Intro to Analysis Slide # 15
```
Recurences for Quicksort

Let Q(N) = time complexity of quicksort on input of size N

How to solve for Q(N)?
Methods for solving recurrences

Iteration
Substitution
Master theorem

```                                          Intro to Analysis Slide # 16
```
Master theorem

Theorem 4.1

Let a>= 1 and B > 1 be constants. Let f(n) be a function, and let T(n) be defined by
T(n) = aT(n/b) + f(n)

where n/b may be either or . Then:
if for some constant > 0 then
if then
if for some constant and if a*f(n/b)<= c*f(n) for some constant c < 1 and all large n, then

```                                          Intro to Analysis Slide # 17
```
Bubble vs. Insertion Sort
 Worst case Average case Bubble sort Insertion Sort

Bubble Sort
 Comparisons Element moves worst case (N-1)N/2 3(N-1)N/2 average case (N-1)N/2 3(N-1)N/4 best case (N-1)N/2 0

Insertion Sort
 Comparisons Element moves worst case (N+1)N/2 - 1 (N-1)N/2 average case (N+1)N/4 - 1/2 (N-1)N/4 best case N - 1 0

```                                          Intro to Analysis Slide # 18
```
Bubble vs. Insertion Sort
Timing Results
Worst Case
 N Bubble Insertion 100 1 1 200 5 3 400 19 11 800 79 42 1600 317 166

Average Case
 N Bubble Insertion 100 1 0 200 3 1 400 14 5 800 56 21 1600 228 84

What is wrong with this Picture?