## CS 662 Theory of Parallel Algorithms Numerical Methods part 2

[To Lecture Notes Index]
San Diego State University -- This page last updated April 30, 1996

### Contents of Numerical Methods part 2 Lecture

Matrix Matrix Multiplication and Scan

Let A and B be a n*n matrices

Compute C = A*B

We have

Scan can compute in log(n) time using n/2 processors

Since there are elements of C

Scan takes log(n) time with processors to compute C
Slowdown

Let A, B, and C be a n*n matrices such that
C = A*B

Let be (n/2)*(n/2) matrices such that
and

Then

Strassen's Method

Let A, B, and C be a 2*2 matrices such that
C = A*B
, , and
Define the sums and products:
 sums products

Then we have:

Using 7 multiplications and 15 additions

On n*n matrices the above method take O( ) operations

### Linear Equations

Solving a system of linear equations like:

can be reduced to solving Ax = b for x where:

Gauss-Jordan Method
```for j = 1 to n do
for h = 1 to n do
for k = j to n + 1 do
if (h != j) then
end for
end for
end for

for j = 1 to n do

end for
```
Gauss-Jordan Method - SIMD
```for j = 1 to n do
for h = 1 to n do in parallel
for k = j to n + 1 do in parallel
if (h != j) then
end for
end for
end for

for j = 1 to n do in parallel

end for
```

T(n) = O(n)
P(n) = O( )
Another Approach

If Ax = b then x =

So just compute

Special Case

Let

Define

We have:
1)
2)
3) if A = BC then
4)

So how long does it take to compute ?

Note that A(BC) = (AB)C when A, B, C are n*n matrices

So we can use generalized scan operator with matrix multiplication

Sequential matrix multiplication takes

So with n processors we can compute in log(n) time

But using a scan within a scan to perform all matrix multiplications we can compute in time using processors

Theorem. L. Csanky 1976
The A be a nonsingular n*n matrix. Then can be computed in parallel using processors in time.

proof.
Uses some cool math to do some fancy stuff
Details available on demand
Method is not practical due to number of processors required and numerical problems

It is an open problem if time is best possible for computing inverse of an n*n matrix

It is an open problem if time is best possible for triangular matrices
x = General Case

Let where are n/2*n/2 matrices

We get:

So

Compute and recursively

This requires O( ) sequential time, where 2 < x < 2.5