1. Sparse Vector & Matrix Algebra Lecture #4 EEE 574 Dr. Dan Tylavsky

2. Sparse Vector & Matrix Algebra © Copyright 1999 Daniel Tylavsky 4 Sparse Vector Algebra –Integer Sets - Sets of integers may be thought of as a sparse storage technique for a vector of integers. –Ex: List 1: {2,4,5,6} –Ex: List 1: {2,4,5,6} (Mutually exclusive lists.) – List 2: {3,7,9} – List 3: {1,10}

3. Sparse Vector & Matrix Algebra © Copyright 1999 Daniel Tylavsky –Merging Sparse Integer Sets Single Switch Technique/Single Use –Ex: Consider Merging »List 1: {2,4,5,6} List 2: {1,2,7} List 3: {1,5} N=Greatest Integer in any list M=Total # Elem. in all lists. = Unique Elem. in all lists. Write Operations –N Clear (Switch Vector) –M Write “1” into Switch –M Write into List Compare Operations –M Comparisons of Switch

4. Sparse Vector & Matrix Algebra © Copyright 1999 Daniel Tylavsky –Merging Sparse Integer Sets Single Switch Technique/Single Use –Resultant list is unordered. List: {2,4,5,6,1,7} To order list, scan switch array in order (ascending/descending). Requires N Comparisons Sort Scheme Requires M(M-1)/2

5. Sparse Vector & Matrix Algebra © Copyright 1999 Daniel Tylavsky To merge two new lists, no need to clear array. Simply use a different switch value. (Multiple Use Technique.) EX: Merge: List A {1,3,5}, List B {2,3,6} To clear array for next use, only clear location in NewList.

6. Sparse Vector & Matrix Algebra © Copyright 1999 Daniel Tylavsky –Adding Sparse Vectors Expanded Accumulator –Ex: Add the following sparse vectors stored in unordered compact form: Step 1: Merge Row Indices –{1,3,5,9,4}

7. Sparse Vector & Matrix Algebra © Copyright 1999 Daniel Tylavsky Step 2: Zero Merged List Positions in real vector of length N. (Expanded Accumulator) {1,3,5,9,4} Step 3: Write first vector into accumulator. Step 4: Add subsequent lists to accumulator.

8. Sparse Vector & Matrix Algebra © Copyright 1999 Daniel Tylavsky Step 5: Use merged integer list to retrieve only relevant data.

9. Sparse Vector & Matrix Algebra © Copyright 1999 Daniel Tylavsky –Adding Sparse Vectors Expanded Pointer Array –Ex: Add the following sparse vectors stored in unordered compact form: Step 1: Merge Row Index: {1,3,5,9,4} Step 2: Store in expanded pointer array the position that each element of A will occupy.

10. Sparse Vector & Matrix Algebra © Copyright 1999 Daniel Tylavsky Step 3: Clear first M position of A array. Step 4: Use R1Indx and Switch to write A1 into A. Step 5: Clear Switch array using merged list if necessary, {1,3,5,9,4}. Step 5: Use R2Indx and Switch to add A2 to A.

11. Sparse Vector & Matrix Algebra © Copyright 1999 Daniel Tylavsky –Inner Product Calculation of Unordered Sparse Vectors Inner product of col. Vectors: a,b: –Ex: Perform an inner product of the following sparse vectors stored in unordered compact form:

12. Sparse Vector & Matrix Algebra © Copyright 1999 Daniel Tylavsky Step 1: Use an expanded pointer array to identify all non- zeros in A1. Step 2: Using pointer array, compare A2 with A1 and accumulate all non-zero products. Position pointer array value at location k gives location of index k in array A1.

13. Sparse Vector & Matrix Algebra © Copyright 1999 Daniel Tylavsky –Sparse Matrix Addition Assumption: Matrices stored in RR(C)U. Procedure: –Use multiple switch technique to symbolically add row by row. Symbolic result is data structure for numerical result. –Using created data structure perform numerical result. –(For something this simple, symbolic and numerical routines would be interwoven; however they will be kept separate in this example to emphasis the difference between symbolic and numerical operations.)

14. Sparse Vector & Matrix Algebra © Copyright 1999 Daniel Tylavsky Initialize: Switch=0, End=0, IR=0 IR=IR+1 Merge RIndxA and RIndxB using multiple use switch technique with IR as switch value. Save merged indices as RIndxC. Increment End on each entry into RIndxC. Mark end of row of C data structure: ERP(IR)=End IR=N? Perform numerical segment. No Yes

15. Sparse Vector & Matrix Algebra © Copyright 1999 Daniel Tylavsky Symbolically add the following matrices stored in RR(C)U.

16. Sparse Vector & Matrix Algebra © Copyright 1999 Daniel Tylavsky Initialize: Switch=0, IR=0 Clear C(k) from 1 to N IR=IR+1 Use Switch array to point to where each col. is stored in C(k). Add A and B to appropriate locations in C from ERPA/B(IR-1)+1 to ERPA(IR). IR=N? End N Y

17. Sparse Vector & Matrix Algebra © Copyright 1999 Daniel Tylavsky You can show that the arrays at completion have the form:

18. Sparse Vector & Matrix Algebra © Copyright 1999 Daniel Tylavsky –Multiplying a Sparse Matrix by a Dense Vector. A  b=x –A is sparse. –b is dense. –x is dense. No need to perform symbolic manipulation. Assume: –A and b are store in RR(C)U, ordered compact form.

19. Sparse Vector & Matrix Algebra © Copyright 1999 Daniel Tylavsky Initialize: IR=0 Clear x(k) from 1 to N IR=IR+1 Set pointer to begin/end or row: IBEG=ERPA(IR-1)+1, IEND=ERPA(IR). Accumulate Inner Product and store in dense x using Inner Product Alg.* IR=N? End N Y * You may wish to modify the inner product algorithm to take advantage of b being dense. There is no need to use a switch array since A indices can be used as pointers to locations in b.

20. Sparse Vector & Matrix Algebra © Copyright 1999 Daniel Tylavsky –Multiplying a Sparse Matrix by a Sparse Vector. A  b=x –A is sparse. –b is sparse. –x is sparse? If x is stored as a sparse vector: –Perform symbolic manipulation. If x is stored as a dense vector: –No need to perform symbolic manipulation. Assume: –A and b are store in RR(C)U, CR(C)U

21. Sparse Vector & Matrix Algebra © Copyright 1999 Daniel Tylavsky Initialize: Switch=0, IR=0 Clear x(k) IR=IR+1 Set pointer to begin/end or row: IBEG=ERPA(IR-1)+1, IEND=ERPA(IR). Accumulate Inner Product and store in sparse x using Inner Product Alg. IR=N? End N Y If x is sparse, perform symbolic program segment to get CRO/U. If x is dense, assume x is stored in ordered compact form.

22. The End