1. LZRW3 Data Compression Core Dual semester project April 2013 Project part A final presentation Shahar Zuta Netanel Yamin Advisor: Moshe porian

2. Contents Project Goals Top Architecture Micro Architecture Problems & Solutions Work method Test Plan Live simulation Synthesis Results Part B Schedule

3. Project Golas Implementation of LZRW3 data compression algorithm High performance- data transfer of 1Gbps Adapted to data templates of 2Kbyte to 32Kbyte Internal memory on FPGA only ( Virtex-5 ), no interface to external memory Implementing strong debugging capabilities via GUI

4. algorithm overview INPUT FILE -------------- -------------- -------------- ------------- INPUT FILE -------------- -------------- -------------- ------------- A copy item consists of two bytes that represent from 3 to 18 bytes. literal item consist of one byte which represents himself LZRW3 COMPRESSOR OUTPUT FILE [----][-----]- [-------][---- -------][----] OUTPUT FILE [----][-----]- [-------][---- -------][----] GROUPS OF ITEMS (literal/Copy) LITERALS ONLY

5. Top Architecture Rx PATH Tx PATH INPUT BLOCK memory LZRW3 COMPRESSOR CORE LZRW3 COMPRESSOR CORE COMPRESSED FILE memory GUI XILINX VIRTEX 5 ON XUVP505 BOARD UART

6. Top Architecture Rx PATH Tx PATH INPUT BLOCK memory LZRW3 COMPRESSOR CORE LZRW3 COMPRESSOR CORE COMPRESSED FILE memory GUI XILINX VIRTEX 5 ON XUVP505 BOARD UART

8. Problems & solutions

9. Problem # 1 Hash table (stage III) should not update if we build a copy item. The decision if a copy item will be built must consider results of the comparison(stage IV) In the original design we have four clock cycles untill stage IV will have the comparison result but, we have to deal with some cases like lzrw3 ABC(index,length)ABC

10. Original design Hash Table STAGE III Address Alignment Tentative Next Address RAM BANKS COMPARATOR UPDATE ENABLE STAGE IV clk MEM REG. 4 1 2 3 2 DATA O 1

11. Heart of the problem Dealing with cases like ABCABC mean that we need to update the hash table entry with ABC offset very fast. ABC BCA CAB ABC Two clocks only :Stage III samples 2 nd ABC need to read 1 st ABC offset! 1 st ABC at Stage III 2 nd ABC at Stage III

12. Solution Hash Table STAGE III Address Alignment Tentative Next Address RAM BANKS COMPARATOR UPDATE ENABLE STAGE IV clk MEM REG. ABC offset 1

13. Problem # 2 The GOLDEN MODEL does not start building a new copy item if it’s in the last 18 bytes of the file, if it’s in the middle of building a copy item it will finish it.

14. Solution # 1 Maintain a FIFO buffer that will hold the two last groups, then when end of file is reached we will send the buffered bytes and not compressed bytes. This solution would demand hardware addition and would demand thorough synchronization which could make the system unstable.

15. Solution # 2 Taking advantage of the pipeline design: Until the data reaches stage IV (the comparator) it goes through 9 buffers. Buffer another 9 will complete to 18 clocks buffer before a received data will be compared. FORWARD the EOF core input signal to stage IV and make it cause automatic comparison failure in the last 18 bytes which will result that no copy items will be created.

16. OUTPUT GROUP Hash Func Hash Func Hash table Hash table INPUT FILE memory COMPARATOR 45 HASH REGTABLE REG GROUP REG 3 Bytes buffer INPUT REG 123458976 INPUT BYTE End Of File When EOF FARWARD UP, NEW COPY ITEMS will not be created Implementation 9 NEW BUFFERS 9 clocks buffer in the original design

17. Work method Generating DIRECT/RANDOM input file. Compress the input file using the GOLDEN MODEL. Compress the same input file using our CORE. Perform a comparison between the two files using DIFF software.

18. Work method (cont.) ExamDiff DIRECT input RANDOM input C CODE C CODE GM CORE simulation charsASCII value C C

19. ExamDiff

20. Test Plan

21. Basic set: include large amount of tests Second set: Basic tests + Random client-ready toggles Third set: Second set + Random inputs valid toggles

22. Basic set examples Random input (Length, Num Of Vars) CASEOUTPUT LENGTH VAR.INPUT Reasonable compression23K32K10RANDOM Each variable repeated 18 times740732K10RANDOM Very high compression ratio ~ 90 % compression 388632K1RANDOM Output = input32K 256RANDOM

23. Basic set examples Direct input PERFORM A COPY ITEM COPY ITEM SHOULD NOT PERFORM

24. Basic set examples Files concatenation DATA PROVIDER DATA CLIENT BUSYFINISH C L A E R READY LZRW3 CORE

25. Live simulation

26. Synthesis Results

27. ADD PLAN AHEAD/SYNPLIFY REPORT MAX clock rate Blocks usage

28. Part B

29. Assemble lzrw3 core with peripheral units Logic Test End To End Burning to FPGA Hardware test & optimization Verification Environment - GUI (Visual Studio) All system Test & Debug

30. Schedule

31. DateGoals 11/04/2013-9/05/2013Assemble units together 12/05/2013-09/06/2013Test logic End To End 16/06/2013- 10/06/2013Synthesis 24/06/2013- 17/06/2013Implementation 08/07/2013- 25/06/2013Verification Environment – GUI 23/07/2013- 09/07/2013Lab Tests 08/08/2013Part B Final presentation