LZRW3 Data Compression Core Dual semester project April 2013 Project part A final presentation Shahar Zuta Netanel Yamin Advisor: Moshe porian

Contents Project Goals Top Architecture Micro Architecture Problems & Solutions Work method Test Plan Live simulation Synthesis Results Part B Schedule

Project Golas Implementation of LZRW3 data compression algorithm High performance- data transfer of 1Gbps Adapted to data templates of 2Kbyte to 32Kbyte Internal memory on FPGA only ( Virtex-5 ), no interface to external memory Implementing strong debugging capabilities via GUI

algorithm overview INPUT FILE INPUT FILE A copy item consists of two bytes that represent from 3 to 18 bytes. literal item consist of one byte which represents himself LZRW3 COMPRESSOR OUTPUT FILE [----][-----]- [ ][ ][----] OUTPUT FILE [----][-----]- [ ][ ][----] GROUPS OF ITEMS (literal/Copy) LITERALS ONLY

Top Architecture Rx PATH Tx PATH INPUT BLOCK memory LZRW3 COMPRESSOR CORE LZRW3 COMPRESSOR CORE COMPRESSED FILE memory GUI XILINX VIRTEX 5 ON XUVP505 BOARD UART

Top Architecture Rx PATH Tx PATH INPUT BLOCK memory LZRW3 COMPRESSOR CORE LZRW3 COMPRESSOR CORE COMPRESSED FILE memory GUI XILINX VIRTEX 5 ON XUVP505 BOARD UART

Problems & solutions

Problem # 1 Hash table (stage III) should not update if we build a copy item. The decision if a copy item will be built must consider results of the comparison(stage IV) In the original design we have four clock cycles untill stage IV will have the comparison result but, we have to deal with some cases like lzrw3 ABC(index,length)ABC

Original design Hash Table STAGE III Address Alignment Tentative Next Address RAM BANKS COMPARATOR UPDATE ENABLE STAGE IV clk MEM REG DATA O 1

Heart of the problem Dealing with cases like ABCABC mean that we need to update the hash table entry with ABC offset very fast. ABC BCA CAB ABC Two clocks only :Stage III samples 2 nd ABC need to read 1 st ABC offset! 1 st ABC at Stage III 2 nd ABC at Stage III

Solution Hash Table STAGE III Address Alignment Tentative Next Address RAM BANKS COMPARATOR UPDATE ENABLE STAGE IV clk MEM REG. ABC offset 1

Problem # 2 The GOLDEN MODEL does not start building a new copy item if it’s in the last 18 bytes of the file, if it’s in the middle of building a copy item it will finish it.

Solution # 1 Maintain a FIFO buffer that will hold the two last groups, then when end of file is reached we will send the buffered bytes and not compressed bytes. This solution would demand hardware addition and would demand thorough synchronization which could make the system unstable.

Solution # 2 Taking advantage of the pipeline design: Until the data reaches stage IV (the comparator) it goes through 9 buffers. Buffer another 9 will complete to 18 clocks buffer before a received data will be compared. FORWARD the EOF core input signal to stage IV and make it cause automatic comparison failure in the last 18 bytes which will result that no copy items will be created.

OUTPUT GROUP Hash Func Hash Func Hash table Hash table INPUT FILE memory COMPARATOR 45 HASH REGTABLE REG GROUP REG 3 Bytes buffer INPUT REG INPUT BYTE End Of File When EOF FARWARD UP, NEW COPY ITEMS will not be created Implementation 9 NEW BUFFERS 9 clocks buffer in the original design

Work method Generating DIRECT/RANDOM input file. Compress the input file using the GOLDEN MODEL. Compress the same input file using our CORE. Perform a comparison between the two files using DIFF software.

Work method (cont.) ExamDiff DIRECT input RANDOM input C CODE C CODE GM CORE simulation charsASCII value C C

ExamDiff

Test Plan

Basic set: include large amount of tests Second set: Basic tests + Random client-ready toggles Third set: Second set + Random inputs valid toggles

Basic set examples Random input (Length, Num Of Vars) CASEOUTPUT LENGTH VAR.INPUT Reasonable compression23K32K10RANDOM Each variable repeated 18 times740732K10RANDOM Very high compression ratio ~ 90 % compression K1RANDOM Output = input32K 256RANDOM

Basic set examples Direct input PERFORM A COPY ITEM COPY ITEM SHOULD NOT PERFORM

Basic set examples Files concatenation DATA PROVIDER DATA CLIENT BUSYFINISH C L A E R READY LZRW3 CORE

Live simulation

Synthesis Results

ADD PLAN AHEAD/SYNPLIFY REPORT MAX clock rate Blocks usage

Part B

Assemble lzrw3 core with peripheral units Logic Test End To End Burning to FPGA Hardware test & optimization Verification Environment - GUI (Visual Studio) All system Test & Debug

Schedule

DateGoals 11/04/2013-9/05/2013Assemble units together 12/05/ /06/2013Test logic End To End 16/06/ /06/2013Synthesis 24/06/ /06/2013Implementation 08/07/ /06/2013Verification Environment – GUI 23/07/ /07/2013Lab Tests 08/08/2013Part B Final presentation