1. Run-Time FPGA Partial Reconfiguration for Image Processing Applications
Shaon Yousuf
Ph.D. Student
NSF CHREC Center, University of Florida
Dr. Ann Gordon-Ross
Assistant Professor of ECE

2. Introduction
Run-time reconfiguration is an important feature in SRAM-based FPGAs that allows changes in functionality dynamically
Enables benefits such as flexibility, hardware reuse and reduced power consumption
Drawbacks of run-time reconfiguration
Entire fabric is reconfigured even for slight design changes
System execution stalls completely
Time to load a design onto the fabric from external memory (reconfiguration time) increases with bitstream size
Run-time reconfiguration is enhanced by run-time partial reconfiguration (PR) which mitigate these drawbacks
Design B
Design A
Design C
Flexibility
Designs loaded when required
Hardware Reuse
Current required design replaces old one on the same fabric
Design A
Power Savings
Design A, B, & C stored in external memory
Est time for slide: 0:45 min
Total = 0:45 min
Configuration controller
Design B
Design C
Design C Required
Design B Required
Design A Required
External memory
FPGA Fabric

3. Partial Reconfiguration (PR)
PR allows the ability to reconfigure a portion of an FPGA dynamically by dividing the FPGA into two types of regions
Static region - contains static portion of the design (Static Modules)
Partially reconfigurable region (PRR) - loaded with a partial reconfiguration module (PRM)
PR benefits in addition to full reconfiguration benefits
Only reconfigured PRR is stalled while static region and other PPRs continue operating
Smaller bitstreams sizes
Reduced power consumption
Reduced memory requirements
Reduced time to reconfigure
Central Controlling Agent
ICAP
Mem controller
Module A
Module B
Module C
Module D
Static modules
Reconfigurable Modules (PRMs)
Module: A & B
PRR 1
Est time for slide: 0:45 min
Total = 1:30 min
Static modules
Static region
Modules: C & D
PRR 2
Example with 2 PRRs
FPGA Fabric

4. PR Challenges and Motivations
PR is hard
Complicated design flow
Requires manual intervention and knowledge about target device
Can decrease system performance as compared to full reconfiguration if system design is not carefully considered
Despite these challenges, PR can potentially prove to be beneficial for certain application types
Resource savings, flexibility, power savings, etc.
Thus, it becomes necessary to explore PR for different application types
Provides valuable insights
Potentially ease PR for designers of similar application types
Manual Steps
Xilinx PR Implementation Flow
HDL Design Description
HDL Synthesis
Set Design Constraints
Timing/ Placement Analysis
Implement Static Design and PR Modules
Est time for slide: 2:00 min
Total = 3:30 min
Merge
Final Generated Bitstreams

5. Contribution PR architecture benefits for a JPEG encoder system
JPEG encoder/decoder systems are a key enabling technology for low-power and high-performance image transmission for on-line satellite communication
The JPEG encoder PR architecture provides increased flexibility as compared to a non-PR architecture
Leveraging the JPEG encoder PR architecture we propose a PR architecture for a JPEG encoder/decoder (codec) system
The proposed PR codec architecture will provide significant benefits in terms of resource savings and power savings as well as flexibility
Study of the PR architecture of the JPEG systems can be adapted to realize potential benefits for similar applications types
Est time for slide: 0:30 min
Total = 4:00 min

6. JPEG Encoder/Decoder Process
JPEG encoding process for color images is divided into four main steps
Color Space Transformation - RGB to YCbCr
Forward Discrete Cosine Transform (FDCT)
Quantization
Entropy Encoding
8x8 data block of each color component
Est time for slide: 2:00 min
Total = 6:00 min
Table Specifications
Table Specifications
Compressed Image Data
RGB to YCbCr
FDCT
Quantizer
Entropy Encoder 6

7. JPEG Encoder/Decoder Process
JPEG encoding process for color images is divided into four main steps
Color Space Transformation - RGB to YCbCr
Forward Discrete Cosine Transform (FDCT)
Quantization
Entropy Encoding
JPEG decoder performs these steps in reverse
Reconstructed
8x8 data block of each color component
Est time for slide: 2:00 min
Total = 6:00 min
Table Specifications
Table Specifications
Compressed Image Data
YCbCr to RGB
IDCT
Dequantizer
Entropy Decoder 7

8. JPEG Encoder Architecture
HOST DATA
HOST PROG
RAM
8x8 Data blocks
Control Signals
Host Interface
MUX
Data
Control Signals
Control Signals
Header Generator
H: 0xFF
Buffer
Pipeline Controller
Est time for slide: 2:00 min
Total = 8:00 min
RGB2YCbCr & FDCT2D
ZigZag
Quantizer
Run Length Encoder
Huffman Encoder and Byte Stuffer
Skipping ahead to when processing is almost done…… 8 8

9. JPEG Encoder Architecture
HOST DATA
HOST PROG
RAM
8x8 Data blocks
Control Signals
Data
Control Signals
Host Interface
MUX
Control Signals
Buffer
Header Generator
EOI : 0xFF
Pipeline Controller
Est time for slide: 2:00 min
Total = 8:00 min
RGB2YCbCr & FDCT2D
ZigZag
Quantizer
Run Length Encoder
Huffman Encoder and Byte Stuffer
Encoding Complete! 9 9

10. JPEG Encoder PR Architecture
Pipeline controller module
Ability to replace with updated module
Ability to replace with different controller module
HOST DATA
HOST PROG
RAM
Control Signals
Data
Control Signals
Host IF
Est time for slide: 1:30 min
Total = 9:30 min
MUX
Byte Stuffer
Control Signals
Buffer
Pipeline Controller
Header Generator
RGB2YCbCr & FDCT2D
ZigZag
Quantizer
Run Length Encoder
Huffman Encoder

11. JPEG Encoder PR Architecture
RGB2YCbCr and FDCT module
Ability to replace with an updated module
Ability to skip color space transformation by replacing with a module that only does DCT
Ability to replace different DCT types
HOST DATA
HOST PROG
RAM
PR Region
Control Signals
PR Module
Data
Control Signals
Host IF
Est time for slide: 1:30 min
Total = 9:30 min
MUX
Byte Stuffer
Control Signals
Buffer
Pipeline Controller
Header Generator
RGB2YCbCr & FDCT2D
ZigZag
Quantizer
Run Length Encoder
Huffman Encoder 11

12. JPEG Encoder PR Architecture
Entropy encoder modules (Zigzag, Run length, Huffman, Header Generator, Byte Stuffer)
Ability to update each individual module
Ability to employ different entropy encoding schemes
Ability to replace Huffman code tables and update header accordingly
HOST DATA
HOST PROG
RAM
PR Region
Control Signals
PR Module
Data
Control Signals
Host IF
Est time for slide: 1:30 min
Total = 9:30 min
MUX
Byte Stuffer
Control Signals
Buffer
Pipeline Controller
Header Generator
RGB2YCbCr & FDCT2D
ZigZag
Quantizer
Run Length Encoder
Huffman Encoder 12

13. JPEG Encoder PR Architecture
Quantization Module
Ability to replace with an updated module
Ability to change quantization matrix tables to control image quality
HOST DATA
HOST PROG
RAM
PR Region
Control Signals
PR Module
Data
Control Signals
Host IF
Est time for slide: 1:30 min
Total = 9:30 min
MUX
Byte Stuffer
Control Signals
Buffer
Pipeline Controller
Header Generator
RGB2YCbCr & FDCT2D
ZigZag
Quantizer
Run Length Encoder
Huffman Encoder 13

14. JPEG Encoder PR Architecture
Advantages of JPEG Encoder PR Architecture
Provides flexibility by allowing the ability to replace different modules
More interesting benefits arise when the encoder architecture is combined with a decoder architecture
HOST DATA
HOST PROG
RAM
PR Region
Control Signals
PR Module
Data
Control Signals
Host IF
Est time for slide: 1:30 min
Total = 9:30 min
MUX
Byte Stuffer
Control Signals
Buffer
Pipeline Controller
Header Generator
RGB2YCbCr & FDCT2D
ZigZag
Quantizer
Run Length Encoder
Huffman Encoder 14

15. JPEG Codec PR Architecture
HOST DATA
PR Region
HOST PROG
RAM
PR Module
Control Signals
Data
Control Signals
Host IF
MUX
Control Signals
Buffer
Byte Stuffer
Encoder Pipeline Controller
Decoder Pipeline Controller
Est time for slide: 1:00 min
Total = 10:00 min
DEMUX
Header Generator
RGB2YCbCr & FDCT2D
YCbCr2RGB & IDCT2D
Reorder
ZigZag
Quantizer
Dequantizer
Run length Encoder
Run length decoder
Huffman Encoder
Huffman Decoder 15

16. JPEG Codec PR Architecture
Encoder Architecture
Encoder Data Path
HOST DATA
PR Region
HOST PROG
RAM
PR Module
Control Signals
Control Signals
Host IF
MUX
Data
Control Signals
Buffer
Byte Stuffer
Decoder Pipeline Controller
Encoder Pipeline Controller
Est time for slide: 1:00 min
Total = 10:00 min
DEMUX
Header Generator
RGB2YCbCr & FDCT2D
ZigZag
Quantizer
Run length Encoder
Huffman Encoder 16

17. JPEG Codec PR Architecture
Decoder Architecture
Decoder Data Path
HOST DATA
PR Region
HOST PROG
RAM
PR Module
Control Signals
Control Signals
Host IF
MUX
Data
Control Signals
Buffer
Byte Stripper
Byte Stuffer
Encoder Pipeline Controller
Decoder Pipeline Controller
Est time for slide: 1:00 min
Total = 10:00 min
DEMUX
Header Generator
Header Decoder
YCbCr2RGB & IDCT2D
Reorder
Dequantizer
Run Length Decoder
Huffman Decoder 17

18. JPEG Codec PR Architecture Contd.
Benefits
Resource savings
Same hardware resources shared between encoder and decoder
Power savings
PR module bitstreams stored in memory and loaded on demand (decoding or encoding) as opposed to both occupying actual hardware resources
Increased flexibility
Encoder and decoder PR modules can be updated as needed or replaced with one of another type as per application requirements
Architecture limitations
For a PR module loaded into a particular region
The loaded PR module’s size and resource requirements (slices, FIFOs, BRAMs, DSPs) cannot exceed the maximum available in the PR region
PR module port connections, both incoming and outgoing, cannot exceed the PR regions maximum incoming and outgoing port connections, respectively
Est time for slide: 1:00 min
Total = 10:00 min 18

19. Experimental Setup Software Hardware
Xilinx ISE with PR patch 12 installed
Synthesize options
Optimization Goal - Speed
Optimization Effort - Normal
Hardware
Xilinx Virtex-4 LX60
Est time for slide: 0:20 min
Total = 10:50 min

20. Results – Architecture Specifications
Input image specifications
Color images only (3 components, RGB input)
Supported resolution 800x600
JPEG Encoder system
JPEG baseline encoding JPEG ITU-T T.81 | ISO/IEC
Standard JFIF header v 1.01 automatic generation
Design operates above 100 MHz
Hardcoded Huffman tables and two programmable quantization tables, one for luminance and one for chrominance at 50% quality settings
50% quality reduced
100Mhz
Est time for slide: 0:40 min
Total = 11:30 min T H
JPEG Encoder System 20 20

21. Results: JPEG Encoder PR Architecture Floorplan
Quantizer Module
Pipeline controller Module
FDCT Module
Huffman Encoder Module
Est time for slide: 1:00 min
Total = 12:30 min
Zigzag Module
Byte Stuffer Module
Header Generation Module
Run length Encoder Module 21

22. Results: Resource Requirements
JPEG Encoder Architecture
Total Slices = 5,531
Total DSP48s = 9
Total FIFO/RAMB16s = 27
JPEG Encoder PR Architecture
Total Slices = 5,678
Est time for slide: 1:00 min
Total = 13:30 min
Slice Requirements 22 22

23. Results: Resource Requirements
JPEG Encoder Architecture
Total Slices = 5,531
Total DSP48s = 9
Total FIFO/RAMB16s = 27
JPEG Encoder PR Architecture
Total Slices = 5,678
Est time for slide: 1:00 min
Total = 13:30 min
DSP48 Requirements 23 23

24. Results: Resource Requirements
JPEG Encoder Architecture
Total Slices = 5,531
Total DSP48s = 9
Total FIFO/RAMB16s = 27
JPEG Encoder PR Architecture
Total Slices = 5,678
Est time for slide: 1:00 min
Total = 13:30 min
FIFO/RAMB16s Requirements 24 24

25. Results: PR vs Non PR Architectures
Encoder architecture slice requirement = 5531
Predicted decoder architecture slice requirement ~ 5600
Est time for slide: 1:00 min
Total = 14:30 min
Put 11,200 in graph 25

26. Results: PR vs Non-PR Architectures
Predicted total slice requirement for non-PR codec architecture ~ 11200
Predicted PR codec architecture slice requirement ~ 6100
11200 Slices
Slice Macro Overhead :
7% or 416 Slices
Est time for slide: 1:00 min
Total = 14:30 min
Put 11,200 in graph 26

27. Results: PR vs Non-PR Architectures
Predicted total slice requirement for non-PR codec architecture ~ 11200
Predicted PR codec architecture slice requirement ~ 6100
Predicted resource savings from PR codec architecture = (( ) * 100)/11200 = 45%!!
45% Savings!!
Est time for slide: 1:00 min
Total = 14:30 min
Put 11,200 in graph 27

28. Conclusions and Future Work
We created a JPEG encoder PR architecture for image encoding
The architecture provides increased flexibility and potential power savings with a slice macro overhead of only 4%
The architecture forms a base for a JPEG codec PR architecture
The JPEG codec architecture is predicted to benefit from increased flexibility and power savings as well as area savings as much as 45% relative to a non-PR codec architecture
Future Work
Complete proposed JPEG codec PR architecture
Extend work to development of PR architectures for MPEG/H.264 encoder and decoder systems
Est time for slide: 0:30 min
Total = 15:00 min

29. QUESTIONS?
This work was supported in part by the I/UCRC Program of the National Science Foundation under Grant No. EEC We also gratefully acknowledge tools provided by Xilinx.