NTSC to VGA Converter Marco Moreno Adrian De La Rosa Good morning everybody. I will be talking about utility-based partitioning of shared caches. This is the work I did with my advisor Yale Patt at the University of Texas at Austin. EE382M-4 Adv Emb Arch

Project Goal NTSC Video Source RCA Video Cable TLL5000 Spartan3 FPGA 15-pin VGA Cable

Project Challenges YUV -> RGB color space conversion Input to display sync timing Interlaced video to progressive scan

Project Hardware TLL5000 – base development module Analog Devices: ADV7180 - NTSC decoder Analog Devices: ADV7125 - VGA DAC Xilinx: Spartan III - FPGA TLL6219 – MX21 daughter board Digital camera NTSC source + A/V cable LCD monitor + 15-pin VGA cable

ADV7180 NTSC Decoder Analog Input Digital Outputs Composite Component Svideo Digital Outputs LLC 27MHz clock HS, VS/Field 8-bit Output Port P

ADV7180 Block Diagram

ADV7125 Video DAC Converts 8-bit values on RGB ports to analog levels for display

Color-Space Conversion YUV Data Format RGB Data Format Color-space conversion Conversion quality / speed

YUV Data Format Compatible with black & white TV infrastructure U & V color difference signals. Single Image format Video format

YUV Analyzer Sunray Image – YUV Tools http://www.sunrayimage.com/ Analysis data used for conversion quality reference

RGB Data Format Additive color model R, G and B are color contributions rather than color differences Single Image Format Video Format Three channels for color, 8-bit values VGA connector – separate lines for vertical and horizontal synchronization signals.

Color-space conversion Multiple sources for information Multiple recommendations for values used in calculation Ultimately chose a mix to give values closed to reference YUV tool analyzer R = 1.164*((int)Y - 16) + 1.596*((int)V - 128); G = 1.164*((int)Y - 16) - 0.714*((int)V - 128) - 0.344*((int)U-128); B = 1.164*((int)Y - 16) + 2.018*((int)U - 128);

Conversion C++ code Allocate memory Open file and write to mem //memory allocation int *yuv = (volatile unsigned int *)malloc(YUVBLOCK); //Open the file for reading in binary format int fd = open("flower_droplet.uyvy", O_RDWR); //write file to yuv memory block read(fd, yuv, YUVBLOCK); close(fd); //open file for writing fd = open ("converted.bmp", O_CREAT|O_RDWR); printf("fd %d\n", fd); // write header write(fd,&header[0],4); . write(fd,&header[13],2); //Calculate values and store to array R = 1.164*((int)Y - 16) + 1.596*((int)V - 128); G = 1.164*((int)Y - 16) - 0.714*((int)V - 128) - 0.344*((int)U-128); B = 1.164*((int)Y - 16) + 2.018*((int)U - 128); //Write entire array to memory at once write(fd,&BLOCK[0],YUVBLOCK); printf("\n\n"); Allocate memory Open file and write to mem Create new file Write header for new file Calculate all pixel RGB data Write out to new file Rev 1.0 :~4hr Rev 2.0 : ~4sec

Conversion Verilog code Implemented with shifts and adds only Asynchronous logic block Bitwise operations // CALCULATE Y COMPONENT OF R, G AND B assign ycomp = ((y & 8'hf0) == 8'h00) ? (8'h00 - y_16 - {3'b0, y_16[7:3]}) : ((y_16) + {3'b0, y_16[7:3]} + {5'b0, y_16[7:5]} + {7'b0, y_16[7]}); // CALCULATE V COMPONENT OF R assign r_vcomp = ((v & 8'h80) == 8'h00) ? (9'd0 - v_128 - {3'b0, v_128[7:3]} - {4'b0, v_128[7:4]}) : (v_128 + {3'b0, v_128[7:3]} + {4'b0, v_128[7:4]}); // CALCULATE V COMPONENT OF G assign g_vcomp = ((v & 8'h80) == 8'h00) ? (9'b0 - {1'b0, v_128[7:1]} - {3'b0, v_128[7:3]} - {4'b0, v_128[7:4]} - {5'b0, v_128[7:5]}) : ({1'b0, v_128[7:1]} + {3'b0, v_128[7:3]} + {4'b0, v_128[7:4]} + {5'b0, v_128[7:5]}); // CALCULATE U COMPONENT OF G assign g_ucomp = ((u & 8'h80) == 8'h00) ? (9'b0 - {2'b0, u_128[7:2]} - {4'b0, u_128[7:4]} - {5'b0, u_128[7:5]} - {6'b0, u_128[7:6]}) : ({2'b0, u_128[7:2]} + {4'b0, u_128[7:4]} + {5'b0, u_128[7:5]} + {6'b0, u_128[7:6]}); // CALCULATE U COMPONENT OF B assign b_ucomp = ((u & 8'h80) == 8'h00) ? (9'b0 - {u_128, 1'b0} - {6'b0, u_128[7:6]}) : ({u_128, 1'b0} + {6'b0, u_128[7:6]}); // ADD COMPONENTS TO CALCULATE R, G AND B assign r_wire = ycomp + r_vcomp; assign g_wire = ycomp - g_vcomp - g_ucomp; assign b_wire = ycomp + b_ucomp; // CHECK FOR R, G OR B LESS THAN ZERO assign r_zero = (r_wire[8] && 1) ? (9'h00) : (r_wire); assign g_zero = (g_wire[8] && 1) ? (9'h00) : (g_wire); assign b_zero = (b_wire[8] && 1) ? (9'h00) : (b_wire); // CHECK FOR R, G, OR B GREATER THAN 255 AND SET FINAL VALUE assign r = (r_zero[8:7] > 255) ? (8'hff) : (r_zero[7:0]); assign g = (g_zero[8:7] > 255) ? (8'hff) : (g_zero[7:0]); assign b = (b_zero[8:7] > 255) ? (8'hff) : (b_zero[7:0]); Y-component = (y - 16) * 1.0010101; Y-component = (y - 16) + (Y - 16 ) >> 3 + (y - 16) >> 5 + (Y - 16) >> 7; V-component of R = (V - 128) * 1.0011; V-component of R = (V - 128) + (V - 128) >> 3 + (V - 128) >> 4; V-component of G = (V - 128) * .10111 V-component of G = (V - 128) * (V - 128) >> 1 + (V - 128) >> 3 + (V - 128) >> 4 + (V - 128) >> 5; U-component of G = (U - 128) * .010111; U-component of G = (U - 128) >> 2 + (U - 128) >> 4 + (U - 128) >> 5 + (U - 128) >> 6; U-component of B = (U - 128) * 10.000001; U-component of B = (U - 128) << 1 + (U - 128) >> 6;

Conversion quality / speed Pixels Y V U   0x4b 0x7c 0x67 75 124 103 R G B Ref. 62 82 18 C++ 80 Verilog 63 76 17 Unoptimized C++ code Rev 1.0 : ~ 4hr Rev 2.0 : ~4sec FPGA operating @ ~27Mhz pixel clock Frame rate 60 Hz One frame - ~16 msec

Video Synchronization NTSC to VGA timing generation Interlaced video to progressive scan Solution architecture

VGA Sync Signals

VGA Sync Timing (Typical) Horizontal Regions Vertical Regions VGA mode Resolution (HxV) Pixel clock(Mhz) VGA(60Hz) 640x480 25 (640/c) VGA(85Hz) 36 (640/c) SVGA(60Hz) 800x600 40 (800/c) SVGA(75Hz) 49 (800/c) SVGA(85Hz) 56 (800/c) XGA(60Hz) 1024x768 65 (1024/c) XGA(70Hz) 75 (1024/c) XGA(85Hz) 95 (1024/c) 1280x1024(60Hz) 1280x1024 108 (1280/c) a(ms) b(us) c(us) d(us) 3.8 1.9 25.4 0.6 1.6 2.2 17.8 3.2 20 1 16.2 0.3 1.1 2.7 14.2 2.1 2.5 15.8 0.4 1.8 13.7 1.0 10.8 0.5 2.3 11.9 a(lines) b(lines) c(lines) d(lines) 2 33 480 10 3 25 1 4 23 600 21 27 6 29 768 36 38 1024

VGA Progressive Video Line 1 Line 2 Line 3 Line 4 … Line 522 Line 523

NTSC Interlaced Video

Field 1 / Field 2 scanning Field 1 start (even lines) (odd lines) VS/Field (from NTSC decoder) HS (from NTSC decoder) Generated VGA HSYNC Generated VGA VSYNC

De-Interlacing

Video Stream Architecture RCA Input LLC - 27MHz pixel clock ADV7180 NTSC Decoder VGA VSYNC Generator VGA VSYNC VGA HSYNC 4:2:2 De-Interlace / Pixel Reformat 4:4:4 Color Space Converter ADV7125 Video DAC VGA Analog Color Channels

De-Interlace Line Buffers WR_BUF_SEL 2 x RAMB16_S18 WE WR_BUF_SEL ? WR_PTR : RD_PTR ADDR[9:0] DIN[15:0] YUV 4:4:4 16-bits per clk YUV 4:2:2 8-bits per clk DOUT[15:0] 1 DOUT[15:0] DIN[15:0] RD_BUF_SEL ? WR_PTR : RD_PTR ADDR[9:0] WR_BUF_SEL WE RD_BUF_SEL

ADV7180 I2C Interface Serial protocol for interconnecting IC’s I2C block registers memory mapped to CPU Linux mmap application command line interface Controls various timing and format options iMX.21 Wishbone 0xCC000000- 0xCC000100 SCLK FPGA I2C Controller ADV7180 NTSC Decoder SDA Registers Registers Input Select H/V SYNC shape Video Format Interrupt Control

Lessons Learned Keep Verilog code simple Edge triggers are extremely useful Tie one-off possibilities to switches Not all TLL5000 boards have same clocks

Potential future work Complete optimization of C++ code Implement video stream with ARM controller rather than full-fpga implementation FPGA vs. ARM performance analysis FPGA vs. ARM power analysis Fix horizontal positioning error Smooth de-interlace w/ interpolation DMA image data to iMX.21 SDRAM

Backup

Hsync generation

Vsync generation

Field 1 / 2 & data transmission

Direct Y-data NTSC to VGA

RGB channel separation

Buffered, CSC, Partially Aligned