Embedded Systems Architecture IR Remote Tic-Tac-Toe Anuwat Saetow John O’Donnell Chris Smallwood Olga Kardonik May 14, 2011

Project Outline Goal: Develop a tic-tac-toe game using the iMX21 and TLL5000 platform capable of using any IR input device as a controller and output to VGA monitor Two Main Tasks IR input (application code, driver, hardware) VGA output (protocol, verilog)

Task Breakdown Anuwat Saetow, Olga Kardonik: Learn VGA Protocol Code graphics in Verilog Test code for graphics generation & output John O’Donnell, Chris Smallwood: Build & test IR Sensor Develop driver code Develop button sensing algorithm and game code in user application

Project Diagram TLL5000 monitor user application: game code, mode control, Calibration, Signal decode ir_sensor.ko: GPIO ctrl reg, Sample PE7, Output FPGA FPGA: VGA Controller VGA Mezzanine Connector Port E[7] Infra-Red Detector IR waves Remote Control

John O’Donnell Chris Smallwood IR Input John O’Donnell Chris Smallwood

Components of IR Input Hardware Application code Driver code Obtain all necessary hardware components to build IR circuit Test hardware to ensure functionality Application code Calibrates the remote Runs the game Decodes the button presses during the game Driver code Sets the GPIO ctrl registers during module initialization Samples PE[7] (bit used as input from IR circuit) Writes display data to FPGA

Hardware Components iMX21 TLL5000 platform RS 276-640 IR Detector 7805 +5V Voltage Regulator 1 x 3.3uF C 3 x 1Mohm R 1 x diode 1 x 9V battery IR remote

IR Sensor Schematic OUT GND IN 7805 5V Regulator 1 3 2 IR Detector VCC 3.3uF GND 2Mohm To GPIO input: 1Mohm RS 276-640 iMX21 PORTE[7] (mezz board J5 pin 15)

IR Sensor Hardware

Waveform of one button push Duration 37ms, minimum pulse width 400us, low preamble pulse 2.7ms

Start Sequence Zoom-in Idle state is high; low preamble pulse = 2.7ms Preamble pulse is 786 counts, so sample rate is 291kHz (3.44us) Min pulse width is 400us, so min sample count = 116

End of Button Sequence When signal stays high for over (8 * start pulse), end detection.

Read Button Routine Data start=0 Read GPIO Data start? State change? Y N Y Y N N COUNT+ COUNT=0 COUNT=0 Data start=1 PULSE[n] = COUNT n+ COUNT=0 COUNT>X? RETURN

iMX21 GPIO Port Logic PORTE[7]- IR sensor SSR: GPIO Input Register

User Application Responsible for: Initializing IR driver Calibrating buttons Determining button pushes after calibration (during game) Running game code Sending encoded display data through driver to FPGA

User Application Flow Chart Program Start Calibration Game Start X win O win Y Y Y X win? N Tie? N O win? X’s turn O’s turn N

Calibration During calibration stage, each button is pressed and its pulse data stored in a 9x100 two-dimensional array calibration_button_archive[9][100] 9 different buttons Stores widths of up to 100 transitions per button push Unused elements set to 0 Example calibration_button_archive[][] array: button\n 1 2 3 4 5 6 … #1 785 114 98 240 120 126 #2 779 99 122 236 243 #3 780 117 90 112 239

Determining Button Pushes Used Sum-of-Absolute-Differences (SAD) algorithm to determine correct button pushes during the game Example: Current Button Data Calibration_button_archive[][] Deltas SAD#1: 358 SAD#2: 36 SAD#3: 271 Button 775 120 103 115 219 239 #1 785 114 98 240 120 126 #2 779 99 122 236 243 #3 780 117 90 112 239 #1 10 6 5 125 99 113 #2 4 7 17 #3 120 14 25 107

IR Driver Responsible for: Device registration (char) Mapping to FPGA and GPIO control registers Setting GPIO control registers Done in module initialization, no need to change during game Readl() Sample Status Register (SSR) to retrieve IR input via ioctl() Writel() encoded display data to FPGA Via ioctl()

Memory Mapping (GPIO) #define GPIOE_BASE 0x10015400 #define GPIOE_MASK 0x00003F #define GPIOE_SIZE 0x40 /********** Perform Memory REMAP for GPIO ************/ if (check_mem_region(GPIOE_BASE, GPIOE_SIZE)) { printk(KERN_ERR "IRsensor: Unable to acquire GPIOE address.\n"); return -EBUSY; } request_mem_region(GPIOE_BASE, GPIOE_SIZE, IRsensor_NAME); gpio_ptr = (volatile unsigned int *)__ioremap(GPIOE_BASE, GPIOE_SIZE, 0); if (!gpio_ptr) printk(KERN_ERR "IRsensor: Unable to map GPIOE.\n"); release_mem_region(GPIOE_BASE, GPIOE_SIZE);

GPIO Ports Six total GPIO ports, labeled A-F We used port E because we found some pins on the mezzanine that mapped to it Each GPIO port has its own set of 17 control registers Only three control registers for port E needed for our project

GPIO Control Registers GPIO IN USE Register (GIUS) Determines whether pin is being utilized as GPIO or some other peripheral function Data Direction Register (DDIR) Determines whether each pin of port is input/output Pull-Up Enable Register (PUEN) Determines whether each pin of port is pulled-up or tri-stated when not driven internally or externally

Other GPIO Ctrl Regs (not used) Output Configuration Registers (OCR) When pin is set as output, determines what driver of pin is Input Configuration Registers (ICONF) Controls what data drives “A_out” & “B_out” Interrupt Configuration Registers (ICR) Used to setup GPIO interrupts

Anuwat Saetow Olga Kardonik VGA Output Anuwat Saetow Olga Kardonik

VGA specifications VGA – Video Graphics Array. VGA video signal contains 5 active signals: two digital signals for synchronization of the video - HSYNC and VSYNC; three analog signals (0-0.7 v) to control the color – R, G, B. Other colors are produced by changing the analog levels of R, G, and B.

640x480 mode 1. VSYNC signal tells the monitor to start displaying a new image and the monitor starts in the upper left corner with pixel (0,0). 2. HSYNC signal tells the monitor to refresh another row of 640 pixels. 3. After 480 rows of pixels are refreshed with 480 HSYNC signals, a VSYNC resets the monitor to the upper left corner and the process continues. The video signal must redraw the entire screen 60 times per second to provide for motion effect and to reduce flicker. This period is called refresh rate. 60 Hz refresh rate which used in 640x480 mode is corresponding to 40 ns per pixel or to 25 MHz pixel clock frequency.

Horizontal and Vertical Timing Pixel clock frequency 25.175 MHz Horizontal frequency (HSYNC) 31.46 kHz Screen refresh (VSYNC) 60 Hz Horizontal Parameter Time, us pixels Visible area 25.4 640 Front porch 0.6 16 Sync pulse width 3.8 96 Back porch 1.9 48 Whole line 31.8 800 Vertical Parameter Time, ms lines Visible area 15.253 480 Front porch 0.318 10 Sync pulse width 0.064 2 Back porch 1.049 33 Whole frame 16.683 525

Sync and Data Signals Front porch Back porch A B C VIDEO DATA Sync pulse width Data blank interval HSYNC and VSYNC signals have negative polarity in 640x480, 60 Hz resolution.

Schematic of the “Output” part FPGA HSYNC, VSYNC generator monitor VGA 3x [7:0] ADV 7125 DAC Video data generator Clock divider FPGA_CLK3 is used for the generation of clock signal for ADV7125 . Monitor is divided to 9 virtual fields to accommodate the game.

Digital to Analog Converter and 15-pin high-density D-sub VGA connector

Mapping between FPGA and the digital part of the DAC ADV7125 NET VGA_R[0] LOC=R3; #VDAC_IO24 NET VGA_R[1] LOC=T5; #VDAC_IO16 NET VGA_R[2] LOC=T4; #VDAC_IO23 NET VGA_R[3] LOC=U3; #VDAC_IO10 NET VGA_R[4] LOC=U4; #VDAC_IO9 NET VGA_R[5] LOC=V3; #VDAC_IO5 NET VGA_R[6] LOC=V4; #VDAC_IO4 NET VGA_R[7] LOC=R7; #VDAC_IO20 NET VGA_G[0] LOC=V5; #VDAC_IO3 NET VGA_G[1] LOC=U5; #VDAC_IO8 NET VGA_G[2] LOC=U6; #VDAC_IO7 NET VGA_G[3] LOC=T6; #VDAC_IO15 NET VGA_G[4] LOC=R6; #VDAC_IO21 NET VGA_G[5] LOC=R8; #VDAC_IO19 NET VGA_G[6] LOC=T8; #VDAC_IO13 NET VGA_G[7] LOC=T7; #VDAC_IO14 NET VGA_B[0] LOC=P8; #VDAC_IO27 NET VGA_B[1] LOC=W1; #VDAC_IO0 NET VGA_B[2] LOC=V2; #VDAC_IO6 NET VGA_B[3] LOC=U2; #VDAC_IO11 NET VGA_B[4] LOC=U1; #VDAC_IO12 NET VGA_B[5] LOC=T2; #VDAC_IO17 NET VGA_B[6] LOC=T1; #VDAC_IO18 NET VGA_B[7] LOC=R2; #VDAC_IO25 NET VGA_VSYNC LOC=N7; #VDAC_VSYNC NET VGA_HSYNC LOC=N3; #VDAC_HSYNC NET VGA_CLOCK LOC=R1; #VDAC_IO26 NET VGA_SYNC_N LOC=V6; #VDAC_IO1 NET VGA_BLANK_N LOC=U7; #VDAC_I02 NET VGA_PSAVE_N LOC=R5; #VDAC_IO22

Testing the Output To test the output part before it was integrated with the real input part we wrote an interactive simulator of the input and the code of Tic-Tac-Toe game. int check_win() { int i; for (i = 0; i < 3; i++) if((matrix[i*3] == matrix[1 + i*3]) && (matrix[1 + i*3] == matrix[2 + i*3]) && (matrix[1 + i*3] != 0)) return WIN; if((matrix[0 + i] == matrix[3 + i]) && (matrix[3 + i] == matrix[6 + i]) && (matrix[0 + i] != 0)) } if((matrix[0] == matrix[4]) && (matrix[4] == matrix[8]) && (matrix[0] != 0)) if((matrix[2] == matrix[4]) && (matrix[4] == matrix[6]) && (matrix[2] != 0)) return NEXT; 0 1 2 3 4 5 6 7 8

Encoding between User and FPGA user_application.c sends 22-bit data to the FPGA with the following mapping: field9 field8 field7 field6 field5 field4 field3 field2 field1 17 16 15 14 13 12 11 10 9 8 7 6 5 4 3 2 1 0 color Mode of operation 21 20 19 18 bits 1 1 1 1 Grid, Calibration and Active mode 0 0 0 1 X or Red Win display 0 0 1 0 O or Blue Win display 0 0 1 1 Invalid input display 0 0 0 0 Blank display Field Data 0 0 black 0 1 red 1 0 blue 1 1 green

Progress in the game We started from the simple boxes colored red and blue

Waveforms: HSYNC and Data Note, the frequency on the picture corresponds to the mode specifications.

Waveforms: VSYNC and Data Note, delta X on the picture corresponds to “front porch + pulse width + back porch” of the mode specifications.

Parameters in top.v for the VGA driver // ----------------------------------------------------------------------------- //VGA 640x480 @ 60Hz: V_refresh 31.46875kHz, Pix_freq 25.175MHz param h_disp = 640; // visible display param h_front = 16; // front porch param h_width = 96; // pulse width param h_back = 48; // back porch param h_sum = 800; // total pulse param v_disp = 480; // visible display param v_front = 10; // front porch param v_width = 2; // pulse width param v_back = 33; // back porch param v_sum = 525; // total pulse

Parameters in top.v for the game visualization // ----------------------------------------------------------------------------- // Grid, X, O localparam v_grid1 = 160; localparam v_grid2 = 320; localparam h_grid1 = 213; localparam h_grid2 = 427; localparam grid_width = 8; localparam x_width = 12; localparam x_trim = 60; localparam o_radius_1 = 60**2; localparam o_radius_2 = 45**2; localparam center_1y = 73; localparam center_2y = 240; localparam center_3y = 407; localparam center_1x = 100; localparam center_2x = 320; localparam center_3x = 540;

Equations for Grid, X, and O if (((vpix_count >= (v_grid1 - grid_width)) && (vpix_count <= (v_grid1 + grid_width))) || ((vpix_count >= (v_grid2 - grid_width)) && (vpix_count <= (v_grid2 + grid_width)))) begin if ((((vpix_count < ((hpix_count - (center_1x - x_width)) + center_1y)) && (vpix_count > ((hpix_count - (center_1x + x_width)) + center_1y))) || ((vpix_count < (((center_1x + x_width)) - hpix_count + center_1y)) && (vpix_count > (((center_1x - x_width)) - hpix_count + center_1y)))) && ((vpix_count < (center_1y + x_trim)) && (vpix_count > (center_1y - x_trim)) && (hpix_count < (center_1x + x_trim)) && (hpix_count > (center_1x - x_trim)))) begin if (((((center_1x - hpix_count)*(center_1x - hpix_count)) + ((center_1y - vpix_count)*(center_1y - vpix_count))) < o_radius_1) && ((((center_1x - hpix_count)*(center_1x - hpix_count)) + ((center_1y - vpix_count)*(center_1y - vpix_count))) > o_radius_2)) begin

Previously recorded demo … btw, we had really big success in the lab. People around asked to try the game!

Summary Problem Statement: Build a game that can use any IR remote control and VGA monitor to demonstrate hardware/software codesign, using simple and robust input interfaces. Most consumer devices require dedicated controller and display devices, adding to product cost. This project was not scoped to improve on the state of the art in games. Project achieved the desired goal of implementing a simple game using the lab hardware and simple user interfaces. Lessons learned: Reached limit of FPGA resources creating VGA graphics. Lost time due to starting with SVGA standard, should have used onboard VGA test pattern first. Needed to minimize code in read-button algorithm to maximize sampling resolution. References: VGA Standard: Wikipedia & various web-based resources. MC9328MX21RM User Manual “Multi Infrared Remote Control Cloner”, Anuwat Saetow, EE464H Senior Design Project, Fall 2006