1. Playing Card Recognizer ECE 4025 February 28, 2002 Group 5 Robert Barrett Jason Hodkin Chung Tse Mar Jay Silver David Winkler Yu Ming Wu

2. Motivation Short-term: –Demonstrate effectiveness and usefulness of character recognition Long-term: Design autonomous on-line card playing system

3. Project Scope Image Restrictions: –One card per image –No occlusion of card –High contrast solid background –Constant light source Image Allowances: –Scaling –Rotation –Font style

4. How does it work? System diagram Down Sample & Convert to Binary High Resolution Color Image Binary Image of Card Corner Label Connected Objects & Flood Fill Card Area Cross- correlate with Templates Compute Scores and Compare Card Value and Suit Use Major Axis to Find Corners for Scaling and Rotation Extract Corner for Comparison

5. Color to Gray Scale Conversion [R G B] Y R U R V R Y G U G V G Y B U B V B = Y U V We are only interested in intensity (Y) [R G B] YRYGYBYRYGYB = intensity Y R = 0.299 Y G =0.587 Y B =0.114

6. Gray Scale to Binary Conversion If grayScaleValue > mean + 2*  Pixel = 1; Else, Pixel = 0 25 13 1 50 28 40 247 5 225 13 18 254 190 12 5 7 210 18 255 32 11 5 36 20 13 0 0 0 0 0 0 1 0 1 0 0 1 1 0 0 0 1 0 1 0 0 0 0 0 0

7. Labeling 1 1 1 0 0 0 0 0 1 1 1 0 1 1 0 0 1 1 1 0 0 0 1 0 1 1 1 0 0 1 1 0 1 1 1 0 0 0 0 0 1 1 1 0 2 2 0 0 1 1 1 0 0 0 3 0 1 1 1 0 0 3 3 0 1 1 1 0 0 0 0 0 Identify blobs Determine characteristics of blobs Area = 24 Area = 4 Area = 5

8. Flood Fill FloodFill ( pixelToEliminate ) FloodFill ( allNeighboringPixels) Erase all neigboring pixels end

9. Calculate Angle of Major Axis of Best Fit Ellipse (orientation)

10. Rotate 25 13 1 50 28 40 247 5 225 13 18 254 190 12 5 7 210 18 255 32 11 5 36 20 13

11. Isolate the Important Information Right Side of Card Bottom of Card Top of Card Left Side of Card Scan each row and column for a white pixel

12. Locate Identifiable Symbol and isolate it top left bottom right

13. Choose an Appropriate Template and Scale to Match the Template Choose a font that should be consistent with most card fonts, but that is different from the data set. Template Image Freecell (A card game on most windows OS’s)

14. Take a Normalized Cross Correlation  [X(r+m,c+n)*H(r,c)] = Correlation(m,n) nm

15. Take a Normalized Cross Correlation

16. Highest “Score” is a Match Max Corr is 0.5581 Max Corr is 0.3016

17. Implementation Overview EVM Correlation Calculation Templates Final Result Color to Binary HOST Corner Extraction Scaling Label Flood FillRotate Decision Model RTDX Camera

18. Implementation Overview Image File Decision Model HOST Corner Extraction Scaling RTDX EVM Correlation Calculation Templates Final Result

19. Realtime Constraint Realtime constraint depends on the application –How fast the cards change –Bounded by camera frame rate Set goal of 1 frame/second

20. Computational Complexity 1 frame/second 64x64 = 4096 pixels/template 100 cross-correlations/template 13 templates for number, 4 templates for suite 100 * 4096 * 17 * 1 = 7 Million MACs/second

21. Next Steps Implementing processing on EVM –Cross-correlation Implementing processing on host –Corner extraction –Scaling –Decision model and display Synchronizing host with EVM using RTDX API