1. Lab 2: Capturing and Displaying Digital Image
ECE 354
Lab 2: Capturing and Displaying Digital Image

2. Big Picture Introduction
Understand the existing SOPC builder setup with new modification
Capturing image using camera
Storing the captured image in the frame buffer
Display the image on a CRT monitor with the use of a VGA controller
Perform simple image processing
Primarily written in C code

3. Additional Hardware TRDB_D5M Digital Camera Serial Connector and Cable
CRT Monitor

4. Skills to learn Integrate code with preexisting code
Understanding previously written code
Connect to a device that you did not design

5. Steps taken to complete project
Begin by looking over camera documentation
Understand SOPC builder setup given
Write a C program to convert the color image from camera to black and white
C program to transfer data from FIFO to flash memory
Gain knowledge of how the DE2_NIOS_HOST_MOUSE_VGA project displays an image to the CRT monitor
Use that knowledge to display your image
Finally have fun performing image processing

6. Lab2 Setup
[1]
[1] TRDB_D5M_Userguide

7. Overview of DE2_NIOS_HOST_MOUSE_VGA project
Implements a monochrome display, with a preloaded image, where the user can draw on it with a mouse
USB mouse should be connected USB HOST port
CRT monitor should be connected to the VGA port
You will integrate your code so that the project displays the image received from digital camera

8. SOPC builder setup: Based on DE2_NIOS_HOST_MOUSE_VGA
The program code is stored in SRAM memory
Parallel Input/output interface(PIO) for LED, switches
Camera Avalon Interface(Camera_IF) helps in transferring image data
The dedicated SDRAM is used as a frame buffer that holds a single frame of video at the time.
The SDRAM is controlled and accessed by using the four-port controller hardware
Controller can be used to read or write to the SDRAM through its four FIFO buffers.
Flash controller is used to control the data transfer to flash memory
VGA controller displaying data on to VGA

9. Block diagram of Terasic camera system
TRDB_D5M_Userguide

10. Hardware Modules for this Lab
SDRAM-Multiport Controller:
This module is a generic 4 port SDRAM controller.
It uses two FIFO buffers for inputs (writing) and two FIFO buffers for outputs. The FIFO buffers are made using Altera MegaFunctions and are 16 bit wide and can hold up to 512 words of this size. In this system the SDRAM controller is used as a frame buffer to hold a single picture frame
Flash Controller
VGA controller
Camera Avalon Interface module

11. To get started with the lab
Run the DE2_NOIS_MOUSE_VGA_project present in the project folder given to you
Download the project content files put up online under lab2
Run SOPC builder and understand each of the components added and configuration present
Hardware component files are present under IP folder
It contains SDRAM_4 port controller components
Camera Avalon Interface
Software components are present under the software folder
hello_led_0 has all the required software component files
Some part of code is given in this project which is present in file hello_led.c

12. Information for C program
C code to capture camera data and store it in frame buffer
Picture taken at 640x480 has 3 data per pixel times 8 bits (at least) each, this would mean a total of 7.37 Mbit or Kbyte
So efficient storage design required
Recommended C code function
Camera Capture
Copying RGB(pixels) data from FIFO to Flash memory
Using flash memory to store the pixel data before transferring to VGA output
Program space memory is less (SRAM).Avoid unnecessary buffers. Write programs as efficient as possible

13. Displaying image through VGA controller
Look through VGA.h for VGA functions
Write C code similar to that shown below that calls functions from VGA.h to display each pixel

14. Reading pixels from Flash Memory(hello_led.c)
The methods called inside this file are NIOS II APIs.
Use this skeleton to develop your code.

15. Function.c This file needs to be updated with your code.
Various skeleton functons are defined. You need to modify/extend these functions.

16. Additional information
The image should be 1-bit per pixel with a resolution of 640x480
Uncompressed

17. Image Processing
It is required that you implement two forms of image processing (out of 5)
Each bullet below correspond to one image processing technique you need to show on the VGA
Recommendations include:
Add timestamp onto image
Counter to keep track of number of pictures taken
Rotate, mirror, invert image
Simple edge detection (challenging)
Detect changes in images (challenging)

18. References for software and hardware design in Lab2
I recommend looking over section II of the NIOS II Software Developer’s Handbook
Go through project titled- Embedded Demonstrator for Video Presentation and Manipulation by Cato Marwell Jonasse (Google it) to get clear insight on hardware description for this project
(You can go through chapter 9 specifically)
DE2_NIOS_HOST_MOUSE_VGA project
NIOS II Software Developer’s Handbook

19. Where does this project lead?
Next lab will be on the topic sending data over a network
Sending image between DE2 boards
Don’t worry if you have not taken computer networking

20. Questions and Comments

21. Back up

22. Readout modes: D5M camera
The Terasic D5M camera supports a function called :
1. binning -This function reduces the resolution of the image by averaging pixels together
2. Skipping - reduces the output resolution without affecting the field-of-view. It does this by not sampling entire rows and columns of pixels. A skip 2X mode skips one of pixels for every pair of output.