1. Lab 14 Department of Computer Science and Information Engineering National Taiwan University Lab14 – Camera 2014/12/30 1 /14

2. Lab 14 Department of Computer Science and Information Engineering National Taiwan University  Learn how to use DSP and CMOS sensor to take pictures on PXA270 by using the miniGUI from GPRS driver. 2014/12/30 2 /14 TIDSP-5502 CMOS Sensor

3. Lab 14 Department of Computer Science and Information Engineering National Taiwan University  Host System  Windows XP  Build System  VirtualBox + Ubuntu 8.04  Target System  Creator XScale PXA270  Software  LCD, GPRS and DSP Drivers  JPEG library and GPRS program  You can download all software from RSWiki CSL Course SoftwareRSWiki CSL Course Software 2014/12/30 3 /14

4. Lab 14 Department of Computer Science and Information Engineering National Taiwan University  PXA270  TIDSP-5502  CMOS Sensor 2014/12/30/14 4 CMOS TIDSP-5502

5. Lab 14 Department of Computer Science and Information Engineering National Taiwan University  A digital signal processor (DSP) is a specialized microprocessor with an architecture optimized for the operational needs of digital signal processing.  Digital signal processing algorithms typically require a large number of mathematical operations to be performed quickly and repeatedly on a series of data samples. 2014/12/30/14 5 Reference: wiki info: DSP, en.wikipedia.org/wiki/Digital_signal_processoren.wikipedia.org/wiki/Digital_signal_processor

6. Lab 14 Department of Computer Science and Information Engineering National Taiwan University  An active-pixel sensor (APS) is an image sensor consisting of an integrated circuit containing an array of pixel sensors, each pixel containing a photodetector and an active amplifier.  There are many types of active pixel sensors including the CMOS APS used most commonly in cell phone cameras, web cameras, etc.  Such an image sensor is produced by a CMOS process (and is hence also known as a CMOS sensor), and has emerged as an alternative to charge- coupled device (CCD) image sensors. 2014/12/30/14 6 Reference: wiki info: APS, en.wikipedia.org/wiki/Active_pixel_sensoren.wikipedia.org/wiki/Active_pixel_sensor

7. Lab 14 Department of Computer Science and Information Engineering National Taiwan University  Step 1: Download drivers2.tar.bz2 and refer to Lab12 to add the following drivers to the kernel.drivers2.tar.bz2  Creator-pxa270 DSP5502  Creator-pxa270 LCD  Creator-pxa270 GPRS  You can build them as modules.  Step 2: Create related device files if they does not exist.  % mknod /dev/lcd c 120 0  % mknod /dev/gprs c 122 0  % mknod /dev/mtdsp c 123 0 2014/12/30/14 7

8. Lab 14 Department of Computer Science and Information Engineering National Taiwan University  Step 3: Download gprs-remote-ctrl.tar.bz2.gprs-remote-ctrl.tar.bz2  Step 4: Modify configuration files and Makefile.  In gprs-ctrl-build/jpeg-6b/microtime_configure,  Delete “.$HOME/bin/microtime ”.  Modify “ --enable-shared ” to “ --enable-static ”.  Add the following to the front of the file.  MICROTIME_USR_LOCAL_DIR=  mkdir -p $MICROTIME_USR_LOCAL_DIR  mkdir -p $MICROTIME_USR_LOCAL_DIR/bin  mkdir -p $MICROTIME_USR_LOCAL_DIR/include  mkdir -p $MICROTIME_USR_LOCAL_DIR/lib  mkdir -p $MICROTIME_USR_LOCAL_DIR/man  mkdir -p $MICROTIME_USR_LOCAL_DIR/man/man1 2014/12/30/14 8

9. Lab 14 Department of Computer Science and Information Engineering National Taiwan University  In gprs-ctrl-build/creator-pxa270-gprs-1.0/microtime_configure,  Delete “.$HOME/bin/microtime ”.  Delete “ export LINUXDIR=$MICROTIME_LINUX_DIR ”.  Add the following to the front of the file.  export MICROTIME_USR_LOCAL_DIR=  export MICROTIME_BUILD_PACKAGE_DIR=  export LINUXDIR=  In gprs-ctrl-build/creator-pxa270-gprs-1.0/Makefile,  Add a new flag “ -static ” at the end of “ CFLAGS ”.  In gprs-ctrl-build/creator-pxa270-gprs-1.0/creator-pxa270- gprs.c,  Delete the code of initializing UART about 5 lines at line 227. 2014/12/30/14 9

10. Lab 14 Department of Computer Science and Information Engineering National Taiwan University  Step 5: Compile the JPEG library.  Execute “ gprs-ctrl-build/jpeg-6b/microtime_configure ”.  Modify the generated Makefile, where “ AR= arm-unknown-linux-gnu-ar rc ”, but not “ AR= ar rc ”.  % make libjpeg.a  % cp libjpeg.a /lib  Step 6: Compile the GPRS program.  Execute “ gprs-ctrl-build/creator-pxa270-gprs-1.0/microtime_configure ”.  Copy “ cmosr.out.bin ” and “ creator-pxa270-gprs ” to PXA270.  Step 7: Connect the DSP module and CMOS to PXA270 and execute.  $./creator-pxa270-gprs  If you use drivers as modules, don’t forget to insmod modules before executing.  By pressing keypad 7, the CMOS sensor will take a picture, and store it as “ dsp_data.jpg ” in PXA270. 2014/12/30/14 10

11. Lab 14 Department of Computer Science and Information Engineering National Taiwan University  Step 8: Display images on the LCD at the same time.  On PXA270, there is an LCD device which can display not only messages but also a 128*128 image with 4 gray levels.  We take a colored picture by CMOS and save the image in JPEG format. 2014/12/30/14 11

12. Lab 14 Department of Computer Science and Information Engineering National Taiwan University  The LCD driver has already provided an ioctl command to display an image on an LCD.  You can implement a new function in creator_lib.c, and use this command in GPRS program.  You can refer to the example source code, i.e., lcd.c, to see how to set the data to an LCD image buffer.lcd.c  Since the captured image is colored, i.e., each pixel consists of three components: red(R), green(G) and blue(B), if you want to display on the LCD, please convert it to 4 gray levels.  Any reasonable conversion is acceptable.  E.g., convert RGB color space to YUV color space, then quantize the luminance Y to 4 gray levels or apply edge detection filter, then convert the edges to black and white, etc. 2014/12/30/14 12

13. Lab 14 Department of Computer Science and Information Engineering National Taiwan University  For more information, please refer to the following sites:  Grayscale, http://en.wikipedia.org/wiki/Grayscalehttp://en.wikipedia.org/wiki/Grayscale  RGB, http://en.wikipedia.org/wiki/RGB_color_modelhttp://en.wikipedia.org/wiki/RGB_color_model  YUV, http://en.wikipedia.org/wiki/YUVhttp://en.wikipedia.org/wiki/YUV  Edge detection, http://en.wikipedia.org/wiki/Edge_detectionhttp://en.wikipedia.org/wiki/Edge_detection  Note that the resolution of the image (320*240) is different from the LCD (128*128), so you can scale the resolution before you display.  Any reasonable scaling method is acceptable.  E.g., nearest neighbor, http://tech-algorithm.com/articles/nearest-neighbor- image-scaling/ or bilinear, http://tech-algorithm.com/articles/bilinear-image- scaling/http://tech-algorithm.com/articles/nearest-neighbor- image-scaling/http://tech-algorithm.com/articles/bilinear-image- scaling/ 2014/12/30/14 13

14. Lab 14 Department of Computer Science and Information Engineering National Taiwan University  Take pictures by the CMOS.  Please modify the DSP driver ( creator-pxa270-dsp5502.c ) and the GPRS program ( dsp.c ) to take 320*240 pictures.  The default resolution is 160*120.  Display the image on PC and on the LCD.  You can use tftp to upload your image file.  $ tftp -p -l 2014/12/30/14 14