DSPGateway Architecture in Detail Yoshiya Hirase Nokia Research Center Nokia Japan Co. Ltd. 1 © 2005 Nokia V1-Filename.ppt / yyyy-mm-dd / Initials

Agenda DSP Gateway Features Demonstration Using Nokia 770 2 © 2005 Nokia CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo

DSP Gateway Features Open Source Software Simple interface for DSP Integrated in Nokia 770 Internet Tablet Simple interface for DSP Efficient inter-processor communication (ARM <-> DSP) Block data transfer via shared memory DSP Task management Dynamic Loader Fault-tolerant mechanism Power management Simple interface for DSP conventional Linux device files Efficient inter-processor communication (ARM <-> DSP) Minimum copy count for maximum throughput Memory management IPBUF: Inter-processor buffer mechanism (“Global”, “Private”, “System”) mmap function Multi-task environment for DSP applications Fault-tolerant mechanism Power management 3 © 2005 Nokia CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo

History 2003/6: Released the DSP Gateway version 1.0 for OMAP1510 at SourceForge 2004/8: Supported OMAP1610/5912 2004/9: Merged into the Linux OMAP tree 2005/2H: Nokia 770 released with DSP Gateway The latest DSP Gateway ver 3.3 supports OMAP 15XX (including 5910) and 16XX (including 1710 and 5912). 4 © 2005 Nokia CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo

DSP Interface: Device Files DSP task devices interface to DSP task DSP control device miscellaneous DSP control (e.g. DSP configuration) DSP memory device DSP memory access DSP task watch device for dynamic loader DSP error detection device error info delivery 5 © 2005 Nokia CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo

Inter-Processor Communication Communications via OMAP mailbox mechanism OMAP ARM DSP Mailbox registers (16bit + 16bit) interrupt (16bit + 16bit) interrupt internal memory IPBUF IPBUF: Inter-communication buffer mechanism memory IPBUF 6 © 2005 Nokia CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo

Inter-Processor Communication: IPBUF Global IPBUF: Used to transfer block data to/from DSP task Prepared by the system and managed with alloc & free functions Fixed block size, restricted block count Private IPBUF: A task can define an exclusive buffer, with arbitrary size System IPBUF: Used to transfer extra data between systems of ARM and DSP ARM DSP 1 task A task A (alloc) (free) 1 2 3 4 Global IPBUF Global IPBUF ARM DSP task B Private IPBUF task B task B Private IPBUF ARM DSP System IPBUF system system System IPBUF 7 © 2005 Nokia CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo

Inter-Processor Communication: Mailbox commands 8 © 2005 Nokia CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo

Memory Mapping DSP internal memory External memory DARAM, SARAM SDRAM Visible for ARM and DSP 9 © 2005 Nokia CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo

Task Management: Dynamic Loader 10 © 2005 Nokia CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo

Task Management: DSP Application Life Cycle DSP Gateway ARM DSP process 1 Linux application Device driver DSP kernel DSP application DSP task dynamically created open() open() activate DSP task complete open() return write() write() input data / control command task running write() return output data read() read() read() return DSP task dynamically deleted Example -dynamic task- close() close() deactivate DSP task complete close() return 11 © 2005 Nokia CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo

MP3 Pseudo program (ARM side) int fd_mp3file, fd_dsp; DATA buf[BUFSZ]; int Decode_MP3(void) { // Open MP3 decoder device fd_dsp = open(“/dev/dsptask/mp3dec”, O_RDWR); // Initialize MP3 decoder ioctl(fd, MP3_DECODE_INIT); // Decode MP3 audio file while (ReadMp3File(fd_mp3file, buf, BUFSZ) != EOF) { // Send encoded data to MP3 decoder in DSP write(fd_dsp, buf, BUFSZ); } // Close MP3 decoder device close(fd_dsp); Issue TCTL command to DSP Issue BKSND command to DSP 12 © 2005 Nokia CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo

MP3 Pseudo program (DSP side) Uns rcv_tctl(struct dsptask *task, Uns ctlcmd, Uns *ret, Uns arg) { switch (ctlcmd) { case MP3_DECODE_INIT: InitIMDCT(); InitFilters(); break; ... } Uns rcv_bksnd(struct dsptask *task, Uns bid, Uns cnt) /* Decode received MP3 data */ Dequantize(); ReduceAlias(); IMDCT(); /* Send decoded PCM data to the Audio Device Driver */ Handle MP3_DECODE_INIT of TCTL command Handle BKSND command 13 © 2005 Nokia CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo

Nokia 770 Internet Tablet Features Display High-resolution (800x480) touch screen with up to 65,536 colors Memory Engine: DDR RAM 64MB Memory: Flash 128MB (>64MB for user) Memory card: 64MB RS-MMC (Reduced Size - MultiMediaCard) Supported file formats Audio: MP3, Real Audio, AAC, WAV, AMR Image: JPEG, GIF, BMP, TIFF, PNG, Animated GIF format, SVG-tiny, ICO Video: MPEG1, MPEG4, Real Video, H.263, 3GP (RED items are decoded on DSP, BLUE items are routed through DSP) Connectivity WLAN: 802.11b/g Bluetooth specification: 1.2 USB device mode for PC connectivity RS-MMC 3.5 mm stereo audio out 14 © 2005 Nokia CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo

Multimedia Software Architecture in Nokia 770 Linux DSP/BIOS Multimedia Player GStreamer Plugin #1 (Codec) Plugin #2 Plugin #3 Codec #2 Codec #3 DSP Gateway ARM DSP 15 © 2005 Nokia CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo

Future Work OMAP2420 support (including IVA support) Driver Abstraction, and expansion for generic AMC (Asymmetric Multi-Core) processors 16 © 2005 Nokia CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo

Demo: Play Media Files using DSP Gateway System log message Device status Play sound or movie 17 © 2005 Nokia CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo

References: Project page: Articles: http://sourceforge.net/projects/dspgateway Articles: 森　英悟, 小林　俊裕, 高橋　清隆. “オープン・ソース・ソフトウェアDSPGatewayを用いたLinuxで使うOMAP DSP部のソフトウェア開発環境.” Interface: CQ Publishing, May 2004, 156-164 http://www.cqpub.co.jp/interface/contents/2004/200405.htm 18 © 2005 Nokia CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo

Thank you 19 © 2005 Nokia CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo

Appendix: File manager Image 20 © 2005 Nokia CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo

Appendix: Audio Player Image 21 © 2005 Nokia CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo

Appendix: BKSND command in log 22 © 2005 Nokia CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo

Appendix: TDEL command in log 23 © 2005 Nokia CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo

Appendix: Task Status Transition 24 © 2005 Nokia CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo

Appendix: Log message while playing MP3 25 © 2005 Nokia CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo

Appendix: Performance Evaluation Example Memory mapping improves data transfer well Data send latency (to DSP) by 44%-68% Data receive latency (from DSP) by 27-55% MPU bksnd := MPU sends a data blocks to DSP MPUbkreq_DSPbksnd := MPU sends a block request to DSP -> DSP receives it & then sends the block back -> MPU receives the data block from DSP Memory write/read := MPU writes/reads data on the mapped memory 26 © 2005 Nokia CELF_Nokia_Japan_v01.ppt / 2006-01-20 / NRC-Tokyo