Renesas Electronics America Inc. © 2010 Renesas Electronics America Inc. All rights reserved. ID 410L: Low cost audio in 8/16 bit applications using ADPCM Carl Stenquist, C&I Staff Applications Engineer 12 & 14 October 2010 Ver. 1.0

2 © 2010 Renesas Electronics America Inc. All rights reserved. Mr. Carl Stenquist Staff Applications Engineer 5 years designing and supporting firmware applications for Renesas MCUs. Support & manage middleware: ADPCM, File System, MMC, CAN applications, flash programming. PREVIOUS EXPERIENCE 6 years embedded software consultant and lead of Embedded FW team, AF-Group, Sweden. Energy consumption logging and presentation system 'EnerLog‘; Enercom, Sweden. Teaching of Electronics, especially Control Systems at Royal Institute of Technology, Sweden.

3 © 2010 Renesas Electronics America Inc. All rights reserved. Renesas Technology and Solution Portfolio Microcontrollers & Microprocessors #1 Market share worldwide * Analog and Power Devices #1 Market share in low-voltage MOSFET** Solutions for Innovation ASIC, ASSP & Memory Advanced and proven technologies * MCU: 31% revenue basis from Gartner "Semiconductor Applications Worldwide Annual Market Share: Database" 25 March 2010 **Power MOSFET: 17.1% on unit basis from Marketing Eye 2009 (17.1% on unit basis).

4 © 2010 Renesas Electronics America Inc. All rights reserved. 4 Renesas Technology and Solution Portfolio Microcontrollers & Microprocessors #1 Market share worldwide * Analog and Power Devices #1 Market share in low-voltage MOSFET** ASIC, ASSP & Memory Advanced and proven technologies * MCU: 31% revenue basis from Gartner "Semiconductor Applications Worldwide Annual Market Share: Database" 25 March 2010 **Power MOSFET: 17.1% on unit basis from Marketing Eye 2009 (17.1% on unit basis). Solutions for Innovation

5 © 2010 Renesas Electronics America Inc. All rights reserved. 5 Microcontroller and Microprocessor Line-up Superscalar, MMU, Multimedia  Up to 1200 DMIPS, 45, 65 & 90nm process  Video and audio processing on Linux  Server, Industrial & Automotive  Up to 500 DMIPS, 150 & 90nm process  600uA/MHz, 1.5 uA standby  Medical, Automotive & Industrial  Legacy Cores  Next-generation migration to RX High Performance CPU, FPU, DSC Embedded Security  Up to 10 DMIPS, 130nm process  350 uA/MHz, 1uA standby  Capacitive touch  Up to 25 DMIPS, 150nm process  190 uA/MHz, 0.3uA standby  Application-specific integration  Up to 25 DMIPS, 180, 90nm process  1mA/MHz, 100uA standby  Crypto engine, Hardware security  Up to 165 DMIPS, 90nm process  500uA/MHz, 2.5 uA standby  Ethernet, CAN, USB, Motor Control, TFT Display High Performance CPU, Low Power Ultra Low Power General Purpose

6 © 2010 Renesas Electronics America Inc. All rights reserved. Notes continued from previous page (continued from notes section of previous page) Renesas knows that to best facilitate the further growth and success of ubiquitous computing, we cannot offer just one CPU core or a single family of microcomputers. Thus, taking advantage of the broad span of leading technologies we have built up, we have decided to concentrate our future R&D efforts on five major CPU cores capable of excelling at major elements of the huge task. Each is optimized for addressing the requirements of diverse sets of key applications. With that business plan in mind, allow me to explain the relative positioning of these five architectures within our strong portfolio of MCUs and MPUs. An important design trend in recent years has seen system engineers taking full advantage of all the computing power that IC makers have made available — often right up to the limits of project constraints. As a result, there have been more and more design-ins of chips with 32-bit architectures. Renesas now has three complementary 32-bit microcontroller and microprocessor families aiding that trend. At the top end of the features-and-capability spectrum we offer the devices in the SuperH family, a superscalar RISC architecture that executes two instructions per clock cycle. Devices in the SuperH family deliver up to 1200 DMIPS performance, so they’re ideal for and popular in multimedia, Real-time industrial-control, server, and automotive engine- control applications. We also recommend them for performing video and audio processing on Linux-based systems Our second series of 32-bit system design solutions is the V850 family, which today is the top-selling line of 32-bit microcontrollers, worldwide. The V850 architecture provides high performance (up to 500 DMIPS), yet consumes low power while doing so. System designers have found these devices to be particularly well suited for automotive applications. The lower- frequency V850 chips are optimized for low power. Thus, they are excellent choices for portable medical equipment, for example.

7 © 2010 Renesas Electronics America Inc. All rights reserved. 7 Microcontroller and Microprocessor Line-up Superscalar, MMU, Multimedia  Up to 1200 DMIPS, 45, 65 & 90nm process  Video and audio processing on Linux  Server, Industrial & Automotive  Up to 500 DMIPS, 150 & 90nm process  600uA/MHz, 1.5 uA standby  Medical, Automotive & Industrial  Legacy Cores  Next-generation migration to RX High Performance CPU, FPU, DSC Embedded Security  Up to 10 DMIPS, 130nm process  350 uA/MHz, 1uA standby  Capacitive touch  Up to 25 DMIPS, 150nm process  190 uA/MHz, 0.3uA standby  Application-specific integration  Up to 25 DMIPS, 180, 90nm process  1mA/MHz, 100uA standby  Crypto engine, Hardware security  Up to 165 DMIPS, 90nm process  500uA/MHz, 2.5 uA standby  Ethernet, CAN, USB, Motor Control, TFT Display High Performance CPU, Low Power Ultra Low Power General Purpose

8 © 2010 Renesas Electronics America Inc. All rights reserved. 8 Microcontroller and Microprocessor Line-up Superscalar, MMU, Multimedia  Up to 1200 DMIPS, 45, 65 & 90nm process  Video and audio processing on Linux  Server, Industrial & Automotive  Up to 500 DMIPS, 150 & 90nm process  600uA/MHz, 1.5 uA standby  Medical, Automotive & Industrial  Legacy Cores  Next-generation migration to RX High Performance CPU, FPU, DSC Embedded Security  Up to 10 DMIPS, 130nm process  350 uA/MHz, 1uA standby  Capacitive touch  Up to 25 DMIPS, 150nm process  190 uA/MHz, 0.3uA standby  Application-specific integration  Up to 25 DMIPS, 180, 90nm process  1mA/MHz, 100uA standby  Crypto engine, Hardware security  Up to 165 DMIPS, 90nm process  500uA/MHz, 2.5 uA standby  Ethernet, CAN, USB, Motor Control, TFT Display High Performance CPU, Low Power Ultra Low Power General Purpose R8C / H8 / M16C ADPCM Audio Solution

9 © 2010 Renesas Electronics America Inc. All rights reserved. Audio Feedback Hidden equipment When difficult to read a display Innovation

10 © 2010 Renesas Electronics America Inc. All rights reserved. Easy to Add Audio Embedded audio doesn’t need D/A - A PWM signal can be filtered for playback Less memory is used with 4:1 ADPCM encoding Use low cost external audio memory Can be rewritten: Example with Hew Target Server via E8 debug circuit Position

11 © 2010 Renesas Electronics America Inc. All rights reserved. Agenda What is ADPCM? - Encoding and Decompressing audio Flow of data through hardware Conversion of PWM to Analog signal SubAtomic Particle PCB solution Processing Requirements Loading audio data to external memory LAB

12 © 2010 Renesas Electronics America Inc. All rights reserved. Key Takeaways After this session you will have seen: 1.What ADPCM is. How it works. 2.How a PWM signal is created from the decoded audio data and is converted to an audio signal. 3.How HEW and HEW Target Server work together to write an image to an external flash chip

13 © 2010 Renesas Electronics America Inc. All rights reserved. DPCM = Differential Pulse Code Modulation Sound data is continuous  Only save difference in output between samples. Algorithm estimates (predicts) next sample Prediction is offset with the actual DPCM sample.  Only the difference between the prediction and the actual sample needs to be saved, and not the ‘whole’ difference.

14 © 2010 Renesas Electronics America Inc. All rights reserved. Each 4-bit sample approximates difference between present value and previous, encoded as Bit# Sign Magnitude (0 to 7) Magnitude is 0-7 ADPCM: Unit step weight varies, or adapts with step size non-linearly…  ADPCM = Adaptive Differential Pulse Code Modulation

15 © 2010 Renesas Electronics America Inc. All rights reserved. Unit stepsize is weighted (quantized) using a non-linear table Unit weight changes each sample via another non-linear table Example. Current unit stepsize is 400: – Magnitude is 5: Increase weight by 4 steps in table – Next sample is 2: Decrease weight by 1 step in table ADPCM… Sample mag- nitude Table index change Output(n)=Output(n-1) +- stepsize*(b2 + b1/2 + b0/4 +1/8) Sign determined by b3

16 © 2010 Renesas Electronics America Inc. All rights reserved. ADPCM Encode 16 bit PCM data (mono)4 bit ADPCM data hello.wav 64 Kbytehello.dat 16 Kbyte 4 bit ADPCM Data16 bit PCM data (mono) Compress 4:1 with e.g. ADPCM.exe Decompress using target decode library ADPCM Decode

17 © 2010 Renesas Electronics America Inc. All rights reserved. Q&A How many 16-bit audio samples are embedded in one byte of audio memory? 1, 2, 4, or 8? 2 (4 bits/sample, 4:1 compression) How can the ADPCM algorithm create 16-bit audio from 4-bit samples? The information encoded is nonlinear, that is, the samples include a bit- weight scaling mechanism.

18 © 2010 Renesas Electronics America Inc. All rights reserved. 1MB Audio Data Size 1 byte decodes to 2 sound samples of 16 bits each. With sample rate 8 kHz (phone), how much memory for 1 sec. of sound? Calculation: 1 second sound => 8000 samples samples * ½ byte = 4 kB of ROM. How much audio in 1 MB chip? 1 MB/4kB/s = 4 minutes Example: R8C 4 kB RAM ROM 1 sec >4 minutes 4 kB/sec (8kHz)

19 © 2010 Renesas Electronics America Inc. All rights reserved. Sub Atomic Particle PCBs with ADPCM For debugging and programming the target MCU SPB available for R8C & M16C 1 MB audio memory Two channel audio amplifier with LP-filter

20 © 2010 Renesas Electronics America Inc. All rights reserved. Data Flow in Hardware MCU peripherals used: SCI (using SPI protocol) RAM for temp sound unpack storage One timer for sample update timing 1 or more (stereo) timers in PWM output mode LP Filter AMP MCU SPB Board Sample Timer SCI PCM sound buffer RAM PWM timer Instead of DAC! 1MB Serial flash mem hello.dat SPI-bus

21 © 2010 Renesas Electronics America Inc. All rights reserved. Converting PWM to Analog One Sample Period at Silence

22 © 2010 Renesas Electronics America Inc. All rights reserved. Converting PWM to Analog One Sample Period at Silence

23 © 2010 Renesas Electronics America Inc. All rights reserved. Tricky 60 second question Match number with letter 1. The sampling frequency determines… 2. The PWM counter source frequency determines… A. …the highest possible frequency that can be reproduced. B. …the bit resolution of the PWM signal. Answer: 1A, 2B.

24 © 2010 Renesas Electronics America Inc. All rights reserved. Decimal values. Sizes are of entire demo R8C/25M16C/26A Timer SamplingTimer RD Channel 1 ATimer B1 PWMTimer RD Channel 1 C&DTimer A2/A3 Serial Flash Interface SSUUART0 Memory ROM3 kB RAMRAM Static320 B350 B Stack170 B150 B SPB ADPCM Audio Resource Usage

25 © 2010 Renesas Electronics America Inc. All rights reserved. ADPCM Processing at 11 kHz Only ISR 1 ch. audio decode + PWM update 2 ch. audio decode + PWM update

26 © 2010 Renesas Electronics America Inc. All rights reserved. Decode Time Example decode function: adpcm_decode4(); Decodes 4 bytes of ADPCM data from 16-bit PCM data. MCUDecode Time M16C/26A56 us R8C/2575 us

27 © 2010 Renesas Electronics America Inc. All rights reserved. Timing Is Important 11 KHz sampling 16 KHz 20MHz  Solution: Decode into a buffer when not busy 63 μsec 90 μsec OK! Missed deadline!

28 © 2010 Renesas Electronics America Inc. All rights reserved.

29 © 2010 Renesas Electronics America Inc. All rights reserved. Loading Sound Data into External Memory Or, use HEW Target Server! HEW Target Server Flash Loader ADPCM data in mot file Use a serial link MMC card Serial Flash or MCU SCI Serial Flash Subatomic Particle Board MCU E8 Circuit

30 © 2010 Renesas Electronics America Inc. All rights reserved. Embedded Audio File Format Bin-to-mot takes these ADPCM data files… …and creates one file with a header (green) Whole file written to flash with VB PC GUI program + HTS + Memloader workspace

31 © 2010 Renesas Electronics America Inc. All rights reserved. Dice Pictures for you to use We will have dice at every station. In your presentation, instruct the attendee to roll the dice to a spot when they complete a section. Here are the pictures for your use.

32 © 2010 Renesas Electronics America Inc. All rights reserved. Checking Progress We are using the die to keep track of where everyone is in the lab. Make sure to update it as you change sections. When done with the lab, your die will have the 6 pointing up as shown here.

33 © 2010 Renesas Electronics America Inc. All rights reserved. From Microphone to Serial Flash

34 © 2010 Renesas Electronics America Inc. All rights reserved. Questions?

35 © 2010 Renesas Electronics America Inc. All rights reserved. Conclusion Adding audio playback to an embedded application is easy and inexpensive. Embedded audio doesn’t need a D/A converter. A PWM signal can be filtered for playback instead. Less memory is used with 4:1 ADPCM encoding. No programming equipment needed for large audio memory since HTS + E8 can be used.

36 © 2010 Renesas Electronics America Inc. All rights reserved. Feedback Form Please fill out the feedback form! If you do not have one, please raise your hand

© 2010 Renesas Electronics America Inc. All rights reserved. 37 Appendix

38 © 2010 Renesas Electronics America Inc. All rights reserved. NAME adpcm_decode4 SYNOPSIS int adpcm_decode4(T_ADPCM *pADPCM); DESCRIPTION Decodes 4 samples at a time of ADPCM data to 16-bit PCM data. The decoded data is stored to the memory area specified by a T_ADPCM structure, which also specifies the number of ADPCM samples to be decoded. ADPCM library decode function

39 © 2010 Renesas Electronics America Inc. All rights reserved. Why M16C is faster…

40 © 2010 Renesas Electronics America Inc. All rights reserved.

Renesas Electronics America Inc.