1. Embedded Streaming Media with GStreamer and BeagleBoard
sudo ifdown usb0 ; sudo ifup usb0 ; ifconfig usb0
ifconfig usb
ESC-321
Presented by Todd Fischer

2. Agenda Introduction to BeagleBoard multimedia architecture features.
GStreamer concepts
OMAP GStreamer hands on exercises
DMAI and GStreamer
OpenMAX and GStreamer
Questions
Use of the word “introduction” since the subject is broad to cover it on 90 minutes

3. Introduction
Primary objective for this class is to introduce audience to GStreamer on OMAP3 with hands on exercises
This doesn’t class won't cover extensively the APIs available and assumes basic knowledge on C programming, Linux and computer graphics.

4. BeagleBoard and OMAP3 architecture
Provides architecture with several multimedia features:
Cortex A8 with Neon
C64x DSP
Video accelerators
How to utilize the hardware features with the software stack?

5. BeagleBoard and OMAP3 architecture
Provides architecture with several multimedia features:
Cortex A8 with Neon
C64x DSP
Video accelerators
How to utilize the hardware features with the software stack?

6. BeagleBoard and OMAP3 architecture
Provides architecture with several multimedia features:
Cortex A8 with Neon
C64x DSP
Video accelerators
How to utilize the hardware features with the software stack?

7. GStreamer Streaming media framework – audio and video
Close to 200 plug-ins available
Higher level than just input / filters / output
Networking, audio/video mixed streams, auto data handling
Various options utilizing hardware accelerators

8. GStreamer Overview Elements Sources, filters, sinks Bins and Pipelines
Containers, pipeline is the overall bin
Pads
Element source / sink connection points
Caps
Capabilities organized by stream type with a set of properties
More details on the Linux graphic software stack in next slides

9. GStreamer Overview Elements Plugin Sources, filters, sinks
Bins and Pipelines
Containers, pipeline is the overall bin
Pads
Element source / sink connection points
Caps
Capabilities organized by stream type with a set of properties
Plugin
Collection of
elements
More details on the Linux graphic software stack in next slides

10. Simple MP3 Player Create dynamically using gst-launch
Source element reads from a file
Filter element converts MP3 to PWM
Sink element passes to ALSA output
gst-launch audiotestsrc freq=1000 num-buffers=100 ! \
alsasink
gst-launch filesrc location=bbb.flac ! flacdec ! alsasink
gst-launch filesrc location=bbb.mp3 ! \
mp3parse ! ffdec_mp3 ! alsasink
# see capture sources
cset numid=18,iface=MIXER,name='Capture Source'
# set to aux in
amixer cset numid=18,iface=MIXER,name='Capture Source' 2
export PKG_CONFIG_PATH=/usr/lib/pkgconfig
gcc -Wall $(pkg-config --cflags --libs gstreamer-0.10) \
audioplayer.c -o audioplayer
./audioplayer bbb.ogg
gst-launch filesrc location=a.mp3 ! mad ! alsasink

11. Hands On Exercise 0 cd gst . src Period character to source the filee
Start up Windowing environment with two terminals
cd gst
. src
List all plug-ins
gst-inspect
List video plug-ins
gst-inspect -a | grep -i 'class.*video'
List filesrc plug-in element properties
gst-inspect filesrc
Period character to source
the filee

12. Hands On Exercise 0 a2 gst-launch filesrc location=bbb.flac !
Play audio file a2
Actual command
gst-launch filesrc
location=bbb.flac !
flacdec ! alsasink
List all plug-ins
gst-inspect
List video plug-ins
gst-inspect -a | grep -i 'class.*video'
List filesrc plug-in element properties
gst-inspect filesrc

13. Simple Audio Player Source Code
Create pipeline, source, filter, sink
Set element properties
Build into pipeline
Connect src and sink pads
Setup pipeline event handler
End of stream
Set pipeline state to play
Run

14. Simple PA System Create dynamically using gst-launch
Source element ALSS audio in
No filters
Sink element passes to ALSA output
gst-launch audiotestsrc freq=1000 num-buffers=100 ! \
alsasink
gst-launch filesrc location=bbb.flac ! flacdec ! alsasink
gst-launch filesrc location=bbb.mp3 ! \
mp3parse ! ffdec_mp3 ! alsasink
# see capture sources
cset numid=18,iface=MIXER,name='Capture Source'
# set to aux in
amixer cset numid=18,iface=MIXER,name='Capture Source' 2
export PKG_CONFIG_PATH=/usr/lib/pkgconfig
gcc -Wall $(pkg-config --cflags --libs gstreamer-0.10) \
audioplayer.c -o audioplayer
./audioplayer bbb.ogg
gst-launch alsasrc num-buffers=100 ! alsasink

15. Keeping Plug-ins Organized
Each known plug-in is added to registry
Most aspects of plug-in are tracked in the registry
Registry support run-in pipeline creation and dynamic filter selection
Use gst-inspect to list plug-ins
gst-inspect | less gst-inspect filesrc

16. Hands On Exercise 1 Using gst-inspect, list All plug-ins
All video plug-ins
Element properties for filesrc plug-in
List all plug-ins
gst-inspect
List video plug-ins
gst-inspect -a | grep -i 'class.*video'
List filesrc plug-in element properties
gst-inspect filesrc

17. Hands On Exercise 2
GStreamer video pipelines
gst-launch videotestsrc ! ffmpegcolorspace ! fbdevsink
List all plug-ins
gst-inspect
List video plug-ins
gst-inspect -a | grep -i 'class.*video'
List filesrc plug-in element properties
gst-inspect filesrc

18. Performance Data Passing
Stream held in buffers with data, timestamp, other info
When possible, buffer memory allocated by sink pad
Use hardware when data copy is necessary

19. Performance Data Transformation
Cortex A8 compiler optimization
NEON
Single Instruction Multiple Data
C64
Video accelerator
DMA and other data movers

20. Davinci Multimedia Application Interface

21. DMAI and GStreamer Davinci Multimedia Application Interface
Exposes OMAP/Davinci hardware using high level of abstraction
Stream audio / video
Graphics display
Hardware optimized frame/data copy

22. OMAP3 Codec Engine
Isolates users for audio/video codecs from those implementing the codecs
Codec can run in several places without calling application being aware
Cortex A8, NEON, C64, hardware accelerator
Uses DSPLink and DSPBios conventions to support DSP based algorithms dynamically

23. Video Phone
Vision – video phone application can create audio and video GStreamer pipelines for capture / rendering
DMAI, Codec engine, DSPLink, C64 algorithms perform heavy lifting
OMAP3 hardware capabilities utilized
Video phone application focused on call management, not streaming data manipulation

24. OpenMAX and GStreamer Addresses codecs using hardware accelerators
GStreamer integration via gst-openmax
All OpenMax codecs available as GStreamer plug-ins
Initial focus on high performance codecs

25. OpenMAX and OMAP3 ARM Inc has Cortex-A8 / NEON DL implementation
Texas Instruments has AAC and MPEG4 IL / DL support

26. GStreamer in 3 Layers