Video Streaming via Transcoding Jianping Fan Department of Computer Science University of North Carolina at Charlotte Charlotte, NC

query object feature extraction Cluster 1 Cluster i Cluster n Subcluster i1Subcluster ij Subcluster im Subregion ij1Subregion ijl Subregion ijr Object ijrm Disk for cluster 1Disk for cluster i 1. Why we need transcoding? Networks?

1. Why we need transcoding? How can I access video in database over networks? Networks a. Videos in database are encoded, fixed and stored. b. Network conditions are always changing! Streaming Videos via Trancoding!

2. Key parameters for transcoding a. Video coding format used in database system. Bit rate for the retrieved video! b. Network conditions such as bandwidth! c. Client’s QoS parameters: Quality such as resolution Delay Quality of Presentation of video

3. Basic requirements transcoding Original bit stream Bit stream sent to client via network a. Bit rate after transcoding should be close to network bandwidth! b. Quality of transcoded video should be accepted by clients! Not induce too much quantization error again! c. Easy for implementation!

4. Case study: Streaming MPEG-1 video over ISDN MPEG-1 Stream ISDN Network Transcoder MPEG-1 Encoder Real-Time Transcoder network MPEG-1 player Error 1 Error 2 Error 1 + Error 2 should be accepted by clients!

5. Quality of Video Parameters for defining Quality of Video: a. Frame rate-The number of still pictures displayed per second to give the viewer perception of motion. b. Color depth-The number of bits per pixel for representing color information. c. Frame resolution-Typically expressed as the width and height in pixels.

6. Simple Transcoding Real-time decoding and real-time re-encoding MPEG-1 Stream MPEG-1 Decoder Real-time H.263 Encoder ISDN Network real-time Too Expensive & Too much Errors!

6. Simple Transcoding Where the bit rate comes from? a.Frame rate b.Frame resolution c.Color depth

7. Transcoding via Spatial Sampling Spatial Resolution Reduction: reduction resolution What will happen when we reduce MPEG video’s resolution?

7. Transcoding via Spatial Sampling Spatial Resolution Reduction: time GOP 4GOP 3GOP 2GOP 1 IBBPBBPBBBP GOP

7. Transcoding via Spatial Sampling Spatial Resolution Reduction: I frame New I frame What will happen if do like this? Sample filtering?

7. Transcoding via Spatial Sampling Spatial Resolution Reduction: How to do it? Y U V How to code I frame? I frame

7. Transcoding via Spatial Sampling Spatial Resolution Reduction: Sample filtering We can do this easily on original image, can we this sample filtering on compressed domain (DCT coefficients)? DC and AC coefficients! I-macroblock

7. Transcoding via Spatial Sampling Spatial Resolution Reduction: Sample filtering

7. Transcoding via Spatial Sampling Spatial Resolution Reduction: If we do resolution reduction on I frames, there are two questions: (1) how we can do this on P & B frames? (2) what’s the influence on motion compensation?

7. Transcoding via Spatial Sampling Spatial Resolution Reduction: Original I FrameOriginal P Frame Transcoding I frame ? How we can use transcoded I frame in the decoder side to get P frame?

7. Transcoding via Spatial Sampling Spatial Resolution Reduction: Original I FrameOriginal P Frame Transcoding I frameTranscoding P frame transcoding ?

7. Transcoding via Spatial Sampling Spatial Resolution Reduction: I or P frame P frame Motion vectors & DCT coefficients of compensated errors How to sample the motion vectors?

8. Transcoding via Temporal Sampling Temporal Resolution Reduction (frame dropping): Frame dropping (1) Which frame should be dropped? (2) What is the influence if we drop some frames?

7. Transcoding via Temporal Sampling Temporal Resolution Reduction (frame dropping): (1) We cannot drop I frames in a GOP (group of pictures). (2) If we drop a P frame, how about its B frames? Drop B frame together or refine motion vectors? IBP (1) It is no problem for us dropping B frames, but bit rate for them is low!

8. Transcoding via Temporal Sampling Temporal Resolution Reduction (frame dropping): I BB P B BP I BB P How we can get compensation of B from another P frame? trancoding

8. Transcoding via Temporal Sampling IPP Motion vector 1Motion vector 2 Motion vector 3 (1) What’s the relationship between motion vector 3 and motion vector 1 and motion vector 2? DCT I P (2) What’s the relationship between the DCT coefficients between I frame and frame after one P is dropped?

9. Transcoding via Averaging Video Averaging Videos Select DC coefficient for each block! DC AC 63 DC

9. Transcoding via Averaging Video Averaging Videos If we just select DC coefficients for I frame, what’s influence on P frames and B frames? I P ? averaging

10. Transcoding via Reducing Color Depth Color depth reduction via binary: binary 8 bits 1 bit (1) How to do this on the compressed domain? (2) What’s the influence on P and B frames?

11. Transcoding via Re-Quantization Re-Quantization Quantization is an efficient way to reduce the redundancy information, but also induce errors in the picture! In current video and image coding standards such as MPEG-*, H.26*, JPEG-*, they use two parameters to define quantization: (1) Scale; (2) Quantization table. We can only change scale to realize re-quantization! What’s the influence for motion compensation?

12. Transcoding with Network Condition MPEG-1 Stream ISDN Network Adaptive Transcoder Parameters should be used for adaptive trancoder design: a. Frame rateb. resolution c. Color depthd. Averaging

12. Transcoding with Network Condition MPEG-1 Stream ISDN Network Adaptive Transcoder Problem: Adaptive Transcoder Design Subject to: Network bandwidth (changing) Source video bit rate ? Network should be used for selecting parameters!

13. Transcoding with Client QoS MPEG-1 Stream ISDN Network Adaptive Transcoder Problem: Adaptive Transcoder Design Subject to: Client’s QoS Source video bit rate ?

14. Open Discussion a. Why we select streaming for video not download? b. How transcoder can adapt? c. Can we do transcode fully on compressed domain?

How can we support content-adaptive video transcoding?

Do we keep the dominant video contents?