1. Chapter 7 – End-to-End Data Two main topics Presentation formatting Compression We will go over the main issues in presentation formatting, but not much detail More detail will be covered in compression, especially JPEG and MPEG

2. Presentation Formatting/Encoding The receiver must be able to extract the same message from the signal as the transmitter sent Encoding is sometimes called argument marshalling Marshalling is actually not trivial – because compilers and application programs have a lot of latitude in how they lay out structures (records) Look over the next high-level graphic

3. Application data Presentation encoding Application data Presentation decoding Message …

4. Taxonomy Base types – lowest level Integers, floating point, characters, etc. Flat types Structures Arrays Complex types – highest level Types requiring pointers

5. Examples Case 1 – sending an ordered string of integers (say financial market data) over the internet – no problem breaking this up into a string of bytes and no problem reassembling the data at the end

6. Examples continued Case 2 – sending a database of student records over a network. Students would have different numbers of courses that took, so the records would be of different length but the fields would probably be fixed length Packing and unpacking the data would have some difficulties involved

7. Examples continued Case 3 – a hierarchical database stored in a format with pointers needs to be transmitted over the Internet Packing and unpacking is a large problem Pointers are implemented by memory addresses and will change from one machine (sender) to another (receiver) Marshalling must serialize a complex, pointer implementation of a database – quite difficult!

9. Two Conversion Strategies Sender converts to common format, common format is transmitted, receiver decodes from common format Seems natural, but … Receiver-makes-right – transmit and let the receiver figure it all out Surprisingly this is often the better approach See the reasons on page 533

10. Call P Client stub RPC Arguments Marshalled arguments Interface descriptor for Procedure P Stub compiler Message Specification P Server stub RPC Arguments Marshalled arguments Code

11. Data Compression Blue.bmp = 293 KB Blue.jpeg = 4 KB Not much information Length, width of each area Color of each area

12. Data Compression - Why Bandwidth is a scarce resource, someone still has to pay for it Often important to compress the data at the sender then transmit the compressed form then decompress it at the receiver.bmp is a good format for application programs like “Paint” but it is much better to transmit with the.jpeg file format

13. Two classes of Compression Lossless Data recovered from the compression/decompression process is the same as the original Lossy Some information might be removed by the compression/decompression process

14. Why not always Lossless? Lossy algorithms typically achieve an order of magnitude (10x) better compression than a lossless algorithm 10x makes a big difference in the amount of data that must be transmitted Still images, video and audio are all intended for human eyes or ears – which can tolerate errors and imperfections – because the brain can compensate

15. Lossless Algorithms Run length encoding (RLE) Replaces consecutive occurrences of a symbol with a single symbol and the number of times it occurs (example: AAACCCC is 3A4C) Differential Pulse Code Modulation Records differences from the base symbol Dictionary-Based Each string is replace with its index in a dictionary

16. Lossless Example – Differential Encoding Basic idea is to encode changes. Concept is also used in some lossy algorithms No need to store all the information in each of the following pictures – for the last two just the changes which are much smaller Frame 1 A B C Frame 2 A B C D E F then just store “D E F” for Frame 2 and “add” it to Frame 1 to restore Frame 2

17. Image Compression (JPEG) JPEG = Joint Photographic Experts Group More than a compression algorithm, also defines the format for image or video data JPEG compression takes place in stages Aside first: Fourier transforms and filtering

18. Fourier Transform Consider the following graph It is a weighted sum of 5 sine waves But the coefficients of the higher frequency terms are very small So the entire figure can be approximated well by the low order terms

19. 1 st Order Approximation Only the first term – not a good approximation

20. 2 nd Order Approximation The first two terms give a better approximation

21. 4 th Order Approximation Skipping ahead to 4 terms the approximation is excellent – almost exact

22. 5 th Order Approximation would be Exact Since the original function is a weighted sum of the first 5 sin terms - sin(kt) - the information that uniquely represents the function is the set of coefficients [10;5;2;1;.5] As we saw we could drop the.5 coefficient and retain most of the shape of the curve – hence our information loss would be very slight. A simple example of lossy.

23. Fourier vs. DCT The Discrete Cosine Transform (DCT) is very similar to the Fourier Transform (see pages 544-5 and note that we are using a 2- dimensional transform)

24. Source image JPEG compression DCTQuantizationEncoding Compressed image

26. MPEG A very difficult algorithm! Like JPEG for a single frame, but it has three basic kinds of frames Encoding is very difficult and computationally intensive, hence slow, often done offline Decoding is the only part usually done real time

27. Three Phases Study over the three phases of JPEG DCT Quantization similar to our example of dropping the 5 th coefficient and retaining a graph that was very similar to the original Encoding phase

28. Frame 1Frame 2Frame 3Frame 4Frame 5Frame 6Frame 7 I frameB frame P frameB frame I frame MPEG compression Forward prediction Bidirectional prediction Compressed stream Input stream

29. 16  16 macroblock with Y component 8  8 macroblock with U component 8  8 macroblock with V component 16  pixel region Color frame

30. SeqHdrGroup of picturesSeqHdrGroup of picturesSeqEndCode GOPHdrPicture SlicePictureHdrSlice Macroblock SliceHdrMacroblock MBHdrBlock(0)Block(1)Block(2)Block(3)Block(4)Block(5) … … … …