The State of Television in Our Crazy World What’s Your A/V Quality?

We’re Digital We regained the 700MHz spectrum –which we sold in 2008 to balance the Federal budget. –Now our cell phone coverage is way better Everyone has those digital converter boxes for their TVs Everything is fine, right?

Hmm, Maybe Why doesn’t the picture fill the screen? Why does the picture look blurry & blocky? Sometimes I have no picture at all

What Happened? With every change, there is a challenge As SMPTE members, let’s see we can fix some of these problems

History – Let’s get into our WABAC 1 Machine 1 Wavelength Acceleration Bidirectional Asynchronous Controller

First, are we re-inventing the wheel? Movies until the 1950s were shot in 35mm (4:3) – Movie attendance decreased in the 1950s Introduction of wide-screen (1.67:1, 1.85:1, and 2.35:1) – Movie attendance decreased in the 2000s Introduction of home theater

In the early 80s – most people didn’t notice Just display it But how? – Put borders – Anamorphic scaling – Center cut Who cares? – It’s just an old movie after all

With Home Theaters - Film We Care – Great picture quality – Sound fidelity We Complain – Twitter – Facebook – Blog

What Has Happened to Viewers People want to watch – at home – while mobile – in a theater The experience must be great everywhere

Why doesn’t the picture fill the screen?

Different Devices and Formats How do we display the formats Label the format at the origin so we try to get it right We need a plan

SMPTE Working Group AFD Dr Kerns Powers – Sarnoff –He cut out little squares and tried to fit all of the aspects ratios on a grid 1.33:1 (4:3) – (35mm) 1.67:1 (European Wide) -(16mm) 1.85:1 (American Wide) 2.20:1 (70mm/PanaVision) 2.35:1 (Anamorphic Wide Screen/CinemaScope)

The Discovery 16:9 is the geometric mean – Everything could be converted to it and from it Most new movies are shot in CinemaScope using Center Cut so the action stays in the 1.78:1 middle

SMPTE 2016 – Specification Active Format Description – The AFD is most commonly on line 11 in the Video Index Information (HD recommendation) – Can be on line 9 through VANC-Y (last line) AFDDescription 0ESTI: reserved; ATSC: undefined 1reserved 2ETSI: 16:9 active picture (top aligned); ATSC: "not recommended" 3ETSI: 14:9 active picture (top aligned); ATSC: "not recommended" 4ETSI: box > 16:9 (center): wider than 16:9 active picture. The aspect ratio of the source area is not given, and the size of the top/bottom bars is not indicated. ATSC: bar data (indicating the extent of top, bottom, left, and right bars) should be transmitted when using this code. 5–7reserved 8Full Frame image, same as the frame (4:3 or 16:9). 94:3 Image: Full Frame in 4:3 frame, Pillarbox in 16:9 frame. 1016:9 Image: Letterbox in 4:3 frame, Full Frame in 16:9 frame. 1114:9 Pillarbox/Letterbox image. 12unused 134:3 with shoot and protect 14:9 centre. The term "shoot and protect" is not explained in the standard, but means that the areas above and below the central 14:9 region of the 4:3 active picture can be trimmed without losing important detail. 1416:9 with shoot and protect 14:9 centre. Here, the areas to the right and left of the central 14:9 region of the 16:9 active picture can be trimmed without losing important detail. 1516:9 with shoot and protect 4:3 centre. Here, the areas to the right and left of the central 4:3 region of the 16:9 active picture can be trimmed without losing important detail.

AFD Right 4:3 Source, 16:9 TV, AFD #9 16:9 Source (4:3) Protected, 4:3 TV, AFD #15 16:9 Source, 4:3 TV, AFD #10

4:3 Source to a 16:9 TV

16:9 Source to a 4:3 TV

Next Step Labeling Benefits of labeling the aspect ratio –Save bandwidth Compress the original aspect ratio. Play the others. –Put the station logos and graphic overlays in the right place –Display the image properly Get the up/down conversion correct

What Does this mean AFD Must Be –Tightly coupled with the video asset –Frame accurate –Survive work-flow processing – compression, insertion, transmission, etc. –Updated when scaling AFD Must Not be –In the Active Video Set your VTRs, disk recorders, and frame grabbers properly AFD Can be –Held as meta data per frame

Where does it lead Every pixel may not light up –Black on the top/bottom for 16:9 content shown on a 4:3 TV –Black on the sides for 4:3 content shown on a 16:9 TV Anamorphic Scaling can fix some of this with the correct AFD settings –The user must stop fiddling with the knobs –They will continue to fiddle until the picture automatically does the right thing Set and preserve the AFD!

System Considerations AFD must be carried through the entire workflow –Production –Processing –Distribution Tape decks, Capture devices must –Preserve the VANC data Not call it active video If the AFD is missing, –Pick a safe, repeatable behavior

After All That, Does it Work? NO! –At home, the consumer has too many knobs to twiddle MAYBE! –Set-top boxes (STB or IRD) are becoming AFD aware The Head-End (HE) can set the artistic action This is part of the consumer electronic (CE) specifications There is HOPE!

System Considerations Test the Behavior –Ensure devices maintain AFD –Ensure scalars change AFD correctly –Ensure scalars react properly when AFD is missing Monitor/Test the result –Send out a known A/V sequence –Record the processed A/V sequence –Verify that the results are as expected frame-by-frame

So, We Better Monitor!

Why Monitor? Save Time – Prevent angry tweets, calls Save Money – Prevent the negative press – Ensure advertisement is shown Altruistic – Ensure quality

How Do We Monitor Employ students to watch After a while they look like this Then they make mistakes

Why Automated Monitoring? Saves Time – Finds errors that you may have missed – Logs errors for further analysis Saves Money – The employee can do something more valuable

Problems seen while Monitoring Up/Down scaling problems Loss of A/V quality Loss of A/V sync Encoding, transmitting, decoding problems Loss of ancillary, PSIP Data

Types of Broadcast Monitoring Full Reference No Reference Reduced Reference

Full Reference Monitoring Compares Reference and Processed A/V streams Identifies all errors Problem –Both streams must be co-located –Must be temporally and spatially aligned Usage –Best in a lab or controlled environment

No Reference Evaluates the processed sequence Identifies –Blocking, contouring, blurring –Loss of high frequency –Quantization noise –Packet loss Problem –Wrongly identifies “Artistic Content” Usage –Used when the reference is not present –No need for synchronization

What is Artistic Content? Does anyone remember Jay Leno doing this?

Reduced Reference Compares some extracted information from the reference and processed Identifies most errors Problem –Extracted information must be co-located –Extracted information must be aligned Usage –Best when a back-channel is present and when full reference cannot be used

What is SMPTE up to? Working Group 22TV In-service monitoring –Lip-Sync detection –Audio quality –Video quality Based on finger printing technology –Reduced Reference

What is Finger Printing Extract information from the audio and video separately Resilient to data processing/ compression – Independent of resolution, frame rate, sampling – Independent of logo insertion, graphic overlay, etc. Computed at the reference point when the data is known to be correct Compared at the processing point

What is Finger Printing (continued) Standards-based algorithm which everyone can use Reduced Reference Requires us to go back and finger print old movies and shows

What Finger Printing is NOT! Watermarking – This is carried within the audio/video and can be destroyed by compression Proprietary approach – All vendors must work together with broadcasters A short-term fix – We must go back and finger print old movies

Hope In-service dynamic monitoring –Check A/V quality –Check A/V offset (lip-sync) –Alert on errors Errors not identified –AFD correct – or any other control variables It does check if the picture has been scaled though

It Should See this Error Can you see the error – yes I am playing it slow?

How about if I just show you this Frames 1-4

Conclusion You may not fill the entire screen – Live with it Monitor everything to make sure it works – Find and fix problems before your customers LEAVE SMPTE is working and with your help it can do more

What Does Video Clarity Do? Full Reference Monitoring – We find all errors – Save sequences around every error A/V Quality Analysis – Quantitative measurements – Perceptual measurement – View and hear