High Definition Video: Workflow from Cameras to CODECs APAN Fall Meeting Xi’an, China Aug 2007 Michael Wellings Director, Engineering Vice Chair, APAN HD WG THINK FORWARD. THINK RESEARCHCHANNEL.

Topics of Discussion ResearchChannel Background Recent and future demonstrations Advanced Systems iHDTV™ Open Source Project HD Video Workflow Discussion

ResearchChannel Background

ResearchChannel Programs  More than 3,200 hours of programming in the Video on Demand library  1,100 new programs added this year

Advanced Network Forum AJA Video Systems Inc. Australia's Academic Research Network (AARNet) Fujinon Howard Hughes Medical Institute IBM Corporation Intel Corporation Internet2 Johnson & Johnson Library of Congress Microsoft Research National Academies National Academy of Engineering Duke University George Mason University Indiana University, School of Informatics Johns Hopkins University Massachusetts Institute of Technology National University of Singapore New York University Pennsylvania State University Rice University Rutgers, The State University of New Jersey Stanford University Medical Center Texas A&M University Tulane University, A.B. Freeman School of Business ResearchChannel :: Participants Universidad de Puerto Rico Universidade de São Paulo University of Alaska - Fairbanks University of Chicago University of Hawaii University of Maryland - College Park University of Pennsylvania University of Southern California University of Virginia University of Washington University of Wisconsin-Madison Virginia Tech Yale University National Academy of Sciences National Institute of Nursing Research National Institute of Standards and Technology National Institutes of Health National Library of Medicine National Science Foundation Pacific Northwest Gigapop Poznań Supercomputing and Networking Center R1edu.org SURFnet Vulcan Northwest, Inc. Wisconsin Public Television

ResearchChannel : New Participants American Meteorological Association Arizona State University California State University, Monterey Bay Indiana University Loma Linda University Pacific Northwest National Laboratory University of Kentucky University of Michigan Isilon Systems

U.S. Television Distribution DISH Network satellite system12 million homes Cable systems12.9 million homes

Google Video

Technology Milestones August 1999 First-Ever Streaming of High-Definition Television over the Internet November 1999 Network Sets Record Speed April 2000 First Live HDTV Over Internet Newscast July 2004 First Transmission of Full Bandwidth HD Video Between Computers September 2005 First-Ever Live HD Images from Seafloor to Land Available as IP-Based Feed November 2005 Interactive Multipoint HD Videoconference Demonstrated November 2006 Partner in World-wide testing of ConferenceXP

Technology :: Leading the Way

Continuous Ocean Observing

Consortia Partnerships

Recent and Future Demonstrations

 Sept iGrid San Diego iHDTV - USA118: Global N-Way Uncompressed Interactive Conferencing Underwater research using HD from SS Thompson – USA119: 20,000 Terabits Under the Sea  Nov SC05 Seattle iHDTV /HD Storage, capture and editing using SRB (Storage Resource Broker) Recent Demonstrations

 Nov 2006 SC06 Tampa iHD1500 system  Dec 2006 Internet2 Fall Member Meeting Chicago iHD1500 system HD CoDecs compared HD Camera hands-on Recent Demonstrations

Future Demonstrations  ResearchChannel Annual Meeting Oct07  Cinegrid / GLIF Sept07  Internet 2 Fall Member Meeting Oct07  SC07 Reno Nov07  APAN 08 Hawaii Jan08

Advanced Delivery Methods and Services

ResearchChannel Advanced on-line Services  ResearchChannel-HD Currently VC-1 SMPTE 421M AVC (h.264, MPEG-4 Part-10) coming soon Unicast: hd.cac.washington.edu/ResearchChannel-HDVideohttp://media-wm- hd.cac.washington.edu/ResearchChannel-HDVideo Multicast: HDVideo.nschttp://researchchannel.org/multicast/ResearchChannel- HDVideo.nsc Solicitation for HD content

ResearchChannel Advanced on-line Services  ResearchChannel satellite feed on the network mbps  Live 1080i HD video of Mt Rainier mbps  Live 720p HD video of Mt Rainier  HD VOD 720p WMV-HD  HD VOD 1080i MPEG2  KEXP-FM Audio-on-demand; uncompressed audio webcasting

ResearchChannel Advanced on-line Services  Developing VOD archive for oceanographic HD video Partnership with Isilon – to be demonstrated at SC07 Reno  KEXP-FM on-line services have a world-wide audience that equals Seattle broadcast listener numbers

ResearchChannel Advanced on-line Services Visions ’05 - Emmy Award Nominee First-ever live HD 1080i video/IP streamed around the world SMPTE 292M multicast and 1.5gbps MPEG 2 multicast WMV-HD multicast and unicast

ResearchChannel Advanced on-line Services  Seattle Science Foundation Project – testbed for new Medical technology with HD video archiving – seattlesciencefoundation.org  WWAMI: collaborative medical education via Advanced video technology  RC HD Lab on-line  Advanced venue on UW campus commissioned  Successful launch of Experimental Internet ResearchChannel-HD

iHDTV Open Source Project

iHDTV™ Software Suite  iHDTV Explained iHD1500 iHD270 HD to the Desktop  WindowsXP-based application suite  Released as Open Source  Account created on SourceForge  iHD_Trusted_Partners_Group formed to direct software development

iHDTV™ Development / Deployment  SourceForge load SVN Tree transferred iHDTV_trusted_partners formed – 47 members 4 Core (UW / U-Mich / U- Wisconsin) Executive team to guide development will meet monthly  Deploy globally N-way implementation this summer with 6 nodes

iHDTV™ iHD270  iHD270 (first introduced aug 99) Sony HDCam tm compression SDTI data format 270mbps AJA Xena I/O Requires: AJA Xena-HD capture cards Dual Processor P4 Windows XP Sony HDCam Hardware encoder/decoder Gige network connection

iHDTV™ iHD270  1920x i 8bit 4:2:2 Sony HDCam source material 995mbps active lines, 1188mbps total

iHDTV™ iHD1500  iHD1500 Uncompressed SMPTE 292M 4:2:2 Data rate total approx 1.5 gbps Requires: Uncompressed HD capture cards PCI Express platforms Windows XP 2 x gige or 1x10gige network connection Jumbo frame transport and routing Specs: /Main_Page /Main_Page

iHDTV™ iHD1500 Latency about 4 frames end to end plus network delay 4 frames=133ms or ~ 1/7 sec

iHDTV™ iHD1500  iHD1500 Enhancements: New i/o board options GUI front end Self-installing package Uncompressed HD Server: Single link Dual link Quad link HD Screen rendering for OptIPortal integration NTT i-VISTO interoperability

High Definition Workflow A Report “in progress…”

HD – a Background

Sample Rates  NTSC 4:2:2 Y, Pb, Pr color matrix –4x3.38 (compromise rate from 3.58MHz) = 15.5MHz –Color difference 6.75MHz  HD 4:2:2 (really 22:11:11) Y, Pb, Pr color matrix –22x3.38 = 74.25MHz –Color difference MHz  10 bit – 1024 gradations  8 bit – 256 gradations

Sample Rates  4:2:2 Chroma sampled at ½ rate of Luma  4:1:1 Chroma sampled at ¼ rate of Luma  4:2:0 Chroma sampled at ½ rate of Luma on every other line of each field

Sample / Bit Rates  Broadcast 4:2:0 MPEG-2 TS 19.2mbps  DVD 4:2:0 MPEG-2 PS VBR  HD DVD/Blu-Ray 4:2:0 8mbps 4:2:0 8mbps

HD Resolutions  720p 60Hz – SMPTE 274M 1280x720 24p 1280x720 30p 1280x720 60p  1080i/p 60Hz – SMPTE 296M 1920x p 1920x p 1920x i 1920x p

HD Bitrates – Uncompressed 720p 10- bit 4:2:2  720p 60Hz - SMPTE 296M 1280x720 30p 742mbps 30MHz Y, 15MHz Chroma 1280x720 60p 1485mbps 30MHz Y, 15MHz Chroma

Progressive / Interlaced  1080/60i and 1080/30p occupy the same datarate – 1.485gbps  Interlaced pictures produce spacial errors associated with the time difference in motion between fields  This can create a smoothing effect which is easier on the eye during fast motion  High Framerate progressive (60 frame or greater) is best

HD Bitrates – Uncompressed 10801/p 10- bit 4:2:2  1080i/p 60Hz – SMPTE 274M 1920x p 1485mbps 30MHz Y, 15MHz Chroma 1920x i 1485mbps 30MHz Y, 15MHz Chroma 1920x p 2970mbps 60MHz Y, 30MHz Chroma

Imaging chips  Solid State Camera front end (tube cameras came first!)  CCD = Charge Coupled Device  CMOS = Complementary metal oxide semiconductor

Imaging chips CCDs collect charge at each element, and that charge is read out sequentially Charge to Voltage conversion takes place off-chip A>D takes place off chip Digital level = luminance level These are monochrome sensors!

Imaging chips CMOS chips collect charge at each element Charge to Voltage and A>D functions take place on-chip CMOS imagers exhibit more noise than CCDs, and tend not to be used in higher end cameras

Single chip vs 3 chip  CCDs gather charge according to the amount of light impinging on the element  This charge is converted to a voltage representing light level  This light level does not specifically contain color information  Color reference is obtained through the use of filtering in front of the image sensor

Bayer Filter on-chip  Bayer Filter Diagram  4:2:2 sampling  Green sampled at 2x Red and Blue  Single chip color solution

Color Processing  3 chip RGB output from image processing Digital sampling and CODECS operate on component video – Y, Pb, Pr (Y’,Cb’,Cr’) Y = Luma channel Pb= subsampled blue matrix Pr= Subsampled red matrix

NAB 2007

HD Cameras  SONY XDCam ex 3 chip ½” imager (details unknown) MPEG-2 long GOP 30-50mbps Writes to Express Flash memory cards 120 min record time Target price $8000

HD Cameras  Grass Valley/Thomson Infinity 3 chip CMOS 2.4 million pixels each Ships with DV25 and JPEG2000 codecs MPEG2 optional Writes at up to 100mbps to IOMega REVPro drives Target price $26K without lens Ships in August 07

New HD Cameras at NAB 07  SONY XDCam HD Long GOP MPEG-2 Writes to dual-layer BluRay Pro disk Up to 50mbps 2/3” 3 chip 2million+ pixel ships in 1 st QTR 08 Target price less than $30K without lens

New HD Cameras at NAB 07

Codec comparison Project  SMPTE StEM HD Mini-Movie Designed for critical evaluation of HD equipment as well as encoders and decoders Delivered as tga files, uncompressed Imported into Final Cut Pro

SMPTE Standard Evaluation Material

Codec comparison Project  7 Uncompressed Quicktime Movies on 7 sequences

 5 movies exported: DVCPro 100 HDV MPEG-4 Basic H.264 Part 10 (AVC) HD uncompressed  All FCP I/O via AJA Kona 2 board Codec comparison Project

 HD sequence played out to HDCam Tape recaptured 8bit 4:2:2 in FCP  HD sequenced played out to NTT HD1000 MPEG-2 encoder recaptured from output of NTT HD1000 MPEG-2 decoder as 8bit 4:2:2, and 4:2:0

Codec comparison Project  DVCPro 100 sequence recorded to tape and played back  HDV sequence re-rendered in FCP to uncompressed  HDCAM SR recorded to tape and played back  XDCAM HD recorded to tape and played back

Codec comparison Project  MPEG-4 Basic re-rendered to uncompressed in FCP  MPEG-4 h.264 Part 10 re-rendered to uncompressed in FCP

The edit  Five scenes of interest selected  10 new sequences created for each of the 9 compressed-decompressed files plus the original  HDV and DVCPro were scaled 133 and 150% respectively to match the uncompressed material

Difference mask  Luminance of each compressed file inverted and added to the original  50% luma field indicated no loss  Material lost in codec process exhibited visually due to the subtraction process

The movies  Several HD movies resulted from this project 9 with the edited 5 scenes – each an example of a particular codec – showing the difference mask full screen 9 with the encoded and decoded video overlayed in the center of the frame of the origional for direct comparison 5 complete sequences of codec representation The original uncompressed movie

CODEC Comparison CODEC Bit Rate (Mbps) File Size (MB) Compression RatioTypeSizeColor SpaceEncode Time HDCam144n/a10.4:1Hardware1440X10803:1:16:30 min HDCam SR440 =1500/B3Hardware1920X10804:4:46:30 min DVCProHD :1Software1280x10804:2:217 min XDCam HD35 Hardware 6:30 min MPEG-2 min MPEG-2 4:2:018n/a93:1Hardware1920X10804:2:06:30 min HDV MPEG-2 min MPEG-4 H :1Software1920X10804:2:01 hr 5 min MPEG-4 Basic :1Software1920X10804:2:024 min VC :1Hardware1920X10804:2:028 min JPEG200046/25* Hardware JPEG /75* Hardware JPEG /190* Hardware

equipment  SONY HDW-500 VTR  SONY HDCAM-SR VTR  SONY XDCAM –HD VTR  Panasonic DVCPRO-100  Apple Dual 2.7GHz G5 4GB RAM Mac OS-X Final Cut Pro AJA Kona 2 I/O  NTT HD 1000 MPEG Encoder  NTT HD 1000 MPEG Decoder

HD Multiburst Waveform Uncompressed Reference

HD Multiburst Waveform DVCPro 100

HD Multiburst Waveform SONY HDCam

HD Multiburst Waveform SONY HDCAM SR

HD Multiburst Waveform HDV

HD Multiburst Waveform H.264 /AVC

HD Multiburst Waveform SONY XDCAM HD 30

HD Multiburst Waveform JPEG 2K 25mbps

HD Multiburst Waveform JPEG 2K 75mbps

HD Multiburst Waveform JPEG 2K 250mbps

Seating Dist

Screens  Small screen up close vs. large screen further away  Dot pitch most important factor  Native HD resolution displays make sense for larger (24 to 50 inch) sizes due to optimum viewing distances  Low-res screens *appear* sharper from a distance – this is due to the apparent dot pitch of the digital display

High Definition Workflow Apple FCP EDIT-SAN Architecure

Edit System Software  Apple Final Cut Pro / Final Cut Studio 2.0  OSX 10.4  XSAN 1.4

Edit Suite Hardware Systems  4 MacPro Dual Quad Core Xeon platforms  AJA Kona 3 i/o  3 XSERV  30TB XRAID

Shared storage  27.3TB shared media pool  Incremental backup of projects nightly IBM Tivoli Storage Manager StorageTek Silo archive Source tape is backup for media pool currently  Multi-resolution capable

HD Edit workflow SMPTE 292M HDW-500A PESA 292M HD Router Final Cut Studio edit suites XSAN XRAID Stack

X-SAN Diagram

ResearchChannel HD Migration  SD cameras to be replaced with HD cameras Likely Sony XDCam HD MPEG-2 4:2: mbps IMX High speed file dump mode from Pro Blu-Ray dual layer disc  4:3 safe zone shooting  HD capture and edit as uncompressed HD or IMX  Can output anamorphic, 4:3, 16:9 letterbox for SD  HD project archive

Edit Room Layout

XSERVs in Master Control

XRAID

XSAN

Studio M

Credits NAB 2007 reports: Noah Pitzer Jamie Alls Jabran Soubeih Dave Robertson Jack Hoffman Special thanks to: Staff engineer Jabran Soubeih for CoDec measurements Tektronix Guide to Standard and High Definition Digital Video Measurements – dexes/video_audio/registration/

For more information  