视频处理相关技术

Basic Concept Image Sequence （图像序列） FPS （ Frames Per Second ） Resolution 1080p ， 720i ， DVD ， VCD ， HD Compression Codec H.264 、 MPEG Quick Time 、 Real Video 、 WMV Real-time Captured / Video files Live stream media

Frames Single Picture/Still Shot, which is shown as part of a large video or movie Frames are run in succession to produce film or video Pixel Color: 8bits, 10bits, 24bits, … FPS 、 24 、 30 、 、 120

VCD 352x240(NTSC) 352x288 (PAL) DVD 720x480(NTSC) 720x576(PAL) HDTV 720p, 1080p, 1080i, 60fps/50fps

Progressive vs Interlaced 逐行扫描 隔行扫描

Video Compression 640x480, 8 bit YUV, 24fps 7.37M bytes/second 26.5G bytes/hour MPEG-1 187K bytes/second

720p HDTV, 60fps, progressive 1.3Gbits/second HDTV: 20Mb/s bandwidth Compression by a factor of 70

Compression Lossless/Lossy > compression -> decompression -> 3 2 0, high error , low error 5 4 2, lossless

Codecs H.261, H.263, H.264 （视频编码专家组） MPEG-1,2,..,4 （活动图像编码专家组） DivX, XviD,… Real Video QuickTime

Special Requirement Real-time compression -> Real-time decompression Video Meeting Chatting Real-time decompression with High compression rate/ High Quality Movie Film

Fields Theory Decoder/Encoder Development DirectShow / Mplayer / FFMpeg, etc Application using OpenCV

DirectShow A Simple Movie Player A Simple Camera Player

DirectShow Part of DirectX before Direct3D, DirectShow, DirectInput, DirectSound, … Removed to Windows Platform SDK in 2005 COM based Interface based

Framework

DirectShow Filters The basic building block, which can Read files Get video from a video capture device Code/decode streams Pass data to another filter Pass data to the graphics or sound card Input/Output Pins

Filter Graph

Program // Create the filter graph hr = CoCreateInstance(CLSID_FilterGraph, NULL, CLSCTX_INPROC_SERVER, IID_IGraphBuilder, (void **) & g_pGraph); // Create the capture graph builder hr = CoCreateInstance(CLSID_CaptureGraphBuilder2, NULL, CLSCTX_INPROC_SERVER, IID_ICaptureGraphBuilder2, (void **) & g_pCapture); hr = g_pGraph->AddFilter(pSrcFilter, L"Video Capture"); hr = g_pCapture->RenderStream(&PIN_CATEGORY_PREVIEW, &MEDIATYPE_Video, pSrcFilter, pGrabber, pNullRender);

Tools GraphEditPlus Generate Source Code in C++/C# on GraphEdit DirectShow.net C# development

OpenCV Video Files FFMpeg based Cameras DirectShow based

Input Source VideoCapture video(0); VideoCapture video(1); VideoCapture video(“hello.avi”);

Read Frame cv::Mat frame; video.read(frame);

Process Frame Just Like Image Sample Face Detector