Distributed Video System realized on mobile device with efficient Feedback channel 分散式影像編碼在手機上的實現與有效率 的回饋通道 1 Chen,chun-yuan 陳群元 Advisor:Prof. Wu,Ja-Ling.

Slides:



Advertisements
Similar presentations
Transform-domain Wyner-Ziv Codec for Video 教師 : 楊士萱 老師 學生 : 李桐照 同學.
Advertisements

ERROR-DETECTING AND ERROR- CORRECTING CODES ․ A simple scheme for detecting errors is to append a parity check digit to each block of message. Even parity:
Pricing and Power Control in a Multicell Wireless Data Network Po Yu Chen October, 2001 IEEE Journal on Select Areas in Communications.
Teacher : Ing-Jer Huang TA : Chien-Hung Chen 2015/6/3 Course Embedded Systems : Principles and Implementations Weekly Preview Question CH3.5 ~ CH /10/31.
Limin Liu, Member, IEEE Zhen Li, Member, IEEE Edward J. Delp, Fellow, IEEE CSVT 2009.
期末專題 - 吊人頭遊戲 第 35 組 組員 : 電機系 49841XXXX XXX 電機系 49841OOOO OOO.
Speaker: Pei-Ni Tsai Adviser: Dr. Kai-Wei Ke 2011/7/27.
Sang-Chun Han Hwangjun Song Jun Heo International Conference on Intelligent Hiding and Multimedia Signal Processing (IIH-MSP), Feb, /05 Feb 2009.
Chapter 3 Growth of Functions Asymptotic notation Θ-notation: f(n) = Θ(g(n)) , g(n) is an asymptotically tight bound for f(n) 。 Θ(g(n)) = {f(n)|
STAT0_sampling Random Sampling  母體: Finite population & Infinity population  由一大小為 N 的有限母體中抽出一樣本數為 n 的樣 本,若每一樣本被抽出的機率是一樣的,這樣本稱 為隨機樣本 (random sample)
: The Playboy Chimp ★★☆☆☆ 題組: Problem Set Archive with Online Judge 題號: 10611: The Playboy Chimp 解題者:蔡昇宇 解題日期: 2010 年 2 月 28 日 題意:給一已排序的數列 S( 升冪.
Monte Carlo Simulation Part.2 Metropolis Algorithm Dept. Phys. Tunghai Univ. Numerical Methods C. T. Shih.
Reinventing Compression: The New Paradigm of Distributed Video Coding Bernd Girod Information Systems Laboratory Stanford University.
2009fallStat_samplec.i.1 Chap10 Sampling distribution (review) 樣本必須是隨機樣本 (random sample) ,才能代表母體 Sample mean 是一隨機變數,隨著每一次抽出來的 樣本值不同,它的值也不同,但會有規律性 為了要知道估計的精確性,必需要知道樣本平均數.
信度.
Robust Audio Tool (RAT) Speaker : Wei-Shin Pan DATE : 09/07/02.
:Problem D: Bit-wise Sequence ★★★☆☆ 題組: Problem Set Archive with Online Judge 題號: 10232: Problem D: Bit-wise Sequence 解題者:李濟宇 解題日期: 2006 年 4 月 16.
微帶線濾波器 國立聯合大學 電機工程學系 主講人 : 徐振剛 何奕叡. 目 錄  設計程序理論說明  1. 微波諧振電路  2. 傳輸線特性  3.Chebyshev filter & Butterworth filter  4. 傳輸線殘段設計濾波器和步階阻抗式低通濾波器  設計模型.
: Happy Number ★ ? 題組: Problem Set Archive with Online Judge 題號: 10591: Happy Number 解題者:陳瀅文 解題日期: 2006 年 6 月 6 日 題意:判斷一個正整數 N 是否為 Happy Number.
Chapter 8 消費可能性 偏好 選擇 Part 3 家庭的選擇
各種線上電子資源的特異功能 SpringerLINK 的 Alert, Serials Update, News 2003/4/28 修改.
: Problem A : MiniMice ★★★★☆ 題組: Contest Archive with Online Judge 題號: 11411: Problem A : MiniMice 解題者:李重儀 解題日期: 2008 年 9 月 3 日 題意:簡單的說,題目中每一隻老鼠有一個編號.
Distributed Video Coding Bernd Girod, Anne Margot Aagon and Shantanu Rane, Proceedings of IEEE, Jan, 2005 Presented by Peter.
: Count DePrimes ★★★★☆ 題組: Contest Archive with Online Judge 題號: 11408: Count DePrimes 解題者:李育賢 解題日期: 2008 年 9 月 2 日 題意: 題目會給你二個數字 a,b( 2 ≦ a ≦ 5,000,000,a.
Distributed Video Coding. Outline Distributed video coding Lossless compression Lossy compression Low complexity video encoding Distributed image coding.
短缺,盈餘與均衡. 遊戲規則  老師想出售一些學生喜歡的小食。  老師首先講出價錢,有興趣買的請舉手。
Wyner-Ziv Coding of Motion Video
:Nuts for nuts..Nuts for nuts.. ★★★★☆ 題組: Problem Set Archive with Online Judge 題號: 10944:Nuts for nuts.. 解題者:楊家豪 解題日期: 2006 年 2 月 題意: 給定兩個正整數 x,y.
資料結構實習-一 參數傳遞.
資料結構實習-二.
演算法 8-1 最大數及最小數找法 8-2 排序 8-3 二元搜尋法.
Learning Method in Multilingual Speech Recognition Author : Hui Lin, Li Deng, Jasha Droppo Professor: 陳嘉平 Reporter: 許峰閤.
Chapter 10 m-way 搜尋樹與B-Tree
Wyner-Ziv Residual Coding of Video Anne Aaron, David Varodayan and Bernd Girod Information Systems Laboratory Stanford University.
論文研討 2 學分 授課教師:吳俊概. 第一節 論文發表的目的 第二節 論文發表的歷程 第三節 投稿過程 第四節 退稿處理 學術期刊論文的製作與發表.
1 Department of Electrical Engineering Stanford University Anne Aaron, Shantanu Rane and Bernd Girod Wyner-Ziv Video Coding with Hash-Based Motion Compensation.
: Problem E Antimatter Ray Clearcutting ★★★★☆ 題組: Problem Set Archive with Online Judge 題號: 11008: Problem E Antimatter Ray Clearcutting 解題者:林王智瑞.
Teacher : Ing-Jer Huang TA : Chien-Hung Chen 2015/6/30 Course Embedded Systems : Principles and Implementations Weekly Preview Question CH7.1~CH /12/26.
Distribute Video Coding 林明德. Lossless Compression 不考慮 X 和 Y 的相關性,直接傳送 X 和 Y 各需要使用 3bits ,總共傳出 的 bit 數為 6bits 。 (1) 考慮 X 和 Y 的相關性 (2) 將 Y 直接傳出,使用 3bits.
RTP Real-Time Transport Protocol Speaker: Hsiao-Ting Wang Advisor: Quincy Wu Date: July 2 nd 2009.
: Place the Guards ★★★☆☆ 題組: Problem Set Archive with Online Judge 題號: 11080: Place the Guards 解題者:陳盈村 解題日期: 2008 年 3 月 26 日 題意:有一個國王希望在他的城市裡佈置守衛,
1 Department of Electrical Engineering, Stanford University Anne Aaron, Shantanu Rane, Eric Setton and Bernd Girod Transform-domain Wyner-Ziv Codec for.
: Finding Paths in Grid ★★★★☆ 題組: Contest Archive with Online Judge 題號: 11486: Finding Paths in Grid 解題者:李重儀 解題日期: 2008 年 10 月 14 日 題意:給一個 7 個 column.
Distributed Video Coding Bernd Girod, Anne Margot Aaron, Shantanu Rane, and David Rebollo-Monedero IEEE Proceedings 2005.
Distributed Video Coding VLBV, Sardinia, September 16, 2005 Bernd Girod Information Systems Laboratory Stanford University.
Arko Barman Computer Vision & Artificial Intelligence Lab Department of Electrical Engineering Indian Institute of Science, Bangalore.
Distributed Video System realized on mobile device with efficient Feedback channel 分散式影像編碼在手機上的實現與有效率 的回饋通道 1 Chen, chun-yuan 陳群元 Advisor: Prof. Wu, Ja-Ling.
High Efficient Distributed Video Coding with Parallelized Design for Cloud Computing 適用於雲端架構下兼具高效能與平行化設計之 分散式視訊編碼 CMLab, CSIE, NTU 1 Cheng, Han-Ping 程瀚平.
Streaming Way to bring you lived.. What is Streaming technology.
Abhik Majumdar, Rohit Puri, Kannan Ramchandran, and Jim Chou /24 1 Distributed Video Coding and Its Application Presented by Lei Sun.
ROBUST VIDEO STABILIZATION BASED ON PARTICLE FILTER TRACKING OF PROJECTED CAMERA MOTION IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY,
Statistical Characteristics of Simple Wyner-Ziv Frames Jin-soo KIM.
Progressive Side Information Refinement with Non-Local Means Denoising in Distributed Video Coding 使用於分散式視訊編碼之非區域平均去雜訊循 序旁資訊改善技術 Wang, Pin-Hsiang 王品翔 Advisor:
Sub pixel motion estimation for Wyner-Ziv side information generation Subrahmanya M V (Under the guidance of Dr. Rao and Dr.Jin-soo Kim)
Rate-distortion Optimized Mode Selection Based on Multi-channel Realizations Markus Gärtner Davide Bertozzi Classroom Presentation 13 th March 2001.
Brief Overview of Wyner-Ziv CODEC and Research Plan Jin-soo KIM.
Speed up in feedback channel for a LDPCA base distributed video coding system on mobile device 在手機裝置上對低密度奇偶校驗碼為 基礎之分散式編碼中的回饋通道加速 Chen,chun-yuan 陳群元 Advisor:
Distributed Video System realized on mobile device with efficient Feedback channel 分散式影像編碼在手機上的實現與有效率 的回饋通道 1 Chen, chun-yuan 陳群元 Advisor: Prof. Wu, Ja-Ling.
Wyner-Ziv Coding of Motion Video Presented by fakewen.
C.K. Kim, D.Y. Suh, J. Park, B. Jeon ha 強壯 !. DVC bitstream reorganiser.
1 Department of Electrical Engineering, Stanford University Anne Aaron, Shantanu Rane, Rui Zhang and Bernd Girod Wyner-Ziv Coding for Video: Applications.
1 Department of Electrical Engineering, Stanford University EE 392J Final Project Presentation Shantanu Rane Hash-Aided Motion Estimation & Rate Control.
Group 2 Block 胡貴蓉 Project 2 JMVC code tracing.
Distributed Video System realized on mobile device with efficient Feedback channel 分散式影像編碼在手機上的實現與有效率 的回饋通道 1 Chen,chun-yuan 陳群元 Advisor:Prof. Wu,Ja-Ling.
BITS Pilani Pilani Campus EEE G612 Coding Theory and Practice SONU BALIYAN 2017H P.
JPEG implementation and improvement
Progress & schedule Presenter : YY Date : 2014/10/3.
基於邊緣吻合向量量化編碼 法之資訊隱藏 張 真 誠 逢甲大學 講座教授 中正大學 榮譽教授、合聘教授 清華大學 合聘教授
Kyoungwoo Lee, Minyoung Kim, Nikil Dutt, and Nalini Venkatasubramanian
強壯的進度 2011/12/28 我是強壯XD.
Presentation transcript:

Distributed Video System realized on mobile device with efficient Feedback channel 分散式影像編碼在手機上的實現與有效率 的回饋通道 1 Chen,chun-yuan 陳群元 Advisor:Prof. Wu,Ja-Ling 吳家麟 2012/6/6

Outline  Motivation and introduction  DVC architecture with mobile device  Propose an efficient feedback channel  Transcoding  Experiment Result  Conclusion  Future work 2

Motivation  Video communication is essential in Mobile device  Decrease the coding time and battery consuming on mobile. 3

Motivation 4

Introduction  For Mobile device video codec  Conventional video coding (ex:MPEG-4 H.264)  Heavy weight encoder, light weight decoder.  Distributed Video Coding  Light weight encoder, heavy weight decoder. 5

Conventional video codec 6

DVC to H.264 Transcoder 7 Clouding server: DVC to H.264 transcoder DVC encoder H.264 decoder

Channel Coding R Y =H (Y) Source encoder Correlation is exploited by motion estimation R Y =H (X|Y) Source encoder H (X|Y)H (Y) Distributed  Slepian-Wolf Theorem (1973) Wyner-Ziv Theorem (1976) Channel encoder Source decoder Side Information creation Channel decoder X Y Virtual channel H (X, Y)

Distributed Video Codec 9 DCT WZ Frames LDPCA Encoder Buffer LDPCA Decoder Key Frames Conventional Intraframe Encoder Conventional Intraframe Encoder SI generation& refinement DCT SI Reconstruction Mode selection CRC CRC check CRC-8 WZ bitstream Correlation Noise Modeling IDCT& Deblocking filter Decoded WZ Frames Decoded Key Frames

 Our DVC codec:  DVC transcoder 10

DVC to H.264 Transcoder 11 Clouding server: DVC to H.264 transcoder DVC encoder H.264 decoder

DVC Decoding Process  DVC Decoding without Feedback channel  Side-Information generation  LDPCA 12

DVC Decoding Process  DVC Decoding  Side-Information generation  Send-Request at Feedback channel  LDPCA 13

Outline  Motivation and introduction  DVC architecture with mobile device  Propose an efficient feedback channel  Transcoding  Experiment Result  Conclusion  Future work 14

DVC Encoder Realization  Implement DVC encoder on Mobile  Key frame encoding  WZ frame encoding  Efficient intra mode realization  Feedback channel transmission  Network connection between mobile and server  Determine payload of a packet with LDPCA protocal 15

Key frame encoding on Mobile  Previous ver. use JM9.5  Transcoder ref x.264 on PC 16

DVC Encoder Realization  Implement DVC encoder on Mobile  Key frame encoding  WZ frame encoding  Efficient intra mode realization  Feedback channel transmission  Network connection between mobile and server  Determine payload of a packet with LDPCA protocal 17

WZ coding with NDK 18

DVC Encoder Realization  Implement DVC encoder on Mobile  Key frame encoding  WZ frame encoding  Efficient intra mode realization  Feedback channel transmission  Network connection between mobile and server  Determine payload of a packet with LDPCA protocal 19

Efficient intra block coding  Previous ver. Transcoder Use JM code.  Intra block encoding consumes lots of time in encoding. 20 Clouding server: DVC to H.264 transcoder

DVC Encoder Realization  Implement DVC encoder on Mobile  Key frame encoding  WZ frame encoding  Efficient intra mode realization  Feedback channel transmission  Network connection between mobile and server  Determine payload of a packet with LDPCA protocal 21

Network connection  Create network connection to connect remote server.  TCP socket in C 22 Clouding server: DVC to H.264 transcoder

DVC Encoder Realization  Implement DVC encoder on Mobile  Key frame encoding  WZ frame encoding  Efficient intra mode realization  Feedback channel transmission  Network connection between mobile and server  Determine payload of a packet with LDPCA protocal 23

ACK at Feedback channel 24 … syndromes

Time consuming at Feedback channel  Packet header occupy large part of network bandwidth.  Network latency in communication at Feedback channel 25

Outline  Motivation and introduction  DVC architecture with mobile device  Propose an efficient feedback channel  Transcoding  Experiment Result  Conclusion  Future work 26

Efficient feedback channel  we propose two methods to decrease time consuming at feedback channel.  Estimate the syndromes size per WZ frame  Estimate the syndromes size per bitplane 27

Number of requests per bitplane per band DC AC1AC2 AC3 AC4AC5 AC6AC7AC8 AC9AC10 AC11 AC12 AC13 AC14 Bitplane Number

Estimate the syndromes size per WZ frame  Estimate syndromes size by correspond WZ frame in previous GOP  Ex. GOP 4 29 Key frame Key frame WZ frame WZ frame WZ frame First GOP Key frame Key frame WZ frame WZ frame WZ frame Second GOP

formula  (ES bt ) WZn =(S bt ) WZ(n-GOPsize)  WZn means the WZ frame’s index  n-GOPsize means the correspond WZ frame in the previous GOP.  bt is the bitplane index.  (ES bt ) WZn means the estimated syndromes bit for the bt’th bitplane of WZn frame.  (S bt ) WZ(n-GOPsize) means the syndromes

Efficient feedback channel  we propose two methods to decrease time consuming at feedback channel.  Estimate the syndromes size per WZ frame  Estimate the syndromes size per bitplane 31

Number of requests per bitplane per band DC AC1AC2 AC3 AC4AC5 AC6AC7AC8 AC9AC10 AC11 AC12 AC13 AC14 Bitplane Number

Number of requests per bitplane per band DC AC1AC2 AC3 AC4AC5 AC6AC7AC8 AC9AC10 AC11 AC12 AC13 AC14 Bitplane Number

Estimate by previous bands  1.formula  2.time ratio

Number of requests per bitplane per band DC AC1AC2 AC3 AC4AC5 AC6AC7AC8 AC9AC10 AC11 AC12 AC13 AC14 Bitplane Number

formula  (ES acn bt ) WZn = ( S dc bt-1 ) WZn  WZn means the n’th WZ frame.  acn means the n’th AC band.  bt means the bitplane index of this AC band.  Which ES acn bt mean the estimated syndrome size for the bt’th bitplane in n’th AC band.  dc means the DC band.  S dc bt-1 means the syndromes bits for bt-1’th bitplane in DC band.

Outline  Motivation and introduction  DVC architecture with mobile device  Propose an efficient feedback channel  Transcoding  Experiment Result  Conclusion  Future work 37

Transcode to mobile codec  Because DVC only encode luminance part in YUV, so we got only Y for output.  Our mobile device can only read mp4 and 3gp. Y UV R G B Frame 1 Frame 2 Frame 3 Frame4 Frame 5 Frame 6 Frame 7 Frame n.mp4

Outline  Motivation and introduction  DVC architecture with mobile device  Propose an efficient feedback channel  Transcoding  Experiment Result  Conclusion  Future work 39

Experiment Result  規格  實驗數據 :  時間 (encode&decode) 、 packet 數量 ( 傳幾次 )+  RD  使用 x264 對 quality 的影響 (intra 改 quality 應該 會變,但變快 ) 40

Outline  Motivation and introduction  DVC architecture with mobile device  Propose an efficient feedback channel  Transcoding  Experiment Result  Conclusion  Future work 41

Contribution  Realize DVC encoding part on Mobile device  Implement Feedback channel  Implement encoder’s component with x264  Propose an efficient Feedback channel  Transcode the decoded sequence to mobile codec 42

Outline  Motivation and introduction  DVC architecture with mobile device  Propose an efficient feedback channel  Transcoding  Experiment Result  Conclusion  Future work 43

Future Work  Take a more precise estimation of syndrome size.  Use motion information to estimate the syndrome size.  Build a reliable DVC codec which is loss tolerate 44

45 Thank You

To do  Real time decoder  Efficient intra coding  Print MSG on Mobile 46

DVC decoding time  Side Info Creation: 0.85 sec  Overcomplete Transform: 2.35 sec  Correlation Noise Modeling: 2.33 sec  Conditional Bit Prob Compute: 0.60 sec  Update Side Info: 7.14 sec  Ldpca Decode: sec  Motion Learning: 1.79 sec  Others: 1.80 sec 47

Distributed Video Coding D. Varodayan, A. Aaron, and B. Girod, “Rate-Adaptive Codes for Distributed Source Coding,”EURASIP Signal Processing Journal, Special Issue on Distributed Source Coding,,November Channel Encoder Channel Decoder LDPC Encoder LDPC Decoder

Distributed Video Codec 49 DCT WZ Frames Extract bit-plane bit-plane 1 LDPCA Encoder Buffer bit-plane 2 LDPCA Decoder Request bits Key Frames Conventional Intraframe Encoder Conventional Intraframe Encoder Interpolation/ Extrapolation DCT Reconstruction

Distributed Video Codec 50 DCT WZ Frames LDPCA Encoder Buffer LDPCA Decoder Request bits Key Frames Conventional Intraframe Encoder Conventional Intraframe Encoder Interpolation/ Extrapolation DCT Reconstruction

Distributed Video Codec 51 DCT WZ Frames LDPCA Encoder Buffer LDPCA Decoder Key Frames Conventional Intraframe Encoder Conventional Intraframe Encoder SI generation& refinement DCT SI Reconstruction Mode selection CRC CRC check CRC-8 WZ bitstream Correlation Noise Modeling IDCT& Deblocking filter Decoded WZ Frames Decoded Key Frames

52

我最少需 要 48bits 的 parity bits 1584bits LdpcaEncode 我還需要 更多的 parity Bits… 我解完了 一個 bitplane 了 ! Buffer 依序解完 所有 bitplanes 24

DVC to H.264 Transcoder 54 Clouding server: DVC to H.264 transcoder DVC encoder H.264 decoder

 接著講 header overhead  每次都傳一點,等於傳的大部分都是 header(20Bytes)  傳輸太多次,會造成大量的 delay 55

Outline  Motivation and introduction  DVC architecture with mobile device  Propose an efficient feedback channel  Transcoding  Experiment Result  Conclusion  Future work 56