1. SAMPLE ADAPTIVE OFFSET IN THE HEVC STANDARD
HARSHA NAGATHIHALLI JAGADISH ( )

2. Acronyms
AVC: Advanced Video Coding BS: Boundary Strength CODEC: COder/DECoder Croma: Chrominance CTU: Coding Tree Unit CU: Coding Unit DCT: Discrete Cosine Transform DFT: Discrete Fourier Transform HEVC: High Efficiency Video Coding ITU-T: International Telecommunication Union (Telecommunication Standardization Sector) IEC: International Electrotechnical Commission ISO: International Standards Organization JBIG: Joint Bi-level Image Experts Group JPEG: Joint photographic experts group

3. JCT-VC: Joint collaborative team on video coding
LOT: Lapped Orthogonal Transform
Luma: Luminance
MB: Macro Block
MPEG: Moving picture experts group
OBMC: Overlapped Block Motion Compensation
PU: Prediction Unit
QP: Quantization Parameter
SAO: Sample Adaptive Offset
TU: Transform Unit

4. Introduction
The HEVC standard specifies two in-loop filters, a deblocking filter and a sample adaptive offset (SAO).
Since the deblocking and SAO attenuate different artifacts, their benefits are additive when used together.
SAO is a process that modifies the decoded samples by conditionally adding an offset value to each sample after the application of the deblocking filter, based on values in look-up tables transmitted by the encoder.

5. Motivation
A larger transform could introduce more artifacts including ringing artifacts that mainly come from quantization errors of transform coefficients[2]
A higher number of interpolation taps can also lead to more serious ringing artifacts. Hence, it is necessary to incorporate SAO technique in HEVC.
SAO not only is useful in HEVC but also can be applied on top of AVC and other prior video coding standards.

6. Evolution In Windows Media Video 9 [3]
deringing filter was applied for reducing ringing artifacts
samples are classified into two categories: edge and nonedge
WMV9 is a post-processing technique, which might lead to serious flickering artifacts from picture to picture.
The features of the SAO in JCTVC-E049 [4]
Step 1 :it allows local adaptation
Step 2 : sequential stages are combined into one stage by selecting only one classifier per region to reduce the encoding latency.
Step 3: 2-D edge classification patterns are removed, and only four 1-D edge classification patterns are used.
Step 4: Fast distortion estimation

7. Brief Overview Sample Adaptive Offset (SAO)Filter
Calculates edge and band offsets signaled to decoder
Offsets added to reconstructed pixels
SAO is not restricted to block boundaries
[2]

8. HEVC Encoder
In-loop filters
Fig.2 HEVC Encoder Diagram
[1]

9. Sample Processing
Two SAO types that can satisfy the requirements of low complexity are adopted in HEVC:
Edge offset (EO) and
Band offset (BO).
For EO , the sample classification is based on comparison between current samples and neighboring samples.
For BO, the sample classification is based on sample values.
To reduce side information, multiple CTUs can be merged together to share SAO parameters.

10. a)Edge offset mode
In the edge offset mode, EO uses four 1-D directional patterns for sample classification:
horizontal, vertical, 135° diagonal, and 45° diagonal as shown in fig.3
[5] 3

11. Contd..
For a given EO class, each sample inside the CTB is classified into one of five categories
The current sample value, labeled as “c,” is compared with its two neighbors along the selected 1-D pattern.
Categories 1 and 4 are associated with a local valley and a local peak along the selected 1-D pattern, respectively.
Categories 2 and 3 are associated with concave and convex corners along the selected 1-D pattern, respectively.

12. Contd..
The meanings of edge offset signs are illustrated 4

13. Contd..
Gibbs phenomenon can be used to simulate a few video compression artifacts, especially the ringing artifacts.
The dotted curve -original samples
The solid curve - reconstructed samples by discarding high frequencies of the original samples

14. b) Band offset mode In the band offset mode :
The selected offset value directly depends on the sample amplitude.
The full sample amplitude range is uniformly split into 32 segments called bands.
The sample values belonging to four of these bands (which are consecutive within the 32 bands) are modified by adding transmitted values.

15. For example 8-bit samples ranging from 0 to 255, the width of a band is 8, and sample values from 8k to 8k + 7 belong to band k.
The main reason for using four consecutive bands is that in the smooth areas artifacts can appear. Another reason is the number of bands in BO should be equal to the number of signaled offsets in EO
Fig 6

16. Example of BO
The dotted curve is the original samples, while the solid curve is the reconstructed samples corrupted by quantization errors and phase shifts due to coded motion vectors deviating from the true motions.
In this example, the reconstructed samples are shifted to the left of the original samples, which systematically results in negative errors that can be corrected by BO for bands k, k + 1, k + 2, and k + 3. 7

17. Implementation
A. Fast Edge Offset Sample Classification
A fast algorithm can be described in the following equations: [5]
sign3(x) = (x > 0)?(+1) : ((x == 0)?(0) : (−1)) ……………..(1)
edgeIdx = 2 + sign3(c − a) + sign3(c − b) …………….(2)
edgeIdx2category[] = {1, 2, 0, 3, 4} ….………….(3)
category = edgeIdx2category[edgeIdx] ………………(4)
Also, data reuse between samples can be further applied for the next sample classification.
For example, assuming the EO class is 0 (i.e., using 1-D horizontal pattern) and the samples in the CTB are processed in the raster scan order, the sign3(c–a) of the current sample does not have to be calculated and can be directly set to the −sign3(c–b) of the neighboring sample to the left.
Likewise, the sign3(c–b) of the current sample can be reused by the neighboring sample to the right.

18. B. Fast Band Offset Sample Classification
The sample range is equally divided into 32 bands in BO.
Since 32 =2^5, the BO sample classification can be implemented as using the five most significant bits of each sample as the classification result.
In this way, the complexity of BO becomes very low, especially in hardware that only needs wire connections without logic gates to obtain the classification result from the sample value

19. C. Fast Distortion Estimation
To reduce the memory access and operations, a fast distortion estimation method [6] can be implemented as follows
Let k= sample positions, s(k)= original samples, and x(k)= pre-SAO samples.
where k belongs to C and C is the set of samples that are inside a CTB and belong to a specified SAO type (i.e., BO or EO), a specified starting band position or EO class, and a specified band or category.
The distortion between original samples and pre- SAO samples can be described in the following equation:
The distortion between original samples and post-SAO samples
In (6), h is the offset for the sample set.

20. The delta distortion is defined in the following equation
In (7), N is the number of samples in the set, and E is the sum of differences between original samples and pre-SAO samples .It is as defined in the following equation:
Next, the delta rate-distortion cost is defined in the following equation:
In (9), λ is the Lagrange multiplier, and R represents the estimated number of bits of side information.
For a given CTB with a specified SAO type (i.e., BO or EO), a specified starting band position or EO class, and a specified band or category, a few h values (i.e., offsets) close to the value of E/N are tested, and the offset that minimizes ΔJ will be chosen.

21. Test Sequences [27] BasketballPass_416x240_50.yuv
RaceHorses_832x480_30.yuv
RaceHorses_416x240_30.yuv
SlideEditing_1280x720_30

22. System Configuration Operating System : Windows 10 (64 bit)
RAM : 16 Giga Bytes
Processor: Intel Core 2.60GHz

23. Results 1) SlideEditing_1280x720_30 sequences frame 100 [27]
(For 150 frames)
Fig. 8a Original
Fig.8b SAO=0
Fig.8c SAO=1

24. Metrics used PSNR=20 log 10 MAXf 𝑀𝑆𝐸 The formula used for Y’CbCr
MAXf is the maximum value of the sample.
The formula used for Y’CbCr
The format used is 4:2:0

25. b) For Random Access Mode
a) For Low Delay
QP=32
Total no.of Frames
Bit-Rate(kbps)
Y-PSNR (dB)
U-PSNR
(dB)
V-PSNR
YUV-PSNR
Time (s)
SAO=0
150
SAO=1
38.68
b) For Random Access Mode
QP=32
Total no.of Frames
Bit-Rate
(kbps)
Y-PSNR
(dB)
U-PSNR
V-PSNR
YUV-PSNR
Time (s)
SAO=0
150
SAO=1
38.93
c) For Intra Mode
QP=32
Total no.of Frames
Bit-Rate(Kbps)
Y-PSNR
(dB)
U-PSNR
V-PSNR
YUV-PSNR
Time (s)
SAO=0
150
SAO=1
SAO=1 is SAO filter ON
SAO=0 is SAO filter OFF

26. 2) BasketballPass_416x240_50.yuv sequences frame 14
(Performed for 250 Frames)
Fig. 9a Original Image
Fig. 9b SAO=0
Fig.9c SAO=1
SAO=1 is SAO filter ON
SAO=0 is SAO filter OFF

27. a)Random access QP=16 Total no.of Frames Bit-Rate(kbps) Y-PSNR (dB)
U-PSNR
V-PSNR
YUV-PSNR
Time(s)
SAO=0
250
SAO=1
QP=32
Total no.of Frames
Bit-Rate(kbps)
Y-PSNR
(dB)
U-PSNR
V-PSNR
YUV-PSNR
Time(s)
SAO=0
250
SAO=1
QP=50
Total no.of Frames
Bit-Rate(kbps)
Y-PSNR
(dB)
U-PSNR
V-PSNR
YUV-PSNR
Time(s)
SAO=0
250
SAO=1
SAO=1 is SAO filter ON
SAO=0 is SAO filter OFF

28. b)Intra mode QP=16 Total no.of Frames Bit-Rate(kbps) Y-PSNR (dB)
U-PSNR
V-PSNR
YUV PSNR
Time(s)
SAO=0
250
SAO=1
QP=32
Total no.of Frames
Bit-Rate(kbps)
Y-PSNR
(dB)
U-PSNR
V-PSNR
YUV-PSNR
Time(s)
SAO=0
250
SAO=1
QP=50
Total no.of Frames
Bit-Rate(kbps)
Y-PSNR
(dB)
U-PSNR
V-PSNR
YUV-PSNR
Time(s)
SAO=0
250
SAO=1
SAO=1 is SAO filter ON
SAO=0 is SAO filter OFF

29. c) Low Delay QP=16 Total no.of Frames Bit-Rate(kbps) Y-PSNR (dB)
U-PSNR
V-PSNR
YUV-PSNR
Time(s)
SAO=0
250
SAO=1
QP=32
Total no.of Frames
Bit-Rate(kbps)
Y-PSNR
(dB)
U-PSNR
V-PSNR
YUV-PSNR
Time(s)
SAO=0
250
SAO=1
QP=50
Total no.of Frames
Bit-Rate(kbps)
Y-PSNR
(dB)
U-PSNR
V-PSNR
YUV-PSNR
Time(s)
SAO=0
250
SAO=1
SAO=1 is SAO filter ON
SAO=0 is SAO filter OFF

30. 3) RaceHorses_832x480_30.yuv sequences frame 1
Fig.10a Original Image

31. Fig.10b SAO=0
SAO=1 is SAO filter ON
SAO=0 is SAO filter OFF

32. Fig.10c SAO=1
SAO=1 is SAO filter ON
SAO=0 is SAO filter OFF

33. a)Random Access (i) Resolution 416x240_30 QP=32 Total no.of Frames
Bit-Rate(kbps)
Y-PSNR
(dB)
U-PSNR
V-PSNR
YUV-PSNR
Time(s)
SAO=0
150
SAO=1
(ii) Resolution 832x480_30
QP=32
Total no.of Frames
Bit-Rate(kbps)
Y-PSNR
(dB)
U-PSNR
V-PSNR
YUV-PSNR
Time(s)
SAO=0
150
SAO=1
SAO=1 is SAO filter ON
SAO=0 is SAO filter OFF

34. b)Intra Mode (i) Resolution 416x240_30 QP=32 Total no.of Frames
Bit-Rate(kbps)
Y-PSNR
(dB)
U-PSNR
V-PSNR
YUV-PSNR
Time(s)
SAO=0
150
SAO=1
(ii) Resolution 832x480_30
QP=32
Total no.of Frames
Bit-Rate(kbps)
Y-PSNR
(dB)
U-PSNR
V-PSNR
YUV-PSNR
Time(s)
SAO=0
150
SAO=1
SAO=1 is SAO filter ON
SAO=0 is SAO filter OFF

35. c) Low Delay (i) Resolution 416x240_30 QP=32 Total no.of Frames
Bit-Rate(kbps)
Y-PSNR
(dB)
U-PSNR
V-PSNR
YUV-PSNR
Time(s)
SAO=0
150
SAO=1
(ii) Resolution 832x480_30
QP=32
Total no.of Frames
Bit-Rate(kbps)
Y-PSNR
(dB)
U-PSNR
V-PSNR
YUV-PSNR
Time(s)
SAO=0
150
SAO=1
SAO=1 is SAO filter ON
SAO=0 is SAO filter OFF

36. SAO Filter Code [28]
Void TComSampleAdaptiveOffset::invertQuantOffsets(ComponentID compIdx, Int typeIdc, Int typeAuxInfo, Int* dstOffsets, Int* srcOffsets) { Int codedOffset[MAX_NUM_SAO_CLASSES]; ::memcpy(codedOffset, srcOffsets, sizeof(Int)*MAX_NUM_SAO_CLASSES); ::memset(dstOffsets, 0, sizeof(Int)*MAX_NUM_SAO_CLASSES); if(typeIdc == SAO_TYPE_START_BO) for(Int i=0; i< 4; i++) { dstOffsets[(typeAuxInfo+i)%NUM_SAO_BO_CLASSES]=codedOffset[(typeAuxInfo+i)%NUM_SAO_BO_CLASSES]*(1<<m_offsetStepLog2[compIdx]); } else //EO{ for(Int i=0; i< NUM_SAO_EO_CLASSES; i++) dstOffsets[i] = codedOffset[i] *(1<<m_offsetStepLog2[compIdx]); assert(dstOffsets[SAO_CLASS_EO_PLAIN] == 0); //keep EO plain offset as zero

37. Contd..
switch(typeIdx) { case SAO_TYPE_EO_0: offset += 2; startX = isLeftAvail ? 0 : 1; endX = isRightAvail ? width : (width -1); for (y=0; y< height; y++) signLeft = (SChar)sgn(srcLine[startX] - srcLine[startX-1]); for (x=startX; x< endX; x++) signRight = (SChar)sgn(srcLine[x] - srcLine[x+1]); edgeType = signRight + signLeft; signLeft = -signRight; resLine[x] = Clip3<Int>(0, maxSampleValueIncl, srcLine[x] + offset[edgeType]); } srcLine += srcStride; resLine += resStride; } } break; .
Fig 11.
[2]

38. Contd.. case SAO_TYPE_BO: {
const Int shiftBits = channelBitDepth - NUM_SAO_BO_CLASSES_LOG2;
for (y=0; y< height; y++)
for (x=0; x< width; x++)
resLine[x] = Clip3<Int>(0, maxSampleValueIncl, srcLine[x] + offset[srcLine[x] >> shiftBits] ); }
srcLine += srcStride;
resLine += resStride;
break;
default:
printf("Not a supported SAO types\n");
assert(0);
exit(-1);
}}}
For BO, all pixels are processed
For EO, pixel availability is checked according to SAO type and if pixel is not available in boundary condition it is skipped for processing (i.e., offset of 0 )

39. Conclusion
SAO locates after deblocking and is a in-loop filtering technique that reduces the distortion between original samples and reconstructed samples.
It has been observed that SAO can improve video compression in both objective and subjective measures with reasonable complexity.
Different modes of configuration –Random access mode, Low delay mode and Intra mode configuration have been used for different test sequences of various complexity to implement the SAO filter.
The bit-rate and PSNR have been compared for these test sequences.

40. References
G. J. Sullivan et al, “Overview of the High Efficiency Video Coding (HEVC) Standard”, IEEE Trans.on Circuits and Systems for Video Technology, Vol. 22, No. 12, pp , Dec
C. –M. Fu et al , Members of IEEE “Sample Adaptive Offset in the HEVC Standard” IEEE Transactions on circuits and systems for video technology, vol. 22, No. 12, Dec
S. Srinivasan et al , “Windows Media Video 9:Overview and applications,” Signal Process. Image Commun., vol. 19,no. 9, pp. 851–875, Oct
C.-M. Fu et al, CE13: Sample Adaptive Offset with LCU-Independent Decoding, document JCTVC-E049, Mar
Yang Zhang, Zhi Liu, Jianfeng Qu : “Sample Adaptive Offset Optimization in HEVC” , Sensors & Transducers, Vol 182, Issue11,Nov.2014, pp
C.-M. Fu, C.-Y. Chen, Y.-W. Huang, and S. Lei, “Sample adaptive offset for HEVC,” in Proc. IEEE 13th Int. Workshop MMSP, Oct. 2011, pp.1–5.
V. Sze and M. Budagavi, “Design and Implementation of Next Generation Video Coding Systems (H.265/HEVC Tutorial)”, IEEE International Symposium on Circuits and Systems (ISCAS), Melbourne, Australia, June 2014.
V. Sze, M. Budagavi and G.J. Sullivan (Editors), “High Efficiency Video Coding (HEVC): Algorithms and Architectures”, Springer, 2014.
G. J. Sullivan et al, “Overview of the High Efficiency Video Coding (HEVC) Standard”, IEEE Trans. on Circuits and Systems for Video Technology, Vol. 22, No. 12, pp , Dec
I.E.G. Richardson, “Video Codec Design: Developing Image and Video Compression Systems”, Wiley, 2002.

41. Contd..
I.E.G. Richardson, “The H.264 advanced video compression standard”, 2nd Edition, Hoboken, NJ, Wiley, 2010.
K. Sayood, “Introduction to Data compression”, Third Edition, Morgan Kaufmann Series in Multimedia Information and Systems, San Francisco, CA, 2005
HEVC tutorial by I.E.G. Richardson:
A.Norkin et.al, (2012) HEVC deblocking filtering and decisions. In: Proc. SPIE. 8499, Applications of Digital Image Processing XXXV, no , Oct. 2012
A.Norkin, K.Andersson, V.Kulyk (2013) “Two HEVC encoder methods for block artifact reduction”. In: Proceedings of the IEEE international conference on visual communications and image processing (VCIP) 2013, Kuching, Sarawak, pp. 1–6, Nov. 2013
A.Norkin, K.Andersson , R.Sjöberg (2013) AHG6: on deblocking filter and parameters signaling, Joint Collaborative Team on Video Coding (JCT-VC), Document JCTVC-L0232, Geneva, Jan. 2013
B. Bross et al, High Efficiency Video Coding (HEVC) Text Specification Draft 8, document JCTVC-J1003, Jul.2012.
Joint Video Team of ITU-T VCEG and ISO/IEC MPEG, ITU-T Rec. H.264, ISO/IEC AVC, 2005.
M. Yuen and H. R. Wu, “A survey of hybrid MC/DPCM/DCT video coding distortions,” J. Signal Process., vol. 70, no. 3, pp. 247–278, Nov.1998.

42. Contd..
W.-S. Kim and D.-K. Kwon, CE8 Subset c: Necessity of Sign Bits for SAO Offsets, JCTVC-H0434, Feb
G. Laroche, T. Poirier, and P. Onno, On Additional SAO Band Offset Classifications, JCTVC-G246, Joint Collaborative Team on Video Coding,Nov
N.Ahmed, R.Natarajan and K.R.Rao, “DiscreteCosine Transform”, IEEE Trans. On Computers, Vol.C-23, pp.90-93, Jan.1974
E. Maani and O. Nakagami, Flexible Band Offset Mode in SAO,document JCTVC-H0406, Feb
K. Minoo and D. Baylon, AHG6: Coding of SAO Merge Left and Merge up Flags, JCTVC-J0355, Joint Collaborative Team on Video Coding, Jul.2012.
E. Alshina et al, AHG6: On SAO Type Sharing Between U and V Components, JCTVCJ0045,Joint Collaborative Team on Video Coding, Jul
Access to HM Software Manual:
Test Sequences: ftp://ftp.kw.bbc.co.uk/hevc/hm-11.0-anchors/bitstreams/
Access to HM 16.9 Reference Software:

43. Contd..
HEVC tutorial by I.E.G. Richardson:
Multimedia Course website-
Joint Collaborative Team On Video Coding Information website-
MPL website:

44. THANK YOU 