1 Exposing Digital Forgeries in Color Array Interpolated Images Presented by: Ariel Hutterer Final Fantasy,2001My eye

2 References  Alin C.Popescu and Hany Farid: Exposing Digital Forgeries in Color Filter Array Interpolated Images.  Yizhen Huang: Can Digital Forgery Detection Unevadable? A Case Study : Color Filter Array Interpolation Statistical Feature Recovery.  Hagit El Or Demosaicing.

3 Outline  Introduction  Digital Cameras  Interpolations  Detecting CFA Interpolation  Results  Crack Methods  Computer Graphics

4 Introduction- forgeries  Low cost: cameras,photo editing software.  Images can be manipulated easily.  Splicing.

5 Introduction- forgeries  Images have a huge impact in public opinion.  Legal world.  Scientific evidence.

6 Introduction - preventing forgeries approaches  Two principal approaches to prevent forgeries: Digital watermarking:  Means that image can be authenticated.  Drawbacks:  Specially equipped digital cameras,that insert the watermark.  Assume that watermark cannot be easily removed and reinserted. (but ….it is???) Statistic analysis:  Most color digital cameras, introduces specific correlation:  A third of the image are captured by a sensor.  Two thirds of the image are interpolated.  Images manipulated must alter this specific statistic.

7 Outline  Introduction  Digital Cameras  Interpolations  Detecting CFA Interpolation  Results  Crack Methods  Computer Graphics

8 Digital Cameras  Most Color digital Cameras have a single monochrome Array of sensors

9 Digital Cameras  How does color form with monochrome sensor for each pixel?

10 Digital Cameras-Bayer Color Array  Half pixels are Green,quarter are Red and quarter are Blue

11 Digital Cameras-Bayer Color Array  Several possible arranges Bayer Diagonal Bayer DiagonalStriped Psudo-random Bayer

12 Digital cameras - forming color

13 Digital cameras - forming color

14 Digital cameras - forming color Interpolation

15 Digital cameras - forming color  Bayer Array For almost all Digital Cameras  Color Interpolation different for each make of Digital Camera Interpolation

16 Outline  Introduction  Digital Cameras  Interpolations  Detecting CFA Interpolation  Results  Crack Methods  Computer Graphics

17 Interpolations  Naive – per channel interpolation Nearest neighbor,Bilinear interpolation  Inter-channel dependencies and correlations – Reconstruct G channel, then reconstruct R & B based on G. Reconstruct all 3 channels constrained with inter-channel dependence.  Adaptive reconstruction – Measure local image variations (e.g. edges, gradients, business) and reconstruct accordingly.

18 Interpolations - Aliasing R R G G G G G G G G G G G G G G G B B B R R R R R R R R R R R R R R R R R R R R R G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G B B B B B B B B B B B B B B B B B B B B B Interpolate

19 Interpolations - Aliasing R R G G G G G G G G G G G G G G G B B B R R R R R R R R R R R R R R R R R R R R R G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G G B B B B B B B B B B B B B B B B B B B B B Interpolate Result

20 Interpolations - Samples

21 Interpolation-Bilinear Bicubic  Red and Blue Kernels: Separable 1-D filters Rw Rw = ½(Rnw+Rsw)

22 Interpolation-Bilinear Bicubic  Green kernels 2-D filters:

23 Interpolation- Gradient Based  First, calculate Green channel: Calculate derivates estimators Determination of Green’s values

24 Interpolation – Evaluation Tools

25 Interpolation -Results OriginalLinearKimmel

26 Outline  Introduction  Digital Cameras  Interpolations  Detecting CFA Interpolation  Results  Cracks Methods  Computers Graphics

27 Detecting CFA Interpolation  In Each pixel only one color derives from the sensor,two others derive from interpolation from their neighbors.  The correlation are periodic.  Tampering will destroy these correlations.  Splicing together two images from different cameras will create inconsistent correlations across the composite image.

28 Detecting CFA Interpolation  Two different tools: EM algorithm : EM algorithm  Produce Map of Probabilities and interpolation coefficients  Used to detect kind of interpolation Farid’s Indicator: Farid’s Indicator  Produce Map of Similarities  Used to quantify the similarity to CFA Interpolated Image

29 EM Algorithm (Expectation/Maximization):  Two possible models: M1:the sample is linearly correlated to its neighbors M2:the sample is not correlated to its neighbors

30 EM Algorithm (Expectation/Maximization):  f(x,y) – color channel  alpha - parameters,where(0,0) = 0. denotes the specific correlation.  n - independent and identically samples drawn from a Gaussian distribution, with 0 mean and unknown variance

31 EM Algorithm (Expectation/Maximization):  Two-step iterative algorithm: E-step : calculate the probability of each sample M-step: the specific form of the correlation is estimated.  Based in Bayes rule:

32 Farid’s indicator  The similarity between the probability and a synthetic map is obtained by:  Where:  Similarity measure is phase insensitive

33 Farid’s indicator  How to use it: CFA-Interpolated : if at least one channel is greater than threshold1 Non CFA Interpolated: if all 3 channels are smaller than threshold2 Ind(cfa-isf) Ind(cfa-sf) CFA InterpolatedNon CFA InterpolatedUnknown threshold2threshold1 result

34 Huang indicator  Motivation: Farid’s Indicator is proportional to image size.  Table of Green Channel Indicator  Huang Indicator: Indicator function 32x32128x128256x256512x512 Farid Huang

35 Outline  Introduction  Digital Cameras  Interpolations  Detecting CFA Interpolation  Results  Cracks Methods  Computers Graphics

36 Results  Detecting different interpolation methods  Detecting tampering  Measuring Sensitivity and robustness

37 Detecting different interpolation methods  Hundreds of images from 2 digital cameras  Blur 3x3  Down sampled  Cropped  Resample in CFA Interpolations

38 Detecting different interpolation methods

39 Detecting different interpolation methods

40 Detecting different interpolation methods

41 Detecting different interpolation methods

42 Detecting different interpolation methods  Coefficients are 8 to each color so we are a 24-D vector,LDA classifier,results: 97% Interpolations kinds was detected  2D projection of LDA

43 Detecting tampering  Hiding the damage of the car Air-brushing,smudging,blurring and duplication

44 Detecting tampering  Result: Left F(p) : for tampered portion Right F(p) : for unadulterated portion

45 Measuring Sensitivity and robustness remember False0%Median 5x5 97 Bilinear100%Gradient based 100% Bicubic100%Adaptive color plane 97% Median 3x3 99 Variable number of gradients 100%  Testing different interpolations with Farid’s indicator

46 Measuring Sensitivity and robustness  Testing influence of jpeg

47 Measuring Sensitivity and robustness  Testing influence of Gaussian Noise

48 Outline  Introduction  Digital Cameras  Interpolations  Detecting CFA Interpolation  Results  Crack Methods  Computer Graphics

49 Cracking  What’s a “true digital image”  General Model

50 True digital image  It was taken by a CCD/CMOS digital camera, or other device with similar function and remains intact after shooting except for embedding ownership and other routinely added information.

51 General Model  where: W all images S all possible images tacked by an ideal camera c. N are S enlarged because of noise.  Detection method: Pm(I), a projection of Image I I is true when: I is Artificially CFA-interpolated

52 General Model  The result image should be as close as possible to the original  The mean of the difference to an ideally CFA interpolated image should be controlled in a specific range.  Such difference should be distributed averagely.

53 General Model  Im: Tampered Image  Im’: cracked Image  Int(I) : Ideal Interpolated Dif(Im,Im’) Dif(Im’,Int(Im’)) K2 K1 Dif(Im,Im’,Int(Im’))

54 General Model  We are looking for  We want to minimize the 3d distance:

55 Outline  Introduction  Digital Cameras  Interpolations  Detecting CFA Interpolation  Results  Crack Methods  Computer Graphics

56 Computer Graphic  A naïve approach: Computer Graphic will be detected like non CFA-Interpolated.

57 Computer Graphics  Huge improvement of dedicated hardware in the last 7 years  SGI:Onyx2,Infinity reality 3(2000) : 12 bits * 4 channels No shaders End User license,250,000$  Pc d/core, geforce 8(2006): 32 bits * 4 channels Shaders w/24 parallels pipes 1,500-5,000$

58 Computer Graphics  2001,Final fantasy,first Film made with PC.

59 Computer Graphics  See cg not like an Image, see it like REALITY. Render Reality high resolution,by 32bits for each color Optical distortions, ghost and blurring Sensor CFA sampling and noise Interpolation Image

60 Computer Graphics From Image Forgeries to Science Fiction  Image forgeries are a “positive issue“ for development of: Simulators. Trainers. Robots………

61 Computer Graphics From Image Forgeries to Science Fiction

62 Conclusion  Detection CFA-Interpolated methods are not enough robust.  Compression like jpeg destroy the interpolation correlation.  Interpolation can be artificially made. The End