1. Overview of State-of-the-Art in Digital Image Forensics H. T. SENCAR and N. MEMON Ashwini Chapte 12/5/08 ECE-643 New Jersey Institute of Technology

2. What is Digital Forensics?? Was the picture captured using a digital camera? Scanner? or generated by computer graphics? Which camera brand took this picture? What model? What technologies were employed? What processing has been done? Has it been tampered or manipulated? Does it contain hidden data?

3. Playing detective with digital images

4. Overview

5. Topics covered in this presentation: 1.Introduction 2.Image Source Identification 2.1 Image formation in digital camera and scanner Digital Camera Pipeline Scanner Pipeline 2.2 Source Model Identification Image features CFA and Demosaicing Lens Distortions 2.3 Individual Source Identification Imaging sensor imperfection Sensor Dust Characteristics

6. Image Source Identification Image formation in digital camera and scanner –Digital Camera Pipeline –Scanner Pipeline Source Model Identification –Image features –CFA and Demos icing Artifacts –Lens Distortion Individual Source Identification –Imaging Sensor Imperfections –Sensor Dust Characteristics

7. Image Source Identification Used to find the digital data acquisition device (cameras, scanner, camcorder.,) 2 major outcomes: –Class properties of source –Individual source properties They refer to 2 operational settings –For class property analysis- single image required –For source properties analysis– many images and potential device required Success behind this technology: –Assumption that all images by single DDAD have particular intrinsic characteristics because of the their image formation pipeline and unique hardware components.

8. Digital camera pipeline

9. How it works? When a digital camera captures a photo, the camera creates each pixel using a charge-coupled device—a microchip that is made up of millions of capacitors that get electrical charges depending on how intense the lighting is in a certain spot. Each of these capacitors has a lens and a color filter that creates one single pixel from a mosaic made up of red, green and blue filters. The colors and brightness levels that we can physically see in our digital pictures are created by a demosaicing software, which is custom built for every camera model due to each camera's individual specs and subtle differences. Because of this, a certain camera model will generate distinct pixels—and unique relationships between its neighboring pixels—which can pinpoint the exact make and model of the camera.

11. Scanner Pipeline

12. Source Model Identification Features used to differentiate models: –Processing techniques –Component technologies Example: –the optical distortions due to a type of lens, –the size of the imaging sensor –the choice of CFA and –the corresponding demosaicing algorithm, –and color processing algorithms Drawback of this feature: Not reliable identification as Many models and brands use components by –Same manufacturer –Same processing steps –Same algorithms

13. Image Features A select number (about 34) of features designed to detect post processing are incorporated with new features to fingerprint camera-models. These features are then used to construct multi-class classifiers. The results obtained on moderate to low compressed images taken by 4 different camera-models yielded an identification accuracy of 97%. When repeated on five cameras where three of them are of the same brand, the accuracy is measured to be 88%.

14. Observations of the experiments: 2 Concerns: 1.First is that as they provide an overall decision, it is not clear as to what specific feature enables identification which is very important in forensic investigations and in expert witness testimonies 2.Second concern is the scalability of performance with the increasing number of digital cameras in the presence of hundreds of digital cameras

15. Conclusion of the experiments In general, this approach is more suitable as a pre-processing technique to cluster images taken by cameras with similar components and processing algorithms.

16. CFA and Demosaicing Artifacts Concept exploited: These 2 features are the most pronounced differences among different digital camera-models. Demosaicing is a form of interpolation which in effect introduces a specific type of inter-dependency (correlations) between color values of image pixels. In digital cameras with single imaging sensors, the use of demosacing algorithms is crucial for correct rendering of high spatial frequency image details, and it uniquely impacts the edge and color quality of an image. The specific form of these dependencies can be extracted from the images to fingerprint different demosaicing algorithms and to determine the source camera-model of an image.

17. Experiment conducted & results The accuracy in identifying the source of an image among four and five camera-models is measured as 86% and 78%, respectively, using images captured under automatic settings and at highest compression quality levels. An accuracy of more than 95% can be achieved in identifying the source of an image among four camera-models and a class of synthetic images and studied the change in performance under compression, noise addition, gamma correction and median filtering types of processing This approach was enhanced by first assuming a CFA pattern, thereby discriminating between the interpolated and un-interpolated pixel locations and values.

18. Lens Distortion Concept exploited: lens radial distortion deforms the whole image by causing straight lines in object space to be rendered as curved lines. This feature was exploited to differentiate the camera models as the radial distortion occurs due to the change in the image magnification with increasing distance from the optical axis, and it is more explicit in digital cameras equipped with spherical surfaced lenses. Therefore, manufacturers try to compensate for this by adjusting various parameters during image formation which yields unique artifacts.

19. Experiments and Results: These distortions are quantified using first-order radial symmetric distortion model. These parameters are computed assuming a straight line model by first identifying line segments which are supposed to be straight in the scene and computing the error between the actual line segments and their ideal straight forms. Once computed these features are used to build classifiers and the measurements obtained from images captured with no manual zooming and flash and at best compression level by three digital camera-models resulted with an identification accuracy of approximately 91%

20. Individual Source Identification Concept Exploited: Characteristics like the form of hardware and component imperfections, defects, or faults which might arise due to inhomogeneity in the manufacturing process, manufacturing tolerances, environmental effects, and operating conditions are helpful in matching an image to its source. For example, the aberrations produced by a lens, noise in an imaging sensor, dust specks on a lens will introduce unique but mostly imperceptible artifacts in images which can later be extracted to identify the source of the image.

21. Challenges: Reliable measurement of these minute differences from a single image is very difficult and they can be easily eclipsed by the image content itself. These artifacts tend to vary in time and depend on operating conditions Therefore they may not always yield positive identification

22. Image sensor imperfections Concept Exploited: This approach focuses on matching the source by identifying and extracting systematic errors due to imaging sensor, which reveal themselves on all images acquired by the sensor in a way independent of the scene content. These errors include sensor’s pixel defects and pattern noise which has two major components, –fixed pattern noise –photo response non-uniformity noise

23. Experiments and results: The initial work in this field, fixed pattern noise caused by dark currents in (video camera) imaging sensors is detected. Dark current noise refers to differences in pixels when the sensor is not exposed to light and it essentially behaves as an additive noise. It was compensated within the camera by first capturing a dark frame and subtracting it from the actual readings from the scene, thereby hindering the applicability of the approach. Also experiments on 12 cameras showed the uniqueness of the defect pattern and also demonstrated the variability of the pattern with operating conditions.

24. Continued… Difference in the dimension of the array can be used to distinguish between digital camera and scanner images. In realizing this, classifiers are built based on (seven) statistics computed from averaged row and column reference patterns extracted from both scanned images at hardware resolution (e.g., no down-sampling) and digital camera images. Using the above technique, average accuracy of more than 95% is achieved in discriminating digital camera images from scanned images. When the images are compressed with JPEG quality factor 90 an accuracy of 85% is obtained in identifying the source scanner of an image among four scanners.

25. Sensor Dust Characteristics This method is based on sensor dust characteristics of single digital single-lens reflex (DSLR) cameras which are becoming increasingly popular because of their interchangeable lenses. The sensor dust problem emerges when the lens is removed and the sensor area is opened to the hazards of dust and moisture which are attracted to the imaging sensor due to electrostatic fields, causing a unique dust pattern before the surface of the sensor. As sensor dust problem is persistent and most generally the patterns are not visually very significant, traces of dust specks can be used for two purposes: 1. To differentiate images taken by cheaper consumer level cameras and DSLR cameras. 2.to associate an image with a particular DSLR camera

26. Experiments & Results Using an empirical dust model characterized by intensity loss and roundness properties; the authors proposed a technique to detect noise specks on images through match filtering and contour analysis as dust patterns might not indicate anything if they have been cleaned. In the experiments, ten images obtained from three DSLR cameras are used in generating a reference pattern which is then tested on a mixed set of 80 images (20 taken with the same camera and 60 with other cameras) yielding an average accuracy of 92% in matching the source with no false-positives.