1. David Andrzejewski, Univ. of Wisconsin-Madison (USA) David G. Stork, Ricoh Innovations, Inc. and Stanford Univ. (USA) Xiaojin Zhu, Univ. of Wisconsin-Madison (USA) Ron Spronk, Queen's Univ. (Canada) 1

2.  Visual arts  Digital authentication of Bruegel, Perugino (Lyu et al, 2004)  Jackson Pollock (Taylor, 1999) (Irfan and Stork, 2009)  Writings  Authorship of the Federalist Papers (Mosteller and Wallace, 1964)  Ronald Reagan’s radio addresses (Airoldi et al, 2007) 2

3. 3 http://www.artchive.com http://www.artchive.com Haags Gemeentemuseum, The Hague

4. 4

5.  Better understand compositional style 1. Develop a formal representation of the paintings 2. Extract these representations from paintings 3. Train a generative model 4. Learn relative visual weights of colors 5. Classify true Mondrians versus 1.“fakes” created by the generative model in step 3 2.“earlier states” of the Transatlantic paintings 5

6. Vertical/horizontal lines locations extents Rectangles locations sizes colors can span multiple lines 6

7. 7

8. Semi-automatic extraction tool 8

9.  Hypothesize an underlying probabilistic model that generates observed data  Many uses in machine learning  Make predictions (Naïve Bayes)  Generate new examples (Markov model)  Interpret parameter values (Linear regression)  Given data, learn/train model parameters  Our approach: Maximum likelihood estimation (MLE) 9

10. Canvas aspect ratios (kernel density estimator) 10

11. Number of horiz/vert lines (Poisson) Horiz/vert line spacing (Dirichlet) 11

12. Segments are deleted / invisible / left alone (Polya) 12

13. Rectangle colors (Multinomial) 13

14. Don’t allow unrealistic “hanging” lines Require ≥ 1 vertical line 14

15. Rectangle colorMultinomial probability White0.754 Red0.085 Yellow0.062 Blue0.065 Black0.034 Line typeSpacing Dirichlet Vertical1.80 Horizontal1.61 15

16.  Calculate visual “center of mass”  Assume true Mondrians centered at [0.5,0.5]  Learn color weights via linear programming 16 RedYellowBlueBlack 0.2370.1430.2270.392

17.  Completed in Europe, but then altered after Mondrian’s arrival in the United States  A variety of techniques (x-ray, UV, etc) were used to recover the earlier states (Cooper & Spronk, 2001 ) 17

18. Composition with Red, Blue, and Yellow (1937-1942) 18

19. Composition with Red, Yellow, and Blue (1935-1942) 19

20. No. 9 (1939-1942) 20

21.  Very popular technique in machine learning  At each iteration, choose a rule to “split” on  Resulting partitions should be more “pure” with respect to target classification (true Mondrian or computer-generated fake?)  Key feature: resulting trees easy to interpret  Estimate accuracy with leave-one-out cross- validation  Control over-fitting with pruning 21

22.  45 true Mondrians versus 45 generated “fakes”  45 true Mondrians versus 11 “earlier states” ClassifierAccuracy Majority baseline50% Decision tree (no pruning)70% Decision tree (with pruning)68% ClassifierAccuracy Majority baseline81% Decision tree (no pruning)72% Decision tree (with pruning)75% 22

23.  Analysis of results  Transatlantic dataset  < 1% pixels blue  # horiz / # vert < 0.9  Low visual “density”  THEN Transatlantic 23

24.  Formal representation and feature extraction  Generative model  Fitting simple statistics of Mondrians cannot create realistic synthetic paintings  Color weights align well with our intuitions  Classification  Can reliably discriminate true Mondrians vs. computer- generated  Cannot do so for true Mondrians vs Transatlantic “earlier states” ▪ Underlying images were “nearly complete” (!) 24