1. Yoshinori Dobashi (Hokkaido University) Yusuke Shinzo (Hokkaido University) Tsuyoshi Yamamoto (Hokkaido University) Modeling of Clouds from a Single Photograph

2.  Introduction  Related Work  Proposed Method  Results  Conclusion

3.  Realistic Display of Clouds ◦ Synthesizing images of outdoor scenes ◦ Realistic density distribution ◦ Long research history  Modeling of clouds ◦ Procedural approach ◦ Physically-based approach outdoor scene

4.  Procedural approach ◦ Use of simple heuristically-defined rules ◦ Low computational cost ◦ Difficult to adjust parameters  Physically-based approach ◦ Simulating physical process of cloud formation ◦ Realistic clouds ◦ High computational cost

5.  Image-based approach ◦ Use of a single photograph ◦ Not to reconstruct the same clouds ◦ Using the photo as a guide to synthesize similar clouds ◦ Three types of clouds cirrusaltocumuluscumulus

6.  Introduction  Related Work  Proposed Method  Results  Conclusion

7.  Procedural modeling ◦ Fractals [Vos83] ◦ Textured ellipsoids [Gar85] ◦ Metaballs + noise function [Ebe97] ◦ Spectral synthesis [Sak93] ◦ Real-time modeling/rendering system [SSEH03] [Gar85][Ebe97][SSEH03] Difficult to adjust parameters

8.  Physically-based modeling ◦ Numerical solution of atmospheric fluid dynamics [KH84, MYDN01, MYDN02] ◦ Controlling cloud simulation [DKNY08] [KH84][MYDN01][MYDN02][DKNY08] High computational cost

9.  Image-based modeling ◦ Use of infrared satellite images [DNYO98] Not applicable to photo taken from the ground No cloud types satellite imagesynthesized clouds

10.  Introduction  Related Work  Proposed Method  Results  Conclusion

11. cirrus alto- cumulus alto- cumulus

12. cirrus alto- cumulus alto- cumulus Calculation of cloud image (common process) Calculation of cloud image (common process)

13.  Overview cirrus altocumulus cumulus intensity opacity intensity opacity intensity opacity (input photographs)(cloud images)

14.  Processes input image sky image intensity opacity 1. Removing cloud pixels2. Interpolating sky color 3. calculation of intensity/opacity

15. 1. Removing cloud pixels  Processes input image sky image intensity opacity 2. Interpolating sky color 3. calculation of intensity/opacity

16.  Processes input image sky image intensity opacity 1. Removing cloud pixels2. Interpolating sky color 3. calculation of intensity/opacity

17.  Processes input image sky image intensity opacity 1. Removing cloud pixels2. Interpolating sky color 3. calculation of intensity/opacity = cloud image

18.  Use of chroma to identify cloud pixels ◦ Clouds are generally white (or gray) ◦ Remove pixel if chroma < threshold Input photographremoved cloud pixels

19.  Interpolation of sky colors by solving Poisson equation: sky image cloud pixels p c p c : cloud pixel = R, G, B

20.  Intensity of clouds (single scattering) viewpoint sun sky cloud

21.  Intensity of clouds (single scattering) viewpoint sun sky I sun  I sun cloud

22.  Intensity of clouds (single scattering) viewpoint sun sky I sky I sun  I sky  I sun cloud

23.  Intensity of clouds (single scattering) viewpoint sun sky I sky I cld I sun  I sky  I sun cloud (p: pixel, = R, G, B)

24.  Computing cloud intensity (  ) & opacity (  ) by minimizing: input image sky image unknowns store these in cloud image

25. cirrus alto- cumulus alto- cumulus Modeling of cirrus

26.  Cirrus clouds ◦ Thin and no self-shadows ◦ Two-dimensional texture input photograph

27.  Use of cloud image as 2D texture ◦ Removing effect of perspective transformation by specifying cloud plane input image cloud image cloud plane cirrus cloud texture

28. cirrus alto- cumulus alto- cumulus Modeling of altocumulus

29.  Altocumulus ◦ Thin but with self-shadows are observed ◦ Using Metaballs to define three-dimensional density distribution input photo metaball position radius center density field function metaball

30.  Converting cloud image into binary image binary imagecloud image

31.  Distance transform of binary image ◦ Distance from a white pixel to the nearest black pixel binary image distance image

32.  Generating 2D metaballs at white pixels ◦ Use distance for radius of metaball binary image

33.  Generating 2D metaballs at white pixels ◦ Use distance for radius of metaball distance image

34.  Minimizing difference between cumulative density and cloud intensity cloud image cumulative density at pixel p center density field function [Wyvill90]

35.  Example cloud image cumulative density image

36.  Interactive specification ◦ Orientation of cloud plane and viewing angle cloud plane

37.  Projecting metaball center onto cloud plane  Scaling metaball radius in proportion to distance from viewpoint viewpoint cloud plane 2D metaballs

38.  Calculating bounding box of all metaballs  Subdividing bounding box into grid  Computing density at each grid point bounding box density distribution

39. cirrus alto- cumulus alto- cumulus

40.  Cumulus ◦ Generating surface shape ◦ Calculating densities inside surface shape cloud imagesurface shape3D density distribution

41.  Converting cloud image into binary image cloud imagebinary image

42.  Distance transform of binary image  Extracting medial axes ◦ Pixels where distances are local maxima binary image distance image medial axis

43.  Use distance at medial axis as thickness of clouds  Propagate thickness by optimization cloud image medial axis thickness image colorization by optimization [Levin 04]

44.  Propagate thickness by optimization thickness similarity between pixels p and q Use of same technique proposed in colorization by optimization [Levin 04]

45.  Constructing surface shape ◦ Assuming symmetric shape with respect to image plane front viewside view

46.  Calculating bounding box of surface shape  Subdividing bounding box into grid  Calculating density at each grid point

47.  Generating binary volume ◦ Assign 1.0 to grid points inside surface shape ◦ Assign 0.0 to other grid points  3D distance transform of binary volume  Normalizing distance so that maximum distance is 1  Assign normalized distance as density at each grid point ◦ Use of Perlin noise to add extra details

48.  Density distribution input photographdensity distribution

49.  Introduction  Related Work  Proposed Method  Results  Conclusion

50.  Computer ◦ CPU: Intel Corei7 (3.33 GHz) ◦ Main memory: 4GB ◦ GPU: NVIDIA GeForce GTX 295  Computation time ◦ within 10 seconds

51.  Cirrus input image cloud texture input image cloud texture

52.  Altocumulus input synthesized input synthesized

53.  Cumulus input synthesized input synthesized

54.  Modeling process

55.  A cloud scene

56.  Modeling of clouds from a photograph ◦ Methods for three types of clouds (cirrus, altocumulus, cumulus) ◦ Generating realistic 3D clouds similar to the input photograph  Future Work ◦ Improving shapes of cumulus clouds

57.  END

58. photographsynthesized