1. TVCG

2. Introduction

3. Introduction Motivation:
Design an effective method for synthesizing mosaics with irregularly shaped tiles.
Can be used at the entertainment industry
Goals:
The method should work with any set of given tiles
The resulting mosaic should be visually pleasant while being more efficient than manual work

4. Contribution
An optimization based method to produce man-made lookalike 3D mosaics.
Using a combination of continuous an combinatorial optimization methods in a smart pipeline.
Smart definition of the problem

5. Related Work

6. Related Work Irregular packing Polygon containment
Computer simulated mosaic
Reassembly of fragments

7. Irregular packing
Packing Irregular objects into a container object is well known problem
It is NP-hard:
R. Fowler, M. Paterson, and S. Tanimoto, “Optimal packing and covering in the plane are NP-complete,” Inf. Process. Lett., vol. 12, no. 3, pp. 133–137, 1981

8. Polygon containment
Polygon containment is a variant of the packing problem
Single polygon containment with translation only
B. Baker, S. Fortune, and S. Mahaney, “Polygon containment under translation,” J. Algorithms, vol. 7, no. 4, pp. 532–548,
Translation & Rotation
F. Avnaim and J. Boissonnat, “Polygon placement under translation and rotation,” in Proc. 5th Annu. Symp. Theoretical Aspects of Comput. Sci., 1988, vol. 294, pp. 322–333.
Translation & Rotation with multiple polygon containment
V. J. Milenkovic, “Multiple translational containment part ii: Exact algorithms,” Algorithmica, vol. 19, no. 1, pp. 183–218, 1997
Mathematical programming formulation
R. B. Grinde and T. M. Cavalier, “Containment of a single polygon using mathematical programming,” Eur. J. Oper. Res., vol. 92, no. 2, pp. 368–386, 1996.

9. Computer simulated mosaic
Many variation of the problem:
2D – extensive work
S. Battiato, G. Di Blasi, G. M. Farinella, and G. Gallo, “Digital mosaic frameworks—an overview,” Comput. Graph. Forum, vol. 26, no. 4, pp. 794–812, 2007.
2D – similar problem
L.-P. Fritzsche, H. Hellwig, S. Hiller, and O. Deussen, “Interactive design of authentic looking mosaics using Voronoi structures,” in Proc. 2nd Int. Symp. Voronoi Diagrams Sci. Eng., 2005, pp. 1–11.

10. 2D – extensive work
S. Battiato, G. Di Blasi, G. M. Farinella, and G. Gallo, “Digital mosaic frameworks—an overview,” Comput. Graph. Forum, vol. 26, no. 4, pp. 794–812, 2007.

11. 2D – similar problem
L.-P. Fritzsche, H. Hellwig, S. Hiller, and O. Deussen, “Interactive design of authentic looking mosaics using Voronoi structures,” in Proc. 2nd Int. Symp. Voronoi Diagrams Sci. Eng., 2005, pp. 1–11.

12. 3D – square tiles
Y.-K. Lai, S.-M. Hu, and R. R. Martin, “Surface mosaics,” Visual Comput., vol. 22, nos. 9–11, pp. 604–611, 2006

13. 3D – irregular tiles
V. Dos Passos and M. Walter, “3D virtual mosaics: Opus palladium and mixed styles,” Visual Comput., vol. 25, no. 10, pp. 939– 946, 2009.

14. Reassembly of fragments
A common approach in computer aid reassembly of fragments, is to identify pair-wise matches among fragments, then assemble them into clusters using a connectivity graph.
Matching can be done with shape cues
] H. C. da Gama Leit~ao and J. Stolfi, “A multiscale method for the reassembly of two-dimensional fragmented objects,” IEEE Trans. Pattern Anal. Mach. Intell., vol. 24, no. 9, pp. 1239– 1251, Sep
Combined with texture, color and normal cues
C. Toler-Franklin, B. Brown, T. Weyrich, T. Funkhouser, and S. Rusinkiewicz, “Multi-feature matching of fresco fragments,” ACM Trans. Graph., vol. 29, no. 6, pp. 185:1–185:12, Dec
Those methods relies on the fact that the fragments are all originated in one object, unlike the mosaic settings.

15. Problem Description

16. Problem Description M – the base surface, a triangle mesh
- Irregularly shaped meta object

17. Problem Description Where:
Where:
P – tile, a duplicate of an element in L with set of parameters Θ
Θ – tile configuration i.e. position, orientation
n – number of tiles, variable
Every tile, P, lies on the tangent plane to the base surface at the tile centroid
Goal:
Optimizing tile selection and configuration to sufficiently cover the base surface

18. Overview of the Method

19. Overview of the Method

22. Tile Configuration Optimization

24. Tile Configuration Optimization
Initial Tile Placement
Curvature based tiles distribution/size
We define a “desired tile size”(DTS) function based on curvature:
where Kmax is maximum principal curvature in the mesh vertex v
Than we sample n points using a distribution based on the DTS function
At each point we pose a tile in the correspondent tangent plane, where tiles are chosen from L in a “round-robin” fashion
Tiles are placed at 10% of their DTS value to avoid tile’s overlaps

26. Tile Configuration Optimization
Prerequisite
Delaunay triangulation
Delaunay triangulation for a set P of points in a plane is a triangulation DT(P) such that no point in P is inside the circumcircle of any triangle in DT(P)

27. Tile Configuration Optimization
Prerequisite
Voronoi diagram
Voronoi diagram is a partitioning of a plane into regions based on distance to points in a specific subset of the plane. That set of points (called seeds, sites, or generators) is specified beforehand, and for each seed there is a corresponding region consisting of all points closer to that seed than to any other. These regions are called Voronoi cells/regions.

29. Tile Configuration Optimization
Approximate Chordal Axis Transform
We perform Delaunay triangulation for the points that represents the tiles
Edges are classified to interior (same tile) and exterior (different tiles)
Triangles are classified to junction/external/internal triangle
The chordal axis is obtained by connecting the midpoints as follows:
The resulting polygons are called CAT regions

31. Tile Configuration Optimization
After performing the CAT we’re projecting the CAT regions to tile plane – denoted as V
The problem is reduced for to “extremal polygon containment problem with two polygons” – which is familiar problem
We now formalize the new problem as non-linear optimization problem

32. Tile Configuration Optimization
P – tile represented by the samples of its edges (p1,p2,…pl)
Q – the CAT region projection, container polygon – represented by its edges
The strategy here is to define a feasible region for every sample in P

33. Tile Configuration Optimization
We compute Voronoi diagram with P as seeds
Every edge of Q intersects with one or more Voronoi region, for every point we denote as all the edges e from Q that intersects with the corresponding Voronoi region

34. Tile Configuration Optimization
We denot the new location for every poin p as: W
here is the rotation matrix implied by the angle θ
and the parameters to optimize are (t,θ,s) = (translation,rotation,scale)
We denote as the starting point of edge
The range of the rotation angle is limited to ] for every iteration to keep containment constraint.
Thus, were getting the following non-linear constrained optimization problem:

35. Tile Configuration Optimization
Additionaly we add a series of barriers: {} to keep synchronization
This optimization problem is solved by exterior software package KNITRO, implementing the interior-point algorithm
The authos state that local maximizer with initial value of (s,θ,t)=(1,0,(0,0)) is satisfactory

36. So far..

37. Tile Configuration Optimization

38. Tile Combination Optimization
Shape Matching – will be used in later algorithms
Given “hole region” and a meta object we do the following:
Align hole & object centroids
Compute the optimal 2D orientation using the “registration method”
Uniformly scale the meta object such that it’ll have the same area as the hole region
The similarity measure is defined to be the symmetric deifference area of the two shapes – i.e. small means similarity
The shape matching algorithms, returns the transformation for the meta-object and the similarity measure

40. Tile Combination Optimization
Tile Permutation
We choose a subset of the current tile layout
For every tile we extract the hole formed by its neighbors tiles as follows:
To get the hole region we use the Delaunay triangulation
We start with the tile samples P
We grow the region by add adjacent triangles until reaching a different tile or if the edge is shortet than 10% of average tile edge

41. Tile Combination Optimization
Tile Permutation
We now form a bipartite graph of tiles and holes such that, for every tile i and hole j the weight of the connecting edge is its similarity measure
A minimum accumulated weight match is corresponding with the beset permutation of the subset
We apply the “Hungarian algorithm” to solve this assignment problem
To avoid problems – a subset can’t contain two adjacent tiles
holes
tiles

44. Tile Combination Optimization
Tile Replacement
We measure ratio between each tile and its surroundings
Than, replaces only 25% smallest ratios
We need do avoid to many repetition of this algorithm since it tends to prefer some tiles
The authors state that usually more than 3 times is usually not very effective either way

47. Tile Combination Optimization
Hole filling
We find holes the same way we did in the Tile Permutation algorithm
If hole is bigger than 50% of average tile size, fill the hole with the most fitting meta-object

49. Tile Combination Optimization
Comparison of with and without Combinataion Optimization

50. Experiments and Results

51. Experiments
Tiles sets used in experiments:

52. Experiments Different experiments results:
Each picture is using a different α & n
According to authors running time is most correlated with n
For n=2000 running time is min, in the examples only the bunny exceeds 2000 tiles
Covrage rate is between for rock & stone tiles, and for clip-art tiles
We can see that the geometry of the base surface is kept

53. Experiments
More tiles leads to better coverage rate:

54. Experiments
By treating user segmentations as open boundaries, multiple patterns can be achieved on on surface:

55. Experiments Anisotropic Mosaics
Sometimes we want tiles to orient according to a specific orientation:
This could be achieved by making the orientation angle θ, not a parameter to be optimized but a parameter that is set according to our specification

56. Experiments
Comparison with planar mosaic methods based on centroidal Voronoi tessellation
Significant coverage rates differences

57. Experiments
Comparison with surface mosaic method based on Vornoi’s polygons
Other method doesn’t look like man-made mosaic

58. Experiments Comparison with manual design
Significant time saving, and comparable aesthetic results

59. Class discussion

60. Does the paper make reasonable assumptions? What are they?
There aren’t any stated assumtions, but it is implied that authors treat tile and CAT regions as relatively small

61. How novel is the solution? Explain.
Pretty novel, though, for mosaic generation, each paper tackles a slightly different problem, so that makes them, the first one to solve this specific problem
The solution is based on a lot of algorithms, but the changes the authors made and the packing of all together make the solution novel, to my opinion

62. Is the solution technically sound? Explain.
I think the solution is technical sound, they explain the motivation behind any step they make, and show results that support the steps e.g. the combination optimization

63. How well is the solution evaluated? Explain.
I think that the paper is nicely showing the potential of the algorithm through diverse experiments and results, though not showing any failure/catastrophic results
The comparison part is a bit problematic, first of all, mosaic are naturally artistic, so measuring it is not clear. They compare it with 3 different categories: planar, surface, manual. Each category is being compared differently in a way that shows the superiority of the suggested method, so it doesn’t make much sense to me.
As I stated earlier, each mosaic paper is trying to solve a slightly different problem (tiles variety, limitations, etc.) so comparing them is no trivial.

64. Writing level: Is the paper clearly written. Is it self-contained
Writing level: Is the paper clearly written? Is it self-contained? Explain.
I think that the paper writing level is generally ok, explaining every step motivation and writing clearly. Though the proposed method is pretty complex and requires several reads to fully understands
The paper is not self-contained, may algorithms involved are only mentioned by name, besides Delaunay triangulation & Voronoi diagram which are crucial to the algorithms are not explained (maybe its topic basics, but I didn’t know what those mean)

65. What didn't you understand?
There are contradictions inside the paper, for example, the number of elements is said to be computed by the ration of average meta-object size and surface size, though, they later show that increasing the number of tiles improves coverage.