1. Daniel Pilon Senior project officer at NRCan
Using FME for Topographical Data Generalization at Natural Resources Canada
Daniel Pilon
Senior project officer at NRCan

2. Overview
Map generalization at NRCan
Principles of good map generalization
ETL generalization patterns

3. Very complex: manually or automatically
Map Generalization
The process of reducing detail on a map
Very complex: manually or automatically

4. Generalization Paradigm Shift
Manually
Comply to very strict specifications
Prepackage products 250K, 1M, 7.5M
Finality on its own
Art Craft
Process
Automatically (faster turnaround)
Less complex specifications
On demand scale production
Integrated into other products
Commodity
Process

5. Generalization Paradigm Shift Example WMS
Web Mapping Service
1 :
1 :
1 :
1 :
1 :

6. Generalization Paradigm Shift Example WMS
1 :

7. Generalization Paradigm Shift Example WMS
1 :

8. Generalization Paradigm Shift Example WMS
1 :

9. Generalization Paradigm Shift Example WMS
Threshold: 1:
1 : (serving 1: )
1 : (serving 1: )

10. FME for Generalization…Why?
Few generalization solutions
No complete generalization solution
Some European solutions
Very complex software to use…
Tackle European problems

11. FME for Generalization…Why?

12. FME for Generalization…Why?
Few generalization solutions
No complete generalization solution
Some European solutions
Very complex software to use…
Tackle European problems
FME as a generalization tool…
General purpose spatial manipulation engine
Reliable and fast
Easily add extensions (python)

13. Overview
Map generalization at NRCan
Principles of good map generalization
ETL generalization patterns

14. The principles of Good Genralization Automation
Generalization
cases
Simple and efficient decomposition of the tasks performed by the cartographers
Objective tools to characterize map objects
Tools to edit modify the spatial objects
Reactive control on the map object’s state during the process
Unambiguous rules in order to guide the process
Measure
Operator
Constraint
Behaviour
pattern

15. Generalization Case
It is a formalization of the cartographer’s knowledge (generalization rules) and a communication tool between the cartographer and the software engineer
Generalization cases do not tell us what algorithms or what parameters to use

16. Generalization Case example
For the Vegetation
For all the Vegetation which breaks the constraint Dimension: Minimum area
If the Vegetation breaks the constraint: Position proximity
Aggregate the area
Else
Eliminate the area

17. The principles of Good Genralization Automation
Generalization
cases
Simple and efficient decomposition of the tasks performed by the cartographers
Objective tools to characterize map objects
Tools to edit modify the spatial objects
Reactive control on the map object’s state during the process
Unambiguous rules in order to guide the process
Measure
Operator
Constraint
Behaviour
pattern

18. Measures are used: Measures
To calculate the characteristics of map objects (or group of map objects )
Before the generalization: to know how to generalize
After the generalization: to assess the success of the applied generalization operations

19. Measure Examples Unary measures Length Area/Perimeter
Circularity ratio
Oriented
Bounding box
Triangulation
Bounding box

20. Measure Examples Binary measures Spatial relationship Distance
Triangulation

21. Measure Examples N-ary mesaures H3 H1 H2 S1 H1 H1 H3 H1 H1 H3 H2 H2 H2
Stream ordering (Strahler or Horton)

22. The principles of Good Genralization Automation
Generalization
cases
Simple and efficient decomposition of the tasks performed by the cartographers
Objective tools to characterize map objects
Tools to edit the spatial objects
Reactive control on the map object’s state during the process
Unambiguous rules in order to guide the process
Measure
Operator
Constraint
Behaviour
pattern

23. Generalization Operators
Generalization operators are typical transformations applied on spatial objects. They allow the decomposition of the generalization process into several sub-problems in order to manage complexity
Generalization operator are implemented using different generalization algorithms. Ex.: For line simplification: Douglas & Peucker, Lang, Sherbend

24. Generalization Operator Examples
Selection
Simplification
Smoothing
Collapse

25. Generalization Operators Examples
Aggregation
Typification
Displacement

26. The principles of Good Genralization Automation
Generalization
cases
Simple and efficient decomposition of the tasks performed by the cartographers
Objective tools to characterize map objects
Tools to edit modify the spatial objects
Reactive control on the map object’s state during the process
Unambiguous rules in order to guide the process
Measure
Operator
Constraint
Behaviour
pattern

27. Constraints
Constraints are rules applied to data in order to comply with requirements of the target map specifications (ex: An area must respect the minimum size threshold)
Constraint are used:
To trigger a generalization operation
To assess the result of a generalization operation
To achieve generalization, a feature must fulfill several constraints

28. The principles of Good Genralization Automation
Generalization
cases
Simple and efficient decomposition of the tasks performed by the cartographers
Objective tools to characterize map objects
Tools to edit modify the spatial objects
Reactive control on the map object’s state during the process
Unambiguous rules in order to guide the process
Measure
Operator
Constraint
Behaviour
pattern

29. Generalization Behaviour
Provides the link between constraints, generalization operator and measures
Implements mechanism for choosing algorithms and parameters
Implements mechanism for choosing alternate scenarios in case of constraint failure

30. Generalization Behaviour Example for Line Simplification
ETL Generalization
Patterns
Reset to the original geometry
Determine an alternate
algorithm and/or parameters
Failed
Measures on
the feature
Apply the
algorithm on the
feature
Asses the results
through a set of
constraints
Determine the
algorithm to use
Success

31. Overview
Map generalization at NRCan
Principles of good map generalization
ETL generalization patterns

32. ETL Generalization Patterns
General and reusable solution to a commonly occurring problem in generalization when used in an ETL context
Three stages pattern
Meta-algorithm pattern

33. Three Stages Pattern
Used when generalization operator(s) are applied on different part of the same feature
Solution:
Create a pseudo-object representing each generalization operation to be done on the real objects
Evaluate the constraint against the pseudo-objects created in 1
Apply on the real objects the pseudo-objects that meet the constraint evaluated in 2

34. Three Stages Pattern Example
Example with amalgamator operator
If you implement a one stage process

35. Three Stages Pattern Example
Example with amalgamator operator
If you implement a three stages process
First create the amalgamation zones
Second you remove unwanted zone
Dissolve the area

36. Meta-Algorithm Pattern
Used when it is too complex to program the behaviour and the constraint in an ETL
Solution:
Select the algorithm(s)
Select the constraint(s) to implement
Select a behaviour pattern
Implement the solution in a high level language
Wrap the solution in a transformer

37. Meta-Algorithm Pattern Example
Example with Sherbend
Implementation of the Wang algorithm

38. Meta-Algorithm Pattern Example
Line simplification
Basic generalization operation
We all thought of the Douglas-Peucker algorithm to solve that problem…
Does it simulate the work of the cartographer well enough?
An example with contours

39. Meta-Algorithm Pattern Example
Contour line simplification
Red: Original contours
Black: Contours with DP 50m

40. Meta-Algorithm Pattern Example
Contour line simplification
Black: Contours with DP 50m
The results are unacceptable
Does it simulate the work of the cartographer? No
Douglas-Peucker line simplification is not doing a good job of generalizing natural features

41. Meta-Algorithm Pattern Example
Contour line simplification
Red: Original contours
Black: Contours with Sherbend algorithm with a parameter of 150m
The Sherbend algorithm will remove the bends below the tolerance and keep the ones above it

42. Meta-Algorithm Pattern Example
Example with Sherbend
Implementation of the Wang algorithm
Implementation of constraints
Self intersection
Line crossing
Sidedness

43. Meta-Algorithm Pattern Example
Implementation of constraints
Self intersection
Line crossing
Sidedness

44. Meta-Algorithm Pattern Example
Example with Sherbend
Implementation of the Wang algorithm
Implementation of constraints
Self intersection
Line crossing
Sidedness
Implementation of a scenario to resolve conflicts (behaviour pattern)

45. Meta-Algorithm Pattern Example
Scenario to resolve conflitcs
Implementation of an iterative process to resolve constraints

46. Meta-Algorithm Pattern Example
Example with Sherbend
Implementation of the Wang algorithm
Implementation of constraints
Self intersection
Line crossing
Sidedness
Implementation of a scenario to resolve conflicts (behaviour pattern)
Creation of a new transformer

47. Meta-Alorithm Pattern
Meta-Algrorithms can be developed by user in FME
FME Python extension (Python caller)
Open source libraries
Shapely for spatial manipulation and spatial relationship (
RTree for spatial indexing (
Already implemented the following meta-algorithms
Douglas Peucker
Convex
Spike
Smoothing
Sherbend

48. Future Works Develop new generalization patterns applied to ETL
Develop new meta-algorithms
Start the generalization of the hydrographic network

49. Thank You! Questions? For more information: Daniel Pilon
Natural Resources Canada