From Oracle Spatial to DGN Using FME Server and WeoGeo.com Jeff Hobbs GIS Coordinator San Jose Water Company 408.279.7833 (US) jeff_hobbs@sjwater.com Twitter: @jeffhobbs Blog: www.jeffhobbs.net Jeff Hobbs GIS Coordinator

What We’re Currently Doing (in Short) SJWC currently uses Intergraph’s Modular GIS Environment (MGE) to maintain the majority of its GIS graphics. MGE is akin to the pre-ArcGIS version of ArcInfo. It’s old school and limited in functionality. SJWC is in the process of migrating our GIS to an Intergraph GeoMedia/Oracle Spatial system. One of our reasons for not migrating in whole up to this point has been our reliance on Intergraph Field View. Field View is based on an Intergraph MGE dump. Until we eliminated our use of Field View, we’ve needed to keep our GIS in MGE. People currently generate MicroStation DGN files using Field View. That’s what this slide is showing. MGE  MicroStation with Field View as the “middleware” piece. This is further illustrated in the next few slides. Intergraph MGE

Data Maintained in Intergraph MGE Intergraph MGE is uses to maintain our GIS data. MGE is a tile-based GIS. Intergraph MGE (Tiled Dataset)

Weekly Exported To… Weekly Export Every week we take our Intergraph MGE project and create an export file. This file is used by Intergraph Field View. Weekly Export

Intergraph Field View Intergraph Field View Intergraph Field View is/was the primary GIS data viewing application used at SJWC. Our Geographic Data Integration (GDI) Portal has since replaced the majority of the functionality provided by Field View for users who are on the network (as opposed to field-based users). One of the pieces of functionality we’ve yet to replace is the ability for our Engineering Department to export to MicroStation DGN from Field View. This presentation discusses SJWC’s use of WeoGeo.com’s SaaS solution to provide this export functionality inside of the GDI Portal. Intergraph Field View

Field View Sample Seamless Dataset Field View looks like a giant seamless CAD file. When you click on a graphic, the majority of the graphics have attribution behind them. This is a screenshot of the map in Field View. NOTE: I’ve blurred the screenshot as this is sensitive data. The Field View functionality is akin to something like ArcExplorer. The export file that Field View uses is created by Intergraph MGE. Seamless Dataset

Field View Export to DGN This screenshot shows the Export to MicroStation DGN inside of Field View. The point here is simple – this is EASY. This is a good thing. Whatever we create to replicate this process in the GDI Portal needs to be as simple as possible.

Result DGN MicroStation V8i Levels without Names & All GIS Layers This is a sample DGN export from Field View. Again, we’ve blurred for security reasons. The result DGN file does not have level names. The user also has _no_ control over what GIS layers in Field View are exported. The export automatically exports every layer that’s in the view. You have no control over what layers are exported; on the bounding box of data that’s exported. Levels without Names & All GIS Layers

Where We Want to Go… Internal GIS Portal The goal is to create the same DGN file from the GDI Portal directly. The back-end database that feeds the GDI Portal is an Oracle Spatial database. Internal GIS Portal

Only Selected GIS Layers Requirement (1) MicroStation V8i The requirements of the DGN file that we’ll be creating from the new process are: It be natively in MicroStation V8 file format (the export from Field View is in V7 that needs to be upgraded to V8) The levels all have level names Only certain GIS layers be exported Levels all Have Names & Only Selected GIS Layers

Requirement (2) The export process must be EASY. As simple if not more simple than the Field View process. (Easy)

Option (1) Our first option was to look at Bentley Map. This is Bentley’s newest GIS. It natively reads Oracle Spatial data and can easily create the DGN file. SJWC also owns a limited number of concurrently licenses.

Bentley Map Now that’s some functionality!! After doing research on Bentley Map we found it to definitely be full featured. The screenshot shows just some of its functionality. We would be putting a full GIS into the hands of our users. Now that’s some functionality!!

Bentley Map… Not easy to use nor setup After doing additional research, it turned out the Bentley Map, although powerful, was going to offer us a number of limitations. A few of them are: It was not going to be easy to setup by an administrator (me or my staff) The GUI isn’t all that intuitive This is a full-fledged GIS. Good if you’re a GIS person, bad if you just want a simple export Although there’s now a standard for annotation inside of Oracle, very few vendors actually take advantage of the functionality. As a result, we were not going to be able to use our text inside of Bentley Map. This meant that we were going to have to dynamically generate labels. Not a very good option…especially for folks that love labels. Not easy to use nor setup

Option (2) Option 2 – FME Server

FME Server… On the positive side, it would be easy to call from an end-user stand point. There would be some setup by an administrator. But once that setup was completed, the end user could easily be exposed to a _very_ simple interface.

FME Server… On the down side…FME Server is _not_ cheap. NOTE: Although this is not cheap, the software, in my opinion, is well worth the money. It’s just not an easy ROI for the type of application we’re developing. Down the road, we may look to purchase this software and bring this ETL process in-house.

What We Need… What SJWC needed was EASY + CHEAP

Easy + Cheap = WeoGeo.com So… EASY + CHEAP = WeoGeo. WeoGeo offers FME Server in the cloud as a Software as a Service (Saas) solution Software as a Service (SaaS)

How it Works… Other Data This slide shows how the WeoGeo process works. FME Desktop reads our Oracle Spatial database along with other data sources. FME Desktop then run this data through a workspace and the output is a FFS file. In the same workspace, the result FFS is uploaded using Python along with a FME Server workspace and a CSV schema file up to Amazon storage. On Amazon lives FME Server. FME Server (run by WeoGeo) uses the FME workspace that was uploaded using Python, the schema CSV, and the dataset dump FFS file and creates DGN files as needed. Other Data

The Data Explained… All data use by FME workspace to create DGN The workspace run by FME Server to create the DGN This slide goes into detail about the functionality of each of the three files that are uploaded as part of the ETL process using Python (this expands upon the previous slide) A “configuration” file used by the FME workspace – symbology, level, name…

The GUI… This shows the GUI inside of the GDI Portal. If you are NOT logged in, you DON’T have access to the Export tool. We do this because we only give certain staff members the ability to export to DGN. This is based on Active Directory group membership.

The GUI… Once you are logged in, if you have the right privileges, you will see a button appear in your toolbar. This button will allow you to export to DGN. If you do not have the right privileges, you will not have a button in your toolbar, regardless of being logged into the GDI Portal. NOTE: In this case, the user is zoomed out to a very small scale. If you are zoomed out too far, we disable the Export to DGN tool. We do this so the user doesn’t look to export a humongous DGN file and bog down the system for a great length of time.

The GUI… If the user has the right privileges and is zoomed in reasonably close, the Export to DGN button is enabled.

The GUI… When you push the button, a bounding box is passed to the WeoGeo API. The API then calls FME Server. FME Server then accepts the bounding box and runs the FME workspace that was uploaded as part of the Python script. On the client side, we run the user though a Flickr photo stream to keep them busy while the processing is happening on the server.

The GUI… After the server processing has been complete, the client is notified. Upon notification of the export being complete, the URL is returned from the WeoGeo API. This URL allows the user to download the DGN file that was created by FME Server.

Only Selected GIS Layers Result MicroStation V8i Here’s the result of the export. This is a native MicroStation V8 DGN File The levels all have names We only export the GIS layers that are useful for this one business process Levels all Have Names & Only Selected GIS Layers

Thank You! Questions? For more information: Jeff Hobbs San Jose Water Company jeff_hobbs@sjwater.com http://www.sjwater.com @jeffhobbs Blog: http://www.jeffhobbs.net Jeff Hobbs GIS Coordinator San Jose Water Company 408.279.7833 (US) jeff_hobbs@sjwater.com Twitter: @jeffhobbs Blog: www.jeffhobbs.net Place your logo here