Chapter 2- managing GIS Data

Different how? Sometimes very large Often shared by multiple users Many sources, many file types Sometimes local, sometimes served Don’t always follow latest file conventions Formats can be complicated

Organizing GIS Data Use organized folders Be aware of where you are saving –The default location is not a good place! Name files with descriptive names –snailhabitat, not rastercalc1

Use folders Store data on the C:\drive –NOT the Desktop –NOT in your user folder Develop an organizational system and USE it Separate working folders from permanent data Put downloads in separate folders when assembling data

Naming conventions NEVER use spaces in folder names. Ever. Use letters, numbers, or underscore only Keep folder names short: ~8-15 characters –Not a requirement, but a good practice

Important Although Windows permits spaces in file and folder names, in GIS they are a BAD IDEA. They often work, but sometimes a certain program or function will fail if it encounters a space in a folder name. Real GIS users FLINCH when they see spaces anywhere in folder/file names—even when they’re allowed!

File extensions A 3-4 letter code that designates the file type –.doc,.docx,.xls,.xlsx,.mxd,.tif,.png,.jpg Change computer settings to make these visible Very helpful when downloading and importing data sets

Lots of data out there, but how to find it? ArcGIS Online (not all downloadable) State and federal government sites GIS Clearinghouses (store metadata) University/research organizations Some is great, some is worthless

Geo.data.gov

Searching Search in your favorite search engine for: –topic + GIS –topic + shapefile –topic + e00 –placename + GIS –placename + shapefile –placename + e00

Searching Try some of the clearinghouse sites Keep notes when you find a good site

Challenges Every site is different Many kinds of data formats Not all sites work as advertised Not all data downloads are valuable You need to develop a lot of general computer savvy You need to be willing to try, fail, explore, invent, search, learn

Zipping Many data sets are zipped and must be extracted before ArcGIS can use them Several zip utilities available –I like 7-zip and it is free –Download one and learn to use it

Source scale Data come at many scales Need to find data at a suitable scale for your project The purple, orange and black lines represent source scales of 1:25 million, 1:5 million, and 1:50,000, respectively.

Available scales Small scale data (world, country, state) are relatively easy to find over large areas Large scale base data down to 1:24,000 common in the United States Very large scale data (parcels) are difficult to find and very localized Specialized data (geology) typically only at small scales or localized special areas

Metadata Data about the data! Contains information about data needed to understand the data and evaluate its quality Should be provided with every data set distributed to the public May be stored as part of the data set or in a separate file

Metadata Useful for assessing the quality (fitness for a given purpose) Always look on web sites—it may be a separate download than the GIS data Download it and keep with the data

Metadata styles Item Description –Brief set of attributes –Quick to create –Default style Standards-based metadata –Extended information –Must set the metadata style to see it Item Description

To see more metadata Set the metadata style in Customize > ArcMap Options. Use FGDC or North American Profile for most US data.

Document downloads Update the metadata if you have changed the data set I like to put the source citation in the metadata credits so I know where it came from

Map documents and data frames The map document Stores collections of data for viewing and analysis Contains one or more data frames Stores properties for each layer (symbols, etc) Stores a page layout for printing Stores references to files-- not the actual data C:\mgisdata\usa\states.shp

GIS data are often shared…

Map docs and data files Same data can be used by many map documents Edits made in one document appear in ALL Shipping map document without its data is useless Changing locations of document or data can cause problems Files on disk Points to data

Data sources for layers Listed in Source tab Stored as pathname

Pathnames Absolute paths always start at the top of the data tree C:\mgisdata\usa\states.shp Relative paths start at the location of the map document..\usa\states.shp.. Means go up one level

Broken data links Occur when a map document cannot find data using the stored pathname because: Data were moved or deleted Drive is not available Map document copied to different computer Data not sent with map C:\mgisdata\usa\supersites.shp ?? D:\mgisdata\usa\supersites

When to use… Absolute paths –Data placed on central server for access by many –Data will not be moved or rearranged –You want to be able to transfer map docs without transferring data also Relative paths –When you plan to keep data with its map documents and move them together as a set –When you want to distribute maps/data to other organizations

Which type of pathname will still work if you…

File locks Programs place “locks” on files they use. Other programs cannot modify a locked file. To remove a lock, close the program using it. ArcGIS doesn’t always recognize when a lock has been removed Save, close ArcMap, and reopen to get rid of most file lock problems Reboot computer as last resort

The ArcCatalog Interface Folder tree Display window Menu/tool bars

Connecting to folders Shortcuts to frequently used folders or servers Must be added for drives other than C:\

Three view tabs ContentsPreview Description

Contents tab Viewing mode Large icons List Details Thumbnails

Description tab User can view metadata to evaluate data quality User can create, edit, import, or export metadata

Preview tab- Geography Zoom In Zoom Out Pan Full Extent Previous/Next Extent Identify Create Thumbnail Tools

Preview tab – Table

File Properties Right-click file name

The Catalog window Provides access to ArcCatalog functions within ArcMap Helps avoid file lock issues

Catalog window Access to most ArcCatalog functions within ArcMap

The Catalog window Open it here Right-click to manage

Previewing in the Catalog window Preview data and metadata

Warning! ArcCatalog and the Catalog window make permanent changes to stored files. They don’t warn you that they are doing so. They don’t have an “Undo” button. Be very careful when working in the Catalog.

Know the difference! Table of Contents Contains layers and properties Changes affect the map display only Catalog window Manages files and folders Changes are permanent

Assembling a geodatabase- Creating items Most items can be created by right- clicking the appropriate container and choosing the item Type in name while it is still selected and press enter Right-click a folder Right-click a geodatabase

Assembling a geodatabase Decide on coordinate system to use Search/download/find data sets Import to geodatabase format –Subset if needed using a query or clip –Project to coordinate system during or after import Update metadata

Import/export Same operation, only direction changes Import coverages, shapefiles, rasters, other feature classes Other formats might need conversion first (e00, CAD)

Using ArcCatalog to export Right-click the feature class to be exported Save in your geodatabase Enter query to select subset of features (optional)

Using ArcToolbox to export Open Feature Class to Feature Class tool Fill out as before (same tool)

Using a query during export

The Clip tool Use Clip to extract major roads within New Jersey

Changing the coordinate system If needed, use the Environments button on the tool to set the desired output coordinate system

Using ArcMap to export See what you are selecting with the query Convert the projection when exporting

1. Set the coordinate system 1.Set the data frame to the output coordinate system chosen for the new geodatabase Note: There are many different datum folders in the State Plane folder. Usually we choose the ordinary NAD 1983 datum.

2. Add the data set 2.Add the data set you want to export to the map. Rotated because the coordinate system is set for New Jersey

3. Query the table 3.Query the table (optional) to select a subset of the features

4. Export the data set 4.Export the (selected) features using the coordinate system of the data frame. 5.Save in your geodatabase

ArcToolbox features Hundreds of functions organized into toolsets Expandable by purchasing extensions. Runs in ArcMap or ArcCatalog Many functions in the toolbox are not accessible from the menus and toolbars. Create custom toolsets with your favorite tools Create your own tools

Using a tool Access info and Help Error tips Double-click

Searching for tools Search window Find tools based on name or keyword See where tool lives Click to open tool Hover for description

Background processing Lets you keep working while a tool runs, BUT Slower Confusing Crashes more Turn it off!

Using ESRI Data & Maps Comes free as DVDs/Downloads with purchase of ArcGIS Contact your administrator to find it Encourage its placement on a central file server Use the metadata to learn more about each data set and evaluate whether it is suitable for what you want to do

Contents Layer files Pre-symbolized layers for adding to maps. Spatial data containers Feature classes for exporting or adding to ArcMap.

Redistribution rights Data from many sources Licensed to Esri customers Static map view and printouts may be freely distributed Many data sets cannot be given to others Consult Help to learn which before giving copies to anyone Cite sources on maps ESRI Data and Maps (2013) [Download]. Esri, Inc., Redlands, CA.