Working with Field Data in ArcGIS ArcMap September 2016

Our Agenda Start with a brief lecture, covering: Some Theory: Datums and Projections Ways to Map Data: Vectors vs. Rasters Requirements for Formatting Coordinates (longitude/latitude) ArcMap Layout The Map Table of Contents window Catalog window Adjusting your map view Zooming Panning Changing Map Scale Going back to previous views Working with Map Layers Turning layers on and off Moving layers up and down in the menu Zooming to layers Opening and closing Attribute Tables and Layer Properties Identifying and selecting features Adding data

Our Agenda (continued) Continue with a hands-on exercise, including: Examine data Open ArcMap with Table of Contents and Catalog windows Create a File Geodatabase Add data to Table of Contents Plot data (longitude and latitude) Edit data points (spatial and tabular) Use Field Calculator

The Basic Theory

(2D representation of the earth) Datums & Projections Datum (3D frame of reference for measuring locations relative to the center of the earth) (converted to) Projection (2D representation of the earth)

Projections Various types of projections are used to represent the 3-dimensional earth on a flat (2-dimensional) surface. typically tailored to a particular purpose or geographic area. Try to minimize distortion in Shape, Area, Distance or Direction (SADD) Examples: State plane projections used to represent various U.S. states.

Ways to Map Data

Forms of Geospatial Data Data mapped using GIS takes one of two basic forms: Vectors Rasters The Nature of Geographic Information. (n.d.). Retrieved August 31, 2015, from https://www.e-education.psu.edu/natureofgeoinfo/

Vector vs. Raster Data Vectors Rasters Can take the form of Points Lines (e.g. roads) Polygons (e.g. areas or regions) Work well for Accurately representing the size and shape of entities such as buildings, political jurisdictions, real estate boundaries, roads, etc. Non-continuous data. Reduced disk storage space requirements Surface of map is divided into a gridwork of square cells, each of which is assigned a data value reflecting the characteristics of that particular square. Smaller cells result in a more detailed map, but also greater disk storage requirements. Work well for Representing continuous data. Avoiding the need for multiple feature types (points, lines and polygons) Supporting rapid computations and multi-variate data.

Formatting Coordinates Data

Express coordinates in decimal degrees. Longitude is: referred to as the “X” axis Positive in the Eastern Hemisphere, Negative in the Western Hemisphere Latitude is: Referred to as the “Y” axis Positive north of the Equator, Negative south of the Equator. For example, the Digital Scholarship Lab is located at: Longitude = -80.623192 Latitude = 28.065724 Obtained from ESRI Support, http://support.esri.com/technical-article/000012745

ArcMap Layout

Table of Contents window Catalog window Map Use “x” to remove window Use pushpin to secure window Use pull-down menu to show windows. Table of Contents window Catalog window Map

Adjusting your map view Zoom Change Map Scale Panning Previous Extent Hover over the icons to see descriptions

Working with Map Layers

Viewing Data Layers Data is displayed in layers on the map, which are listed in the Table of Contents Top layer on the map is at the top of the list in the Table of Contents. Check layer to view features. or Uncheck layer box to not view features.

Moving Layers To change a layers position in the “stack” of layers on the map, click on its name in the Table of Contents and drag it up or down in the list. (Table of Contents must be displayed as List by Drawing Order in order to move layers up and down)

Attribute Tables and Layer Properties Right-clicking on a layer name in the Table of Contents is a quick way to: Open the layer’s Attribute Table Zoom to the Layer on the map Export the data in the layer Access the layer’s Properties. All of these are commonly performed functions.

Identifying and Selecting Features Use the “Select Features” tool to select one or more features on the map Use the “Identify” tool to pull up information on one or more features on the map

Adding Data Two ways to load data into ArcMap: Click on the Add Data icon at the top of the page (looks like a plus (+) sign superimposed on a yellow diamond). or Click-and-drag files from the ArcCatalog window into the map area or Table of Contents.

Adding Data Using the Add Data icon

Adding Data Using the Catalog window Expand the directory containing the data. Click-and-drag the desired data from the Catalog window to either the map area (in the center of the screen) or the Table of Contents (on the left side of the screen)

Hands-on Exercise For this part of the workshop, please log onto one of the 5 GIS workstations located along the west wall of the Digital Scholarship Lab.

Hands-on Exercise Part 1: Examine Data

View Data in CSV Open the data file located at: C:\GIS_field_data\public_housing.csv Expand the columns to see all the data Things to notice: Separate columns for Latitude and Longitude There is a single header row – every column has a name There are no spaces in the column names – use underscores None of the column names starts with a number There are no merged cells or elaborate formatting Close the CSV file. **Very important

Hands-on Exercise Part 2: Open ArcMap with Table of Contents and Catalog windows

Open ArcMap ArcMap is the primary component of ArcGIS for making maps Click on the Windows Start icon Click on All Programs Expand the listing for ArcGIS and click on ArcMap Click “Cancel” on the “Getting Started” window. If you had pre-existing map documents (MXDs) or worked with templates, you would select appropriately here. We just want an empty map to work in today.

Table of Contents and Catalog Windows If not already open, open the Table of Contents and/or Catalog windows. Hint: use the “Windows” pull-down menu. Connect the Catalog window to your working directory. At the top of the Catalog window, click on the Connect to Folder icon (looks like a folder with a plus (“+”) sign on it). Expand the Computer list and navigate to C:\GIS_field_data. Click on “GIS_field_data” and then on “OK”

Hands-on Exercise Part 3: Create a File Geodatabase

Create a Geodatabase In the Catalog window, navigate to the C:\GIS_field_data folder and right-click. Select “New” and “File Geodatabase” Name the geodatabase “Public_Housing” If you initially name the new geodatabase “New File Geodatabase.gdb” (as below), right-click on it and select “Rename” to correct the name. You now have a storage place for your data.

Hands-on Exercise Part 4: Add data to Table of Contents

Adding your Data Click on the Add Data icon. Navigate to your working directory (C:\GIS_field_data) Select your CSV file Click on “Add”.

Alternate Method to Load Data In the Catalog window on the right side, Expand the C:\GIS_field_data folder Click-and-drag public_housing.csv to either the map area (in the center of the screen) or the Table of Contents (on the left side of the screen) Why can’t we see the public_housing data on the map? The public_housing data is in a .csv (comma-separated values) file rather than a shape file or feature class. Therefore, data is not displayed on the map automatically. You will plot these points in the next step.

Hands-on Exercise Part 5: Plot data

View the Data Table Open the Attribute table for the Public Housing data to verify the fields containing the longitude and latitude of each point. To open the data table, Right-click on public_housing.csv in the Table of Contents Select “Open” Inspect the contents of the table The coordinates are in columns named “longitude” and “latitude”. You are looking at the same data you viewed in Windows Explorer, but here we are going to use the data to plot points. Close the table Before using data, you have to know the coordinate system used when the data was collected. In this case, the coordinate system is “WGS 1984”.

Display XY Data Right-click on public_housing.csv in the Table of Contents. Select “Display XY Data…” In the Display XY Data window, Set the pull-down menus for “X Field:” and “Y Field:” to the appropriate column headings in the data table “X Field:” is used for the longitude, while “Y Field:” is used for the latitude. ArcMap will often “guess” at the correct column headings if the headers are labeled appropriately. You have to assign the coordinate system for the data. Click on “Edit…” to open a Spatial Reference Properties window (for entering the correct coordinate system) In the upper part of the window, expand the “Geographic Coordinate Systems” list Expand the “World” list Select “WGS 1984” Confirm that GCS_WGS_1984 is displayed in the “Current coordinate system:” window. Click on “OK” In the Display XY Data Window, click on “OK”. If you get a “Table Does Not Have Object-ID Field” message, click on “OK”

Viewing the Housing data The ArcMap Table of Contents should now have a new layer: public_housing.csv Events. Note the “Events” at the end of the layer name. The layer indicates the symbol associated with the points plotted on the map. An Events layer is only temporary and cannot be used outside this specific GIS model file. To save the layer permanently: Right-click the Events layer Select Data  Export Data In the Export Data window, click on the browse button. Navigate to your C:\GIS_field_data folder. Set “Save as type:” to “File and Personal Geodatabse feature class” Double Click on Public_Housing.gdb Be sure the “Look In:” field looks like this Change the name to “Public_Housing” Select Save, followed by OK When asked if you want to add the exported data to the map as a layer, select “Yes”. You will then see the new “Public Housing” layer in the Table of Contents. Be sure you use an Underscore in the name: Public_Housing

Let’s add a background map When displaying data, having a map in the background helps place the data in context. Click on the pull-down menu to the right of the “Add Data” icon (plus sign on top of a yellow diamond). Choose “Add Basemap…” Choose a basemap. Try adding the Light Gray Canvas basemap. Click “Add”.

View the data The data displayed on the map is contained in Attribute Tables. Try this: Right-click on “Public_Housing” and select Open Attribute Table. This is the data table that was used to create the points and populate the “Identify” table in the next slide. Close the Attribute table.

Hands-on Exercise Part 6: Edit data points

What is that? Right-click Public_Housing and Zoom to Layer. Select the “Identify” tool (circle with an “i” in it) Using the Identify tool, click-and-drag a box around the points to the SE of Hawaii. The Identify box gives you information on the 4 points. Where should these points actually be? You are going to move the points to the correct general area.

Getting ready…. Close the “Identify” box. Add the “GuamArea” Shapefile to your map. (C:\GIS_field_data) Add the “World Street Map” as a Basemap and uncheck the Light Gray Base Map “Zoom to Layer” on the World Street Map. In the Table of Contents, change the symbology of the “Guam Area” so there is no fill and the outline line is thicker. (Double click the symbol to access the “Symbol Selector” Uncheck or right-click and remove the “Events” layer.

Move the points to Guam Right-click in the general blank area at the top of the screen in order to access the toolbars. Select Editor From the drop-down menu on the left side of the Editor toolbar, select “Start Editing.”

Editing Select “Public_Housing” to edit and then “OK”. Using the Edit tool on the Editor toolbar, draw a box around the points that are incorrectly located to the SE of Hawaii. Click-and-drag the highlighted points with the X to the far right of your map where the “Guam Area” is shown as a box. The points will ‘snap’ to the box.

General Editing… You just moved 4 points to the general area of Guam. Select the Magnifier and draw a box around the Guam Area so you zoom in closer.

More Editing….. Notice that your points are probably along the edge of the “GuamArea” shapefile because of a “Snapping” Setting that helps with Editing. On the Editor Tool, click on the “Edit Tool” icon again and then move each point individually to the Guam island.

Attribute Editing Since we have moved the points, the Latitude and Longitude in the attribute table are no longer valid. We need to remove those values from the attribute table. For this exercise, Click on the “Attribute” icon on the Editor Toolbar. You will select each point individually and delete the values in the Latitude and Longitude fields. (You could also edit in the attribute table by accessing it in the Table of Contents.)

Attribute Editing….. The Attribute entry for your selected point will appear to the right of your map. Highlight each of the Latitude and Longitude numbers and delete the values. Do this for each of the 4 points you have placed on the island of Guam.

Almost done…. Save Often! After you have deleted the Latitude/Longitude values for these 4 points, go back to the Editor toolbar and use the dropdown to “Stop Editing.” Select “Yes” to save your edits. If you feel you made a mistake, select “No” and start a new Editing session to correct the points. At anytime during the editing process, you can “Save Edits”. Don’t wait until the end to “Save Edits” if you have a large dataset. Save Often!

Accuracy of Edits Due to our time constraints, you only moved the points to the Guam island, which isn’t really accurate, but this gives you a starting point for more editing. The address is in the attribute table, so given more time, you could locate the correct streets and move the points to the exact locations. You would need to zoom in very close to the street level and move the points to the real location based on the address. This can be a time consuming effort, so plan ahead when you have a dataset that needs to be checked for accuracy!

Hands-on Exercise Part 7: Use Field Calculator

Add Field Data Notes Since the Guam points aren’t actually on the street locations and you can’t just delete the address, create a new field that contains a note regarding the inaccuracy. In the Table of Contents: Right-click on “Public_Housing” and select “Open Attribute Table”. From the “Table Options” pull-down menu at the top of the Attribute Table, Select “Add Field”

Add Field Type “Notes” as the Name of the field Change the “Type” to TEXT. Change the “Length” to 150 so you don’t run out of character spaces for your notes. “OK” In the Attribute table, move to the right side to see your new field. All the records are automatically populated with a <NULL> value.

Select Data Points Select the 4 points in Guam: At the top of the Attribute Table, click on the “Select by Attributes” icon. Be sure “Method” is “Create a New Selection.” Double-click “STATE_NAME” so it appears in the bottom section Single-click the Equal (=) sign Select the “Get Unique Values” button Scroll to “GU” and double-click on it Once the formula is entered as shown, click “Apply”

Selection You should have 4 out of 5397 records selected. Toggle between the “Show all Records” and “Show Selected Records” buttons at the bottom of the Attribute Table.

Field Calculator Use the Field Calculator to add attributes to selected records in a dataset. Be sure only the 4 records are selected. Right-click the “Notes” heading and Select “Field Calculator” “Yes”, calculate outside an Edit session. In the empty box, type: “Points moved to general area only. Not accurately placed yet.” Be sure to enclose the words with quotes. Click on “OK”.

Field Calculator Result of Field Calculator: To Populate remaining fields, use the “Switch Selection” tool at the top of the Attribute Table to automatically select the other 5393 records.

Field Calculator Use the Field Calculator to add attributes to selected records in a dataset. Be sure the 5393 records are selected. Right-click the “Notes” heading and Select “Field Calculator” “Yes”, calculate outside an Edit session. Change the words to: “Have not verified.” Be sure to enclose the words with quotes. Click “OK”.

Selected Records and Calculations Important: If you have records selected within any of your data layers, any calculations performed will only apply to the selected records. All unselected records will be unaffected by the calculations. If you no longer need to have records selected, it is always a good idea to unselect your records.

Unselecting records At this time: 2 ways to unselect records Click on the “Clear Selection” icon from within the attribute table (unselects only those records selected within the specific data layer) Click on the “Clear Selection” icon from the main ArcMap toolbar (unselects all records selected in all data layers. At this time: Clear all selections in any data layers by clicking the “Clear Selection” icon on the ArcMap toolbar. In the Attribute table, confirm there are 0 records selected.

Sorting Double click the “Notes” heading to sort the field. Scroll to the top. Double Click the “Notes” heading again. By double clicking the heading, you are sorting by Ascending and Descending order. You should see that only 4 records have a note about the Points being moved.

ZIP Codes Move to the “ZIP” field and double click on the header to sort. Note that there are 4 records without any ZIP code information. Many records have only 3 or 4 digits for a ZIP code. Those records are missing the initial “00” or “0” to complete the ZIP code. You will use “Field Calculator” to correct this format problem.

ZIP Codes Right-click the “ZIP” heading and select “Properties...” Note that the field TYPE is “Long”, which is a numerical field type. ArcMap has therefore ignored any leading zeros in ZIP codes. ZIP codes are a special number and can be treated as text. Close the “Field Properties” box.

ZIP Codes In the drop-down menu for the attribute table’s “Table Options” tool, select “Add Field.” Name: ZipText (no spaces in name) Type: Text Length: leave at default 50 OK

Populate ZipText field Move to the right end of the attribute table. Right-click on the “ZipText” heade3r and select “Field Calculator”. “Yes” for outside an edit session. Double Click “ZIP” in the upper box so it populates the lower box. Select OK. By putting the values into a text field we can add the leading 0’s to the zip code in the next few steps.

ZipLength Field Calculation Add another new field: Name: ZipLength Type: Short Integer OK Right-click the field name, select “Field Calculator.” Delete the existing code in the bottom box. Set “Type” to “String” Double Click Len( ) in Functions Double Click “ZipText” in Fields

ZipFinal field Sort the ZipLength field and note that there are a few records without values. Most records have either 3, 4 or 5 digits. Add a new field: Name: ZipFinal Type: Text Length: leave default 50 “Select by Attribute” the records with a ZipLength of 3 (Be sure to delete the previous selection equation in the lower box of the “Select by Attribute” window. Should select 153 out of 5397 records.

ZipFinal field With the 153 records selected, right-click the ZipFinal field and select “Field Calculator.” “yes” to outside an edit session Delete any previous previous formula in lower box. Type in the box: “00” - be sure to include quotes. Select the “&” button In the Fields box: Double Click “ZipText” OK

ZipFinal field Scroll down the attribute table and verify that only zip codes with a length of “3” have a value in the “ZipFinal” field. The following steps will populate the field for ZipLengths of 4 and 5. “Select by Attribute” the records with a ZipLength of 4. Be sure to edit the previous selection equation in the lower box of the “Select by Attribute” window. Should be 501 out of 5397 records.

ZipFinal field With the 501 records selected, right-click the ZipFinal field and select “Field Calculator.” “yes” to outside an edit session Edit the previous formula. Change “00” to “0” OK Scroll down the attribute table and note that ZipFinal now has values in records with a ZipLength of 4.

ZipFinal field “Select by Attribute” the records with a ZipLength of 5 Be sure to edit the previous selection equation in the lower box of the “Select by Attribute” window. Should be 4739 out of 5397 records. With the 4739 records selected, right-click the ZipFinal header and select “Field Calculator.” “yes” to outside an edit session Edit the previous formula so only [ZipText] appears OK Clear selections

Concluding Remarks This exercise has covered some of the more common aspects of using ArcMap to analyze field data, including: Plotting data Editing data Performing calculations based upon data. If there are any other aspects of ArcGIS use that you would like to see addressed in future Digital Scholarship Lab workshops, please let your instructor know.

Getting Help! If you are unsure how to proceed, ArcGIS has extensive Help Resources.

Want to learn some more? Contact me!! Rob Sippel rsippel@fit.edu 321-674-7585 Register with ESRI (the company that makes ArcGIS) It’s free!!! https://accounts.esri.com/signup?redirect_uri=http%3A%2F%2Ftraining.esri.com%2F gateway%2Findex.cfm%3Ffa%3Dsearch.results Come by the library and try out our GIS workstations (3rd Floor –at least for the time being) Take some ESRI tutorials. http://training.esri.com/gateway/index.cfm?fa=search.results&searchterm=&search= Search&cannedsearch=1&orderby=Price%20asc (some are free, others aren’t) Download some geospatial data and play with it. Consult GIS Research Guide (Rob Sippel) http://libguides.lib.fit.edu/GIS

License Use Restrictions The library’s ArcGIS licenses are authorized for use in teaching or performing research, as spelled out in the following sources: Rubric for determining permitted uses under Educational Site License Program (https://maps.cise.jmu.edu/lic/Esri%20Educational %20Site%20License%20-%20Permitted%20Uses.pdf) Esri Educational Site License Program description (https://www.esri.com/~/media/Files/Pdfs/industrie s/university/academic_programs/pdf/site-license- overview.pdf)

Questions?