1. QA/QC: A Checklist for Quality and Control
Hello my name is Robert Borchert.
Thank you for joining this session this afternoon.
Today I am going to talk to you about a subject that I feel should be near and dear to just about everyone's heart, and that is Quality Assurance and Quality Control.
This presentation is not going to be about a nuts and bolts approach to QA/QC, but rather it is going to be about my companies approach to QA/QC and how we went about developing our program.
Robert D. Borchert
GIS Technician

2. Biographical Information
GIS Technician for seven years
Major – Geography, Minor in GIS
E911 and Electrical Distribution GIS
Currently a GIS team member with United Services Group in Elk River, Minnesota
Project Coordinator
MapInfo, AutoCAD Map, ArcGIS a
First just a little bit about me.
I have been in the GIS field since I have a degree Geography from Saint Cloud State University in Minnesota with a minor on GIS
I started out developing GIS for E911 on a county wide basis and have successfully completed two complete upgrades to E911 and assisted on several other project.
I currently work for United Services Group in Elk River, Minnesota. We are a department of the Transmission Division of Great River Energy.
The GIS team at United Services Group concentrates on developing and maintaining GIS of Electrical Distribution for cooperatives. We currently serve a good share of the cooperatives in Minnesota, but we also have clients in other neighboring states. We also provide services for American Samoa Power Authority in the South Pacific.
My duties at United Services Group include Project Management and GIS development.
My GIS software experience includes MapInfo, AutoCAD Map, and ArcGIS

3. QA/QC Discuss QA/QC Developed a checklist Developed a procedure
What is QA/QC?
What QA/QC is not
Developed a checklist
Developed a procedure
Examples of our QA/QC checklist and things to be aware of in QA/QC
Today I am going to cover a variety of subjects all centered around Quality Assurance and Quality Control.
I will start of with a discussion of QA/QC and what it means to my organization.
I will talk about what QA/QC is and what QA/QC is not
Our organization developed a Checklist for QA/QC which I will share the developmental concepts behind that checklist and the procedures we used in it conception.
I will also share some examples of our checklist and some things I feel you should be aware of when implementing QA/QC
This discussion will not be software specific, however I am more than willing talk about specific procedures with anyone after the presentation.

4. Do you need QA/QC? Fueling your car? Submit a Papar
Diesel or Gas?
Submit a Papar
Czech spelling
Create an Enterprise GIS
QA/QC
Would you put diesel fuel in your gasoline-powered car?
Would you submit a paper to GITA without first checking the grammar and spelling?
How many of you noticed the incorrect spelling on this slide. I bet it detracted from what I was saying and for some of you it lowered my legitimacy.
Would you ever create an Enterprise Geographic Information System for your agency or a client and not perform Quality Assurance and Quality Control (QA/QC) on it?
Of course, you would not.
However, some people do. Maybe it is just that they do not have the proper methodology, tools, or the knowledge to implement QA/QC. For just about every GIS application involving computer technology there are certain functions that come with the software that provide some sort of QA/QC

5. Do you need QA/QC? F 7 Fueling your car Submit a Paper
Separate Pumps
Labeled
Submit a Paper
Spell Check
Grammar Check
Check
Create an Enterprise GIS
Not a lot of built in tools
No universal answer for QA/QC
Fueling your car
Separate Pumps
Labeled
Submit a Paper
Spell Check
Grammar Check
Czech
Create an Enterprise GIS
Not a lot of built in tools
No universal answer for QA/QC
F 7
In the example of putting diesel fuel in your car, many gas stations have a separate pump for diesel and/or have the pump noticeably labeled.
In addition, your car may have a little sign near the gas cap indicating which fuel you should be putting in your car. This is a form of QA/QC.
In the example of submitting a paper or thesis without checking spelling and grammar, many word processing programs have a spell checker and grammar checker built right into the program.
However, a spelling and grammar checker may not be enough, as you can see Czech is a legitimate word and may not be caught
In writing this paper, I probably ran the spelling and grammar checker hundreds of times
However, when it comes to an enterprise GIS there may not be a lot in place to perform QA/QC depending on your software.

6. QA/QC for GIS Built in QA/QC Significant Tools
There is no QA/QC button or hot key
How you use the tools you have
Educate the GIS Technicians
What to look for
Knowledge of existing tools
Checklist
Procedures
Many GIS programs have tools in place that may be used, but you may not be aware of them or they may not be comprehensive enough to cover your needs adequately.
I have significant experience with three different GIS software applications and have yet to find any one single tool in the software that will perform acceptable QA/QC. There is no QA/QC button. There are tools out there that you can use to perform QA/QC and it is just a matter of how you use them.
We have found that the best approach to QA/QC is a combination of
Educating the GIS technicians and users as to what to look for.
Knowledge of the tools that your software contains.
And a QA/QC checklist for the user to follow.
This rest of this presentation will give a broad perspective of the QA/QC checklist that my company has assembled for use in developing and maintaining GIS for electric distribution utilities
However this discussion should be applicable to any industry with some modification.

7. What is QA/QC
From the "WordNet: An Electronic Lexical Database" following are defined as:
Quality: “a degree or grade of excellence or worth”
Assurance: “freedom from doubt”
Control: “the activity of managing or exerting control over something”
From these definitions, I take:
Quality Assurance/Quality Control: “An activity intended to give something a level of excellence that you know will work”
Lets start off defining QA/QC
The "WordNet: An Electronic Lexical Database" , which is an online dictionary, defines the following as such:
Quality: “a degree or grade of excellence or worth”.
Assurance: “freedom from doubt”
Control: “the activity of managing or exerting control over something”.
From these definitions, I take:
Quality Assurance/Quality Control: “An activity intended to give something a level of excellence that you know will work”

8. How does this apply to GIS?
Standardized Methods
Checks
Balances
Corrective Actions
Produces a Stable Product
Correct
Consistent
Reliable
How I feel this applies to a GIS is that the
QA/QC program should be a standardized method of checks, balances and corrective actions that produce a stable product that is correct, consistent across its breadth and can be relied on the give the correct information

9. Accuracy and Precision
US Navy School
Accuracy & Precision
Accuracy in determining the correct details
Precision on how well you can make those determinations every time
When I was attending the US Navy Aviation Machinist Mate School one of the first things our instructors taught us was precision and accuracy.
My instructors defined accuracy as how close you get to the mark,
and they defined precision as how well you could perform that operation every time.
The same principals apply to QA/QC in that you are accurate in determining the correct information, and that you are precise in doing it consistently

10. QA/QC Checklist… Not a static document Change and grow Start Small
Big picture items
Grow your checklist
New situations
Concerns from customers
Encourage new ideas and methods
A QA/QC checklist should not be a static document;
it needs to change and grow as you and your organization gains experience and grows. Our checklist started small and grew larger and larger as we encountered new situations or data concerns from our customers.
We encourage new methods and sharing of ideas during meetings to help foster new QA/QC techniques and discrepancies.

11. QA/QC Checklist… Twelve main categories
Documentation and File Structure
Project Checklist
Data Checklist
Data Structure Checklist
Attributes Checklist
Data Conversion Checklist
Network Checklist
Relationship Checklist
Annotation Checklist
Custom Applications Checklist
Wall Map/Map Book Checklist
Project Checklist for Clients
Our current checklist, which is the focus of this paper, consists of twelve main categories, which each have a number of subcategories. As a reminder, this checklist is for use in electrical distribution GIS systems that we have converted from one format to another or maintain. You may note that several items in our checklist appear to be redundant. But we will discuss the differences
The main categories for the QA/QC checklist are:
Documentation and file structure
Project checklist
Data structure checklist
Data checklist
Attributes checklist
Data conversion checklist
Network checklist
Relationship checklist
Annotation checklist
Custom Applications checklist
Wall map/map book checklist
Project checklist for clients

12. Documentation and File Structure
All pertinent documents
Contain a brief description of the project
What client requested
How it was completed
Original abstract
Outline of steps taken
Metadata
Standardized file structure
Any other documentation
Spatial references
Specific output types
Any paper documents used
Documentation and file structure
Documentation should include all pertinent documents generated during the project and should contain a brief description of the project, a listing of what the client requested and how it was completed. The documentation should include a copy of the original project abstract and an outline of the actual steps taken to complete the project. Documentation (metadata) that should be included are things such as file structure, data specifics, output specifics, deliverables and any other documentation that will help the end user understand their GIS. It may be important to consider having a standardized file structure in place for your organization. In our organization, this means that all folders for a client are stored in the same way on our server. Each client will have a unique folder and then an identical subfolder structure for all clients. For example, on a project for ABC Cooperative, their folder on our working drive would be “I:\ABC_Cooperative”, and the sub folders under this folder would be the same for all other clients that we work with. It is nice to have standardized file structure on your server and on the client’s computers so that anyone who is not completely familiar with a given project can still provide customer support. When checking the file structure you should clean up any extra folders/files created for a project but are not necessary or finding files that you can combine into one. I will discuss file and data structure in more depth a little later. I would caution you to make a backup of your data first, to prevent accidentally deleting something that was actually important, and consulting with everyone who actually worked on a project.
Other important documentation that should be included, if applicable, are spatial reference, scale intended for use, specific output types, any paper documents used to create the project, and a list of all deliverables sent to the client.

13. Project Checklist Relevant Name Opening the Project
Open with no errors
Desired location
Extent of system
Specific location
Correct drawing order
Layer States or definition queries
Symbols
Does the GIS do what it is supposed to do?
Project Checklist
This category of the checklist pertains mostly to the cosmetic qualities, operational characteristics and reference documents for the project.
First in this category would be to ensure the name of the project is relevant. The name of the project you deliver to your customer should be relative to what it is, as well as easily understood by the end users. For example, during the creation of the ABC_Cooperative GIS, I would probably have a project file called ABC_Cooperative_Roberts_Anno, which is the file I was using when I created annotation. Well, since ABC_Cooperative is going to be using this GIS for a variety of operations relating to their cooperative this would not be a relevant name. We would use a name like “ABC_Cooperative GIS”.
Other performance options to look for would be to make sure the project opens with no errors and that data has the correct path to the source. Check to ensure that the printers you have on your system are not included, this could prevent an error message when the end user starts their GIS. Does the project open to the desired location (i.e. zoomed to the extent of the system or other location)? Do layers draw in the proper order and do the symbols match what the client wants? For example, should the conductor draw on top of the fuse or beneath it?
Can your GIS use layer states or definition queries? If it does you should check that they are performing as desired and have them documented for the end user so they know how the GIS is supposed to react. For example, ABC_Cooperative has three different scales at which they view customer locations: township, section and quarter section. When at the township level they do not want the section and quarter section detail customers to be visible until you zoom in to a predetermined scale.

14. Data Structure Checklist
Relevant Data
Correctly Named
Naming Conventions
Proper Location
i.e. C:\AMFM\ABC_Cooperative
Pre-Defined Model
Other billing or staking programs
Data Structure Checklist
In this area of our checklist, we check to ensure that all the data in the GIS is relevant, named correctly, and is in its proper location. Sometimes during the creation of a GIS, the technician may give a temporary name to a file for a specific use, or as a reminder. During the creation of the ABC_Cooperative GIS, I had a temporary buffer around the street data that I used to snap conductor. What we stress here are file naming conventions, non-duplicated data, and that data is of the correct type. If you are using a pre-defined model, you should ensure that your GIS conform to it. This is important for ABC_Cooperative, as they need their GIS to work with their billing system.

15. Data Checklist Data in Correct Layer Viewable when it should be
All fuses in fuse layer
All transformers in transformer layer
Etc…
Viewable when it should be
Document
Data Checklist
The data checklist will utilize the GIS software itself and focuses on cosmetic appearances and usability. During this phase, we check to ensure that the data is correctly drawn and is consistent. We want to make sure items appear in the correct layer. Are switches in the switch layer, or do some appear in the fuse layer? Make sure all your data sets are viewable when it they should be and are editable if so desired.

16. Attributes Checklist Actual Data Symbolize from an attribute
Is that field value complete?
Subtype or block reference?
Label from an attribute
Document
Attributes Checklist
When checking the attributes you will be getting into the actual data itself. Make sure that the attributes you symbolize from are complete and if those attributes use a domain or block reference that they are complete. Many GIS systems have the ability to label from the data itself, these attributes are important as well.

17. Data Conversion Checklist
Converted or imported to GIS from another platform
Check for lost data
One to one match
Proper Conversion
Polygon to polygon
Point to point
Polyline to polyline
Same relative location
Document
Data Conversion Checklist
This section pertains to a GIS that you created from another platform or that has had new data imported from another system. Checking the data conversion properties is one of the areas that you may find yourself performing quite often during the creation or maintenance of your GIS. The GIS technician should document data that was not converted and why. Check for lost data and that there is a one to one match of old verses new. Polygons converted to polygons, were all points converted to points, etc? Did features appear in the same relative location as they did before? Do features have unique ids if necessary?

18. Network Checklist Connectivity Loops Direction of Flow
Sources or sinks
Rules in place and working
Document
Network Checklist
This area is specifically for those GIS systems that utilize some sort of trace or route finding option. An important aspect of an electrical GIS is the snapping of conductor to sectionalizing devices. You need to check to make sure all features are connected that are supposed to be and that a path exists to all locations. Look for loops in your system that could cause it to function improperly during a trace. This is very important for electrical distribution systems. A loop is something that causes the flow or trace to “loop” back on itself, similar to the idea of a short circuit. You should check to make sure all sources and sinks are set properly. In addition, if you have rules set up you should verify that they are working properly. A rule could be something such as “A” phase conductor can only attach to “ABC”, “AB”, or “AC” phase conductor. You should include documentation as to how you created the network and the rules that you set up.

19. Relationship Checklist
Relationships to Other Datasets
One to one
One to many
Database connections
Document
Relationship Checklist
Many GIS systems utilize some sort of relationship or database connection. For example, in the ABC_Cooperative GIS, if I use an info tool on a customer I not only see the attributes of that point feature, but I also have a link to billing based on customer id that will show me other specific information on that customer such as transformer number, phone numbers, or threat codes, etc. We take that a step further by having a link from that transformer id to a transformer table that shows us the history and technical specifications for the transformer. You need to check to make sure that those relationships or database connections are working, as they should be. In addition, you will want to include documentation on all of the relationships or database connections that you have built.

20. Annotation Checklist Visible Relative Dynamic or stored in a database
When and where it should be
Relative
Dynamic or stored in a database
Multiple copies
Document
Annotation Checklist
Many of us use annotation, and many of us agree that annotation can be a very frustrating stage of GIS development. Typically, when we perform QA/QC on annotation we print out a series of paper maps and go through them by hand. Many GIS platforms have rules for the features to help place annotation and avoid overlap or masking. Some rules will give certain annotation priority over others. You will want to check that you created annotation for all features that need it, and that it is of consistent size and nature. Annotation can take up a lot of space in your GIS database so you will want to check for multiple copies of the same annotation and that it is stored properly. Should your annotation be stored in the project file or as part of the database? Both of these options have an effect on database performance depending on platform. Once again, you will want to provide documents on how annotation was set up.

21. Custom Applications Checklist
Do they load?
Do they work on the clients hardware?
Plot Routines
Custom Applications Checklist
Often a client will request a custom application for their GIS. You should check that all applications load and work as desired. You may want to run a check on the client’s hardware to ensure that the application works properly. For ABC_Cooperative we created a custom printing application that worked fine with our printers. When we loaded it onto their system, we had to reconfigure it to work with their printers.

22. Wall Map/Map Book Checklist
Common Output Formats
Only contact with GIS for some
Base opinion of what they think
Education
Page Numbering
Correct Order
Scale
Symbols
Size
Color
Effective use of the page
Is annotation visible?
Wall Map/Map Book Checklist
Two very common output formats for a GIS are wall maps and map books. The map book may very well be the only contact some of your clients have with the GIS you built, and they may very well base there impression of the quality of your work on that map book.
It may be very important to help educate your client as to what they are seeing and the differences they may see
Verify that you numbered your pages properly and assembled them in the correct order. Is the scale appropriate for each page? Are symbol color, size, scale and text readable and distinguishable? Did you make effective use of your page? In addition, make sure all annotation is visible where it should be and then check the annotation again.

23. Project Checklist for Clients
Point of Delivery
Correct File Path
All Layers Available
Correct View
Correct Features On or Off
Printers and Other Hardware Configured
QA/QC Checklist for Client
Project Checklist for Clients
Now that you have done QA/QC on your GIS, you need to deliver it to your client. Check to see that all data is “pathed” correctly. Check that all layers are available. Does the GIS start with the correct predetermined view? Do certain features turn on and off at the intended scale? Is the GIS doing the job that the client wanted? Create a QA/QC checklist for the clients to go over themselves that they can sign off. Make sure it utilizes the client’s printers and other hardware.

24. QA/QC: A Checklist for Quality and Control
Never Ending Process
All Stages of GIS
Development
Deployment
Maintenance
Creating a Checklist
Discussions and Team Meetings
Feedback from Clients
Quality Assurance and Quality Control are a never-ending process. Some sort of QA/QC should exist throughout all stages of GIS development, deployment and maintenance. We developed our QA/QC checklist through discussions and experience amongst our clients and ourselves.

25. QA/QC: A Checklist for Quality and Control
Tool Not an Answer
All Inclusive
Not All Consuming
Main Intent
Develop and maintain a reliable GIS that you know works
The main intent of a QA/QC checklist should be to help you develop and maintain a reliable and functional GIS that will not cause conflicts with the end user and you know it works. A Quality Assurance & Quality Control Checklist by itself will not guarantee a perfect product. It should be considered a tool and not an answer. The quality of your GIS will depend on many factors including QA/QC.

26. Best Practices 1. Train Your People and Your Customers
Notice right from wrong
Use of GIS
Use of Tools
Techniques and Practices
2. Develop Tools
Automate repetitive tasks
Automate bulky tasks
I will conclude with some practices that I think are relevant and may not work in your situation, but they could give you the ideas you need to make QA/QC work for you. These guidelines are not a replacement of QA/QC but they can help you avoid little errors in the first place.
1. Train your people and your customers in the proper use of the GIS.
2. Learn to notice what is right and what is wrong. For example, the only conductors that an “A” phase conductor can only connect to are “ABC”, “AB”, “AC”, or “A” phase conductor. A transformer should connect to a primary conductor and a secondary conductor.
3. Your organization may have a programmer on its staff. Use that person to develop programs or helper applications to automate repetitive tasks. We had our programmers develop a map plot routine that can be used for each of our GIS systems. It will automatically use a polygon feature to zoom to the desired map page extents, populate multiple fields with specific information (page number, township, range, section are some examples) assign an alphanumeric name to the file and save it as a PDF or plot file. This way I can confidently create map books and be assured that they will be correct. Of course, you have to QA/QC the program.

27. Best Practices 3. Encourage Dialog 4. Use Your QA/QC Program!
Discussions
Panel Sessions
Etc…
4. Use Your QA/QC Program!
4. Encourage your people to talk about QA/QC problems they have encountered and to discuss any techniques they have developed.
5. Use your QA/QC program.

28. Questions?