1. Validation Workshop at The Wales Annual Meeting
May 2019

2. Outline for workshop
Introduction & general principles - a presentation based on the Irish validation workshop
The What, Who, Why, When, Where and How
Working in small groups on problems

3. Validation: The VCR is king!
Validation is the what you do when you confirm/reject or query a record in the DDB. To do this you need to check
grid reference/site are consistent
Date is valid
Record is taxonomically correct
Editing a record involves changing a record because it has a correctable error.
Validation is one of your most important roles as a VCR. This is because you are better placed to do it than anyone else.

4. Why do we need to validate records?
Most datasets contain errors:
What % of records are incorrect in some way?
(0.03%) – 0.5% - (10%) (from Andy Amphlett)
1 error per 200 records is fairly typical
Are these errors significant?
See DDb > my county > data validation - this will flag up records with serious problems eg wrong taxon, hectad
It is impossible to eliminate all errors, in all datasets, so that is not the target of validation
The important part is to validate the tiny percentage of records with serious errors
Errors may be relatively insignificant (eg. minor typo in locality spelling, or grid ref incorrect by 100m. BUT some errors are SERIOUS, eg wrong taxon, wrong hectad, wrong date class etc.
Existing validations on DDb – while many are ok, some ‘confirmed’ data sets were poorly checked. DDb has much more powerful tools to identify unusual or suspect records than were available in the past.
Range of error percentages are from datasets that Andy Amphlett has worked on. 0.5% (1 error per 200 records) is fairly typical for many datasets.

5. How to validate records
Start with Mapmate
When entering data into Mapmate set up the Validation tool
(Data entry-Records-Change default-validation)
Click the warnings that you wish to have – eg 1ST record for hectad
It may be useful to set up the Taxon-Validation tool
This has to be set up for a taxon eg E. peplis - asking it to warn you if it is
entered in your VC (often a typo for E. peplus)
We advise synching to the DDB after each data entry session, as then you can validate each data set quickly
When using the DDB with records just updated, search with the date entered and once the more unusual or notable records have been checked, confirm ALL records in the set
Errors may be relatively insignificant (eg. minor typo in locality spelling, or grid ref incorrect by 100m. BUT some errors are SERIOUS, eg wrong taxon, wrong hectad, wrong date class etc.
Existing validations on DDb – while many are ok, some ‘confirmed’ data sets were poorly checked. DDb has much more powerful tools to identify unusual or suspect records than were available in the past.
Range of error percentages are from datasets that Andy Amphlett has worked on. 0.5% (1 error per 200 records) is fairly typical for many datasets.

6. DEADLINE for validation of records for Atlas 2020 is 31/03/2020
Advise the following approach to validation
NOW
Check, (correct where necessary), and validate records as soon as they arrive at the DDb
Do not let the backlog of unchecked records get any bigger.
Prioritise record checking
Check and validate all records
DEADLINE for validation of records for Atlas 2020 is 31/03/2020

7. How to Validate: Using the DDB Validation Tools
The DDB validation tool is essential when dealing with a backlog
Find it using: DDb > my county > data validation tab
Validation of Singletons
(One off or isolated occurences) – require +ve confirmation or rejection
Look at the County report, click on 2nd tab “Data validation”
Scroll down to Singletons & Anomalies. Click on blue result nos and display results to see table of records
Examine table of records. Can you see any obvious problems?
Change the Validation state if necessary.
Tick box on far left. Scroll to page bottom and select change validation state in dialogue box. Add comments to explain the change.
‘my county’ tab on DDb. Links to queries for records to target for validation for Atlas2020 are suggestions for starting points for detailed checking of records from your county.

8. Validation states: what they mean for the record
Mapped?
Returned by default queries?
Unchecked (i.e. newly added to the DDb) 
Accepted / Confirmed
Needs verification / May need verification X
Doubtful
Rejected
Duplicate
Deleted externally (in the origin database)
DDb retains all records and all versions of records (unlike MapMate).
Visibility of records, on maps or returned by queries, determined by Validation state.
Records arrive at DDb in ‘Unchecked’ state.
Do not spend time investigating duplicates – that is a very poor use of time, and not necessary for Atlas 2020.
Remember: you cannot ‘delete’ records from the DDb

9. Frequently Asked Questions
Is it necessary to examine every record in detail when validating records? NO
When you Confirm a record does that mean that you have to be 100% certain the record is correct? NO Confirmation of a record means that you are happy to accept it as likely to be correct
If I Confirm or Reject a record can I change my mind later? YES
You can always assign a ‘new’ validation state to a record, which will
over-write the previous version.
Can I reject or query an unusual record that might possibly be correct? YES Look for supporting evidence before confirming an anomalous record

10. More general guidance
Don't get stuck on difficult, often old, records. Prioritise those you are entering now.
The DDb retains full details of records tagged as doubtful or rejected, a record cannot be deleted.
Use the facility to edit very sparingly
Do not edit or change a record in the DDb if it should really be happening in MapMate (either your own, or another VCRs’) in other words, edit the record where it was made
Hieracium and Taraxacum records are much lower priority, (key datasets for those genera have been verified but not validated)

11. In this workshop..
Try peer-to-peer discussion and problem solving in small groups
There will be some support from Pete Stroh
Not all issues can resolved here, so we will make a list of things to be followed up
Hopefully everyone will leave with a better understanding of the process and know what to do next

12. Discussion kick starter
What are the two biggest issues your group has about validation?
Discuss for 5 mins
Jot down on sheets of paper I’ve placed
near each group
Report back