1. Mock-ups for Discussing the CMS Administrator Interface

2. User Name:
Full Name:
Password:
Confirm Password:
PIN:

3. Admin Home Page Import Book Configure Extraction Tools (not clickable)
FROntIER (as is, all setup work done outside CMS, but testing done inside CMS)
GreenDDA
GreenFIE
ListReader
OntoES (OntoES + snippets with regex tool for creating rules for PCF ontologies)
OntoSoar
Initiate Pipeline Run
Manage Pipeline Run (check progress, kill & restart tasks, manage jobs and batches, submit completed work)
Edit User Account (only for super-admins)

4. Database Schema user (id, username, email, password, name, type)
job (id, name, book, max_page)
batch (id, pages, status, job_id, user_id)
form (id, form_name, version)
eventlog (id, user_id, batch_id, session_id, log-info, time_written)
status_code (status, next_status, description)
user_type (type, description)
P: patron
A: adjudicator-patron
M: administrator
S: super-administrator
N: not assigned or not submitted
S: submitted (initial patron submission)
D: done (sent downstream to Gedcomx)
Q: quality recheck by patron
R: resubmitted (patron resubmission)
A: adjudicator re-check
C: checked by adjudicator
T: testing (under control of CMS admin)

5. Workflow Pipeline PCF = Person, Couple, Family directories
PRF = Precision, Recall, F-score
1.pages
2.tools
<toolName>
PCF: extracted
2.1.tool-ontology-extracted
2.2.ontology-extracted
2.3.extracted-text-cleaned
3.json-from-osmx
3.1.ontology-merged
3.2.value-cleaned
3.3.date-value-parsed
3.4.constraint-checked
3.5.violation-corrected
PCF: osmx & json
4.json-working
PCF: json
5.json-final
CMS: setup & test
6.osmx-checked
6.1.ontology-merged
6.2.value-cleaned
6.3.date-value-parsed
6.4.constraint-checked
6.5.authority-checked
PCF:
osmx & json
ground-truth
PRF-report
7.osmx-enhanced
7.1.target-ontology-generated
7.2.value-standardized
7.3.information-inferred
8.gedcomx
8.1.gedcomx-generated
8.2.reports-generated
Q, A, T
D, C
Q, A, T
N, T
COMET bypass
S, R, C, T

6. Import Book Browse file of available books Includes browse pages
Future organization: books may be categorized, ordered, …
Select book from file for import
Add meta.xml text
shortTitle
biblio
url (on dithers) auto-assigned
Filename (in file of available books) auto-assigned
Auto-assign id (six digits + shortTitle)
Initialize book authority files (NameAuthority.txt, PlaceAuthority.txt)
Initiate PDF-indexer
Split pages
Generate 1.pages files for each page
.html
.pdf
.png
.txt
.xml
Import Book

7. Test tool on selected page(s)
User-selected list or range of test pages
Generate filled in forms (in quality-check mode)
Allow filled-in form(s) to be ground-truthed
Produce PRF reports for ground-truthed form(s)
Test tool in ensemble with other tools
Results of a test run
For each page/form, the data and form json
Regular COMET display of first page and filled-in form
Ground-truth reports
Differences in COMET display of auxiliary information
No “Annotation Actions” or click for instructions link
Batch sequence according to list or range of test pages
Instead of “Submit Batch”, “Generate Ground Truth”
A list of links to any and all ground-truth reports
For all tools
Test <list, e.g. 031,032,099> or <range, e.g >
Include Tools: [ ] FROntIER, [ ] GreenDDA, [ ] GreenFIE, [ ] ListReader, [ ] OntoES++, [ ] OntoSoar
The tool being worked on is included by default.

8. For all tools Generate Ground Truth range.030-031.PRFreport-soft.txt
Person/range PRFreport-soft.txt
Person/030.PRFreport-soft.txt
Person/031.PRFreport-soft.txt
Couple/range PRFreport-soft.txt
Couple/030.PRFreport-soft.txt
Couple/031.PRFreport-soft.txt
Family/range PRFreport-soft.txt
Family/030.PRFreport-soft.txt
Family/031.PRFreport-soft.txt

9. Use FROntIER interface, as is, to create an extraction ontology
Use CMS testing interface to test the created extraction ontology
Run test (as can be done for all tools)
Set ground truth (as possible for all tools)
Future: add the CMS regex-checker tool
FROntIER

10. GreenDDA Select training set Train ML tool to training set Select book
Apply trained ML tool to book
GreenDDA

11. GreenFIE GreenFIE Regular GreenFIE interface in experimental mode
? Ability to run other tools to initialize the GreenFIE interface
? Ability to selectively save/retract generated rules
Changes to COMET interface
Add GreenFIE “Generate Rule” button
Replace all below the form by “Save” button to save all rules and:
? Initialize from tools request
? List of generated rules with check boxes
Make page number a type in box
GreenFIE
GreenFIE
Regex
Regex
Regex
Regex

12. ListReader ListReader Next Stop
Select book and run ListReader (future: also set text abstraction parameters)
One record from all three forms on the lhs
Highlights on the rhs as generated by ListReader
Control buttons
Next (to tell ListReader to save the current labeling and ask for the next)
Stop (to stop the labeling cycle)
ListReader
ListReader
Next
Stop

13. OntoES Generate extraction rules
FROntIER-like rule-creation facilities
Re-organized for better UX
Built-in regex checker
OntoES

14. OntoSoar

15. Initiate Pipeline Run Book selection box Page Range – Tools FROntIER
GreenDDA
GreenFIE
ListReader
OntoES
OntoSoar
Run Type
with COMET
without COMET
default
Run

16. Manage Pipeline Run Display progress for book
Show progress for each page
Report failures
Allow admin to kill and restart tasks
When running through COMET:
Display progress of each job
Percent complete
Status of each batch: Unassigned/InProgress/Done
Status of InProgress batches
User assigned
Page/form status
Eventlog for batch (upon request)
Retract batch from user
Assign batch to user
Reassign batch to user
User status
Batches completed
Batches assigned
Eventlog report for user (upon request)
Submit completed work to FS
Manage Pipeline Run

17. Edit User Account Select user: id username email name search Find user
Set any/all:
Password
Username
Full name
Change privileges
Disable account
Edit User Account
Select user: id
username
name
search

18. Edit User Account User: id username email name Change password
Find user
Set any/all:
Password
Username
Full name
Change privileges
Disable account
Edit User Account
User: id
username
name
Change password
Change username
Change
Change full name
Change privileges
o patron
o adjudicator-patron
o administrator
o super-administrator
o Disable account
done