Send Me a Disk, Ok? -Sharing Genealogical Information With Your Relatives Beau Sharbrough PO Box 3170 Grapevine TX

Thank you. To the CIG. I’m grateful for the invitation to be here. To the CIG. I’m grateful for the invitation to be here. To Russ and Birdie Holsclaw. They took care of me the past three days, sharing their home, their cars, their community, and their son Will. To Russ and Birdie Holsclaw. They took care of me the past three days, sharing their home, their cars, their community, and their son Will. To Roger Ebert. I was starting to worry about my weight. To Roger Ebert. I was starting to worry about my weight.

General Topics Five steps to understanding what they’re saying Five steps to understanding what they’re saying Discussion of software developers’ methods of merging files Discussion of software developers’ methods of merging files Significance of GENTECH Genealogical Data Model Significance of GENTECH Genealogical Data Model

Five Steps to Combining Your Research

Step 1. Determine What Form the Data Is in. Which program do they use? Which program do they use? What type of disk drives do they have? What type of disk drives do they have? What general field usage have they adopted? What general field usage have they adopted?

Step 2. Exchange Pedigree and Group Sheet Examples. Look for detail, accuracy, thoroughness. Look for detail, accuracy, thoroughness. Are there full or partial dates? Are there full or partial dates? Do the citations for US places include counties? Streets? Cemetery names? Do the citations for US places include counties? Streets? Cemetery names? Are nicknames used in place of “real” names? Are nicknames used in place of “real” names? Are sources cited? Are sources cited?

Step 3. Agree on Usage of Fields. RESIdes or ADDRess? RESIdes or ADDRess? Will you both use CHRIsten? Will you both use CHRIsten? How will you document sources? How will you document sources? How will you document the research of others? How will you document the research of others?

Step 4. Convert Your Information. Nobody Can Avoid This Step. Agree with your relative what information you will convert and how Agree with your relative what information you will convert and how Normally, this means saying things like, "I’ll put in the counties after I get it from you" Normally, this means saying things like, "I’ll put in the counties after I get it from you"

Step 5. Exchange Only the Individuals You Want. NEVER just import the whole family on top of the information you already have. NEVER just import the whole family on top of the information you already have. Computer routines for merging data are improving, but not complete or effective yet. Computer routines for merging data are improving, but not complete or effective yet.

There Are No Effective Routines for Merging Data Sets at Present. The problems of … The problems of …  Identity  merging methods and  data formats … are too new for generalized solutions to be available in the marketplace … are too new for generalized solutions to be available in the marketplace Good theoretical solutions don’t even exist Good theoretical solutions don’t even exist

Merging Data Sets

Customers who just assume that someone will know what they want and have it ready when they recognize that need had parents that spoilt them rotten.

WHY? Family history record-keeping is increasingly becoming a digital process. Family history record-keeping is increasingly becoming a digital process. Linking one’s information to the information already gathered by other family members and researchers is becoming more and more common. Linking one’s information to the information already gathered by other family members and researchers is becoming more and more common.

We Have to Put Our Information Together Somehow

A Few Basics Computer programs store the data that we enter in FILES Computer programs store the data that we enter in FILES Each genealogical program stores the information in its own way, called a PROPRIETARY FORMAT Each genealogical program stores the information in its own way, called a PROPRIETARY FORMAT Most programs can also read and write in GEDCOM format Most programs can also read and write in GEDCOM format

A word about exchange … A A B B Import Routine Export Routine Possible Intermediate Format

A Few Basics Merging is copying Merging is copying From a SOURCE From a SOURCE To a TARGET To a TARGET Sometimes called the SURVIVING INFORMATION Sometimes called the SURVIVING INFORMATION

MERGING DATABASES merging the files into a single one merging the files into a single one merging the duplicated individuals merging the duplicated individuals merging the rest merging the rest  sources  repositories

The database merging process is evolving More input sources More input sources More freedom to choose the features you like. More freedom to choose the features you like. GenBridge GenBridge

Freedom has a price Enter a name Enter a name Program won’t break it up Program won’t break it up Enter a place Enter a place Program won’t break it up Program won’t break it up

Legacy Trick You can open two family files at the same time, and copy and paste a person and their descendents from one set into another, like grafting a tree branch from one tree to another. You can open two family files at the same time, and copy and paste a person and their descendents from one set into another, like grafting a tree branch from one tree to another.

Making automatic citations Legacy – individual level Legacy – individual level TMG and FTM – field level TMG and FTM – field level

The Current Merging Art Merging Databases Merging Databases Merging Individuals Merging Individuals Merging the Rest Merging the Rest Spotting Duplicates Spotting Duplicates

Merging Individuals If you want to merge duplicates, most programs will make you choose which “tags” to keep and throw the rest away.

MERGING INDIVIDUALS: The old way Copy the info Copy the info Delete one of the people Delete one of the people Type the info into the new one Type the info into the new one

MERGING INDIVIDUALS: The middle way View both persons View both persons Select what you want Select what you want The program does the rest The program does the rest

MERGING INDIVIDUALS: The future way Computer spots likely dups Computer spots likely dups Recommends them to you Recommends them to you You control the process You control the process

Merge Sources for most popular software Their own files Their own files GEDCOM GEDCOM In some cases, files from other programs In some cases, files from other programs In some cases, CD and internet databases In some cases, CD and internet databases Still, it ends up being like pouring two cans of paint together.

Merging the Rest Most programs don’t even import and merge place tables, source tables, etc. I don’t know of any program that recognizes the same source in two separate datasets.

Merging The Rest source citations, master sources, repositories, and places source citations, master sources, repositories, and places Most programs just combine the tables, creating duplicates Most programs just combine the tables, creating duplicates LG will combine a source, with exact spelling LG will combine a source, with exact spelling UFT and FTM merge master sources UFT and FTM merge master sources PAF and TMG merge master sources and repositories PAF and TMG merge master sources and repositories

Limits to Storage Some programs have really limited storage, and only store conclusions Some programs have really limited storage, and only store conclusions If you have two birth dates, they put your favorite one in and throw the other away, or store it in a note. If you have two birth dates, they put your favorite one in and throw the other away, or store it in a note. Some programs have a lot of storage, and let you make your own “tags” such as executrix. Some programs have a lot of storage, and let you make your own “tags” such as executrix.

SPOTTING DUPLICATES Some programs have “merging routines” based on: Some programs have “merging routines” based on: Soundex Soundex Spelling of name Spelling of name Birth date Birth date TMG and Legacy use a large variety of match choices TMG and Legacy use a large variety of match choices

Spotting duplicates Soundex for names (AQ) Soundex for names (AQ) Exact spelling or soundex (PAF 3.0) Exact spelling or soundex (PAF 3.0) Exact spelling and exact birth date (FTM) Exact spelling and exact birth date (FTM) Many name compares (TMG and UFT) Many name compares (TMG and UFT) Soundex surname and user choice of # of letters in first name (LG) Soundex surname and user choice of # of letters in first name (LG) Warn if duplicate name entered (most) Warn if duplicate name entered (most)

Merging tips Match on parent soundex reduces false positives (Gaylon Findlay) Match on parent soundex reduces false positives (Gaylon Findlay) If your program won’t let you choose initials, but has a number-of-letters, try that with 1. If your program won’t let you choose initials, but has a number-of-letters, try that with 1. Beware of people about whom you know very little. Beware of people about whom you know very little. Beware of blank dates. Beware of blank dates.

Signs that you can merge better today than you could before More formats allowed More formats allowed Easier individual merging Easier individual merging Identifying routines are becoming more sophisticated Identifying routines are becoming more sophisticated More storage of conflicting data allowed More storage of conflicting data allowed More variety in the software marketplace More variety in the software marketplace

Signs that we aren’t getting there yet No formal studies on known datasets to quantify false positives and false negatives No formal studies on known datasets to quantify false positives and false negatives No implementation of information sciences in commercial products No implementation of information sciences in commercial products No implementation of AI in commercial products No implementation of AI in commercial products No formal discussion of algorithms No formal discussion of algorithms

MERGING SUMMARY Users can merge from a wider variety of data formats than in the past. Users can merge individuals more easily.

MERGING SUMMARY Routines to help identify candidates for merging are becoming quite sophisticated. More programs store conflicting data today.

It’s also encouraging that they are not all doing the same thing. The resultant diversity and innovation offer us more chances to connect Where-We’ve-Been to Where-We’re-Going than we’ve ever had before.

The GENTECH Genealogical Data Model Purpose: To define and communicate the meanings of family history data. Purpose: To define and communicate the meanings of family history data.

Genealogical Data Model Request for Comment Request for Comment Project by genealogists and developers to describe genealogy processes. Project by genealogists and developers to describe genealogy processes. Describes the relationships between the various kinds of family history information. Describes the relationships between the various kinds of family history information. Overview of what genealogists do Overview of what genealogists do Not a genealogy program. Not a database design Not a document saying what genealogists SHOULD do.

Every genealogist says that they do research differently. The GDM describes the process that they do differently.

Stop Starting with Conclusions Don’t start with conclusions, start with evidence. Don’t start with conclusions, start with evidence.

Some features of Evidence in the GDM REPOSITORY REPOSITORY SOURCE SOURCE REPRESENTATION TYPE REPRESENTATION TYPE REPRESENTATION REPRESENTATION CITATION CITATION

CONCLUSIONS ASSERTIONS about … ASSERTIONS about …  PERSONA  EVENTS  CHARACTERISTICS  GROUPS  ASSERTIONS

XML is eXtended Markup Language The Title of My Book The Title of My Book Jonathan Sharbrough Jonathan Sharbrough circa 1734 circa 1734

Future digital research programs publish pedigrees and registers in some XML format programs publish pedigrees and registers in some XML format repositories publish records in the same format repositories publish records in the same format local links, remote sources local links, remote sources external authorities external authorities

A new culture most quoted sites - “authorities” most quoted sites - “authorities” many link sites - “hubs” many link sites - “hubs” links define culture, tribe, families links define culture, tribe, families

The digital future of family history is a virtual library where it is... Easy to find the conclusions Easy to find the conclusions Easy to identify the evidence Easy to identify the evidence Easy to identify the thought process that links them. Easy to identify the thought process that links them.

Missing ingredients agreement on LexML standard agreement on LexML standard wide acceptance of LexML standard wide acceptance of LexML standard wide implementation of LexML wide implementation of LexML

Send Me A Disk, Ok? Do’s and Don’ts Do’s and Don’ts Merging Technique Merging Technique GENTECH GDM GENTECH GDM Beau Sharbrough PO Box 3019 Grapevine TX