Tag Archives: myheritage

Pruning Your Family Tree

Cruft is a term used in computer programming circles to mean the useless code in a computer program that accumulate over time. Cruft is the stuff you added at one point that might have been important then, but is now irrelevant, and worse it causes the rest of your program to slow down. You might have needed, for example, to support what is now an obsolete computer platform at one point, but the code for that shouldn’t still be in your program today.

Family trees also accumulate cruft over time, and just like in computer programs those extra people and extra information can slow you down. There are a number of reasons that bad information can enter your tree, but the most common and most problematic is when you import a GEDCOM from a relative without checking first to see if everyone in the tree is actually related to you. If you get a GEDCOM file from a relative with 2000 people in it and only 200 of them are actually related to you, you’ve just added 1800 that are irrelevant to your tree. Moreover, if you upload your family tree to a site like Ancestry.com or MyHeritage.com where they can do some form of automatic matching between your tree and other trees as well as with records on the site, you’re going to get all kinds of matches for people who are not actually related to you. Following up these false leads is a big waste of time.

I recently uploaded a family tree to one of these web sites and started getting matches to people not related to me. It illustrated to me that when I imported a tree from a relative a couple of years ago I did not properly check out the tree first. When I share a tree with someone, I usually only export those people who are related to the person I’m sending the tree to, plus spouses. This insures the person doesn’t get a lot of records that are not relevant to them. When you receive a GEDCOM from someone else, you should also check it out, create a test file where you import it, add yourself to the file, and then see if everyone in the file is related to you. I obviously forgot to do this with this particular file a few years back, and ended up with about 300 extra people in my tree that I was not related to, which was what was causing these false hits in the matching program (technically they’re not false hits, okay, but from my perspective they’re just as annoying even if they are my fault).

After receiving quite enough of these messages from the web site I decided it was time to remove the incorrect records from my family tree file. While my initial guess about that GEDCOM file was correct, and it was indeed the source of most of the incorrect records, I also discovered something else interesting – that there were other people in my family tree that were not related to me, some of them that I wanted to keep. The important thing here is that while most genealogy programs will let you select all your relatives (and their spouses), it’s not so simple to select your relatives and delete everyone else. The issue of the spouses, by the way, is a simple one. If you only had the program select your actual relatives, your sibling’s spouse would not be chosen. Your sibling’s kids would be chosen, but they would be missing a parent since strictly speaking that spouse is not your blood relative. Thus you need your genealogy program to select spouses as well.

The people I found in my tree that were not related to me fell into a few categories. Most were from GEDCOM imports, with most of those from that one GEDCOM I suspected, but also a few others here and there.

Some of the people were really cruft in that they were small sections that were someone isolated from the rest of the tree. I suspect they were descendants of someone I deleted at some point. They should probably have been deleted a long time ago, but were somehow still in my tree – probably due to a bug in the genealogy program.

Then there were the parents of spouses. I sometimes like to add information on parents of spouses that I add to the tree. This is mainly so that if I want to research the spouse at some point in time, that I know a bit more about them to help me with the research. Knowing the names of a person’s parents can be very important when doing research. The problem, of course, is that if I do a standard selection of people in my tree that are not relatives or their spouses, these parents get left out – yet I still want them in the tree. The solution here is not simple. There is not an automated way to include these people. The answer is probably (and I have not done this yet) to flag those parents in some way. Some genealogy programs let you define custom flags, and then assign them to people. If you carefully check out all the non-relatives in your tree and see which ones you want to keep, you can then flag them for future reference. Each time you add non-relative parents, you can flag them. In the future if you go to prune your tree again, you can do a standard selection of relatives and spouses, and then add the flagged people. Anyone left over can then be removed from your tree.

The Future of Sharing (Genealogical Data)

It’s no secret that the current standard for sharing genealogical data, GEDCOM, is woefully out of date. The last official revision to the GEDCOM standard, 5.5, was completed in 1996. A minor update, 5.5.1, was released in 1999 but never officially approved (even though some of its provisions have been adopted by various genealogy programs). Revision 5.5.1 added one very important feature – support for UTF-8 character encoding, which is a form of Unicode, which support multiple character sets (including, for example, Hebrew).

GEDCOM has, for all intents and purposes, been abandoned by the Church of Latter Day Saints (the Mormons) which created and owns the standard. The church has indicated that they will not be updating it, and indeed are replacing the need for it with a new API (Application Programming Interface) which will allow genealogy programs to exchange data with their website (FamilySearch.org). One problem with this approach is the need to go through their website, and the fact that they have not made this API publicly available (i.e. it’s not a public standard, just a private interface to their web site). Another major problem is that there is no data format that allows one to create a family tree that can be shared independently, like GEDCOM is used today. FamilySearch in no way needs such a format, since their mammoth size and importance in the genealogical world will force genealogy program to support its API, as many have already done.

Over the years, there have been many attempts to either upgrade or replace GEDCOM. These efforts have all failed. In general the problem has been that the companies that create genealogy program need to agree to adopt any new standard, and they really haven’t had much incentive to do so. Supporting the import of GEDCOM files allows them to support a basic file interchange, which never will support the full feature-set of their programs which have become much more sophisticated since 1996, but is enough to allow customers to exchange information with their relatives. If they supported a fully-featured GEDCOM replacement (that for example would better support photographs and evidence management), it would only make it easier for customers to try other programs. Thus the disincentive for the companies to support a modern replacement for GEDCOM.

Another problem with replacing GEDCOM has been arguments over the data model used. GEDCOM is based on a nuclear family data model (i.e. one mother, one father and their children). It assumes a nuclear family structure, and other forms of families are harder to support. This problem has caused some to support a data model based not on the family but on the individual. This is philosophical debate, and as you might imagine different people take very strong positions in this battle.

Even with this history, there are a few new initiatives to come up with a replacement for GEDCOM. One initiative that has garnered some attention recently is BetterGEDCOM. The BetterGEDCOM initiative came from the frustration of many genealogists over the lack of updates to GEDCOM and is an attempt to create an open forum for the creation of a new standard. Like many attempts at ‘openness’, however, it has run into its own in-fighting and conflicts. It remains to be seen how successful this attempt with be. Another recent initiative is the International OpenGen Alliance (OpenGen). This effort is a bit more of a top-down approach, being managed by the company that runs AppleTree.com, an online family tree web site. OpenGen is, however, a non-profit organization that is supposed to include more than just the team at AppleTree. There have been some attempts between BetterGEDCOM and OpenGen to coordinate, or at least follow each others’ efforts closely. Time will tell which effort, if either, will be successful in creating a new genealogical data sharing standard.

In case you think it isn’t complicated enough, other web sites beyond FamilySearch.org are also developing their own APIs for exchanging genealogical data. OneGreatFamily.com last year introduced an API called GenealogyCloud. It seems that no third-party applications yet support this API.

Geni.com, which boasts nearly a hundred million profiles on their site, and nearly 50 million that are interconnected in what they call their World Family Tree, just yesterday introduced their own API. Unlike FamilySearch.org, however, they are releasing documentation and sample applications on their web site. This will allow anyone to write applications that interact with Geni.com, similar to the way Facebook allows outside developers to create application that access information on Facebook. This is a very positive step. It’s not coincidence that one of the other large family tree web sites, AppleTree.com, is pushing another initiative to replace GEDCOM (OpenGen). These large sites need to create ways to exchange data and interact with other programs and web sites in order to maintain their growth rates.

MyHeritage.com, another one of the big family tree web sites, has taken a slightly different approach in that they have their own application (Family Tree Builder) that runs on a computer, which can sync data to their web site. While this approach allows them more control over what modifies data on their platform, it has its shortcomings as well, not the least of which it requires Windows to run (this coming from a Mac user). I suspect that MyHeritage.com will release their own public API in the future, if only to compete with Geni.com, their biggest competitor.

We can always hope that FamilySearch.org, Geni.com, MyHeritage.com and AppleTree.com will all come together and create a single API and data format for sharing data, but unfortunately if the past is any guide, this is unlikely to happen.

One indication of the direction the wind is blowing in this regard will be the upcoming RootsTech conference, taking place in February 2011 in Salt Lake City. This conference is the first RootsTech conference, although according to the organizers it replaces three earlier technical conferences – The Conference on Computerized Family History, the Family History Technology Workshop and the FamilySearch Developers Conference. Note that these previous conferences were all connected in some way to the Mormon church. It’s unclear how open this new conference will be to new ideas, or if it is really only looking for input for the existing Mormon church efforts such as FamilySearch.org. I imagine representatives from most of the genealogy software companies and web sites will be in attendance at the conference, as will people associated with the BetterGEDCOM and OpenGen efforts. During the week of the conference there will probably be a lot of blogging about what is going on, but the real test will be after the conference if companies announce intentions to seek a common API or data format to move forward with, or whether everyone will just continue the same disjointed approach that has been pursued for nearly 15 years.