Tag Archives: genealogycloud

Genealogy standards, another look

Technical, , , , , , ,

Over a year ago I took a look at genealogy data standards and where they were headed in my article The Future of Sharing (Genealogical Data). In some ways a lot has changed since I wrote the article, but in some ways we’re really at the same point we were then, with no clear picture of the future. This past week’s 2nd annual Rootstech conference (my last article mentioned the then-upcoming 1st Rootstech) has brought some of the questions asked into focus, so I thought it was worth reviewing what has happened.

GEDCOM X

On the face of it, the biggest news to come out of the conference was the release of long-awaited successor to GEDCOM, GEDCOM X. FamilySearch, the online presence of the LDS church which was the creator and maintainer of the original GEDCOM standard, released this new standard at the conference a few days ago. FamilySearch hits a lot of the right keywords in the release – the format can be XML or JSON based, is released under an Creative Commons license, supports metadata including Dublin Core and FOAF, the development is hosted on Github, it offers both a file format (like tradition GEDCOM) and an API, and more. Yet there are also some strange decisions that seem to have been made, and no explanation seems to be given. One that stands out is the decision to base the file format MIME, a format created for sending e-mail attachements (MIME is an acronym of Multipurpose Internet Mail Extensions). So far the logic behind many of the decisions that have been made seem very opaque. The entire development of GEDCOM X seems to have been done up to this point without any input from the industry at large, or even the well know efforts to improve GEDCOM, such as the Better GEDCOM group. Indeed, the answer in their FAQ about these efforts seems largely patronizing:

Have you heard about FHISO (BetterGEDCOM), OpenGen, ?
Of course. We’ve heard about them and many others who are making efforts to standardize genealogical technologies. We applaud the work of everybody willing to contribute to the standardization effort, and we hope they will continue to contribute their voices.

In other words, at least to my ears, it’s saying they know other people want to improve GEDCOM, but they are going to do their own thing and maybe they’ll listen occassionally (but no promises). In short, while it’s great that FamilySearch has come out with a new standard, their approach to doing so does not seem geared towards gaining widespread adoption from the industry at large, or at least not in such a friendly manner.

Of course, the huge advantage FamilySearch has over just about anyone else is the very large developer network they’ve cultivated for accessing familysearch.org. They are essentially a non-profit organization which has many commercial companies using their current API. To the extent that they transition these existing companies from their legacy API to GEDCOM X, they will certainly have a major advantage over other efforts to replace GEDCOM.

Progress On Other Fronts

So what happened to the other efforts mentioned in my last article?

The most visible effort has been the BetterGEDCOM wiki, which is moving from an informal group to a formal organization called the Family History Information Standards Organisation (FHISO) which will now sponsor the wiki. While they have been the most active effort to create a replacement for GEDCOM, they seem to have been overtaken by the too-many-cooks problem and how they plan on coming to a consensus remains to be seen, let alone how they convince industry organizations and companies to agree with them. It will be interesting to see FHISO’s response to GEDCOM X, and if they will focus their efforts on trying to implement their ideas within the GEDCOM X framework, or if they will continue to try to go it alone.

The OpenGen International Alliance, started by the people at AppleTree.com, doesn’t seem to have taken off. Either for the matter has AppleTree, which may explain the why the OpenGen site hasn’t been updated in the past year (and refers to an upcoming webinar last March).

APIs

One of the most interesting developments last year was the introduction of Application Programming Interfaces (APIs) for genealogy web sites. Indeed, the rumors around what would become GEDCOM X was that it was only an API, and not a file format, but luckily that turned out not to be true and it is both. The only APIs that had been released before my last article were Geni.com‘s API and OneWorldTree.com‘s GenealogyCloud API.

Geni seems to at least gotten some traction with their API, with future support for syncing data coming from AncestorSync. Presumably this uses Geni’s API. I haven’t heard of other uses of the Geni API, however. If you know of other developers using the Geni API, let me know in the comment.

I have not heard of anyone using the GenealogyCloud API. If you know any anyone using GenealogyCloud, let me know in the comments.

As I predicted in the last article, MyHeritage introduced their own API, smartly named Family Graph. I say smartly because it is clearly mimicking Facebooks’ Social Graph API. They’re not comparing themselves to Geni, but to Facebook, which is smart. The other very smart thing they did was introduce a contest to develop applications that use the Family Graph API. If no one uses your API, what’s the point right? The winner receives $10,000. The deadline for that contest is actually in about a week from now, with judging by a panel taking place in the first half of March and the results announced on March 15th. The real test will be the quality of the applications submitted, and whether the applications were submitted by individual developers or by larger companies. If the contest results are published next month with no major applications, then this will in my estimation be a setback for MyHeritage, not an achievement.

Conclusion

It will be very interesting to see how the introduction of GEDCOM X is accepted by the genealogy companies at large that are needed to make a new format successful. FamilySearch has some key advantages in that they are a non-profit organization (even though in many ways they compete with the large commercial companies like Ancestry.com and MyHeritage.com) and that they already have a large developer network. While many of the largest genealogy companies are not currently part of that developer network, if all of the ones who are start adopting GEDCOM X as their export format of choice, I think it will be hard for other companies to not adopt it. GEDCOM X’s dual format/API functionality also gives it a major edge, especially if FamilySearch’s legacy API is replaced by the API functionality in GEDCOM X.

Some have predicted there would never be a true replacement for GEDCOM, and others have said that technology such as AncestorSync’s upcoming products would make the need for a file format unnecessary. I think both of these assertions are incorrect. There will be a replacement for GEDCOM, and it is necessary. Whether or not GEDCOM X is the ideal replacement seems to me to be a moot point. They will get the traction they need to push GEDCOM X into the mainstream. The real question is will they truly make it an open standard, or will they continue to hold it close to the chest? The real test will be when other groups insist on various features, and how they handle those demands. FamilySearch has put in all the trappings of an open and transparent development process, so let’s hope they keep in that direction.

The Future of Sharing (Genealogical Data)

Technical, , , , , , , , , ,

It’s no secret that the current standard for sharing genealogical data, GEDCOM, is woefully out of date. The last official revision to the GEDCOM standard, 5.5, was completed in 1996. A minor update, 5.5.1, was released in 1999 but never officially approved (even though some of its provisions have been adopted by various genealogy programs). Revision 5.5.1 added one very important feature – support for UTF-8 character encoding, which is a form of Unicode, which support multiple character sets (including, for example, Hebrew).

GEDCOM has, for all intents and purposes, been abandoned by the Church of Latter Day Saints (the Mormons) which created and owns the standard. The church has indicated that they will not be updating it, and indeed are replacing the need for it with a new API (Application Programming Interface) which will allow genealogy programs to exchange data with their website (FamilySearch.org). One problem with this approach is the need to go through their website, and the fact that they have not made this API publicly available (i.e. it’s not a public standard, just a private interface to their web site). Another major problem is that there is no data format that allows one to create a family tree that can be shared independently, like GEDCOM is used today. FamilySearch in no way needs such a format, since their mammoth size and importance in the genealogical world will force genealogy program to support its API, as many have already done.

Over the years, there have been many attempts to either upgrade or replace GEDCOM. These efforts have all failed. In general the problem has been that the companies that create genealogy program need to agree to adopt any new standard, and they really haven’t had much incentive to do so. Supporting the import of GEDCOM files allows them to support a basic file interchange, which never will support the full feature-set of their programs which have become much more sophisticated since 1996, but is enough to allow customers to exchange information with their relatives. If they supported a fully-featured GEDCOM replacement (that for example would better support photographs and evidence management), it would only make it easier for customers to try other programs. Thus the disincentive for the companies to support a modern replacement for GEDCOM.

Another problem with replacing GEDCOM has been arguments over the data model used. GEDCOM is based on a nuclear family data model (i.e. one mother, one father and their children). It assumes a nuclear family structure, and other forms of families are harder to support. This problem has caused some to support a data model based not on the family but on the individual. This is philosophical debate, and as you might imagine different people take very strong positions in this battle.

Even with this history, there are a few new initiatives to come up with a replacement for GEDCOM. One initiative that has garnered some attention recently is BetterGEDCOM. The BetterGEDCOM initiative came from the frustration of many genealogists over the lack of updates to GEDCOM and is an attempt to create an open forum for the creation of a new standard. Like many attempts at ‘openness’, however, it has run into its own in-fighting and conflicts. It remains to be seen how successful this attempt with be. Another recent initiative is the International OpenGen Alliance (OpenGen). This effort is a bit more of a top-down approach, being managed by the company that runs AppleTree.com, an online family tree web site. OpenGen is, however, a non-profit organization that is supposed to include more than just the team at AppleTree. There have been some attempts between BetterGEDCOM and OpenGen to coordinate, or at least follow each others’ efforts closely. Time will tell which effort, if either, will be successful in creating a new genealogical data sharing standard.

In case you think it isn’t complicated enough, other web sites beyond FamilySearch.org are also developing their own APIs for exchanging genealogical data. OneGreatFamily.com last year introduced an API called GenealogyCloud. It seems that no third-party applications yet support this API.

Geni.com, which boasts nearly a hundred million profiles on their site, and nearly 50 million that are interconnected in what they call their World Family Tree, just yesterday introduced their own API. Unlike FamilySearch.org, however, they are releasing documentation and sample applications on their web site. This will allow anyone to write applications that interact with Geni.com, similar to the way Facebook allows outside developers to create application that access information on Facebook. This is a very positive step. It’s not coincidence that one of the other large family tree web sites, AppleTree.com, is pushing another initiative to replace GEDCOM (OpenGen). These large sites need to create ways to exchange data and interact with other programs and web sites in order to maintain their growth rates.

MyHeritage.com, another one of the big family tree web sites, has taken a slightly different approach in that they have their own application (Family Tree Builder) that runs on a computer, which can sync data to their web site. While this approach allows them more control over what modifies data on their platform, it has its shortcomings as well, not the least of which it requires Windows to run (this coming from a Mac user). I suspect that MyHeritage.com will release their own public API in the future, if only to compete with Geni.com, their biggest competitor.

We can always hope that FamilySearch.org, Geni.com, MyHeritage.com and AppleTree.com will all come together and create a single API and data format for sharing data, but unfortunately if the past is any guide, this is unlikely to happen.

One indication of the direction the wind is blowing in this regard will be the upcoming RootsTech conference, taking place in February 2011 in Salt Lake City. This conference is the first RootsTech conference, although according to the organizers it replaces three earlier technical conferences – The Conference on Computerized Family History, the Family History Technology Workshop and the FamilySearch Developers Conference. Note that these previous conferences were all connected in some way to the Mormon church. It’s unclear how open this new conference will be to new ideas, or if it is really only looking for input for the existing Mormon church efforts such as FamilySearch.org. I imagine representatives from most of the genealogy software companies and web sites will be in attendance at the conference, as will people associated with the BetterGEDCOM and OpenGen efforts. During the week of the conference there will probably be a lot of blogging about what is going on, but the real test will be after the conference if companies announce intentions to seek a common API or data format to move forward with, or whether everyone will just continue the same disjointed approach that has been pursued for nearly 15 years.