Category Archives: Technical

MyHeritage announces integration with 23andMe

logo-MyHeritageIn one of the more interesting announcements in the genealogy world of late, MyHeritage has announced a partnership with genetic testing company 23andMe.

logo-ftdnaUnlike the partnership between MyHeritage and Family Tree DNA which dates back to 2008 and was extended in 2012, this new partnership is not just an affiliate sales relationship.

23andMe LogoMyHeritage has announced that they will be integrating their service directly with 23andMe, allowing one to confirm relationships in your tree on MyHeritage using the DNA data on 23andMe. Certainly a very interesting development, and it will be interesting to see how far they go with this integration.

It will also be interesting to see how the other players in this field respond to this development. MyHeritage, in addition to their namesake site also runs the large Geni.com family tree site. Will Family Tree DNA stay partnered with MyHeritage? Will they seek similar integration with MyHeritage? Will that even be allowed by 23andMe?

If not, will they seek to partner with another large family tree web site? Ancestry.com has their own family trees, but also their own DNA tests, so it would seem that’s not possible. From there, the size of family tree sites drops dramatically. TribalPages? OneGreatFamily? WikiTree? If the integration is done right, this creates a real strategic advantage for MyHeritage and 23andMe, that it would seem only Ancestry.com can really compete with at this stage.

See MyHeritage CEO Gilad Japheth speak about the new partnership on Bloomberg TV here:

and read more on MyHeritage’s blog post announcing the new partnership.

Transitioning from Blogger to WordPress

transition

[Update 5 Feb 2014: The Blogger Importer plug-in has undergone a major overhaul, and has fixed a lot of problems and made lots of improvements. One of the major improvements is the importation of images, so at the very least Step 7 below can probably be skipped. Some other tweaks might also be unnecessary. It claims to handle internal links better, for example. In any case, check each step after the import below and make sure they’re necessary.]

I’ve been using WordPress for a couple of years now on other sites I’ve developed, such as Genealogy.org.il (the web site of the Israel Genealogy Research Association), my other blog Lexigenealogy, Kanczuga.org (for descendants of the destroyed Jewish community of Kanczuga, Poland), and some others I’ve worked on, but I’ve always been hesitant to move this blog over to WordPress since I knew it would be a lot of work, and I liked that Blogger (Google’s blogging platform that I started with) was a very easy and inexpensive way to keep the blog running.

However, as time went on I felt stronger and stronger that the site needed a redesign, and I never felt the Blogger platform allowed me enough flexibility to do what I wanted. Thus, this redesign on WordPress. I welcome your comments on how the new site looks, and if you see anything that seems wrong or out of place, please let me know. As you’ll see below, it’s not so easy to transfer a blog from Blogger to WordPress, and it’s very possible some things didn’t make the transfer 100% accurately.

The rest of this post is to explain the steps I went through to transition the site, of interest to other bloggers who might be considering the same. If technical details don’t interest you, the rest of this post will be very very boring to you. For those who want to take a look at the original site, it is still live at bloodandfrogs.blogspot.com (but will not be updated in the future).

Step 1) Install WordPress

I won’t go into details here as there are plenty of places online to help you setting up WordPress, but I’ll give a quick overview. I have a web host that lets me quickly add domains to sub-directories of my main account, so I took a different domain, assigned it to a sub-directory, and installed WordPress in that directory. I set up a MySQL database on my web host, and configured this new WordPress installation to connect to that database. Using a new domain for the setup with very helpful as it allows me to have a live site working even before I consider switching the original domain over to the site. Before I can do that, I need to insure that all of the links on my original Blogger site will work on the new WordPress site.

2) Set up Permalinks

Permalinks refer to the format of the web link to your articles. It’s the permanent link to the articles you post on your site, so someone can share or bookmark a specific article (instead of just the site itself). The default for WordPress is to just show the article id number, which looks something like: bloodandfrogs.com/?p=123 Not very pretty. However, WordPress lets you set the format of the link to include the year, month, day, article title, etc. in whatever combination you’d like. Thus with pretty permalinks the link to an article might be:

bloodandfrogs.com/2013/09/transitioning-from-blogger-to-wordpress

That format shows the year, month and the name of the article. However, there’s a problem, in that Blogger has a different permalink structure, which looks like:

bloodandfrogs.com/2013/09/transitioning-from-blogger-to-wordpress.html

In order to insure that all the links from the old Blogger site work on the new WordPress site, we need the permalinks to match. Luckily, WordPress lets you customize the display of permalinks, by specifying the pattern like so:

/%year%/%monthnum%/%postname%.html

This custom pattern creates the text after the domain name in the exact same format as Blogger. Well, not exact exact. The problem is that how Blogger and WordPress determine how to format the name of the article is a bit different. Basically, Blogger cuts the number of words in a link, while WordPress lets you have as many words as you want (up to the maximum allowed in a URL). In order to insure the article title is formatted the same, you need to run a script to correct the imported articles. First we need to import the articles, however, so back to that in a minute.

3) Importing Posts and Comments from Blogger

WordPress offers an Import screen in the Tools menu of the Dashboard. In Import, there are options for many different blogging platforms like Blogger, LiveJournal, Tumblr, etc. If you click on Blogger, it will initially take you to a page to install a plug-in called Blogger Importer. Once installed, the same link on the Import page will instead take you to the settings page for the plug-in. Blogger Importer does some, but not all of the job. According to the plug-in description: Items imported

  • Categories
  • Posts (published, scheduled and draft)
  • Comments (not spam)

Items not imported

  • Pages
  • Images (the images will appear in your new blog but will link to the old blogspot or picassa web locations)

That doesn’t tell the whole story, however. When I ran the importer, it imported all my posts, and most of my comments. Why were 20 or so comments left out? I’m not sure. Not a huge deal, but I don’t even know which comments were left out. The good news is that even though it doesn’t import Pages or Images, there are workarounds for them.

4) Fixing Permalinks

Earlier, we set up Permalinks to be like Blogger, but as I mentioned, it’s not exactly like Blogger. Now that we’ve imported the posts, we can go back and fix the links that need to be fixed. For this, I used a PHP script from rtCamp from their guide on moving from Blogger to WordPress. They provide code that you copy into a text file with a .php extension and upload to your server. Once it’s in your WordPress directory, you load the file from the web and it processes all the link names and fixes them. If you have internal links to those pages, I’m not sure if it would fix those. Probably not.

In the end, there will be lots of little details like this that need to be checked and manually fixed. Once you’ve run the script, you could and should delete it from your server. In theory, all of your posts are now exactly what they were on Blogger, and assuming you haven’t changed your domain then you shouldn’t need any fancy redirection tools to keep external links to your site, since they should be identical to the old site (at least for Posts).

5) Fixing Tags

Blogger doesn’t have Categories for Posts, but it does have Tags. Oddly, when importing the Tags from Blogger they become Categories in WordPress. It’s odd because WordPress also supports Tags, and one would think Tags would be imported as Tags. In order to fix this, I found a plug-in called simply Categories to Tags Converter, which does exactly what it says. The plug-in hasn’t been updated in over two years (usually a bad sign) but it worked great and everything seems to have been converted properly.

6) Importing Pages

Unlike Posts, there is not a simple way to import your Pages. Hopefully you don’t have as many pages as Posts. Although in my case I had hundreds of Posts, I only had five Pages. Here’s what you need to do to get them on your site. Create corresponding Pages on your WordPress site for each of your Blogger pages. Copy the text from each Blogger Page, and copy it into the corresponding WordPress Page. Depending on your Pages, this might be good enough.

If you have complicated code in your pages, you may need to go back and make changes. In most cases with my pages everything worked fine. I did have some internal links to other Pages and Posts that I needed to modify, and in one case I had some custom CSS I had created that I needed to track down in Blogger and copy over to WordPress.

It’s good to view every page and test all the links, compare the appearance between the old and new sites, etc. This isn’t a perfect process, and mistakes will pop up.

Once your Pages look and perform the same, you need to add redirections. These are codes that tell the browser (or search engine) that the location of the resource they are looking for has changed. It’s important to use what’s called a 301 redirection, which tells the search engines like Google that this is a permanent change, and allows you to retain any search rankings you received on the previous page on the new one. The reason this is necessary is that Blogger places all of its pages in a directory /p/, and WordPress doesn’t let you do this. Thus, you need to have a different URL for Pages, and need to direct incoming traffic from the old /p/ location to the new one without the sub-directory. Therefore, a page on Blogger might be called:

www.bloodandfrogs.com/p/search.html

but on WordPress it would be:

bloodandfrogs.com/search

and you need to tell the browser to search engine how to figure out that change.

For this problem, I use a plug-in called simply Redirection. Redirection lets you set up a series of redirections, where you simply give it the original URL and the new URL. I set up redirections for the five Pages, and I’m done. Keep in mind that since the URL is changing anyways (removing the /p/ at the minimum) you don’t need to keep anything else in the name the same. The redirection will work even if the Pages have completely different names. Indeed if you wanted to delete a Page altogether, you could create a redirection to send people trying to reach that page to someplace else on your site.

7) Importing Images

Earlier, when we imported the posts from Blogger, the images were not imported. Since the images are stored on a Google server somewhere with a link that does not include your blog’s name, you could just leave those images alone and let them load from Google’s servers. Of course, if you ever close the account you used to set up your Blogger blog, then all those images will disappear. I also would feel a bit like I was taking advantage by using their image hosting for Blogger but not actually using Blogger. Technically there’s nothing wrong with it, as it’s all part of your Google account, but it still seems off to me. In any case, I’d prefer to have my images local to my blog.

There are a few plug-ins that allow you to import images from Blogger. I used one called SEO Blogger to WordPress Migration using 301 Redirection which in addition to doing redirection also has a module for importing images. After it imports all the images, it goes through all of your posts and changes the links from the original links to their new locations on your server. Pretty neat.

There’s another plug-in called Blogger Image Import that does the same thing, although I haven’t used it. With any plug-in like this, however, make sure to backup your blog before using it, as it is making a large number of changes and if it messes up, you’ll be in trouble. There’s no undo with these. I did this step before I imported my Pages, so I’m not clear if the links would be updated on Pages, or just in Posts. In any case, the images used on Pages will be imported, I’m just not sure if the links to them will similarly be updated. Since I don’t use a lot of images in my Pages, this wasn’t a big deal for me. I just went in and manually modified a couple of links to the images.

8) Adding Google Back

Since Blogger is a Google platform, it has built-in support for a bunch of Google technologies, like Google Analytics, Google+ Comments and Feedburner. These are not built into WordPress, so if you want them back you need to add some code to WordPress. There are plug-ins that can help.

For Google Analytics, I used Joost de Valk’s Google Analytics for WordPress. You install the plug-in, log into Google and give permission for the plug-in to access your Analytics information. Then you just select your site from a drop-down menu, save the settings, and you’re done.

For Google+ Comments, I’m not settled on exactly what I want to use. It’s only in the last few months that I switched to Google+ Comments on the blog, and with mixed results. I think the Google+ comments are better than the original Blogger comments, but there’s no way to really track them. I don’t always know when new comments are posted, even though I think I’m supposed to be notified. If I change the domain name in the future, all the Google+ comments will be lost as well, which is bizarre to me. This is the problem with using an external commenting system.

For the moment, I’m looking at different options for comments. Some plug-ins such as Social Comments promise to integrate WordPress, Google+, Facebook and Disqcus comments all together. What commenting system do you use on your blog? Right now, unfortunately, all the comments added through Google+ in the past few months are now missing from the site.

As I already use Google Feedburner for my RSS feeds, all that needs to be done to keep the feed going is to log in to Feedburner and change the URL of the original feed from the Blogger location (/feeds/posts/default) to the WordPress location (/feed). Since people should have been directed to the Feedburner address (http://feeds.feedburner.com/bloodandfrogs) instead of the Blogger address, no one who subscribes to this blog should see any difference (although how the blog posts look will likely be a bit different).

9) Widgets

Blogger has a lot of pre-built Widgets to place in various places in the layout of the blog. Mostly, I used them in the sidebar on the right side. One thing I like in Blogger is the appearance of their Archive widget. It shows all the posts of the most recent month, and allows easy navigation to find other posts from previous months and years. WordPress doesn’t support this directly, but I found an old plug-in that does something very similar, called Collapsible Archive Widget. It hasn’t been updated in over two years, but so far it seems to be working pretty well.

By default it doesn’t show the post names, because that would require loading all post names from the whole history of the blog into the page. Presumably Blogger itself doesn’t do this, but loads the data as needed, but this plug-in doesn’t seem to do that, and warns against using this feature if you have a lot of posts. I’ve turned it on for now, and will see how it goes.

Some widgets I had were just HTML code, so I just copied them over. The widgets for things like Google+, Google Friend Connect, etc., however, were part of Blogger and I will need to find replacements for them. I haven’t reproduced everything in my right-column on the old site, but I hope to add more to in in the future as I get things up to speed.

For Twitter, I used a widget that comes with Jetpack. There’s also a Facebook widget, but I’m using Facebook’s own widget, so I didn’t bother to use Jetpack’s.

10) Transfer the domain

This section is for those who registered a custom domain through Blogger when they still allowed that option. I set up my blog with a custom domain back in 2010 (i.e. my blog is on bloodandfrogs.com, not bloodandfrogs.blogspot.com). Google will no longer register a domain for you, but they did then, and I’ve renewed through them each year since. So how does one transfer a Blogger custom domain to another registrar?

You can’t imagine how complicated it is…or rather you can’t imagine how complicated it is to figure out how to do this…

There is nothing in Blogger that tells you anything about the fact that your domain was registered via Google, or how to access it. Apparently, what happened is that a Google Apps account was set up for your domain, with an e-mail address they never tell you about, and you need to figure all this out. Here’s the deal:

1) You have an account on Google Apps with the email address [email protected]yourdomain.com (obviously with your domain instead of yourdomain.com)

2) Using the following URL, you can reset the password for your Google Apps Account: http://google.com/a/cpanel/yourdomain.com/ResetAdminPassword

3) Once you’ve reset the password (they send an e-mail, you click on the link, fill out the form, etc.), you log in to Google Apps, and go to the Domains section, and then click on Advanced DNS Settings. That gives you the login credentials to connect to the registrar which actually controls your domain (either enom or godaddy probably). Once you connect to the registrar, you can change the name servers to the name servers of your self-hosted WordPress site (or transfer the domain to another registrar).

Transferring the domain is actually very complex. I won’t go into all the details here, just keep in mind that once you can access the registrar (in my case it was GoDaddy) you need to both unlock the domain AND remove privacy protection. When I first tried to transfer the domain I unlocked it (which allows it to be transferred) but when I tried to transfer it, it was rejected. Privacy shouldn’t affect the transfer process, but here it does. The domain was connected to a third-party proxy service which needed to be disabled. Get this, the username for this service is also something not given to you. By recovering the username, I found out it’s one digit different from the username for the registrar. The password was the same. How are you supposed to know this? I have no idea. Thus I’m sharing what I discovered here.

At this point, make sure to set the nameservers for your domain to match your web host’s name servers. You could have specified this when you initiated the transfer, but if not, make sure to change the nameservers now. If you think you can keep the Google nameservers when your transfer your domain (i.e. to point to your Blogger blog until you’re ready to switch to WordPress), let me clear that up – you can’t. Google will disable the DNS for the domain as soon as your transfer it away. Basically, when you transfer the domain, have the new site ready.

Lastly, don’t forget to decouple the yourdomain.blogspot.com domain from the custom domain. This is easy enough to do in the Blogger settings – just go to Settings – Basic, and under Publishing, it should show both your custom and standard blog address. Just click the X next to the custom domain, and it will be removed. Once you do this, there will be no DNS support for your domain from Google, so when you do it make sure your new site is fully set up and ready to go…

11) Switching the domain in WordPress

In WordPress settings, you can define your WordPress Address and your Site Address. If your site is in the your primary directory for the domain, then these will be the same. If you want to keep WordPress in a subdirectory, but still have it show up as the main site, then these will be different. I originally set these up with my dummy domain for testing.

Once the domain was being transferred, I went into WordPress Setting -> General and switched these to be https://bloodandfrogs.com. I had already configured the domain on my web host to point to the same directory for both my dummy domain and the real one, so there was nothing needed there. When the domain transferred, everything worked properly.

12) Other services – Twitter and Facebook

In my original blog, I used a service called Twitterfeed which looked at the RSS feed of my blog and tweeted new posts automatically (to my twitter feed twitter.com/bloodandfrogs). WordPress allows more control over the Twitter process through many different plugins. Although I could have left the Twitterfeed service running (since my feed didn’t change) I decided instead to switch to using a plugin called WP to Twitter, which automatically tweets new posts among other functions, and offers a much larger degree of control.

I also used an RSS-based solution for posting to my Facebook page (fb.com/jewishgenealogy) called RSS Graffiti, which you might have noticed when this site moved, as it must have detected a different RSS feed and re-posted the last 10 posts or so to Facebook. That’s the nature of an RSS solution. It’s not a fine instrument, but a blunt force tool.

For Facebook I had wanted to use the official Facebook plugin for WordPress, which I use on other sites, but for some reason it wouldn’t let me connect to my Facebook account. Instead I’ve used Jetpack, a plugin that connects your self-hosted WordPress blog to WordPress.com and adds many features, which has a social publishing feature called Publicize that supports Facebook, Twitter, LinkedIn, Tumblr and Path. I may switch to using Jetpack for Twitter as well, but for the moment I’ve left that to WP to Twitter.

Conclusion

That was a lot of work, and there’s still a lot left to do. There are lots of little things to fix, and eventually I need to actually take advantage of WordPress and add additional functionality to the site. My initial goal was just that  everything from the old site worked properly on the new site, and that all the links worked.

I also haven’t categorized any of the posts, something I probably should go back and do since Blogger doesn’t have categories, and everything on the site is now ‘Uncategorized’.

Lastly, one odd thing is that images in my posts that were centered in Blogger, and not centered in WordPress. I need to go back and center images. I may not do that with every post, perhaps just the most recent and the most popular.

Did you find this guide to transitioning from Blogger to WordPress useful? Let me know in the comments.

Scanning documents and photos using your phone

As the cameras built-in to cell phones get better and better (and indeed every year the improvement has been significant), more and more people are using their phones instead of dedicated scanners for duplicating documents and photographs.

There are many iPhone apps for scanning documents, such as CamScanner Pro, Doc Scan Pro, Genius Scan+, JetScanner, Prizmo, Scanner Pro, TurboScan, and the genealogy-focused Shoebox (now owned by Ancestry.com).

Android also has their share of apps, like CamScanner, Document Scanner, Droid Scan Pro PDF, Handy Scanner Pro, Mobile Doc Scanner, and there’s also an Android version of Shoebox.

Other popular apps like Evernote are also suitable for this kind of document and photo scanning.

Perspective correction in Genius Scan+

Most if not all of these apps do something pretty neat, which is that they will take your document or photo and if the corners are not 90 degrees, it will correct the perspective. This is necessary because most people holding a camera in their hand cannot possibly get the camera lens to be perfectly parallel to the document or photo they are photographing. This causes the photo they take to be skewed (where for example one side of the photo is smaller than the other side), and thus the need for this correction technology.

The problem with this technology is that you can never correct something like this without losing some quality. Wouldn’t it be better if you could insure your phone was parallel to the document when you take the photograph? When people traditionally took photographs of documents, they used big bulky copy stands. They are generally bulky and expensive. So what solution exists for your cell phone? Turns out this is a problem several people have taken a look at, and there are some very interesting solutions.

ScanDock

Let’s start out with free. Designer Kyle Koch has designed a cardboard stand he calls the iPhone Document Scanner, which he provides the plans for in EPS format. You can print out the designs and cut these out using cardboard or other materials yourself.

ScanDock

If you want to order it pre-made, it’s also available from Ponoko in either corrugated cardboard or MDF. A newer version, called the ScanDock (maybe soon to be ScanDeck as there is another ScanDock – see below), is available only in cardboard, and the plans are not available for free. Oddly, the different products all have different and bizarre shipping options. The best option seems to be the ScanDock which costs $25 and can be shipped to the contiguous US for $8.50. If you’re in New Zealand, shipping is only $3. I think this is really just a good do-it-yourself solution, I’m not sure I would pay for this over the options that follow.

Steady Stand

Another option is a series of stands from Scottish photo accessory company Modahaus, which they call Steady Stands. These stands are manufactured out of plastic, and are design to diffuse the light that hits the document or object inside the stand, which helps eliminate harsh shadows.

Steady Stand 200

The Steady Stand comes in a variety of sizes, and can be used to photograph physical objects, or used to create videos by dragging the whole stand over an object or document (such as a map) to create steady videos. I could see using the smaller one as a kind of Flip Pal replacement. The downside is that the FlipPal doesn’t need lighting, the plus side being that you don’t need to bring a full scanner with you, and the object you’re scanning doesn’t need to be flat. The smallest stand is £15 (yes, that’s British Pounds – currently about $23.37) and the largest is £31.20 (currently about $48.62)

ScanJig

The only solution I’ve seen that doesn’t shoot straight down onto a flat document, is the ScanJig. The ScanJig is also unique in that is designed to work with a larger variety of devices, from iPhones up to iPad Minis. There is a separate version for the Galaxy III and IV phones as well.

ScanJig

This is perhaps the most complicated stand in terms of setup. I’m not sure sure this is something I’d want to carry around with me. For some types of documents and workflows, this might be a good solution. I image if it’s on your desk and you’re copying lots of sheets of paper, this might be easier to swap out pages quickly. The ScanJig is currently $29.95 (although it says it is regularly $39.95) plus shipping (which was $4.95 to Boston).

StandScan

One of the most interesting solutions is called the StandScan. It has two features that I really like – it folds flat, and it has the option to add LED lighting. Of course, due to its design that has opaque walls on three sides, it’s possible the LED lighting would be required to get good quality in many lighting situations, so maybe it’s not an option after all, but a requirement.

StandScan

You can get the StandScan with or without the LED lighting, and there are two options for powering the lights, 9V (1 9V battery) or 12V (8 AA batteries). The stand without lighting is $19.95, the stand with lighting and a 9V battery pack is $29.95, and the scanner with lighting, both battery packs and an AC adapter is $37.95. Shipping seems to be $15, whether to the US or it seems anywhere in the world (and by that I mean I check how much it cost to ship to Israel).

Scandock

One other option is Scandock, which is built like a more traditional copy stand, with built-in lights on overheard arms, and a weighted base. It has its own camera app, which uses color patches on the stand itself to insure proper color calibration, and the stand has a silicon cover for documents to insure they lay flat.

Scandock

In short, the Scandock is much more advanced than the offerings above, but is also priced accordingly, which the cheapest version costing $279. Also, it’s not exactly a portable solution.

Lomography Smartphone Scanner

If everything so far seems more or less the same, you won’t get this one confused with the others. That’s because this product is not for scanning documents, but for scanning photo negatives and slides. It’s from analog photography company Lomography, and it’s called the Smartphone Scanner.

Smartphone Scanner

This device has a light source on the bottom, and you put in a transparency and scan it using your phone. It supports all iPhones and several Android devices. It has its own iPhone app, although you don’t need to use it (it is, however, helpful if the app you’re using can reverse the colors of image it captures).

The price is $59 in the US, and €59 in Europe, plus shipping. It’s also available for $59 (including shipping in the US) from photojojo, one of my favorite photo gadget companies.

Interestingly, two other phone stands were crowdfunded on Kickstarter, and succeeded in being shipped out (not as common as you might think with hardware products), but both seem to have disappeared – ScanBox and Scandy. The team behind ScanBox seems to have moved onto other products (LED light bulbs), while Scandy’s web site has disappeared completely.

I haven’t used any of these solutions unfortunately, but if one of the companies wants to send me a stand, I’m happy to review it on the blog.

Have you used any of these products? What software do you use for document scanning with your phone? Share your experiences in the comments.

Collaborating without having to be in sync

I’ve mentioned the Family History Information Standards Organization (FHISO) previously. It is one of two organizations, along with FamilySearch (with their Gedcom X effort), trying to define future data formats for genealogy. 

Back in March, FHISO announced an open Call for Papers in order to solicit ideas for future genealogy standards. This is the first step in the FHISO’s efforts to create new standards.
Yesterday I submitted a paper, titled Asynchronous Collaboration: A Proposal, which outlines my ideas for facilitating collaboration between different researchers, while not forcing researchers to fully merge their databases. The key here is that people can accept family trees from other people, without having to merge their entire tree into their own, and with a query mechanism for figuring out conflicts, or requesting additional information like sources and media related to individual records.
This proposal submitted to FHISO and published on my other blog, Lexigenealogy, which is where I now publish my more technical genealogy writings, as well as other technical work related to lexicography.
Keep in mind that this is a fairly technical proposal. It’s not really light reading. If you’re interested in the technical aspects of genealogy, and in furthering the creation of new technical standards, I think you’ll find it interesting.
If you’re interested, hop over to Lexigenealogy and take a look.

Evidence-based Genealogy

Most people who research their family carry out conclusion-based genealogy. Entire family trees are built solely based on conclusions, i.e. who a person was, when they were born, where they died, etc. sometimes with little evidence to back up the conclusions being made. While building a tree without sources does not necessarily mean the tree has faulty information, it is impossible to show what is right and wrong. Where we especially run into trouble is when we try to collaborate with others in our research. Most of the popular genealogy programs that people use (Family Tree Maker, Legacy Family Tree, MacFamilyTree, Master GenealogistReunionRootsMagic, etc.) are conclusion-based. One can use them to build a 10,000-person tree without entering a single piece of evidence. That’s not to say anything bad about these programs, they all have the ability to cite sources for each piece of data added to their databases. The problem is they do not require citations.

There is another model for genealogy research – evidence-based genealogy. This model starts with the sources. Instead of the individual being the core of the tree, the sources become the core. In most cases these programs are designed as a kind of companion app for conclusion-based apps like those mentioned above. The focus of these apps are not producing nice charts, but create a solid chain of evidence to back up each piece of information you are researching.

One of the ways evidence-based applications work is by developing a series of source templates for all kinds of sources, insuring you extract all the information out of a source when entering it into the system. One example of this are templates for each year of a census having their own templates, as each year the questions were slightly different.

In general the applications available for doing evidence-based genealogy seem to be intended as companion apps to traditional conclusion-based apps like those mentioned above. Below I’ve linked to the evidence-based applications I was able to find under active development, with blurbs from their web sites describing their applications. My observations follow this listing.

What is Clooz 3?

  • A Program to Consolidate, Index, Analyze and Report Document and Image Data
  • A Family History and Genealogy Research And Analysis Tool
  • A Windows Desktop Application

Why Should I Use Clooz 3?

  • Gather, Analyze and Validate Clues and Evidence About Potential or Suspected Ancestors
  • Analyze Family History Challenges Using Factual Document-based Research Strategies
  • Organize and Index Collected Documents and Images
  • Support “One Name” Surname Studies (large or small)

Welcome to Custodian 3, the database software which helps you to store, index and organise the information you have gathered from all kinds of family history records.

Use Custodian for general family history research, one-name studies, indexing projects, local history and one-place studies. Keep a computer-based version of all your paper records, documents and lists in one place and minimise endless searching for paperwork.

A New Kind of Genealogy Tool

  • Evidentia is genealogy software created from the ground up for the user who wants to take their research to the next level. It turns what you know into evidence you can use.
  • Too many conflicting sources about Grandpa William’s birth? Evidentia presents all your evidence on one screen, making it easier for you to separate fact from fiction.
  • Not sure how strong your evidence about when and where your great grandparents were married is? Evidentia’s reports help you to identify the gaps in your sources.
  • Evidentia is the software program that will help you feel confident in your research.

The Family Pack is an ambitious open source project to create a new cross platform genealogy program. The project will involve designing a new genealogical database, creating a program to make use of it, and finally, organising ways of providing some standard universal data sets. 

The database design owes much to the impressive GenTech Genealogical Data Model. One idea taken directly from the GenTech GDM is that of Personas. A personas is a small separate particles of information that relates to a person. We can collect these personas from separate sources and then build them up to create individual person. The database uses the concept of a Reference Statement to link together Personas, Events, Dates, Places and Attributes (Characteristics) records. This idea is loosely based on the GenTech’s Assertion record. The Reference Statement is the heart of the new database and provides a very flexible way of entering and organizing evidence. It can consist of any typed statement and could be a simple statement by the researcher reviewing other statements, a transcription of a historical documents (certificates, census, parish records etc.), a description of a photograph or transcribed tape recording.

What is GenQuiry? 

GenQuiry is designed to help you manage your family history research. It was born through my own experience of researching families in Wales, trying to make sure that I made the most of my visits to archives 300 miles away, keeping track of the searches I had done and the clues I still had to follow up, and recording my reasoning for deciding which was my ancestor out of three “John James” all living in the same small Pembrokeshire parish around 1800! 

If you’ve ever had problems:

  • keeping track of searches you’ve made — and their results, positive or negative — and the searches that you plan to make
  • planning what to do on a visit to some archives, making sure you don’t forget anything vital at the far end of that long trip
  • keeping track of which sources are relevant for a particular place and how to access them, so that you don’t overlook an important clue because the existence of a source slipped your mind
  • citing your sources consistently, so that you always know where a particular piece of information came from
  • recording how you reached a conclusion from a variety of conflicting evidence

then GenQuiry may help you. 

GenQuiry can support: family history research, one-name studies, one-place studies or research into more general topics. You can use any or all of the features, to suit the way you choose to work, and integrate it with your existing physical or electronic filing system.

While some of these applications have been around for a long time, more of them are new. This category is emerging, and I suspect there will be more applications and services that will enter this space in the future. Ideally the features of these apps would be combined with a mainstream conclusions-based application, to allow one to carry out both aspects of genealogy – person-centric conclusion-based genealogy, and document-centric evidence-based genealogy.

Clooz has been around since 1997, and is available for Windows only. Originally developed by Elizabeth Kelley Kerstens, it was sold in 2011 to Ancestral System LLC, which developed version 3 in 2012. Clooz has a mailing list, Facebook page and a separate Rootsweb mailing list.

Custodian has been around almost as long as Clooz, coming on the market in 1998. It is also Windows-only. Developed by husband-wife team Sonja and Phil Smith, the latest major version, Custodian 3, was released in 2003. Developed in the UK, it is my impression the application is a bit skewed towards UK sources, but it does provide many templates for sources in the US and elsewhere. I couldn’t find any community forum or social media page for Custodian.

Evidentia is a relatively new application, released in December 2012. Its developer, Ed Thompson, previously developed a WordPress plugin called RootsPersona for publishing family trees online. Developed using JavaFX, the application can run on Mac, Windows and even Linux. Evidentia is under very active development, and it is evolving quickly. One thing the author has tried to do recently is bring his templates in line with the formats from Elizabeth Shown Mills’ Evidence Explained. Evidentia is also designed to support the Genealogical Proof Standard. Lastly, Evidentia has a very active Google+ Community where the author interacts directly with users. It has been interesting to see the fast evolution of Evidentia, and I’m looking forward to see how it continues to evolve. One thing the author recently mentioned in the Google+ community is that version 2 will directly support images of sources, something it does not currently do.

The Family Pack is an open-source evidence-based genealogy program being developed by Nick Matthews. The application is roughly based on the GenTech Genealogical Data Model which was developed by NGS. Currently it is in pre-alpha, meaning it’s not ready for regular use. It’s also currently only being released for Windows, although it is being designed to be compiled for Linux and Mac as well (it is written using the cross-platform framework wxWidgets). The first version released was only a few months ago in March. It is still a long way from reaching production stage, but it’s worth keeping track of…

GenQuiry is a Windows application being developed by Helen Wright. It is based on Microsoft Access, and thus is unlikely to make it to another platform. It is currently in beta testing, and supported by donations. It’s not clear if that model will continue after the program reaches production. The first beta was released in January 2012. One can communicate with the author in the applications’ forum. GenQuiry was a finalist in the Rootstech 2012 Developer’s Challenge.

I haven’t used any of these applications, but I hope to work evidence-based genealogy into my genealogy workflow soon. Have you used any of these programs? Do you know of others? What do you think of them? The authors mentioned are certainly welcome to chime in as well.