search - B&F: Jewish Genealogy and More

I previously wrote about how many genealogy sites have been removing or restricting access to parts or even the entire Social Security Death Index (SSDI), due to pressure from lawmakers who have tried to make it seem like access to the SSDI was contributing to identity theft. I won’t go over that again, but you can read my earlier post Changes in Access to the SSDI and Vital Records.

One company, Mocavo, seems to be bucking the trend of most of the genealogy companies out there to restrict access to the SSDI, and has actually introduced a very nice new search engine for the SSDI, which seems to include all the information in the database, including social security numbers. In order to see the results, you need to sign up for a free account on Mocavo if you haven’t already. I suspect launching this SSDI search engine is largely a way to bring in users to their site. The search results also allow you to add comments, something I haven’t seen before on SSDI databases. How this will be used will be interesting.

I’m not a regular user of Mocavo, but it seems that this is the first actual database they’ve added to their site, and thus this comment feature is also new. With their search engine, they have the ability to ‘follow’ a page, but this is the first time I’ve seen the ability to comment. I don’t know if you are notified of someone else’s comments on the same record, or if you have to go back and check regularly. I also don’t see a way to send messages to other users (such as if someone commented that the person in the record was their great-grandmother, and they have a picture of her) but perhaps this is coming. If you are notified of other comments, then it might not matter too much, although without private messaging people would have to post their e-mail addresses publicly to take a conversation further, which is not ideal. As this is a brand new feature, however, I’m sure they’re working on something.

Some other features of Mocavo are also interesting, such as being able to mark a search result as already read, and being able to say that a specific result is about the person you were searching for, maybe close, not who you are looking for, or is a broken link. Mocavo presumably uses this data to improve its search results.

As I tell people who ask me about it, we don’t know if lawmakers will decide to restrict access to the SSDI in the future. I always suggest going through one’s family records and searching through the SSDI for anyone likely to have had a social security number and copying down everything into your personal database. If in the future access is removed, you may not be able to get the information later. Information that can be very helpful from the SSDI records includes birth date (take with a grain of salt), death date (more likely to be accurate), what state their social security number was issued in, the person’s last residence, and where their last benefit was sent (which may be different from last residence and indicate where a spouse moved). So if you haven’t done so already, take a look at your family tree, figure out who is likely to have had a social security number (someone who was working from the late 1930s on) and search through Mocavo’s search engine, or one of the others available online (see my older article about that).

So it’s been two weeks since I last posted, and that’s because I’ve been busy on a new project. When Mocavo, a search engine dedicated to genealogy, launched a couple of weeks ago, I was inspired to figure out exactly how they were returning the results they were returning, and how one can create topic-specific search engines.

It was not clear to me exactly how Mocavo collected its results – if for example it uses results from another search engine, and just releases the results that match a pre-set list of genealogy-oriented web sites, or if Mocavo is indeed operating its own search ‘spider’ to crawl the web and collect its own results. It seemed clear that while Mocavo did find good results within a number of major genealogy web sites, it didn’t appear to find results on many minor sites, or on major general web sites that might have small genealogy sections. For example, if someone posted a web page on their family on their own web site, or started a Yahoo Group to discuss a particular town or surname (such as described in my earlier post on mailing lists), it did not seem to appear on Mocavo. I don’t know what algorithm Mocavo uses, but I’m guessing it can’t currently find particular sites within larger general websites like Yahoo, so it ignores Yahoo altogether (to eliminate the chance of false positives).

Not knowing anything about how Mocavo put together their site, I decided to see what I could put together myself. Using tools provided by Google (I suspect Mocavo uses the same tools, just their paid versions that allow them much greater customization) I worked over the past couple of weeks to put together my own genealogy search engine. It is a bit more inclusive in how it determines which sites to search than Mocavo. It is thus more likely to find small genealogy sites, but also more likely to find some less-than-relevant results. That’s a compromise I’ve struck, which I think returns many interesting results than you might not find on Mocavo. Of course, Mocavo has the advantage of being a real company with employees who get paid to update the search results, so they can improve their results over time. As this is not my full-time job, I don’t have that luxury. Don’t think, however, that I’m trying to compete with Mocavo. This is just my own attempt at creating a useful search tool for genealogists, inspired by Mocavo.

Unfortunately one of the downsides of Google’s free search tools is ads. I can’t stop the ads from showing up unless I’m willing to pay Google for that privilege. I don’t know why Google shows ads more aggressively on custom search engines like this one then they do on their own search engine, but they do. I’m sorry about that, but there isn’t anything I can do about it.

When looking through the tools available to me, I tried to figure out how I could improve the results for genealogists. I came up with an interesting idea, but Google restricts how useful it can be. Basically when towns have undergone name changes or have different names in different languages, and a record shows up under a name of a town that is different than the version of the name you are searching, you will not get results. Google will actually help here with major cities, so for example if you search for Wien it knows to search for Vienna, but it does not know every version of every town one might be searching for, nor frankly should it as this technique can actually reduce the usefulness of search results when alternate names overlap. In any event, Google allows you to define synonyms for search terms, but limits the amount you can do.

As I was limited, I had to choose a small area to try this technique out on, and I chose the Galicia region of the former Austro-Hungarian Empire. It is a particularly good region to choose, as it has been controlled by many different countries over time, had many different languages spoken, and most towns have many names. It’s also small enough of a region that it fits within the limits of what Google allows me to do. Part of the problem is that Google only allows uni-directional synonyms, which means you need to know which town name to search or the synonyms won’t kick in. To use a set of names that was easily definable, I’ve chosen to use the names at the top of the Locality Pages from the JewishGen Community Database for the given towns. Basically, whichever name appears at the top of a Locality Page for the given town is the one you should use – except don’t use accent marks or apostrophes in the name. You should use dashes if they are in the name. Obviously it must be a town that was part of Galicia. While this will only help people who are searching with one of the hundreds of Galician towns in their search query, if you are not searching with one of these town names, the search engine will still work well to help you find results among the many sites it does search.

I am open to all feedback on this search engine, and welcome feedback in the comments. Please leave comments on the search page itself, and not on this post, as this post is just an introduction and in the future people will just go straight to the search page.

Without further ado, I introduce B&F Enhanced Genealogy Search.