B&F has a new server

The short version of what I’m about to write is that this web site is running on a new web server, which should mean the site will be running faster than ever before. If you see any problems on the site, please contact me so I can fix them. For those interested, I am including more details below.

I am grateful for all the users of this site, and hope everyone who has visited has benefited in some way when they’ve come to the site. This web site has been around since 2010, when it was originally launched as a Blogger site. Back in 2013 I moved the site to self-hosted WordPress, which allowed me to expand the site’s content and functionality, but also meant I had to maintain the site and had to deal with the fact that many other sites were running on the same server. My web host was actually great, and I continue to run other sites on the same host without any problems, but they had no good solution for moving this site to a bigger server when it needed it.

B&F Compendium of Jewish GenealogyWhen I created the B&F Compendium of Jewish Genealogy, I stretched the capacity of the server to its limits. The average WordPress site only has a few Pages (as opposed to Posts, of which there can be many), maybe a few dozen, but the Compendium uses over 25,000 Pages and is continuing to grow. Whether WordPress was the right platform to develop the Compendium on is a different question, but as I had it running in WordPress I had to find a way to increase the capacity of the server without breaking the bank (since this site is a labor of love, and I make no money from it).

For those who use WordPress, think about the fact that on the WordPress editor page, all Pages are loaded into a drop-down menu for selecting a parent page. Imagine now that you have 25,000 pages and that the menu is obviously part of the page that loads.

Amazon LighsailOver the past several months I’ve been working to move my site to a service operated by Amazon called Lightsail. Lightsail is essentially a simplified version of Amazon’s EC2 cloud computing service. You pay a set amount a month and get a VPS server that you control. For $5 a month you get a Linux server with 512MB of RAM and 20GB of storage, and the best part is that you can easily upgrade the server if you need more power. The $5/month is more than I paid before, but it’s still quite reasonable for the added power I get, and if I need to I can move to a more powerful server for simple increments in cost.

Migrating a web site to a new server is never easy, and not to be taken lightly. There are always unexpected problems, and my site had some fairly unique problems. One of the problems I ran into was that the database that holds all the information for the site was so large that it was difficult to even export to a format I could move to the new server. At first the exported database was over 300MB. I looked into the database and saw a lot of the records had to do with plug-ins I used. I had to disable those plug-ins, and remove their data from the database manually, which brought the size down to a still difficult 100MB. When I was finally able to export the database, I found it impossible to import to the new server. The web interface to the database, which was the easiest way to import the data, would run out of memory before completing the import. I tried uploading the file to the server directly and importing the data via the command line, but still had trouble.

Eventually I found a great little piece of code that helped me import the data by breaking it down into smaller chunks and importing it piece by piece (BigDump, which I recommend highly for those who need to import large databases). However, even with this new tool I ended up with an error message I didn’t understand. After asking for help online, I found out that the error was due to the original web site running on an older version of the database software (MySQL) and that the newer version didn’t allow a date field to be empty (set to 0 essentially). When you had a data field in the database that you didn’t have a value for, it was supposed to be set to Jan 1 1970 instead. Go figure, but I had 53 times in the database where I had a zeroed out date, and doing a find and replace in the database fixed the problem and the database finally imported.

Other problems were more mundane. As I tried different parts of the site, I noticed certain things didn’t work properly.

The contact form didn’t work, which it turns out was because the new server didn’t handle mail the same way as the old server. After re-configuring the mail, the contact form began to work.

The maps on the Compendium city pages were not being displayed because for some reason Google thought this was a different site. After setting up new credentials for the the Google Maps API, the maps began to work again as well.

The more insidious problems had to do with file and folder permissions. Having moved much of the files over from my old server, the file permissions on many things were wrong. Now I find file permissions in UNIX to be a form of the dark arts, but slowly I’ve been fixing the problems. I noticed, for example, that I couldn’t upload new images, which is because the server couldn’t create a new folder to upload images to for this month. Sure enough, it was a permissions problem. Upgrading plug-ins has also been difficult due to permission issues. This is probably an issue that will continue to cause issues for some time until I’ve worked out all the bugs.

For the last few days the site has been running completely on the new server. I imagine there will still be problems going forward, and I ask that if you run into anything you think is weird on the site, please please contact me and tell me what you saw. As with the above examples, the problems can be hard to predict, so if you see something odd, such as an error message, or even a missing image, don’t assume I already know about the problem. Please contact me and let me know, so I can fix it for everyone.

Avotaynu Black Friday Sale

Black Friday sale on Avotaynu books

[Note that this sale is now over]

It’s hard to do Jewish genealogy and not come across the books and other publications of Avotaynu, the long time publisher of books for Jewish family researchers as well as the publisher of their eponymous journal. While many printed publications have been overtaken by the Internet, many are just as useful today as when they were first printed. This is particularly true of the many books on names by Alexander Beider.

Avotaynu tries to estimate the total number of books it will sell to avoid doing re-prints, and as such they end up keeping many books in their warehouse. In order to cut down on their inventory they’re having a Black Friday sale (well, actually November 19-27, so more than a week) on many of their books. Of note, when these books sell out there are no plans to re-print them, so if you’ve been thinking about buying one of these books, this might be your last chance.

Avotaynu Black Friday Sale
Avotaynu Black Friday Sale

The books on sale are Avotaynu Guide to Jewish Genealogy, Dictionary of Ashkenazic Given Names, Dictionary of Jewish Surnames from Galicia, A Dictionary of Jewish Surnames from the Russian Empire: Revised Edition, Jewish Personal Names, A Dictionary of German-Jewish Surnames, Russian-Jewish Given Names: Their Origins and Variants, Where Once We Walked: Revised Edition, A Practical Guide to Jewish Cemeteries, Every Family Has a Story, Biographical Dictionary of Canadian Jewry: 1909–1914. Births, bar mitzvahs, marriages, deaths and other records of Canadian Jewry, Library Resources for German-Jewish Genealogy, Jewish Vital Records, Revision Lists in the Lithuanian Archives, and Eliyahu’s Branches.

See the sale page for the specific discounts which range from 40% to 74% per book.

Note that you need to clock on the Order Now links on the sale page in order to get the discount. If you go to the regular order pages for the books it will charge you full price.

Lastly, I don’t get anything from Avotaynu for promoting this sale. I just like a lot of the books they have on sale, and I want people to know about maybe not being able to get them at a later date. You’re welcome to tell them you heard about the sale on this site, but I don’t benefit from any sales they make.

Introducing archival records info in the Compendium

I’m happy to announce a new feature of the B&F Compendium of Jewish Genealogy. Last month I added over 150 Polish towns to the Compendium, adding to the over 200 towns I added back in August, and bringing the total number of Polish towns to over 1350. Those new towns were in preparation for the feature I am introducing today.

For nearly 800 towns there is now information on what archival records exist for those towns, and links to the sites that have further information on those records.

The information currently comes from two sources:

The first source is the Polish State Archives (PSA), where I provide links to information on all Jewish records listed in their PRADZIAD database. Additionally, I provide links to their szukajwarchiwach.pl site which provides further information on the records, and in many cases provides the digital scans of the records themselves.

The second source is FamilySearch. As you may know, FamilySearch has millions of microfilms they have collected over decades, which are now on their way to all being digitized and placed on their web site. Unfortunately, most of those films can only be accessed at their Family History Centers or a FamilySearch Affiliate Library. For all of the films that have Jewish records from towns currently in Poland I provide links to the FamilySearch Catalog page that lists the film, as well as a link to the film itself if it has been scanned.

It is my hope that in the future I will be able to add information from German and Ukrainian archives, as well as any other archive with records on Jews from towns currently in Poland, and thus build a complete picture of what records exist for the towns in the Compendium.

Here’s how it works. When you go to a town page (see the full List of Polish Cities), if there are archival records then at the top of the list of resources you’ll see a green box (this may change later) that tells you how many listings exist from each source, and a link to display all of them. If you click on the link you’ll go to a separate page that lists all of the archival records, with links to their original sites to find out more.

As an example, let’s take a look at Kraków. In the picture below you can see part of the Krakow page, and if you look below the crest and map, you’ll see the green box under the heading Archival Records. In the box it says that there are 7 listings from the Polish State Archives for Krakow, and 25 from FamilySearch.

We can then click on the link in the green box to go to the Archival Records for Krakow page. On that page you will see the 7 listings from the Polish State Archives, and then the 25 from FamilySearch. In the picture below you can see the last two listings from the Polish State Archives, and the first two listings from FamilySearch:

For each listing I show the archive that the records are in (or were in, in some cases). For the PSA I note the Fond number and the name of the Fond (in Polish). For FamilySearch I list the Film number, and if there is one, an item number indicating where in the film the records can be found (in films that have more than one set of records).

Note that for each listing there are three links.

For the two PSA listings at the top, there is a link to the PRADZIAD catalog, a link to the szukajwarchiwach.pl site, and a numbered link in the Comments column that goes to the resource page for that listing.

For the two FamilySearch listings on the bottom, there is a link to the FamilySearch Catalog, a link to the film itself (if scanned), and again a numbered link in the Comments column that goes to the resource page for that listing. Note the icon of key with a red X next to the Film links (), which indicates that the film can only be viewed online while in a Family Research Center, or a FamilySearch Affiliate Library (a searchable map of Family Research Centers and FamilySearch Affiliated Libraries). When the film is viewable online from any location, there will be no icon. If the film has not yet been scanned yet, then there will be no film link at all.

It’s worth noting that FamilySearch has indicated that all of their films will be scanned in the next few years, so you should always check the Catalog entry and double-check to see if the film has been scanned. If you find a film that has been scanned that has no link in the Compendium, then please click on resource page link (the Comments number), and add a comment indicating that the film is now scanned so I can add the link. About 84% of all the film listings are locked (viewable only in Family History Centers and FamilySearch Affiliate Libraries), 13% are unlocked, and the remaining 3% are not yet scanned.

I hope people find this useful. As always, let me know what you think and if you find any problems.