Tag Archives: wordpress

B&F has a new server

The short version of what I’m about to write is that this web site is running on a new web server, which should mean the site will be running faster than ever before. If you see any problems on the site, please contact me so I can fix them. For those interested, I am including more details below.

I am grateful for all the users of this site, and hope everyone who has visited has benefited in some way when they’ve come to the site. This web site has been around since 2010, when it was originally launched as a Blogger site. Back in 2013 I moved the site to self-hosted WordPress, which allowed me to expand the site’s content and functionality, but also meant I had to maintain the site and had to deal with the fact that many other sites were running on the same server. My web host was actually great, and I continue to run other sites on the same host without any problems, but they had no good solution for moving this site to a bigger server when it needed it.

B&F Compendium of Jewish GenealogyWhen I created the B&F Compendium of Jewish Genealogy, I stretched the capacity of the server to its limits. The average WordPress site only has a few Pages (as opposed to Posts, of which there can be many), maybe a few dozen, but the Compendium uses over 25,000 Pages and is continuing to grow. Whether WordPress was the right platform to develop the Compendium on is a different question, but as I had it running in WordPress I had to find a way to increase the capacity of the server without breaking the bank (since this site is a labor of love, and I make no money from it).

For those who use WordPress, think about the fact that on the WordPress editor page, all Pages are loaded into a drop-down menu for selecting a parent page. Imagine now that you have 25,000 pages and that the menu is obviously part of the page that loads.

Amazon LighsailOver the past several months I’ve been working to move my site to a service operated by Amazon called Lightsail. Lightsail is essentially a simplified version of Amazon’s EC2 cloud computing service. You pay a set amount a month and get a VPS server that you control. For $5 a month you get a Linux server with 512MB of RAM and 20GB of storage, and the best part is that you can easily upgrade the server if you need more power. The $5/month is more than I paid before, but it’s still quite reasonable for the added power I get, and if I need to I can move to a more powerful server for simple increments in cost.

Migrating a web site to a new server is never easy, and not to be taken lightly. There are always unexpected problems, and my site had some fairly unique problems. One of the problems I ran into was that the database that holds all the information for the site was so large that it was difficult to even export to a format I could move to the new server. At first the exported database was over 300MB. I looked into the database and saw a lot of the records had to do with plug-ins I used. I had to disable those plug-ins, and remove their data from the database manually, which brought the size down to a still difficult 100MB. When I was finally able to export the database, I found it impossible to import to the new server. The web interface to the database, which was the easiest way to import the data, would run out of memory before completing the import. I tried uploading the file to the server directly and importing the data via the command line, but still had trouble.

Eventually I found a great little piece of code that helped me import the data by breaking it down into smaller chunks and importing it piece by piece (BigDump, which I recommend highly for those who need to import large databases). However, even with this new tool I ended up with an error message I didn’t understand. After asking for help online, I found out that the error was due to the original web site running on an older version of the database software (MySQL) and that the newer version didn’t allow a date field to be empty (set to 0 essentially). When you had a data field in the database that you didn’t have a value for, it was supposed to be set to Jan 1 1970 instead. Go figure, but I had 53 times in the database where I had a zeroed out date, and doing a find and replace in the database fixed the problem and the database finally imported.

Other problems were more mundane. As I tried different parts of the site, I noticed certain things didn’t work properly.

The contact form didn’t work, which it turns out was because the new server didn’t handle mail the same way as the old server. After re-configuring the mail, the contact form began to work.

The maps on the Compendium city pages were not being displayed because for some reason Google thought this was a different site. After setting up new credentials for the the Google Maps API, the maps began to work again as well.

The more insidious problems had to do with file and folder permissions. Having moved much of the files over from my old server, the file permissions on many things were wrong. Now I find file permissions in UNIX to be a form of the dark arts, but slowly I’ve been fixing the problems. I noticed, for example, that I couldn’t upload new images, which is because the server couldn’t create a new folder to upload images to for this month. Sure enough, it was a permissions problem. Upgrading plug-ins has also been difficult due to permission issues. This is probably an issue that will continue to cause issues for some time until I’ve worked out all the bugs.

For the last few days the site has been running completely on the new server. I imagine there will still be problems going forward, and I ask that if you run into anything you think is weird on the site, please please contact me and tell me what you saw. As with the above examples, the problems can be hard to predict, so if you see something odd, such as an error message, or even a missing image, don’t assume I already know about the problem. Please contact me and let me know, so I can fix it for everyone.

Transitioning from Blogger to WordPress

transition

[Update 5 Feb 2014: The Blogger Importer plug-in has undergone a major overhaul, and has fixed a lot of problems and made lots of improvements. One of the major improvements is the importation of images, so at the very least Step 7 below can probably be skipped. Some other tweaks might also be unnecessary. It claims to handle internal links better, for example. In any case, check each step after the import below and make sure they’re necessary.]

I’ve been using WordPress for a couple of years now on other sites I’ve developed, such as Genealogy.org.il (the web site of the Israel Genealogy Research Association), my other blog Lexigenealogy, Kanczuga.org (for descendants of the destroyed Jewish community of Kanczuga, Poland), and some others I’ve worked on, but I’ve always been hesitant to move this blog over to WordPress since I knew it would be a lot of work, and I liked that Blogger (Google’s blogging platform that I started with) was a very easy and inexpensive way to keep the blog running.

However, as time went on I felt stronger and stronger that the site needed a redesign, and I never felt the Blogger platform allowed me enough flexibility to do what I wanted. Thus, this redesign on WordPress. I welcome your comments on how the new site looks, and if you see anything that seems wrong or out of place, please let me know. As you’ll see below, it’s not so easy to transfer a blog from Blogger to WordPress, and it’s very possible some things didn’t make the transfer 100% accurately.

The rest of this post is to explain the steps I went through to transition the site, of interest to other bloggers who might be considering the same. If technical details don’t interest you, the rest of this post will be very very boring to you. For those who want to take a look at the original site, it is still live at bloodandfrogs.blogspot.com (but will not be updated in the future).

Step 1) Install WordPress

I won’t go into details here as there are plenty of places online to help you setting up WordPress, but I’ll give a quick overview. I have a web host that lets me quickly add domains to sub-directories of my main account, so I took a different domain, assigned it to a sub-directory, and installed WordPress in that directory. I set up a MySQL database on my web host, and configured this new WordPress installation to connect to that database. Using a new domain for the setup with very helpful as it allows me to have a live site working even before I consider switching the original domain over to the site. Before I can do that, I need to insure that all of the links on my original Blogger site will work on the new WordPress site.

2) Set up Permalinks

Permalinks refer to the format of the web link to your articles. It’s the permanent link to the articles you post on your site, so someone can share or bookmark a specific article (instead of just the site itself). The default for WordPress is to just show the article id number, which looks something like: bloodandfrogs.com/?p=123 Not very pretty. However, WordPress lets you set the format of the link to include the year, month, day, article title, etc. in whatever combination you’d like. Thus with pretty permalinks the link to an article might be:

bloodandfrogs.com/2013/09/transitioning-from-blogger-to-wordpress

That format shows the year, month and the name of the article. However, there’s a problem, in that Blogger has a different permalink structure, which looks like:

bloodandfrogs.com/2013/09/transitioning-from-blogger-to-wordpress.html

In order to insure that all the links from the old Blogger site work on the new WordPress site, we need the permalinks to match. Luckily, WordPress lets you customize the display of permalinks, by specifying the pattern like so:

/%year%/%monthnum%/%postname%.html

This custom pattern creates the text after the domain name in the exact same format as Blogger. Well, not exact exact. The problem is that how Blogger and WordPress determine how to format the name of the article is a bit different. Basically, Blogger cuts the number of words in a link, while WordPress lets you have as many words as you want (up to the maximum allowed in a URL). In order to insure the article title is formatted the same, you need to run a script to correct the imported articles. First we need to import the articles, however, so back to that in a minute.

3) Importing Posts and Comments from Blogger

WordPress offers an Import screen in the Tools menu of the Dashboard. In Import, there are options for many different blogging platforms like Blogger, LiveJournal, Tumblr, etc. If you click on Blogger, it will initially take you to a page to install a plug-in called Blogger Importer. Once installed, the same link on the Import page will instead take you to the settings page for the plug-in. Blogger Importer does some, but not all of the job. According to the plug-in description: Items imported

  • Categories
  • Posts (published, scheduled and draft)
  • Comments (not spam)

Items not imported

  • Pages
  • Images (the images will appear in your new blog but will link to the old blogspot or picassa web locations)

That doesn’t tell the whole story, however. When I ran the importer, it imported all my posts, and most of my comments. Why were 20 or so comments left out? I’m not sure. Not a huge deal, but I don’t even know which comments were left out. The good news is that even though it doesn’t import Pages or Images, there are workarounds for them.

4) Fixing Permalinks

Earlier, we set up Permalinks to be like Blogger, but as I mentioned, it’s not exactly like Blogger. Now that we’ve imported the posts, we can go back and fix the links that need to be fixed. For this, I used a PHP script from rtCamp from their guide on moving from Blogger to WordPress. They provide code that you copy into a text file with a .php extension and upload to your server. Once it’s in your WordPress directory, you load the file from the web and it processes all the link names and fixes them. If you have internal links to those pages, I’m not sure if it would fix those. Probably not.

In the end, there will be lots of little details like this that need to be checked and manually fixed. Once you’ve run the script, you could and should delete it from your server. In theory, all of your posts are now exactly what they were on Blogger, and assuming you haven’t changed your domain then you shouldn’t need any fancy redirection tools to keep external links to your site, since they should be identical to the old site (at least for Posts).

5) Fixing Tags

Blogger doesn’t have Categories for Posts, but it does have Tags. Oddly, when importing the Tags from Blogger they become Categories in WordPress. It’s odd because WordPress also supports Tags, and one would think Tags would be imported as Tags. In order to fix this, I found a plug-in called simply Categories to Tags Converter, which does exactly what it says. The plug-in hasn’t been updated in over two years (usually a bad sign) but it worked great and everything seems to have been converted properly.

6) Importing Pages

Unlike Posts, there is not a simple way to import your Pages. Hopefully you don’t have as many pages as Posts. Although in my case I had hundreds of Posts, I only had five Pages. Here’s what you need to do to get them on your site. Create corresponding Pages on your WordPress site for each of your Blogger pages. Copy the text from each Blogger Page, and copy it into the corresponding WordPress Page. Depending on your Pages, this might be good enough.

If you have complicated code in your pages, you may need to go back and make changes. In most cases with my pages everything worked fine. I did have some internal links to other Pages and Posts that I needed to modify, and in one case I had some custom CSS I had created that I needed to track down in Blogger and copy over to WordPress.

It’s good to view every page and test all the links, compare the appearance between the old and new sites, etc. This isn’t a perfect process, and mistakes will pop up.

Once your Pages look and perform the same, you need to add redirections. These are codes that tell the browser (or search engine) that the location of the resource they are looking for has changed. It’s important to use what’s called a 301 redirection, which tells the search engines like Google that this is a permanent change, and allows you to retain any search rankings you received on the previous page on the new one. The reason this is necessary is that Blogger places all of its pages in a directory /p/, and WordPress doesn’t let you do this. Thus, you need to have a different URL for Pages, and need to direct incoming traffic from the old /p/ location to the new one without the sub-directory. Therefore, a page on Blogger might be called:

www.bloodandfrogs.com/p/search.html

but on WordPress it would be:

bloodandfrogs.com/search

and you need to tell the browser to search engine how to figure out that change.

For this problem, I use a plug-in called simply Redirection. Redirection lets you set up a series of redirections, where you simply give it the original URL and the new URL. I set up redirections for the five Pages, and I’m done. Keep in mind that since the URL is changing anyways (removing the /p/ at the minimum) you don’t need to keep anything else in the name the same. The redirection will work even if the Pages have completely different names. Indeed if you wanted to delete a Page altogether, you could create a redirection to send people trying to reach that page to someplace else on your site.

7) Importing Images

Earlier, when we imported the posts from Blogger, the images were not imported. Since the images are stored on a Google server somewhere with a link that does not include your blog’s name, you could just leave those images alone and let them load from Google’s servers. Of course, if you ever close the account you used to set up your Blogger blog, then all those images will disappear. I also would feel a bit like I was taking advantage by using their image hosting for Blogger but not actually using Blogger. Technically there’s nothing wrong with it, as it’s all part of your Google account, but it still seems off to me. In any case, I’d prefer to have my images local to my blog.

There are a few plug-ins that allow you to import images from Blogger. I used one called SEO Blogger to WordPress Migration using 301 Redirection which in addition to doing redirection also has a module for importing images. After it imports all the images, it goes through all of your posts and changes the links from the original links to their new locations on your server. Pretty neat.

There’s another plug-in called Blogger Image Import that does the same thing, although I haven’t used it. With any plug-in like this, however, make sure to backup your blog before using it, as it is making a large number of changes and if it messes up, you’ll be in trouble. There’s no undo with these. I did this step before I imported my Pages, so I’m not clear if the links would be updated on Pages, or just in Posts. In any case, the images used on Pages will be imported, I’m just not sure if the links to them will similarly be updated. Since I don’t use a lot of images in my Pages, this wasn’t a big deal for me. I just went in and manually modified a couple of links to the images.

8) Adding Google Back

Since Blogger is a Google platform, it has built-in support for a bunch of Google technologies, like Google Analytics, Google+ Comments and Feedburner. These are not built into WordPress, so if you want them back you need to add some code to WordPress. There are plug-ins that can help.

For Google Analytics, I used Joost de Valk’s Google Analytics for WordPress. You install the plug-in, log into Google and give permission for the plug-in to access your Analytics information. Then you just select your site from a drop-down menu, save the settings, and you’re done.

For Google+ Comments, I’m not settled on exactly what I want to use. It’s only in the last few months that I switched to Google+ Comments on the blog, and with mixed results. I think the Google+ comments are better than the original Blogger comments, but there’s no way to really track them. I don’t always know when new comments are posted, even though I think I’m supposed to be notified. If I change the domain name in the future, all the Google+ comments will be lost as well, which is bizarre to me. This is the problem with using an external commenting system.

For the moment, I’m looking at different options for comments. Some plug-ins such as Social Comments promise to integrate WordPress, Google+, Facebook and Disqcus comments all together. What commenting system do you use on your blog? Right now, unfortunately, all the comments added through Google+ in the past few months are now missing from the site.

As I already use Google Feedburner for my RSS feeds, all that needs to be done to keep the feed going is to log in to Feedburner and change the URL of the original feed from the Blogger location (/feeds/posts/default) to the WordPress location (/feed). Since people should have been directed to the Feedburner address (http://feeds.feedburner.com/bloodandfrogs) instead of the Blogger address, no one who subscribes to this blog should see any difference (although how the blog posts look will likely be a bit different).

9) Widgets

Blogger has a lot of pre-built Widgets to place in various places in the layout of the blog. Mostly, I used them in the sidebar on the right side. One thing I like in Blogger is the appearance of their Archive widget. It shows all the posts of the most recent month, and allows easy navigation to find other posts from previous months and years. WordPress doesn’t support this directly, but I found an old plug-in that does something very similar, called Collapsible Archive Widget. It hasn’t been updated in over two years, but so far it seems to be working pretty well.

By default it doesn’t show the post names, because that would require loading all post names from the whole history of the blog into the page. Presumably Blogger itself doesn’t do this, but loads the data as needed, but this plug-in doesn’t seem to do that, and warns against using this feature if you have a lot of posts. I’ve turned it on for now, and will see how it goes.

Some widgets I had were just HTML code, so I just copied them over. The widgets for things like Google+, Google Friend Connect, etc., however, were part of Blogger and I will need to find replacements for them. I haven’t reproduced everything in my right-column on the old site, but I hope to add more to in in the future as I get things up to speed.

For Twitter, I used a widget that comes with Jetpack. There’s also a Facebook widget, but I’m using Facebook’s own widget, so I didn’t bother to use Jetpack’s.

10) Transfer the domain

This section is for those who registered a custom domain through Blogger when they still allowed that option. I set up my blog with a custom domain back in 2010 (i.e. my blog is on bloodandfrogs.com, not bloodandfrogs.blogspot.com). Google will no longer register a domain for you, but they did then, and I’ve renewed through them each year since. So how does one transfer a Blogger custom domain to another registrar?

You can’t imagine how complicated it is…or rather you can’t imagine how complicated it is to figure out how to do this…

There is nothing in Blogger that tells you anything about the fact that your domain was registered via Google, or how to access it. Apparently, what happened is that a Google Apps account was set up for your domain, with an e-mail address they never tell you about, and you need to figure all this out. Here’s the deal:

1) You have an account on Google Apps with the email address bloggeradmin@yourdomain.com (obviously with your domain instead of yourdomain.com)

2) Using the following URL, you can reset the password for your Google Apps Account: http://google.com/a/cpanel/yourdomain.com/ResetAdminPassword

3) Once you’ve reset the password (they send an e-mail, you click on the link, fill out the form, etc.), you log in to Google Apps, and go to the Domains section, and then click on Advanced DNS Settings. That gives you the login credentials to connect to the registrar which actually controls your domain (either enom or godaddy probably). Once you connect to the registrar, you can change the name servers to the name servers of your self-hosted WordPress site (or transfer the domain to another registrar).

Transferring the domain is actually very complex. I won’t go into all the details here, just keep in mind that once you can access the registrar (in my case it was GoDaddy) you need to both unlock the domain AND remove privacy protection. When I first tried to transfer the domain I unlocked it (which allows it to be transferred) but when I tried to transfer it, it was rejected. Privacy shouldn’t affect the transfer process, but here it does. The domain was connected to a third-party proxy service which needed to be disabled. Get this, the username for this service is also something not given to you. By recovering the username, I found out it’s one digit different from the username for the registrar. The password was the same. How are you supposed to know this? I have no idea. Thus I’m sharing what I discovered here.

At this point, make sure to set the nameservers for your domain to match your web host’s name servers. You could have specified this when you initiated the transfer, but if not, make sure to change the nameservers now. If you think you can keep the Google nameservers when your transfer your domain (i.e. to point to your Blogger blog until you’re ready to switch to WordPress), let me clear that up – you can’t. Google will disable the DNS for the domain as soon as your transfer it away. Basically, when you transfer the domain, have the new site ready.

Lastly, don’t forget to decouple the yourdomain.blogspot.com domain from the custom domain. This is easy enough to do in the Blogger settings – just go to Settings – Basic, and under Publishing, it should show both your custom and standard blog address. Just click the X next to the custom domain, and it will be removed. Once you do this, there will be no DNS support for your domain from Google, so when you do it make sure your new site is fully set up and ready to go…

11) Switching the domain in WordPress

In WordPress settings, you can define your WordPress Address and your Site Address. If your site is in the your primary directory for the domain, then these will be the same. If you want to keep WordPress in a subdirectory, but still have it show up as the main site, then these will be different. I originally set these up with my dummy domain for testing.

Once the domain was being transferred, I went into WordPress Setting -> General and switched these to be https://bloodandfrogs.com. I had already configured the domain on my web host to point to the same directory for both my dummy domain and the real one, so there was nothing needed there. When the domain transferred, everything worked properly.

12) Other services – Twitter and Facebook

In my original blog, I used a service called Twitterfeed which looked at the RSS feed of my blog and tweeted new posts automatically (to my twitter feed twitter.com/bloodandfrogs). WordPress allows more control over the Twitter process through many different plugins. Although I could have left the Twitterfeed service running (since my feed didn’t change) I decided instead to switch to using a plugin called WP to Twitter, which automatically tweets new posts among other functions, and offers a much larger degree of control.

I also used an RSS-based solution for posting to my Facebook page (fb.com/jewishgenealogy) called RSS Graffiti, which you might have noticed when this site moved, as it must have detected a different RSS feed and re-posted the last 10 posts or so to Facebook. That’s the nature of an RSS solution. It’s not a fine instrument, but a blunt force tool.

For Facebook I had wanted to use the official Facebook plugin for WordPress, which I use on other sites, but for some reason it wouldn’t let me connect to my Facebook account. Instead I’ve used Jetpack, a plugin that connects your self-hosted WordPress blog to WordPress.com and adds many features, which has a social publishing feature called Publicize that supports Facebook, Twitter, LinkedIn, Tumblr and Path. I may switch to using Jetpack for Twitter as well, but for the moment I’ve left that to WP to Twitter.

Conclusion

That was a lot of work, and there’s still a lot left to do. There are lots of little things to fix, and eventually I need to actually take advantage of WordPress and add additional functionality to the site. My initial goal was just that  everything from the old site worked properly on the new site, and that all the links worked.

I also haven’t categorized any of the posts, something I probably should go back and do since Blogger doesn’t have categories, and everything on the site is now ‘Uncategorized’.

Lastly, one odd thing is that images in my posts that were centered in Blogger, and not centered in WordPress. I need to go back and center images. I may not do that with every post, perhaps just the most recent and the most popular.

Did you find this guide to transitioning from Blogger to WordPress useful? Let me know in the comments.