Yearly Archives: 2013

Transitioning from Blogger to WordPress

transition

[Update 5 Feb 2014: The Blogger Importer plug-in has undergone a major overhaul, and has fixed a lot of problems and made lots of improvements. One of the major improvements is the importation of images, so at the very least Step 7 below can probably be skipped. Some other tweaks might also be unnecessary. It claims to handle internal links better, for example. In any case, check each step after the import below and make sure they're necessary.]

I’ve been using WordPress for a couple of years now on other sites I’ve developed, such as Genealogy.org.il (the web site of the Israel Genealogy Research Association), my other blog Lexigenealogy, Kanczuga.org (for descendants of the destroyed Jewish community of Kanczuga, Poland), and some others I’ve worked on, but I’ve always been hesitant to move this blog over to WordPress since I knew it would be a lot of work, and I liked that Blogger (Google’s blogging platform that I started with) was a very easy and inexpensive way to keep the blog running.

However, as time went on I felt stronger and stronger that the site needed a redesign, and I never felt the Blogger platform allowed me enough flexibility to do what I wanted. Thus, this redesign on WordPress. I welcome your comments on how the new site looks, and if you see anything that seems wrong or out of place, please let me know. As you’ll see below, it’s not so easy to transfer a blog from Blogger to WordPress, and it’s very possible some things didn’t make the transfer 100% accurately.

The rest of this post is to explain the steps I went through to transition the site, of interest to other bloggers who might be considering the same. If technical details don’t interest you, the rest of this post will be very very boring to you. For those who want to take a look at the original site, it is still live at bloodandfrogs.blogspot.com (but will not be updated in the future).

Step 1) Install WordPress

I won’t go into details here as there are plenty of places online to help you setting up WordPress, but I’ll give a quick overview. I have a web host that lets me quickly add domains to sub-directories of my main account, so I took a different domain, assigned it to a sub-directory, and installed WordPress in that directory. I set up a MySQL database on my web host, and configured this new WordPress installation to connect to that database. Using a new domain for the setup with very helpful as it allows me to have a live site working even before I consider switching the original domain over to the site. Before I can do that, I need to insure that all of the links on my original Blogger site will work on the new WordPress site.

2) Set up Permalinks

Permalinks refer to the format of the web link to your articles. It’s the permanent link to the articles you post on your site, so someone can share or bookmark a specific article (instead of just the site itself). The default for WordPress is to just show the article id number, which looks something like: bloodandfrogs.com/?p=123 Not very pretty. However, WordPress lets you set the format of the link to include the year, month, day, article title, etc. in whatever combination you’d like. Thus with pretty permalinks the link to an article might be:

bloodandfrogs.com/2013/09/transitioning-from-blogger-to-wordpress

That format shows the year, month and the name of the article. However, there’s a problem, in that Blogger has a different permalink structure, which looks like:

bloodandfrogs.com/2013/09/transitioning-from-blogger-to-wordpress.html

In order to insure that all the links from the old Blogger site work on the new WordPress site, we need the permalinks to match. Luckily, WordPress lets you customize the display of permalinks, by specifying the pattern like so:

/%year%/%monthnum%/%postname%.html

This custom pattern creates the text after the domain name in the exact same format as Blogger. Well, not exact exact. The problem is that how Blogger and WordPress determine how to format the name of the article is a bit different. Basically, Blogger cuts the number of words in a link, while WordPress lets you have as many words as you want (up to the maximum allowed in a URL). In order to insure the article title is formatted the same, you need to run a script to correct the imported articles. First we need to import the articles, however, so back to that in a minute.

3) Importing Posts and Comments from Blogger

WordPress offers an Import screen in the Tools menu of the Dashboard. In Import, there are options for many different blogging platforms like Blogger, LiveJournal, Tumblr, etc. If you click on Blogger, it will initially take you to a page to install a plug-in called Blogger Importer. Once installed, the same link on the Import page will instead take you to the settings page for the plug-in. Blogger Importer does some, but not all of the job. According to the plug-in description: Items imported

  • Categories
  • Posts (published, scheduled and draft)
  • Comments (not spam)

Items not imported

  • Pages
  • Images (the images will appear in your new blog but will link to the old blogspot or picassa web locations)

That doesn’t tell the whole story, however. When I ran the importer, it imported all my posts, and most of my comments. Why were 20 or so comments left out? I’m not sure. Not a huge deal, but I don’t even know which comments were left out. The good news is that even though it doesn’t import Pages or Images, there are workarounds for them.

4) Fixing Permalinks

Earlier, we set up Permalinks to be like Blogger, but as I mentioned, it’s not exactly like Blogger. Now that we’ve imported the posts, we can go back and fix the links that need to be fixed. For this, I used a PHP script from rtCamp from their guide on moving from Blogger to WordPress. They provide code that you copy into a text file with a .php extension and upload to your server. Once it’s in your WordPress directory, you load the file from the web and it processes all the link names and fixes them. If you have internal links to those pages, I’m not sure if it would fix those. Probably not.

In the end, there will be lots of little details like this that need to be checked and manually fixed. Once you’ve run the script, you could and should delete it from your server. In theory, all of your posts are now exactly what they were on Blogger, and assuming you haven’t changed your domain then you shouldn’t need any fancy redirection tools to keep external links to your site, since they should be identical to the old site (at least for Posts).

5) Fixing Tags

Blogger doesn’t have Categories for Posts, but it does have Tags. Oddly, when importing the Tags from Blogger they become Categories in WordPress. It’s odd because WordPress also supports Tags, and one would think Tags would be imported as Tags. In order to fix this, I found a plug-in called simply Categories to Tags Converter, which does exactly what it says. The plug-in hasn’t been updated in over two years (usually a bad sign) but it worked great and everything seems to have been converted properly.

6) Importing Pages

Unlike Posts, there is not a simple way to import your Pages. Hopefully you don’t have as many pages as Posts. Although in my case I had hundreds of Posts, I only had five Pages. Here’s what you need to do to get them on your site. Create corresponding Pages on your WordPress site for each of your Blogger pages. Copy the text from each Blogger Page, and copy it into the corresponding WordPress Page. Depending on your Pages, this might be good enough.

If you have complicated code in your pages, you may need to go back and make changes. In most cases with my pages everything worked fine. I did have some internal links to other Pages and Posts that I needed to modify, and in one case I had some custom CSS I had created that I needed to track down in Blogger and copy over to WordPress.

It’s good to view every page and test all the links, compare the appearance between the old and new sites, etc. This isn’t a perfect process, and mistakes will pop up.

Once your Pages look and perform the same, you need to add redirections. These are codes that tell the browser (or search engine) that the location of the resource they are looking for has changed. It’s important to use what’s called a 301 redirection, which tells the search engines like Google that this is a permanent change, and allows you to retain any search rankings you received on the previous page on the new one. The reason this is necessary is that Blogger places all of its pages in a directory /p/, and WordPress doesn’t let you do this. Thus, you need to have a different URL for Pages, and need to direct incoming traffic from the old /p/ location to the new one without the sub-directory. Therefore, a page on Blogger might be called:

www.bloodandfrogs.com/p/search.html

but on WordPress it would be:

bloodandfrogs.com/search

and you need to tell the browser to search engine how to figure out that change.

For this problem, I use a plug-in called simply Redirection. Redirection lets you set up a series of redirections, where you simply give it the original URL and the new URL. I set up redirections for the five Pages, and I’m done. Keep in mind that since the URL is changing anyways (removing the /p/ at the minimum) you don’t need to keep anything else in the name the same. The redirection will work even if the Pages have completely different names. Indeed if you wanted to delete a Page altogether, you could create a redirection to send people trying to reach that page to someplace else on your site.

7) Importing Images

Earlier, when we imported the posts from Blogger, the images were not imported. Since the images are stored on a Google server somewhere with a link that does not include your blog’s name, you could just leave those images alone and let them load from Google’s servers. Of course, if you ever close the account you used to set up your Blogger blog, then all those images will disappear. I also would feel a bit like I was taking advantage by using their image hosting for Blogger but not actually using Blogger. Technically there’s nothing wrong with it, as it’s all part of your Google account, but it still seems off to me. In any case, I’d prefer to have my images local to my blog.

There are a few plug-ins that allow you to import images from Blogger. I used one called SEO Blogger to WordPress Migration using 301 Redirection which in addition to doing redirection also has a module for importing images. After it imports all the images, it goes through all of your posts and changes the links from the original links to their new locations on your server. Pretty neat.

There’s another plug-in called Blogger Image Import that does the same thing, although I haven’t used it. With any plug-in like this, however, make sure to backup your blog before using it, as it is making a large number of changes and if it messes up, you’ll be in trouble. There’s no undo with these. I did this step before I imported my Pages, so I’m not clear if the links would be updated on Pages, or just in Posts. In any case, the images used on Pages will be imported, I’m just not sure if the links to them will similarly be updated. Since I don’t use a lot of images in my Pages, this wasn’t a big deal for me. I just went in and manually modified a couple of links to the images.

8) Adding Google Back

Since Blogger is a Google platform, it has built-in support for a bunch of Google technologies, like Google Analytics, Google+ Comments and Feedburner. These are not built into WordPress, so if you want them back you need to add some code to WordPress. There are plug-ins that can help.

For Google Analytics, I used Joost de Valk’s Google Analytics for WordPress. You install the plug-in, log into Google and give permission for the plug-in to access your Analytics information. Then you just select your site from a drop-down menu, save the settings, and you’re done.

For Google+ Comments, I’m not settled on exactly what I want to use. It’s only in the last few months that I switched to Google+ Comments on the blog, and with mixed results. I think the Google+ comments are better than the original Blogger comments, but there’s no way to really track them. I don’t always know when new comments are posted, even though I think I’m supposed to be notified. If I change the domain name in the future, all the Google+ comments will be lost as well, which is bizarre to me. This is the problem with using an external commenting system.

For the moment, I’m looking at different options for comments. Some plug-ins such as Social Comments promise to integrate WordPress, Google+, Facebook and Disqcus comments all together. What commenting system do you use on your blog? Right now, unfortunately, all the comments added through Google+ in the past few months are now missing from the site.

As I already use Google Feedburner for my RSS feeds, all that needs to be done to keep the feed going is to log in to Feedburner and change the URL of the original feed from the Blogger location (/feeds/posts/default) to the WordPress location (/feed). Since people should have been directed to the Feedburner address (http://feeds.feedburner.com/bloodandfrogs) instead of the Blogger address, no one who subscribes to this blog should see any difference (although how the blog posts look will likely be a bit different).

9) Widgets

Blogger has a lot of pre-built Widgets to place in various places in the layout of the blog. Mostly, I used them in the sidebar on the right side. One thing I like in Blogger is the appearance of their Archive widget. It shows all the posts of the most recent month, and allows easy navigation to find other posts from previous months and years. WordPress doesn’t support this directly, but I found an old plug-in that does something very similar, called Collapsible Archive Widget. It hasn’t been updated in over two years, but so far it seems to be working pretty well.

By default it doesn’t show the post names, because that would require loading all post names from the whole history of the blog into the page. Presumably Blogger itself doesn’t do this, but loads the data as needed, but this plug-in doesn’t seem to do that, and warns against using this feature if you have a lot of posts. I’ve turned it on for now, and will see how it goes.

Some widgets I had were just HTML code, so I just copied them over. The widgets for things like Google+, Google Friend Connect, etc., however, were part of Blogger and I will need to find replacements for them. I haven’t reproduced everything in my right-column on the old site, but I hope to add more to in in the future as I get things up to speed.

For Twitter, I used a widget that comes with Jetpack. There’s also a Facebook widget, but I’m using Facebook’s own widget, so I didn’t bother to use Jetpack’s.

10) Transfer the domain

This section is for those who registered a custom domain through Blogger when they still allowed that option. I set up my blog with a custom domain back in 2010 (i.e. my blog is on bloodandfrogs.com, not bloodandfrogs.blogspot.com). Google will no longer register a domain for you, but they did then, and I’ve renewed through them each year since. So how does one transfer a Blogger custom domain to another registrar?

You can’t imagine how complicated it is…or rather you can’t imagine how complicated it is to figure out how to do this…

There is nothing in Blogger that tells you anything about the fact that your domain was registered via Google, or how to access it. Apparently, what happened is that a Google Apps account was set up for your domain, with an e-mail address they never tell you about, and you need to figure all this out. Here’s the deal:

1) You have an account on Google Apps with the email address bloggeradmin@yourdomain.com (obviously with your domain instead of yourdomain.com)

2) Using the following URL, you can reset the password for your Google Apps Account: http://google.com/a/cpanel/yourdomain.com/ResetAdminPassword

3) Once you’ve reset the password (they send an e-mail, you click on the link, fill out the form, etc.), you log in to Google Apps, and go to the Domains section, and then click on Advanced DNS Settings. That gives you the login credentials to connect to the registrar which actually controls your domain (either enom or godaddy probably). Once you connect to the registrar, you can change the name servers to the name servers of your self-hosted WordPress site (or transfer the domain to another registrar).

Transferring the domain is actually very complex. I won’t go into all the details here, just keep in mind that once you can access the registrar (in my case it was GoDaddy) you need to both unlock the domain AND remove privacy protection. When I first tried to transfer the domain I unlocked it (which allows it to be transferred) but when I tried to transfer it, it was rejected. Privacy shouldn’t affect the transfer process, but here it does. The domain was connected to a third-party proxy service which needed to be disabled. Get this, the username for this service is also something not given to you. By recovering the username, I found out it’s one digit different from the username for the registrar. The password was the same. How are you supposed to know this? I have no idea. Thus I’m sharing what I discovered here.

At this point, make sure to set the nameservers for your domain to match your web host’s name servers. You could have specified this when you initiated the transfer, but if not, make sure to change the nameservers now. If you think you can keep the Google nameservers when your transfer your domain (i.e. to point to your Blogger blog until you’re ready to switch to WordPress), let me clear that up – you can’t. Google will disable the DNS for the domain as soon as your transfer it away. Basically, when you transfer the domain, have the new site ready.

Lastly, don’t forget to decouple the yourdomain.blogspot.com domain from the custom domain. This is easy enough to do in the Blogger settings – just go to Settings – Basic, and under Publishing, it should show both your custom and standard blog address. Just click the X next to the custom domain, and it will be removed. Once you do this, there will be no DNS support for your domain from Google, so when you do it make sure your new site is fully set up and ready to go…

11) Switching the domain in WordPress

In WordPress settings, you can define your WordPress Address and your Site Address. If your site is in the your primary directory for the domain, then these will be the same. If you want to keep WordPress in a subdirectory, but still have it show up as the main site, then these will be different. I originally set these up with my dummy domain for testing.

Once the domain was being transferred, I went into WordPress Setting -> General and switched these to be http://bloodandfrogs.com. I had already configured the domain on my web host to point to the same directory for both my dummy domain and the real one, so there was nothing needed there. When the domain transferred, everything worked properly.

12) Other services – Twitter and Facebook

In my original blog, I used a service called Twitterfeed which looked at the RSS feed of my blog and tweeted new posts automatically (to my twitter feed twitter.com/bloodandfrogs). WordPress allows more control over the Twitter process through many different plugins. Although I could have left the Twitterfeed service running (since my feed didn’t change) I decided instead to switch to using a plugin called WP to Twitter, which automatically tweets new posts among other functions, and offers a much larger degree of control.

I also used an RSS-based solution for posting to my Facebook page (fb.com/jewishgenealogy) called RSS Graffiti, which you might have noticed when this site moved, as it must have detected a different RSS feed and re-posted the last 10 posts or so to Facebook. That’s the nature of an RSS solution. It’s not a fine instrument, but a blunt force tool.

For Facebook I had wanted to use the official Facebook plugin for WordPress, which I use on other sites, but for some reason it wouldn’t let me connect to my Facebook account. Instead I’ve used Jetpack, a plugin that connects your self-hosted WordPress blog to WordPress.com and adds many features, which has a social publishing feature called Publicize that supports Facebook, Twitter, LinkedIn, Tumblr and Path. I may switch to using Jetpack for Twitter as well, but for the moment I’ve left that to WP to Twitter.

Conclusion

That was a lot of work, and there’s still a lot left to do. There are lots of little things to fix, and eventually I need to actually take advantage of WordPress and add additional functionality to the site. My initial goal was just that  everything from the old site worked properly on the new site, and that all the links worked.

I also haven’t categorized any of the posts, something I probably should go back and do since Blogger doesn’t have categories, and everything on the site is now ‘Uncategorized’.

Lastly, one odd thing is that images in my posts that were centered in Blogger, and not centered in WordPress. I need to go back and center images. I may not do that with every post, perhaps just the most recent and the most popular.

Did you find this guide to transitioning from Blogger to WordPress useful? Let me know in the comments.

The problem of borders…

One of the problems one runs into when researching their family, particularly those researching family from continental Europe and farther east, is how often the borders changed. The video below is a great visualization of that problem – it shows the borders of Europe over the past thousand years.

I don’t know who put together the original video, and exactly where the data came from, but it’s a very impressive video. I do know that the years being shown at the bottom were added later by someone else, so they probably don’t match up exactly to the map changes, but you can take them as an estimate.

One very interesting thing to watch in the video (it helps to focus on one area when you watch) is Poland. A little before half-way through when the years shown are in the late 1500s Poland is massive. It later disappears completely, later emerging again in a much smaller form.

So what do you think? Learn anything that can help your family research?

Scanning documents and photos using your phone

As the cameras built-in to cell phones get better and better (and indeed every year the improvement has been significant), more and more people are using their phones instead of dedicated scanners for duplicating documents and photographs.

There are many iPhone apps for scanning documents, such as CamScanner Pro, Doc Scan ProGenius Scan+, JetScanner, Prizmo, Scanner Pro, TurboScan, and the genealogy-focused Shoebox (now owned by Ancestry.com).

Android also has their share of apps, like CamScanner, Document Scanner, Droid Scan Pro PDFHandy Scanner ProMobile Doc Scanner, and there’s also an Android version of Shoebox.

Other popular apps like Evernote are also suitable for this kind of document and photo scanning.

Perspective correction in Genius Scan+

Most if not all of these apps do something pretty neat, which is that they will take your document or photo and if the corners are not 90 degrees, it will correct the perspective. This is necessary because most people holding a camera in their hand cannot possibly get the camera lens to be perfectly parallel to the document or photo they are photographing. This causes the photo they take to be skewed (where for example one side of the photo is smaller than the other side), and thus the need for this correction technology.

The problem with this technology is that you can never correct something like this without losing some quality. Wouldn’t it be better if you could insure your phone was parallel to the document when you take the photograph? When people traditionally took photographs of documents, they used big bulky copy stands. They are generally bulky and expensive. So what solution exists for your cell phone? Turns out this is a problem several people have taken a look at, and there are some very interesting solutions.

ScanDock

Let’s start out with free. Designer Kyle Koch has designed a cardboard stand he calls the iPhone Document Scanner, which he provides the plans for in EPS format. You can print out the designs and cut these out using cardboard or other materials yourself.

ScanDock

If you want to order it pre-made, it’s also available from Ponoko in either corrugated cardboard or MDF. A newer version, called the ScanDock (maybe soon to be ScanDeck as there is another ScanDock – see below), is available only in cardboard, and the plans are not available for free. Oddly, the different products all have different and bizarre shipping options. The best option seems to be the ScanDock which costs $25 and can be shipped to the contiguous US for $8.50. If you’re in New Zealand, shipping is only $3. I think this is really just a good do-it-yourself solution, I’m not sure I would pay for this over the options that follow.

Steady Stand

Another option is a series of stands from Scottish photo accessory company Modahaus, which they call Steady Stands. These stands are manufactured out of plastic, and are design to diffuse the light that hits the document or object inside the stand, which helps eliminate harsh shadows.

Steady Stand 200

The Steady Stand comes in a variety of sizes, and can be used to photograph physical objects, or used to create videos by dragging the whole stand over an object or document (such as a map) to create steady videos. I could see using the smaller one as a kind of Flip Pal replacement. The downside is that the FlipPal doesn’t need lighting, the plus side being that you don’t need to bring a full scanner with you, and the object you’re scanning doesn’t need to be flat. The smallest stand is £15 (yes, that’s British Pounds – currently about $23.37) and the largest is £31.20 (currently about $48.62)

ScanJig

The only solution I’ve seen that doesn’t shoot straight down onto a flat document, is the ScanJig. The ScanJig is also unique in that is designed to work with a larger variety of devices, from iPhones up to iPad Minis. There is a separate version for the Galaxy III and IV phones as well.

ScanJig

This is perhaps the most complicated stand in terms of setup. I’m not sure sure this is something I’d want to carry around with me. For some types of documents and workflows, this might be a good solution. I image if it’s on your desk and you’re copying lots of sheets of paper, this might be easier to swap out pages quickly. The ScanJig is currently $29.95 (although it says it is regularly $39.95) plus shipping (which was $4.95 to Boston).

StandScan

One of the most interesting solutions is called the StandScan. It has two features that I really like – it folds flat, and it has the option to add LED lighting. Of course, due to its design that has opaque walls on three sides, it’s possible the LED lighting would be required to get good quality in many lighting situations, so maybe it’s not an option after all, but a requirement.

StandScan

You can get the StandScan with or without the LED lighting, and there are two options for powering the lights, 9V (1 9V battery) or 12V (8 AA batteries). The stand without lighting is $19.95, the stand with lighting and a 9V battery pack is $29.95, and the scanner with lighting, both battery packs and an AC adapter is $37.95. Shipping seems to be $15, whether to the US or it seems anywhere in the world (and by that I mean I check how much it cost to ship to Israel).

Scandock

One other option is Scandock, which is built like a more traditional copy stand, with built-in lights on overheard arms, and a weighted base. It has its own camera app, which uses color patches on the stand itself to insure proper color calibration, and the stand has a silicon cover for documents to insure they lay flat.

Scandock

In short, the Scandock is much more advanced than the offerings above, but is also priced accordingly, which the cheapest version costing $279. Also, it’s not exactly a portable solution.

Lomography Smartphone Scanner
If everything so far seems more or less the same, you won’t get this one confused with the others. That’s because this product is not for scanning documents, but for scanning photo negatives and slides. It’s from analog photography company Lomography, and it’s called the Smartphone Scanner.
Smartphone Scanner

This device has a light source on the bottom, and you put in a transparency and scan it using your phone. It supports all iPhones and several Android devices. It has its own iPhone app, although you don’t need to use it (it is, however, helpful if the app you’re using can reverse the colors of image it captures).

The price is $59 in the US, and €59 in Europe, plus shipping. It’s also available for $59 (including shipping in the US) from photojojo, one of my favorite photo gadget companies.

Interestingly, two other phone stands were crowdfunded on Kickstarter, and succeeded in being shipped out (not as common as you might think with hardware products), but both seem to have disappeared – ScanBox and Scandy. The team behind ScanBox seems to have moved onto other products (LED light bulbs), while Scandy’s web site has disappeared completely.

I haven’t used any of these solutions unfortunately, but if one of the companies wants to send me a stand, I’m happy to review it on the blog.
Have you used any of these products? What software do you use for document scanning with your phone? Share your experiences in the comments.