J-Walk Blog
Many bloggers like to keep track of other blogs that link to them. It may seem a bit narcissistic, but there are some good reasons for doing so. The vast majority of bloggers don't get paid for their efforts. Therefore, knowledge that another blogger liked what you wrote (or linked to) is one of the few tangible blogger rewards. Those who run marketing blogs, of course, always want to know what's being said about them. In addition, I've discovered quite a few good blogs because I traced their blog to an incoming link (bloggers who link to my blog often have a lot in common with me). And finally, it's always useful to know if some other blogger is ragging on you (it could happen).
Bloggers have a long tradition of being concerned with links. Many blogs feature an extensive "blog roll" of links on their main page. The primary function, I think, is the hope that a member of your blog roll will return the favor and include your blog on his or her blog roll. Personally, I think blog rolls are a bit old-fashioned and only serve to add clutter, but they are still widely used.
Another reason bloggers are concerned with links is because that's pretty much how the Web works. The number of people who actually find your blog is determined in large part by the number and quality of the incoming links. Google and other search sites use incoming links to determine the placement in their search results.
On 11 August, 2005, I started an experiment. I posted a blog entry titled Attention Bloggers! (referred to in this document as the "experimental URL"). I asked fellow bloggers to create a link to that blog entry. I would then attempt to identify all of the incoming links.
Three days later, I posted a list of 114 incoming links that I'd found. See Link Experiment Results – Part I. I asked those who provided a link that I didn't find, to identify themselves. I then listed these unfound links at Link Experiment Results – Part II. This listed consisted of 10 blogs, plus eight more who identified themselves in the Comment section.
So, at least 132 bloggers linked to that experimental blog entry, and I identified 114 of them – for a "find" rate of 86%. It's extremely likely, however, that other unfound links exist, and the blog owners simply haven't checked back to tell me. Therefore, my actual success rate is probably closer to 80%.
* * *
The remainder of this article describes the tools and services I used to help find those incoming links. I used all of the tools listed here except Feedster, MSN Search, and Talk Digger. I forgot about Feedster, I didn't even consider MSN Search, and I just learned about Talk Digger today. As it turns out, those are three of the best tools available for this type of task.
You may be familiar with some or all of these sites. But if you're like me, it's not uncommon to discover a great tool and then quickly forget about it. There are probably some good tools that I haven't discovered, so I make no claims that this list is comprehensive.
Keep in mind that this discussion focuses on identifying incoming links -- which is not the most common use for these search sites. Most people use blog search sites to find relevant articles using keywords, not URLs.
Note: When I began this project, my original intention was to keep track of how I found each of the incoming links. That proved to be an incredibly complicate manual task, that I quickly abandoned.
In the past, the most common way to identify incoming links was to examine your site's log files. These massive files are generated by your Web server, and you need some time of log file analysis software to make any sense of them. Among the data summarized is a list of referring sites.
If your blog has its own domain name, you probably have access to your log files. If you use a blogging services such as Blogger or TypePad, you won't have access to this data. In any case, referrer logs are getting to the point where they are almost useless – thanks to spammers. Chances are, you'll find dozens of referring sites that simply faked their visits. If you click on these referring URLs, you'll end up at porn or casino sites. These spammers don't do it because you'll click the links. They do it because some not-so-bright Webmasters actually list their referrers on their site or blog. Search engines see these bogus referrers as outgoing links, and the porn sites and casino sites may get an additional boost in the search engines.
Most blogging software tools enable trackback. This is a list of entries in other blogs that refer to the particular blog item. It's similar in concept to a comment – except that the comment is actually someone else's blog entry.
I've never used this feature, so I have no idea how reliably it works. I've read that trackbacks are often used by spammers. No surprise there.
I've had Blogdex in my bookmark list for years. You can enter a URL, and it lists blogs that link to the document. For my main blog URL, it found only 25 incoming links (many of them from my own site). For my experimental URL, it found only five.
I'm not sure what's up with Blogdex. The front page reports a problem due to a hard drive failure. At first I thought that may account for the lack of results. Then I noticed that the message is from October, 2004.
I think it's safe to say that Blogdex is no longer a contender.
Blogdigger is a search engine that reads RSS feeds. Among its features is the ability to search for incoming links. I entered my blog URL and it found only 58 linking sites. It appears that blog owners must add their blog RSS feed URL in order to be included in searches – something that significantly reduces the value of this tool.
Bloglines is primarily an online RSS aggregator, but it also has a Citations search tool that can help you identify incoming links. I tried a search for the URL of my experimental item, and it found 73 incoming links. A search for the main page of my site yielded 6,154 entries – many of them linked from my own site (there is no way that I know of to exclude your own blog from the search results). To limit the search, the Advanced Search feature has a "yesterday only" option, but it doesn't work.
Overall, the Bloglines Citations feature is pretty good, but it certainly has the potential to be a lot more useful than it currently is.
BlogPulse is a service of Intelliseek. It's Search feature lets you enter keywords to find, or you can enter a URL and get incoming links. There's also an option to get the results in the form of an RSS feed.
I tried a URL search, and BlogPulse found 1,608 messages that link to the main page of my blog. When I entered the experimental URL, it found 58.
BlogPulse is a lot more than a search site. Check out their main page for more interesting stuff. I particularly like their Trend Results, which displays a chart that shows the frequency of keywords over time. They also have a list of the most linked-to blogs today. As far as I know, BlogPulse is the only service that does this.
BlogsNow is a newcomer, and not fully working at this point. It monitors 8.6 million blogs, and displays a list of the most frequently linked stories. I think BlogsNow will eventually have a public search feature. The site owner, Andreas Wacker, created a search for my experimental blog entry, and it found 71 links to that item. Not bad at all.
I'm ashamed to admit that I had forgotten all about Feedster when I did my link search. Had I used it, I think I would have found a few more.
The Link Search feature allows you to enter a URL and get a list of linking sites. I entered the URL for my experimental entry, and it found 91 links. When I entered the main URL for my site, it found 2,152. I've got to remember to use this more often.
IceRocket is a Web search site that enables you to limit your searches to blogs. Even the advanced search isn't very flexible, though. For example, it's impossible to enter a URL and search for incoming links only (your own blog is always included in the results). You can, however, limit your search to a specific date range.
IceRocket also has a Link Tracker application that you can install that displays the number of incoming links for every item you post – below the entry, just like a permalink (it uses Javascript and PHP scripts). I haven't tried this, but it sounds promising. IceRocket also has a graphical Trend Search tool that looks identical to the one found at Technorati.
IceRocket certainly has some potential, but if you're looking for a way to identify incoming links, it's not of much use at this time.
Everybody knows about Google, but can it help you find incoming links to your blog? Yes it can – but it's certainly not as useful as the blog-specific tools. If you go to Google's Advanced Search page, you can enter a URL and search for pages that link to the URL. Problem is, it doesn't work very well. I entered the URL for my experiment item, and it found no links to it. When I did a search for keywords (j-walk blog link experiment, no quotes) it found 19,700 entries. Entries on the first dozen or so pages were mostly relevant.
Google's search for pages that link to a URL works better when you enter the domain name alone. When I entered my blog URL, it found 9,360 incoming links – many of them from my own site (there's no way to exclude those). If I preceded my domain with "www." it found only 25.
Frankly, I'm very surprised that Google doesn't offer any blog-specific search tools. I'm sure they're working on it.
Like Google, MSN Search allows you to search for links to a specific URL. Unlike Google, however, using MSN Search to find links to my experimental URL actually returned some results – 136 to be exact. A search for links to my general site yielded 19,712, about twice as many as Google.
Color me surprised! I didn't even think to use MSN Search while I was looking for my links. As it turns out, it's may be the best tool of all. I admit that I've always dismissed MSN Search because… well it was lousy. But the new and improved MSN Search may finally provide some competition for Google. I tried searching for links to several other of my specific blog entries, and I was very impressed. I found a few links that I hadn't seen before.
Free tip for MSN: Include some type of blog-specific search (like IceRocket), and bloggers will come flocking.
PubSub can show you a 30-day link count history for your blog. It works only at the site level (it won't find links to a particular blog entry). I have my doubts about its accuracy, though. In addition to inbound links, it also displays outbound links and number of entries -- two things that are simple for me to count directly. Yet PubSub doesn't even come close to being accurate for these items (at least for my blog).
Talk Digger displays the incoming link results from nine search engines. You can enter your main blog URL or the URL for a specific entry. I just found out about this today (it could have saved me a lot of time!). The number of links to my experimental URL corresponds to what I found manually, except for IceRocket (it shows zero) and MSN (it shows 753). This site is still in beta, and it's worth keeping an eye on.
Technorati claims to be "the authority on what's going on in the world of weblogs." This service is currently tracking 15 million sites, and 1.4 billion links. They offer a good Website URL Search that seems to be fairly comprehensive. Some sites (such as Boing Boing) include the Technorati search URL for every blog item. It's a slow search that often yields nothing. Maybe they should try using MSN Search?
For my blog, Technorati finds 1,814 links from 1,077 sites. This information is also used for ranking purposes, and it gives J-Walk Blog a ranking of 318 among all blogs. When I entered the experimental URL, Technorati came up with 92 incoming links -- but it would only display four of them! When I clicked "Show all" it stripped the URL down to the domain name (maybe it's the question mark in my permalink URLs). I did a keyword search (j-walk AND link AND experiment) it did much better, and found 91 posts which I could view.
One nice thing about Technorati is that you can create a Watch List for a particular search (requires a free account). However, it appears that you can't create a Watch List for a particular permalink URL (at least I couldn't).
Although it has quite a few quirks, Technorati is a great tool and every blogger should spend some time to explorer it.
The table below summarizes the number of links returned for two searches:
|
Tool |
Links to a Specific Entry |
Links to General Site |
| MSN Search | 136 | 19,712 |
| Technorati | 92 | 1,814 |
| Feedster | 91 | 2,152 |
| Bloglines | 73 | 6,156 |
| BlogPulse | 58 | 1,608 |
| IceRocket | 54 | 1,122 |
| Blogdigger | 24 | 58 |
| Blogdex | 5 | 25 |
| 0 | 9,360 |
In terms of sheer number of links returned, MSN Search is the surprising winner. For the experimental URL, I don't think it found all of them, but it did better than any other single search site. Based on this little experiment, my opinion of MSN Search went up a notch or two.
But the bottom line remains: No single site is able to identify every incoming link – at least not yet. All of the sites listed in this table are worth exploring. Check 'em out, and see which ones you like. Most of them offer other tools that are useful to bloggers.
I'm still puzzled about why I was not able to find some of the incoming links. Maybe the sites are too new? Maybe they're too small? Maybe they've just been overlooked. One thing that certainly doesn't hurt is to take advantage of pinging if your blog software supports it. Pinging aggregator sites will help get your blog noticed. You might also consider using Ping-o-Matic, which does mass pinging with a single click.
Note: I have no idea to what extent these results can be generalized. It's entirely possible that if this experiment were repeated, the search sites would have an entirely different ranking.