The World Wide Web was designed to primarily do three things - inform, discover and connect. A globally connected series of documents could instantly bring you to the thoughts and experiences of someone across the world. In the earliest designs of the Web, it was through hyperlinks that you would find those new voices. Links brought you new sources of data, and those downstream documents led you even further to new people and ideas.

As the Web evolved, and incorporated photos, videos, streaming, and all manner of media, discovery expanded to include search. Without an explicit link, you could still find pointers to new content in the results of your query. Destination sites, acting as content hubs, would surface new content, usually within their network, of recommendations you might like. Ads, essentially links with pretty pictures, would offer another exit.

WebCrawler: One of the Web's first search engines

When blogging was the main medium of first person information sharing, prior to the rise and later domination of real time social streams, the way we discovered new voices was through links to others. I'd mention those I agreed with and highlight, with more links, those I didn't. One popular feature in practically everyone's sidebar was a blogroll, to show those with closest ties or just who we liked to read. And there were custom search engines, like Technorati, which when combined with tools like Google Alerts, could let you know when somebody mentioned you on the Web.

Technorati: The original blog and link search engine

But over time, a number of things happened to chop away at this fluffy cloud of friendly discovery.

1. Many Blogs Gravitated Toward Internal Linking, Not External Linking

The big sites realized that keeping visitors on their own site was more profitable and aided their metrics more than sending them away did. And while there is obvious irony in my posting to my own discussions on this from the past, we actually had lengthy discussions about these internal linking practices in 2007 (Part two and part three) 

Arguments a decade ago in favor of internal linking were that site visitors were familiar with companies and topics discussed, and could see previous coverage by their publication to learn more if they weren't. And any link off site started feeding the ad revenues of a potential competitor.

2. Dedicated Blog Search Sites Didn't Graduate to Quality Businesses

Technorati was a specialized blog search engine that skipped the general Web and went directly to blogs for its content. Its leaderboard of bloggers was closely watched, as were trending topics on the site that led to see what the blogosphere was discussing. But it was seemingly always in financial trouble, and has pivoted beyond recognition to whatever it is now.  

A 2010 interview I had with the company's leadership team claimed a pivot to quality, but their CEO was gone a year later, and so are pretty much all the discovery tools that initially aided me to find some of the best voices of the Web 2.0 era. And yes, Google Blog Search quietly disappeared not too long afterward.

3. Blog Discussions Pivoted to Real Time Streams and Sharing

As I noted in 2009 (yet another internal link, am I right?), linking between communities was declining in favor of retweets on Twitter or sharing into the stream. The microburst of a little site traffic would provide that one time dopamine hit, but not leave a trail for later web spiders to find.

As the social streams of Twitter and Facebook took over, and bloggers (me too) got distracted, the share became the canonical mention. Your mentions on Twitter, or your notification of shares on Facebook, were faster delivered and easily quantifiable. And individual profile owners are quite unlike the publishers looking to deep dive into their analytics to discern where traffic came from.

Today, Brent Simmons laments the result of all the mentions going to the streams and leaving the Web. With links decreasing, and blog search being a relic, he yearns for a way to find again how his business is mentioned on the web or when he is being linked to. Joe Gregorio, like me, wistfully remembers analysis of referral logs to find how people found you... all through links.

Simmons' proposal is a limited one, keeping it to the MacOS/iOS community. A small project like that shouldn't be too challenging, but it speaks to a bigger problem, where the connections we once demanded are an afterthought behind the latest viral tweet and trending Facebook share. Streams are ephemeral, but the Web was built to last. It'd be great to see new voices building, informing and connecting again.

Disclosures: I work at Google and am on the Google Analytics product. So I think about publisher tools and visitor stats more than most.

Over the last two days, the popular location based service and king of check-ins, Foursquare, has been suffering from intermittent outages related to database overload and storage troubles. And this is wonderful news. Not because I wish them ill will, but it seems this is a necessary strain practically all successful companies go through at some point as they achieve rapid user growth. How users react to what are hopefully temporary bumps, and how the service reacts after the fact can help determine if the company will continue to be embraced, like a Twitter, or dumped, like a Technorati.

Foursquare's rise from its SXSW 2009 debut to near pervasiveness among the digerati, becoming synonymous with the location check-in movement, despite big players like Facebook and Google participating, and more direct competition like Gowalla and Mytown, is a testament to the service's viral nature, including game mechanics, close ties to friending and following, and a casual attitude that eliminated the dark feeling of insecurity many people once had about announcing their locations, and replacing it with something one not just accepts, but expects.


Foursquare seems to be in the position to choose their own destiny - to forge ahead as an independent company, flush with cash from a summer VC round, or to court willing suitors at a later date, hopefully at a higher valuation than deals they famously turned down in a public negotiation for control. While two days of shakiness are no doubt unwelcome by the company or its addicted userbase, it seems a rite of passage that the biggest Web services go through in a circuitous route to the top.

Twitter is by far the most famous for continued bouts with downtime, closely tied to scalability issues, and its fail whale mascot is the icon of downtime that all other interruption pages wish to be when they grow up. The company's status blog at http://status.twitter.com/ has actually seen 15 different posts since September 1st, contrasted with only 10 posts from the company's official blog. (http://blog.twitter.com/)

But the blue bird and white whale on a teal background are not the first to document downtime doom. Amazon Web Services promotes a service health dashboard, while Salesforce.com displays a system status at Trust.Salesforce.com. Google provides a status dashboard for Google Apps, with dedicated pages for Blogger status and Feedburner status and others. Even pokey old Web 1.0 dinosaur Hotmail has a status dashboard.

Common to all these services? Previous downtime issues that hit the community hard. Truth is, until there is a major downtime incident, most companies do not offer insight into their status. They may feel like they are teflon, and they can't be taken down, or they simply didn't think about what could happen if something went wrong. It takes a public shakeup that calls the service's stability and viability into question to push for improved system integrity and redundancy, and despite best efforts, some never quite solve the problem.

Back in 2007 and 2008, Technorati was as much known for its Technorat Monster escaping as anything else. This cutesy downtime message foretold the demise of the service, which had to strip out much of its functionality simply to stay up, and was pretty much abandoned by all but a core of users who watched the company evolve its business model and reduce their technical demands. And so far as I know, Technorati never launched a status page. They didn't do what the many other successful companies (above) did when push came to crash. Foursquare, it looks like, is working hard to push this recent event and will come out of this stronger because they are following the proven path - of ownership, late hours, and public transparency.

Tonight may suck for the company's employees, who are taking a public flogging that cans of Red Bull and pizza won't fix. But they should wear the fatigue as a badge of pride. In time, they could look back at this week's events with warm and fuzzy feelings, remembering the outage, which resulted from "too many checkins", as a clear tell they were graduating from Web curiosity to Web titan.

In the last few years, much of the Web's focus has trended toward social networks and microblogging, and away from more traditional blogging. Twitter and Facebook are on the tips of most people's tongues, even when those sites often serve as much to link to news and blogs elsewhere, in addition to the much-lauded status updates. Regator, the blog search engine which debuted in mid 2008, and introduced a solid iPhone app one year later, is poised to revamp its service while keeping blog content front and center. The result, in preview now, is a solid data mine for content, trends and topics. In theory, the site could outright replace Web 1.0 engines like Technorati, and it looks to be significantly more useful - no matter your focus of interest.

Regator still presents itself as surfacing the world's best blog posts from the world's best blogs. Not all make it in the highly selective network, but there are many quality sources that do in practically every subject, from Academics and Arts to Politics, Religion and Tech.

Regator's new preview steps up its blog coverage by showing trending topics, as Technorati never got quite right and Twitter and Google News have for their own properties. From global trends (such as Times Square and the Gulf of Mexico) to topical trends (such as Steve Jobs and the iPad 3G), Regator quickly updates you at a glance on what the blog world is talking about.

As the site does now, you can still see popular and new stories by topic, as well as those with audio or video. But you can take a step further with two new sections, including trends and monitoring.


With Regator's deep archive of blog topics, you can compare up to three keywords, and see how they are spiking over time. Not unsurprisingly, Gizmodo spiked well beyond Engadget around their iPhone 4 coverage, and Apple dwarfs almost anybody else during Steve Jobs keynotes. You can see first day launches of products like Google Buzz, and the ensuing fade of coverage, or you can see consistent updates on more broad terms, such as "laptop", which only shows dips every weekend.


Should one-time visits to the site not be enough, you can tap into Regator's blog monitor to follow keywords and send you updates on these keywords - much like Google News alerts and TweetBeep.


Regator may have started out as a way for the community to help find the best of the Web through up arrows and down arrows, but its upcoming revamp, which you can gain access through via invites, looks solid enough to be the highest quality blog search available, with deep trendspotting. I will keep using Icerocket search for finding all mentions from all blogs, but Regator has trimmed the fat and returned with the best.

Just go to http://preview.regator.com with the invite code: "louisgraysentme" and check it out for yourself.

When Technorati CEO Richard Jalichandra joined the company near the end of 2007, he found the blog search engine in "a lot of trouble". Technorati was bleeding cash, with an expensive infrastructure based on "incredibly complicated technologies", and was losing the war on fighting spammers, who were threatening to render the company's deep index and authority measurements completely useless. In parallel, the struggles came at a time when expanded competition from formidable players, including Google, made staying on top almost impossible, even as the company's formidable task to index the fast-growing live Web expanded.

"Please Put On Your Oxygen Mask First..."

The situation looked dire, one Jalichandra compared to when the oxygen masks fall from a depressurized airplane, saying you can't save the passenger next to you, until you save yourself.

"The first thing I had to make sure of was that we didn't go out of business," Jalichandra said in an interview Monday at the company's San Francisco headquarters. "In the fall of 2007, it all came to a head, and we knew we had to do something or there wouldn't be anything left."

Since 2003, Technorati was among the first leaders of the Web 2.0 wave, recognizing the imminent growth of the live Web, encompassing blogs and other social media activities. Not just comfortable with blog search, Technorati also enabled site owners to claim blogs, track backlinks to their content, measure authority, ranked by unique site backlinks over time, to ping the site with new updates, chart changes in content distribution on the Web, and present Technorati widgets on their blogs. Add on a few fun features, including popular pages, and the always quizzical "Where's the Fire", best known as "WTF?", and Technorati had its focuses widely diversified.

Near Death By a Thousand Cuts...

The 2007 crisis forced the company to revisit everything and make some very painful decisions, cutting costs, headcount and features - many of which had been treasured by the blogosphere.

"(Previous CEO David) Sifry came up withe great idea of boiling the ocean of the live Web and it grew like crazy," Jalichandra said. "But there are a couple of problems with boiling the ocean when people are looking for a cup of coffee."

Technorati, once known as the go-to site for data on the blogosphere, had soon found itself eclipsed by more nimble competitors, such as Google Blog Search, and while many bloggers, including myself, grew frustrated with the site's rapidly killing features we liked, and the not too unfamiliar appearance of site downtime, which would happen multiple times a week, for several hours at a time, the site came incredibly close to going away altogether without drastic change.

"The first six months (of my term as CEO) was all about stabilizing the company," Jalichandra said. "At some point you have to keep the lights on, particularly the kind of company that is venture funded. It has to get revenue, and has to make a profit. We had to write the plan for growth, a two-pronged plan, to grow the site and launch the ad network. We are still dedicated and committed to growing Technorati.com."

One Must Have Revenue to Invest Back In the Company

The company's focus on the advertising network seems like a dramatic change in course for what was instead seen as a technology service company, but it was a move aimed to leverage the existing database in the site, while also driving revenue into the company. It was suggested that, as revenue grew, it could be reinvested back into Technorati's site, quite possibly to start replacing some of the lost or deprecated services much beloved by the blogosphere.

In October of 2009, Technorati revamped its Web site, and while gone were many of the dedicated pages for topics and individual blog posts, which I had liked, in its place was a detailed directory with top blogs ranked by category, with a new ranking of authority, and a higher focus on relevancy. While the old Technorati would highlight the most recent updates as they flowed in, the new Technorati tries to deliver recent updates from the most authoritative sources - a move that's not without its controversy, of course.

"Search is not a democracy," said Jalichandra. "There is always an editorial choice in search. It may be an algorithmic choice, but it's a choice."

Look Mom, No Spam and No Downtime!

The new site relaunch led not just to new content and a new look, but the elimination of downtime, as the company says they have not experienced any appearances of the Technorati Monster having escaped since the introduction, adding it is believed the the issue of spam is practically non-existant. Despite so many headaches and a popular perception that the site is irrelevant or archaic, the company's leadership thinks the two years needed to bring the site back to the point where innovation can occur has been both necessary and worth it.

David White, vice president of publishing, says that working for Technorati has been a "really interesting challenge", but one that has changed as the mainstream is not just aware of blogs, but is embracing the live Web in their daily lives, as Technorati plays less of a role as evangelist for the blogosphere, but instead one that focuses on finding quality content.

"Technorati started by indexing all the blogs," White said. "Our job is different now, and that is to find the quality. Technorati is an important site, and it has made an impact on how we interact with the blogosphere, which has had a huge impact on media as a whole."

Will It Be Enough? Can I Have My Widgets Back Now?

As the move to quality infiltrates the company's blog directory, Technorati promises many of the tools that have gone away are coming back. The company fixed blog claiming last week, and promised Monday to bring back authority over time charts, as well as introducing new widgets, which had long since obsoleted.

Many bloggers, including the visible futurist Stowe Boyd, have already given up on Technorati, after seeing the company's early promise seem to crash and burn, through a confusing product strategy, spam-filled results, poor uptime, and bad news followed by no news. It seems the darkest times are behind them, and the company will have to make good on its promises before many bloggers learn to trust them again. But in my meeting with them Monday, they clearly said they're not done fighting.

To hear Jalichandra in his own words, listen to the 8 minute clip I recorded on Cinch:

With all the recent news around Bing and Google making deals with Twitter to gain a foothold in the world of real-time search, not much has been said about the often frustrating world of blog search, and not a peep has been made about Icerocket, a company I'd myself largely neglected until a few months ago, having mentally relegated it to the Technorati-like category of Web 2.0 dustbins. But very quietly, Icerocket has built out what I believe to be the best blog search out there, and its Twitter search is quite potent as well. I have been using it exclusively to find links to my blog, and its relevancy blows Google Blog Search out of the water, barring any upgrades from Mountain View.

Aside from an October post from Doc Searls, Icerocket's improvements have practically been invisible. But it's a disservice to the site, which has easy to find RSS feeds for any search, total number of positive hits in the database by URL, and improvements on the standard Twitter search that include the "smack your forehead and say 'duh'" additions of retweeting and replying to any successful search result, which Twitter's search engine lacks today.

In years past, most bloggers have turned to Technorati or Google Blog Search to find relevant posts. Google Blog Search does a great job finding keywords, and until the most recent revamp, Technorati did the best job in finding which sites were linking your way. (In March, I even said Technorati was trumping Google for blog reactions) The October update to Technorati (covered here) added some interesting twists, but the "reactions" function that I liked has pretty much become impossible to use, or disappeared.

Icerocket, which CrunchBase reports gained its first funding round way back in 2004, from Mark Cuban, has not gotten some of the buzz that other real-time search companies, including Collecta, Topsy and OneRiot, have, but its results are top-notch. Like any good blogger, I've bookmarked a search on the site to find links my way, and more often than not, Google's Blog Search doesn't find them for me in the database.

Compare for yourself: Google Blog Search vs. Icerocket Blog Search


Recent Links to the Blog, Via Icerocket

Icerocket does more than just find the most answers. It highlights the ranking of the domain being searched in its database (mine has hovered just below 2,000), and shows the rank of linking blogs - a simple way to assume "Authority", including the total number of references these downstream blogs have. Also, helpfully, the results are sorted by date! Not bad. Google Blog Search won't do that. Icerocket also lets you display total results for a specific post, by URL.


Some Notable Blogs' Rankings in Icerocket


More than 20 Blogs Linked to My Recent FTC Badges Post

Similarly, Icerocket thumps Twitter Search for the same results.

Compare for yourself: Twitter Search to that of Icerocket Twitter Search


Some Twitter Results from Icerocket's Search

Icerocket's Twitter search results are the same as those presented by the microblogging giant, but in addition to the standard responses, you see the number of people following those mentioning a keyword, you see more results on one page, and have the smart additions of "Reply" and "RT" at the end of each result, having you one step away from making an action on the search. In contrast, to do so on Twitter Search, you would need to click the timestamp of the specific update, and then, from that window, hit reply or retweet.

You can even tell Icerocket to show all tweets, just those containing links, or those without links. Also helpful? A cool tool that shows common followers on Twitter of selected users, up to four, saying who follows all of those specified, and who all four follow. (Here's an example with @dewitt, @louisgray, @arrington and @jenna.)


Want to See Who Follows Who? Try Icerocket.

Just about the only thing Icerocket is not doing when it comes to Twitter search results is having the newest flow in from the top to present the illusion of real time. Instead, to get the newest answers, you still have to refresh. It's a small price to pay for a single site that offers great blog search and Twitter search results in one. I've switched. I just wonder if Icerocket, considered an older property, can have new users recognize its new life.

There are a select few Web 2.0 companies who have suffered such a roller coaster of peaks and valleys the way Technorati has. Once a clear industry leader for blog search, statistics, and individual site "authority", Technorati's influence withered away thanks to an aggressive push by Google into the blog search arena, statistical gaming by number-crazed bloggers, management changes, odd product launches, and inconsistent uptime. But with a major relaunch tonight, the company has tried to throw off the shackles of the old and rise again, armed with more cash in the bank, a talented editorial staff, and a new look. All of a sudden, the site looks relevant again.

Back in early 2007, Technorati was among one of the favorite topics on this blog. You could see the tumult at the company, as then-CEO David Sifry wrote on a Tuesday in a comment here that he was "very very happy at Technorati", only to announce he was looking for a new CEO that Friday, three days later. You could see debate that summer over people trying to game the then much-watched "Authority", which counted up external links to your site in a six month period. But by early 2008, we were using phrases to discuss the company that included "totally toast", and the new Twitter generation, less than two years removed from Technorati's heyday, scarcely remembers the once respected innovator.


Rising and Falling Blogs On Technorati
But as of tonight, they are back in the game. They ditched the old metrics for attributing authority, as it was considered too static, and now will aim to reward authors for posting frequency, context and linking behavior. Interestingly, they have also introduced authority by topics, meaning that technology blogs can be compared to others in their sector, as can sports blogs, music blogs, and so on. This means that aside from the overall Top 100, sites like TechCrunch don't have to measured head to head against Huffington Post, and we smaller blogs can get a better idea of who our peers are. (See:
A Totally New Technorati.com & Technorati Media Rising)


It's Nice to Be Considered a Top 100 Tech Blog
This new ranking system is looking to be more dynamic - changing along with the real-time nature of the Web. Blogs will rise and fall, and be noted on the site. Blogs and individual posts will be featured, and "hot blogosphere items" of all topics can make the front page.


An Individual Blog's Technorati Profile
Occasional louisgray.com contributor and friend to the site, Eric Berlin, is the blogging channel editor, so we wish him well and look forward to hearing more about that role. Additionally, JS-Kit's Echo will enable comments to be placed underneath all blog listings and tags, possibly adding conversation to the data.

Technorati may not be the big giant we once thought they would be, and they will need to have some consistent successes to become a blogosphere darling again, but they are back in the conversation and worth watching.

See Also:
VentureBeat: Big changes coming at Technorati — the CEO’s perspective
TechCrunch: The New Technorati

Operating a blog focused on early adopters means being willing to adapt as technology and the world around us evolves. One of the more-recent additions to the blog was my embedding of buttons from TweetMeme, to show how often my posts were being linked to on Twitter, and making it easier for others to retweet these items, even thought I've already gone on record as not being a big fan of retweeting myself.

I made the change not because of a change in my own practices, but due to recognition that many people are turning to Twitter to share links and find new links from peers. While most of my posts only get a few dozen tweets, some have numbered over a hundred. And as this occurs, in parallel, the total number of links back to the Web site from other blogs is decreasing.

Not too long ago, one of the most common items to see featured on a blog was one's Technorati Authority, showing the number of unique blogs that linked back to you in the previous six months. Looking back at some of my "State of the Blog" entries I used to post monthly, I can see that approximately one year ago, that number stood at nearly 1,000. (See: here) Now, that same measure is only slightly above 500, a decrease of nearly half, despite my posting schedule being fairly regular, and the site's visibility rising over the same period.


Just Over a Year Ago, Technorati Had More Traffic Than Twitter...
Discussion of Technorati Authority used to be a big topic on this blog as recently as 2007. (See: Why My Technorati Ranking Is Slip-Sliding Away, Technorati Needs to Stamp Out Viral Tag Spam Now, Is Technorati Going After Spam Blogs?, and My Technorati Link Stats Make No Sense) But now, not only is Technorati largely overlooked, but so is linking, for the most part. It's easier to send a link via Twitter, or to share a post on Google Reader or FriendFeed.

This change is why in July of 2008 I said The Importance Of Blog Linking Seems to Be Declining, and why Steve Rubel is now saying his stats reflect the way the Web is changing. Not only are people turning to social sites instead of blogs to share links, but often, many blogs do a lot more linking to themselves than they do in linking to other sources, whether set by editorial policies, inertia, or just simplicity. Not surprisingly, we talked about that back in 2007 as well. (Internal Linking On Some Tech Blogs Is Out of Control)

TweetMeme, which started out as a secondary project by Nick Halstead and his team at Favorit, now looks like a real winner, having gained significant visibility after rebooting last July after Twitter API issues had killed the service following initial launch. The company just announced new ways to display statistics by domain, and it will now enable the ability to retweet items no matter where they are found, including on RSS feeds. Once that gets posted, you can be sure I will find a way to get that button in my RSS feed, and if done well, I will post the chicklet of domain activity alongside my FeedBurner number in the blog's sidebar.

I may still prefer to write long blog posts and link out, but I can't force everyone else to swim upstream. Retweet away.

To laud Technorati is going against the flow, to say the very least. The once-omnipresent blog search engine has practically been reduced to a state of irrelevancy, thanks to inconsistent uptime, odd product launches and withdrawals, nonsensical redesigns, executive turnover and aggressive competition from others - primarily Google Blog Search. In previous posts on Technorati, I've referred to them as "totally toast" and "fighting off irrelevance". But surprisingly, especially in recent months, the moribund site has consistently beaten Google in terms of finding new and accurate links to my blog or mentions of the site, while Google's results have actually gotten less relevant over time, including false positives from blog rolls and the like. No doubt this had much to do with why Rob Diana, in January, said for the most part, that blog search sucks.

Google is set up to find all of the world's information, and it is doing a fantastic job at that, as we all know, and it is the gold standard for search in practically every regard. But it's maybe too good. The company's over-aggressive spiders are just as likely to trigger false positives in terms of knowing what is a blog and what is not, or what is a blog post or what is simply sidebar information. Last August, I highlighted one issue, when MyBlogLog activity was spawning false positives. On other occasions, I've seen updates from aggregation sites, like Socialmedian, do the same. At this point, my bookmarked blog search from Google to find reactions excludes no fewer than four sites, to try and filter down the accurate results.

And as I'm fighting off false positives with Google, Technorati is quietly finding me mentions that I can't get using Google, which relies on keywords instead of links. Not even the advanced blog search page on Google lets me find links to a site the way most bloggers want to find.

Technorati, for instance, found me links from LivingstonBuzz.com, BlackWeb20.com, and from Regular Geek in the last few days, which were pointing my way, but didn't mention my name or site domain - and I'm finding this to be the rule, rather than the exception. While at one point I'd stopped visiting Technorati, I've now returned to the site on a frequent basis to see responses, and participate in the conversation wherever it may be.

Technorati's benefits over Google Blog Search may no doubt be short lived. Maybe Google Blog Search will solve some of their issues soon, and develop new features, while Technorati has been relatively stagnant. And I'm still waiting for somebody to come up with the "inverse Technorati" idea I floated back in October of 2007. I'm not saying Technorati is perfect, or winning me over as a major force to be reckoned with in innovation, but if I want to know who is linking my way and extending the conversation, they're still doing a good job, and beating Google, which is a significant feat.

By Rob Diana of Regular Geek (Twitter/FriendFeed)

In the past few weeks, the 'blog search engines are bad' complaint has resurfaced. This seems to reappear every few months, and for good reason. Blog search is not really changing. I complained a few months ago that blog search was broken and a month later reiterated my feelings when Technorati went down and nobody cared.

This time, Steve Rubel complains that blog search is in a pitiful state. His lament seems to be that the blog search engines are not returning real links to his blog posts. I could be wrong in my summary, but this is the real point of trackbacks, which most blog platforms still support, I thought. I am not saying that I disagree with Steve, I do believe that blog search is in terrible shape. Mark Evans questions whether blog search is just too hard a problem:

Maybe the blogosphere is simply too difficult to track given it changes so quickly and there’s so much to spider. Or maybe Google believes there are bigger opportunities elsewhere.

Mark even asked for a better blog search (and a few other things) from Google for Christmas. Part of the problem that people are mentioning is that Technorati and Google Blog Search are returning links from blog rolls and not just links from within posts. Google Blog Search is actually asking for help if you see this problem.

However, I think the conversation regarding "finding links" is missing something. What do we really want in blog searching? Are we only searching for links to our posts? Are we searching for blogs that are talking about a specific topic? Are we just searching for new blogs to read? Are we really just trying to find out where our blog ranks, like Technorati's authority?

Part of the problem is that we are focusing on one issue with blog search. Basic link searching will probably always suck because of the problem with spam blogs. They will add links to some sites and detract from others. Spam is just a hard problem to fight, so we probably will have to live with some of those problems. The other questions are much more interesting, but I am not sure that people really want those types of features. Do we want to use something like Technorati to find new blogs? Or are we now using FriendFeed and Google Reader shared items to find new things?

My thinking is that beyond Technorati Authority, people have left blog search behind. Because of this, blog search is not going to be seeing innovation by itself. If you look at Compete and compare traffic of Technorati, Google Blog Search, IceRocket and BlogPulse, you will see that the combined traffic has decreased over the past year. My estimates put the decline in traffic from Technorati and Google Blog Search at 30%. Obviously, there is little traffic or revenue to be had. As you can see, there is no real reason for anyone to try to compete with Technorati. It would be a competition for a dying space.

If you are trying to find who is talking about your post, blog links are only a small part anymore. Social media has changed where the conversation is occurring. The conversation is still happening on other blogs and in the blog comments, but there are other sites included as well. Social news sites like Digg, Mixx and Reddit have comments from various people. Other social media sites like Twitter and FriendFeed also promote comments on their sites. It is time to stop complaining about blog search. It is time to start looking at the bigger picture, find the conversations and join them.

Read more by Rob Diana at RegularGeek.com.

By Jesse Stay of Stay N' Alive (Twitter/FriendFeed)

This weekend's blog flareup on whether Twitter should track the "authority" of a user, based primarily on the number of followers, has a number of people up in arms. One side says it makes sense. After all, Technorati and Google have always tracked influence. Others say the following number can be easily manipulated, and has no weight. First of all, before we address the issues, why am I writing this on LouisGray.com and not my own blog, StayNAlive.com?  It largely comes down to numbers.  LouisGray.com has near 4,000 RSS subscribers, while my blog only has 500.  Aside from the fact that I enjoy the team of great writers I work with on this blog, I have a much louder, and because of that, more authoritative, voice here.  More people listen with a larger audience than those with a small audience.  And like it or not, all bloggers trying to compete play the numbers game - that's simple marketing.
Read more by Jesse Stay at Stay N' Alive.

The launch of a leaderboard for the once-feared Twitterank was inevitable. After all, in the online world, if you can measure something and give it a score, then by all means the next step is to rank people from high to low, and provide a leaderboard. It's happened with blog "influence" (Technorati). It's happened with mentions on Techmeme. It's even happened with how frequently people's items are shared on Google Reader (Feedheads, RSSmeme and ReadBurner). As ranking one's Twitter influence has been tried several times by a bunch of different sites, from Twinfluence to Twitter Grader, Twitterank was practically destined to join the crowd. On Friday, the site launched a "Top 50" list and after watching the dust settle a bit, I have to be extremely amused by the results.


Every ranking system has its flaws. And considering Twitterank's algorithm is both secret and changing, according to its author, Ryo Chijiiwa, initial hiccups are no surprise. But glancing at the top 50 tells me that Twitterank must measure influence in a very odd way, contrary to just about every other measure I've seen out there.

For example, according to Twitterank, the #1, highest scoring person in all the world is Scott Beale of Laughing Squid. (@laughingsquid) Scott's account garners a score of 237.591. His own Twitter account shows he (as of Monday after midnight) is following 1,636 people, has 19,307 followers, and has made 5,285 updates. This does not rank him among the top 50 on Twinfluence in total reach, but he does reach #20 on Twitter Grader.

In the #2 position on Twitterank is Brian Solis (@briansolis), who weighs in with a score of 235.847, and Twitter activity of 582 following, 8,033 followers and 3,524 updates. This activity garners him the #43 position on Twinfluence and #22 overall on Twitter Grader.

While Twitterater's top list does have a lot of "household names" like Dave Winer, Michael Arrington, Jeremiah Owyang and Steve Rubel, there are some big oddities, including at least one account that has never sent a message on Twitter at all.


Let's be honest, there's no way I should be this high.
For example, Loic Lemeur (not pictured, but at 226.91) actually ranks below me in the rankings, despite his following and being followed by almost five times as many people, and sending ten times the amount of updates. Meanwhile, Leo Laporte gets a 179.87 ranking, well off the top 50 list, despite having more than 60,000 followers, behind only president-elect Barack Obama and Kevin Rose of Digg (that I know of). And the ever-present Robert Scoble gets only a 188.63, also keeping him off the Top 50.


Leo Laporte, with 60,000 followers, misses the leaderboard?


And Scoble, Mr. Twitter, doesn't break 200 either?
So how does that make any sense? I was going to guess that Scott Beale ranked highly thanks to his high followers to following ratio, but Leo Laporte's ratio is an astonishing 120 to 1, so that, in theory would rank higher. And Scoble's real numbers are off the charts in almost every metric.

Another canary in the coal mine - the account of @google, which ranks #13 overall, according to Twitterank's Top 50, but has only 366 followers, isn't following anyone and has never updated their Twitter account.

So... @google, a user with no updates, has a higher Twitterank than does Scoble, who tops out at 39,000 followers, and more than 15,000 updates. Whatever you think about the content of Robert's tweets, whether they be too frequent or too off-topic, to say that an unused account is among the top in the world is as they say in the Web world... a big FAIL.

That Twitterank has an algorithm which measures something is clear as it gets some of the names you'd expect, but there are still a lot of questions around this service. Right now, it's basically a toy, and has little value.

In late August, BackType launched to track comments across thousands of blogs, and seemingly nearly as many disparate platforms. The service effectively turned blogging on its head, providing a place where comments were king, and the posts themselves, secondary. With BackType, you can see a single individual's comments from around the Web, and follow multiple people, helping to discover new sites. (My ID can be found here) Today, BackType debuted a new trends application, at http://trends.backtype.com/, based on their API, which shows trends within those comments, including how frequently keywords were mentioned, who said them the most often, and on which blogs those topics came up.


BackType Trends Lets You Search Millions of Comments
The effort to harness large groups of comments and make sense of the noise is not new. Fav.or.it pulled in more than 13,000 comments per day as recently as August, forming the basis for its report on the blogosphere's commenting statistics in aggregate. (See: Fav.or.it Comments On Commenting Statistics) But despite that one-time blip, most comprehensive reports on the state of the Web (yes, Technorati, we mean you) walk right past comments and act as they don't exist. Now that BackType has been pulling in comments for a few months, they're ready to position themselves as the authority on comments, and will be writing about these statistics in upcoming pieces, said Christopher Golda, founder of BackType.


The Presidential Election Has Been a Big Topic of Late
The new trends site, found at http://trends.backtype.com/ lets you graph mentions of keywords or phrases from the millions of comments BackType has pulled from around the Web. The tool can be used to gauge a topic's popularity, based on the total raw number of results, or to see spikes in activity, like with "Joe the Plumber", or the word "Macbook", which, as you can guess, spiked upward around Apple's recent announcement, and then quickly fell.


MacBook Mentions Spiked With Apple's New Introductions
The Trends tool shows results over the last two weeks, graphing the total counts by date, and displaying the top five blogs where the keyword is mentioned, as well as the top five authors of that keyword.

The combination lets you know, for example, that Daniel Pritchett, Robert Scoble and Mark Dykeman are the three most frequent users of the word FriendFeed, and that you're most likely to see FriendFeed discussed on Scobleizer, TechCrunch or Startup News (Y! Combinator).


Who Mentions FriendFeed Most Frequently, and Where? BackType Knows.
Similarly, using the same query, you can find that Twitter is most frequently discussed on ProBlogger, TechCrunch and Mashable.

So what has garnered commenters' interests in the last two weeks? Unsurprisingly, a lot more than just tech, but also, politics and finance. "Obama" clocks in with 80,851 separate comments. McCain tries to match with 69,414 mentions and his running mate, Sarah "Palin" counters with 33,593. On the financial side, you see 7,765 mentions of "crisis", 2,344 mentions of "recession", and 2,681 for "depression".

You can even combine searches using boolean expressions. Searching for "depression OR recession" garners 4,666 responses, but "depression AND recession" only returns 359. Good thing, because searching for these negative keywords has me heading to the medicine cabinet...

What I'd like to be able to do in the future with BackType Trends is compare one term versus another over time, or extend the search over a greater period, or on specific blogs, for example, seeing if Twitter mentions are going up or down at ReadWriteWeb or Mashable. It's also not the most robust, beautiful presentation I've ever seen, but the data is more important than its shell. We're seeing the foundation of what could be a very interesting repository of data, one I'll be tapping into time and again.

Follow me on BackType here: http://www.backtype.com/louisgray.