Summary: This is a reprint of my Search Insider column from last Friday.

Chris Morrison at VentureBeat has been one of the privileged few to get a sneak preview of Powerset; he recently reported that the semantic start-up?s unofficial tagline is, ?We?re not a search engine.?

According to Morrison, this is standard for any company looking to dodge the hype of the ?Google-killer? moniker — fair enough; although, based on Powerset?s behavior to date, they don?t seem inclined to dodge hype of any variety.

There?s another reason for Powerset and its ilk to shun the search engine label, though: search isn?t broken.
Remember Gord?s Breaking the Google Habit series? Over five of his Search Insider columns, he discussed how people form habits and what it takes to change. We got a more scientific understanding of what we knew instinctively already: habits are darn hard to break, even if you want to break them.

Take overeating. Despite pills and patches and pop psychology, millions are locked in a seemingly unbreakable cycle — and that?s something that people want to give up. That?s something that goes to the heart of people?s senses of self-esteem and wellbeing, something that can extend or diminish life expectancy.

There?s no equivalent downside for using Google, which means that merely offering a slightly better version doesn?t represent a convincing argument. Nobody is going to change search engines because the top 10 results are slightly more relevant.

So those companies looking to compete have to take a different approach: the we?re-not-a-search-engine approach. This is the approach demanded of disruptive technologies since the beginning of time. Don?t offer a faster horse, build a car.

The road to success requires would-be Google-killers to solve a problem that Google doesn?t solve, to create a new habit under a new circumstance, where it can flourish free from the inexorable pull of ingrained attitudes.

This is why David Berkowitz reported last September that MySpace was the fourth largest search engine: because they?re competing in a different arena.

Twine is another great example; it represents a totally new way of interacting with data. You can create a habit of using Twine without threatening your Google use, transitioning slowly and imperceptibly until you wake up one day and say, ?Remember when we all thought Google couldn?t be beaten??

This is also why it?s so important for Google to snap up a token presence in every emerging Web 2.0-3.0-4.0 space. They know that they?re unlikely to be threatened on their own turf, and they want to make sure they?re at least in the ring wherever the fight?s going to be.

The great philosopher Osho said, ?If you want to do something with darkness, you have to do something with light, not with darkness at all. You have to light a candle, and suddenly there is no darkness.? I?m not suggesting that Google represents the Forces of Evil here, but the concept is transferable: light the candle of a new habit, and the old habit disappears.

Will Powerset be the candle of a new habit? That remains to be seen. Ultimately, though, someone will be the candle; as Osho also said, ?Habits die hard. But they die certainly — if one persists, they die.?

Summary: This piece was published two days ago in Search Insider, and got some positive feedback, so I thought I’d share it with you here as well. It discusses how search is evolving from a content focus to a user focus.

A few days ago in the Online Spin, Max Kalehoff wrote

?all of our silo-plagued concepts of media categories ? like Internet, newspaper, television, stuffed into this latest ZenithOptimedia report ? will eventually connect and blur onto the digital grid. That will lead us to adopt models based less on media formats and audience proxies, and more on actual people, content, behaviors, receptivity, relationships and performance.

He was talking about advertising, of course, but the trend he describes is universal. From personalized search to the social graph, the relentless trend of search is towards understanding the people who are at its core.

While focusing on people might seem to be an obvious direction, this is a massive shift in tactics from the early days of search. Remember? It was a major feat for a search engine to even index a site, let alone worry about matching it to the hopes and dreams of the person running the query.

Successes were based on volume: the number of pages a site could index was an indication of its quality. Now we take it as a given that we?ll receive several million responses for a moderately ordinary query.

Directories and portals provided migrants from the Land of the Yellow Pages with a familiar, organized interface. But once users realized that they could get where they wanted to go far faster with freeform searching, directories went the way of the dinosaur.

As the size of indexes increased, the volume of noise rose, and engines turned their attention to deciphering the value of the content itself. ?We?ve got 500 pages that say the same thing,? the reasoning went. ?How do we know which one to list first??

Google?s inbound/outbound link quantification went a long way towards answering that question—but in a vacuum. Third-party sites are used as proxies for third-party opinions. The user doesn?t figure at all.

And now, finally, we?re beginning to open our eyes and remember that there is a human being at the origin of that query. There is a human being consuming that ad. There is a human being building that relationship.

Google?s working on the concept with personalized search. Facebook stumbled over it—and kicked it in the gut by mistake. And, like sperm at an egg, a thousand alts are pecking away, trying to get in. They know there?s another quantum leap to be had.

Last week on my own blog, I described my current state of information overload. This phenomenon is not new, nor is it exclusive to me. It is, however, the single most pressing reason why focusing on people will be the key to the next great evolution in search.

As I struggle to stay afloat in an infinitely-growing sea of information, one thing has become clear: we don?t need more content. We need our content, the content that will add the most value to our experience. On a good day, I?ve got roughly two hours to read the millions of blogs that are begging to deliver me content.

If you don?t make it about me, I?m going to drown.

This piece was published in SearchInsider on Friday. In it, I make the point that the way to improve search is to remember that the people searching, and their intentions, are what really matter.

Microsoft is the latest to take a stab at deciphering intent, outlining updates to Live Search in a pair of well-titled blog posts: Do What I Mean, Not What I Say! (part I and part II). In it, they describe new semantic features that they hope will take them further along the road to disambiguating user intent:

1. AutoSpell Correction
   Microsoft?s description of this is kind of funny, because they make a direct comparison to Google?s treatment of suspected spelling errors without, of course, referring directly to Google:

   If we are absolutely, completely, totally, “no doubt about it” confident you misspelled one of your search terms, we automatically deliver a page that includes spell-corrected results, rather than a page of misspelled results accompanied by a “Did you mean _______?” link at the top?
   ?With AutoSpell correction I get the correct result the first time, regardless of the misspelling. Instead of being two clicks away from pizza, I’m just one. Being two clicks away just keeps people hungry, rather than satisfying their intent!

   Google, of course, has been checking spelling for years, but evidently their agonizingly long two-click process of asking, ?Did you mean?? has deprived searchers of much-needed pizza.

2. Stemming
   Stemming is about knowing when you mean ?books? instead of ?book? but not ?cables? instead of ?cable?.
3. Equivalencies
   Equivalencies are abbreviations and other instances of words whose meanings were heretofore known to searchers but not to engines. The example Microsoft gives is ?CA CHP?, meaning California, California Highway Patrol (although why you?d need the redundant California is beyond me).
4. Intelligent ?Stop Word? Retention
   Stop words, like ?a?, ?the?, ?in?, etc., are typically discarded by Google, even though sometimes they add value. The example Microsoft gives is ?The Office?, which is an entirely different query than ?office?.

These features are aimed at what I call ?structural semantics?: the ability to ascribe meaning based on the structure of the query.

Laurie Petersen at Online Media Daily recently discussed a different approach from Acxiom:

DATABASE MARKETING GIANT ACXIOM TODAY officially launches its Relevance-X products designed to allow marketers to make online media buys…
“We’re really excited about this,” said Rich Howe, Acxiom’s chief marketing and strategy officer. “We’re bringing our knowledge and experience in direct marketing to the online channels to give clicks context–going far beyond basic information such as age, gender and household income to include the attitudes, beliefs and lifestyles of consumers that are much more predictive.” [emphasis mine]

According to the article, Acxiom has been seeing click-through rates double or triple in tests of the Relevance-X system: powerful numbers.

The ability to successfully map a user?s intention has tremendous implications for how people access the Internet, and it?s a worthwhile exercise to imagine an Internet that responded and adapted directly to the individual, in real time. Imagine, for example, how your usage of StumbleUpon might change if you knew that StumbleUpon would deliver the exact web page most relevant to you personally, every time.

The company that I blog for, VortexDNA, is also in the business of intention mapping. Our technology is based on the knowledge that an individual’s purpose and values can be used to predict relevance. Human intention is structured according to the mathematics of complex systems, and therefore can be understood.

Microsoft, Acxiom, and VortexDNA: we?re all saying the same thing. Hugh MacLeod of gapingvoid says it too:

?it’s not the wine per se that is interesting, it’s the conversations that happen around the wine that is interesting. And that is true for all social objects. People matter. Objects don’t.

You said it, Hugh. People matter. Not objects, or keywords, or stemming, or equivalencies. These are all tools to help us understand the people behind the queries and deliver what they want.

When it comes to search, the road to heaven is paved with user intention.

Imagine if we were able to find things in our ‘real’ lives the way we find them on the web. Here are some things we search for on a daily basis:

• Snooze button on alarm
• Slippers
• Glasses
• Clothing, including shoes and socks
• Breakfast
• News items, either via television, newspaper or radio
• House keys
• Car keys
• Briefcase or purse
• Shortest route to destination
• Food or coffee en route
• [Insert here: all the computer stuff we search for, either online or locally, all day]
• Physical files or artifacts
• A fresh roll of toilet paper
• Our printout amongst many
• Shortest route to client office
• Room in massive complex where client meeting is being held
• Name of client’s significant other
• Any topics that should be strenuously avoided
• Parking space
• Bottle of wine your significant other asked you to pick up
• Those olives (not the green ones, the black ones, but not the crappy black ones they serve on airplanes)
• Way to avoid construction on the way home
• That one ingredient you were sure you had
• The gravy boat
• Any topics that might actually engage your teenage children in conversation
• A TV show that you’re genuinely interested in watching
• The other half of your pajamas

Show me the search engine that can find all that—then I’ll be impressed.

Over at Search for Better Search, the topic of conversation is “Winning Users’ Trust”. They’ve got an all-star list of heavy hitters weighing in: John Battelle, Om Malik, Michael Arrington, etc. I was struck by the fact that, for the majority of the commentators, trust was a much bigger issue than data. It didn’t seem to matter what information search engines stored, as long as they were transparent and gave users control:

John Battelle: The way they can do this is by giving their users access to, and editing permissions over, the data they keep.

Gary Price: Transparency is key… For search engines to sustain user trust, they must be transparent about the filtering they use to display results, capture information and disclose biases.

Michael Arrington: User trust is built by giving users control…

Richard MacManus: I think the number one best practice is to engage the community and have conversations with them… Once that starts to happen, people begin to trust corporations more.

Dr Riza Berkan: Search engines must openly declare what they are doing with the data and all tracking devices, almost like a confession.

Ars Technica: The compliment (sic) of power must always be control; a search engine that learns from you must also trust you by you giving you the tools to curate and prune your search history and to opt-out at will.

Matt Marshall: Somehow you’ve got to protect identities, and if you’re collecting information about us, you should let us know what you’re collecting, and where to find it.

Only one person, Om Malik, had anything at all to say about whether corporations should be storing data at all:

Search engines have to make it clear that they don’t store any data whatsoever. The only way they can do that is if search is their only business. Email, personalized home pages, mobile clients, IM clients, search history - they are the enemies of private searching.

I don’t know about you, but I found this really interesting. There’s a big difference between people not wanting their information shared at all and being able to control when and with whom it gets shared. If search engines follow Om’s worldview, then trust is a non-issue (presuming you believe that they’re not out-and-out lying about not storing data). If you don’t have any of my data, it doesn’t matter whether or not I trust you to protect it from subpoenas or hackers—you can’t give them what you don’t have.

If search engines listen to everyone else, on the other hand, one of the single most critical factors for web businesses over the next few years will be the ability to engender trust.

These two concepts are vastly different in practice. If trust is more important than privacy, then a company that says up front, “We collect all of your search history and use it to target you directly,” will do better than a company that says, “We will not use your search history for anything other than making our algorithm better, and oh by the way those ads today that match your search query from two weeks ago? Pure coincidence.”

In other words, it’s not what they do. It’s whether they keep their word.

Which is more important to you? Trust or privacy? Cast your vote to find out what other people think!

This piece appeared on Friday in Search Insider. It’s gotten some interesting comments, so feel free to respond either here or there.

On Tuesday, the BBC reported that Yahoo and Microsoft are adding new features as part of their ongoing attempts to convince the world that they are serious contenders to the Google throne. Yahoo will provide on-the-fly query suggestions, while Microsoft is quadrupling the size of its index. Both companies are touting the fact that they will soon include links to photos and video on the results pages.

Hmmm? where have I heard this before? In addition to Google?s own Universal Search, I mean. Oh, yeah! Ask.com did this back in June! And, despite rave reviews of Ask3D, comScore shows them slipping ever since, from a 5.0% market share in June to 4.7% in July to 4.5% in August.

Yet Yahoo and Microsoft insist on trying to woo searchers away from Google by launching new features. Unfortunately, they?re in a bit of a lose-lose situation right now. If they upgrade, they?re only playing catch-up. If they don?t upgrade, they fall even further behind.

They should have called me first. Me or Jeremy Kaplan, the editor of PC Magazine. Kaplan was interviewed on MarketWatch for his thoughts on the matter, and he had this to say:

It seems like it?s really a mindshare thing more than anything. I think most of the search engines seem to be able to cull the same information. It?s just a question of getting the brand out and transforming the way people search, and that?s definitely an uphill battle.

It?s likely that Jeremy Kaplan has access to a broader dataset than I do; even so, I surveyed myself and found his observation to be true.

For example, I have absolutely no inclination or disinclination towards Microsoft search. In fact, I?m quite confident that it delivers similarly useful results to Google. In addition, I am somewhat ashamed to admit that I have MSN as my home page because it came with the browser, and I?m doubly ashamed to admit that I?m too lazy to spend the three seconds it would take to make Google my home page. Nonetheless, every time I have to run a search, I launch Explorer and type Google into the address bar.

Thank you in advance for your many words of advice on how to change my inefficient habits. I realize I need help. My point here, however, is not about my own loss of street cred; it is that Google?s hold on the market, or at least on that share of the market sitting at my desk, is so strong that I invest effort to bypass the Microsoft search bar on my home page.

I invite you to think about your own habits when you search, and whether the promise of a couple of new features would be sufficient to entice you to change your behavior. What would it take?

Whatever the answer to that question may be, I don?t think it?s accessible to Microsoft and Yahoo. They?ve never achieved escape velocity, the minimum speed necessary to bust out of the Earth?s atmosphere, and now it?s too late: they?ve begun to decelerate.

No, there are only two possibilities for another search engine to unseat Google, and they would pretty much have to happen simultaneously:

1. A new search engine, or coalition of search engines, will have to offer both the novelty to capture the imagination of early adopters and the substance to cross the chasm, and
2. Google will have to make a major misstep.

Charles Knight at AltSearchEngines understands this, which is why he?s fighting for alternative search engines to collaborate. He realizes that, combined, they have a lot more momentum than they do individually, and a much greater chance of reaching escape velocity.

Within a few years, the Universal Interface that he champions could be in outer space—while Microsoft and Yahoo watch from the ground and fiddle with features.