This is the fourth in what looks to become a 5-part series on relevance in search. If you’re just now tuning in, here’s what you missed so far:

1. The opening gambit: an overview of relevance in search and a proposal of four qualities that can be used to define it
2. Part II: an exploration of the human connection, the first on the list of qualities
3. Part III: deconstructing discovery: how the unexpected can be made relevant

Today we’ll move on to the subjective nature of search, or the idea that the degree of relevance changes from person to person and moment to moment. The fifth and final installment will cover the measurement conundrum.

Interestingly, the original post was sparked by a conversation about subjective relevance begun by Hamlet Batista. So even though this is the third point on the list, it contributes significantly to our ability to generate meaningful relevance.

Subjectivity essentially says that it is the end user who decides whether or not a result is relevant. I can show you a result all day, and tell you that it matches every other site you’ve ever visited, that 92% of people in your demographic found it useful, and that it’s got your name on it, but if you don’t agree that this is a site you want, you win. Every time. No discussion.

Subjectivity also means that what is relevant to you today may not be relevant to you tomorrow. Ever change your mind about whether you’re interested in something? Me too. Show me someone who hasn’t.

What does this mean to companies like ours who are seeking to improve relevance? It means that we’re shooting at a moving target. It means that we have to operate on a basic principle: relevance isn’t a fact; it’s a transient state.

It is also one the basis of the principal arguments against personalization. Because most personalization efforts have been driven by user history, by definition they tend to be self-reinforcing. Aaron Chronister over at TheMadHat just put up a piece called ‘Personalized Search and Why It Sucks‘; his first reason that it sucks is that it tends to look backwards rather than forwards:

Personalized Search limits the discovery of new sites and new information. The higher placement of sites you have visited and browsed before pushes anything new that you may have not seen before down the page. Users will have to start finding new content in other ways than search; through blogs, news sites, etc. The users don?t like this and the search engines certainly don?t want people finding content in other ways. If the result sets are filled with things I?ve already seen I?m going to look elsewhere.

Aaron’s viewpoint is based on a personalization model that uses history as the basis for ranking sites. When he says that this limits discovery, he is reinforcing the idea, discussed in Part III, that we may find things that are entirely new to be totally relevant to us, in fact more relevant at times than something we’ve seen before. This phenomenon ties into two realities:

1. We don’t know what we haven’t seen yet, so something new may be better than anything we’ve seen so far.
2. Our tastes and interests change all the time.

So how can we begin to increase relevance when we know we’re always going to be on the back foot? One way is to accept the flaws of historical relevance as worthwhile tradeoffs for the benefits of that approach. Another—and this is the outlook VortexDNA has chosen to follow—is to seek out a method of personalization that doesn’t rely on history.

Although this might sound like an impossibility, it is in fact entirely attainable. Think about the way you recommend things to your friends, for example. You’re likely to know their interests and things they’ve liked in the past—that’s the historical model. You also might have found something entirely new that, based on what you know about them as a person, you think they might like. That’s the VortexDNA model.

The question of the subjective nature of relevance has been tackled by Google before, when they realized that the same algorithm that gives weight to incoming links (thereby favoring older sites) works against users when the nature of what they’re searching for is novel. An article in the New York Times referred to the fact that a search for ‘Google Finance’ didn’t bring up a link to the company’s own stock quotation service, prompting Google engineers to develop a new QDF (Query Deserves Freshness) protocol.

So while stories about the 2005 London bombings might have been more relevant to people initially, stories about the 2007 London bombings became more relevant on and after June 29, 2007. While stories about LinkedIn might have been more relevant six months ago, stories about Facebook are more relevant now. And neither of those two is relevant to someone who doesn’t care about social networking to begin with.

To me, what this says about subjectivity is that relevance has to take into account the user’s present range of interests, desires and state of mind if it is to stand a chance of being effective.

Am I unraveling this concept or merely making it more difficult? I’d love to hear your thoughts.

This entry was posted on Thursday, August 2nd, 2007 at 8:38 pm and is filed under People matter, Relevance, Search. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.