I had the privilege this week of attending a lecture by Professor Hal Varian, Chief Economist for Google. Varian discussed the advent of computer-mediated transactions and how they transform our business practices.

There were a couple of interesting points he raised: historical (in a pre-literate and pre-numerate era, how could people shipping barrels of olive oil have any confidence that the amount of oil that left was the same amount that arrived?), logistical (computer-mediated transactions enable more and more complex contractual arrangements), and conceptual (behavioral targeting, etc.).

This last, conceptual, is a big thing for Google these days, since they’ve been in the behavioral targeting business for all of two weeks. It’s also where Varian started to get into Web Genome Project territory. I found one thing he said particularly interesting:

In general, people have no problems with the intended use of data (more relevant content, etc.). What people are worried about is the unintended use of data (AOL’s massive data spill, etc.). The problem, therefore, is not so much a privacy problem, but rather a security problem.

That’s a pretty interesting comment, and it certainly rings true to me. “I don’t want Google knowing all this stuff about me,” people say. “Who knows what they’re going to do with it? What if somebody unscrupulous gets their hands on it?”

The core proposition of the Web Genome Project is personalisation with privacy. In light of Varian’s comments, however, it’s worth revisiting that proposition, because in fact it’s much stronger than that. The WGP model means that no clickstream or historical data is ever collected in the first place. If a thief were to break in, the vault would be empty; there’s just nothing there. So the model actually eliminates the entire question of privacy. It doesn’t much matter whether I can keep your data private if I don’t have any data on you to begin with.

Gratifying stuff from someone who’s earned his stripes. What are your thoughts about privacy vs. security?

Note: This piece was published on this blog last week, but disappeared during migration to a new server! So bear with me if you’ve already read it.

One month ago, I predicted that the currently-accepted-as-standard behavioral targeting model doesn?t stand a chance. Simon Levine was kind enough to point out that ISP-based targeting isn?t standard ? it?s a new, wrong model, so let?s re-posit it: ISP-based behavioral targeting doesn?t stand a chance.

That was after NebuAd had pretty much ceased all activity, when Phorm was still doing okay. Here?s what?s happened since then:

Last week, 15 people filed suit against NebuAd for privacy violations.

?The collection of data by the NebuAd device was wholesale and all-encompassing,? the lawsuit alleges. ?Like a vacuum cleaner, everything passing through the pipe of the consumer?s internet connection was sucked up, copied, and forwarded to the California processing center. Regardless of any representations to the contrary?all data?whether sensitive, financial, personal, private, complete with all identifying information, and all personally identifying information, was recorded and transmitted to the California NebuAd facility.?

?Consumer advocates also were alarmed by the sheer scope of information available to NebuAd. Unlike older behavioral targeting companies that only collected data from a network of publishers, Internet service providers have access to everything?including activity at search engines and at non-commercial sites, such as sites operated by religious groups.

?The plaintiffs, who are seeking class-action status, allege that NebuAd violated a federal wiretap law, California privacy law and computer fraud law, among others.

Wendy Davis reported further on the story:

?It could be a landmark case,? said attorney Bennet Kelley, founder of the Internet Law Center in Santa Monica, Calif. and former privacy director at ValueClick. He added that a victory for the plaintiffs could mean that broadband providers will be left with no choice but to seek users? affirmative consent before selling information about people?s Web activity to ad companies.

?privacy advocates say that the case doesn?t invite courts to make new standards as much as to clarify that existing laws already prevent Internet service providers from selling users? clickstream data without their consent.

?There?s a very high likelihood that some of these new advertising proposals that take advantage of Web traffic are illegal,? said Marc Rotenberg, executive director of the Electronic Privacy Information Center.

Across the pond, Virgin Media publicly dismissed the idea of working with Phorm, even though they were one of three ISPs slated to use the platform. Orange, the ISP, also rejected any collaboration.

?perhaps the ISPs are realizing that the proposition might be a public relations disaster should angry subscribers voice complaints.

With so many rats making their way off the ISP-based BT ship, who would be foolish enough to go in the other direction?

What do you think the companies, courts and governments should do? Should they put a stop to ISP-based BT now?

In the past two posts, we’ve explored some of the benefits of customer segmentation. In this post, I’m going to look at some of the pitfalls, specifically what happens when you use the wrong criteria for segmentation.

Fortunately, the recent U.S. election offers some insights into how people react when they feel they’re being unfairly targeted. Because a victory for either party would have resulted in a first, the issue of segmentation came up pretty regularly. Would blacks vote for Obama because he’s black? Would women vote for McCain because of Sarah Palin’s gender?

We can never know what is in someone’s heart of hearts, and what truly drove them when they went into the polling booth. But if media coverage, op-ed pages, blog posts and general commentary are any guide, the answer to both of the above questions is a resounding, ‘No!’

Back in August, Gail Collins of the New York Times had this to say:

The idea that women are going to race off to vote for any candidate with the same internal plumbing is both offensive and historically wrong. When the sexes have parted company in modern elections, it?s generally been because women are more likely to be Democrats, and more concerned about protecting the social safety net. ?The gender gap traditionally has been determined by party preference, not by the gender of the candidate,? said Ruth Mandel of the Eagleton Institute of Politics.

As for Obama, despite an obvious and rightful outpouring of national pride at overcoming one of our oldest racial hurdles, Time Magazine calmly pointed out that segmentation itself couldn’t have won the election for him:

…most blacks have not been blinded by race. Though proud of his blackness, those who did vote for him were far more thoughtful in making the decision and based their vote on promises that he now must keep. To think that this election was a shoo-in for him among blacks because of our affinity for our own people is disingenuous at best and at worst insults our intelligence. And it ignores the fact that many other blacks have run for President and walked away without winning a primary, much less the presidency. Neither Shirley Chisholm, Jesse Jackson, Al Sharpton, Alan Keyes nor any other black candidate amassed black support the way Obama did.

Both of these attitudes are the ideal: we want people to vote for or hire or marry the person who is best suited to the job, not the person with the right gender or skin tone or career prospects.

What this means in terms of segmentation is that, in order to get it right, it’s critical to use the right criteria — and just as critical to avoid the wrong criteria. In this election, it wouldn’t have been wise to make predictions based on gender or race.

Imagine a health insurance company that segmented customers based on whether they enjoyed reading John Grisham novels. The segments would be unlikely to behave as predicted. Other companies with more accurate segmentation strategies would garner more low-risk customers, and the John Grisham Insurance company would go into the death spiral I discussed earlier in the week.

Here’s the bottom line: segmentation is great, but you’d better have your segments right.

I welcome your thoughts.

Summary: In this post, I discuss the financial benefits of segmentation, and explore a few of its moral implications. In order for segmentation to be at its most effective, it has to consider profitability as well as concern for the wellbeing of all stakeholders, including customers and society.

What if a company had no waste?

What if a company knew exactly who its customers were (I mean, exactly), knew the most efficient way of getting them to purchase, and knew the most efficient way of delivering a product?

Would that company be able to offer its wares at a cheaper price than a company that didn’t know any of those things?

You betcha.

Would that company’s customers benefit from the reduced waste? Of course. All of us, at one time or another, have subsidized waste; in fact, we do it frequently. Ever buy from a company that advertises? Then the price you pay includes the cost of that ad reaching lots of people who don’t buy.

Before you assume that I’m decrying advertising, let me say that advertising is itself more efficient than the alternative. I have a book on my desk called ‘Masters of Copywriting’; it contains a piece from J. George Frederick, written in the early 20th century:

Of course, the public pays the cost of advertising, as it has always paid the cost of all selling. The constantly overlooked fact is that selling expenditure of older days was unseen; it represented salesmen’s expense and other high cost methods of selling. Sales cost per unit of merchandise was far greater in older days than now. To-day the public sees the sales expenditure, in the form of publicly displayed, spectacular advertising. Because advertising comprises a considerable grand total in volume and bulks large in public consciousness, it is mistakenly regarded as being an additional burden of selling cost, whereas, in truth, it is only an altered and more visible, but on the other hand, lower selling cost. If what is now spent for advertising were spent for salesmen, circulars to the trade, and old-time sales methods, nobody would be noticing it, or considering anything to be amiss — yet it would bulk to tremendously greater proportions in the attempt to accomplish the same results that advertising produces in the present era.

The bottom line of segmentation is this: that it allows companies to become more efficient in the distribution of their products.

Here’s an interesting scenario, taken from a textbook on insurance pricing:

Imagine there’s an insurance company, Equal Treatment Corp., that sells policies to all comers for $200.

One day a competitor, Segmentation Limited, comes on the scene. Segmentation Limited looks at Equal Treatment’s marketplace and notices something interesting: the people therein are either bookworms or skateboarders.

Further, Segmentation Limited notices that skateboarders are significantly more likely to get into accidents (imagine that). In fact, when they do the math, they realize that a skateboarder policy should actually cost $250.

Of course, if they go into Equal Treatment’s market selling policies for $250, while Equal Treatment keeps selling at $200, they’re going to get killed.

Luckily, Segmentation notices something else. While skateboarders are more likely to get into accidents, bookworms are less likely. When they do that math, they find a bookworm policy should really only cost $150.

So they come into the market, offering bookworms policies for $150 and skateboarders policies for $250. What do you think happens next?

Correct! The bookworms all move over to Segmentation, while the skateboarders all stay with Equal Treatment.

But it’s what happens after that that’s really interesting. Equal Treatment now has a very different mix of clients. Before, the bookworms were essentially subsidizing the skateboarders, but now there aren’t enough bookworms, and Equal Treatment begins to lose money.

So Equal Treatment raises its premiums. More bookworms flee. The only people still willing to insure with Equal Treatment are the riskiest people, the ones most likely to require the company to pay out claims. Equal Treatment raises its premiums again in order to handle the higher risk, and the death spiral continues.

From this simplistic scenario, it would seem that segmentation is the way to go in the insurance business, and blanket pricing is no good. But the world isn’t simplistic.

For example, consider your reaction if we replaced the category ’skateboarders’ with the category ‘cancer patients’ or ‘autistic children’.

The difference, obviously, is that one group is engaging in obviously risky and unnecessary behavior, while the other group may be suffering from challenges it didn’t ask for.

So segmentation on its own has moral implications, just as free-market practices can result in risky behavior by financial institutions. Effectively applying segmentation means considering not just the profitability of the company itself, but the wellbeing of all stakeholders — including customers and the wider society.

Fortunately, history has shown us that companies that have this sort of inclusive stakeholder focus consistently outperform those that don’t, meaning that rewards accrue where they are due.

What do you think about the benefits of segmentation to society and the accompanying moral implications?

In my last post, I wrote about what I perceive to be the fundamental problem with behavioral targeting: the value proposition to consumers is less than the cost to consumers.

The post received highly thoughtful comments from Pete and Jim. Both of these people are from anti-Phorm websites (as you’ll see if you click through), and both of them have interesting perspectives on the behavioral targeting problem. I thought I’d aggregate them here and add my spin. So, without further ado, I present my Three Behavioral Targeting Keys to Success.

BT Key to Success #1: Data about an interaction with one supplier, taken without consent, should not be used to promote a new interaction with a different supplier
Trust me, when I explain this one it’s going to be more than obvious.

Imagine for a moment that you go to a Target store. As you’re walking through the store picking out goodies (including, let’s say, diapers), a fellow shopper is surreptitiously observing what you buy. As soon as you pay, that so-called ‘fellow shopper’ ditches her bogus shopping cart, runs outside, and calls Walmart to give them a full report. Walmart then sends you coupons for baby food.

This scenario sounds lousy. But the idea of ISP-based behavioral targeting is even worse, because there’s no existing trust relationship with that phony fellow shopper, while there is one with your ISP. Imagine if the post office read your mail, found out which books you bought from Amazon and how much you paid for them, and then sold that information to Barnes & Noble.

BT Key to Success #2: Recognize that certain ’safe havens’ of communication have to exist for a healthy society
Facebook got a taste of this with the backlash from its Beacon advertising system. People became enraged at the idea that they couldn’t control which activities were kept private.

In that case, the ramifications were generally small — a wife found out too soon about a surprise gift from her husband. In other cases, though, the ramifications may be quite large, manifesting not only in tangible repercussions (consider political communications in repressive countries), but also in a diminished sense of stability and security in society at large.

BT Key to Success #3: Make sure that consumers not only want what you’re offering, but have a clear and transparent choice to accept or reject it
Jim pointed out that Google’s model is easily controlled by the user, as contrasted with ISP-based behavioral targeting:

You don’t want to be tracked by google? Fine, block google’s cookies. Don’t want google to profile your e-mail? Don’t use Gmail. Don’t want them recording your searches? Use a different search engine. You don’t have that kind of assurance when companies start intercepting all your traffic. They take control, not you. You have to rely on their promises to behave fairly, to honour your preferences and you have *no way to make sure they’re doing it*. This makes users uneasy. They feel as though they can’t trust their service providers, something I have to say most service providers have brought upon themselves through years of silly money-grabbing schemes like this and generally deplorable levels of customer care.

In fairness, Phorm is now opt-in only, although I’m not certain how thoroughly they explain what they do at time of opt-in. My understanding is that their technology is presented as protection from phishing sites, which it may well do in addition to its other uses.

Further, you could also make the argument that, just as you could choose to not use Google or Gmail, you could also choose to use a different ISP. This one is a bit trickier, however, and brings some other considerations into play. If ISP-based behavioral targeting were to become the de facto standard, choosing a different one wouldn’t make a difference. And, ultimately, the issue is one of invasion of sensitive information.

When professional service providers gain access to sensitive areas of your life, it is encumbent upon them to behave with corresponding sensitivity. You don’t want your mammogram tech telling you you’ve got nice tits. The privilege of access to sensitive areas of our lives carries with it the responsibility of appropriateness.

Okay, enough for today… I hope I’ve given you some food for thought! That’s what your comments always do for me; I welcome them below.