The Crowd Is Wise (When It’s Focused)

FEW concepts in business have been as popular and appealing in recent years as the emerging discipline of “open innovation.” It is variously described as crowdsourcing, the wisdom of crowds, collective intelligence and peer production — and these terms apply to a range of practices.

The overarching notion is that the Internet opens the door to a new world of democratic idea generation and collaborative production. Early triumphs like the Linux operating system and the Wikipedia Web encyclopedia are seen as harbingers.

In the new model, innovation is often portrayed as a numbers game. The more heads, the better — all weighing in, commenting, offering ideas. Collective knowledge prevails, as if a force of egalitarian inevitability.

But a look at recent cases and new research suggests that open-innovation models succeed only when carefully designed for a particular task and when the incentives are tailored to attract the most effective collaborators. “There is this misconception that you can sprinkle crowd wisdom on something and things will turn out for the best,” said Thomas W. Malone, director of the Center for Collective Intelligence at the Massachusetts Institute of Technology. “That’s not true. It’s not magic.”

The Netflix Prize is a stellar example of crowdsourcing. In October 2006, Netflix, the movie rental company, announced that it would pay $1 million to the contestant who could improve the movie recommendations made by Netflix’s internal software, Cinematch, by at least 10 percent. In other words, the company wanted recommendations that were at least 10 percent closer to the preferences of its customers, as measured by their own ratings.

(Cinematch analyzes each customer’s film-viewing habits and recommends other movies that the customer might enjoy. More accurate recommendations increase Netflix’s appeal to its audience.)

The contest will end next week because a contestant finally surpassed the 10 percent hurdle on June 26, and, according to the rules of the competition, rivals have 30 days from that date to try to beat the leader. The frontrunner is a seven-person team, and its members are statisticians, machine learning experts and computer engineers from the United States, Austria, Canada and Israel. It is led by statisticians at AT&T Research.

The leading team is a very elite crowd, indeed, but it is also one that was made possible by the Internet. The original three AT&T researchers (one has since joined Yahoo Research, but remains on the contest team) made good strides in the first year of the contest. But to make further progress, they went looking for people with other skills and perspectives. So they reached out eventually to a pair of two-person teams, who were among the leaders in the rankings posted on the contest Web site.

Read more . . .

Related articles by Zemanta

• Bits: Improving Netflix for a $1 Million Prize (bits.blogs.nytimes.com)
• Can a machine know what movies you like? (ycorpblog.com)
• The Social Nervous System Has More Than One Sense (radar.oreilly.com)

Random Posts:

• Forging a Hot Link to the Farmer Who Grows the Food
• On a Roll: Autonomous Navigation Lasers and Robotics Push “Smart” Wheelchair Technology to the Cutting Edge
• Catch of the Freezer
• India is morphing into a global R&D hub, but can it ever take on Silicon Valley?
• Yellow Is the New Green – Sewage Treatment Wastes Energy

Related posts:

1. A $1 Million Research Bargain for Netflix, and Maybe a Model for Others
2. India and China Wise Up to Innovation
3. ChallengePost launches problem-solving site with Wozniak, Betaworks onboard
4. An EBay For Innovation
5. You’re Leaving a Digital Trail. What About Privacy?

Tags: collaborative production, collective intelligence, collective knowledge, democratic idea, harbingers, Innovation, netflix, numbers game, wisdom of crowds

This entry was posted on Sunday, July 19th, 2009 at 9:07 am and is filed under Innovation, NY Times. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.