High-powered internet applications typically need teams of experts to maintain them. Not any more, say European researchers who have built a system to create applications that manage and fix themselves.

Part of the internet’s potential lies in its ability to link hundreds, thousands, or even millions of devices.

Whether a user is downloading a video from a peer-to-peer service, performing scientific research on a grid, or using “cloud computing” to manage a business, programs that let many devices and applications work together are crucial.

The problem, says Peter Van Roy, coordinator of the EU-supported SELFMAN project, is that it’s getting harder to keep those systems working.

“The central challenge when you build big internet applications is how to keep them running without having to tweak and manage them all the time,” he says.

The SELFMAN team set out three years ago to solve that problem by finding out how to build programs that take care of themselves in the rough-and-tumble internet environment.

“We wanted to make big internet applications easy,” Van Roy says, “so that all the management problems you normally have are handled by the system itself.”

The payoff, he says, will be huge. “It will take the internet to the next level.”

Self-management – four key features

The SELFMAN researchers identified four vital functions for a distributed application to manage itself – self-configuring, -tuning, -healing and -protecting.

Software is continually being patched, updated or replaced. For a distributed system to configure itself, it needs to keep track of all its components, update them as needed, and make sure that all parts of the system can still talk to each other.

“Our system can ask a component, what version are you? Who are you talking to? It can then replace an old version with a new one as needed,” says Van Roy.

Self-tuning means that the system can instantly adjust to changing loads and to components leaving or joining the network.

“Suppose one node is getting overloaded,” says Van Roy. “Our load-balancing algorithm allocates new nodes close to that hotspot. It spreads the heat to the other nodes and the hotspot cools down.”

The internet is an unpredictable environment. Routers crash, cables get cut, parts of the system overload and grind to a stop, and components come and go.

“With SELFMAN,” Van Roy says, “each node stores some of the data and each piece of data is replicated a certain number of times. If a node crashes, the other nodes detect the crash, find a new node and give it the missing data. The system heals itself.”

One of the biggest problems SELFMAN tackled was self defence.

The researchers discovered that a system’s security depends on its topology – how nodes are linked to each other. They found that “small world” networks – in which most nodes are not directly linked, but in which any node can communicate with another in a few steps – were the safest.

“With a small world network, it’s easier to detect, isolate, and eject bad nodes,” says Van Roy. “The security service observes the system’s behaviour. If it notices that certain parts of the network are acting abnormally, it takes action.

Read more . . .

Related articles by Zemanta

• Cloud vs Traditional Hosting (brilliantthinking.net)
• Where is YOUR data? (cloudave.com)
• Rethinking The Cloud: From Client/Server To P2P (cloudave.com)
• If Load Balancers Are Dead Why Do We Keep Talking About Them? (java.sys-con.com)
• BotchagalupeMarks for September 10th – 08:43 (johnmwillis.com)

Related posts:

1. The internet at forty
2. The Cyber Sea: World’s Largest Internet Undersea Science Station Boots Up
3. Seed’s Goal Is To “Redefine Journalism For The Internet Age,”
4. Charging by the Byte to Curb Internet Traffic
5. Internet addresses set for change

Tags: cloud, cloud computing, internet, internet applications, internet environment, management problems, self management, unbreakable internet

This entry was posted on Sunday, October 18th, 2009 at 12:10 pm and is filed under Communications, Consider This, Innovation, Job Changes, Need to Know, Shape of Things to Come?. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.