By Thomas Mennecke 12/20/02
A storm is brewing the world of P2P networking. Feeling compelled by the inadequacies of Gnutella, Mike Stokes, the lead programmer from Sharaza broke ranks with the major Gnutella developers and implemented his own protocol, dubbed "Gnutella2".
This has caused a sharp divide in the Gnutella world. The line has clearly been drawn between those who support and oppose this concept. Expressing concern and opposition to the protocol is BearShare, LimeWire, XoloX and Ares (former Gnutella client), while Gnucleus and a large community of users, support the new idea.
Gnucleus, a well respected open source client, has been particularly supportive on this issue. When asked if he supported the protocol, John Marshall, the lead programmer of Gnucleus responded:
"I would like to add support for the G2 protocol into GnucDNA. Gnucleus and any other applications using the DNA would then have the advantage of running both the Gnutella and G2 protocols underneath."
In addition, the Gnucleus camp feels optimistic about the new protocol. "I feel that G2 is a very clean protocol that can be extended easily without creating incompatibility headaches between clients. It's as ‘Gnutella’ as we make it, I would like to run with this protocol and create the next generation with it. However, other clients do not like the 'Gnutella2' name so they will not implement it based on that fact."
Supporters of Gnutella2 see this endeveor as more of a "free vs commercial" situation. John adds, "The backstory of this whole issue is Mike of Shareaza developed a new protocol for a couple months, branded it as Gnutella2, and tested out a working version in his client. The GDF’s first reaction was negative because they claimed it used the same ideas from other proposals. Once the protocol specs were released this was obviously false, but the GDFs reaction was still negative so Mike has not bothered to release the rest of the specs. Currently the feeling on the GDF is that the Gnutella2 name should not be used for this new protocol because they want nothing to do with it. I would like to implement it though. Now its more like "Free vs Commercial" clients, which [the latter] would rather develop their own next generation protocol (which would probably never happen).
Gnutella2 has yet to become recognized by the major developers, who instead call it "MP", or "Mike's Protocol". Some developers contend that the situation is so severe, that the future of the Gnutella network, and subsequently their lively hood, is in peril.
Our goal with this article is not to take sides on this situation. Instead,
we're publishing Shareaza's interview today, and in the following days we'll
present the opposing viewpoint. Contribution to the opposition will include
interviews with BearShare, LimeWire and XoloX.
Slyck.com: The Gnutella network has been around for a considerable amount of time. Although potential was always there, it's never fully realized this potential. What in your opinion, are the problems facing Gnutella today?
Shareaza: I think the potential everyone sees in the Gnutella network stems from the hope that it will become the ideal peer to peer system - an open system truly created and supported by the community that uses it, rather than an opportunistic company out to make a few advertising bucks at the expense of its users. A system where people get together to share information however they want to, without the hassles that have become associated with P2P.
That's a powerful concept. The problem is that with all the talk about how great it's supposed to be, not much actually gets done. Gnutella is always getting "slightly better", but gradual improvement isn't going to cut it. To become the ideal peer to peer system everyone wants to see, Gnutella must have the courage to break free from the mistakes of the past and radically redesign itself. That's the only way it can become a first-class network.
Of course, doing that isn't easy. Making dramatic changes means taking risks. Shareaza and other forward thinking developers are taking those risks. Sadly it seems that some of the other well known members of the Gnutella community now have too much to lose.
Slyck.com: I'm assuming these problems are insurmountable. Would you agree with this? Please elaborate why Gnutella can't continue to exist in its current state.
Shareaza: I don't think the old Gnutella network is in imminent danger of collapsing. A number of claims have been made over the years that the network would soon reach a point at which it could no longer operate. As we've seen, this has not happened - and I don't think it will.
However the fact that it's not going to collapse does not make it a good network. In its current state the Gnutella network has a pretty bad reputation in the P2P community - hard to connect, can't find files, downloads don't work - much of which is justified. It simply does not perform as well as we would expect from a modern peer to peer system. Unless this fact changes, Gnutella won't be able to grow - in fact it's likely to continue its slow decline in the face of newer, better designed networks.
Could we continue patching up the original Gnutella network? Probably. Incremental improvements are better than nothing. But why keep tinkering with a fundamentally limited, out of date system that was never designed to grow when we can take the opportunity to start over and create something really powerful?
That's what Gnutella2 is about: a fresh start with a first-class network that actually works. It's a long overdue escape from the dark ages of old-Gnutella.
Slyck.com: How different is Gnutella2 from the current Gnutella protocol?
Shareaza: In short, it's completely different. Gnutella2 is an entirely new protocol and network, with very little in common with the old Gnutella. Instead of being limited by compatibility, Gnutella2 is designed from scratch and takes advantage of modern peer to peer technologies.
The original Gnutella protocol was designed for a single, closed P2P application with a limited set of features. It was a relatively simple peer to peer network that worked well in its time. But that's pretty much as far as it went. I don't think anybody expected it to grow into the much more diverse Gnutella we see today, with more sophisticated features and many different applications to choose from.
That growth was made possible by tacking new ideas onto the old Gnutella protocol, finding tricks for hiding extra information in the old structures. The new ideas were good, but at the core it's still the old limited Gnutella - it's like icing a stale cake.
Gnutella2 is a fresh start. It's designed from the ground up to meet the requirements of the diverse peer to peer environment we're in today, but also with a view to growth and expansion in the future. Unlike the original Gnutella network, Gnutella2 is designed to deliver on its promises.
The primary goal of the Gnutella2 network's object location / searching architecture is that even if only one single user has an object / file you are looking for, it should be possible to find them. That's what I think is important. Once you can make sure that the rarest possible file can be located, you can build a good network on that. If you search Gnutella2 for something too common like "the", you won't see every file with "the" in its name. But if you search for a specific item, and one or more people have it, you're sure to find it.
There are four major "levels" of Gnutella2:
Level One: A New Protocol Gnutella2 is based upon a flexible new protocol designed to support current and future P2P technologies directly. It provides a common, powerful communication structure and allows seamless introduction of new features and extension of current systems, even with multiple developers working independently.
Level Two: A New Data Transport Architecture Gnutella2 makes use of reliable, semi-reliable and unreliable transmission media, bringing it up to speed with other P2P networks. As part of the common framework, the best media can be selected for each application.
Level Three: A New Set of Base Services Taking full advantage of the first two levels, Gnutella2 delivers a scalable and efficient global search solution with all the usual enhancements like full metadata, exclusions, phrases, ranges, partial sources, etc. If at least one person has the file you're after, you can find it.
Level Four: A New Implementation Standard Gnutella2 defines a core set of features that must be supported to participate such as partial sharing and location, secure file integrity and automatic corruption repair, link compression and live queues as well as community features such as optional profiles, user and shared file browsing. This common functionality makes sure the network is not "let down" by users running inadequate software.
Slyck.com: What kind of support, if any, have you received from the Gnutella community? From leading developers such as LimeWire and BearShare?
Shareaza: I've received fantastic support from countless members of the Gnutella community and a number of other developers. I think there is a general agreement that Gnutella desperately needed an image boost that could be backed up by real, drastic improvement in the network, and that's exactly what Gnutella2 has delivered.
I believe that Gnutella2 is needed, and is needed now. Unfortunately, not all of the existing Gnutella developers share that view - perhaps most notably two of the larger commercial developers, LimeWire and BearShare.
Gnutella2 exists because I believe that it's well and truly time to abandon the limitations of the old system and create a powerful new network based on modern P2P technology. Gnutella should not be a prisoner of its past. Some other developers would rather we stick with the old system and make gradual, incremental improvements. For whatever reason, they don't want to risk investing in such a fundamental leap forward. I disagree. For Gnutella to be taken seriously it had to change radically, and Gnutella2 was the perfect solution. And it's available and working right now.
Some other developers disagree with the way Gnutella2 has been developed - they feel that it should not have the "Gnutella2" name without their approval. Once again I don't think they're seeing the bigger picture. The Gnutella idea needed a new image backed up by a network that can actually perform. That's what the community wants to see. Gnutella2 delivers exactly that, and just like its predecessor it remains an open platform which welcomes all developers and users who want to contribute. Many have already.
I think Gnutella2 is a very important step, helping "Gnutella" become the ideal peer to peer network we all want to see (and leaving behind its tainted past). I respect the opinions of all other developers, but I think if they don't see and embrace the value of Gnutella2 they are doing their users a disservice.
Shareaza: It's been an exciting time. The Gnutella2 network is performing very well, and shows every indication that it will continue to do so. It's so refreshing to connect to a network where everything actually works - connecting is painless, searching is reliable and downloads actually do something. Of course Shareaza can still connect to the older Gnutella1 network for compatibility, but G1 just can't compete with Gnutella2's performance.
The purpose of the beta testing period has been to make sure that everything works the way it was intended, and to allow some time for fine-tuning performance through real-world experience. It's been very valuable, and based on this research things have been tuned quite a bit. The current beta release is very stable, and performs at a first-class level.
In terms of the network size, the highest Gnutella2 node capacity I've seen so far is about 200,000 nodes and growing. Node capacity is a bit different to the number of unique users, as the network has a high level of built-in redundancy. That would equate to somewhere between 60,000 and 100,000 actual unique users online.
Slyck.com: Will [Shareaza] eventually phase out Gnutella support? Will the [Gnutella2] protocol remain open?
Shareaza: The magnitude of change in Gnutella2 means that it exists as a separate network along-side Gnutella1, providing a more modern and powerful alternative for those users who want it.
Technically Shareaza could phase out support for the old network; however I don't foresee that happening. It costs nothing to remain compatible, so it makes sense to do so as long as there are users there. On the other hand when others develop new Gnutella applications they may well choose to support Gnutella2 only, which would maximise performance for limited development time.
The Gnutella2 protocol is intended (and designed) to be an open, commonly extensible architecture for developing peer to peer applications. While it's true that closed or proprietary systems have some advantages in terms of locking out unwanted participants, I believe that there are other ways to address this problem, and that a truly successful network needs to be built openly. The web/HTTP is probably the best example of that. [Any third party developer is free to create their own client to connect to Gnutella2.]
Slyck.com: What future implementations of Gnutella2 can we expect? Can you share some thoughts on your vision of the future of P2P networking?
Shareaza: There are a number of things I think we will see in the P2P future:
- The era of one network or brand dominating the scene is over
- Competition between applications and networks will become more intense
- There will be a diversification in the applications of P2P technology
- It will get better
The increasing competition is both good and bad - on one hand it provides more choice, which is a good thing. Developers should not be able to hide behind a large network and force inferior software or invasive advertising on their users. On the other hand, fragmentation of the P2P world limits the size of any one network.
Diversification in the applications of peer to peer technology is going to be the interesting area. What started off as strictly a file sharing tool is slowly becoming something more. We've seen the beginnings of this with URI links such as magnet, bridging web documents and peer to peer content delivery. I'm convinced we'll see this go a lot further, leveraging ad-hoc networks in much more exciting ways.
Slyck.com: Some have suggested that Gnutella2 is merely a bastardised version of a LimeWire proposal known as GUESS. Is there any truth in that?
Shareaza: No truth at all. Gnutella2 and LimeWire's GUESS proposal are actually very different concepts with different goals. GUESS is a proposal for an incremental change in the original Gnutella network which would improve the way queries are delivered. The underlying network would remain the same old Gnutella, while some of the key behaviours are changed. Gnutella2 on the other hand is much broader - it's a whole new peer to peer network and communications architecture, redesigned from the ground up to embrace the latest P2P technologies and leave the inefficient baggage of "old Gnutella" behind.
There are of course some similarities between the Gnutella2 network and LimeWire's GUESS proposal, just as there are some similarities with other popular networks such as eDonkey2000 and FastTrack. However Gnutella2 was certainly not based upon the GUESS proposal in any way. In fact if anything I think we may see some aspects of Gnutella2 showing up in the GUESS proposal as it matures - I believe there's already some evidence of that. Even if that happens, I believe the Gnutella2 architecture is far superior to what has been proposed to date, in terms of its current potential performance, scalability, elegance and future expandability.
Slyck.com: Some developers are upset that you didn't discuss you concept with them first before creating G2...any significance to this?
I’ve been very keen to discuss the Gnutella2 network architecture with other P2P developers, and some constructive discussion has taken place already. Many good suggestions and criticisms were made, and the appropriate changes were incorporated for everyone’s benefit. I’m confident that this will continue, and as a consequence the network will just keep getting better.
As for getting some early beta testing done before opening things up for discussion, I stand by that completely. I think it’s much more productive to discuss things that you have some real-world experience with, rather than making armchair speculations about what might work.
If you can back up various design decisions with evidence that they actually work, you can discuss them in a much more informed manner and make better choices, ultimately producing a better product. And that’s what this should all be about – creating a better product.
That’s one motivation. The other motivation is shock value. By unleashing the Gnutella2 concept so quickly and backing it up with beta software to demonstrate how it might work (and more importantly, that it can be done), I’m effectively forcing change and improvement one way or another.
Understandably, some members of the Gnutella community who are quite content with the current situation may be upset by all of this. But I think that’s part of the problem – we don’t need people contentedly profiting from old Gnutella – we need radical change for the better, and we need it now. Gnutella2 is that change.
Editors note: As you will see with our follow-up article, the camps have envisioned the future of Gnutella in two very different ways. Ultimately, it will be up to the P2P community to decide which protocol will continue to exist in the file-sharing world.
Check out Shareaza here and Gnutella2 here.
Tom from Slyck.com is a regular contributer to MP3 Newswire. Tom's insights on other digital music issues can be read on his site and we encourage you to check it out.
The ultraslim iRiver SlimX MP3/CD player is available at Amazon
Other MP3 Stories:
Malicious MP3s? MS and WinAmp Flaws Found.
Jon Johansen - v - Hollywood
Broadcast Flag - to be, or not to be?
Review: Archos Jukebox Camera
Consumer attitudes and the record industry