Hello Everybody,

Recently I’ve been doing some thinking about how to generalize the packet abstraction. Till now we had a single (partial) P2P protocol implementation. I’d like that we now start work on the BitTorrent protocol, so that we develop and design file transfers for both protocols, plus keep in mind other protocols for file transfers for which we could add support later.

Now we already have some infrastructure for processing Gnutella packets. It’ll be great if we can reuse what we already have for BitTorrent. I can think of refactoring the current Gnutella classes by trying to generalize concepts (classes) and leave only the protocol specific problems to be implemented in subclasses. In order to break as little as possible, the mechanism of this refactoring should be to keep existing Gnutella classes intact for now. In a new package (Protocols::Generics) define a number of generic classes, which could (and should) borrow some code from Gnutella. Implement Protocols::BitTorrent on top / specializing from the generic concepts. When we get BitTorrent to work that way, we refactor Gnutella code to reuse the code from Protocols::Generics. We might need to evolve Protocols::Generics at this phase too but that’s fine.

It will be good if we can predict what we would need for protocols which we will implement in the future and take their requirements in account. However, we should be very careful not to premature design something, which we cannot directly verify that makes sense and what may eventually never be required. Having that in mind I read the specifications of BitTorrent and (partially) JXTA. I think it will be correct to say that in general there are two types of packets (or messages, depending of terminology). These are binary and text (possibly XML) packets/messages. Gnutella and BitTorrent use purely binary messages. JXTA uses XML messages but JXTA defines two formats – binary and XML. The format which is more suitable for the used transports can be used.

Binary messages typically have a header, payload and sometimes a trailer. Headers and trailers can be of fixed or varying size (depending on flags), payload length depends on the packet type and possibly the contents of the payload. Gnutella has a fixed sized header and no trailer. BitTorrent, although not defined exactly with these terms in the specs, has a header of length 4 bytes. The first four bytes give the payload length. The first byte of the payload specifies the message type. JXTA binary messages have a fixed header consisting of protocol marker and a version number. Total length of the payload can be determined only by reading the header for the case of Gnutella and BitTorrent, but not for JXTA binary messages. Reading text messages (MIME, HTTP, JXTA XML) is a completely different story.

We would like to generalize as much as possible without hurting flexibility. Doing proper generalization would allow us to be compatible with existing classes (like PacketDump, for example).

Based on the above summary of differences and similarities between the protocols, I would like to propose the following:

• BitTorrent::Packet and Gnutella::Packet be derived from Protocols::Generics::BinaryPacket.
• PacketReader, PacketWriter, PacketSession from Gnutella::PacketProcessing be copied to Protocols::Generics and modified to use the more abstract BinaryPacket class.
• Due to the differences in the way the packet length is extracted from binary packets, differences in packet structure, PacketReaderHelper can be provided by each protocol that wants it’s packets to be read using PackerReader.
• PacketReader and PacketWriter read/write the binary data to/from a Connection object. Reading from stream and datagram oriented connections would be different so PacketReader and PacketWriter should mask these differences.
• Protocols::Generics::BinaryPacket could later be derived form a more general class Protocols::Generics::Packet. Some of the code from BinaryPacket can be pulled up in Packet.
• At a later point, Http::Message (not existing yet) can be derived from Protocols::Generics::TextPacket (or better TextMessage).
• If that makes sense, the generic classes MessageReader and MessageWriter could be created to substitute Http::HeaderReader (-Writer) and Http::BodyReader (-Writer)

I’m still in doubt whether inheritance or template instantiation should be preferred. Why not a combination of both? PacketSession could be a template with the concrete packet type as a template argument. If we use inheritance, than classes like PacketDump directly work. Otherwise we will need a third class to do required transformation.

I know that the information I just presented here might not be detailed enough for more specific suggestions on your side. Still, any comments are welcome, at least comments about choosing the right names or about the high level view of the issues. I hope that as soon as I provide some preliminary implementation of BinaryPacket there will be more room for discussion.

Thanks,

Peter