Extension negotiation protocol
This proposal has been finalised, has support from both Azureus and Transmission developers, and has now been implemented in both clients.
- Azureus supports this from v18.104.22.168 b2.
- Transmission supports this from 0.82.
amc1 at users.sourceforge.net
When this was written in August 2007, there were three used known extension protocols at the moment by BitTorrent clients:
The last of these is not properly documented, either explicitly in a document, or implicitly in the source code being publicly available, so we will ignore it for this document.
Currently there are a few more, but those fall out of the scope of this document.
It is not technically possible for a client to properly support both the Azureus Messaging Protocol (AZMP) and the LibTorrent Extension Protocol (LTEP) at the same time - if both clients support the AZMP, then according to the specification, both have implicitly agreed to switch over to use AZMP, and then send the AZ_HANDSHAKE message. On the other hand, if both clients support LTEP, then the clients should still talk over the BitTorrent protocol, and send the LTEP handshake to each other.
At the time of writing, there is only one client which supports both protocols at the same time, Transmission, so this document is written in an attempt to define an agreed convention on how to handle the situation of when two clients have a choice between which extension protocol to use.
Defining this now is much more preferable than allowing clients to implement their own arbitrary behaviour. By defining this behaviour, it would allow clients to provide support both protocols at the same time without worrying about a lack of interoperability with other clients.
Note: Only one extension protocol should be used between two peers.
As mentioned before, at the time of writing, only Transmission supports both protocols at the same time. However, I have been examining the viability of possibility of offering basic LTEP support in Azureus (namely just exchanging the LTEP handshake without supporting any messages over it).
The proposal I have come up with attempts to allow Azureus the ability to implement as much or as little of the LibTorrent extension protocol without putting itself in a disadvantageous situation in the majority of cases. The goal is also to allow full compatibility with existing and new releases of Transmission without the clients being unable to communicate with each other.
Currently, the only client which defines any behaviour regarding choosing which protocol to use is Transmission (as of version 0.70), where if both protocols are available, it defaults to using LTEP. Therefore, we use this our default situation - any client by default which indicates support for both clients should use LTEP by default. However, the behaviour for clients which are aware of this negotiation scheme can choose a different behaviour.
This proposal suggests using two reserved bits in the BitTorrent handshake - bits 47 and 48 (as currently described here).
If a client indicates no support for either extension type, then the value of these bits are ignored. If a client indicates support for AZMP, but not LTEP, then the value of these bits are ignored, and AZMP is used (if both clients indicate support for it). If a client indicates support for LTEP, but not AZMP, then the value of these bits are ignored, and LTEP is used (if both clients indicate support for it).
If both clients indicate support for both protocols, then the value of these bits will determine what extension protocol is used.
The current values are (in binary, bit 47 and bit 48):
- 00 - Either means the client is not aware of the negotiation protocol, or that they are going to force the other client to use LTEP.
- 01 - Means that the client prefers LTEP, and will only talk AZMP if the other client indicates that they are going to force that protocol to be used.
- 10 - Means that the client prefers AZMP, and will talk AZMP if the other client indicates that they either prefer to use AZMP, or that they are forcing AZMP to be used.
- 11 - Indicates that the client is going to try and force AZMP to be used, and it will be used unless the client indicates that they are forcing LTEP (or that they are not aware of the negotation protocol).
I'll use the following terminology to describe the messages:
- 00 - Force LTEP
- 01 - Prefer LTEP
- 10 - Prefer AZMP
- 11 - Force AZMP
Although I use the term "prefer" and "force", they are not equivalent when used. If one client tries to force AZMP, but the other LTEP, then LTEP will be used. If one client indicates they prefer AZMP, but the other client indicates they prefer LTEP, then LTEP will be used.
Another way of puttiing it is (with the higher rule given greater preference):
- FORCE overrides PREFER
- LTEP overrides AZMP
I'm going to use the following example - note that I make use of possible future versions of clients, it does not mean that the clients will behave as described, it is just an example! It also up to client developers to decide what bits to set for the negotiation protocol - the examples I give have some explanation about why a client might choose to set the bits that it does, but in the real world, clients can use whatever logic they wish.
Let's have four clients:
- Transmission 0.7 - supports LTEP and AZMP, doesn't understand the negation protocol.
- Transmission 0.9 - supports LTEP and AZMP, supports the negation protocol.
- Azureus 3.1 - supports AZMP and supports LTEP, but only exchanges the handshake, contains no extension support.
- Azureus 3.2 - supports AZMP and LTEP, supports some messages over LTEP, but not all messages - it uses AZMP for that.
The four clients will have the following bits set:
- Transmission 0.7 - 00 (Force LTEP). It doesn't understand the negation protocol, so falls back to the default behaviour.
- Transmission 0.9 - 01 (Prefer LTEP). It supports the negation protocol, and prefers to send messages over LTEP if possible.
- Azureus 3.1 - 11 (Force AZMP). It only has basic support for the LTEP handshake, so would try to force AZMP to offer all the extensions it has available.
- Azureus 3.2 - 10 (Prefer AZMP). It has some level of support for LTEP, including extensions, so it doesn't mind using LTEP.
|Client 1||Client 2||Protocol||Notes|
|TR07||AZ31||LTEP||This is an unwanted situation, because AZ31 doesn't support any extensions over LTEP, even though TR07 would support some extensions over AZMP.
This is why we would want clients to use something like Prefer LTEP instead of Force LTEP.
|TR09||AZ31||AZMP||This is a situation which benefits from this proposal - AZ31 is using Force AZMP to indicate that falling back to LTEP would not be beneficial to the peer.
TR09 allows AZ31 to indicate what is best.
|TR09||AZ32||LTEP||Although AZ32 might prefer AZMP, TR09 won't gain or lose extra functonality by using LTEP or AZMP, so either protocol can be picked here.|
|AZ32||AZ32||AZMP||Although both clients could use LTEP for some of the extensions, they default to using AZMP because more extensions are supported via that mechanism.|
There are two main disadvantages to my proposal.
The first is that currently, Transmission 0.7 talks to current versions of Azureus using AZMP. When (if) Azureus introduces any kind of LTEP support in the future, Transmission 0.7 will use LTEP, preventing the two clients performing peer exchange. To combat this, my goal is to introduce this negotiation protocol behaviour to be implemented sooner rather than later (especially in Transmission). That way, if Azureus was to introduce LTEP support, the majority of Transmission clients would be aware of this negotiation protocol, and behave accordingly.
Secondly, it doesn't provide much scope for indicating why a client wants to use a given protocol. All it does is give a simple indicator of what protocol is preferred and how strongly they prefer to use it. An alternative might be to agree to use one extension, to switch over to that extension, to then have some exchange indicating what protocols support what messages, and then agree what protocol to use.
I've discounted that idea because there is no guarantee that this process will happen quickly enough - there will be too many messages exchanged before agreeing what to do. Both AZMP and LTEP use the presence of the reserved bits in the BT handshake as indicators that you agree to use the protocol - switching protocols after more messages other than the BT handshake have been exchanged may be difficult to implement.
For clients that wish to support both protocols, but do not want to implement the behaviour here (though it is recommended), there are two potential alternatives:
- Always default to using LTEP - this is the agreed behaviour by default anyway if neither of the reserved bits are set.
- Let the receiving connection decide - the sender connection will indicate it supports both protocols (and may or may not some of the reserved bits being set), and the receiving connection could wait until the handshake has been received and then send its handshake out - only announcing support for one particular extension. If the receiving connection does not send a handshake which decides the protocol to use, then both clients should default to using LTEP (this would be the behaviour when connecting to an existing Transmission 0.7 client).
- joshe for discussing this proposal with me, as well implementing the changes on the Transmission side.
- hydri for reading through and discussing alternatives; and
- parg for reading through it and giving it a half-hearted thumbs up. ;)