The Bittorrent Protocol

The Bittorrent Protocol

The Bittorrent ProtocolA Technical LookPresented by Cody Sand

What is Bittorrent?Peer-to-Peer (P2P) File Transfer ProtocolCentralized Discovery, Decentralized DistributionCurrently uses TCP transfer.Designed to facilitate file transfers among multiple peers across unreliable networks. Tit-for-Tat protocolAny downloaders MUST upload, discourages leeching.UsesContent Distribution (Downloading)Music, Movies, SoftwareValve/Blizzard DownloadsStreaming video (beta)

HistoryDesigned by Bram Cohen in 2001. Assisted in Development of Valves Steam platform, before focusing on Bittorrent fulltime.First implementation released in July 2001.Accounted for roughly 33% of all internet traffic today.Nearly 100,000,000 users worldwide.

How it worksPeer distributing the file breaks it into small pieces, typically 64k 1MB in size.Pieces over 512k tend to decrease protocol efficiencyPeer creates a checksum for each piece and records it in a file (a .torrent file)When another peer receives a piece, it is checked against this file for integrity purposes.This model allows uploading/downloading of one or more files by many peers.

Why Bittorrent?For large, popular files, more customers require more bandwidth.Bandwidth costs go up.Bittorrents solution:Unload some distribution to usersAllows for savings in bandwidth and costs.

StructureOrdinary Web ServerA static metainfo file (.torrent).A bittorrent trackerAn original downloader or the seedOriginal content providerWeb BrowserConsumer (Downloader) with client software.Web Server / Bittorrent TrackerWeb ServerHosts the tracker, which is the most essential piece.Trackers implemented to run with all major servers.TrackerAssists communication between peers.Responds to HTTP Get Requests.Used to track peers, download statistics.Tracker GET RequestsStandard HTTP Get RequestParametersinfo_hash 20byte SHA-1 hash code to ID torrent filepeer_id 20 character string, identifies user/client.ip IP address of peer.port the port the client is listening on.uploaded total amount uploaded so far.downloaded total amount downloaded so farleft number of bytes left to downloadevent state of torrent (ie. started, completed, stopped)numwant Optional key stating the number of desired peers. default is 50.

Info_hash calculated using info dictionary of .torrentPeer_id unique id that identifies user and bittorrent client.8Example GET Requesthttp://some.tracker.com:999/announce?info_hash=12345678901234567890&peer_id=UTABCXYZ&ip=174.250.2.22 &port=6888&downloaded=1234 &left=98765&event=stopped Tracker ResponseError Message (Failure)3-digit error codehuman-readable error messageList of peers (Success)Dictionary formatinterval time client should wait before rerequestingpeers list of dictionaries coresponding to peersid the peer_id of remote peerip IP address of remote peerport port number

3-digit error think 404.10Torrent file (.torrent)Static metainfo in BEncoded format.Provide client with informationTracker Location (URL)File(s) InformationStorage information

BEncodingFormat for loosely structured data4 typesByte stringsIntegersListsDictionariesBEncoded Byte StringA sequence of bytes, not necessarily charactersFormat:Example5:hello7:seminarBEncoded IntegerFormatieExamplei42eBEncoded ListFormatleExamplel5:helloi42e7:seminareList:5:hell0I42e7:seminarBEncoded DictionaryAssociative arrayFormatde alternating key/valueExampled5:hello5:class4:falli2009eehello => classfall =>2009.torrent FilesStructureBEncoded Dictionaryannounce string value containing tracker URL.info points to dictionary contain info about filesOptional fieldsannounce-list a list of alternate trackerscomment any comment the author wants to includecreatedby author of the .torrent filecreationdate the date the .torrent was created.OthersCan put ANY field into .torrent. Ignored if not supported.17Single File Torrentsinfo dictionary fieldslength size of the file, in bytesname a string containing the filenamepiece length number of bytes in each piecepieces string value containing concatenated 20-byte SHA-1 hashes for each piece of file.Multiple File Torrentsinfo dictionary fieldsfiles dictionary containinglength size of the file, in bytespath path to the file, including filenamename name of top-most directorypiece length number of bytes in each piecepieces string value containing concatenated 20-byte SHA-1 hashes for each piece of file. For verification.Peer MessagingHandshakingPeer connects to remote peer, citing specific filePeer states(Un)Choked Is the remote peer allowed to send data?(Un)Interested Is the peer interested in receiving data?Data Transfer takes place when one side is interested, and the other is not choking.Peer-Wire MessagesUsed to communicate between peersPeer Wire MessagingChokeInform the other peer they are being chokedUnchokeInform the other peer they are no longer being chokedInterestedInform remote peer that data is desiredUninterestedInform remote peer that data transfer is not desiredPeer Wire MessagingHaveAnnounce that peer has a piece of the torrentRemote peer may then request that pieceBitfieldSend a bitfield representing pieces peer needs/has.RequestOfficial request for a specific piece(s).PieceTransfer of a piece(s)CancelSent if peer requested data, but is no longer interested.Process

Tracker goes onlineProcess

Content Provider creates .torrent file using bittorrent client software

Process

Client uploads the .torrent file to the server.Process

Client re-downloads the .torrent file from the serverand opens it using a bittorrent client.Tracker indexes the torrent

Index029317AHBS: first.torrent26Process

Client software sends HTTP Get request, sending informationto server; PeerID, and lets tracker know we have full file

Index029317AHBS: first.torrent27Process

Client software sends HTTP Get request, sending informationto server; PeerID, and lets tracker know we have full file

Index029317AHBS: first.torrentAdds seed to peer listPeers123.234.12.3428Process

Index029317AHBS: first.torrentPeers123.234.12.34

More peers download the .torrent file29Process

Index029317AHBS: first.torrentPeers123.234.12.34Peers open the .torrent file, and each send an HTTP Get request identifying that the want to start downloading the file.

30Process

Index029317AHBS: first.torrentPeers123.234.12.34111.222.33.44222.124.232.1125.55.66.42122.45.77.99

Tracker adds peers to list31Process


Tracker returns the list of peers to the new peers.32Process


Peers begin forming direct connections with other peers, forming a swarm.33Process


Peers begin exchanging messages((choked))((unchoked))((Interested))((Uninterested))34Process


Peers begin exchanging messages((have))((bitfield))((Piece))((Request))35Peer to Peer Process

Handshake((interested))((Unchoked))((Bitfield)) / ((Have))((Request))((Piece)) State: Choked/UninterestedState: Choked/InterestedState: Unchoked/UninterestedData Transfer BeginsOptionalProcess

Index029317AHBS: first.torrentPeers123.234.12.34111.222.33.44222.124.232.1125.55.66.42122.45.77.9955.121.33.6575.45.65.95111.85.243.11

As peers continue to connect to tracker, more peers are added to the peer list.37Process


New peers receive up-to-date peer lists.

Existing peers periodically report to tracker, also receiving new peer list.

38Process


Peers continue exchanging messages and data until all peers receive the entire file.

As pieces are received, the program calculates the SHA-1 hash for each piece and checks it against .torrent file. Any piece with incorrect checksum must be discarded and redownloaded.

39Process


As peers disconnect, they send another HTTP request, stating that theyre no longer sending.40Process

Index029317AHBS: first.torrentPeers123.234.12.34111.222.33.44222.124.232.1125.55.66.42122.45.77.9955.121.33.6575.45.65.95

As peers disconnect, they send another HTTP request, stating that theyre no longer sending.Tracker removes peer41Process

Index029317AHBS: first.torrentPeers123.234.12.34111.222.33.44222.124.232.1125.55.66.42

As peers disconnect, they send another HTTP request, stating that theyre no longer sending.42Future of BittorrentBittorrent continues to increase its presence online.Bram Cohens company, Bittorrent, Inc. is now in control of the protocol.UDP implementationIncreased speeds without TCPs congestion controlStreaming VideoPiece selection algorithms to increase speed and ensure sequential delivery.The End.Thank You.

Documents

The Bittorrent Protocol