Upload
karyn-sheppard
View
56
Download
0
Embed Size (px)
DESCRIPTION
VoD BitTorrent Framework. Background. BT is a very popular peer to peer protocol with many implementations: http://en.wikipedia.org/wiki/Comparison_of_BitTorrent_clients The first one was created in Python by Bram Cohen. Real world motivation. BitTorrent is common and Free - PowerPoint PPT Presentation
Citation preview
VoD BitTorrent Framework
Background
• BT is a very popular peer to peer protocol with many implementations:– http://en.wikipedia.org/wiki/
Comparison_of_BitTorrent_clients
• The first one was created in Python by Bram Cohen.
Real world motivation
• BitTorrent is common and Free
• Good chance there is a swarm for your movie
• VoD servers are expensive and complicated
• Its better to use P2P for VoD
Some definitions• Delay (D) = How much time I am allowed to hold before
starting to play M (Movie).
• Prefetch Time (P) = How much time before I watch byte B from M, it should arrive.
• Playback Rate (R) = How many bytes/second I watch
• B(t) – the index of the bit that must arrive by t.
• B(t) = R *(P+t-D)
Rules and Examples
• B(t) = R * (P+t-D)
• If R=5 KB/sec and D=6 seconds and P=2 seconds, than:– At t=6 seconds, I must have 10KB continuous from
the start of the movie.– At t=18 seconds I must have 70KB continuous from
the start of the movie.
What does one VOD client do
• Your client tries to get from the Bittorent clients pieces of data needed to play the video.
• Each piece needs to arrive before its deadline.
• If the piece does not arrive by the deadline you can download it “instantly” from the “server”.
• Each VOD client counts the data from server (DFS).
What is my goal
• Each byte taken from the server costs money, therefore you need to minimize DFS.
• Group - all the VOD clients that run concurrently.
• Goal - Minimize the DFS for the group.
How do I create my client• Download GROUP_VOD_CODE.tar.gz
from “BitTornado - Start with this Code” link in the course web page.– It is also in /users/courses/hanochworkshop/group_vod
• GROUP_VOD_CODE.tar.gz is a regular BT with StreamWatcher.py to monitor DFS. Any changed section is marked ###### GROUP VOD ######
• Modify it to have lower group DFS
• Do not change StreamWatched.py.
Standard BT parameters
• --saveas F
• --max_upload_rate
• F.torrent
My VOD client new parameters• delay – Wait before prefetch, in seconds.• prefetchT – How much time in advance the byte must
arrive. In percent of movie length.• rate – VOD rate in KB/sec• out_dir – Where to place the .csv files.• order – How many run when client spawned.• gap – How much to wait between spawns – unit is
pewfetchT.• group_size – How many concurrent clients.• alg – Name of the algorithm. Default is ORIG.• Verbose – Print and kill the whole swarm when
complete.
How do I test my idea
• Run the environment:./run_multiple.sh
~/GROUP_VOD_CODE/btdownloadheadless.py <GAP> ~/out
• ~/GROUP_VOD_CODE/btdownloadheadless.py: Your client.
• <GAP>: An integer. How many prefetchT between spawns of your clients.
• ~/out: .csv files
Execution example
run-all.sh
Tracker
BTBTBT
BTBT
BTBT
BT
BTBT
VOD BT
run_multiple.sh
VOD BTVOD
BTVOD
BT
How does it work?
• Run tracker
• Run the “world”
• Run your clients
How does it work?
• Both tracker and the “world” run on the same computer.
• Running a test:– Reserve time on pc-hanoch-w3/w4 in:
• http://bt-p2p-vod.wikispaces.com/ • Or from course www - "Scheduling computers for tests.“
– cd /users/courses/hanochworkshop/group_vod/vod_tests– ./run_multiple.sh ~/btdownloadheadless.py <GAP> ~/out
Output to screen==================================================================================
total time: 1 min 07 sec
share rating: 0.904 (4816.0 KB up / 5328.0 KB down)
saving: picture-100.bmp (27.5 MB)
percent done: 18.9
time left: 4 min 52 sec
download to: /home/pc-hanoch-w4/home/hillelav/name/picture.pc-hanoch-w4-4.bmp
download rate: 93.4 kB/s
upload rate: 80.6 kB/s
seed status: 2 seen now, plus 0.340 distributed copies
peer status: 28 seen now, 10.7% done at 1220.3 kB/s
--------------------------------StreamWatcher-------------------------------------
Csv stats: ~/out/1/statistics-order-4-gap-0.csv
DFS is: 8372224 bytes
DFS/Total is: 29 %
FullPieces: 63 / 220
DirtyPieces: 1 / 220
FaildHashChecks: 0
Prefetching 5 %
Playing point: 56 / 220 ( 25 %)
Only last VOD client prints
Output to .csv
• In ~/out you will find 4 directories: /1, /2, /3 and /4.
• In each directory, you have 4 files
• statistics-order-<1..4>-gap-<GAP>.csv
• Order is how many VOD clients ran before.
• <GAP> is the parameter <GAP>.
.csv filealg,dfs,p2p
ORIG,5,0
ORIG,10,2
ORIG,15,6
ORIG,19,11
ORIG,23,16
ORIG,28,20
ORIG,31,25
ORIG,34,31
ORIG,37,36
ORIG,39,41
.
.
.
Required plot 1 - total DFS and total P2P – GAP = 2
File percentage
GAP = 0
Required plot 2 - local DFS GAP = 2
GAP = 0
The main script - run_multiple.sh
• Location:– /users/courses/hanochworkshop/group_vod/vod_tests
• run_multiple.sh:– Runs run_all.sh four times.– Parameters:
• Your client – PATH/btdownloadheadless.py• Gap – wait Gap*PrefetchT between VOD spawns.• Csv location – where to store the .csv files.
run_all.sh defaults• Start
– VODClient:– Max_upload rate = 90– prefetchT = 5%– Rate = 90– Alg = ORIG– Verbose = TRUE # For last client in group
• Stop– Kill all python threads
• Kill – Kill all python threads and scripts.
To run client separately:
• PATH/btdownloadheadless.py
--saveas F
--delay D
--prefetchT P
--rate R
--out_dir O Can add the parameters
--order X in the IDE
--gap G
--group_size S
--alg ‘ALG'
--verbose
F.Torrent
Python
• Very common script language
• IDE: – PyDev – WINGWARE Python IDE – not free, but very good.
• References:– python.org :http://docs.python.org/reference/ – http://wiki.python.org/moin/BeginnersGuide/NonProgrammers– http://wiki.python.org/moin/BeginnersGuide/Programmers
• The book "Think like a Computer Scientist in Python" has a very good reputation:
Getting started• Run the environment
– ssh pc-hanoch-w3 or pc-hanoch-w4– cd /users/courses/hanochworkshop/group_vod/vod_tests– run_multiple.sh MyClient GAP output-directory– Note tracker prints a message for each GET.
• You can connect to the tracker, from the CS network, with a browser at:– http://pc-hanoch-wk:6970/
• Do ls -l /home/pc-hanoch-wk/home/hillelav/$USER to see downloaded files
Directory Structure
• Root – – high level application
• Root/BitTornado – – services
• Root/BitTornado/BT1 –– Actual work
Client Operation
• btdownloadheadless.py – First to run.– Performs initializations.
• get_response – parse the .torrent• Infohash – the ID of the .torrent• startEngine – Start connection with peers• startRerequester - Start connection with tracker
– Run rawserver.listen_forever from RawServer.py.
– listen_forever is: • Polling port • executing func() from a task queue.
Receiving messages
• From listen_forever loop it goes to:– BT1/Connecter.py/got_message
• CHOKE • UNCHOKE• INTERESTED• NOT_INTERESTED• BITFIELD• REQUEST• CANCEL• PIECE• HAVE
Influencing decisions• Can be done in one of the callbacks which
spring from Connecter.py / got_message:
• Downloader.py– HAVE->got_have->send_interested
• Tune for VoD?
BT tracker
• Send GET request in Rerequester.py / announce
• Receive GET data in thread in Rerequester.py / _rerequest
Policy
• Choker.py/_round_robin – decide who to choke / optimistically un choke / send_have to.
• Downloader.py / _request_more.
• PiecePicker.py / next - rarest piece, etc. Tune for VoD?
• got_unchoke->_request_more->next…
More about the code
• Start your project from “BitTornado - Start with this Code” link.
• Click “BitTornado - Legacy Code Guide” for basic explanations about the code.