Posted: 5/23/2005 12:51 PM
Post subject: Request: btfriend improvements
Quote
torbjrn
Joined: 3/2/2005
Posts: 18
Location: Linköping, Sweden
I finally got around to installing btfriend in order to seed the movies, although I limited the upload to 900 kB/s. (Perhaps one day I'll setup real traffic shaping...) However, this causes one inconvenience: hard disk accesses. To reduce the bursts, I want to disable the hash checking when seeds are juggled. (The bittorrent client is running as an unprivileged user without write access anyway.) This might not work when new files are made available, because those are automatically downloaded and might be re-seeded before completion. Solution #1: btfriend doesn't try to seed non-existing files. Solution #2: modify bittorrent to only check hashes for files with write permissions. Solution #3: btfriend puts the torrent file in a different folder (and preferably sends an e-mail when doing so). I could attempt this myself, but not in another two weeks due to exams.
Posted: 5/23/2005 12:56 PM
Post subject: Re: Request: btfriend improvements
Quote
Bisqwit
Editor, Active player (296)
Joined: 3/8/2004
Posts: 7469
Location: Arzareth
Yeah, Bittorrent is quite tough at harddisk unless you have enough RAM to cache all of your shared AVI files. Solution #1 seems like an easily implementable option (but I'm not inspired to work at it any time soon). As for solution #2, did you know there's this option?
btdownloadcurses wrote:
--check_hashes <arg> . . . . . . . whether to check hashes on disk (defaults to 1)
Posted: 5/23/2005 1:05 PM
Post subject: Re: Request: btfriend improvements
Quote
torbjrn
Joined: 3/2/2005
Posts: 18
Location: Linköping, Sweden
Bisqwit wrote:
As for solution #2, did you know there's this option?
btdownloadcurses wrote:
--check_hashes <arg> . . . . . . . whether to check hashes on disk (defaults to 1)
Yes, it is a requirement to reduce the bursts. I can't use it though if I don't want my client to spread invalid data from incomplete downloads. Files with write permissions are incomplete. Write protected files are symlinks to complete ones.
Posted: 5/23/2005 4:49 PM
Quote
SXL
Joined: 2/7/2005
Posts: 571
I tried to look at the code, but even though it's easy not to check hash of files, I couldnt find a way to identify complete files from the others. more than that, when I first ran the client, it began a lot of differents files (since it tried to help people looking for them, thanks to priorities and the score system) but never finished most of them. the client just detected that the file was requested, but never checked that I had it already or not, hence filling my hard drive with gigs of unusable incomplete files. I tried to increase (locally) the priorities of those files, but met the same problem : identify from within the client complete/incomplete files. but without having to hash them... I thought of an idea : make a more user-friendly client, that would keep a list of the files => managing a collection. create a nesvideos client that would make 3 categories : complete files, wanted incomplete files, and unwanted files (not everyone would want to download the whole collection). using such a client could encourage people to seed more files ? unfortunately, my programming skills are far too low to begin such a project, so I gave up the idea... and the use of the btfriendly client :( but, maybe someone would want to create one, or might think up a better idea... //edit : sorry for the poor english :)
I never sleep, 'cause sleep is the cousin of death - NAS
Posted: 5/29/2005 8:32 PM
Post subject: Re: Request: btfriend improvements
Quote
torbjrn
Joined: 3/2/2005
Posts: 18
Location: Linköping, Sweden
torbjrn wrote:
Solution #1: btfriend doesn't try to seed non-existing files.
Well, I just decided I had the time for this. It was nice to automatically download new files, so I will probably do solution #3 later. http://www-und.ida.liu.se/~torso690/btfriend.py-diff
Posted: 6/23/2005 11:59 AM
Quote
Bisqwit
Editor, Active player (296)
Joined: 3/8/2004
Posts: 7469
Location: Arzareth
If your btfriend.py program has recently started spitting error messages that look like this: 
Traceback (most recent call last):
  File "./btfriend.py", line 265, in ?
    CheckSharing()
  File "./btfriend.py", line 256, in CheckSharing
    Share(file, key, prio)
  File "./btfriend.py", line 224, in Share
    DownloadTorrent(key, file)
  File "./btfriend.py", line 162, in DownloadTorrent
    response = urlopen(url).read()
  File "/usr/lib/python2.3/urllib2.py", line 129, in urlopen
    return _opener.open(url, data)
  File "/usr/lib/python2.3/urllib2.py", line 326, in open
    '_open', req)
  File "/usr/lib/python2.3/urllib2.py", line 306, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.3/site-packages/BitTorrent/zurllib.py", line 33, in http_open
    fp = HTTPHandler.http_open(self,req)
  File "/usr/lib/python2.3/urllib2.py", line 901, in http_open
    return self.do_open(httplib.HTTP, req)
  File "/usr/lib/python2.3/urllib2.py", line 895, in do_open
    return self.parent.error('http', req, fp, code, msg, hdrs)
  File "/usr/lib/python2.3/urllib2.py", line 352, in error
    return self._call_chain(*args)
  File "/usr/lib/python2.3/urllib2.py", line 306, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.3/urllib2.py", line 412, in http_error_default
    raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
urllib2.HTTPError: HTTP Error -1:
It is because the tracker nowadays requires that when downloading torrent files directly from the tracker, the info hash must be hex-coded. Therefore, a new version of btfriend.py has been released, addressing this problem. http://bisqwit.iki.fi/src/btfriend.py <- download here. EDIT: Nevermind that. I guess Debian folks had broke it. I downloaded v4 of Bittorrent and it works fine.
Posted: 6/23/2005 2:18 PM
Quote
Nach
Emulator Coder
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Latest Debian packages of Bit Torrent haven't been working for me. However getting v4 from official site, or installing Debian packages for Bit Tornade works good.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Posted: 8/16/2005 10:06 AM
Quote
torbjrn
Joined: 3/2/2005
Posts: 18
Location: Linköping, Sweden
I left btfriend running while I was on vacation, and this is what I found when I came back: 
-------------------------------------------------
Rechecking sharing  Mon Aug  8 07:57:26 2005
-------------------------------------------------
Traceback (most recent call last):
  File "/usr/local/bin/btfriend.py", line 265, in ?
    CheckSharing()
  File "/usr/local/bin/btfriend.py", line 234, in CheckSharing
    response = LoadBittorrentTrackerData()
  File "/usr/local/bin/btfriend.py", line 63, in LoadBittorrentTrackerData
    return bdecode(urlopen(TRACKER + '/scrape').read())
  File "/usr/lib/python2.4/urllib2.py", line 130, in urlopen
    return _opener.open(url, data)
  File "/usr/lib/python2.4/urllib2.py", line 358, in open
    response = self._open(req, data)
  File "/usr/lib/python2.4/urllib2.py", line 376, in _open
    '_open', req)
  File "/usr/lib/python2.4/urllib2.py", line 337, in _call_chain
    result = func(*args)
  File "/usr/lib/python2.4/site-packages/BitTorrent/zurllib.py", line 33, in http_open
    fp = HTTPHandler.http_open(self,req)
  File "/usr/lib/python2.4/urllib2.py", line 1021, in http_open
    return self.do_open(httplib.HTTPConnection, req)
  File "/usr/lib/python2.4/urllib2.py", line 996, in do_open
    raise URLError(err)
urllib2.URLError: <urlopen error (110, 'Connection timed out')>
Perhaps it should handle errors in a better way (i.e. not crash).
Posted: 8/16/2005 10:07 AM
Quote
Bisqwit
Editor, Active player (296)
Joined: 3/8/2004
Posts: 7469
Location: Arzareth
I usually run btfriend like this: while :;do ./btfriend.py;sleep 2;done I.e., in a loop, in case it crashes, and with a 2-second restart delay in case the error is a permanent one that would eat the CPU to death if it wasn't that delay.