What are the 4 other people doing then, if they aren't seeding, and they're not wanting the data either?
If what Bag of Magic Food said is right, that btfriend.py will download any torrent that needs seeding, then it would be those other 4 people who are running btfriend.py.
Since the data.xml file is XML, wget can't operate on it directly. I put together a simple sed command to extract the torrent filenames, but in a test on a few of them I found one of them was obsolete (redirect to OldMovies.html). The XML file does have obsoleted data at the top but I don't really have the time/interest/knowledge to write an XML parser that intelligently chooses which torrents to download. Downloading everything would waste resources, which I would guess you wouldn't want.
It will take 1000 seconds for all of the 5 peers to get a full copy of the file.
Ok, I think we agree. In my case, I'm talking about a scenario where there's just one guy who wants the file, and there's only one seeder providing original data. Just like you say, even though there's 4 other people in the swarm, that one guy who wants the data is still going to get it in 1000 seconds.
I can see the advantage to having extra ("helper") peers when you have more than one person wanting the file. They would provide redundancy and, save some drastic download/upload speed imbalances, would therefore help the transfer along.
As for wget, all I can say is that when I tried without the custom user-agent string I got error 404 (which is usually file not found) with the comment "Mirroring forbidden." Altering the user-agent string cleared that error.
Anyway I just came to volunteer some bandwidth, not cause a fuss. I'll respect your choice not to have mirrors on the seeds and delete my torrent archive.
Edit: I think we posted/edited at the same time. I'll leave this alone for tonight but FYI, I just chose that user-agent ("Firefox") so it would show up in your logs as an obvious sore-thumb and you could filter it out. Any user agent string is fine with me.
Ideally those peers will be sharing with one another and the seeder will never need to repeat any information no matter how many clients are connected.
Right but there's no real advantage to having those peers, is there? If we assume that either the seeder or downloader is going at max speed (or, likely, both) then those peers just change the route of the data, not the speed that its delivered.
I came up with a way to solve my own question, and thankfully no one on the server side has to do anything. I was able to use wget to download the whole set of torrents, which I can now start downloading/seeding.
Here's the command I used:
The user-agent is there because the site has protections against mirroring which block it if a user-agent of wget is reported. I think those protections are there to prevent rival sites from downloading everything, so I felt OK bypassing them. The --wait 4 is crucial: it makes each download four seconds apart, instead of one after the next, so I don't hammer the server. This spreads the download out over about 20 minutes. The movies.cgi page is a page listing every torrent on the site.
If any admins don't like what I've done feel free to delete this post and send a privmsg. Otherwise I'll probably update my torrent set every couple weeks and seed away.
(For Windows users, wget is available through Cygwin. For mac users, Fink. For Linux, it's usually built in.)
The problem with making one zip for the whole site is that it would have to be altered every time a movie is obsoleted or published.
True it would have to be updated periodically, but not constantly. Plus, it's not too hard to write a cron job/Scheduled Task to zip files.
Bag of Magic Food wrote:
I thought BTFriend went for any file that was needed.
Even if it is, you still have the problem that volunteer seeders need to get the content before the data is needed by peers. E.g. if someone's trying to download a file, and there's only one person who has the data to seed, a bunch of other people joining that torrent in order to "help" will actually just distract the one seeder, and the end result for the peer will be the same. Right?
I'd like to help seed torrents that need it, but I don't want to have to download them all one by one. Perhaps this has been discussed before, but has anyone thought of creating a big ZIP file with all the torrents? I certainly have the drive space spare, it's just a matter of getting the files onto my system.
My understanding of the btfriend.py script is that it only juggles torrents you already have, just like Azureus would. Maybe that's incorrect?