Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Marvel Super Heroes Vs. Street Fighter 0031, AKA Secret Character Theme.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
RAR generally performs best in comparison to other archives when compressing a media file.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
sixofour wrote:
Moon landing was fake.
Well yeah of course, since we all know the moon doesn't actually exist, and you can't go somewhere non existent.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
ShinyDoofy wrote:
Nach wrote:
That's why we now need to use self extracting 7-Zip files. So Windows users can extract them without needing any programs, and be horribly mortified and scared by the .exe, and UNIX users can just use 7za to extract self extracting 7-Zip exe files.
Thank God I can spot irony every once in a while. A little part (possibly even a big one) died inside of me when reading this. While I already can't stand having win32 emulators, this is just too much sucking up to M$.
Can you spot sarcasm too?
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
arukAdo wrote:
So for bin based movies you gonna put extra size, just for the self decompression system?
7z even with the self extracting code can very likely end up smaller than a comparative zip file.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Bisqwit wrote:
Anyone have idea why it's missing?
I read a report a while back how Debian for CGI security purposes removed SCRIPT_URL and some other environment variables from PHP.
This should only be an issue when running PHP in CGI mode though.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
That's why we now need to use self extracting 7-Zip files. So Windows users can extract them without needing any programs, and be horribly mortified and scared by the .exe, and UNIX users can just use 7za to extract self extracting 7-Zip exe files.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
arflech wrote:
so we don't need to package several files together (which is the sole purpose of TAR).
I've seen people distribute single files (UNIX executables) in TAR format, just to preserve the execute bit.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Bisqwit wrote:
Plain LZMA compression on the other hand, is fine. Maybe we should do .tar.lzma? ;)
I wouldn't think that.
They've been discussing changing the LZMA format bundled with 7z.
The LZMA format produced by these utilities is also different than the one currently produced by 7z (as well as supposedly being superior).
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Bisqwit wrote:
Nach wrote:
Bisqwit wrote:
Nach wrote:
Bisqwit, on Debian, install p7zip-full, and then use the "7za" program.
Thing is, I also use Gentoo and Fedora.
For your servers???
For computers. I access the tasvideos site on computers, you know.
Yes, but the site needing to decompress runs on a server I imagine.
And personally, all my computers these days are servers ;)
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Bisqwit wrote:
Nach wrote:
Bisqwit, on Debian, install p7zip-full, and then use the "7za" program.
Thing is, I also use Gentoo and Fedora.
For your servers???
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Bisqwit, on Debian, install p7zip-full, and then use the "7za" program.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
adelikat wrote:
I can't think you guys enough.
Yeah, I don't think about them either, too painful.
Current status is that we copied over the main code, and got a duplicate site up and running. Bisqwit has very kindly been hard and work improving things to be more compatible than the type of dedicated servers he's been running it on till now.
However, as it stands, parts of the sites can't load within the current server limitations, we hope to cut it down more. Once we have everything working, we hope to perform the transfer. No ETA yet however.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
OmnipotentEntity wrote:
Dreamhost
"enough" (df says I have 2.7TB)
150 MB (but that can be changed up)
"enough" (read Dreamhost ToS)
debian
don't know about that.
and I don't think so.
Yeah, I don't believe Dreamhost would cut it based on what I read from their website.
Bisqwit wrote:
eEccelerator dependency --definitely the largest hurdle for installing a PHP version that can run the site -- has been removed. Manual compilation of stuff no longer required at least on Debian squeeze/sid.
Additionally, now it also supports XCache.
Excellent :D
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
OmnipotentEntity wrote:
Nach wrote:
adelikat wrote:
OmnipotentEntity wrote:
Dedicated servers from a reputable renter are generally between $100-$200 for a low-mid range unit. per month.
Well, with Nach's ingenuity, trimming some fat, & some deals, we will get something cheaper than this range.
Just to clarify, it's not a "dedicated" server. It's a really powerful server shared between 15 or 20 other users, but guaranteed a nice chunk.
Hopefully, when trimming some fat, we'll get TASVideos working with less. Just hope we have room to grow.
If that's the case then I already have a server rented that you can use. I've already paid for a year and I'm not doing anything with it. :/
Can we have some details?
Who is the hosting company?
How much HD space?
How much RAM?
How much bandwidth?
Which OS? (name the distro)
Can we run a torrent tracker on it?
Root login?
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Bisqwit wrote:
Nach wrote:
Bleh, evil magic numbers, don't bother defining len, and use sizeof(str) instead.
I did that because this is C, not C++ -- "const int x = <expression>;" is not a constant in C as it is in C++, and I opted to do this instead of using #define. Also, I'm not fond of (sizeof(x)-1) -- the nul terminator there is a trap.
If you must declare len, at the very least use sizeof(str), as opposed to magic numbers. What you did is pure evil, and should be avoided. You should make your code least prone to errors when something changes. If he decided to change the text to "Hi, my name is Sally!", he'd have to completely redo the len with your definition.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Bisqwit wrote:
Bisqwit wrote:
not the solution handed on a silver platter…
But I can go with the old-fashioned [spоiler] tag.
#include <stdio.h>
int main()
{
int a=0, len=5+2+5+1;
static const char str[] = "Hello, world ";
for(;; a=(a+1)%len) printf("%.*s%.*s\n", len-a%len, str+a%len, a%len, str);
}
Bleh, evil magic numbers, don't bother defining len, and use sizeof(str) instead.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
adelikat wrote:
OmnipotentEntity wrote:
Dedicated servers from a reputable renter are generally between $100-$200 for a low-mid range unit. per month.
Well, with Nach's ingenuity, trimming some fat, & some deals, we will get something cheaper than this range.
Just to clarify, it's not a "dedicated" server. It's a really powerful server shared between 15 or 20 other users, but guaranteed a nice chunk.
Hopefully, when trimming some fat, we'll get TASVideos working with less. Just hope we have room to grow.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Just to clarify, those willing to donate? What methods are you willing to donate by?
Paypal?
Credit Card?
Bank Transfer?
Western Union?
Mailing a check?
Something else?
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
You wouldn't have to upload them manually. It's quite easy to create an FTP script or copy a whole directory.
rsync really only helps when you plan on constantly syncing, which isn't really relevant.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
ShinyDoofy wrote:
While I like the idea, wouldn't it be easier to have the encodes on said second server and have btfriend kick in?
No, as that means the storage server needs to run a constant program, and it can't take advantage of a CDN.
ShinyDoofy wrote:
I think most users would simply grab the encode from http instead of getting it via torrent
You really think someone is going to download the torrent, then rip it apart just to get the URL seed?
ShinyDoofy wrote:
(speed issues, poor seeder count, possibly clogging your uplink and so on). At least that's what I myself would think.
That's what I specifically said to use a CDN.
ShinyDoofy wrote:
As archive.org didn't (doesn't?) like Zer0 uploading mkv files, I'd already most (all?) my encodes on my current server for rsync'ing it over.
What archive.org likes or doesn't like is irrelevant to us.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
In addition to the main site, it could be a useful idea to get a cheaper host with a decent capacity (100GB?), combine it with a CDN, and then we could offer URL seeds within torrents. It could also be an easy way to provide a link to those without bit torrent clients.
Essentially, instead of creating and uploading a .torrent, and the encoder running a torrent client, the encoder via FTP can upload his encode to his account on a special server, and then a torrent link appears on the main site, which operates much faster than we've had it operate previously. It would also avoid the issue of a user running into a video which lacked seeders.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
ShinyDoofy wrote:
For €40 a month, one could get an AMD Athlon LE 1660 with 2.8GHz with 2 Gigs of RAM and a 250GB HDD.
Do you mean buy? Or rent at a server farm / management facility somewhere?
Does that include hosting bills for Internet access?
ShinyDoofy wrote:
As said before, I'd love to keep this site alive, but I'm not going to pay it all by myself. Sadly, though, the EUR/USD exchange rate is pretty bad (for USD people), so I'd understand if people didn't want to join or not chip as much. adelikat already said he'd like to do so, but I don't know much actually. Nach, what's your thought on that?
Well, if we can get a certain amount of users willing to donate a consistent fee, then we should have no problem.
Some of these sites also offer discount if you order 1 or 2 years in advance.
For example, we could get the second option I posted for 2 years for ~$850 if we were willing to pay in advance.
Question is, can we raise that amount of money? Can we re-raise it over the course of every 2 years?
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
arukAdo wrote:
When i asked to bisqwit on IRC about if the site could work on a non-dedicated server he sayd yes, so apparently its possible
Why paying 10$ per month for just some RAM, when for 4$ per month you get a unlimited hosting, bandwidth + disk space...
http://www.hostmonster.com/
You need to research hosting better than that. "Unlimited" doesn't mean much, the main thing to worry about is RAM and CPU usage. These so called unlimited hosting will most likely not let you install what you need, and even if you did, you'd be limited to 32MB total, or something almost as ridiculous, and your CPU share depending on the host can be almost non existent.
It may be great if you just want a small site, maybe a minimally dynamic site, but nothing compared to the needs of TASVideos.
arukAdo wrote:
The best thing of all is you can cut the site into 3/4 parts by reading what bisqwit posted
This mean shiny can take maybe the most "light" part, then nach another (without needing to upgrade) and a last one on a non-dedicated hosting
After talking more with Bisqwit, I doubt that'd work well.
arukAdo wrote:
That been sayd, now if theres a particular reason to have a dedicated server, that fine, but i guess you can combine with shiny or others for thoses need, and let the rest on non-dedicated
Maybe also the charge on a non-dedi could be a problem, that something i cant tell, but well, 10$ a month for just some RAM is realy wasting money i think
In fact, after getting a clearer picture from Bisqwit, I don't think $10 is quite right either.
To run everything, we need as follows:
A machine which provides root access, so we can setup EVERYTHING, since a lot of custom software is in use. We can't just default with whatever the hosting provider wants to give.
Bisqwit says 800MB of RAM is required, 512MB of RAM has ran things too slowly in the past.
Now I believe this number can be cut down if we try to do some things Bisqwit has not done in the past.
Namely, use a fully static cache for most of the main site, and only update it periodically. What this means however, that when a new run is published, it may appear on the forum, and also receive its own movie page, you won't notice it on frontpage/whatsnew/console till a few hours later. I'm not sure to what extent that'd be viewed as a bad thing. It may require modifying the code of the site though to make sure editors see the latest version when editing, but still have guests use the cached version whenever possible.
I also can't run things on my existing server, since the site apparently needs eAccelerator, which conflicts with other software I have installed.
The current design also requires more work than is necessary, since the images and page scripts for example don't have expires/caching enabled on them, so your browser has to grab them each time you view another page, putting more load on the server than is required, especially if you keep viewing the same pages over and over.
Based on an analysis performed on the current site, I think bandwidth requirements can get at least a 20% reduction, and memory usage can get anywhere from 10%-50% reduction depending on how much of the code is modified. If we consider most hits are from none members, and we can serve a separate tree to logged in and not logged in viewers, we can definitely see these improvements.
Looking at current hosting plans from respectable hosting companies that guarantee both RAM and to a certain extent CPU, and root and let you do what you want with the machine for the most part, best deals I know of that I can work with are as follows:
512MB RAM
16GB Storage
200GB Transfer
$30/month
640MB RAM
24GB Storage
300GB Transfer
$40/month
(I scaled back how much they actually give by a small % to reserve for OS install size, and management tools)
I believe that first setup with the right tweaking should be able to serve all our serving needs. However, if my estimates are a bit too idealistic, or we find that we are going over CPU limits due to the tremendous amount of processing we're doing, we may need the second option.
Thoughts?
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
adelikat wrote:
Only $10? Upgrading shouldn't be a problem then, if $10 is too much for you, talk to me about it.
$10 a month.
Well if the site can generate at least $10 a month, via ads/donations or whatever, then yes, I can go for that.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
