Access Blocked
Having problems with diff or editing and are being referred to this page? This site tries to protect itself from dumb mirroring scripts that issue some thousand requests within minutes bringing our server to its knees.These measures have been taken:
- checking for referrer information
- hidden trap link
- limiting requests/minute
Referrer information
A first measure is to block access to any function
other than viewing a page, if there is no referrer
information present. What does this mean?
Every time you click on a link, your browser sends
a request to our server to get the desired page
and pictures. This request not only contains the
pagename which you would like to see, but also
which page you are coming from (referrer information).
However, mirroring scripts don't send this information.
So checking for the referrer information is an easy way
to distinguish between scripts and regular browsers.
On the other hand, many standard configurations
of browsers and firewalls refuse to send referrer
info too.
If there is no referrer information, then everything
but viewing a page is blocked (e.g. diff, edit, save, ...).
If you get the "AccessBlocked" message as a regular user,
then either you are not using a standard browser, or have
configured your browser in a way to not send the referrer
information, or a proxy you are using is removing this information.
Of course, to fix the problem, you have to disable this information-secrecy option from the proxy/firewall/somethingblocker you are using. As the referrer information has to originate from within this site, it is no longer possible to link to diff etc. from other websites. Note: you can still link to pages themselves.
Of course, to fix the problem, you have to disable this information-secrecy option from the proxy/firewall/somethingblocker you are using. As the referrer information has to originate from within this site, it is no longer possible to link to diff etc. from other websites. Note: you can still link to pages themselves.
Hidden trap link
As a second measure, there's a trap link on the pages for scripts to follow if they don't obey the robots.txt file. Users should not normally be able to see this link. (You can see it in the source, but don't try it out or your address will be blocked for 48 hours. Really. We mean it.)
Limit requests/minute
Some scripts issue many requests per minute. We now enforce a limit of 35 requests/minute. Any IP issuing more requests is first warned and subsequently blocked for 6 hours.
If you are a regular user and got blocked by this, then send us an email.
Maybe you should reconsider your browsing habits :o)
Note: If you use a "pre-fetching" feature in your browser, you may get
blocked without warning because the pre-fetching downloads hundreds of pages
from this site without your explicit request, first getting a warning (which
it does not display because it's a prefetch) and then blocking your access
before you even know it.
The above measures should shield TASvideos from the most offensive scripts.
Text mostly copied from Sensei's Library
FractalFusion: How come some external links link to this page? For example: http://www.mmhp.net.
AccessBlocked last edited by Bisqwit on 2007-01-11 14:27:07
Page info and history | Latest diff | List referrers
The above measures should shield TASvideos from the most offensive scripts.
- Q>What if I would still like to have a local copy of TASvideos?
- A>Download a ready packed snapshot at TASSnapshot.
- Q>What if I want to do some data processing on the site contents, such as load the list of all movie filenames published since day X?
- A>Consider the SOAP API we have.
Text mostly copied from Sensei's Library
FractalFusion: How come some external links link to this page? For example: http://www.mmhp.net.
- Bisqwit: Because the link should be written http://www.mmhp.net/. Yours is missing the path component of the URL. (The browser fills in / automatically, but it does not change the fact that your URL is missing that part.)
- Bisqwit: The literal reason why it happens is because the site autoconverts the links into a referrer tracking mechanism (yes it is very cunningly hidden), which uses a checksum to prevent spammers and phishers from using the tracking cgi as a bouncer. However when the URL is malformed, the reference checksum (which is calculated from the URL autocorrected by the browser) does not match the one generated on the server, and thus this page is shown instead.
- Bisqwit: The literal reason why it happens is because the site autoconverts the links into a referrer tracking mechanism (yes it is very cunningly hidden), which uses a checksum to prevent spammers and phishers from using the tracking cgi as a bouncer. However when the URL is malformed, the reference checksum (which is calculated from the URL autocorrected by the browser) does not match the one generated on the server, and thus this page is shown instead.
AccessBlocked last edited by Bisqwit on 2007-01-11 14:27:07Page info and history | Latest diff | List referrers