TASVideos

Tool-assisted game movies
When human skills are just not enough

Encoder Guidelines / TAS Videos Channel

Accessing the TASVideos Channel

Decentralize your control from a centralized location.

This page explains the how and why of the Encoder TASVideos Channel Manager site.

Rationale

All site encodes should be uploaded to TASVideos Channel on YouTube. By storing all our encodes in one place we gain the following benefits:

Trusted publishers will be given direct access to the channel, and can make use of our Youtube Uploader. But for all other encoders, upload access is given via the Encoder TASVideos Channel Manager.

Encoder TASVideos Channel Manager

This site is protected via SSL v3 / TLS v1. In order to access it, you should first install our certificate.

Anyone can visit the site in order to view recent activity and statistics of our uploaders.

Those with the appropriate permissions (tvc_upload), generally Encoders and above, have full access. Those with full access can use the upload area, and also download encodes that have been fully uploaded to the site, but not fully sideloaded to YouTube yet.

Requirements

This site requires a browser with many features not found in older browsers. For all the major browsers, version 10 is a milestone which should indicate that the site would work. However, some older may work anyway, and even if it does work, a newer browser will probably work even better.

Here's a list of some technology the site requires:

Usage

If you have the permissions needed to upload/download, the first thing you want to do is generate a password for yourself.

Enter your site password in the first box, and receive your encoder password in the box below it. For any given site password, the exact same encoder password will always be generated. You can each time you use the site, copy the generated password, and paste it into the password prompt that appears on the upload/download pages.

User names are case sensitive, so be sure to enter your credentials exactly.

Once you are logged in, you remain logged in until you close your browser, or press the log out button on the upper right of the upload page. When uploading using a public computer, remember to:

  • Have your browser NOT remember your password to the site.
  • Remember to log out and close your browser when you are done.
  • If you have your encoder password in your clipboard, remember to empty it, or overwrite it with something else.

Before uploading, ensure that your encodes are created according to the Encoder Guidelines, and that you have gone over your encode with the checklist.

Files up to 64 GB can be uploaded via the encoder site (but see below for more details). If for any reason your upload was interrupted, you can resume your upload just by sending the same file again. Most errors will also be automatically resumed by the page itself.

You can check the status of your uploads via the status page. The various statuses are explained below. Once a video is done sideloading, it's YouTube ID will be available.

Connecting a movie with YouTube

Once the video is done sideloading to YouTube, any editor on the site with appropriate permissions can copy the YouTube link from the status page and add it to a movie's page by editing its files as is normally done.

In order to help other staff know which particular movie you're uploading, or has been completed, it is suggested that you include the movie ID or URL in the title or description of your upload. If you're uploading an accepted submission prior to publication, then be sure to mention which one.

The description for a movie on YouTube is automatically copied from its movie page on TASVideos. When a description is edited here, the changes are propagated to YouTube.

In order to edit YouTube tags, or manually force description propagation, use System TVC Control. Don't forget to click Update after you save edits.

Resuming

Once a request has been registered, you can resume your upload at any time. Just fill out the form with the same file again, and it will upload where it left off.

Note: Most errors will be automatically resumed by the page itself if it is left open.

Note: The metadata in the form does NOT need accurate data for resuming, it will be ignored. Just fill out the form metadata with valid gibberish in the case of a resume.

Storage

Our site has a limited amount of storage. Therefore, if many large uploads have been queued, the site may inform you that there is no room for your upload at the moment, and to try again later. If you leave the upload page open, it will automatically retry periodically.

In any case, your upload request will be registered, and your file metadata will be saved.

The maximum allowed file size that can be uploaded is 64GB. However, depending on the amount of free space on the server, the working limit will be lowered. The following table explains the working limit based on how much free space is available:

Free Space Upload Limit
>= 95 GB 64 GB
>= 90 GB 60 GB
>= 80 GB 50 GB
>= 70 GB 40 GB
>= 60 GB 30 GB
>= 50 GB 20 GB
>= 40 GB 15 GB
>= 30 GB 12 GB
>= 20 GB 10 GB
>= 10 GB 5 GB
>= 4 GB 4 GB

If there is <4GB free, then the limit will be whatever is available.

Hashing

Before a file is uploaded, it is hashed, in order to uniquely identify it, and ensure the movie is constant when resuming. When an upload is complete, this hash is also used to confirm integrity.

If an upload integrity check fails, your file is deleted.

Status

Each file on the site has one of the following 5 statuses.

  • Waiting - The encoder informed the site of their desire to upload a file, but uploading has not yet begun.
  • Uploading - The encoder is currently in middle of uploading the file to the site.
  • Queued - The file has been uploaded, and is queued to be uploaded to YouTube.
  • Sideloading - The file is currently being uploaded to YouTube.
  • Done - The file is on YouTube. The encode has been deleted off the TASVideos server. The YouTube ID/link is available. Status entries which have reached the done state are deleted once they are a month old.

Sideload process

The site examines the video queue every hour at 45 minutes past the hour to see if sideloading should begin. When sideloading begins, the site determines if any video is currently being sideloaded, if so, it is continued, if not, the oldest video in the queue begins sideloading. Whenever a sideloading operation completes, the queue is reexamined immediately, regardless of what the current time is.

Download

If you wish to download an encode, the site also supports pause and resume of those as well, so be sure to use a browser which can too. You can also download with cURL:

curl -OJk --digest --user <user name>:<encoder password> "https://encoders.tasvideos.org/secure/download.exe?hash=<file hash>"

Resume with cURL: curl -C <amount downloaded> -OJk --digest --user <user name>:<encoder password> "https://encoders.tasvideos.org/secure/download.exe?hash=<file hash>"

Replacing encodes on TASVideosChannel

If you are replacing an encode on our channel, please upload your new encode like usual. Then contact the senior publisher or an administrator to delete the old encode.

Alternate upload methods

TVCMan - Youtube Uploader Clone

We also offer a clone of our Youtube Uploader modified for this site, known as TVCMan, the TASVideos Channel Manager - command line uploader.

Downloads

Usage

Usage is pretty much the same as for ytu. The biggest difference is that the built in database removes entries for uploaded files. Please see the activity status page to view upload status, and which files you finished uploading recently.

Nothing is reported upon success, but errors are reported if something goes wrong.

Using the command line version of the TASVideos Channel Manager is completely interchangeable with the browser based version. An upload that was started with one can be completed with the other.

Legal

This product includes software developed by the OpenSSL Project for use in the OpenSSL Toolkit.

Author

The author of TVCMan is Nach, the creator of ytu, and the creator of the TASVideos Channel Manager site itself.

API

You're welcome to design your own application to upload to the site.

Overview

The API is based upon standard HTTP/1.1 along with Google's 308 Resumable Upload / Resume Incomplete additions.

Access to the API is protected by SSLv3/TLSv1 and Digest Authentication. If possible, force your application to only perform authentication via Digest, in order to prevent man-in-the-middle attacks. If you are unable to do so, ensure that the certificate chain for the site is valid. Bundling the root certificate may help.

In any case, if you can force Digest Authentication, or fully verify the site's certificate, you should be accessing the API in a mostly safe manner. If you are unable to verify the site's certificate in your application, nor can you force authentication only via digest, then terminate immediately.

URL

Challenge responses

All calls can return HTTP 401 if the login credentials were incorrect, did not contain upload permissions, or were sent improperly.

Irregular responses

Due to occasionally Apache, PHP, or MySQL running out of handlers or other resources, you'll get an invalid response. In these situations, wait 30 seconds, and try your request again.

There are two known responses that would indicate this happened.

  • HTTP 200 with a Content-Type header of application/x-httpd-php5-cgi.
  • HTTP 500 without a Warning header.

Invalid method responses

If you attempt to use an HTTP method not supported, you will receive HTTP 405.

Steps

The API for uploading consists of 3 steps.
  • Registering your file's metadata for upload.
  • Checking upload availability and status.
  • Upload/Resuming.

Send via POST the following parameters:
  • filename - Name of the file you're uploading.
  • filesize - Size of the file you're uploading in bytes.
  • filehash - MD5 hash of the file you're uploading, in 32 hexadecimal characters. Regex: ^[0-9a-fA-F]{32}$.
  • title - Title of the video you're uploading. Up to 100 characters, and must not contain < or >. Regex: ^[^<>]{1,100}$.
  • description - Description of the video you're uploading. Up to 5000 characters and must not contain < or >. Regex: [^<>]{1,5000}.
  • tags - Tags of the video you're uploading. Between 2 and 500 characters, and must not contain < or >. Tags are separated by a comma and must be between 2 and 30 characters in length. Regex: ^([^<>,]{2,30})(,([^<>,]{2,30}*$. Also perform an explicit length check to ensure it is between 2 and 500 characters.

A successful response is HTTP 204.

Failure responses are:

  • HTTP 400 - The POST data was not properly sent.
  • HTTP 413 - The file size exceeds the server limit. An X-Max-File-Size header is also returned with the overall server limit.
  • HTTP 500, with Warning: 199 - An error occurred trying to store your data in the database.

Example:
POST /secure/upload.exe HTTP/1.1
Authorization: Digest username="Nach", realm="TASVideos Channel Uploader", nonce="ABCDEF123", uri="/secure/upload.exe", cnonce="123ABCDEF", nc=00000001, qop="auth", response="a2777775d8535152a2ce8ecc83b162a5", algorithm="md5-sess"
Host: encoders.tasvideos.org
Content-Length: 191
Content-Type: application/x-www-form-urlencoded

filename=video%2Emkv&filesize=1024000&filehash=01234567890abcdef01234567890abcdef&title=TASMan%20II%3A%20Dr%2E%20Nach%27s%20Revenge&description=Really%20cool%20TAS%21&tags=TAS%2CTASMan%2CNach

Send via PUT the following HTTP headers:

  • X-File-Md5 - Value you used for filehash in step 1.
  • Content-Range - bytes */*.
  • Content-Length - 0.

A successful response is one of the following:

  • HTTP 201 - Your file was already fully uploaded.
  • HTTP 308, with Range header - Your file is in middle of uploading: Example: Range: bytes=0-42/1024000. Continue uploading from one byte after what is already on the server, in this example, 43.
  • HTTP 308, without a Range header - Your file is registered for upload, but no uploading has begun yet.

Failure responses are:

  • HTTP 400 - Improper headers received.
  • HTTP 404 - Value in X-File-Md5 has not yet been registered for upload.
  • HTTP 411 - Content-Length header missing.
  • HTTP 413, with a Retry-After header, and an X-Max-Allowed-Size header - Server is currently full, retry in the amount of seconds specified. The X-Max-Allowed-Size header contains the current maximum size allowed to upload, based on free server space. An X-Max-File-Size header is also returned with the overall server limit.
  • HTTP 500, with Warning: 199 - An error occurred trying to store your data in the database.

Example:
PUT /secure/upload.exe HTTP/1.1
Authorization: Digest username="Nach", realm="TASVideos Channel Uploader", nonce="ABCDEF123", uri="/secure/upload.exe", cnonce="123ABCDEF", nc=00000002, qop="auth", response="b5788775d8535152a2ce8ecc83b162a5", algorithm="md5-sess"
Host: encoders.tasvideos.org
Content-Length: 0
Content-Range: bytes */*
X-File-Md5: 01234567890abcdef01234567890abcdef

Send via PUT the following HTTP headers:

  • X-File-Md5 - Value you used for filehash in step 1.
  • Content-Range - bytes begin-end/total. Example: Content-Range: bytes 43-1023999/1024000.
  • Content-Length - amount of bytes being uploaded. Example: Content-Length: 1023957.
  • Content-Type - application/octet-stream.
  • Expect - 100-continue.

Followed by the data you're uploading.

The server will not accept chunks out of order. Ensure that the chunk you're sending begins one byte after the end marker in the Range header response received from step 2, or 0 if the file has not yet begun uploading.

If for any reason your connection breaks during uploading, restart from step 2, so you can resume your upload from where it left off.

You may upload chunks up to 10MB in size (10,485,760 bytes).

Your application should wait for a 100 Continue response from the server before you begin sending data (after you have sent the headers).

A successful response is one of the following:

  • HTTP 201 - Your file is fully uploaded.
  • HTTP 308, with Range header - Your chunk has been received, and the server awaits the next chunk. See the format of Range above.

Failure responses are:

  • HTTP 400 - Improper headers received, or you skipped step 2.
  • HTTP 404 - Value in X-File-Md5 has not yet been registered for upload.
  • HTTP 409 - You tried uploading a latter chunk before uploading all prior chunks, or the total size specified in Content-Range does not match the file size in step 1, or the MD5 of the file did not match what you reported it as. In the last case, your file will be deleted.
  • HTTP 411 - Content-Length header missing.
  • HTTP 413 - The file size exceeds the server limit. An X-Max-File-Size header is also returned with the overall server limit.
  • HTTP 500, with Warning: 199 - An error occurred trying to store your data in the database, or storing your file data on the hard drive.

Example:
PUT /secure/upload.exe HTTP/1.1
Authorization: Digest username="Nach", realm="TASVideos Channel Uploader", nonce="ABCDEF123", uri="/secure/upload.exe", cnonce="123ABCDEF", nc=00000003, qop="auth", response="5b777885d8535152a2ce8abb83b1625a", algorithm="md5-sess"
Host: encoders.tasvideos.org
Content-Range: bytes 0-1023999/1024000
Content-Type: application/octet-stream
X-File-Md5: 01234567890abcdef01234567890abcdef
Content-Length: 1024000
Expect: 100-continue

Wait for: HTTP/1.1 100 Continue

<chunk data>

Alternate download methods

As mentioned above, you can download with cURL. However, you can also write your own downloader.

Everything said in the upload API overview above applies here also. Except resuming is done with standard HTTP 206 instead of Google's HTTP 308 extension.

All API calls are to https://encoders.tasvideos.org/secure/download.exe. Accepted methods are HEAD, GET, and POST. All calls expect a hash parameter containing the MD5 hash of the file you wish to download.

The challenge, irregular, and invalid method responses apply here too.

Valid responses are HTTP 200 and 206. The server has full HTTP 206 support (well, as much as Apache implements anyway).

Invalid response is HTTP 404.

Forum

You can discuss this feature on our forum.

Mistakes

If you started uploading a file in error, close your browser (tab) or application. Then contact Nach or Ilari and ask them to delete your file.


Combined RSS Feed
EncoderGuidelines/TASVideosChannel last edited by Nach on 2015-05-05 00:09:26
Page info and history | Latest diff | List referrers | View Source