Judge Guidelines

Guidelines for judges

With great power comes great responsibility.

This page should summarize and detail what thoughts the judges should adhere to.

Table of contents [expand all] [collapse all]

Guidelines for judges
- Sum-up
Judge requirements
Judging
More reading

Sum-up

Quantity is not quality.
- Keep the number of different branches per a game minimal.
- Avoid doing decisions that undermine this guideline (or other guidelines) now or in the future. For example, don't publish an arbitrarily rule-restricted movie just because there are few of movies for that game; doing so may lead to impossible-to-solve "why A but not B" debates later.
Hard work should have a reasonable chance of being published. More on this below.
Satisfy the audience's expectations.

There are no laws that work in all possible situations. Each case is slightly different, and a judge is required to use multiple perspectives when judging a movie: both short-term and long-term consequences to (including but not limited to): site content quality; authors' feelings; audience's trust in the site staff; site's maintainability; the rules of TAS.

Judge requirements

Judges need to

have the will to follow these guidelines.
have experience on high-precision TAS making. If one does not have experience, one can not judge accurately.
- This must include a minimum of 8 published movies and preferably Expert player rank.
have a good record of watching, voting, and making helpful/informative posts on submissions.
have experience watching and/or making a variety of games on a variety of platforms.
have a reputation that will make them seem like a good candidate for a judge.
A TASVideos homepage. Will be assigned by an admin when promoted to judge.

Judging

Watch the entire movie from the input file!

Especially with the increase of authors/users encoding and uploading submissions to streaming media sites, it is important that the judge verify the authenticity of the input file itself.

The input file should sync properly on an approved version of a rerecording emulator. Any settings necessary for sync should be documented in the submission properly (edit the submission to include this information if necessary).

The input file should complete the game (or goal stated in the submission text).

It should fulfill the claimed goals of the author. I.E. if it is a SMB "walkathon", it should not press B.

It should not start from reset, SRAM, or soft-reset unless of course that is necessary for the goal choice (such as to unlock a character, or take advantage of newgame+, for instance).

Act consistent with the message of the site

Reject movies that break the Rules.
Judge movies in concordance to the Guidelines.

Managing game versions/ports on multiple platforms

As we expand to support more platforms this issue becomes more complicated.

In the past, we generally preferred one version of a game. The preference was for the original, such as Wonderboy for SMS instead of NES Adventure Island.

With so many platforms this policy is harder to achieve. While the situation should be judged on a case by case basis, it should generally be acceptable to have different versions of the same game.

In cases where they are too similar, the original version should be preferred, if applicable, such as favoring NES Super Mario Bros games over the SNES All-stars port. Of course even this example has exceptions. The "warpless" SMB2j is on the all-stars port because the later levels can be accessed without beating the game several times.

Be fair

A judge has the greatest control over the content of this website in the long run. All judges must act towards the goal of having an encouraging and rewarding atmosphere for both the players and the audience. You must be fair towards both.

It is fairness towards the audience when judges disqualify worse submissions and qualify the better ones.

Too many bad publications turn the audience away.
Too few publications turn the audience away (and possibly the players too).

The players must have chances of getting their movie published.

Don’t demand them do tedious laboursome optimizations that can’t actually be noticed in the viewed video ^[1]. (Example: 1-frame difference caused by different timing of a jump. (Exception: If the jump clearly bumps against an obstacle, it’s a visible difference.
For rejected submissions, refer to the relevant guidelines that would improve their chances of being accepted the next time. If possible, point out some specific mistakes.

Every newbie could be the next superstar. Their first submission might be rejected, but don’t destroy their self-esteem. Reject with reason, but only in the necessary amount.

[1]: Notice to new players: Experienced TAS makers have often developed a great observation skill for typical mistakes in TAS movies. Mistakes that might be unnoticeable to you will be noticed by experienced TAS players and the judges. (But of course, we don’t know and see everything.)

Improvements/Obsoletions

It may turn the audience away when meaningless new publications are done. (Publications which don’t look any different and differ only by less than a second.)
- Small improvements have and will be published, but it should in a situation where it seems reasonable that only those small optimizations are left.
For improvements to published movies, game choice is no longer a factor for judgement (generally).
However, if the submitted movie is clearly improvable as well, it (usually) should be rejected just as a non-published submission might be.
- While it is expected that the new run should use all tricks and techniques known at the time, it is not uncommon for new time-saving techniques to be found during the latter stages of making a run. Ideally, the run should be restarted to allow for inclusion of these new discoveries, however if restarting will be especially time consuming, exceptions can be made to this rule per a Judge's discretion.
Improvements to published movies should meet site standards, even if the published movie doesn't (use of correct the correct ROM, cheat codes, etc).
Sometimes an "improvement" may necessitate a new category, such as a huge time saving glitch (see SMW2).
Sometimes an improvement necessitates a double obsoletion.
- This may occur when there is a 1-player any% movie, and a 2-player longer but more entertaining (based on the consensus of the audience) movie but a new 2-player movie is submitted that is faster than the 1-player. (See Contra)
- This may occur when a new any% movie also achieves a lower % than the low%. (See SM glitched any%)

Hacks

By nature, hacks have different criteria for judgement than regular games.

A hack must be judged for its entertainment value as its own separate game but also in context to the original game.
Too many hacks of the same game engine causes the same issues as to many categories of the same game.
The quality of the hack should come into play as well. Hacks where only sprites are altered do not make for quality hacks. By contrast, a good hack alters the levels, the physics, the sprites, expands the overall game play, and even combines elements of other games to the point it doesn't feel like the same game engine.
The popularity of the hack needs to be considered.

Be open

Collect opinions

The audience has voice. If they don’t use it, encourage them to use it.

Read what other people think of the submission.

If nobody says anything, try to ignite discussion.

Be objective

Judge movies, not players.

Be verbose

Inform people of your actions, your thoughts and everything regarding a particular submission.

Try to write more in an encouraging tone than in a dooming tone, even if you are going to reject a submission.

Editing the submissions

Apply the Editor Guidelines to your editions. Specifically:
- Do not delete text written by others. (But you can reformat it for readability.)
- Use clear, easy to read markup and language.
- Attribute clearly.

Generally, when you add a comment to the submission, your markup should look like this for example (assuming you also have a homepage on this site by your name):

  ----
  [user:your username]: Rejected in favor of a [828S|faster movie].

(828S (an example) stands for a link to submission number 828.)

Replacing Submission files

Judges have the ability to replace a submission file.
This can be done when the submitter find a small improvement to their submission; rather than cancel and submit a new movie.
Use this feature only for small improvements not major changes. Anything that could possibly invalidated the voting results should definitely be a new submission (such as a controversial trick, a major improvement, or anything that might affect the entertainment value of the movie).

TASVideos