Watch the entire movie from the input file!
Especially with the increase of authors/users encoding and uploading submissions to streaming media sites, it is important that the judge verify the authenticity of the input file itself.
The input file should sync properly on an approved version of a rerecording emulator. Any settings necessary for sync should be documented in the submission properly (edit the submission to include this information if necessary).
The input file should complete the game (or achieve the goal stated in the submission text).
It should fulfill the claimed goals of the author. I.e., if it is a SMB "walkathon", it should not press B.
It should not start from reset, SRAM, or soft-reset unless, of course, that is necessary for the goal choice (such as to unlock a character, or take advantage of newgame+, for instance). Of course this depends on the goal choice being worthwhile too.
Act consistent with the message of the site
Managing game versions/ports on multiple platforms
As we expand to support more platforms this issue becomes more complicated.
In the past, we generally preferred one version of a game. The preference was for the original, such as Wonderboy for SMS instead of NES Adventure Island.
With so many platforms this policy is harder to adhere to. While the situation should be judged on a case by case basis, it should generally be acceptable to have different versions of the same game.
There are several observed schools of thought in regards to ports and conversions considered similar enough to warrant a strong preference. In some cases it becomes similar to ROM selection for the same platform where different regions have noticeable changes in content.
- The first dictates that the original (chronologically first to be released) version should be preferred. Advantages: indisputable authenticity.
- The second dictates that the more popular (more widely spread and/or recognized among the audience) version should be preferred. Advantages: easier and better recognition, better compatibility with existing records.
- The third dictates that the superior (such as having expanded content, better graphics or music, more glitches, less lag or shorter loading times) version should be preferred. Advantages: potentially better watching experience, potentially more opportunities for creative timesavers. Superiority in this case can be disputed.
- There are cases when a submission aspires to a goal only feasible in versions with altered or expanded content. Examples include using Maria which is only present in the Saturn port of Symphony of the Night, and warpless SMB2j movie made on the SNES All-Stars conversion because the later levels are accessible without beating the game multiple times there.
Obviously there are cases where at least two of these options pertain to a single release, as either original or superior version will likely also be considered the most popular. In any case this should be carefully considered on per-game basis with full understanding of the differences between all versions.
A judge has the greatest control over the content of this website in the long run.
All judges must act towards the goal of having an encouraging and rewarding atmosphere
for both the players and the audience. You must be fair towards both.
It is fairness towards the audience when judges disqualify worse submissions and qualify the better ones.
- Too many bad publications turn the audience away.
- Too few publications turn the audience away (and possibly the players too).
The players must have chances of getting their movie published.
- Don't demand them do laboursome optimizations that can't actually be noticed under normal viewing conditions . For example, if a jump performed one frame late still moves the character along its purported trajectory with no interruptions, it is a negligible flaw. If, however, a late jump bumps the character into an obstacle, thus interrupting its movement, it's an easily noticeable mistake.
- For rejected submissions, refer to the relevant guidelines that would improve their chances of being accepted the next time. If possible, point out some specific mistakes.
Every newbie could be the next superstar. Their first submission might be rejected, but don't destroy their self-esteem. Reject with reason, but only in the necessary amount.
]: Normal viewing conditions refer to real-time playback with no additional indicators to judge the player's performance; i. e., like most people watch these videos
Notice to new players: Experienced TAS makers often possess a great observation skill for typical mistakes in TAS movies. Mistakes unnoticeable to you might still be noticed by experienced TAS players and the judges. (But of course, we don't know and see everything.)
Tiers and goals
Formerly, the movies that were found boring by the audience and by judges were rejected for a bad game/goal choice. After the Vault
was introduced, some unimpressive, but still well-played movies got a chance to be published. However, not all goals can be accepted, only the ones that don't trade off time for entertainment (time trade-offs aren't justified if a movie fails to entertain). Allowing such goals for the Vault would result in vanishing the overall quality border due to hosting countless boring movies with arbitrary goals.
The only two types of goals acceptable to the Vault are "fastest completion" (any%) and full completion (100%/best ending). All other goals like "maximum score", "pacifist", "no damage", or "suboptimal character" are unvaultable.
If a movie does get a good feedback, even an esoteric goal
like "glitchless low%, Speedbooster"
can be accepted. For liked movies the Moons
tier is selected.
But if a movie has an uncommon goal, and at the same time gets poor feedback, it has no way to be published and must be rejected.
Here it becomes important to determine the feedback tinge.
- Read the thread, count both "like" and "dislike" posts. More dislikers than likers, or equal amount of both mean it wasn't recieved well enough.
- Look at the submission votes. Equal "Yes" and "No" or only slight preponderance of "Yes" mean no way for Moons.
- Watch the movie yourself and compare to the current Moons and Vault residents.
Improvements and obsoletions
- Avoid meaningless publications. It may turn the audience away when a large improvement is possible, yet each small incremental change with no visible differences in gameplay finds its way onto the main page.
- Small improvements have and will be published, but only in a situation where it seems reasonable that only those small optimizations are left. If larger known improvements aren't implemented, it may be grounds for rejection.
- For improvements to published movies, game choice is no longer a factor for judgment (generally).
- However, if the submitted movie is clearly improvable as well, it should (usually) be rejected.
- While it is expected that the new run should use all tricks and techniques known at the time, it is not uncommon for new time-saving techniques to be found during the later stages of making a run. Ideally, the run should be restarted or edited to allow for inclusion of these new discoveries, however, if restarting will be especially time consuming, exceptions can be made to this rule per judge's discretion.
- Improvements to published movies should meet the site's standards, even if the published movie doesn't (such as use of correct ROM, cheat codes, etc.).
- Sometimes what is claimed to be an improvement may necessitate a new branch, such as when a huge time-saving glitch is involved (see SMW2), or unusual goal choice is exhibited (as in Princess-only SMB2 run, whose first iteration beat then-current any%).
- Sometimes an improvement necessitates a double obsoletion.
- This may occur when a new submission aims for a goal in a certain branch (for instance, any%) but also achieves primary goals of other branches (such as low% or 100%) better than their respective published movies (see Super Metroid glitched any%). Another notable example is when a 2-player movie, generally considered to be more entertaining than a 1-player version, also outperforms it time-wise (see Contra).
By nature, hacks have different criteria for judgment compared to regular games.
- A hack must be judged for its entertainment value as its own separate game but also in context to the original game.
- Too many hacks of the same game engine causes the same issues as too many categories of the same game.
- Quality of the hack should come into play as well. Hacks where only sprites are altered do not make for quality hacks. By contrast, a good hack alters the levels, the physics, the sprites, expands the overall game play, and even combines elements of other games to the point it doesn't feel like the same game engine.
- Popularity of the hack needs to be considered.
The audience has voice. If they don't use it, encourage them to use it.
Read what other people think of the submission.
- If nobody says anything, try to ignite discussion.
- Judge movies, not players.
Inform people of your actions, your thoughts and everything regarding a particular submission.
Try to express yourself in an encouraging tone, even if you are going to reject a submission.
Editing the submissions
- Apply the Editor Guidelines to your editions. Specifically:
- Do not delete text written by others. (But you can reformat it for readability.)
- Use clear, easy to read markup and language.
- Attribute clearly.
Generally, when you add a comment to the submission, your markup should look like this for example (assuming you also have a homepage
on this site by your name):
[user:your username]: Rejected in favor of a [828S|faster movie].
(an example) stands for a link to submission number 828
Replacing submission files
- Judges have the ability to replace a submission file.
- This can be done when the submitter finds a small improvement to their submission; it is usually better to incorporate it immediately rather than cancel and submit a new movie.
- Check that the movie still satisfies its stated goals and that the movie file is good.
- Use this feature only for small improvements and not major changes. Anything that could possibly invalidate the voting results should definitely be a new submission (such as a controversial trick, a major improvement, or anything that might affect the entertainment value of the movie).
- There is a minimum time a submission can be in queue before a verdict can be made (currently 72 hours). Prior to this time a submission can be set to "judging underway" or "needs more info".
- Use the "judging underway" feature, and use it sooner rather than later. If you decide to handle a submission set this first so other judges know it is already being taken care of.
- Let the conversation play out. The audience has a say, so let them say it. If a thread is very active still (many posts in a short time period), let it sit until the conversation dies down a bit.
- Give a chance for rebuttal. Try to indicate the verdict you are leaning to in a post, or ask questions rather than make statements ("Why did you do x?" instead of "x looks improvable"). Give the author a sufficient chance to respond to assumption made about the movie.
- Avoid saying it "looks" improvable. Instead, take the time to show that it IS improvable. Show an existing WIP, or make a short one of your own. Document some specifics rather than making blanket statements impossible to prove.