The good:
It provides a way to filter TASes by quality (although filtering by movie rating range would work too*). The vault did enable more games to be accepted, though that should have been allowed all along. Vault does get credit for fixing that, and that is a big improvement to this site.
The bad:
The default movie lists hide vault entries, causing them to be overlooked (and less likely to be rated*), even if they actually are high quality. In some respect, if there is a TAS of a game that is considered "bad", then it will tend to be put in Vault regardless of how well the TAS was done because it would be perceived as "boring". I would prefer to have the full list by default, and then let people filter the list however they want.
The ugly:
The vault category name/icon does make those TASes feel second rate. Also, there is also a "penalty" applied to the score calculation to the player score for vaulted TASes. I do not agree with any of the multipliers for tiers; I don't think they are necessary. Because of how the rating system has always worked (which I like), there already is a penalty for runs with a lower rating; that is two 4.5 rated TASes do not equate to one 9.0 TAS. But with the current system, the 2 TASes would be in separate tiers, and so the separation is enhanced with multipliers, which is rather unfair to TASes that happen to be put in vault, especially if they are borderline cases.
*Provided that the movie rating system is working well. Right now, that system is broken along with the submission voting, and I think that is the primary factor why there are complaints. It may be that the tiers are generally working, but it needs the voting to be working to help the judges decide.
Submission voting:
The question is "Did you find this movie ENTERTAINING?" (emphasis added) But some people have been voting as if it was "Should this movie be published?", especially when the TAS is an improvement. This makes it difficult for a judge to determine the entertainment response. I would propose that there be 2 poll questions, the ones mentioned above. That way the "yes for improvement" votes can be moved to the published poll, and then the entertainment can be voted on properly. It might be worth having 5 choices for the entertainment poll, sort of like a 1-5 rating, but with wording like: yes, probably, maybe, not really, no. I think these changes will help the judges determine the entertainment better.
Movie voting:
I honestly have no clue how I am supposed to rate movies, so I refrain from doing so. I don't see a 100 point system as useful, even 10 point is somewhat difficult. But having a description for what that rating means would be very useful, and it would help people give useful ratings, especially help avoiding voting 7 because that is the default. Also, having it be less daunting would help encourage people to vote. As for the technical rating, I can't even imagine what the numbers mean, and the votes from people make no sense either. That definitely needs text to help understand it. Perhaps another category could be useful to either add to diversify, or replace another if it were to make more sense. I can't think of any right now, but it should be considered. I agree with AKheon that an optional review would be useful with the voting. It is sort of like the comments in the submission thread that go along with the voting.
Conclusion:
I am not so sure what to do about tiers right now because I think we should focus on fixing the voting system, which I think is the main cause of the symptom, tier complaints. If the vote results were more useful, it would help make better sense of the tiers.
But in the meantime, I think we should change the name and icon of vault so it is less negative. The original system sort of had tiers (nothing, moon, star) though they were really category marker tags. In fact, I think it is better to think of tiers as just category marker tags for sorting, like they were before the change.
We should drop the score multipliers for tiers as well since that penalizes lower rated movies(or perceived lower) more than is deserved.
Adding reviews(comments) for movies would be interesting. I suppose it could be done now by creating a thread for each movie and having people post there.
