Emulator Coder, Site Developer, Site Owner, Expert player
(3574)
Joined: 11/3/2004
Posts: 4754
Location: Tennessee
it fail again too, for precisely the same reason. If you like the movie, anything less than a 10 risks it being vaulted, and you wouldn't want that.
Also, give this system a chance a little bit before you decide that a few people are incapable of keeping the tiers staight. One of the next logical steps is to have a specialized role for maintaining tiers (expand the starman role?). And guidelines for this would involve paying attention the rating. A movie gets Vaulted, and gets rated high, it has a chance. However, I don't find ratings accurate enough to trust this system to be automated.
There could be a special area between submission and publication, where the accepted movies stay for 2 or 3 days, or till it receives a certain number of rates before it gets publicated. If that makes any difference.
Joined: 4/17/2010
Posts: 11478
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
Discussing Star-worthy movies with Nach, I'm more and more sure we need 2 hand-picked highligts: Gold (=Stars) and Silver (=Moons), then semi-automated Regular (non-marked), Vault (boring crap) and probably those Demo and Playaround (totally automated by category).
It's needed just to keep the awesomeness-based sorting really fair. For example, if a bunch of similar runs is proved to be frame perfect (tased to death), but only one of them is also entertainment-wise perfect, the rest need a Silver. Silver would mark frame perfect records that are cool to watch, but still not Gold.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
To the people who have problems with understanding the tiers, here is my short summary as I understand it:
Stars/moons: what we published earlier, only that stars are now handed out more generously.
Vault: movies we rejected earlier, even though they completed a game in the shortest possible time.
------
I understand the aim of this change. We want to include more movies, and not reject people's hard work just because they chose a game which it turned out people did not find interesting to watch.
The downside with the change, is that the published movies will be less entertaining to watch on average, according to the viewers. (In Bisqwit's words, the signal to noise ratio will be lower.) For example, people subscribed to the TASVideosChannel will see all movies in their feed, without being able to tell which tier they are in. (This is maybe not that easy to fix. An extra channel could be used for Vault movies, but then it would be hard to move movies between channels.)
-------
A minor thing: In the description for the Vault category, there's an (important) unfinished sentence: "Its purpose is to fulfill the site's goal of being".
This gives rise to this decision procedure, correct?
The symmetry of the axis graphic is reflected here, which is why I find this layout intuitive.
Sensibly, adelikat wants to give a place to optimal movies that aren't entertaining, and everyone agrees with this. I think the problem is only trying to add on the Vault category leads to imbalance, which is why people are suggesting additional categories to balance it out. So this is the desired layout from the original proposal (if I understand it correctly):
The problem is that people perceived this as ending up in practice as:
Because the categories seem diluted. I think this happens because Star is trying to take on some of the load Moon has from not having Playaround and Demo to help out. Really Star ought to have remained exactly the same subset of movies, and only movies that obsolete more entertaining runs (as well as previously Grued runs) would be reconsidered and moved to Vault. This would have only reduced the size of Standard movies and given Vault its place.
So that's why I think it may be worthwhile to tryin again but with all the organizational categories.
What I think adelikat wants to remind everyone though is that runs aren't being chopped off any more than they were in the past, and it's in fact the opposite. More runs will be accepted now because of Vault. Just compare the above decision procedures with what we used to have:
Nothing is being lost, except IMO the clarity of categories.
Joined: 5/2/2006
Posts: 1020
Location: Boulder, CO
I know I'm late to the party on this one, but I still wanted to say that I really like this whole concept.
Actual minimum-frame records instead of entertainment is something I have wonted to see for a long time.
I think we should keep things simple on this site, and that's why I would prefer the original system.
Of course, some movies seem interesting, while some movies seem boring, but there are still lots of movies struggling between them. It's subjective, and personally I don't prefer we tier movies this way.
We have movie categories for Demos and Playarounds, and I don't like to see them blended with Stars and Moons.
Even though Vault would lower the entertainment of movies on our site on average, I'm not against that, because it does have some meanings. But I think we shouldn't change the meaning of Stars and Moons. After all, not every movie is special, most of them are just regular ones which don't need tiers.
Although we have many ideal systems in our minds, I still think we should keep things simple on TASVideos, because sometimes the simplest way is the best, and it would keep the administrators and judges less busy.
Recent projects: SMB warpless TAS (2018), SMB warpless walkathon (2019), SMB something never done before (2019), Extra Mario Bros. (best ending) (2020).
Joined: 3/23/2012
Posts: 296
Location: In X position=50 y position=20
HappyLee wrote:
I think we should keep things simple on this site, and that's why I would prefer the original system.
Of course, some movies seem interesting, while some movies seem boring, but there are still lots of movies struggling between them. It's subjective, and personally I don't prefer we tier movies this way.
We have movie categories for Demos and Playarounds, and I don't like to see them blended with Stars and Moons.
Even though Vault would lower the entertainment of movies on our site on average, I'm not against that, because it does have some meanings. But I think we shouldn't change the meaning of Stars and Moons. After all, not every movie is special, most of them are just regular ones which don't need tiers.
Although we have many ideal systems in our minds, I still think we should keep things simple on TASVideos, because sometimes the simplest way is the best, and it would keep the administrators and judges less busy.
I agree. old way is better
Jungon wrote:
if I was to have a Tool-Assisted real life ... I'd..
I could abuse death, just to see if it saves time ..
Moons for valuable perfections, star candidate neighbours, highlight in any way = Silver.
Lightning bolt for notable improvements (even SMB 1 frame improvement was really important, though minimal possible) = Bronze. I personally don't ming slight overlapping with the above 2.
3-mark system allows to fairly highlight all that needs to be highlighted, nothing is missed and nothing is abusively added (thanks to highly-trained monkeys).
This, and lowering the Vault criteria a bit (6- insread of 7-) really goves a room to keep neither-too-good-nor-too-bad runs, that would really serve "fulfilling the site's goal of being" the base for movies solid in both tech quality & entertainment.
That would allow pointing to attracting (in 3 ways) movies manually, and pointing to distracting movies, leaving just-okay runs untouched (unmarked). I find it exhaustive.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
As for rating a movie when it is a submission, I thought of that back in 2008. Seemed like a great idea, all those votes would turn into ratings and improve the rating system and give judges more information for judging. But it clearly failed. Instead people rated 10 on movies, to see them published, and ratings were heavily inflated. We reverted back.
The idea of having more than two categories (well, three) in the submission voting may indeed sound like a great idea at first. However, as you say, it's prone to abuse and to skew up results (someone may vote 10 just to see something published, while someone might vote 0 for a stupid reason, eg. because they don't like the console or improvements to an existing run). A "yes/no" voting system avoids abuse because everybody's vote weights as much as everybody else's, so no one person can skew up the results.
A suggestion that's just come to my head: instead of saying certain runs are "better than others", we could use symbols to mark the best thing(s) each movie has to offer. Viewers should be able to put the mouse over the symbol to se a short description of it, or click it to see a list with all of them (like the current "Tiers" page).
With this system:
-Stars could still be used for the absolute best movies of the site, like they've always been.
-Movies with a high entertainment value could be still marked with moons, and other fields like speed would get other symbols; for example: a Sonic shoe for speedy movies (either by time or by character movement), a wrench for technical feats...
- Symbols would be combinable. For example, a fast AND technical run would get both a Sonic shoe and a wrench, etc.
Antes de una TAS:
-Malo: ¡Voy a conquistar el mundo!
-Héroe: ¡Por poco tiempo!
Emulator Coder, Site Developer, Site Owner, Expert player
(3574)
Joined: 11/3/2004
Posts: 4754
Location: Tennessee
I really like the gold,silver,bronze suggestion (Kind of wish that had come up before the change).
So how about this:
Gold medal = old star criteria
Silver medal = old moons if they were used with consistency
Bronze medal = current moons
Vault = current vault
(And this system doesn't mean that we couldn't add in a demos type tier at some point.)
For example, people subscribed to the TASVideosChannel will see all movies in their feed, without being able to tell which tier they are in. (This is maybe not that easy to fix. An extra channel could be used for Vault movies, but then it would be hard to move movies between channels.)
To avoid that problem, all Vault movies uploaded to TASVideosChannel could include a "[Vault]" tag at the beginning of the title and an explanation in the description clarifying what the Vault is about.
AzumaK wrote: I swear my 1 year old daughter's favorite TASVideo is your R4MI run :3
xxNKxx wrote: ok thanks handsome feos :D
Help improving TASVideos!
-Stars could still be used for the absolute best movies of the site, like they've always been.
Even at the risk of sounding like a broken record, stars were never intended to be a "the absolute best movies" symbol. Rather, it means "recommended movie".
Not all of the best movies get a star, nor are all the starred movies necessarily the best. The point is that there's a relatively small selection of movies that showcase what's cool about TASing. Whether it's considered a top TAS is inconsequential (although often goes hand-in-hand).
Joined: 4/17/2010
Posts: 11478
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
Warp wrote:
Ajavalo wrote:
-Stars could still be used for the absolute best movies of the site, like they've always been.
Even at the risk of sounding like a broken record, stars were never intended to be a "the absolute best movies" symbol. Rather, it means "recommended movie".
Not all of the best movies get a star, nor are all the starred movies necessarily the best. The point is that there's a relatively small selection of movies that showcase what's cool about TASing. Whether it's considered a top TAS is inconsequential (although often goes hand-in-hand).
Stars are more about best of the best, what means not just frame perfected, but entertainment-wise incredible. There may still be several candidates to a star, and the ones that weren't chosen must have a rest in moons (after it gets separated from commonplace movies, which must also be divided from vault).
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
I understand the aim of this change. We want to include more movies, and not reject people's hard work just because they chose a game which it turned out people did not find interesting to watch.
I think the aim of this change is different and has little to do with people's hard work. The current list of games has become more and more like a full list of games, with some holes where the more impopular/unknown games should be. Given shows like the Angry Video Game Nerd, Video Game Vault and many more, there does seem to be an interest in these games. With the addition of the vault tier, TASVideos could become a place where you can find complete playthroughs (and if you're lucky some interesting strats) of just about any, no matter how obscure, game out there.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
There is a major flaw with the current proposal of tiers.
The word "tier" as used here connotates that something in one grouping is significantly better than something in a lesser grouping. However the way things are being assigned to tiers do not mesh with the word tier at all.
Saying that only one movie for a particular game can be in the highest "tier" makes no sense. Either the movies in the higher "tier" are a cut above those in a lesser "tier", or they are not a "tier". As we have plenty of movies in the second "tier" which are not in the higher "tier" for non tier reasons, such as "similar movies", we have a problem. A similar problem exists when going with a "percentage of" definition.
Everything I'm seeing proposed only has two actual "tiers". Movies that entertain, and movies that don't. Everything else is arbitrary based on criteria which are not tier related.
Therefore it seems like under the current mindframe no single system will work. This needs to be split into true tiers, and a way to add on distinctions in general, be it medals, awards, or what have you.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
So how about this:
Gold medal = old star criteria
Silver medal = old moons if they were used with consistency
Bronze medal = current moons
Vault = current vault
Well, the "milestone" moons weren't really on a scale with the others (as gold/silver/bronze would suggest). They should probably be a separate thing entirely, like A2600 ET might be notable for being the first A2600 run even though it's vaulted, while discovering a new major glitch might be notable and yield an extremely entertaining run.
Joined: 4/17/2010
Posts: 11478
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
Nach wrote:
The word "tier" as used here connotates that something in one grouping is significantly better than something in a lesser grouping. However the way things are being assigned to tiers do not mesh with the word tier at all.
Then we need 3 real tiers:
Notable for being "really good"
Unnotable
Notable for being "really bad"
Definitions of bad are written in the Vault description. Good are runs like 8+, as 8s are given intentionally when the run is in fact "really good". Really bad are 6- runs. I'm not saying that must be automated by rating (if we improve rating guidelines and the weighting system, rating can be taken seriously), but it doesn't really matter, because even if the rating decides, all the rest groups shall still be hand-picked:
Recommended/Stars - all must watch them and learn by heart, TASers and newbies.
First platform publications
AVGN
Console-verified
Notable improvement
Whatever
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
If you guys need to create 3D graphics and flowcharts to explain the tiers system, you can be sure nobody will understand it outside of the community. Even I'm not sure I do after reading this thread.
Personally I don't understand why we should separate movies into tiers in the first place, considering movies already have user ratings and can be sorted as such (at least for 8.0+ ratings). Why have a parallel system? Shouldn't stars and moons be automatically attributed based on that rating?
--
The thing that really bothers me though is submissions with alternate goals. Currently, if the goal is not "beating the game ASAP", it needs to be reviewed for entertainment, which is for me contradictory with the idea of allowing more interesting publications to the site.
My solution would be to put goals under vote separately from submissions. Indeed, as long as the goal itself is considered valuable and its objective is not subjectively measured (like artsy runs), I don't see why such movies wouldn't be automatically accepted.
--
tl:dr version:
- Use existing user ratings instead of tiers
- Publish based on goal value rather than movie value
Joined: 4/17/2010
Posts: 11478
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
After unrejecting, shitty game runs were expected to drown the other runs, that's why the former needed to be separated as vault. All the rest is still debatted as I can see.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
This gives rise to this decision procedure, correct?
I don't think the judging process is as simple. It can't be efficiently automated. I think only the division to popular and unpopular speedruns can be automated, everything else (even Stars) should be manual. So I wouldn't start drawing flowcharts for the process, especially with symmetry in mind! :)
SmashManiac wrote:
If you guys need to create 3D graphics and flowcharts to explain the tiers system, you can be sure nobody will understand it outside of the community.
But these charts are not meant for outsiders, they were made for site maintainers, to let them better understand the system they are employing. Also, I agree that multi-tiered site is more confusing for casual viewers, hence I was suggesting single-tiered system similar to old site system but with smart filters.
This and this are single-tiered system (+Stars, which are not really a tier, but a subset).
This and this are multi-tiered system with arbitrary division principles.
However, I'm not saying the tiers should be reverted. I say, let it be tested for a year or so, and when it becomes obvious the system failed, we can return to the thread.
SmashManiac wrote:
Personally I don't understand why we should separate movies into tiers in the first place, considering movies already have user ratings and can be sorted as such (at least for 8.0+ ratings). Why have a parallel system? Shouldn't stars and moons be automatically attributed based on that rating?
If Stars meaning has changed to "the most entertaining movies" then yes, they can be automatically attributed, but until their meaning is (was) "the showcase of the site content", they have to be manually picked.
IF the Stars were just given by the ratings, then it would be a loss of meaning to both Stars and the rating system.
And IF tiers were merely sorted by entertainment and tech quality ratings, it would be better that we simply find ways to improve our rating system instead of to set up different kinds of tiers.
Recent projects: SMB warpless TAS (2018), SMB warpless walkathon (2019), SMB something never done before (2019), Extra Mario Bros. (best ending) (2020).
The symmetry of the axis graphic is reflected here, which is why I find this layout intuitive.
Sensibly, adelikat wants to give a place to optimal movies that aren't entertaining, and everyone agrees with this. I think the problem is only trying to add on the Vault category leads to imbalance, which is why people are suggesting additional categories to balance it out. So this is the desired layout from the original proposal (if I understand it correctly):
The problem is that people perceived this as ending up in practice as:
Because the categories seem diluted. I think this happens because Star is trying to take on some of the load Moon has from not having Playaround and Demo to help out. Really Star ought to have remained exactly the same subset of movies, and only movies that obsolete more entertaining runs (as well as previously Grued runs) would be reconsidered and moved to Vault. This would have only reduced the size of Standard movies and given Vault its place.
So that's why I think it may be worthwhile to tryin again but with all the organizational categories.
What I think adelikat wants to remind everyone though is that runs aren't being chopped off any more than they were in the past, and it's in fact the opposite. More runs will be accepted now because of Vault. Just compare the above decision procedures with what we used to have:
Nothing is being lost, except IMO the clarity of categories.
