Joined: 4/17/2010
Posts: 11475
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
Sorry Aran. I tried reading a whole lot of your posts, and I simply can't. I keep giving up three lines into a sentence of 10.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Sorry Aran. I tried reading a whole lot of your posts, and I simply can't. I keep giving up three lines into a sentence of 10.
He's asking about combining the rating system with the voting system. (I think it was discussed somewhere before?) This sentence seems to concisely sum it up:
Aran Jaeger wrote:
And my suggestion above with the rating system being put in place of the no/meh/yes poll
Joined: 4/17/2010
Posts: 11475
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
It was tried and miserably failed. I wasn't really around, you'd need to ask the elders what exactly happened.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
It was tried and miserably failed. I wasn't really around, you'd need to ask the elders what exactly happened.
I'm curious about how it failed at the time, and how it would work today.
On paper that seems like a solution that fixes multiple problems at the same time (the best kind of solution). And I can't come up with a serious downside.
As we've seen, people can interpret the "meh" differently (middle ground, don't care, etc...), numerical values don't have that problem.
The other problem is that many movies don't have many ratings after being published. It can help here as well. Maybe it's just me (I'm pretty sure it's not :P ) but I can watch a TAS in the workbench, vote and comment, etc... but then afterwards I don't have the reflex to go and rate the publication. Especially because we have to wait for the movie to be published and can forget about it.
Maybe it was due to a problem with how the site could handle it (or not handle it).
Well... Elders, raise your hands if you have an answer!
Joined: 10/29/2014
Posts: 176
Location: Bavaria, Germany
Actually I can see some potential downside of substituting the no/meh/yes poll with the rating system, or allowing ratings to already be done during the submission phase (so that the ratings will automatically be applied once the TAS is accepted, if it does get accepted). Namely from the perspective of providing premature/overhasty evaluations:
Since it usually takes some time between voting with respect to a submission being done and the time when the submission is accepted and on the site so that one can give it a rating, there's some forced ''cooldown phase'' that might act like a puffer, so that people can do their rating evaluations with a clear head and not maybe be potentially biased towards one direction or the other, depending on various possible factors. So from this perspective, some consideration time about what rating one really would want to provide for a movie can make sense, compared to already providing a rating while the discussion is still going strong or while information relevant to the movie is still being updated/provided, and then later realizing that something actually was different about the movie than what one thought.
But then this would come down to knowing (or being aware of) roughly on what aspects one's own rating is based on, and how certain one is of the stability of those aspects being as one thinks of them, or one could just ask about the uncertain aspects. And then again, the same issue could be a problem for the normal no/meh/yes voting too, except when the impact of some change in how someone views a TAS (due to new information towards its technical quality or entertainment) is low enough, then it might not yet make the difference between yes/meh or meh/no (or even yes/no), whereas it could make already a difference of 1 or 2 or so rating points away from one's initial rating.
However, on the other hand-side, one might want to consider that in the alternative scenario (the current one, where the rating system is separated from submission processes), the costs of keeping it this way might involve that one will be missing out on many more ratings towards movies in general that might have otherwise been made.
collect, analyse, categorise.
"Mathematics - When tool-assisted skills are just not enough" ;)
Don't want to be taking up so much space adding to posts, but might be worth mentioning and letting others know for what games 1) already some TAS work has been done (ordered in decreasing amount, relative to a game completion) by me and 2) I am (in decreasing order) planning/considering to TAS them. Those would majorly be SNES games (if not, it will be indicated in the list) I'm focusing on.
1) Spanky's Quest; On the Ball/Cameltry; Musya; Super R-Type; Plok; Sutte Hakkun; The Wizard of Oz; Battletoads Doubledragon; Super Ghouls'n Ghosts; Firepower 2000; Brain Lord; Warios Woods; Super Turrican; The Humans.
2) Secret Command (SEGA); Star Force (NES); Hyperzone; Aladdin; R-Type 3; Power Blade 2 (NES); Super Turrican 2; First Samurai.
(last updated: 18.03.2018)
Emulator Coder, Site Developer, Site Owner, Expert player
(3573)
Joined: 11/3/2004
Posts: 4754
Location: Tennessee
Grincevent wrote:
As we've seen, people can interpret the "meh" differently (middle ground, don't care, etc...), numerical values don't have that problem.
The other problem is that many movies don't have many ratings after being published. It can help here as well. Maybe it's just me (I'm pretty sure it's not :P ) but I can watch a TAS in the workbench, vote and comment, etc... but then afterwards I don't have the reflex to go and rate the publication.
...
Well... Elders, raise your hands if you have an answer!
It was a failure because the average rating given to a submission was far higher than published movies were getting. A movie that might get a 7.5-8 rating would getting like 9.8 instead due to the submission ratings. Basically if people wanted to see a movie published, they rated it as a 10. It makes sense, as anything less might hurt its chances. As a judge it was harder to get a gauge on what people were feeling, and the rating system de-evolved into a 2 or 3 option system (since people were only rating in extremes anyway), which it was to begin with.
Joined: 10/29/2014
Posts: 176
Location: Bavaria, Germany
Adelikat, was this occurrence in the past such that the usual no/meh/yes poll was not part of submissions anymore and instead the rating system was put in its place, or was it a case in which both types were present, side by side? I assume the former was the case, but considering these points I'm not sure if the latter should be given a chance or not. This probably would be something to be decided by the staff, I think.
Edit: Oh and in this context I also want to point out that in the former case, the audience would be forced to provide such precise rating (for those that do want to contribute to this evaluation process), whereas in the latter case I could see a rating to not be required to be provided by a user, and just be there as an optional additional choice to take or to leave empty.
2nd edit: At least I think that if one would see both parts present at submissions, then this could serve as distinguishing indicator between the purposes of the 2 systems (poll & rating).
3rd edit: Or one could put up some disclaimer that informs evaluators that their ratings aswell as the average rating stays hidden/unknown (to maybe the audience or also including the judge or judges in general) until (at least) the end of the submission (when it was rejected or accepted), so that they might understand that the ratings are not meant to have any influence in the voting process in the way that adelikat described.
collect, analyse, categorise.
"Mathematics - When tool-assisted skills are just not enough" ;)
Don't want to be taking up so much space adding to posts, but might be worth mentioning and letting others know for what games 1) already some TAS work has been done (ordered in decreasing amount, relative to a game completion) by me and 2) I am (in decreasing order) planning/considering to TAS them. Those would majorly be SNES games (if not, it will be indicated in the list) I'm focusing on.
1) Spanky's Quest; On the Ball/Cameltry; Musya; Super R-Type; Plok; Sutte Hakkun; The Wizard of Oz; Battletoads Doubledragon; Super Ghouls'n Ghosts; Firepower 2000; Brain Lord; Warios Woods; Super Turrican; The Humans.
2) Secret Command (SEGA); Star Force (NES); Hyperzone; Aladdin; R-Type 3; Power Blade 2 (NES); Super Turrican 2; First Samurai.
(last updated: 18.03.2018)
As we've seen, people can interpret the "meh" differently (middle ground, don't care, etc...), numerical values don't have that problem.
The other problem is that many movies don't have many ratings after being published. It can help here as well. Maybe it's just me (I'm pretty sure it's not :P ) but I can watch a TAS in the workbench, vote and comment, etc... but then afterwards I don't have the reflex to go and rate the publication.
...
Well... Elders, raise your hands if you have an answer!
It was a failure because the average rating given to a submission was far higher than published movies were getting. A movie that might get a 7.5-8 rating would getting like 9.8 instead due to the submission ratings. Basically if people wanted to see a movie published, they rated it as a 10. It makes sense, as anything less might hurt its chances. As a judge it was harder to get a gauge on what people were feeling, and the rating system de-evolved into a 2 or 3 option system (since people were only rating in extremes anyway), which it was to begin with.
I see. Hum, but it was before the vault tier, right? There is a chance the voting behavior (too high to make sure it's published) would change now, but maybe not by much (too high too if the voter wants it to go in moons, or even get a star).
Aran has some interesting ideas... But I don't know how to merge or at least keep things simple enough without influencing too much the ratings.
Joined: 4/17/2010
Posts: 11475
Location: Lake Chargoggagoggmanchauggagoggchaubunagungamaugg
£e Nécroyeur wrote:
These multiple account shenanigans are just petty.
Why?
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
