1 2
Posted: 12/1/2013 10:04 PM
Post subject: The poll question
Quote
thatguy
Editor
Joined: 11/3/2013
Posts: 506
Right, I know there have been discussions about this before, but it still seems that the poll question is causing confusion. As far as I am aware, the poll question "Did you find this movie entertaining" means "Did you find this movie entertaining?" And yet, in the discussion for the recent "Ganonless" Ocarina of Time submission, I see a comment like this "This is a great TAS, I really loved it... I'm going to have to vote reject", from a user with over 1000 posts no less. Conversely, in the recent ET improvement had comments like "Totally uninteresting, but it looks optimal so meh". From these responses I can see that users are treating the poll question in the following way: Yes = publish to moons Meh = publish to vault No = do not publish Whereas in fact, voters should be responding thus: Yes = Entertained Meh = Can't decide No = Bored Having established the meaning of the poll question, I now want to explore its purpose. Does the poll question decide the fate of a movie? No, a judge does that. Does entertainment ultimately alter a judge's decision? IMO, yes, because in the case of a well-optimised video that does not constitute a technical foul, entertainment value determines the tier in which a video is published. If the video is an any% or 100% run, then entertainment decides whether the run is publishable in moons or in the vault. If it is any other category, it determines whether or not it is publishable at all. Does the judge need to be told whether or not a movie should be published? IMO, no, that's what judges are for. They judge things. In this case, they judge the publication and/or tier of TAS movies. So what's the point in the poll question? Well, it is useful for the judge to be able to gauge the reception of a run, at a glance. Judges should obviously read through the comments to see why the movie is deemed entertaining or dull, and to see any other issues that may be brought up, technical foul, poor goal choice etc. But it is far easier to vote than to comment, so by having a poll question, the judge will get more feedback than he would have otherwise. So that, as I see it, is the system, and it works well as long as people actually answer the question the poll asks them. So here are some questions that are worth discussing: If people are treating the question as "Should this be published", wouldn't it be easier to change the question to this? Which question is more helpful for judges? Could we have two polls, one for publish-worthiness and one for entertainment? Or would that be redundant?
Posted: 12/1/2013 10:31 PM
Quote
Noxxa
They/Them
Moderator, Expert player (4110)
Joined: 8/14/2009
Posts: 4089
Location: The Netherlands
Exotic platforms TASer of 2018 Exotic platforms TAS of 2018 Exotic platforms TASer of 2017 Exotic platforms TAS of 2017 Exotic platforms TAS of 2017 Exotic platforms TASer of 2016 Exotic platforms TAS of 2016 Gameboy TASer of 2013
Just for reference - this is the third thread on this matter, and there was more discussion about it before those threads as well. I personally think we should change the poll name to the old "Should this movie be published". Regardless of what the actual poll question is, it seems many people continue to use the poll that way. I occasionally find myself having to remember that the poll is supposed to be for entertainment rather than approval for publication. It seems a bit more intuitive for voting as well. Right now a submission can have all "No"s and 0% rating in the submission list and still be a perfectly fine candidate for publication. Conversely, the system encourages you to say "Yes" even when you don't want to say "yes" to publishing the submission in the first place. It has been mentioned before that there are technical issues with changing the poll, but a poll name change to "Should this movie be published" and poll option name change to "Yes/No/Vault" from "Yes/No/Meh" should not be a big issue, unless the poll options are directly identified by name in the code. Also, as a judge, I find myself gauging entertainment ratings much more by posts than by poll anyway. The poll is not really as reliable, which is made even worse with people misusing it for "should it be published". I find personal viewing experience and posts by other users detailing their viewing experience say a lot more about a run's entertainment value than poll numbers and ratios. Just my two cents.
http://www.youtube.com/Noxxa <dwangoAC> This is a TAS (...). Not suitable for all audiences. May cause undesirable side-effects. May contain emulator abuse. Emulator may be abusive. This product contains glitches known to the state of California to cause egg defects. <Masterjun> I'm just a guy arranging bits in a sequence which could potentially amuse other people looking at these bits <adelikat> In Oregon Trail, I sacrificed my own family to save time. In Star trek, I killed helpless comrades in escape pods to save time. Here, I kill my allies to save time. I think I need help.
Posted: 12/1/2013 10:54 PM
Quote
adelikat
He/Him
Emulator Coder, Site Developer, Site Owner, Expert player (3571)
Joined: 11/3/2004
Posts: 4754
Location: Tennessee
TASer of 2010 NES TASer of 2010 DS TASer of 2010 Arcade TAS of 2010 NES TASer of 2008 NES TAS of 2008 Funny TAS of 2008 TASer of 2007 NES TASer of 2007 Funny TAS of 2007
I do not want two poll questions, nor do I want a poll about publishing, as it would have little value. You silent vote of "no this shouldn't be published" has really no weight. Do you understand the criteria for rejection? Is it suboptimal? Is that why you are voting no? I can't guage any of this from a silent vote. Also, rejection is rather non-subjective these days since entertainment value isn't a factor. The simplest way I can put it is this: Vote on entertainment, DISCUSS technical details. Also discuss entertainment, discussion beats a poll any day.
It's hard to look this good. My TAS projects
Posted: 12/1/2013 11:28 PM
Quote
Radiant
Player (26)
Joined: 8/29/2011
Posts: 1206
Location: Amsterdam
Simply adding one line of descriptive text to the poll. would fix this issue.
Indie game developer
Posted: 12/2/2013 1:00 AM
Quote
Mitjitsu
He/Him
Banned User
Joined: 4/24/2006
Posts: 2997
TAS of 2007 N64 TAS of 2007
It doesn't matter what the question is, as people will change it to suit their agenda.
Posted: 12/2/2013 4:36 PM
Quote
Warp
Banned User
Joined: 3/10/2004
Posts: 7698
Location: Finland
adelikat wrote:
Vote on entertainment, DISCUSS technical details. Also discuss entertainment, discussion beats a poll any day.
I think you (plural) should finally awaken to the fact that no matter how many times you repeat that, people will still find it unnatural to vote "yes" on a submission that they feel is not fit for publication (eg. because its goal is unsuitable, or it doesn't properly beat the previous publication) or "no" on a submission that they don't have any objection to being published (even though they personally didn't find it very entertaining.) Please be a bit more pragmatic about this. Listen to the people.
Posted: 12/2/2013 4:40 PM
Quote
OmnipotentEntity
He/Him
Player (36)
Joined: 9/11/2004
Posts: 2630
Radiant wrote:
Simply adding one line of descriptive text to the poll. would fix this issue.
That line has been there since 2006(?)
Build a man a fire, warm him for a day, Set a man on fire, warm him for the rest of his life.
Posted: 12/2/2013 4:58 PM
Quote
Radiant
Player (26)
Joined: 8/29/2011
Posts: 1206
Location: Amsterdam
OmnipotentEntity wrote:
That line has been there since 2006(?)
"Adding" as in "putting a bit MORE text on top of the poll than what is currently there". Because clearly, numerous users misinterpret the current wording.
Indie game developer
Posted: 12/2/2013 5:01 PM
Quote
feos
Site Admin, Skilled player (1251)
Joined: 4/17/2010
Posts: 11475
Location: Lake Char­gogg­a­gogg­man­chaugg­a­gogg­chau­bun­a­gung­a­maugg
NES TAS of 2011
Radiant wrote:
Because clearly, numerous users misinterpret the current wording.
So you're saying, they interpret "Did you find this movie entertaining?" as something else due to bad wording? I'd rather call it ignorance.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Posted: 12/2/2013 5:23 PM
Quote
Radiant
Player (26)
Joined: 8/29/2011
Posts: 1206
Location: Amsterdam
feos wrote:
So you're saying, they interpret "Did you find this movie entertaining?" as something else due to bad wording? I'd rather call it ignorance.
People tend to interpret it as "Did you find it entertaining? If enough people vote yes, it will be published; if not, it won't be". That is because the thread is about whether the movie gets published; since the vote doesn't explain how it ties into that, it is only natural for people to assume that it means "yes vote = publish, no vote = do not publish". And you are correct that this is ignorance. I think the best way to deal with this ignorance is to educate the voters by adding a bit of explanatory text to the poll.
Indie game developer
Posted: 12/2/2013 9:38 PM
Quote
Warp
Banned User
Joined: 3/10/2004
Posts: 7698
Location: Finland
How about simply giving up and just changing the questions to what people want to answer, not what you want them to be? Something like: Should this be published? * Yes, and I liked it. * Yes, although I didn't like it much. * No, there were problems with it.
Posted: 12/2/2013 9:50 PM
Quote
MrGrunz
Experienced player (603)
Joined: 2/8/2009
Posts: 656
Nintendo 3D TAS of 2014 N64/Wii TASer of 2011 TAS of 2011 N64/Wii TAS of 2011
Mitjitsu wrote:
It doesn't matter what the question is, as people will change it to suit their agenda.
Because this is more true than anyone wants to admit, I think Warp's idea is really good:
Warp wrote:
How about simply giving up and just changing the questions to what people want to answer, not what you want them to be? Something like: Should this be published? * Yes, and I liked it. * Yes, although I didn't like it much. * No, there were problems with it.
Don't focus on the question, but the answers. Yes, Meh, No always have the mentioned probably, no matter what the question above them is. We should still try to keep the number of possible answers rather low, though (if we really do this), because it would get way too confusing then. 3-5 are probably good numbers.
Posted: 12/3/2013 12:33 PM
Quote
Noxxa
They/Them
Moderator, Expert player (4110)
Joined: 8/14/2009
Posts: 4089
Location: The Netherlands
Exotic platforms TASer of 2018 Exotic platforms TAS of 2018 Exotic platforms TASer of 2017 Exotic platforms TAS of 2017 Exotic platforms TAS of 2017 Exotic platforms TASer of 2016 Exotic platforms TAS of 2016 Gameboy TASer of 2013
I don't think more than 3 answer options are needed. I've suggested changing it to something simple like Yes/No/Vault instead of Yes/No/Meh. Warp's idea is fairly similar, if more verbose. As a side note, having more than 3 answer options will very definitely run into site code issues.
http://www.youtube.com/Noxxa <dwangoAC> This is a TAS (...). Not suitable for all audiences. May cause undesirable side-effects. May contain emulator abuse. Emulator may be abusive. This product contains glitches known to the state of California to cause egg defects. <Masterjun> I'm just a guy arranging bits in a sequence which could potentially amuse other people looking at these bits <adelikat> In Oregon Trail, I sacrificed my own family to save time. In Star trek, I killed helpless comrades in escape pods to save time. Here, I kill my allies to save time. I think I need help.
Posted: 12/3/2013 1:17 PM
Quote
Spikestuff
They/Them
Editor, Publisher, Expert player (2633)
Joined: 10/12/2011
Posts: 6437
Location: The land down under.
Flash TASer of 2022 PSX TASer of 2016
Mothrayas wrote:
I don't think more than 3 answer options are needed. I've suggested changing it to something simple like Yes/No/Vault.
That idea is the Idea I like the most. Only that "Did you find this movie entertaining" changed to something else.
WebNations/Sabih wrote:
+fsvgm777 never censoring anything.
Disables Comments and Ratings for the YouTube account. Something better for yourself and also others.
Posted: 12/3/2013 6:19 PM
Quote
Derakon
Joined: 7/2/2007
Posts: 3960
Mothrayas wrote:
I don't think more than 3 answer options are needed. I've suggested changing it to something simple like Yes/No/Vault instead of Yes/No/Meh. Warp's idea is fairly similar, if more verbose.
If you're going to have a "Vault" option, then the options ought to be Moon/Vault/No.
Pyrel - an open-source rewrite of the Angband roguelike game in Python.
Posted: 12/3/2013 6:24 PM
Quote
Radiant
Player (26)
Joined: 8/29/2011
Posts: 1206
Location: Amsterdam
Isn't the point of the Vault that movies that fit its criteria will go there regardless of votes?
Indie game developer
Posted: 12/3/2013 6:47 PM
Quote
Derakon
Joined: 7/2/2007
Posts: 3960
Right, a vote for Vault would be "I don't find this entertaining, but it's technically proficient enough to be published".
Pyrel - an open-source rewrite of the Angband roguelike game in Python.
Posted: 12/3/2013 7:00 PM
Quote
Radiant
Player (26)
Joined: 8/29/2011
Posts: 1206
Location: Amsterdam
Yes, but since popularity is not one of the criteria for the Vault, in practice it doesn't matter if people vote "vault" or "no" on a run, because if it fits Vault criteria it would be published there regardless of votes.
Indie game developer
Posted: 12/3/2013 7:45 PM
Quote
Derakon
Joined: 7/2/2007
Posts: 3960
The problem being that people want to vote even if their vote is meaningless, so we give them an option to register their opinion even though the count for that option will be ignored. It serves the same purpose as the "Meh" vote did, but more explicitly. To look at it another way, a "Vault" vote is a vote for technical proficiency and would help the judges tell at a glance if the run "meets Vault criteria".
Pyrel - an open-source rewrite of the Angband roguelike game in Python.
Posted: 12/3/2013 8:18 PM
Quote
Radiant
Player (26)
Joined: 8/29/2011
Posts: 1206
Location: Amsterdam
I think that people who misunderstand the current setup will misunderstand it even more if you add an important-sounding option that quietly gets ignored by the judges. It would be much clearer to educate the confused users, by adding to The Question's text that "This vote is about Moon Tier. A run can be published in Vault Tier regardless of its entertainment."
Indie game developer
Posted: 12/3/2013 9:38 PM
Quote
RachelB
She/Her
Player (129)
Joined: 12/3/2011
Posts: 1579
Honestly, do we even need the poll anymore? What is it even used for that the discussion doesn't do better?
Posted: 12/4/2013 4:10 AM
Quote
feos
Site Admin, Skilled player (1251)
Joined: 4/17/2010
Posts: 11475
Location: Lake Char­gogg­a­gogg­man­chaugg­a­gogg­chau­bun­a­gung­a­maugg
NES TAS of 2011
Radiant wrote:
It would be much clearer to educate the confused users, by adding to The Question's text that "This vote is about Moon Tier. A run can be published in Vault Tier regardless of its entertainment."
I'd prefer "This vote is to register how much you were entertained (surprise). About technical value, post in the thread"
RachelB wrote:
Honestly, do we even need the poll anymore? What is it even used for that the discussion doesn't do better?
It shows when there's huge dislike of something. http://tasvideos.org/forum/viewtopic.php?t=14634
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Posted: 12/4/2013 5:26 PM
Quote
RachelB
She/Her
Player (129)
Joined: 12/3/2011
Posts: 1579
feos wrote:
RachelB wrote:
Honestly, do we even need the poll anymore? What is it even used for that the discussion doesn't do better?
It shows when there's huge dislike of something. http://tasvideos.org/forum/viewtopic.php?t=14634
Does it really change anything even then though? The comments make it pretty clear that people don't like it for a variety of reasons.
Posted: 12/4/2013 5:30 PM
Quote
feos
Site Admin, Skilled player (1251)
Joined: 4/17/2010
Posts: 11475
Location: Lake Char­gogg­a­gogg­man­chaugg­a­gogg­chau­bun­a­gung­a­maugg
NES TAS of 2011
It varies. Sometimes there can be little to no comments on entertainment, but 10 Yes votes with 2 Noes. Sometimes comments vary quite well, and the votes are still rather positive. And there are cases when comments are also mixed and the votes are negative. So everything works really well helping to make a decision. I mean, the current system as it is intended.
Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.
Posted: 12/4/2013 7:10 PM
Post subject: "lol"
Quote
alden
Player (121)
Joined: 2/11/2007
Posts: 1522
The utility of the poll question is to feed the ego of those who submit popular movies, to shame those who submit unpopular ones, and for everyone else, to provide another excuse to argue.
I make a comic with no image files and you should read it. While there is a lower class, I am in it, and while there is a criminal element I am of it, and while there is a soul in prison, I am not free. -Eugene Debs
1 2