Post by Warp

Posted: 12/1/2013 10:04 PM

Post subject: The poll question

Quote

thatguy

Editor

Joined: 11/3/2013
Posts: 506

Right, I know there have been discussions about this before, but it still seems that the poll question is causing confusion. As far as I am aware, the poll question "Did you find this movie entertaining" means "Did you find this movie entertaining?" And yet, in the discussion for the recent "Ganonless" Ocarina of Time submission, I see a comment like this "This is a great TAS, I really loved it... I'm going to have to vote reject", from a user with over 1000 posts no less. Conversely, in the recent ET improvement had comments like "Totally uninteresting, but it looks optimal so meh". From these responses I can see that users are treating the poll question in the following way: Yes = publish to moons Meh = publish to vault No = do not publish Whereas in fact, voters should be responding thus: Yes = Entertained Meh = Can't decide No = Bored Having established the meaning of the poll question, I now want to explore its purpose. Does the poll question decide the fate of a movie? No, a judge does that. Does entertainment ultimately alter a judge's decision? IMO, yes, because in the case of a well-optimised video that does not constitute a technical foul, entertainment value determines the tier in which a video is published. If the video is an any% or 100% run, then entertainment decides whether the run is publishable in moons or in the vault. If it is any other category, it determines whether or not it is publishable at all. Does the judge need to be told whether or not a movie should be published? IMO, no, that's what judges are for. They judge things. In this case, they judge the publication and/or tier of TAS movies. So what's the point in the poll question? Well, it is useful for the judge to be able to gauge the reception of a run, at a glance. Judges should obviously read through the comments to see why the movie is deemed entertaining or dull, and to see any other issues that may be brought up, technical foul, poor goal choice etc. But it is far easier to vote than to comment, so by having a poll question, the judge will get more feedback than he would have otherwise. So that, as I see it, is the system, and it works well as long as people actually answer the question the poll asks them. So here are some questions that are worth discussing: If people are treating the question as "Should this be published", wouldn't it be easier to change the question to this? Which question is more helpful for judges? Could we have two polls, one for publish-worthiness and one for entertainment? Or would that be redundant?

Posted: 12/1/2013 10:31 PM

Quote

Noxxa

They/Them

Moderator, Expert player (4142)

Joined: 8/14/2009
Posts: 4083
Location: The Netherlands

Just for reference - this is the third thread on this matter, and there was more discussion about it before those threads as well. I personally think we should change the poll name to the old "Should this movie be published". Regardless of what the actual poll question is, it seems many people continue to use the poll that way. I occasionally find myself having to remember that the poll is supposed to be for entertainment rather than approval for publication. It seems a bit more intuitive for voting as well. Right now a submission can have all "No"s and 0% rating in the submission list and still be a perfectly fine candidate for publication. Conversely, the system encourages you to say "Yes" even when you don't want to say "yes" to publishing the submission in the first place. It has been mentioned before that there are technical issues with changing the poll, but a poll name change to "Should this movie be published" and poll option name change to "Yes/No/Vault" from "Yes/No/Meh" should not be a big issue, unless the poll options are directly identified by name in the code. Also, as a judge, I find myself gauging entertainment ratings much more by posts than by poll anyway. The poll is not really as reliable, which is made even worse with people misusing it for "should it be published". I find personal viewing experience and posts by other users detailing their viewing experience say a lot more about a run's entertainment value than poll numbers and ratios. Just my two cents.

http://www.youtube.com/Noxxa <dwangoAC> This is a TAS (...). Not suitable for all audiences. May cause undesirable side-effects. May contain emulator abuse. Emulator may be abusive. This product contains glitches known to the state of California to cause egg defects. <Masterjun> I'm just a guy arranging bits in a sequence which could potentially amuse other people looking at these bits <adelikat> In Oregon Trail, I sacrificed my own family to save time. In Star trek, I killed helpless comrades in escape pods to save time. Here, I kill my allies to save time. I think I need help.

Posted: 12/1/2013 10:54 PM

Quote

adelikat

He/Him

Emulator Coder, Site Developer, Site Owner, Expert player (3600)

Joined: 11/3/2004
Posts: 4739
Location: Tennessee

I do not want two poll questions, nor do I want a poll about publishing, as it would have little value. You silent vote of "no this shouldn't be published" has really no weight. Do you understand the criteria for rejection? Is it suboptimal? Is that why you are voting no? I can't guage any of this from a silent vote. Also, rejection is rather non-subjective these days since entertainment value isn't a factor. The simplest way I can put it is this: Vote on entertainment, DISCUSS technical details. Also discuss entertainment, discussion beats a poll any day.

It's hard to look this good. My TAS projects

Posted: 12/1/2013 11:28 PM

Quote

Radiant

Player (26)

Joined: 8/29/2011
Posts: 1206
Location: Amsterdam

Simply adding one line of descriptive text to the poll. would fix this issue.

Indie game developer

Posted: 12/2/2013 1:00 AM

Quote

Mitjitsu

He/Him

Banned User, Experienced player (532)

Joined: 4/24/2006
Posts: 2997

It doesn't matter what the question is, as people will change it to suit their agenda.

Posted: 12/2/2013 4:40 PM

Quote

OmnipotentEntity

He/Him

Player (36)

Joined: 9/11/2004
Posts: 2624

Radiant wrote:

Simply adding one line of descriptive text to the poll. would fix this issue.

That line has been there since 2006(?)

Build a man a fire, warm him for a day, Set a man on fire, warm him for the rest of his life.

Posted: 12/2/2013 4:58 PM

Quote

Radiant

Player (26)

Joined: 8/29/2011
Posts: 1206
Location: Amsterdam

OmnipotentEntity wrote:

That line has been there since 2006(?)

"Adding" as in "putting a bit MORE text on top of the poll than what is currently there". Because clearly, numerous users misinterpret the current wording.

Indie game developer

Posted: 12/2/2013 5:01 PM

Quote

feos

Site Admin, Skilled player (1237)

Joined: 4/17/2010
Posts: 11276
Location: RU

Radiant wrote:

Because clearly, numerous users misinterpret the current wording.

So you're saying, they interpret "Did you find this movie entertaining?" as something else due to bad wording? I'd rather call it ignorance.

Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.

Posted: 12/2/2013 5:23 PM

Quote

Radiant

Player (26)

Joined: 8/29/2011
Posts: 1206
Location: Amsterdam

feos wrote:

So you're saying, they interpret "Did you find this movie entertaining?" as something else due to bad wording? I'd rather call it ignorance.

People tend to interpret it as "Did you find it entertaining? If enough people vote yes, it will be published; if not, it won't be". That is because the thread is about whether the movie gets published; since the vote doesn't explain how it ties into that, it is only natural for people to assume that it means "yes vote = publish, no vote = do not publish". And you are correct that this is ignorance. I think the best way to deal with this ignorance is to educate the voters by adding a bit of explanatory text to the poll.

Indie game developer

Posted: 12/2/2013 9:38 PM

Quote

Warp

Banned User, Former player

Joined: 3/10/2004
Posts: 7698
Location: Finland

How about simply giving up and just changing the questions to what people want to answer, not what you want them to be? Something like: Should this be published? * Yes, and I liked it. * Yes, although I didn't like it much. * No, there were problems with it.

Posted: 12/2/2013 9:50 PM

Quote

MrGrunz

Experienced player (599)

Joined: 2/8/2009
Posts: 656

Mitjitsu wrote:

It doesn't matter what the question is, as people will change it to suit their agenda.

Because this is more true than anyone wants to admit, I think Warp's idea is really good:

Warp wrote:

How about simply giving up and just changing the questions to what people want to answer, not what you want them to be? Something like: Should this be published? * Yes, and I liked it. * Yes, although I didn't like it much. * No, there were problems with it.

Don't focus on the question, but the answers. Yes, Meh, No always have the mentioned probably, no matter what the question above them is. We should still try to keep the number of possible answers rather low, though (if we really do this), because it would get way too confusing then. 3-5 are probably good numbers.

Posted: 12/3/2013 12:33 PM

Quote

Noxxa

They/Them

Moderator, Expert player (4142)

Joined: 8/14/2009
Posts: 4083
Location: The Netherlands

I don't think more than 3 answer options are needed. I've suggested changing it to something simple like Yes/No/Vault instead of Yes/No/Meh. Warp's idea is fairly similar, if more verbose. As a side note, having more than 3 answer options will very definitely run into site code issues.

http://www.youtube.com/Noxxa <dwangoAC> This is a TAS (...). Not suitable for all audiences. May cause undesirable side-effects. May contain emulator abuse. Emulator may be abusive. This product contains glitches known to the state of California to cause egg defects. <Masterjun> I'm just a guy arranging bits in a sequence which could potentially amuse other people looking at these bits <adelikat> In Oregon Trail, I sacrificed my own family to save time. In Star trek, I killed helpless comrades in escape pods to save time. Here, I kill my allies to save time. I think I need help.

Posted: 12/3/2013 1:17 PM

Quote

Spikestuff

They/Them

Editor, Publisher, Expert player (2312)

Joined: 10/12/2011
Posts: 6344
Location: The land down under.

Mothrayas wrote:

I don't think more than 3 answer options are needed. I've suggested changing it to something simple like Yes/No/Vault.

That idea is the Idea I like the most. Only that "Did you find this movie entertaining" changed to something else.

WebNations/Sabih wrote:

+fsvgm777 never censoring anything.

Disables Comments and Ratings for the YouTube account. Something better for yourself and also others.

Posted: 12/3/2013 6:19 PM

Quote

Derakon

Joined: 7/2/2007
Posts: 3960

Mothrayas wrote:

I don't think more than 3 answer options are needed. I've suggested changing it to something simple like Yes/No/Vault instead of Yes/No/Meh. Warp's idea is fairly similar, if more verbose.

If you're going to have a "Vault" option, then the options ought to be Moon/Vault/No.

Pyrel - an open-source rewrite of the Angband roguelike game in Python.

Posted: 12/3/2013 6:24 PM

Quote

Radiant

Player (26)

Joined: 8/29/2011
Posts: 1206
Location: Amsterdam

Isn't the point of the Vault that movies that fit its criteria will go there regardless of votes?

Indie game developer

Posted: 12/3/2013 6:47 PM

Quote

Derakon

Joined: 7/2/2007
Posts: 3960

Right, a vote for Vault would be "I don't find this entertaining, but it's technically proficient enough to be published".

Pyrel - an open-source rewrite of the Angband roguelike game in Python.

Posted: 12/3/2013 7:00 PM

Quote

Radiant

Player (26)

Joined: 8/29/2011
Posts: 1206
Location: Amsterdam

Yes, but since popularity is not one of the criteria for the Vault, in practice it doesn't matter if people vote "vault" or "no" on a run, because if it fits Vault criteria it would be published there regardless of votes.

Indie game developer

Posted: 12/3/2013 7:45 PM

Quote

Derakon

Joined: 7/2/2007
Posts: 3960

The problem being that people want to vote even if their vote is meaningless, so we give them an option to register their opinion even though the count for that option will be ignored. It serves the same purpose as the "Meh" vote did, but more explicitly. To look at it another way, a "Vault" vote is a vote for technical proficiency and would help the judges tell at a glance if the run "meets Vault criteria".

Pyrel - an open-source rewrite of the Angband roguelike game in Python.

Posted: 12/3/2013 8:18 PM

Quote

Radiant

Player (26)

Joined: 8/29/2011
Posts: 1206
Location: Amsterdam

I think that people who misunderstand the current setup will misunderstand it even more if you add an important-sounding option that quietly gets ignored by the judges. It would be much clearer to educate the confused users, by adding to The Question's text that "This vote is about Moon Tier. A run can be published in Vault Tier regardless of its entertainment."

Indie game developer

Posted: 12/3/2013 9:38 PM

Quote

RachelB

She/Her

Player (127)

Joined: 12/3/2011
Posts: 1579

Honestly, do we even need the poll anymore? What is it even used for that the discussion doesn't do better?

Posted: 12/4/2013 4:10 AM

Quote

feos

Site Admin, Skilled player (1237)

Joined: 4/17/2010
Posts: 11276
Location: RU

Radiant wrote:

It would be much clearer to educate the confused users, by adding to The Question's text that "This vote is about Moon Tier. A run can be published in Vault Tier regardless of its entertainment."

I'd prefer "This vote is to register how much you were entertained (surprise). About technical value, post in the thread"

RachelB wrote:

Honestly, do we even need the poll anymore? What is it even used for that the discussion doesn't do better?

It shows when there's huge dislike of something. http://tasvideos.org/forum/viewtopic.php?t=14634

Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.

Posted: 12/4/2013 5:26 PM

Quote

RachelB

She/Her

Player (127)

Joined: 12/3/2011
Posts: 1579

feos wrote:

RachelB wrote:
Honestly, do we even need the poll anymore? What is it even used for that the discussion doesn't do better?
It shows when there's huge dislike of something. http://tasvideos.org/forum/viewtopic.php?t=14634

Does it really change anything even then though? The comments make it pretty clear that people don't like it for a variety of reasons.

Posted: 12/4/2013 5:30 PM

Quote

feos

Site Admin, Skilled player (1237)

Joined: 4/17/2010
Posts: 11276
Location: RU

It varies. Sometimes there can be little to no comments on entertainment, but 10 Yes votes with 2 Noes. Sometimes comments vary quite well, and the votes are still rather positive. And there are cases when comments are also mixed and the votes are negative. So everything works really well helping to make a decision. I mean, the current system as it is intended.

Warning: When making decisions, I try to collect as much data as possible before actually deciding. I try to abstract away and see the principles behind real world events and people's opinions. I try to generalize them and turn into something clear and reusable. I hate depending on unpredictable and having to make lottery guesses. Any problem can be solved by systems thinking and acting.

Posted: 12/4/2013 7:10 PM

Post subject: "lol"

Quote

alden

Player (120)

Joined: 2/11/2007
Posts: 1522

The utility of the poll question is to feed the ego of those who submit popular movies, to shame those who submit unpopular ones, and for everyone else, to provide another excuse to argue.

I make a comic with no image files and you should read it. While there is a lower class, I am in it, and while there is a criminal element I am of it, and while there is a soul in prison, I am not free. -Eugene Debs