Post subject: New judging policies - Judging time minimum
adelikat
He/Him
Emulator Coder, Site Developer, Site Owner, Expert player (3572)
Joined: 11/3/2004
Posts: 4754
Location: Tennessee
As of now, submissions can not have a judging verdict until a minimum time has passed. Currently that is set at 72 hours (but it easily changed). Prior to this time the only options available to a judge is judging underway and needs for info (actually as of typing this neither of these is available due to a bug). The point of this is to try to curb "quick decisions" which seem to be at the heart of a the majority of judging controversies lately (such as this topic). A judge should make use of the judging underway option to announce they are handling the submission but then let the conversation play itself out before making a decision. There is still a lot a judge can do in this window like setting to judging underway, providing feedback, ask question, and propose possible verdicts. Judging underway has these advantages: For other judges - they know this one is being handled and can focus their attention to other submissions For the author - they know their submission isn't getting neglected, someones is concerned and handling it For the audience - someones paying attention, give helpful feedback! For the admin - submissions that are old and new status are apparently being neglected, no judge has claimed them, better look into this situation. I also have been happy with how Truncated & klmz have both often posted where they are leaning in a post and reasons they feel that way. As Warp posted here, authors should have a chance for rebuttal, especially since the judges conclusions might be incorrect. Anyway, I hope this forced delay helps for a smoother judging process. The time limit was set to a somewhat arbitrary 72 hours. Feel free to use this thread to argue some other value instead.
It's hard to look this good. My TAS projects
Experienced player (961)
Joined: 12/3/2008
Posts: 936
Location: Castle Keep
IMO, there should be 2 (or more?) cases where rejection should not wait 72 hours. -Troll. -Played Realtime (lets say, not a troll, and giving him a second chance).
adelikat
He/Him
Emulator Coder, Site Developer, Site Owner, Expert player (3572)
Joined: 11/3/2004
Posts: 4754
Location: Tennessee
If its truely a troll, perhaps deleting is the better option, there is no time limit for that. As for a realtime run or something obvious, an admin can always intervene, but is it such a big deal for it to sit a few days?
It's hard to look this good. My TAS projects
Editor, Skilled player (1439)
Joined: 3/31/2010
Posts: 2108
While I don't think that premature judging isn't a big problem, this certainly sounds like a good idea to me. I do, however think that the delay might be a bit too long, especially for submissions that have "YES!" written all over them. So, for example, Submisssions that are improvements to published runs could have their delay shortened, since there is a solid base to compare the run to. Alternatively, the votes could be used to factor in how long the delay is, so a highly anticipated movie can be published sooner, although I think I'm making things too complex here.
Player (118)
Joined: 5/13/2009
Posts: 700
Location: suffern, ny
I agree on the troll thing, those submissions should be deleted sooner rather than later. But perhaps we should extend it to 96 hours or 4 days. I remember getting caught up with stuff, coming back to the site, and seeing publications that were submitted and published, before I even got a chance to look at it. I didn't mind much, But I felt a little cheated.
[19:16] <scrimpy> silly portuguese [19:16] <scrimpy> it's like spanish, only less cool
adelikat
He/Him
Emulator Coder, Site Developer, Site Owner, Expert player (3572)
Joined: 11/3/2004
Posts: 4754
Location: Tennessee
scrimpeh wrote:
While I don't think that premature judging isn't a big problem, this certainly sounds like a good idea to me. I do, however think that the delay might be a bit too long, especially for submissions that have "YES!" written all over them. So, for example, Submisssions that are improvements to published runs could have their delay shortened, since there is a solid base to compare the run to. Alternatively, the votes could be used to factor in how long the delay is, so a highly anticipated movie can be published sooner, although I think I'm making things too complex here.
I think these should still wait. Like funnyhair just poitned out, the audience wants there chance to weigh in on it, even if it is obvious. Also if a run is really popular it is nice to see those positive comments (from an author perspective). We've had some big publications that were published immediately and the conversation basically died, and nobody really had a chance to "enjoy the moment".
It's hard to look this good. My TAS projects
Senior Moderator
Joined: 8/4/2005
Posts: 5777
Location: Away
scrimpeh wrote:
While I don't think that premature judging isn't a big problem, this certainly sounds like a good idea to me. I do, however think that the delay might be a bit too long, especially for submissions that have "YES!" written all over them. So, for example, Submisssions that are improvements to published runs could have their delay shortened, since there is a solid base to compare the run to. Alternatively, the votes could be used to factor in how long the delay is, so a highly anticipated movie can be published sooner, although I think I'm making things too complex here.
We've had way too much improvement submissions that themselves had known improvements the authors didn't get a chance to incorporate because the judges/publishers were too hasty with accepting and publishing. That's the reason your suggestion won't work.
Warp wrote:
Edit: I think I understand now: It's my avatar, isn't it? It makes me look angry.
adelikat
He/Him
Emulator Coder, Site Developer, Site Owner, Expert player (3572)
Joined: 11/3/2004
Posts: 4754
Location: Tennessee
moozooh wrote:
scrimpeh wrote:
While I don't think that premature judging isn't a big problem, this certainly sounds like a good idea to me. I do, however think that the delay might be a bit too long, especially for submissions that have "YES!" written all over them. So, for example, Submisssions that are improvements to published runs could have their delay shortened, since there is a solid base to compare the run to. Alternatively, the votes could be used to factor in how long the delay is, so a highly anticipated movie can be published sooner, although I think I'm making things too complex here.
We've had way too much improvement submissions that themselves had known improvements the authors didn't get a chance to incorporate because the judges/publishers were too hasty with accepting and publishing. That's the reason your suggestion won't work.
Yes, case in point: Tempo. Sonikkustar knew I was finishing up that movie and was ready to submit, however, he wasn't around for about 24 hours or so and it was published for he could watch. When he watch, he spotted an improvement that led to this which is something I could have easily redone and updated the submission file.
It's hard to look this good. My TAS projects
Banned User
Joined: 3/10/2004
Posts: 7698
Location: Finland
scrimpeh wrote:
While I don't think that premature judging isn't a big problem, this certainly sounds like a good idea to me. I do, however think that the delay might be a bit too long, especially for submissions that have "YES!" written all over them. So, for example, Submisssions that are improvements to published runs could have their delay shortened, since there is a solid base to compare the run to. Alternatively, the votes could be used to factor in how long the delay is, so a highly anticipated movie can be published sooner, although I think I'm making things too complex here.
I really don't think it harms anybody or anything if even a clear "YES!" submission is published a couple of days later. I don't think anybody would get that impatient. (Besides the vast majority of submissions get pre-encoded to youtube of whatever, so it's not like people would have to wait the 72 hours to see it.) (And besides, it's not completely unheard of that a supposed improvement to an existing run has been rejected, for whatever reason. "Improvement to existing run" is not, and should not, be some kind of "automatically accepted".)