Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Warp:
At the end of the day it matters what the judges think. Votes and comments are only there to help a judge who isn't sure. If a judge feels it should or shouldn't be published, he has the power to mark the movie appropriately, irregardless of how long its been there.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Warp:
At the end of the day it matters what the judges think. Votes and comments are only there to help a judge who isn't sure. If a judge feels it should or shouldn't be published, he has the power to mark the movie appropriately, irregardless of how long its been there.
You make it sound like judges don't really care or appreciate people's comments and votes (and only check them if they have some doubts about the movie). I'm not sure that's exactly what you meant.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Warp wrote:
Nach wrote:
Warp:
At the end of the day it matters what the judges think. Votes and comments are only there to help a judge who isn't sure. If a judge feels it should or shouldn't be published, he has the power to mark the movie appropriately, irregardless of how long its been there.
You make it sound like judges don't really care or appreciate people's comments and votes (and only check them if they have some doubts about the movie). I'm not sure that's exactly what you meant.
I mean that if a judge feels confident he knows the correct decision for a particular run, he doesn't have to wait for votes or comments.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Emulator Coder, Site Developer, Site Owner, Expert player
(3573)
Joined: 11/3/2004
Posts: 4754
Location: Tennessee
Warp wrote:
Is that really something to aim for?
One time for at least a few minutes, YES. It has been my goal for 4 years and I haven't managed to achieve it.
Movies are submitted on a more or less steady pace (which is a great thing), and optimally they would stay on the submission queue long enough for people to have time to vote and comment on them with enough detail. Rushing movies to publication or rejection for the sole reason of keeping the queue at 0 would only mean that people don't have that much time to comment and vote on them, which I don't see as such a good thing.
You are stirring up trouble here without even looking at the facts. Every movie was judged adequately. The audience had their say and the judgement was largely based on that. At least look at the details before you decide it satisfies your thesis topic.
You are stirring up trouble here without even looking at the facts.
I can completely honestly say that I was not trying to criticize or attack anybody here. It was a legit and innocent question. I was just thinking that "getting the queue to 0" was some kind of ubiquitous goal that everyone is somehow taking for granted, and was wondering why this is so, when you think about it.
I apologize if I inadvertently sounded like attacking someone.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
While we're on this topic...
From IRC:
<Nach> if a judge watches a movie immediately after submission, and it kicks the old one in every way, and after watching it, he has to scrape his jaw off the floor and find where his eyeballs rolled to, I don't see what the problem is with him accepting it before anyone else weighed in
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
While we're on this topic...
From IRC:
<Nach> if a judge watches a movie immediately after submission, and it kicks the old one in every way, and after watching it, he has to scrape his jaw off the floor and find where his eyeballs rolled to, I don't see what the problem is with him accepting it before anyone else weighed in
One could come up with scenarios where rushing even such a submission into publication can cause needless work later. Maybe the judge missed something, such as the submission breaking the rules, something which people commenting on the submission might point out. (I'm not saying that this precise scenario would be very likely, but you get the idea.)
<Nach> if a judge watches a movie immediately after submission, and it kicks the old one in every way, and after watching it, he has to scrape his jaw off the floor and find where his eyeballs rolled to, I don't see what the problem is with him accepting it before anyone else weighed in
I remember at least 3 or 4 submissions (some of them by Aqfaq) which had pending or last-minute improvements by the author that couldn't make it in only due to hasty acceptance and publishing. There must always be a window of 2-3 days for at least that purpose.
Site visitors are able to watch new submissions instantly via key input movies, and very soon (usually within 24 hours) via pre-publication encodes, so we aren't really holding anybody off or preventing an access to the content by not doing everything as fast as possible. In my opinion, the only purpose that serves is a feeling of satisfaction with achieving an arbitrary goal of seeing no submissions in queue or something else like that. Which is rather bogus (since the site lives by its submissions), but not really any productive.
Warp wrote:
Edit: I think I understand now: It's my avatar, isn't it? It makes me look angry.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Newcomers won't have a clue what do with key press files, nor want to bother with the hassle. Nor are preencodes necessarily common or easily accessible.
The point isn't an empty queue, but quick updates to new content.
We don't have to cry about something being published within 2 days.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
IMO what should be avoided is having a submission stay in the queue for months even though everybody stopped talking about it a couple of days after it was submitted. I think that has been the problem with some submissions in the past, and it's good that something is actively done in order to avoid those cases.
However, going the other extreme and publishing something within hours of submitting it might be going a bit too far (even if it's crystal-clear to the judge that the submission is valid and publish-worthy). Let people comment and vote on it, if for nothing else, as a token of appreciation that some people still care to watch the submissions and comment on them. Also, as moozooh well put it, give the author a few days to possibly realize some last-minute optimization which he discovered after submitting.
I don't think submissions should be rushed because of newcomers. If they don't have a clue what to do with movie files nor can find pre-encodes, they probably won't be none the wiser about the fact that there were a few days between submission and publication either. I don't think anybody would complain about that.
We don't have to cry about something being published within 2 days.
I like how your point went from the original quote's "before anyone else weigh in" to the current "within 2 days". Two days is, as I said above, reasonable, and a lot of people can weigh in during this time, contrary to what you've said. Now you're retroactively changing your statement and use it to ridicule my argument, nice job. :D
Nach wrote:
Newcomers won't have a clue what do with key press files, nor want to bother with the hassle. Nor are preencodes necessarily common or easily accessible.
I also don't understand how, by your words, the newcomers are neither going to bother with key input movies, nor skim the thread for encodes and streaming media links, yet the very same newcomers will be very upset to not find newest submissions published as soon as possible, or even be able to find them in the first place. How much of this is pulled from thin air?
On the other hand, I could give at least one example of a rushed publication where the improvements didn't have a chance to be incorporated in time and stayed unimplemented for months because the author didn't want to make another submission.
(That's assuming you still remember what particular statement I was challenging.)
Nach wrote:
The point isn't an empty queue, but quick updates to new content.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
moozooh wrote:
Nach wrote:
We don't have to cry about something being published within 2 days.
I like how your point went from the original quote's "before anyone else weigh in" to the current "within 2 days". Two days is, as I said above, reasonable, and a lot of people can weigh in during this time, contrary to what you've said. Now you're retroactively changing your statement and use it to ridicule my argument, nice job. :D
I'm using your supplied time interval and saying less than that is fine.
Anyone at any time can realize a small improvement. It may be 3 hours after the movie was submitted, 3 days, 3 weeks, 3 months, or 3 years. How long should we wait to publish?
Also, once accepted, it doesn't have to be encoded immediately, there's still time for someone to say something. Encoding itself also adds a few hours for longer videos.
I'd find several cases where the judge judged incorrectly and use that as a point for building up a certain minimum time instead of saying sometimes we rush things. I don't believe anything is rushed if a judge judged a video. This is not life and death, decisions don't need to be slept on.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
I'm leaning towards agreeing with Moozooh, for one big reason: the workbench is the point at which runs finally get significant public scrutiny. Even our highly-skilled TASers make mistakes or are unaware of potential glitches. I can't count the number of times when I've seen a run get lots of positive feedback on the workbench, and then someone comes along and says "Hmm, I bet you could do this segment faster by doing this" and it turns out they're right. Now, I haven't done any kind of formal study, but I wouldn't be surprised if at least some of these have happened more than two days after the run was submitted.
There's several reasons why it's desirable to incorporate these improvements into the just-submitted run rather than wait for the next run:
* There's a TASer working on the run right now. He/she has the energy and desire to improve it, and can do so more easily than anyone else at that time, having all the knowledge related to TASing the game fresh in memory.
* It encourages people to give the runs more scrutiny. Suggesting improvements is one of the best parts of member interaction here. If we don't give enough time between submission and publication for this to happen, we're discouraging that interaction.
* It's more satisfying for many of our members if the runs we publish are the best we are currently capable of producing. Publishing a run with known significant improvements, then, is a bit rankling.
I'm sure there's more reasons too, but this should be enough. For my part, if I were to look for a "minimum time on workbench", I'd say at least two working days and two non-working days, since many people get their internet access primarily only on weekends or only at work.
Pyrel - an open-source rewrite of the Angband roguelike game in Python.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Derakon wrote:
* There's a TASer working on the run right now. He/she has the energy and desire to improve it, and can do so more easily than anyone else at that time, having all the knowledge related to TASing the game fresh in memory.
Yes, and many don't want to go back and redo a lot just for a small savings. The odds of savings are more likely to be not near the end than near the end.
Derakon wrote:
* It encourages people to give the runs more scrutiny. Suggesting improvements is one of the best parts of member interaction here. If we don't give enough time between submission and publication for this to happen, we're discouraging that interaction.
I have no hard data on that. I think people can scrutinize a run after it is published just fine. In fact, even more people can do so.
Derakon wrote:
* It's more satisfying for many of our members if the runs we publish are the best we are currently capable of producing. Publishing a run with known significant improvements, then, is a bit rankling.
We already do publish runs now and then are recognized not to be perfect.
Even if we rejected all those, if you were correct, then why isn't every movie of at least of recent publication getting a perfect 10 in the technical ratings? We don't even have our NES SMB runs getting perfect 10s in technical when at the time of publishing, no one knows how to do it better, or even if they did, won't save more than a frame or two.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
* There's a TASer working on the run right now. He/she has the energy and desire to improve it, and can do so more easily than anyone else at that time, having all the knowledge related to TASing the game fresh in memory.
Yes, and many don't want to go back and redo a lot just for a small savings. The odds of savings are more likely to be not near the end than near the end.
There's no requirement that the TASer go back and make suggested improvements, but in my experience, most do unless the improvements are minor compared to the scope of the movie (saving two frames is often worthwhile on a 5-minute movie, less so on a 30-minute movie). Some people aren't willing to touch the movie once it's submitted, of course, and they aren't required to do so. I don't see how that affects my argument, though.
My big concern here, though, is that we rush a movie to publication, an improvement is found, and the TASer complains that they didn't get the opportunity to incorporate that improvement before the movie got yanked out of their hands. The amount of effort involved in making a new publish-worthy movie to obsolete an old one is significant, since generally we frown on "this movie is identical to the previous one except at this point" submissions (And yes, there are exceptions to that; I'm making a broad generalization).
Nach wrote:
Derakon wrote:
* It encourages people to give the runs more scrutiny. Suggesting improvements is one of the best parts of member interaction here. If we don't give enough time between submission and publication for this to happen, we're discouraging that interaction.
I have no hard data on that. I think people can scrutinize a run after it is published just fine. In fact, even more people can do so.
Again, speaking just from my experience and without any hard numbers to back it up, I see a lot more scrutiny on movies before they are published, not after.
Nach wrote:
Derakon wrote:
* It's more satisfying for many of our members if the runs we publish are the best we are currently capable of producing. Publishing a run with known significant improvements, then, is a bit rankling.
We already do publish runs now and then are recognized not to be perfect.
At least in those situations that's an informed decision. It's not "Whoops, we published this and then an improvement was discovered right afterwards"; it's "Hey, there's an improvement, but the TASer is unwilling to incorporate it for whatever reason."
Nach wrote:
Even if we rejected all those, if you were correct, then why isn't every movie of at least of recent publication getting a perfect 10 in the technical ratings? We don't even have our NES SMB runs getting perfect 10s in technical when at the time of publishing, no one knows how to do it better, or even if they did, won't save more than a frame or two.
You'll note that I didn't say "perfect". I said "best we are currently capable of". The only movie I'm currently willing to accept as "perfect" is King's Bounty, since as I understand it all the possible shorter input sequences were exhaustively tried.
So I've made my case for giving a little more time to movies on the workbench. I'm still having trouble with the case for speeding them along as quickly as possible. What does it gain us?
Pyrel - an open-source rewrite of the Angband roguelike game in Python.
Joined: 11/18/2006
Posts: 2426
Location: Back where I belong
With the workbench quickly approaching healthy levels again, this discussion is somewhat moot until the next publication binge occurs. Back to business as usual.
Even if we rejected all those, if you were correct, then why isn't every movie of at least of recent publication getting a perfect 10 in the technical ratings?
Because, hopefully, most people understand what is meant with "technical quality" (or at least what I meant when I implemented it and have tried to emphasize in numerous occasions): It does not mean "does it contain flaws or possible improvements?" At least not exclusively. While flaws may be considered when estimating the technical rating, IMO it should only be a relatively small part of it.
Not all runs can get an entertainment rating of 10 for the simple reason that not all games are suited to that rating, no matter how well you do the run on it. Such games simply are boring to watch (or at least not super-entertaining, which the rating of 10 would require). Likewise not all games lend themselves to a technical rating of 10 because not many (if any) superb TASing and running techniques can be used with them. That's (I hope) the answer why not all runs get a technical (nor entertainment) score of 10, no matter how "perfect" they might be.
And as a side note, seemingly my argument of "if for nothing else, as a token of appreciation that some people still care to watch the submissions and comment on them" has fallen completely on deaf ears. Would showing some appreciation be too much to ask? It's not like it would be more laborious, tedious or technically difficult to do so. Although that's just my view.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Warp wrote:
Nach wrote:
Even if we rejected all those, if you were correct, then why isn't every movie of at least of recent publication getting a perfect 10 in the technical ratings?
Because, hopefully, most people understand what is meant with "technical quality" (or at least what I meant when I implemented it and have tried to emphasize in numerous occasions): It does not mean "does it contain flaws or possible improvements?" At least not exclusively. While flaws may be considered when estimating the technical rating, IMO it should only be a relatively small part of it.
That's not actually what's happening.
Warp wrote:
Likewise not all games lend themselves to a technical rating of 10 because not many (if any) superb TASing and running techniques can be used with them. That's (I hope) the answer why not all runs get a technical (nor entertainment) score of 10, no matter how "perfect" they might be.
If you review the runs, that's obviously not the case. In terms of pure technique I'm surprised that all the NES SMB runs don't all have a perfect 10 (or at least close to it for tech). Some have told me they don't give high scores in tech because "what if".
Warp wrote:
And as a side note, seemingly my argument of "if for nothing else, as a token of appreciation that some people still care to watch the submissions and comment on them" has fallen completely on deaf ears. Would showing some appreciation be too much to ask? It's not like it would be more laborious, tedious or technically difficult to do so. Although that's just my view.
It is appreciated.
I think the converse needs to be spoken for.
Here we have judges spending their free time really thinking about a movie, and encoders using a lot of time and hard drive space, and effort to encode a movie, sometimes many times. Then they got to upload it and edit and all kinds of stuff, and in the end, we sometimes get "Hey? Why so fast?". It's ungrateful that people here are even fathoming using a term like "rushed publication". There will always be mistakes either way, but people helping out don't need to get their noses rubbed in it, which is what happens every time someone complains about a rush.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
Because, hopefully, most people understand what is meant with "technical quality" (or at least what I meant when I implemented it and have tried to emphasize in numerous occasions): It does not mean "does it contain flaws or possible improvements?" At least not exclusively. While flaws may be considered when estimating the technical rating, IMO it should only be a relatively small part of it.
That's not actually what's happening.
You mean that most people do consider the technical rating to be a pure number which indicates the amount of flaws/possible improvements in the run? Considering how people have written about that rating in the past, that's probably (and sadly) the case. I would hope it wouldn't be so, but I can't stop people from having free will, I suppose. My quest for the rating to be more expressive and interesting than a boring flaw count is probably futile.
Warp wrote:
And as a side note, seemingly my argument of "if for nothing else, as a token of appreciation that some people still care to watch the submissions and comment on them" has fallen completely on deaf ears. Would showing some appreciation be too much to ask? It's not like it would be more laborious, tedious or technically difficult to do so. Although that's just my view.
It is appreciated.
I didn't want my text above to sound like an accusation (although it admittedly sounds like one). I was simply suggesting that it would be one way to show in practice that judges appreciate people's opinion by waiting for it and reading it.
Many acceptance/rejection notices by a judge do indeed contain a reference to the forum feedback, which is always nice. I'm just fearing that if some runs are rushed into publication, it may show that the judge wasn't even interested on people's opinions.
I think the converse needs to be spoken for.
Here we have judges spending their free time really thinking about a movie, and encoders using a lot of time and hard drive space, and effort to encode a movie, sometimes many times. Then they got to upload it and edit and all kinds of stuff, and in the end, we sometimes get "Hey? Why so fast?". It's ungrateful that people here are even fathoming using a term like "rushed publication". There will always be mistakes either way, but people helping out don't need to get their noses rubbed in it, which is what happens every time someone complains about a rush.
Well, one way of avoiding such situations is not to rush a publication, wouldn't it?-)
I do appreciate that judges/publishers are eager to publish new material, but sometimes it's good to use some diplomacy.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
Warp wrote:
You mean that most people do consider the technical rating to be a pure number which indicates the amount of flaws/possible improvements in the run? Considering how people have written about that rating in the past, that's probably (and sadly) the case. I would hope it wouldn't be so, but I can't stop people from having free will, I suppose. My quest for the rating to be more expressive and interesting than a boring flaw count is probably futile.
Yes. Or along those lines.
Warp wrote:
I didn't want my text above to sound like an accusation (although it admittedly sounds like one). I was simply suggesting that it would be one way to show in practice that judges appreciate people's opinion by waiting for it and reading it.
Judges do appreciate it. Although it's not always needed. Some runs here get built up in a discussion topic, or frequently discussed over IRC with demos. The judge along with others he has spoken to may have already seen most of the movie a dozen times by the time its published.
Warp wrote:
Many acceptance/rejection notices by a judge do indeed contain a reference to the forum feedback, which is always nice. I'm just fearing that if some runs are rushed into publication, it may show that the judge wasn't even interested on people's opinions.
I think the converse needs to be spoken for.
Here we have judges spending their free time really thinking about a movie, and encoders using a lot of time and hard drive space, and effort to encode a movie, sometimes many times. Then they got to upload it and edit and all kinds of stuff, and in the end, we sometimes get "Hey? Why so fast?". It's ungrateful that people here are even fathoming using a term like "rushed publication". There will always be mistakes either way, but people helping out don't need to get their noses rubbed in it, which is what happens every time someone complains about a rush.
Well, one way of avoiding such situations is not to rush a publication, wouldn't it?-)
I do appreciate that judges/publishers are eager to publish new material, but sometimes it's good to use some diplomacy.
Sure is diplomatic of you to keep using the word rush there.
Problems are rare, stop telling publishers they're rushing. It's not nice.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
I think the converse needs to be spoken for.
Here we have judges spending their free time really thinking about a movie, and encoders using a lot of time and hard drive space, and effort to encode a movie, sometimes many times. Then they got to upload it and edit and all kinds of stuff, and in the end, we sometimes get "Hey? Why so fast?". It's ungrateful that people here are even fathoming using a term like "rushed publication". There will always be mistakes either way, but people helping out don't need to get their noses rubbed in it, which is what happens every time someone complains about a rush.
Nach, for the record, the "we donate our free time so that you bastards could enjoy it" type of argument is very cheap. The reason is that every single thing about this site, or any hobby in general, is taking our free time. Judging takes free time, but making movies does so as well, and even watching them — perhaps THE integral part of the site — takes free time. Now if you're going to convince everyone your free time is more valuable than anyone else's you just won't go very far with that. As much as we should appreciate what judges do, judges should appreciate the watchers and the movie makers. Not any less.
But if you want me to go deeper than that I can also point out that making and watching TASes existed before the institution of submission judgement, so here I unambiguously hint to you that using such an argument here can be seen as self-justification. I hope you won't use that argument in the future.
Think for a moment that this site wasn't made for the administration, but for the audience (and its members who come from, and remain, the audience) to enjoy. Naturally its goal should be constant improvement of the environment in accordance with most productive means of going through the "creation -> submission -> appraisal -> discussion -> publication -> improvement" chain. "Rushed publications", while they seem productive in the sense of speedy content delivery, undermine some of the aspects of the comfortable process of going through that chain. They just forcibly break it. It's not productive, and it's not polite.
Oh yeah, and certainly it shouldn't be harder for our judges and publishers to do some things slower. Don't you agree? If you don't agree I'll throw feces at you, I swear.
Warp wrote:
Edit: I think I understand now: It's my avatar, isn't it? It makes me look angry.
Joined: 3/9/2004
Posts: 4588
Location: In his lab studying psychology to find new ways to torture TASers and forumers
moozooh wrote:
Nach, for the record, the "we donate our free time so that you bastards could enjoy it" type of argument is very cheap.
That wasn't the argument. You turning it into such is disrespectful.
moozooh wrote:
The reason is that every single thing about this site, or any hobby in general, is taking our free time. Judging takes free time, but making movies does so as well, and even watching them — perhaps THE integral part of the site — takes free time. Now if you're going to convince everyone your free time is more valuable than anyone else's you just won't go very far with that. As much as we should appreciate what judges do, judges should appreciate the watchers and the movie makers. Not any less.
Nothing was ever said to the contrary.
moozooh wrote:
But if you want me to go deeper than that I can also point out that making and watching TASes existed before the institution of submission judgement, so here I unambiguously hint to you that using such an argument here can be seen as self-justification. I hope you won't use that argument in the future.
I don't know what you're talking about.
moozooh wrote:
Think for a moment that this site wasn't made for the administration, but for the audience (and its members who come from, and remain, the audience) to enjoy. Naturally its goal should be constant improvement of the environment in accordance with most productive means of going through the "creation -> submission -> appraisal -> discussion -> publication -> improvement" chain. "Rushed publications", while they seem productive in the sense of speedy content delivery, undermine some of the aspects of the comfortable process of going through that chain. They just forcibly break it. It's not productive, and it's not polite.
I fail to see anyone breaking the process.
moozooh wrote:
Oh yeah, and certainly it shouldn't be harder for our judges and publishers to do some things slower. Don't you agree? If you don't agree I'll throw feces at you, I swear.
Perhaps we should wait 6 months before publishing? Yes.
It's ungratefulness like this that I stopped publishing for the most part.
Warning: Opinions expressed by Nach or others in this post do not necessarily reflect the views, opinions, or position of Nach himself on the matter(s) being discussed therein.
