Experienced Forum User, Published Author, Skilled player
(1403)
Joined: 5/31/2004
Posts: 1821
Thanks to everyone for all the nice comments; great to hear that so many people enjoyed the run :)
amaurea: Very interesting comparison video, thx for making it. (Note that in some rooms, like the first cave in level 1-6 we seem to lose time compared to spezzafer, but we safe eggs in this room, which will be used to safe more time later. Also watch the second half of world 5 to get a good look at the speed difference due to 1/1ing, which was used by zvsp in these levels.)
Bobo the King: <3
Experienced Forum User, Published Author, Skilled player
(1403)
Joined: 5/31/2004
Posts: 1821
We (NxCy, Carl Sagan and me) have been working on the TAS a lot lately, and are now at 6-8! Note that we do still have all the extra levels to complete after that.
Experienced Forum User, Published Author, Skilled player
(1403)
Joined: 5/31/2004
Posts: 1821
You are actually quoting me from when I rejected Myst, not accepted (also from when there was no Vault, so I'm not sure why it matters either way).
Given what I've read about movies being eligible for the Vault, and what I generally understand from it, I've come to the conclusion that this movie does not meet the outlined requirements.
What is eligible for the Vault is not defined relative to the most peculiar TASes that are in the Vault. Judging is done by several people who could have slightly different interpretations of the guidelines or just a different experience when seeing the movie. This may be why, when comparing two decisions made different judges, they may seem inconsistent. This is not to say that a single judge was inconsequent: I was not the one who accepted any of those movies you linked.
Experienced Forum User, Published Author, Skilled player
(1403)
Joined: 5/31/2004
Posts: 1821
Warp wrote:
Baxter wrote:
Even with the introduction of the vault, there are still some standards. Gameplay must stand out from non-assisted play, and it must be a "serious game". This TAS does not meet these requirements in a satisfying way. Rejecting.
Please define "serious game".
The term "serious game" is not something I came up with. I got it from the description of vault movies where it is not clearly defined, but where examples are given of games that are not considered to be "serious". I quote: "a game which is a board game, sports game, or game show game is not defined as a serious game".
This is probably a subjective thing, but I judged this TAS to have some of the features that were meant in this remark about serious games on the vault description page. This, in combination with the gameplay not standing out from non-assisted play, lead my to the decision to reject the movie. Note that this had nothing to do with it being a game about ponies. Feel free to disagree and discuss though :).
Experienced Forum User, Published Author, Skilled player
(1403)
Joined: 5/31/2004
Posts: 1821
I've known about that graphical glitch for quite some time. As an extra, I provided a version of the second quest TAS where level 9 was glitched at the bottom of the submission text. An input file of this glitched version was just an extra and was not not the main submission, as, while it is a cool little glitch to see at first, it just messes up with the experience you are trying to provide. So yeah, I even thought that only level 9 glitched is a bad idea... let alone half of the TAS having these ugly graphics. A definite vote against it being included.
Experienced Forum User, Published Author, Skilled player
(1403)
Joined: 5/31/2004
Posts: 1821
I have two suggestions:
When I am looking at the ratings that were given to a certain movie (such as this page), then the usernames of users with a personal page are clickable, linking to the personal page (for instance to this page). When I am browsing the ratings though, and someone gives a rating that stands out, I am interested how this rating compares to the other ratings this user has given, and not in seeing his personal page. I think it would therefore be better if clicking on the username (when browsing ratings) would link to the page with all of the user's ratings (such as this page (I think the "movies rated by this used"-tab should be the first tab btw, rather than the summary)).
My second suggestion is pretty old, I've suggested it (not publically) back when the player point system was first introduced, but it never really gained any traction. I think it would be neat if every day, a snapshot of every user's number of player points was taken and saved somewhere. This way, you can generate a graph at the end of the year of a player's progression during that year. If a player had a lot of publications that year, or movies that were obsoleted etc. Think it would be interesting :)
Experienced Forum User, Published Author, Skilled player
(1403)
Joined: 5/31/2004
Posts: 1821
adelikat wrote:
I've got an idea for a flag, Atlas map encodes that AndyDick does: http://tasvideos.org/forum/viewtopic.php?t=9731
We could flag all the movies he's done, and of course link the video on the movie description.
Experienced Forum User, Published Author, Skilled player
(1403)
Joined: 5/31/2004
Posts: 1821
The addition of the vault enables movies that were previously rejected due to bad game choice to be published. Does this have any effect on currently published movies that were judged not to be a bad game choice? You would think not. Although... maybe we do have some published movies that are better suited for the vault in hindsight? I've expressed my opinion on putting currently published movies into the vault on irc before, but I'll lay it out here again.
Current system
All current movies with an entertainment rating below 5.75 are automatically placed into the vault. People who are able to move movies between tiers place some movies that they feel do not belong into the vault back into the moon tier.
My problem with this system
Future TASes are not automatically placed into the vault when their entertainment value drops below a certain rating, which makes it unfair to currently published movies that are subject to this automatic vaulting. I am not suggesting that future movies should automatically be vaulted based on rating though.
What I suggest
Put all the old movies back in the moon tier that were auto-vaulted. These movies were manually judged already after all. It is always possible to manually move movies between tiers if it seems to be better suited for a different tier, but every time a movie is moved between tiers, it should be required that a post is made in the movie's thread with an argument why the movie's tier was changed (so that the decision can be debated).
What is worse: a moon movie ending up in the vault by accident or a vault movie ending up as a moon? I think the former is what we really want to avoid.
Experienced Forum User, Published Author, Skilled player
(1403)
Joined: 5/31/2004
Posts: 1821
Truncated wrote:
I also don't think it's a good idea to give different amount of points depending on how many people voted. It's better for us to implement ways to get more people to vote, if the problem is that people are not rating movies.
You are actually the first person I heard against the number of ratings affecting the player points. Of course it would be best if all movies had a decent amount of votes, and we should try to encourage that indeed, but fact remains that there are movies that don't get a lot of ratings, and the number of TASes that have very few ratings will only increase due to the implementation of the vault tier. If a vault movie is stuck at a single rating, but this rating would happen to be a 9.5, it will provide the same number of player points as some of the most popular TASes on the site. The idea is not to have a system where more ratings is more player points, but where this kind of situation is avoided. Because of this, more ratings would only provide significantly more player points up to a certain number of ratings that is representative (10-15?).
Experienced Forum User, Published Author, Skilled player
(1403)
Joined: 5/31/2004
Posts: 1821
Truncated wrote:
I understand the aim of this change. We want to include more movies, and not reject people's hard work just because they chose a game which it turned out people did not find interesting to watch.
I think the aim of this change is different and has little to do with people's hard work. The current list of games has become more and more like a full list of games, with some holes where the more impopular/unknown games should be. Given shows like the Angry Video Game Nerd, Video Game Vault and many more, there does seem to be an interest in these games. With the addition of the vault tier, TASVideos could become a place where you can find complete playthroughs (and if you're lucky some interesting strats) of just about any, no matter how obscure, game out there.
Experienced Forum User, Published Author, Skilled player
(1403)
Joined: 5/31/2004
Posts: 1821
feos wrote:
What's the dispersion of amount of raters in total? Is there any sort of sats about the most rated runs? We could see the average of those runs and decide the value for such cases and for few rates, seeing WHICH runs attract more raters.
I don't think there is. There are of course the moviestatistics, but it only lists the top 10 most rated movies. I thought Ilari said the other day that the average number of ratings per movie was about ~15?
feos wrote:
But speaking of much raters, sometimes I see the runs that get lower rates than they actually deserve, but there're too many raters, so I can't change the situation with my single "true" rating. There's a problem of people not caring too much (we need to expand the gidelines list of what to pay attention to when rating).
We already have a page for rating guidelines and voting guidelines. If you think that something is missing there, then you can always add it... I wonder if that will really solve your problem though. I don't think that you should base your rating on what other people rated though... if people were to view all of your ratings, they'd want to see what you like compared to other movies. If you base your ratings per movie on other people's rating to steer the average in the direction you want, then your personal ratings page is not very informative anymore.
Experienced Forum User, Published Author, Skilled player
(1403)
Joined: 5/31/2004
Posts: 1821
These last few posts were about point (2) in the first post, fixing it by multiplying the player points the different tiers general by different numbers. The first few comments were against this, arguing that unentertaining movies will get little player points anyway, and that it could be unfair to movies near the line vault/moon.
Those who argue for such a factor I ask again, is a system that favors high ratings with lots of votes not enough to distribute player points fairly? Why not?
Sure, people must be encouraged to make high quality TASes with great entertainment value, but is the audience's response, the placement in a certain tear etc not enough as a motivation?
===============================
On the other hand, I would like to know the following:
A average rating of a 6.3 currently gives about 100 player points. Just going by intuitions, how many player points would this 6.3 actually be worth if you know that it consisted of:
1 rating
2 ratings
5 ratings
10 ratings
20 ratings
50 ratings
100 ratings
Do you think this should be linear, like 1 rating, 10 points, 2 ratings, 20 points... 10 ratings 100 points, and cut it off there.
Or do you think it should start low, but more up faster as in: 1 rating 8 points, 2 ratings 30 points, 5 ratings 70 points, 10 ratings 90 points, 15 ratings 100 points, 50 ratings 110 points.
Any posts of this type would be appreciated.
Experienced Forum User, Published Author, Skilled player
(1403)
Joined: 5/31/2004
Posts: 1821
Gman wrote:
Anyway, I'm mostly dealing with issue #1 in this post.
You failed to explicitly mention what exactly you are trying to do. Issue (1) in my post refers to taking the number of ratings into account when converting the ratings to player points. Are you trying to change the way the average is calculated which incorporates the number of votes in such a way that the old player point formula can be used? Would the old player point formula still need adjustments?
Also, the average is currently calculated I think by giving the entertainment rating a weight of 2/3 and the technical rating a weight of 1/3. Did you account for this? These ratings can be very different for certain movies; everyone agreeing that the movie is quite a technical achievement, but completely disagreeing on the entertainment value. This also makes you wonder how good the assumption of a normal distribution is.
GMan wrote:
So to start, let's describe our overall goal. What we're really trying to calculate is the 'true' mean (average). That is, if every single possible viewer voted, we could take the mean and with 100% certainty say "this is the mean", because there couldn't possibly be another vote to throw it off. The problem is that not every possible voter casts a vote, so we have some uncertainty.
I also don't know if this is a good assumption. I for instance would never watch a 4 hour rpg TAS of a game I haven't played, so I would never rate such a movie. If I had to, I would probably rate it extremely low. People watch and rate the movies they are in some way interested in, so the ratings that you do get are in no way a good representation of what the distribution of ratings if everyone were to vote, and finding out/approximating what everyone would vote (including people who have never player the game/have no interest in it) does not seem like a good goal then.
Note that, as feos pointed out in the post above, discussing weight of entertainment/technical ratings or the way to calculate the final rating in general given the individual ratings is slightly off-topic.
Experienced Forum User, Published Author, Skilled player
(1403)
Joined: 5/31/2004
Posts: 1821
feos wrote:
I think the tier (at least for vault) must be decided by a judge as he accepts the movie. And the future placing must depend on rating: whether it is a vault movie or a regular one (moons/stars are still assigned by trained monkeys starmen).
While I do think that it should be possible to switch a movie's tier in hindsight, I don't think this should (solely) depend on the ratings it receives.
Whether or not the ratings of some users should weight heavier on the final rating than the ratings of other users is a different topic, that does not directly bear on the player point/rank system. (Unless some user's ratings would not count fully to the total number of ratings maybe...)
Experienced Forum User, Published Author, Skilled player
(1403)
Joined: 5/31/2004
Posts: 1821
GMan wrote:
The current and suggested methods do seem a bit ad hoc, I'm interested in sampling some data to run some tests, is there any easily available?
At the moment, data like that is not easily available. Unless Ilari wips something up.
GMan wrote:
That is, is there a dump of data containing movie votes (note: not the final average score, but the votes themselves; I think using the average might not be the best strategy for this), movie author count, and resulting score?
You can see the individual ratings by clicking the number of votes. Some people have chosen not to show their votes publicly, so you won't be able to get the complete data you'd like probably.
Experienced Forum User, Published Author, Skilled player
(1403)
Joined: 5/31/2004
Posts: 1821
rog wrote:
Movies with less than probably 10 votes or so should probably get a penalty. Above that though, i don't think anything needs to be done.
Do you have some intuitions regarding this penalty? As in, what percentage of the normal points should a movie with only 1 rating get? And movies with 2 ratings? etc :)
