last day (15 days later) » 

3:26 AM
36
Q: Is the voting and reputation system sustainable? How can we improve it or maybe it should be replaced?

Sextus EmpiricusIs voting still sustainable? The number of votes going down and the dynamics changing towards questions and answers only being relevant during the first few days. This makes the rating system, by means of a system that measures popularity (counts votes/likes), not very useful. Or at least, ther...

 
Do you have a better idea to implement instead? As usual, the current system isn't perfect but that doesn't mean we have to abolish it. We'd need an alternative first.
 
quick answers suggest poor questions.
 
It is "Stack Overflow" and "Cross Validated" (the last section), not "StackOverflow" and "CrossValidated".
 
Better would be a system that uses ranking. Like it is done with rating movies. We currently have very often only 0 or 1 vote. That is not very meaningfull. However if people would give a rating then this may already work a bit better with less votes/ratings. Also it might stimulate more people to add their vote/rating.
 
This question shows research effort. It is useful and clear. (That's the tooltip from the upvote button, if someone wonders...). Answering it is tremendously hard, though. The voting system has proven to be very effective. One can easily suggest small changes, and carefully analyze the pros/cons (which would be better than retroactively adding +5 for question upvotes ...*sigh*). But as Mast said: Without a clear, profound alternative, the suggestions of how the system might be improved have to remain very vague...
 
3:26 AM
Follow up to my earlier comment/suggestion. That is of course a very severe change. Some other way to improve the system is to give the dynamics a new swing. The recent change of score from +5 to +10 is not gonna do anything. Something else that might be able to help is change the interface or bring otherwise some more attention to older posts. The first graph is especially interesting (and alarming). The number of votes that are being cast to questions older than 1 week has dropped to almost zero. Changing that will already improve the voting system.
 
we could take all upvotes and replace them with 5x5 star rankings and then replace all the downvotes with 1 1 star ranking, that would adequately represent the current ratio of upvotes to downvotes.
 
If you got a rating system, most people would either vote max or least. You'd get very few users that would attempt to nuance the vote, and inevitably, people would disagree with your judgement.
 
@fbueckert the rating could be linked to classifying and then classifying to some important effects for the questions. E.g. 1=question should be deleted, 2=question should go into que for improvements, 9=question should be in popular list, 10=question should be sticky (or some other additional functions which make people better consider their ranking and gives more meaning to the number; trains people to vote similarly).
 
That sounds awful complicated, and I don't see how that's any better than the current system. Does 2 mean the question should be closed? If anything, it's much worse than the existing system; vote on quality, perform additional actions separate from the quality.
 
@fbueckert It is just one idea. I agree it is an awekward suggestion, but I suggest brainstorming about it first without too much restrictions. The main point is that the current system is breaking down (or at least going downhill very fast). We only don't see it without the graphs/data.
 
3:26 AM
It's almost as if most of the questions new devs have repeatedly have already been well asked and answered
 
Is the system really breaking down, though? I took a fast look at your query, and it doesn't seem to include deleted posts. Maybe I missed that? Not including it skews the results, likely pretty dramatically.
 
With deleted post it would be worse because we have more deletions (and low quality) now. Also, the main deletion step occurs after 1 year (removingany zero score questions) so that is the only point where the comparisons go wrong (due to unequal situations, before/after the main deletion step).
 
You're ignoring voting on a massive chunk of posts. Can your conclusions really be accurate when data is missing?
 
What about the effect that older posts are seldom voted upon, but if they get a vote, they tend to go up – suggesting a large disconnect between intention and outcome for evaluating quality? There are terrible posts lurking on SE just waiting to grow further & further. Scrutiny by community seems to be only when the post is center stage, but the downvotes wander seldom into the archives?
 
@fbueckert I often look at both postwithdeleted and post. However, with votes I have not done that (I believe that might be the reason for the small discrepancy which I will investigate later). But, in many of the statistics this difference is not so large. We remain having the case that these numbers are decreasing in the non-deleted set, and it is more or less the same when you include deleted (although I will have to check with the votes).
Here is an example with images for both cases that do not show a drastic difference: meta.stackexchange.com/a/337718/389297 I believe I made similar comparisons for the data here. But I will have to recheck it.
@fbueckert I checked whether the votes table includes votes from deleted posts, and it seems it does. But, the difference is not so big. So, there is not much difference in including deleted posts ( at least I checked it for the second image on stats.SE data.stackexchange.com/stats/query/1155500/… ; for stack overflow it is probably a bit worse the since about 10% of te votes is on deleted questions/answers on stats.SE this is only about 5% which is much less data.stackexchange.com/stats/query/1155616 but it is not a massive chunck )
 
3:26 AM
Thanks! Am I missing something? Why does it look like cross validated has way more questions than SO on those first two graphs? Is that correct? Or is it an artifact of "year 2012 = 100"? It looks like So gets 7k Qs/day and crossvalidated gets around 100 so it can't be real right?
 
Your "absolute rate of voting" graph is meaningless. The rate of late voting as you've defined it is inevitably going to decrease over time: a question posted in 2009 will have had ten years to accumulate late votes, while a question posted last week can't have accumulated any. You really need some sort of horizon: for example, define a "late vote" as one cast between one week and one year after the question was posted, then only include questions posted at least a year ago.
 
@Rubiksmoose yes it is an artifact. What it means is that cross validated has seen a stronger increase since 2012. The relative number (1000/100 or roughly tenfold increase) is higher but not the absolute number.
@Mark the absolute rate of voting. is for all votes in a particular month, and independent from the age of the question So it is the votes done in that month (and not the votes done on questions in that month). So it means that te votes being cast in a particular month is nowadays mostly on done on fresh questions. The old questions are not getting attention. Hvrdly anybody votes on questions older than 1 week.
@Mark it shows the monthly voting activity. It is decreasing in time. But when we differentiate it we see that it is mostly decreasing in time for the votes being placed on older questions (an activity which has almost become zero). (This is ironic and counterintuitive - since the fraction of old questions is getting higher - and I had expected the opposite)
 
The voting system is not a rating system. That's interesting because I've caught myself doing the following: 1) I read an answer; 2) I like it; 3) I go to upvote it; 4) Then I see the answer already has 3 upvotes; 5) I ask myself whether the answer merits 4 upvotes and conclude not; 6) I don't upvote the answer I like. (On reflection, this isn't entirely irrational. It means that a future answer which is excellent will have more chance of replacing a current answer which is good.)
 
@Rounin indeed it might work for some like that (also when it would be a rating system, people might change their rating based on what the current rating is, in order to change the average more strongly towards their favored rating; although I have a feeling this behaviour is decreasing). But this makes the variation in the voting extremely random. It will depend a lot on whatever people happen to stumble on the question, and in addition in which order. (Related: I still want to check out to see whether the score distribution of answers changes when there are more answers)
 
Maybe the impact of gamification is overrated and there isn't much that can be done rep-wise. There may be other things though that can be done.
 

last day (15 days later) »