last day (15 days later) » 

3:52 AM
9
A: Is there a Demonstrable Correlation between Question/Answer Length and Reputation?

ArtOfCodeTL;DR: Yes (just), but not the one you think. I pulled the statistics. This is a scatter graph of reputation to average post length on Stack Overflow, limited to reputation <= 20000 and post length <= 5000 to remove the outliers. That red line is the trend line. In other words, this is showin...

 
Nice job. Couple more things to investigate though. How does Qs vary vs. As? I'm actually not surprised that high rep implies shorter As. But I suspect Qs might tell a different story. Also, what's interesting from that graph is that there tends to be more consistency as rep goes up. Lower rep users are jumping all over the place with length. Higher rep are staying within a consistent zone.
And another thing to investigate would be - is the reputation in the graph above the current rep as of May 2020 or is it rep at time of posting? If it's the current rep, there is still value, but it wouldn't be as accurate of a measure as rep at time of posting to demo the correlation. However, rep at time of posting was proving v. difficult for me in the SEDE.
 
@ColmBhandal This is both post types, Q and A aggregated. And yep - rep at time of posting is basically impossible, you'd have to do a full rep calculation for every row.
 
Oh wait, I have an idea why the "variance" seems to be very high for low rep users. It's because they have tiny sample sizes to average over. Possibly just one or two Qs. Higher rep users are stabilizing because they've answered many Qs I'd imagine.
Rep at time of posting is possible, for a single user, but yeah you need to do some very computationally expensive stuff in the SEDE, and at the end of it you just get a crude approximiation to rep (upvotes*10 - downvotes*2). So for all users I doubt it would work. I was trying to think of an optimization but no joy.
 
How did you get this graph? Is there a Data.SE query you can link to, or did you use some other programming wizardry? I'd like to get the same data for other sites than SO, if it's just a matter of site switching in a Data.SE query.
 
It was a SEDE query, @Randal'Thor, but I didn't save it. Let me see if I can salvage it...
@Randal'Thor here you go, that's for SciFi. It's not exactly the same graph as for this post because I pulled the CSV into Excel and did some post-processing, but the SEDE graph serves for an overview.
 
3:52 AM
@ArtOfCode Thanks, very interesting!
 
The problem with this answer is that the query used to generate it looks at all post types, which includes tag excerpts, which are generally very short. So what the graph may be showing is that higher-rep users write more tag excerpts. It would be better, I think, to select only questions and answers.
 
@GarethRees On any site except Stack Overflow, very few people write tag excerpts at all. The difference from excluding them will be insignificant.
 
That's not what we found at Literature, where higher-reputation users write substantially more tag excerpts. I'm prepared to believe that excluding tag excerpts and wikis would make little difference at a bigger site like Stack Overflow, but I think it would honest either to re-run the query and re-draw the graph, or else to mention this limitation in your answer.
 
I'm hardly being dishonest, @GarethRees.
 
I've suggested an edit that improves the accuracy of the post.
 
3:52 AM
@GarethRees I've re-run the query on several sites excluding all but Q&A, including Literature. The average difference is 17 characters. That's statistically insignificant - and more to the point, would be invisible on this graph.
 
If it makes no difference to your conclusions, then you should be happy to include the updated graph in your answer. I mean, it was a natural mistake to make — who knew that there were 8 different post types? — and doesn't reflect badly on you if you correct it.
 
Please re-read my last comment, @GarethRees. It would be invisible. Generating that graph is not a trivial amount of work; I'm simply not going to do it for something that won't even show up. I knew there were multiple post types - I simply ignored it here because I suspected (and have now confirmed) that it would make no difference.
 
If it's too much work to generate a new graph, that's fine — just update the answer to explain the data behind the graph that's shown there.
 
Or, since it makes no difference, I could just... leave it as it is. Which is what I've chosen to do.
 
Can you state the sample size in the answer? (How many points in the graph)
Why are there so few points above 5000 total user reputation points? E.g. only 7 users for reputation points range [18000; 20000].
I suggest adding a figure caption so the figure can understood in isolation. E.g. "reputation" is not very specific (from context, it is probably total user reputation, but it is not even in the answer). And it is not clear if "post length" is for all posts or only for answers or only for questions.
"Posts" meaning "questions" (average question length) would explain why there are so few users in the graph with reputation points in the range [18000; 20000] - for many users it is below them to ask a question, even if it would be a good contribution to the knowledge base (ego-driven nature of Stack Overflow (by design)).
It also ought to be stated that posts used in this answer include tag excerpts and tag wikis. You need to tell us things like that.
Please update your answer with the extra information.
 
3:52 AM
Please read the comments above, @P.Mort.-forgotClayShirky_q.
 

last day (15 days later) »