« first day    last day (23 days later) » 

12:28 AM
@gnat So the idea here is to eliminate Qanswers and replace it with a fixed constant. That way we put enough weight onto the question score itself. So the formula becomes (Qscore * X) + sum(Ascore). X will need to be tuned to how much we value the question relative to the answers.
In all cases, I don't think it's possible to "hurt" new questions other than to change the time-decay factor. (but that's a separate issue)
Something will always get on the multicollider. How we select the formula will determine what gets on the multicollider.
As for the top/10 restriction. I actually don't think this is going to to much.
Here's my reasoning:
When the question is young, none of the answers will be filtered out.
So we are left with the same situation as right now, where all FGITW questions with many answers get onto the multicollider.
Once the question gets older and accumulates votes, the crappier answers get counted out. So the Qanswers decreases. This helps to push it down on the multicollider.
In the end, the questions with a single (good) answer are still never going to get on the multicollider.
 
 
5 hours later…
5:17 AM
@Mysticial fixed constant is quite weak at predicting the potentiall hot question: it gives same chances to question with zero and 3-5 answers. Please keep in mind here we're talking about early stage, when answers quality is under regular local community quality control...
meaning garbage gets downvoted
(the fact that current formula counts DVed crap same as upvoted and zero-score answers means formula sucks and needs change)
as for questions with single good answer, that's an interesting use case, I need to think it through...
apparently suggested can be tweaked to account for that, top/5 gives such questions better chance than top/10, top/3 better chances than top/5etc but...
 
@gnat On the contrary, I'm not convinced that the number of answers is at all a good indicator of a "hot" question, unless that's what you define it as.
 
@Mysticial ah! understood
 
You could have a FGITW question with the top 3 answers all the same.
 
@Mysticial FGITW I see. You would better not try this at Workplace: there are at least 4 DVers who would equally quickly DV fastest-gun crap :)
that's the point I am trying to make talking about "local community quality control"
 
Well, that's what I'm saying, we're trying to discourage FGITW questions from getting too much multicollider visibility.
FGITW questions have a lot of answers - duplicate answers.
Open-ended questions have a lot of answers - and they aren't particularly the type of questions we want (at least not on SO).
 
5:24 AM
at WP there wouldn't be top-3 copy paste answers, two at least would be downvoted immediately and you'll get 1 or less answers that count
 
I'm not sure about Programmers though.
So it's not a problem at WP - although they are known to put a lot of questions high on the multicollider.
 
at Programmers, answers to regular questions are now passing reasonably strong quality control. Maybe not as strict as at WP but still
WP questions are put there too high because current formula sucks at handling downvotes. One minute we discussed it at Prog meta I'll look for a quote...
 
@gnat I see the point there.
 
yeah we spoke with ex-WP modm how convenient :)
@BenBrocka interesting that current system ignores downvotes in quite a brutal way. It kind of says we don't give a shit about your DVs; even if you put answer to -5, it will still be added to Qanswers and multiplied by Qscore - in highly popular questions this means almost any non-deleted answer will only add to "hotness", no matter how crappy it is — gnat Jan 26 at 9:21
 
Either way that's a big problem.
On pretty much all the sites.
 
5:28 AM
agree
 
But let's get back to the issue that you mention of young questions.
Ok
Situation 1:
 
ok
 
FGITW:
- Question picks up 10 upvotes
- 8 (largely duplicate) answers pick up 16, 10, 8, 6, 3, 2, ...
Boom, top of the multicollider under the current formula.
 
I see
 
Really advanced question with one awesome answer:
- Question picks up 10 upvotes
- 1 answer with 20 upvotes
no multicollider under current formula
So that's the problem with the current formula.
 
5:31 AM
I see, that's your "single (good) answer" problem right
?
 
yeah
 
and top/10 apparently will have it too, right?
 
And note that these votes come before it gets on the multicollider.
Once it reaches the multicollider, then you'll probably see this:
FGITW:
- Question - 100 votes
- Answers - 250, 160, 120, 80, 60, 20, etc...
And because it's easy, people will pile on new answers that may or may not really add anything.
 
ah! good catch. At this stage, it's OK to assume all votes fair (they not necessarly are, but deviations should be rare enough to ignore)
 
now for the other case.
- Question has 10
- Answer has 20
No multicollider - it stays that way.
That amazing advanced answer gets stuck at 20.
Now someone reddits it:
- Question: +300
- Answer: +500
 
5:34 AM
I see
 
From what I've seen, FGITW questions don't do well on reddit because they're - well - boring.
So redditing that FGITW example won't do much.
 
well that makes good sense
 
It might add a 100 votes to the top answer - but that's it.
 
agree
 
So basically under the current system, the only way for the single good answer to get any recognition is to reddit it.
Sad, but true.
 
5:36 AM
I see. And, by the way, this problem will persist at top/10, it gives too much edge to multi-answered questions
 
Yeah, pretty much.
 
I need some time to think it through
 
Bill (the moderator Bill) himself said he reddited my Pi answer because the 150 votes it had at the time "didn't do it enough justice"
 
apparent solution is to tweak it from top/10 to /5, /3 etc but I need time to figure that
 
TBH, I think reddit is a better indicator of whether a question is good or not than our multicollider.
I don't think the formula needs to be that complicated.
If there are two good answers, then people will vote for both of them.
That's already an advantage for something with only one good answer.
 
5:39 AM
pi answer, this one I see
914
A: How to determine whether my calculation of pi is accurate?

MysticialSince I'm the current world record holder for the most digits of pi, I'll add my two cents: Unless you're actually setting a new world record, the common practice is just to verify the computed digits against the known values. So that's simple enough. In fact, I have a webpage that lists snippe...

 
Yeah
150 from just the multicollider.
Then Bill put it on Reddit.
Now, the only reason it got on the multicollider - is thanks to the other answers.
 
aha I need to think it through
 
If my answer was the only one, it wouldn't have gotten past 20. Definitely no multicollider.
And Bill wouldn't have seen it.
 
or better yet test though - I recently suggested a trial run of the modified formula at Programmers questions, just to catch and study issues like you describe...
5
Q: Trial run of modified "hotness formula" for Programmers questions

gnatCould we please make a trial run of modified "hotness formula" for Programmers questions? Modification details are described in this MSO post as follows: As far as I can tell, substantial part of Qanswers in current formula is fake. (log(Qviews)*4) + ((Qanswers * Qscore)/5) + sum(Ascor...

 
Interesting. Then we can throw out all the downvoted answers.
But TBH, I don't think they will have an effect.
If the top few answers have +20. Those few downvoted answers aren't gonna drag them down.
Especially since people less inclined to downvote answers.
If you look at this one (also one I was involved in):
633
Q: Why is one loop so much slower than two loops?

Johannes GererSuppose a1, b1, c1, and d1 point to heap memory and my numerical code has the following core loop. const int n=100000 for(int j=0;j<n;j++){ a1[j] += b1[j]; c1[j] += d1[j]; } This loop is executed 10,000 times via another outer for loop. To speed it up, I changed the code to: for(i...

Out of that many votes on everything. The bad answer at the bottom only has -10.
 
5:44 AM
DVed answers will have effect at early stage. Counting them out at the top of collider won't help, we discussed it there. One minute I'll lokk for a quote
 
@gnat True.
 
I've been using data provided in this MSO question: Don't let questions stick to the top of the hot questions list forever. There are 9 examples of "sticky questions" there, having total 254 non-deleted answers. By the way none of these 254 answers have negative score, which makes me suspect that testing ignoring only negatives will show very little difference compared to current approach: in all 9 example questions Qanswers will remain the same as now — gnat Jan 25 at 17:51
 
So you could possibly ignore the votes on the negative ones.
 
not just "possibly" ignoring negatives is a conditio sine qua non, the thing is it's likely too little to make a difference
as it looks per my analysis of "sticky hot" questions
just think of it, "none of these 254 answers have negative score"
 
Once they mature, the downvotes won't matter.
Even the downvoted ones may get back up because of drive-bys that don't really read them.
 
5:46 AM
yup for mature questions I think something stronger is needed
 
But TBH, the only real issue is getting onto the multicollider in the first place - because that starts the snowball effect.
 
that's how I came initially to idea of top/10, it only later turned that top/10 is also worth considering for "early stage", too...
(upon further thinking) one benefit of having 0-score answers count is, it's more sensitive to potentially-hot questions at early stage, when there's not yet enough votes to judge better. Frankly, this is the part I really like in current formula. As far as I can tell, modification that suggests cut off at "1/10 of top-voted answer" behaves about like that in the beginning; it only drops after first upvotes received by answers - if this is too early, one tweak to consider is to cut at top/10 - 1, this will keep zero-voted answers in until one of the answers reaches score more than 10. — gnat yesterday
 
Even the dumbest of the FGITW questions don't get more than a couple hundred votes - even at the top of the multicollider for a long amount of time.
 
@Mysticial yes snowball effect is for real
 
The inherentness of "this is stupid I'm not voting for it". Will kick in at some point.
But if you can stick an interesting question to the top of the multicollider. (like the sock question - which is somewhat controversal)
 
5:49 AM
@Mysticial you mean stupid will keep at around +10 while reasonable will go to 100, right
?
 
Gimme a sec to pull out examples.
 
sure I see no need to hurry
meanwhile here's an example of stupid answer that totally ignored question asked...
 
Here are some dumb ones: (these are biased towards ones I've answered since I remember them)
http://stackoverflow.com/questions/7632926/is-this-a-c-program-or-c-program-how-to-decide
http://stackoverflow.com/questions/7911776/what-is-x-after-x-x
http://stackoverflow.com/questions/8321459/what-is-the-difference-between-str-null-and-str0-0-in-c
http://stackoverflow.com/questions/8225776/why-does-sizeofx-not-increment-x
 
> For the "sheep", you need a populist slogan, a pitch, a catch phrase that will trigger sufficient support ("git is fantastic" is a good bet for programmers communities). For the "dogs", you need basic handwaving skills - just enough blah blah to make sure answer isn't flaggable plus make it read sufficiently smoothly to avoid triggering vote-to-delete in case if 20Ker skims through it.
 
I've never been involved in a "good" question that reached the multicollider and didn't get reddited.
That's because, they either don't make the multicollider at all.
Or they just get linked.
 
5:55 AM
@Mysticial hey sizeof(x++) I like it, made me stuck for a while. what's so wrong with it, do I miss something?
 
@gnat That's borderline.
 
ah okay
 
It's interesting if you don't know C that well.
Here's a genuinely good one:
494
Q: What is ":-!!" in C code?

chmurliI bumped into this strange macro code in /usr/include/linux/kernel.h: /* Force a compilation error if condition is true, but also produce a result (of value 0 and type size_t), so the expression can be used e.g. in a structure initializer (or where-ever else comma expressions aren't per...

Completely multicollider powered. Didn't get too much love on reddit though.
That's one of the few examples of something getting on the multicollider without a lot of crap answers to help it.
Hacker news came in much later and pumped about 100 votes into the top answer though.
That's where all the views came from.
 
@Mysticial that one is really good. Except for the macro name, as pointed in the answer :) I'd prefer BUILD_BUG_GUARDS_ZERO
@Mysticial three other examples here look like mediocre quality. And these got high score through collider, that has been tricked by multiple answers right?
 
@gnat Correct
They all have a ton of answers.
I mean like the problem with questions like these:
224
Q: How many levels of pointers can we have?

ParagHow many pointers (*) are allowed in a single variable? Let's consider the following example. int a = 10; int *p = &a; Similarly we can have int **q = &p; int ***r = &q; and so on. For example, int ****************zz;

Is that you really only need one answer.
It's funny to note at Will's comment:
@Thecrocodilehunter: Mmmm, nah. This is on topic. And if any of you reddit people want to go through my questions/answers and massively upvote them, please feel free! — Will Apr 11 '12 at 13:07
He blames it on reddit. When in fact they all came from the multicollider. The question didn't any love at all on reddit.
 
6:09 AM
@Mysticial fuck me sizeof(x++) is mediocre, too. sizeoff is a compile time thing, this should be at fingertips of anyone practicing C. I did't do C for several years, that got me trapped
 
@gnat haha - hence "borderline"
 
okay
 
Suppose we actually got rid of Qanswers and replace it with a constant.
We could make some predictions on all the new types of questions that would take over the multicollider.
 
@Mysticial levels of pointers is 2/3 fake score on answers even if we use conservative top/10 cut off, there are only 5 answers of 12 above 29
@Mysticial how?
 
And 5 answers is already a lot.
 
6:13 AM
5 is a lot here, agree
 
@gnat Well, a lot more of the questions that I answer would get up there.
Something like this might get 100:
39
Q: How does GCC optimize C code?

HaileI wrote this simple C program: int main(){ int i; int count = 0; for(i = 0; i < 2000000000; i++){ count = count + 1; } } I wanted to see how the gcc compiler optimizes this loop (clearly add 1 2000000000 times should be "add 2000000000 one time"). So: $ gcc test.c and t...

We would probably see a lot more Jon Skeet up there.
 
how to make formula detect that early? You're not suggesting to account for hasMysticialAnswer, do you? :)
 
@gnat lol no
In other words, we would see a lot more difficult questions with advanced answers.
 
"more difficult questions with advanced answers" I see. Something like cut off at top/1.5 could likely do that trick I think
need some time to think it through
something feels slippery but I can't figure
 
Difficult questions with advanced answers also tend to be more difficult to understand.
 
6:16 AM
maybe it is OK after all
 
But the ones that are difficult to understand wouldn't get enough votes at the beginning anyway.
 
right
 
So if you can answer a difficult question with an easy-to-understand advanced answer.
Oooooohhhh
I'd say that deserves to be stuck at the top of the multicollider for days.
 
ah and easy-to-understand advanced answer will get good upvotes early and we can use that, something like this?
 
@gnat Yeah. They will get good upvotes both early and late.
A lot of people will not upvote something stupid if it has enough upvotes (myself included).
 
6:19 AM
OK that's something worth thinking about
 
But an easy-to-understand advanced answer is different.
Those will keep going.
 
wait! it's not only difficult, but for formula purposes, can not be discerned with populist garbage. Write gitisfantastic and it will get "good upvotes both early and late"
that's not reliable
 
Early on there isn't going to be much difference between populist garbage and easy-to-understand advanced stuff.
 
per what you wrote before, gitisfantastic would have worse chances at reddit but that's out of scope of formula
 
Under the current formula, the populist garbage will get on the multicollider.
The advanced one won't.
 
6:22 AM
yes, on that I agree
 
Under the new formula, both may get on the multicollider. But the populist one won't get very high - because people will withhold votes once it has "enough".
 
cut off at top/somethingSmall` would level the playing field I think
new formula is K*Qscore right?
 
K?
 
constant
 
Yeah
It would need to be tuned well.
I'm thinking 2 would be good.
Since most questions can be answered well with 2 answers already.
Actually, it's inverse.
Answers tend to be more upvoted than questions. So I think 2 would be good.
 
6:25 AM
let me quote to get details right here without stupid K...
6 hours ago, by Mysticial
@gnat So the idea here is to eliminate Qanswers and replace it with a fixed constant. That way we put enough weight onto the question score itself. So the formula becomes (Qscore * X) + sum(Ascore). X will need to be tuned to how much we value the question relative to the answers.
 
yeah, it's the same thing.
 
I see. That's quite interesting
 
We test it on a few existing questions.
Some questions don't have a lot of votes, but have very upvoted answers.
 
I see
you expect Ascore would take care of that, right?
 
yep
In the majority of cases, the question/answers ratios fall somewhere like this:
Q = 100: A = 150, 50, 20, 10...
For populist questions.
For difficult advanced questions, it's more like:
Q = 100: A = 150, 20, 10
 
6:31 AM
hmm that's way different from what current formula has, I need time to get used to it
 
Under the current formula, I believe the sock question ranks higher than even the branch predictor question.
 
anyway this surel will level playing field for pupolist and advanced questions
 
It definitely will.
While I don't think (Qscore * X) + sum(Ascore) is optimal, it is MUCH better than the current formula.
We don't even need to get into the 1/10 cutoff stuff yet.
 
with current formula, socks has about 1000x20 in Qscore part, vs branch having it at about 3500x8, thats 20K versus 28, pretty close, and socks get it easily because of 20 answers
 
Well, the branch predictor question when it was still young.
 
6:36 AM
ah I see
 
It was 1600 x 4 + 2800 + ... at the end of the month
 
I see 1600x4 is tiny 6K versus 20K at socks, not fair.
 
Yeah, it's not fair.
I think we left out a factor of 5 somewhere on the current formula.
So it probably isn't as bad as it looks.
 
right, factor of 5 let's Ascore play greater role, I was talking specifically about Qscore differnce
 
ah
Anyways, the bigger problem will be convincing the higher ups to try the new formula.
It's a drastic change.
And they probably won't like it.
 
6:45 AM
well that's for sure :)
and you know I am not yet fully comfortable with it too. Look we somehow drifted the terms here...
from "hot" to "advanced"...
 
That's the the discussion is for. :)
 
hot means popularuty, advanced means quality
different things
I need to think this through
 
TBH, if we present it like this the meta regulars might like it. But the higher ups (like the devs might be freaked out)
 
one thing for sure with you formula I'd rather rename the list from hot to something else
 
Which would you rather see on the multicollider?
1. How to divide a number by 2?
2. An awesome analogy on how pointers work.
Alternate example for 2. John Carmack's explanation on transatlantic pings.
 
6:50 AM
@Mysticial regulars will like it for sure. As a Prog regular I myself came to idea of changing formula from another, quality driven crusade. I was worried about quality at first, without knowing that formula stands in the way...
9
Q: Answers quality in hot questions

gnatFor few recent months, I've got a habit of downvoting answers which quality doesn't look OK to me. These probably can be generally described as low effort and/or these lacking relevance to question asked. Opinionated slogans, claims that are not backed up by appropriate references or by ...

@Mysticial an answer to "which would you rather see" depends on whther you ask meta regulars or higher ups
:)
okay taking a timeout to chew the stuff I learned so far. Thanks!
 
same
later
 
 
10 hours later…
5:02 PM
@Mysticial Okay, so far I found no flaws in your approach. Though presenting it as a correction to hotness formula would be a royal road to decline - not because it's a bad idea (it's good I think) but strictly because of presentation...
Just think of it, "hotness formula has some flaws, let's replace it with something completely different" that's a feature request pretending to be a bugfix. As a developer I hate it when customers cheat like that and this hate is quite widespread - a great way to alienate devs. Even if higher-ups would be OK with it, devs will likely influence them into decline...
I'd rather spell this as a clear new feature request "provide a way to promote questions by quality index", possibly / optionally / conditionally (I haven't yet made up my mind on that) including replacement of current hotness formula into the package. The best thing about "provide a way..." is it's very hard to shut down with formal ; no matter if devs / higher ups like it or not, this leaves a door wide open for a long term pressure on getting it done.
 
 
3 hours later…
8:08 PM
@gnat Lemme know how to decide to approach it. I'll be glad to give suggestions and support you in the pending meta post.
I actually thought of making a similar proposal a year ago, but I couldn't think of a good way to present the problem/solution.
But you seem to have put a lot more time into this than I have. So it might actually work out here.
 
8:54 PM
@Mysticial well this is how I see it. "problem/solution" is the right term, there is a scent of XY-problem here. First things first, you want advanced questions to be promoted, if this is done then what happens to hot list doesn't matter, it can stay or go, or whatever
 
I think you're approaching this in a better direction than I was.
"Promoting advanced questions" sounds better than "suppressing lame ones"
So I take it that we can approach the problem like this.
 
@Mysticial Yeah I gave it some thought that's for sure. :) Right, another "stuck point" is in the naming. Until you name the suggested metrics anything "hot", it'll drag you into endless debates about quality-vs-popularity
 
We mention that well written advanced questions/answers currently don't get any love.
At least not without reddit.
One way to fix this is to "improve" the formula.
This way we completely bypass the need to mention "hot" at all.
By promoting advanced questions, we are implicitly suppressing the lame ones since the attention shifts.
We already have a ton of examples of advanced questions that didn't get any love.
 
@Mysticial yes. Quality index or fair score stuff like that
 
There's a completely separate issue with the time-decay that prevents late answers from getting any traction.
 
9:00 PM
time decay is too complicated I am not yet redy to get in there
 
But that's a separate issue. That I don't think we should tackle until this one is solved first.
 
yes
ready=ready
 
I have some ideas about that such as a sliding window for votes instead of a simple age of question. But I won't get ahead of ourselves.
 
okay, now don't drag yourself too deep into formula improvements. If these will led to fair score promoted, fine; if not, push for another way to get it promoted
@Mysticial right. Advanced ideas are better build over working simplified prototype
build and most important test
:)
now, there are different options to get fair score promoted. One is, obviously, replacement of hot list. Another is a separate list. Yet another, a "mix" in a single list, hot stuff at odd lines, fair stuff at odd lines. Yet another,...
 
Yeah, so many ideas. But we wouldn't want to make it too complicated.
 
9:06 PM
...an option to switch - per user or per site. "I/we want hot/fair"
 
The formula can be arbitrarily complicated since it's only the back-end and nobody sees it upfront.
 
right, list the ideas, pick those that look most feasible, present these
@Mysticial "arbitrarily complicated" is slippery idea. It better be explainable and understandable even if complicated. Black magic in such delicate things as scoring is not good
 
Well, from a UX point of view.
Obviously it still needs to be understandable by the devs.
 
the last but not the least, the feature is new and untested. How to approach matters of testing in the proposal. One option is to state this to be decided on later. Another is to make a temporary switch, SE-wide or on some site. Another, again to get users an option to switch into "beta mode". Another,
is to present it at Stack Apps (but that's good only for smoke testing due to narrow audience)
@Mysticial "understandable by the devs" not only. Understandable by the MSO regulars to allow productive discussing and tuning
your chances to go to meta and say "here's black box score, enjoy" and stay alive won't be high :)
 
Well, I wasn't saying we should try a complicated formula. I'm just saying that it doesn't matter for the average user who's using the site.
We definitely don't want to propose anything compliated.
And I think for now, just replacing Qanswers with a constant (2), and dropping the /5 would probably be simple enough for everyone to understand.
 
9:35 PM
@Mysticial agree on that. It took me really little time to figure => good enough. For explanatory purposes it is worth keeping it documented somewhere that 2 stands for as if there are two answers
 
And also to offset the trend that answers are more upvoted than questions.
 
@Mysticial this sounds OK, too
 
There's one thing we might want to consider that might be worth jacking up the constant to higher than 2.
A lot of dumb questions get a simple answer.
The question gets very few votes (and lots of downvotes). While the answers get pretty heavily voted.
Those are probably the same as the silly FGITW ones.
Anyways, the 2 is flexible.
Could be higher. But definitely not as high as 4 or 5.
 
@Mysticial that makes sense
2 sounds like reasonable starting point anyway. Until tested, at least...
 
9:58 PM
Though I think a change like this will require that it be pushed to all sites at once.
It can't really be tested on a single site.
Since there's only one multicollider.
 
@Mysticial well not necessarily, as far as I understand devs already adjust some stuff at per-site basis, would need to re-check to find out
and if we opt for "configurable" option to choose between hot/fair, it will be entirely possible to arrange testing in much more flexible ways
sorry got to go. Later
 

« first day    last day (23 days later) »