last day (24 days later) » 

9:58 AM
2
A: What formula should be used to determine "hot" questions?

gnatAs far as I can tell, substantial part of Qanswers in current formula is fake. (log(Qviews)*4) + ((Qanswers * Qscore)/5) + sum(Ascores) About 1/3 of the answers studied here (83 of total 254) have score less than 1/100 of top voted post in respective question. Given that questions checked wer...

 
related suggestion at Programmers meta: Trial run of modified “hotness formula”...
 
+1 And it's really frustrating when a good question with a single authoritative answer fails to rank high because it doesn't have enough crap answers to bump it up.
Why don't you make a formal feature request to get rid of Qanswers from the formula completely. I think we have enough evidence already.
 
@Mysticial upon discussing these matters, I am not convinced that straightforward throwing of Qanswers altogether is a good idea. At early stages it appears to be doing quite a good magic, "...it's more sensitive to potentially-hot questions at early stage, when there's not yet enough votes to judge better. Frankly, this is the part I really like in current formula..."
...at later stages, totally ignoring Qanswers would be "unfair" to mediocre questions that were made hot by great answers. Don't get me wrong I dislike some things about current formula but have to admit is does some things good (handling of early stage and moderately hot questions looks good to me for example)
 
I think that's more of an issue of what defines a hot question. Is a question with a single good answer considered a "hot question"? The only time I've seen questions with few answers become popular are the ones that get the help of reddit. Never have I seen a question with fewer than 5 answers go +500 from just the multicollider - thanks to the equation putting them at a huge disadvantage. (The branch predictor question, with 2 answers, probably could have done it without Reddit. But that's an extreme case.)
 
@Mysticial an example: single answer from John Carmack made the question really hot (hundreds votes, 100K+ views). If collider wouldn't be able to reflect this, that would be wrong would it?
 
9:58 AM
That John Carmack answer was linked in just about every place you could imagine. So it got just about as much external help as it could possibly get. The multicollider equation wouldn't have mattered at all. I do remember it getting up to 600+ on the multicollider with only 3 answers. But that's because of all the votes pouring in from outside.
In other words, questions with few answers become popular not because of the multicollider, but because they get linked. 3 of my top answers fall into this category. If they get high on the multicollider, it's because of the question getting linked early - as was the case on 2 of my top 5 answers.
 
@Mysticial I see, interesting. So the idea is to get rid of Qanswers at all, right? How would that work at early stages, when Qviews and Ascores are still low?
 
It won't have an effect. Let the pure sum of votes be the primary factor. If there's only a single good answer it will collect a lot of votes, so let it get on the multicollider. By throwing out Qanswers, bad answers have no effect on the question's hotness.
Basically, the Qanswers has the effect of promoting questions like this:
275
Q: Which is better option to use for dividing an integer number by 2?

AbhineetWhich of the following techniques is the best option for dividing an integer by 2 and why? Technique 1: x = x >> 1; Technique 2: x = x / 2; Here x is an integer.

 
@Mysticial interesting. I need some time to chew this approach. As for dismissing bad answers, I don't buy this as a unique advantage because for example, a suggested change with cut off at TopScore / 10 would also take care of that
 
While hurting questions like this:
583
Q: Why does changing 0.1f to 0 slow down performance by 10x?

DragarroWhy does this bit of code, const float x[16] = { 1.1, 1.2, 1.3, 1.4, 1.5, 1.6, 1.7, 1.8, 1.9, 2.0, 2.1, 2.2, 2.3, 2.4, 2.5, 2.6}; const float z[16] = {1.123, 1.234, 1.345, 156.467, 1.578, 1.689, 1.790, 1.812, 1.923, 2.0...

I'm obviously biased here, since I obviously stand to gain the most from getting rid of Qanswers.
The the idea is that Qanswers is that it feeds the bike-shed problem.
@gnat Cutting off at topscore/10 is definitely better than nothing. But then you have cases where you have multiple mediocre answers, and one amazing answer.
 
@Mysticial for dividing-by-2 example, cut off at TopScore/10 would ignore 19 of 22 answers (those scored below 70), would that be fair enough?
 
10:04 AM
The top answer having more than 10x the votes of the other ones.
@gnat Yes it would.
Lemme pull out a recent javascript example that people raged at.
233
Q: Why is i-- faster than i++ in loops?

Afshin Mehrabani Possible Duplicate: JavaScript - Are loops really faster in reverse…? I don't know if this question is valid in other languages or not, but I'm asking this specifically for JavaScript. I see in some articles and questions that the fastest loop in JavaScript is something like: for(var...

1/10 works here as well.
But I have a feeling that gap will only reach that high once the question is already popular.
Here's an example where the 1/10 idea might fail:
1054
Q: Cycles in family tree software

Partick HöseI am the developer of some family tree software (written in C++ and Qt). I had no problems until one of my customers mailed me a bug report. The problem is that he has two children with his own daughter, and, as a result, he can't use my software because of errors. Those errors are the result of...

As is this one:
1232
Q: Our security auditor is an idiot. How do I give him the information he wants?

samA security auditor for our servers has demanded the following within two weeks: A list of current usernames and plain-text passwords for all user accounts on all servers A list of all password changes for the past six months, again in plain-text A list of "every file added to the server from re...

 
 
2 hours later…
11:53 AM
@Mysticial cycles in family tree, let's see. top/10 would leave 6 answers out of 17, dropping about 2/3 of the value contributed by Qanswers in current formula...
@Mysticial idiot auditor, top/10 would keep answers above 66, 9 of 28, again about 2/3 of current formula value
I can't tell if this would be perfect but adjustment by 2/3 looks like worth trying doesn't it?
and I think I figured the reason why I am not comfortable about completely dropping Qanswers (at least not yet0...
this is all about early stages of the "question lifecycle", when there's not enough views / votes to judge...
as far as I understand the formula prior to current one failed at exactly this stage, just look at what Jeff wrote when introducing the change,
> Right now the front page Popular tab is fairly broken -- it's a simple descending sort by views. As Joel said in podcast #18, it is "a self-fulfilling prophecy."
semantics of Qscore*Qanswers part appears to trying to fix just that...
Qscore indicates that question is interesting, Qanswers that it looks answerable. At early stage, when there's not enough votes / views to judge better, combining these looks like a reasonable heuristics (and as far as I can tell it works well - again, at early stages). It's only when question "matures", when two things happen,
first, it gathers feedback, ie gets views and votes allowing to learn better than extrapolate plain numbers...
...and second, the part that relates to reasoning about "more answers might mean answerable" gets quite slippery
does that make sense?
 
 
2 hours later…
1:56 PM
@Mysticial "bad answers have no effect..." I think here's the trap; no formula can guess what answers are "bad" until there's sufficient amount of votes to make a decision; that's just the difficulty of dealing with questions at early stage...
...and that's why I would prefer to try a modification that somehow accounts for this lack of information. For mature questions this is of course not so, and your logic in these makes sense; and it's the mature questions (ones that gained sufficient amount of votes to learn from) where current formula doesn't work well,...
..."doesn't work well" - I think the reason is it keeps applying early stages heuristics for too long, up to the stage where it's no longer relevant, up to the stage when question gets sufficient amount of votes to build upon
 
 
4 hours later…
6:15 PM
@gnat I disagree that a question with only one answer is unanswerable.
I'm not entirely convinced that dropping Qanswers is going to have a detrimental effect on the early stages of a question. The main reason for it is that nothing can get onto the multicollider until it has enough votes in the first place.
The initial votes are from people watching the active lists - which isn't affected by the # of answers. If anything, questions with few answers are disadvantaged here as well since there's less activity so they get bumped less often.
 
6:35 PM
If anything, a question with a lot of low-voted answers is more of an indicator of being unanswerable than a question with a single extremely upvoted answer.
When there's a single extremely upvoted answer that clearly answers the question, it discourages other answers because, "This guy answered it well already. I don't need to add an answer myself."
But in the case of a highly upvoted question and a bunch of lower-voted answers (such as the security auditor question), then it's an indicator that the question is less answerable.
So it's a bit of a counter-argument to your point that lots of answers = answerable.
 
 
3 hours later…
9:33 PM
@Mysticial no-no-no, I am talking about "looks", you are talking about "is". Your reasoning makes sense to me, it only has one limitation: it works only on "mature" questions, when there's enough votes and views for formula to build a popularity guess from these...
...but at early stage, there's too little votes and too little views to build a sensible guess on these
...when question is "young" all we have is number of answers and question score
that's exactly where current formula "plays well" per my observations. It's like, you know, "many answers make it look (note look, not really be) like answerable"
other than that I agree, one (or small amount of) well upvoted answer is a better indicator of a question being answerable, that's the whole idea behind the cut-off. I only have to take into account that answer(s) have first to get these upvotes and...
...while there's not enough to make a better guess, plain Qanswers serves as a rough approximation
not in any way meaning definitively answerable, only a guess that it has better chance to get answered. One thing to take into account here...
...is that at early stage, when question is "cold" (as opposed to "hot"), it is mostly viewed and answered by community regulars, which makes it a whole different game...
...and I think there's a reason to trust "early answers" more than those brought by collider-invaders,
> You know, everyone would want to answer a question, but for community regulars, this natural desire is balanced by established norms. Regulars understand what kind answers are good and likely to bring upvotes and they avoid posting garbage that usually gets downvoted: this sort of sets the quality norm.
> ...For questions that are not too hot (likely 2-3 clicks away from top of the list) it's natural to see things working exactly as intended. Answers and comments quality is mostly maintained by site / tag regulars (business as usual), collider brings moderate amount of interested newcomers from other sites with their views, votes and fresh perspective, everything is nice and fair.
@Mysticial I think dropping Qanswers is going to have a detrimental effect on the early stages, and here is why. You correctly noted it needs "enough votes" to get to collider, but the clever trick (clever at early stages only) is that amount of answers multiplies on question score. And that makes the "early stage game" really interesting...
Formula bases its guess on the votes given to question there, and that's a good trick (again, only at early stages)
 
9:52 PM
I'll get back to you in a bit later since I'm currently in a chat session on Anime. You have some good points that I want to address as well.
 
Okay, meanwhile I'll finish the point. Formula assumes that 1) high question score means community regulars evaluate question as a good one and 2) reasonably high amount of answers (from regulars) means it's answerable, and more, interesting to answer...
Based on this, it sort of "credits" the question, skips waiting for answers upvotes and pushes it to collider. For an early stage, under local community control, that's a pretty good guess as I saw good questions magically getting quickly up in collider
well I think I'm done for now. Looking forward to hear from you, your critique is quite helpful. For a "dessert", an example question where top/10 cut off sucks really bad...
It's one of the questions I studied (listed in "sticky hot" post at MSO), the one about Mountain Lion really failed to deliver with top/10 cut off: 44 question remained of total 52.
I don't know how to deal with this. If honestly I think if this kind stuff is frequent enough, this would mean top/10 cut off idea is flawed
It's only at very strict top/2 cut off (requiring score over 20), there would be 1/4 (13 of 52 answers left), go figure
 

  last day (24 days later) »