Haven't used this downstair toilet for a while, don't think the people who is living there do either - opened the lid, saw a frog 🐸 Thought it's a frog shaped toilet cleaner container hanging on the wall. Flushed, frog jumped into toilet hole (you know, the pipe where human waste travel through) - turns out, it's a real frog living in the toilet next to a puddle of water in the toilet bowl
@Shog9 I think the approach of replacing the icon to increase review participation doesn't work
I think reviewing has been suffering from the "nothing new in a year almost" syndrome in that its systemic issues have been made clear to people and there is no longer any expectation of change coming
It's very discouraging if you see robo reviewing everywhere from people either not caring about review bans or using scripts to bypass the audits while at the same time people who try legitimately get shafted by the occasional mishandled audit. I think right now the audit system is one of the big points of increasing (especially new reviewer) burnout
An errant keys * command was run on production Redis, stalling the service for 30-60s. We will be renaming that cmd… https://twitter.com/i/web/status/877211204220887040
I disagree with the rep. Right now we provide up to 1000 rep for edits. Allowing that for reviews gets you 2/3 of the way through the privileges list and allows you to edit without review. Suddenly adding rep for reviews is going to do exactly what happened with documentation: People will rush in and click a button to get their points, regardless of quality. Queues will drop temporarily but I will bet you my rep that the quality of reviews will drop.
Throwing rep at the problem isn't a great way to solve the problem.
@Magisch we've already done this several times. Bluefeet spent a fair bit of time on that last year. It helped a lot... Until in the span of a couple months we were back at pre-change backlogs.
@Shog9 If you want to adress burnout, you need to find a carrot to put before people besides the one gold badge per queue
A tangible carrot. Something that people want
The desire to make the site better often isn't enough to overcome the necessary systemic barriers to reviewing
If I want to make the site better I won't need or use your review queues for that. That's what people make stuff like charcoal or SOBOTICS or SOCVR for, a place where we control the rules of engagement to a small extent. If you want people to use the official tools, there needs to be an incentive
SE of all places should know this, its success is built on the perpetual carrot
I think a big carrot would be per-emptively removing low quality posts from the queue. It gets frustrating to go through 20-40 reviews and only seeing crap. I'd much rather see good posts that need a few things to make them great.
How about just some more postive feedback when you reviewed? Not just the Thank you for reviewing 20 suggested edits today; come back in 6 hours to continue reviewing. but something like: With your effort 7 posts were bumped to the front page, 5 question were closed and 10 low quality posts were removed. You're quality rank for today is 234 on the total of 596 reviewers.
@DiminutiveColossus literally. There's a bunch of stuff we can't do simply because the schema + volume makes it too heavy.
Like... I'd kill to be able to visit a tag & see on the sidebar, "3 suggested edits, 10 questions awaiting triage, 30 questions pending closure, 2 answers flagged" - and have them link directly to applicable filters.
That'd be worlds better than a link in the top bar.
No idea what's going on here. Presumably /review just hates Freedom.
I turned switches and flipped knobs randomly until it started loading.
I'm reluctant to call it fixed though.
This got me thinking...
Not about a contest though. I don't want people reviewing questions so that they'll get something in return; folks should be doing it because they care about the questions, about their community. If folks don't care, then what's the point?
But maybe folks do care, maybe...
that's the problem: we designed the review system at a time when there was essentially no one reviewing; the notion of having a hundred active reviewers per day sounded pretty good. We've scaled waaay beyond that.
And it's worked really well - I mean, back in 2012, there were mods who pretty much just spent all day handling close flags. Now review handles close flags, bulk of VLQ/NAA flags, automatic system flags... But it's overloaded and creaky.
@Shog9 yes, but it is so hidden away, so many clicks, so many scary looking screens. I do think the ads with a live feedback count would be really useful.
two classes of things: 1) Things that break very very very badly 2) [Things that we can get through the CR process](https://meta.stackexchange.com/questions/291031/what-features-did-the-community-team-discuss-have-implemented-or-have-denied-l)
IIRC, that was originally suggested on meta in late 2012; I'd originally sent it over in... 2013-2014, but it got rejected. I sent it over again in late 2016, and it finally got done in March, tested and deployed in April.
Easily the most work that's been done on review (outside of Docs) in a couple of years; I was pretty psyched
Dev perspective here: the review queue codebase needs to be rewritten, from scratch and with what we know now that it needs to handle. The codebase and system design are gnarly at best.
And yet - this needs to work with all sites, not just SO. And be discoverable and usable by everyone (with the privileges), not just those with the API keys.
Sure, it is not easy, I never suggested that. And there are numerous issues to be addressed and designed for. And it could maybe be like SEDE. Basically you run the community build and maintained review system.
If it so big and complex that you're no longer able to change it, just launch weird idea's.
You have a community with people than can generate those idea's at an instant
Dev gives up due to the sheer underestimating of the complexity by the oblivious community
@Oded There is a precedent to building things on SO first then pushing them to the rest of the network. It isn't like we would have to immediately replace the review system either, we could develop and test the new system while the old system is still in play. I just figure that if there aren't enough devs to make this stuff happen, why not outsource to your massive free labour base?
@DiminutiveColossus still needs to have a UI for users who are not Meta/Chat frequenters and those who may not be comfortable with just an API (or with installing user scripts). Not saying we can't have two systems side-by-side if designed that way, but that the idea of an API only system is a no go.
@Oded Oh for sure, I'm proposing that you make an API, so that the community can work on a separate review system that could eventually be merged back into SO / SE.
"Sharp drop at 10-50 likely indicates that many users find it difficult to work in review queue. Drop after 250 (silver badge) suggests that even after substantial amount of reviews, many users still fail to discover a way to work productively..." Folks dropping off after 50 reviews because they (wrongly) feel it's too hard are unlikely to be attracted to do more by badges and stats
@DiminutiveColossus Making an API isn't free. Reviewing and integrating a completely outsourced system isn't free either. It's not as simple as "make someone else do it"
Here's the thing. If we had resources to throw at it, we'd just do the work instead of doing the work to enable someone else to do the work. If that makes sense.
But making an API would be a lot less work than making a review system.
Hell, coming up with a design and workflow that everyone agrees on is probably the most time consuming part of the whole thing. Outsourcing even just that part to the community is bound to save a bunch of work.
@DiminutiveColossus Maybe, maybe not. Spending time on a system that would either be thrown out eventually or that we then have to maintain indefinitely doesn't sound awesome. And we're still starting with zero available resources.
Sure, but as far as I can tell right now, the consensus is that the current review system is largely broken, can't handle the scale, and there isn't going to be any reasonable amount of dev time devoted to fixing or rewriting it, so.... basically the current review system is a somewhat failed experiment at best.
@AdamLear that's in the API too (even works with custom reasons). Most of what's needed is informational (what's been flagged, what's been reviewed), which is also where the weak points for review plumbing lie.
@AdamLear Yeah, I'm not worried at all about actions... whatsoever. I can handle that. What I want is the review feeding and stuff like that. Basically, you have a list of posts that need to be reviewed for a given queue and a way of selecting which review goes to which person, and I'll handle the user interface part.
Yeah, that's the bit that's gonna take some creative re-engineering, regardless of what interface gets put above it.
user315433
11:32 PM
I don't recall seeing any stats on the number of edit review overrides that happen....
user315433
The /review page is fed some data on current reviewers via a websocket. Is it something that could be shown elsewhere in a way that encourages reviewing? "Come in, everybody else is doing it"
@Shog9 For sure, at least y'all know what I want. I don't expect anything to happen, but if it does I would be ecstatic. I am (and I can think of a few others who are) willing to work for free to make whatever needs to happen happen.
Trivia: on Stack Overflow since April 22nd, 481 edits have been approved by the owner of the post they were applied to after being rejected by review; 160 have been rejected by the post owner after being approved. Moderators have override-approved 1 edit and rejected 3.
Giant is both an adjective and a noun. Not so for tiny. Hm.
user315433
11:48 PM
I forgot seeing those override stats. Hopefully the overrides do something to educate the reviewers. Although it's quite possible that those suggestions were correctly rejected as deviating from the author's intent. The author then changes the intent after seeing the suggestion...
user315433
With tag wikis, one gets to review suggested edits long before being able to edit directly, at 5k versus 20k. Maybe drop the edit review threshold to 1k.?
user315433
It's the less dangerous on the two powers, considering there are two reviewers.
user315433
As for LQRQ, the audits are too often things that were deleted by a mod for reasons one can't easily tell.