« first day (2146 days earlier)      last day (3089 days later) » 
00:00 - 20:0020:00 - 00:00

20:03
@rene Lol nope.
Maybe an automatic flag?
@nicael could be. I clicked Looks OK so you can keep your test post ;)
Could you provide the link to the review?
Ah wait
I can get it myself :)
:D
Interesting, if I delete, will the flag be deemed helpful?
@nicael I think so, yes
20:08
Anyway, deleted-undeleted.
I have no idea why would anyone flag it though.
It has two delete votes btw. Huh.
You might have picked up some haters? Can't imagine how that would happen ...
@rene btw. Could you flag this comment as anything, please? need a test.
gone ...
Thanks
Oh lol
Let's hope that it keeps me clear of repercussions ...
20:12
Thought I could edit it know, but it just disappeared once I clicked "save"
You would need a mod to tell you if that edit made it into the backend
There are no websocket updates for comments, except the new comment notification
@rene I haven't made an edit, unfortunately. Clicked enter, thought it wouldn't disappear that fast.
@nicael Ah, OK
@josliber gone!
20:16
@nicael Thanks! it had been lingering
Does !!/report not work anymore?
@josliber No, all reports to the Tavern are on a 5-minute delay. That includes manual reports.
@hichris123 Ah cool, thanks
And howdy all; haven't been here in forever and a day
Howdy !
dev alert
Hey, Oded is here.
He is going to explain what Nick broke this time ...
20:20
Just to say - thanks for keeping an eye on meta and giving me more search targets for that careers spam.
Guess that now with careers being part of SO, it has become that much more of a spam target.
:) I do think it should be brought up about allowing everyone to create a company page. Some of them are literally advertisements, not looking for employees at all. Like stackoverflow.com/jobs/companies/softoneglobal.
There isn't a recent added Companies feed, right?
Don't think so, not as such.
It would help if you could sort the company list on Creationdate
@Oded Apparently they're still at it: stackoverflow.com/jobs/companies/… is new. There's also stackoverflow.com/jobs/companies/iubiubiu but that one was in one of my searches a while ago.
20:29
Yeah, we can see them editing.
Then you could crawl without having to visit all pages
@Oded uhh.. you switched the camera on their PC on and now you're watching them ... Cool ;)
Isn't this a fun way to be spending your Sunday... fighting spammers...
@rene Fighting spam with... malware... ingenious? :P
What is the caching on that Companies list? It seems like some take only a few minutes to fall out but others take a half hour+.
> This service is 100% safe and secure, join us and become famous!
I'm famous enough ...
20:35
@rene But you can get 20k Instagram followers!
And remember, it's safe & secure.
I guess that comes with the requirement to create an instagram account first ...?
Mm, maybe that comes with?
Or, if you don't like that @rene: you can get Facebook likes! stackoverflow.com/jobs/companies/firstfans
Yes! 1000 Likes!
@rene not really part of the same batch, but removed :)
OK
20:44
Thanks
What is the logic to sort these results? Are the companies without jobs (0 jobs) always first in the list?
@rene It almost seems to be random...
@rene Yeah, I don't know. Was looking at that too (but was able to use an internal / careers mod page to find all the recent "companies" and remove the company pages)
Scraping is going to be a headache this way
:(
20:59
I would probably be blocked by the HA-proxy if I tried
Anyhow - all the IPs that we saw are now blacklisted and also entered into spamram.
So - this wave is over, and we will be discussing spam prevention on careers tomorrow morning.
Spamram also blocks Jobs then?
@Oded or allow community flagging?
@rene and allow for community flagging ;)
Just do the quick win first ...
in Charcoal HQ on The Stack Exchange Network Chat, 2 hours ago, by hichris123
I'm tempted to scrape the listing of Company Pages and see how many match...
21:03
Oh yeah - certainly. I mentioned community flagging in their chatroom about an hour ago though ;)
Sorry you had to spend your Sunday evening like this. :)
@hichris123 on a slow pace it could work, but don't you need an extra fetch on each search result? Or do you only do a title match?
@rene For a lot of those, the title alone matched.
!!/test Resource - If you are weight lifting then taking peptides is a must. You have to take the best bodybuilding supplements for your body. Visit researchpeptides.com to learn more.
> Bad keyword in title
----------
Title - Position 96-108: bodybuilding, Position 109-120: supplements
21:07
@hichris123 you get only 15 results from this with 1251 pages
Yeah. :( I was trying to see if there was a limit parameter or anything but it doesn't look like it.
If you stay under 90 calls per minute or so the HA-proxy will not bark
Maybe mixed that up
A call/second usually works.
For 1250 pages, that's 21 minutes.
150 calls in 90 seconds is allowed
Where'd you get that number?
21:10
@hichris123 here, I tested a bit
Ah, science. :D
but that is per IP so on a shared IP, well, you're screwed
I have a static IP for just my modem so it's probably no issue.
You write the scraper then?
I guess. :P
21:14
OK, let me know if you need help but I only do c#
In python would be preferable because then you can re-use the regex from smokey?
Yeah. It's not bad, since all I have to do is get .tagline and .title and then pipe them to Smokey.
OK, I consider it done then ...
21:39
Okay, it's running. Hopefully 1/sec is respectful enough.
@hichris123 If we don't hear back from you for 4 hours we know it wasn't ...
wait nevermind I missed something... sigh.
lol
All companies have a title, but not all companies have a tagline.
Yeah, the less you need to enter the quicker you can spam
21:51
morning
morning
Hmm, should I log the industries, tags, and location too?
@hichris123 I dunno but tags can be handy. All the movie stuf had 3d models IIRC
Alright, why not.
I'm off to bed, so \i'm not going to see the end-result, sorry @hichris123
22:09
:( Well, night @rene!
22:36
Okay, scraping done. Now to run it through the regexes...
drum rolls
22:54
Results: gist.github.com/hichris1234/b408fbe69e086d25a5f915a0a4c4323f. Either the title or the tagline would be caught by Smokey on all of those -- some are quite legitimate though, since "URL in title" would catch any that have a URL.
But some are spam. Like stackoverflow.com/jobs/companies/arb-bearings is definitely spam.
Hey, you can get a new window from them.
Huh, some of those are duplicates. I guess the sort really is random.
23:12
Okay, duplicates now removed from that.
23:29
@Oded Here's a list of "companies" that are very likely spam: gist.github.com/hichris1234/dd8ec804e95fa48ce343df456dfb5fa8. At best, some are advertisements -- none talk about work environment or why you should work with them.
Are you going to write up an email to careers@so?
@hichris123 Can you append the baseurl to those links next time pls :)
that really sounds lazy.... that wasn't my intention <_<
@Quill I did, refresh the page. :)
:D
(did as in 10+ minutes ago)
@Quill Email about what? "You have a huge spam problem"?
with the list
23:36
Oh, I guess I could. But Oded is probably a decent enough messenger. :P
there's a lot of listings in that list with non-latin characters
Yeah. Some are really suspicious, some are just semi-suspicious.
Come on spammers, it's father's day. Knock this shit off.
Dear spammers, I have altered your access our sites. Pray I do not alter it any further.
ah, I missed these. Poor Nick, always dealing with spammers at inopportune times.
> your access our sites
twitterGrammarFailsFromThe140Refactor++;
Could you potentially scrape the job listings as well, @hichris123?
23:50
Potentially, yeah.
But there's no spam there... at least, I hope not. If a spammer is stupid enough to pay for a job listing...
Yeah... uh, for sure, uh, they'd never, uh, they'd never do that
Oh, yeah, that one...
that's not really spam though... it's just not "on-topic"
FR: Meetups.SO
00:00 - 20:0020:00 - 00:00

« first day (2146 days earlier)      last day (3089 days later) »